Batch Attachment Uploader

[vinayakjha]

Batch Attachment Uploader

The Batch Attachment Uploader allows users to bulk upload attachments (files) to existing records by matching filenames against record identifiers.

Features of the Batch Attachment Uploader

• Validation and Upload: Validate, upload, and roll back attachments as needed.
• Record Disambiguation: Automatically disambiguate records when a filename matches multiple records.
• Upload Control: Ability to stop uploads or rollbacks at any point.
• Network Resilience: Upload and rollback processes are tolerant of network issues and client-side crashes.
• Metadata Attachment: Attach remarks to the metadata of the dataset.
• Numeric Field Matching: When using numeric fields (e.g., Catalog Number) for matching, filenames like 000000001.jpg, 1.jpg, and 0001.jpg are treated as equivalent. However, text fields will differentiate between these formats. If you upload multiple attachments for a numeric field, you will need to include all leading zeros for the format, see below.

Permissions

To use the Batch Attachment Uploader, non-admin users must be assigned the “Bulk Attachment Import” role by an admin in the “Security and Accounts” section. This role grants full access to the uploader and can be customized as needed.

For more information on Security & Accounts and user roles, refer to this guide.

Working with Data Sets

Selecting and Validating Files

1. Navigate to the Attachments gallery by clicking the side menu icon and then click on Import in the top-right corner.

2. A dialog displaying existing attachment datasets will appear. Click on New to create a new dataset.

3. You can select files in two ways:

   • Click Choose files or drag them here.
   • Drag files onto the blank area in the center.

4. By default, no field is selected for matching files to records. Choose the desired matching field from the picklist. Available options include:

   • Collection Object / Catalog Number
   • Collection Object / Alt Cat Number
   • Taxon / Full Name
   • Storage / Full Name
   • Collecting Information / GUID
   • Collecting Information / Collector Number
   • Loan / Loan Number
   • Accession / Accession Number
   • Gift / Gift Number
   • Preparation / Bar Code
   • Preparation / GUID
   • Borrow / Invoice Number
   • Treatment Event / Field Number
   • Exchange In / Exchange In #
   • Exchange Out / Exchange Out #
   • Reference Work / GUID
   • Reference Work / Title
   • Locality / Unique Identifier
   • Locality / GUID
   • Permit / Permit Number
   • Agent / GUID
   • Collecting Trip / Trip Name
   • Deaccession / Deaccession Number

5. After selecting the path, the system will mark files as correctly or incorrectly formatted. For instance, 000000001.txt may be marked as correctly formatted, while Not Correctly Formatted.txt is marked incorrectly if it does not match the numeric field requirements.

   

6. Click on Validate to test matches against records in the database:

   • If a match is found, a formatted version of the record will be displayed with a link to open it in a new tab.
   • If no match is found, the message No match will be displayed.
   • If multiple matches occur, the message Multiple Matches. Click to Disambiguate will appear.
   • Incorrectly formatted records will retain the label ‘Incorrectly Formatted’.
   

Add Multiple Attachments to a Single Record

In many collections, it is common to have multiple attachments associated to a single record. For example, you may need to document different orientations of a specimen, such as ‘dorsal’ and ‘frontal’ views, or include multiple photographs of the same specimen to capture various details.

Add Multiple Attachments

To add additional attachments to a single record in your data set, you can simply add an underscore after the field value you are matching and input any value.

When matching a numeric field, such as a 9-digit Catalog Number, you must include all leading zeroes in the filename to ensure it matches the format exactly when adding attachments to a single record.

This should follow the format you see on the data entry form for Collection Object (e.g. if the catalog number field shows 000493839, this should be the file name).

Example:
You are uploading 3 attachments to a Collection Object with the catalog number 000493839.

Filename	Status
000493839.jpg	Correctly Formatted
000493839_dorsal.jpg	Correctly Formatted
000493839_frontal.jpg	Correctly Formatted

If you upload a single attachment, Specify will treat filenames such as 0000000493839.jpg, 0493839.jpg, and 493839.jpg as equivalents. If you upload multiple attachments, you will need to follow the format exactly.

Filename	Status
493839.jpg	Correctly Formatted
493839_dorsal.jpg	Incorrectly Formatted
493839_frontal.jpg	Incorrectly Formatted

Uploading Data Set

1. If an upload is possible, the Upload button will become visible. Click on it to start the upload process.

   

2. While the upload is in progress, a progress bar will display, along with an option to stop the upload.

   • Clicking Stop will halt the upload. At least one file will always be uploaded unless the stop action is executed immediately. The table will indicate which files were uploaded and which were not.
   • For remaining files after stopping, it will show Cancelled: Stopped By User.
   

3. If you refresh or open the link in a new tab, all previously selected files will display a warning, and you will need to reselect them before uploading. The system will match new selections against previously selected files based on filename, size, and type. If a match is found, the previous file will have (No File) removed.

   

4. Click Upload again. Files that were previously uploaded will not be uploaded again and will show as Skipped: Already Uploaded. Files not correctly matched will display Skipped: No Match or Skipped: Multiple Matches. Incorrectly formatted files will show Skipped: Incorrectly Formatted.

   

Rolling Back Uploads

If you made a mistake when uploading attachments, you can reverse this upload and remove the attachments from the database. This is called a rollback.

1. If a rollback is possible (i.e., at least one file has been uploaded), the Rollback button will become visible. Click on it to delete the uploaded files from the asset server and corresponding records from the database.
2. Similar to the upload process, if you stop the rollback, the system will indicate which files were deleted and which were not.
3. For files that were not uploaded, after the rollback, the message Skipped: No Attachments Found will appear.
4. You can stop the rollback at any point.

[QuinnE]

Hello! We just uploaded our first batch of image - it was magical and went very smoothly.

We did have a question though. When uploading, we can only add in the linked field. However, there are a few fields - such as copyright - that we would like to update for all the records (ours is blank). I assume the metadata is currently pulled from what is embedded in the attachment. Is there any way for us to bulk edit the attachment fields, such as copyright?

Thank you for your help!
Quinn

[SPalpurina]

Hello there,

I am an admin but I cannot see the button Import in the top-right corner of the Attachments gallery. Is there some setting I should turn on to make it visible?

[robinsp]

I’m used to attaching multiple images to each of multiple Collection Objects using an Import Mapping File in Specify 6.8.03. I have not been able to work out a similar process in Specify 7. Is this option currently available?

[Grant]

Hi @robinsp,

Currently, there is no support for attachment mapping files, but we have a feature request on GitHub for this:

Add Attachment Mapping File support

I’ve added you as a requestor!

[Grant]

Hi @SPalpurina,

Can you verify you are on the correct version of Specify 7? Please share a copy of your Specify 7 System Information with us to review!

---

Hi @QuinnE,

That’s great to hear! This tool currently does not support batch update operations for individual fields (or field-level metadata being included in the upload), but we are developing a batch edit utility that will be available in the coming months. This will enable the kind of operation you described!

[SPalpurina]

Apologies. Here it comes: v7.8.13
Specify 7 System Information - 2024-08-28T08_14_21.855Z.txt (664.3 KB)

btw, @cicadina has mentioned that he plans to install soon a new version I am keeping him in the loop just to be informed.

[SPalpurina]

Oh, yes, now I see we need version 7.9.3, right?
Thanks for your time, @Grant. Next time I will keep in mind to check for which release the post is associated with.

[Grant]

Hi @SPalpurina,

Yes, once you update to version 7.9.3 or newer, this tool will be available.

[sorosoro]

HI. Attaching multiple images to a collection Object is important for us, whether it’s done with a mapping file or in any other way. It’d be great to see this functionality in Specify 7.

[vinayakjha]

Keep in mind that you can upload to the same object as long as the catalog number is any kind of formatted (and has a fixed length) right now. If it isn’t, then mapping path will be neededd.

[sorosoro]

Hi @vinayakjha. Do you mean with the batch attachment uploader? So, if the catalog number has a CatalogNumberString format, a mapping file will be needed?

[vinayakjha]

Oops, I’ll try to keep the conversation rolling this time! Yes, I meant with bulk attachment uploader.

If the formatter is defined and is not regex, you don’t need the mapping file (since the size property should be required in that case). So, if the length of your field formatter is 4, and catalognumber of collectionobject is ABCD, you can import multiples via ABCD(0), ABCD(1) and anything other like ABCD(a), ABCD-0, ABCD-1, ABCD-(first), etc.

[sorosoro]

Thanks @vinayakjha, It’s good to know this is possible in Specify 7 although I cannot use it at the moment because of the catalogue number format we defined (regex).

[markp]

To prevent others from the same false assumptions I made today, the following statement

Files that were previously uploaded will not be uploaded again and will show as Skipped: Already Uploaded

Will only apply to files within that same bulk upload. If an attachment already exists in the system, and then an identical file is added to a fresh bulk upload for the same record, it will not be picked up as already uploaded, and will upload as normal.

In essence, the Already Uploaded functionality shouldn’t be used as a shortcut to avoid uploading duplicates overall.