Batch Attachment Uploader

[vinayakjha]

Batch Attachment Uploader

Bulk attachment upload allows you to upload attachments (files) to existing records, by matching the filename against an identifier of the record.

What does Batch Attachment Uploader do?

• Validation, upload, and rolling back of the attachments
• Disambiguation of records if a file name was matched to multiple records
• Ability to stop upload/rollback at any point
• Upload and Rollback are majorly tolerant of network issues and crashes on the client’s side
• Attaching remarks to the metadata of the data set.
• If a numeric field (e.g. Cat #) is used for matching records, 000000001.jpg is treated the same as 1.jpg, and 0001.jpg. However, if a text field is used, all the previous ones would be treated differently.

User Roles and Permissions

If a user is not an admin in order for them to be able to use Batch Attachment Uploader an admin will need to add the “Bulk Attachment Import” role to the user in the “Security and Accounts section”. This role will give the user full access to batch attachment uploader but can be edited as needed.

If you need more information on Security & Accounts and user roles, see this guide.

Data sets, and selecting files, and validating files

• Go to the Attachments gallery by clicking on the side menu icon. Click on Import in the top-right corner.

• The dialog showing existing attachment data sets is visible. Click on New to make a new data set.

• There are two ways to select files

  • Clicking Choose files or drag them here
  • Drag files onto the blank space in the center.

• Duplicate files are not allowed. Hence, if you select the same file again, it opens a dialog saying “ Duplicate Files Found” and will show which ones are duplicates

• By default, no field is selected to match the files to records. Choose the desired path in the picklist to match the files against.

  • There are several different paths to choose from including
    • Collection Object/Cat #
    • Collection Object/Prev/Exch #
    • Taxon/Full Name
    • Collecting Event/GUID
    • Collecting Event/Field #
    • Loan/Loan #
    • Accession/Accession #
    • Gift/Gift #
    • Borrow/Invoice #

• In this example, I am using “Collection Object / Catalog Number” - note that this label may vary across databases. So, the first item in the picklist will always use “Collection Object / Catalog Number”.

• After selecting the path, the files which are correctly and incorrectly formatted are marked. In my case, 000000001.txt is marked as correctly formatted and Not correctly Formatted.txt is marked as incorrectly formatted since the catalog number is numeric field.

• Additionally, 20.txt is also marked correctly formatted (since it doesn’t care about leading zeros).
  

  

• Click on Validate to test match the files against records in the database. If correctly matched, it shows a formatted version of the record with a link to open the record in a new tab.

  • If no match is present, it shows the text No match.
  • If there are multiple matches (not possible in this case, but possible for taxon), it says Multiple Matches. Click to Disambiguate.
  • If the record is incorrectly formatted, it preserves the text - it should still say ‘Incorrectly Formatted’

• Clicking on validation again will re-run the validation query.
  

  

Uploading files

• If an upload is possible, the Upload becomes visible (for example, if any filename is correctly formatted and has not been uploaded before, the button should be visible). Click on Upload to start the upload.
  

  

• Now while the upload is in progress, a progress bar should be visible with an option to stop the upload.

• You can click on stop to stop the upload. Note that some files (at least 1) will always be uploaded (unless you click really fast). The table should say which files were uploaded, and which were not.

• For any remaining files after clicking on stop, it says Cancelled: Stopped By User.

• If you refresh or open the link in a new tab all the files in the selected file will have a warning and you need to reselect the files before uploading. It tries matching the new files you selected to the previously selected files (based on filename, size and type). If it cannot find the file in previously selected files, it adds a new row for that file. If it finds a match for that file, that previous file will have (No File) removed.

• Click Upload again. All the files that were previously uploaded will not be uploaded again, and instead will say Skipped: Already Uploaded for that file. The rest of the files will be uploaded.
• For files which were not correctly matched, it says Skipped: No Match or Skipped: Multiple Matches. The ones incorrectly formatted, it says Skipped: Incorrectly Formatted.

Interruptions

• While the upload is in progress if there is an interruption like your internet going out after some time, it opens a dialog with a timer displaying the time after which it will retry.

  • You can either click Try Now, Cancel, or wait for the timer to finish for the upload to continue.
    
    

• If the upload is still in progress and the tab gets closed when you reopen it prompts you with a dialog saying that the upload was in progress when it was interrupted.

  • Close this dialog, and it tells you which file(s) are definitely uploaded, and the file(s) which were not.
  • You can then reselect all the files (including already uploaded ones - it would just skip them).
    
    

Rolling back files

• If a rollback is possible, the Rollback becomes visible. This happens if at least one file was uploaded.
• Clicking on Rollback will delete the files from the asset server, and the corresponding records from the database.
• Similar to upload if you stop the rollback it will show which files got deleted and which did not
• For the files which were not uploaded, after the rollback it says Skipped: No Attachments Found.
• Similar to upload, you can stop the rollback at any point.
  
  
  
  
  

[QuinnE]

Hello! We just uploaded our first batch of image - it was magical and went very smoothly.

We did have a question though. When uploading, we can only add in the linked field. However, there are a few fields - such as copyright - that we would like to update for all the records (ours is blank). I assume the metadata is currently pulled from what is embedded in the attachment. Is there any way for us to bulk edit the attachment fields, such as copyright?

Thank you for your help!
Quinn

[SPalpurina]

Hello there,

I am an admin but I cannot see the button Import in the top-right corner of the Attachments gallery. Is there some setting I should turn on to make it visible?

[robinsp]

I’m used to attaching multiple images to each of multiple Collection Objects using an Import Mapping File in Specify 6.8.03. I have not been able to work out a similar process in Specify 7. Is this option currently available?

[Grant]

Hi @robinsp,

Currently, there is no support for attachment mapping files, but we have a feature request on GitHub for this:

Add Attachment Mapping File support

I’ve added you as a requestor!

[Grant]

Hi @SPalpurina,

Can you verify you are on the correct version of Specify 7? Please share a copy of your Specify 7 System Information with us to review!

---

Hi @QuinnE,

That’s great to hear! This tool currently does not support batch update operations for individual fields (or field-level metadata being included in the upload), but we are developing a batch edit utility that will be available in the coming months. This will enable the kind of operation you described!

[SPalpurina]

Apologies. Here it comes: v7.8.13
Specify 7 System Information - 2024-08-28T08_14_21.855Z.txt (664.3 KB)

btw, @cicadina has mentioned that he plans to install soon a new version I am keeping him in the loop just to be informed.

[SPalpurina]

Oh, yes, now I see we need version 7.9.3, right?
Thanks for your time, @Grant. Next time I will keep in mind to check for which release the post is associated with.

[Grant]

Hi @SPalpurina,

Yes, once you update to version 7.9.3 or newer, this tool will be available.

[sorosoro]

HI. Attaching multiple images to a collection Object is important for us, whether it’s done with a mapping file or in any other way. It’d be great to see this functionality in Specify 7.

[vinayakjha]

Keep in mind that you can upload to the same object as long as the catalog number is any kind of formatted (and has a fixed length) right now. If it isn’t, then mapping path will be neededd.

[sorosoro]

Hi @vinayakjha. Do you mean with the batch attachment uploader? So, if the catalog number has a CatalogNumberString format, a mapping file will be needed?

[vinayakjha]

Oops, I’ll try to keep the conversation rolling this time! Yes, I meant with bulk attachment uploader.

If the formatter is defined and is not regex, you don’t need the mapping file (since the size property should be required in that case). So, if the length of your field formatter is 4, and catalognumber of collectionobject is ABCD, you can import multiples via ABCD(0), ABCD(1) and anything other like ABCD(a), ABCD-0, ABCD-1, ABCD-(first), etc.

[sorosoro]

Thanks @vinayakjha, It’s good to know this is possible in Specify 7 although I cannot use it at the moment because of the catalogue number format we defined (regex).