Images still not being served from Asset Server

Get Help
18

Hi @phb,

Thank you for the detailed follow-up. That is a completely valid question, and I want to be very clear about what happened and how we’re ensuring it won’t happen again. You and your curators are right to be concerned, and I apologize for the frustration this has caused. We want to make sure you and your team have confidence in the system.

Why This Happened and How We’ll Prevent It

Your Specify Cloud instance had a custom configuration file that directed it to use your self-hosted asset server (images.beatymuseum.ubc.ca). During a routine software update on our platform in early May, this custom file was inadvertently overwritten by the default configuration, causing your instance to revert to using our standard asset server (assets-ca.specifycloud.org).

At no point was any data lost—the application continued to upload assets successfully—but it sent them to the wrong, default location. This created the confusing “split-brain” scenario where some assets were on your server and newer ones were on ours. This was our mistake, and we have taken immediate steps to correct our processes.

To prevent this from ever happening again, we are implementing a multi-layered approach:

  1. Automated Configuration Checks: We are updating our deployment scripts to specifically protect and verify custom configurations before and after any update. This directly prevents the type of overwrite that caused this problem.
  2. Post-Update Audits: In addition to the automated checks, our team has been conducting a manual audit of all instances with custom configurations after every deployment to provide a second layer of verification. This has been in effect since late July, but this started after the “split-brain” scenario began.
  3. Proactive Monitoring: We are implementing an alerting system that will monitor the connection to custom asset servers like yours. If it detects a connectivity failure, our team will be notified immediately, allowing us to notify you and investigate before it has an impact.

Data Integrity and Trust

Specify is a “single source of truth” where the database record is intrinsically linked to its corresponding digital asset. The record in the database holds the metadata and a pointer to the location of the asset file. The “upload confirmed” message you received was accurate: the record was created and the file was successfully transferred. The failure was in our server configuration, which pointed the transfer to the wrong destination.

The core database records (your metadata) were never at risk and are backed up nightly as part of our standard Specify Cloud hosting. The issue was confined to the storage location of the asset files. The immediate plan we went through today sent the files from our server back to yours so you can consolidate all assets and restore your server as the single, authoritative location for assets.

Thumbnails Not Loading

Regarding your second question, thumbnails are generated on demand by the asset server. When a user’s browser requests a thumbnail, it makes a call to the asset server, which generates and serves a smaller version of the original image. See our documentation “Thumbnails” for the technical details of it.

I imagine the discrepancy you are seeing is because you cannot modify an original URL to output a thumbnail just by changing the O to a T, as there is a set of construction steps taken by the app to build a URL:

Specify builds URLs to obtain assets from the Web Asset Server differently depending on the context (either full resolution or thumbnail). The URLs are constructed with the following parameters:

  • Base URL (baseUrl): comes from the attachment server settings (e.g., https://assets-test.specifycloud.org) for the Specify 7 service (configured in Docker).
  • Collection (coll): the collection key for the database (e.g., sp7demofish) (configured in Docker).
  • Filename (filename): the server-side stored filename from each Attachment record (from the attachmentLocation field in the attachment record directly)
  • Type (type): T for thumbnails/previews used in the gallery, originals use O. If T, you must add the scale component.
  • Scale (scale): The size of the thumbnail to generate. Only used for thumbnails.
  • Download Name (downloadname): Optional, you can specify the download name for the image.

Putting it together, a URL like this will make it possible to download the original attachment at full resolution:

${baseUrl}/fileget?coll=${coll}&type=O&filename=${filename}

and this will get you the thumbnail:

${baseUrl}/fileget?coll=${coll}&type=T&filename=${filename}&scale=123

We take our role as stewards of your data platform very seriously. Please let me know if this explanation helps.

Best,

Grant