Fix Missing Resources in a Specify Database

Documentation Technical Docs
,
1

:gear: This guide offers solutions for users facing trouble with their database being in an inconsistent state due to historical migrations or manual changes.

[!warning]
This guide is for IT Administrators. This document describes tools and procedures to repair data and schema inconsistencies caused by historical migrations or manual database changes. It focuses on the run_key_migration_functions Django management command (used when migrating from Specify 6 to Specify 7), outlines the available helper functions, safety considerations, and shows usage examples for creating or synchronizing missing schema records and related resources.

[!hint] Additional Support
If you have any questions or need help from our team, members can contact support@specifysoftware.org and we are happy to perform these steps for you! If you are hosted on Specify Cloud, you can reach out to us with any questions.

Fix Missing Resources

In every Specify 7.12+ installation, there is a Django management command called the run_key_migration_functions , designed specifically for those migrating from Specify 6 to Specify 7, which:

  • Provides a convenience wrapper to re-run several important data-migration helper functions used by Specify.
  • The command exposes a small set of named functions (each typically implemented as an idempotent or one-time migration helper) and runs them inside a single database transaction.
  • It is intended for repair/fix tasks after historical migrations or manual changes left the database in an inconsistent state.

This is run automatically after adding new levels in the Specify Institutional Hierarchy, but can be triggered manually when needed.

Available functions

The command exposes these function names (callable via the management command):

  • apply_patches — apply backend patch migrations (patch-application helpers).

  • fix_cots — create/fix default collection types, collection object type picklists and related COTS defaults.

  • fix_permissions — initialize permissions and add specific edit/stats permissions.

  • fix_business_rules — apply default uniqueness rules, fix global default rules, and adjust catnum rule editability.

  • fix_schema_config — run a series of schema-config update and creation helpers (table/field schema defaults, picklists, deduplication, and discipline-scoped overrides).

  • fix_app_resource_dirs — ensure discipline-scoped app resource directories exist and de-duplicate empty duplicates.

  • fix_tectonic_ranks — create default tectonic ranks and root tectonic nodes; fix related tree-def discipline links.

  • fix_misc — small miscellaneous migrations (e.g. adjust selectSeries defaults).

Safety / behavior notes

  • The command wraps the selected operations in a single transaction.atomic() when run, so either all selected functions succeed or the transaction rolls back on error.

  • Many of these helpers make irreversible changes to the database (creating records, updating schema config, removing duplicates). Always run on a staging copy first and ensure you have a recent backup.

  • Some functions are intended to be safe to run multiple times; others may be one-way fixes. Review the implementation (link above) if in doubt.

  • Pass --verbose to get per-function progress printed to stdout.

Usage examples

These can be run from within the specify7 container:

  • Run the full pipeline (all functions):

    ve/bin/python manage.py run_key_migration_functions

  • Run a single function only (e.g., fix_schema_config):

    ve/bin/python manage.py run_key_migration_functions fix_schema_config

  • Run multiple named functions in order:

    ve/bin/python manage.py run_key_migration_functions fix_cots fix_business_rules

  • Run with verbose logging for more detailed messages:

    ve/bin/python manage.py run_key_migration_functions --verbose

Add Missing Schema Config Records

If you see an error like this one, it means you are missing the necessary schema config record for a table or field:

Error: Unable to find SpLocaleContainer for CollectionObjectType

Fortunately, there is a tool to create new schema config records for tables and fields present in the data model that do not have schema records. To run this, first you need to identify the discipline ID for any discipline in the database you wish to correct the missing schema records for. You can find this by querying the discipline table and finding the discipline.UserGroupScopeId field, which is the primary key (or ID) for the discipline.

It should look something like this (plus many {table}Id fields) once run, showing all fields that are currently missing from your discipline’s schema config:

Discipline: Vertebrate Zoology (id=3)

Missing containers:
- DNASequencingRunAttachment
- LocalityUpdate
- LocalityUpdateRowResult

Missing fields:
- Attachment: spDataSetAttachments
- CollectionObject: collectionObjectType
- DNASequencingRunAttachment: ordinal, remarks, timestampCreated, timestampModified, version, attachment, createdByAgent, dnaSequencingRun, modifiedByAgent
- Discipline: tectonicUnitTreeDef
- Locality: collectingEvents
- LocalityUpdate: status, timestampcreated, timestampmodified, recordset, collection, specifyuser, createdbyagent, modifiedbyagent
- LocalityUpdateRowResult: rownumber, result, localityupdate
- SpQueryField: isStrict

After this, while inside the container running Specify, run the following, replacing 13 with the ID of the discipline you wish to test this on

See all missing tables and fields:

ve/bin/python manage.py sync_schema_config_fields --discipline-id 13

Create all missing table and field records for the schema config:

ve/bin/python manage.py sync_schema_config_fields --discipline-id 13 --apply

Create all missing table and field records for the schema config with logging for each addition:

ve/bin/python manage.py sync_schema_config_fields --discipline-id 13 --apply --verbose