Testing your import
Before running a large import job, we strongly encourage all users of the CSV import tool to prepare and load a test file containing a record (or a few records) representative of the data you will be loading.
- 1 How many records to test?
- 2 Testing recommendations
- 2.1 Fields in repeating field groups, especially if the field group repeats
- 2.2 Fields populated by authorities
- 2.2.1 What you want to see
- 2.2.2 Problem signs
- 2.2.2.1 Not imported as authority
- 2.2.2.2 Imported as wrong authority
- 2.3 Fields populated by term lists
- 2.4 Fields populated from static dropdown lists (option lists)
- 2.4.1 What you want to see
- 2.4.2 Problem signs
- 2.4.2.1 Apparently blank field
- 2.5 Date fields
How many records to test?
Best practice would be to include a value in each field you plan to populate with the CSV import tool. Often you can do this in one row.
However, if a non-repeatable field can be populated from multiple authorities, you would need to include two rows.
Example: Core Media “Creator” is non-repeatable and may be populated from Person/Local or Organization/Local. If you expect to add both person and organizational creators to the Media records you are loading, you would want to test two rows, as shown below.
| REPEATABLE FIELD? | n | n | n |
| IN REPEATING FIELD GROUP? | n/a | n/a | n/a |
| VALUE SOURCE | | personauthorities/person | orgauthorities/organization |
| CSVHEADER | identificationNumber | creatorPerson | creatorOrganization |
|---------------------------+----------------------+--------------------------+-----------------------------|
| | MH2020.1.1 | Ann Acme | |
| | MH2020.1.2 | | The Museum |
Testing recommendations
We recommend you pay particularly close attention to the following data patterns and field types in your testing.
Fields in repeating field groups, especially if the field group repeats
Formatting the CSV data in fields like those shown below can be tricky, so you want to make sure you are getting the expected results.
Fields populated by authorities
The Value Source in the CSV template will mention “authorities”.
Sample CSV data (core collectionobject):
| VALUE SOURCE | | conceptauthorities/concept |
| CSVHEADER | objectNumber | contentConceptAssociated |
|--------------+--------------+----------------------------|
| | 2020.1.1 | Teapot |What you want to see
The term is included as expected in the “Terms Used” list on the right sidebar:
Problem signs
Not imported as authority
Expected term does not appear in the “Terms Used” list on the right sidebar at all.
Imported as wrong authority
Expected term appears in the “Terms Used” list on the right sidebar, but as part of the wrong authority.
Fields populated by term lists
The Value Source in the CSV template will start with “vocabulary:”.
Sample import data:
| VALUE SOURCE | | vocabulary: inventorystatus |
| CSVHEADER | objectNumber | inventoryStatus |
|--------------+--------------+-----------------------------|
| | 2020.10.14.1 | accession status unclear |What you want to see
Click in the populated field in the imported record. The term list will drop down. You should see a check mark next to the term in the term list.
Problem signs
Apparently blank field
The data was sent through and is in the under-the-hood XML as shown below. However, it is not formatted properly as a reference to a term list term, so the user interface does not display it.
Term appears in record, but no check mark next to the value in the dropdown.
The data was sent through as a reference to a term list term as shown below, but something in the way the reference was created doesn’t match the actual term list configuration expectations in the application.
Fields populated from static dropdown lists (option lists)
The Value Source in the CSV template will be a comma separated list of allowable values.
Sample CSV data:
The collection field value for 2020.10.14.2 will import as expected. The other two rows will not.
What you want to see
Click in the populated field in the imported record. The term list will drop down. You should see a check mark next to the term in the term list.
This is from the record imported for 2020.10.14.2:
XML for 2020.10.14.2:
Problem signs
Apparently blank field
The data was sent through and is in the under-the-hood XML as shown below. However, it does not exactly match the key value for the option list, so the user interface does not display it.
XML for 2020.10.14.3:
XML for 2020.10.14.4:
Date fields
This is particularly important for fields where the CSV template data type is “structured date group”.
Ensure that the date format(s) you are using are being parsed as expected.
If you have sent “03/04/96” through as Object production date, this initially looks fine:
However, you should click on the calendar icon in the field to expand and examine the detailed date parsing:
The importer was unable to parse the date at all, so this object would not come up in a search for objects with production date 1996.
If this date had parsed, you’d want to check whether it was interpreted as March 4 or April 3!