This page lists the errors and warnings you might encounter while using the CSV Importer. It includes those you might see in the application web interface, and those you might find in the CSV reports generated by the Processing and Transferring steps.

The errors and warnings are listed in the order you would likely encounter them in a typical workflow.

There is some explanation of what each error means, and what you should try in order to fix it.

If you encounter an error or warning you do not find explained here, please let us know and we will add it!

Warnings vs. Errors

At the BATCH level:

• Error indicates an issue blocking the creation or processing of the batch as a whole.

• Warning indicates a possible issue with the batch as a whole that you should check out. If the issues mentioned by warnings are deemed not of concern, you can proceed. Warnings do not block moving to the next step.

At the RECORD level within a batch:

• Error indicates an issue specific to that record, preventing it from being processed into a transferable CollectionSpace XML record, or preventing a CollectionSpace XML record from being transferred to the CollectionSpace instance specified in the connection used for the batch. Other records in the batch are not affected by this record’s inability to be processed and/or transferred, but this record will be skipped in subsequent steps.

• Warning indicates a possible issue with a specific record. As at the batch level, you may proceed past a warning if you determine the effects are intended or ok.

Outside the context of batch creation and workflow steps, any issues encountered are likely APPLICATION errors, related to unexpected input or inability to properly connect to a CollectionSpace instance.

Creating a connection

Request error user account lookup failed

The CSV Importer could not log in to your CollectionSpace instance using the Url, Username, and Password you entered.

Check that the Url, Username, and Password are correct. The Url should be the base Url for the Services API associated with your CollectionSpace instance.

Creating a batch

Uploaded CSV is invalid

The CSV you uploaded is not readable by the CSV Importer. As the Open Data Institute says, “CSV looks easy, but it can be hard to make a CSV file that other people can work with.” Issues could include things such as:

• different operating systems' end-of-line characters mixed together in the same file

• rows with different numbers of columns

• stray/unclosed quotes

• …and many more

If you have received this error, some specific error messages will have been listed in the body of the error. These errors were identified by a tool called Csvlint, and you can look up what the specific mean errors mean in their error documentation.

The “unknown_error” message isn’t very informative, but we know the following issue(s) can trigger it:

• Some combinations of mixed line endings (i.e. line break characters) and blank lines in your file. Removing blank lines usually fixes the issue.

See User Manual: CSV Importer: Dealing with invalid CSVs for how to find/remove blank lines, and deal with other invalid CSV issues.

Uploaded CSV contains characters with invalid encoding

The error message in the CSV Importer gives you a lot of information. Read that carefully.

It gives you a list of invalid characters in context. If that list is three lines long, that means you have 3 different invalid characters (each of which may occur 300 times in your file!) Unfortunately the CSV Importer can’t show you the actual bad characters, because the web application only knows how to render valid UTF-8 characters. Since we don’t know where the bad characters came from or what their correct encoding is, we can’t force them into valid UTF-8 equivalent characters.

If there are many rows in the list of invalid characters, re-open your CSV and make sure you save it with UTF-8 encoding. (How to do this varies per application. Consult User Manual: CSV Importer: Saving Files as CSV and/or your application’s manual.

See User Manual: CSV Importer: Dealing with invalid CSVs for tips on finding characters with invalid encoding.

CSV contains invalid blank rows

See User Manual: CSV Importer: Dealing with invalid CSVs for how to find/remove blank rows.

CSV has invalid end-of-line character on last row

The error message you see in the CSV Importer explains how to fix this one.

If you need further assistance, contact your normal source of CSV Importer support and provide the file.

Malformed CSV

When you receive this error, we aren’t able to know exactly what the issue is. Information in User Manual: CSV Importer: Dealing with invalid CSVs might be useful in figuring it out.

If you cannot determine what the problem is, contact your normal source of CSV Importer support and provide the file.

Batch processing steps

Preprocessing

WARNING: Fields that will not import

If any field names are listed here, data from those columns will not go into CollectionSpace.

It is fine to go forward if you know your data has extra, un-importable columns. If this is unexpected, check your column names against the relevant CSV template.

Note: column headers/field names are case insensitive. The header is downcased for processing, so column names are returned in all lower case.

ERROR: One or more headers in spreadsheet are empty

There is at least one column that contains data, but nothing in the header row. The Processing step will not know what to do with the data in this column.

Open

Column C is has an empty header

Either add a field name in C1, or delete column C

ERROR: Required field missing

At least one required column is missing from your CSV. The message will tell you specifically what column(s) you need to add.

Note that use of the CSV Importer requires every row/record have a unique record id. This is not a requirement of the CollectionSpace application itself, so you may get errors here about required fields that you don’t have to populate if you are entering one record at a time manually.

ERROR: In one or more rows, required field empty

Open

Row 3 lacks the required objectNumber value

The required column is present in your CSV, but one or more rows has no value in that column. The message tells you what column(s) to examine in your data.

Open your CSV and sort or filter on those columns to find the blank ones. Enter appropriate values or remove affected rows from the CSV.

Processing

Report type: processing_report

• Contains all data from your originally uploaded file

• Error and Warning information is added to columns at the end (right) of your file. The presentation of the error information is designed to allow you to sort/filter to rows with the same error.

• Warnings may or may not be true problems for you and your data. They are intended to flag possible data quality issues. Some of these may be expected and acceptable to you, so you can proceed with warnings, but it is important to always review warnings.
  For example, you may receive an “Unknown value in option list” warning. If you discover you have put “Permanent collection” in your data instead of the required “permanent-collection”, then you need to fix that before load. If you are adding “new-collection” to the collection dropdown but your admin hasn’t added it to the list of usable terms, you can go ahead and load this data with the warning. Once your admin has added the term, everything should work as expected. You just need to ensure you are using the system-expected under-the-hood value, not the display value. 

Report type: uniq_missing_terms

This report is only available if your data included authority or vocabulary terms that are not already present in your CollectionSpace instance

• The report contains one row per unique not-found term. That is, if you have used a new term for a given vocabulary/authority in 100 different records or fields, it will appear here once. 

• The report has four columns: term type (i.e. personauthorities), term subtype (i.e. person or ulan_pa), identifier (the “shortID” the mapping process created for the new term), and value (the term value as it was found in your original data)

• The intent of this report is two-fold: 

  • If new terms are unexpected, you can use this information and search your CollectionSpace instance to investigate whether case differences, punctuation differences, or typos caused your terms not to match. Then you can update your original file with the expected term form. OR…

  • If you want to create fuller term records, before linking this data to the terms, you can use the values in this form to create new CSVs for separate batch import of term records. To do that:

    • Filter this report to a given type/subtype combination (i.e. local person names)

    • Copy the values out into the termDisplayName column of the Person CSV template

    • Add in any more details you’d like to import

    • Batch import each of those CSV forms (one batch per term type).

Warnings and errors

The following describe what you may see in the processing_report CSV.

ERR: multiple_matching_records_found

More than one record with this row’s ID value were found in your CollectionSpace instance when we attempted to check the record status.

Records with this error will not be transferred, since the CSV Importer has no way of knowing which of the matching records you want to update or delete.

The solution to this error is:

• Search your CollectionSpace instance for the records with this id

• Add something to the id values in the CollectionSpace records having duplicate id values, so that the id values are unique. (This can be done temporarily, to facilitate the batch ingest, or permanently)

• Update the id values in your batch CSV to match the id values of the intended records in CollectionSpace

ERR: unparseable_date

The processor cannot convert the given date value into a valid plain (i.e. non-structured) date value.

Records with this error will not be transferred, since you would lose the data in any field(s) with this error.

The processing report tells you the affected column and unparseable date value for each record with this error.

The solution to this error is to use a date format that is accepted by the field you are trying to import data into. For example, if you get this error in Object annotationDate field, test that the date value you enter into the CSV is accepted by the application if you manually enter it into the Annotation Date field of a record.

WARN: boolean_value_transform

One or more values in a boolean field in your CSV was something other than true or false. The processor has attempted to convert your 0, 1, y, n, or other value into true or false, but you may want to review that the values you gave it were converted as expected.

The processing report tells you what each value triggering this warning was converted to, so that you may review.

WARN: duplicate_records

The same record id field value is used in more than one row.

The importer warns you about this because it will cause problems going forward if you add records with duplicate IDs to CollectionSpace. If you are creating new records, you may be able to transfer the records with duplicate ID values to CollectionSpace. However, you will run into “WARN: multiple_records_found_for_term” if you try to use the CSV Importer in the future to (1) update these records; (2) create relationships between these records and other records; or (3) (for authority terms only) use the terms to populate other record fields.

WARN: multiple_records_found_for_term

If you receive this warning, the processing_report with have a column with this as the header value.

For rows where this warning applies, the column is filled in with the following info:

• the column the term is used in (the end of this column’s name indicates which exact authority vocabulary was being searched---person local vs. person ulan, for example)

• the term for which multiple records were found

• how many records were found

• the API record URL for the term that was used in the processed record

EXAMPLE: perhaps you have two Place authority terms with the same term display name value “New York”. Other data in the authority records distinguishes that one record is for the state, while the other is for the city. However, if you use “New York” as a value in a place-authority controlled field in a CSV import, it’s impossible to know which “New York” authority record should be linked to by this field.

The CSV Importer uses the first matching authority record it receives to populate the field. If the first record was for the state and you meant the state, then you have no problem. If you meant the city, then you either need to make a note to manually fix this after you transfer the records OR cancel your batch, update one or both “New York” term display name values to distinguish them, update your CSV to use the new display name value, and re-import.

The best way to avoid this warning is to create unique terms within a given authority or vocabulary.

WARN: unknown_option_list_value

The field value in your CSV does not match one of the known values for the field, as defined in the CollectionSpace user interface configuration used to generate the record mapper instructions used by the CSV Importer.

This may or may not indicate an actual problem in your CSV.

Example of actual problem

You are getting the warning: “collection: permanent collection” when processing a collectionobject batch.

The actual underlying data value that you need to use is “permanent-collection”.

If you go ahead and transfer your records with this warning, you will find that you do not see any value in the collection field of your Object records.

General guideline: If you are trying to import a value that has NOT been custom-added to your instance, make sure you are using a value listed in the VALUE SOURCE row of the CSV Importer template you are using.

Example of not an actual problem

In your CollectionSpace instance, you have requested customization of the department term list to add “Basketry” as a new department. You are getting the warning: “responsibledepartment: Basketry”.

If you go ahead and transfer your records with this warning, you will find that the value has been imported as expected.

General guideline: If you are trying to import a value that was added just for you, you can usually use the value you see in the dropdown list in CollectionSpace, and ignore the warning. If your customization involved just changing the label of an existing value without changing the underlying data value, or if the customization did not set the underlying data value to be the same as the display value, using the display value will not work. Refer to the general guidance section below.

General guidance

Approach 1: Attempt to import ONE test record using the value about which you are being warned. If the value gets imported and is visible in your test record, you can disregard the warning for that value.

Approach 2: Find or create a test record in CollectionSpace using the value you wish to import. Do a search for that record. Export the option list controlled field containing the value of interest. If you use the exact value as exported, you can disregard warnings for that value.

See also: this confusing state of affairs is a known issue/limitation of the CSV Importer.

WARN: unparseable_structured_date

The processor could not convert the given date value into a valid structured date.

Records with this warning will be transferred, as no data is lost in this case. The given date value will be imported as the value of the display date (i.e. the value you see in the record before you click in to show the detailed structured date fields.) The Note field in the date details is populated with "date unparseable by batch import processor". However, the detailed structured date fields will not be populated, which can affect searching, sorting, or reporting on these dates.

The processing report tells you the affected column and unparseable date value for each record with this warning.

There are several approaches to dealing with this warning:

• If you do not care about whether or not structured date detail fields get populated, or want to defer the issue to a later cleanup project, you can ignore the warning

• Fix the data in the CSV and re-import. Move values that are not actually dates to other fields. Use standard date formats where possible. Note that the date processing done by this tool will have somewhat different results than what you get entering structured date values manually (for example: try entering “10-19-22” as Object Production Date manually vs. importing it via the CSV Importer)

• Fix the dates in CollectionSpace after importing. If you received this warning on dates in Object Production Date and Associated Date fields, you could find all affected records in CollectionSpace via this search:
  
  

  Open

  

Transferring

WARNING: Some records did not transfer. See CSV report for details

This one is pretty self-explanatory. Look in the CSV report in the Files box to learn which records did not transfer. Filter the spreadsheet to rows where “XFER_status” column = “failure”. Details on the failure(s) will be in the “XFER_message” column.

ERROR: no appropriate transfer action detected for record

There was a mismatch between the status of this record in CollectionSpace and the type of transfer action(s) you selected. Examples could include:

• Record status = new, but you checked only “Update existing records?” or “Delete existing records?”

• Record status = existing, but you checked only “Create new records?”

If any of these errors was unexpected and the Transferring step is finished, you will need to start over:

• Use the CSV report to create a new import file

  • Delete rows for records that transferred successfully

  • Delete any INFO, WARNING, or ERROR data columns added by the CSV Importer

• Create a new batch and make sure to select appropriate transfer actions on the Transferring step

ERROR: Could not delete because other records in the system are still referencing this authority term

As in the web application, you cannot delete an authority term if it is still being used in other records.

Visit the un-deleted term authority record in CollectionSpace and look in the right sidebar to see what records it is “Used by”. Visit those records and remove the term from them. Then you should be able to delete the term manually or via a new CSV Importer batch.

If your term does not appear to be “Used by” any other records, but you are getting this message, it is likely that records using this term were “soft deleted” at some point. These records no longer show up in the application, but still exist in the underlying data used by the services API. Contact your CollectionSpace admin support and ask them to check for and delete from the database any soft deleted records using the term you wish to delete.

ERROR: No processed data found. Processed record may have expired from cache, in which case you need to start over with the batch.

As described in the Known issues/limitations documentation, processed records are only kept in memory in the CSV Importer for a day.

This indicates there are no longer any records to be transferred to CollectionSpace because too much time has elapsed between processing and attempting to transfer. You will need to start over, reprocess the batch, and transfer the processed records more promptly.

ERROR: [partial URI indicating a record type and ids] DELETE

For an authority term record, this will look like:

ERROR: /conceptauthorities/1824cf1c-958b-4c93-a42b/items/42fe73c3-0f81-4c15-8302-e6144877dbc3 DELETE

For an object, this will look like:

/collectionobjects/850161ed-466c-4af2-bc45 DELETE

If you are getting this error on ALL records you transfer as deletes:

Your CollectionSpace user probably does not have permission to do “hard deletes.” Someone with the TENANT_ADMINISTRATOR Role in your CollectionSpace instance will need to handle batches of deletes for you (or assign you to that Role).

If you are seeing this error on only some delete transfers, while others are successful:

Save a copy of the full CSV report generated by the transfer step and leave the batch in the CSV Importer. If you are a Lyrasis hosted client, or have a Lyrasis Zendesk account, send a copy of that report in a new Zendesk ticket so we can look into the problem. If you are not a Lyrasis hosted client, we are unlikely to have the access to your system that we would need to determine the cause of this issue. However, you are welcome to reach out to the CollectionSpace team in the usual ways and we can check whether there’s anything we can tell you.