Imports Service notes and issues

  • Currently the Imports Service reports success for all records, even if it fails to import a record. You need to check the imported data against your import file. (Jira filed?)
  • Shell script for imports
  • Shell script for deletes. (Note: Currently the script sleeps for 30 seconds after processing each CSID in the todelete.list file, presumably to give it extra time.  It will continue to run if you are patient.)
  • If you delete a record in the CSpace UI (e.g., a person record), it is really a soft-delete, and other fragments of that record will be stored in tables outside the authority you were working with.  If you try to reload that CSID using the Imports Service, it will report a successful import, but the record will not be added.  You need to run a curl-based delete to remove the record completely, and you can do so even for a record that does not appear in the UI.
  • In a file with 1000 person records, there were some ampersands that were not properly formatted to avoid the Imports Services bug documented in CSPACE-3911.  However, rather than failing for that set of records completely, 208 records were imported.  However, they were not the first 208 records in the file.  This makes cleanup and reloading of data pretty hard. Yuteh confirmed that records are not necessarily imported in the order of the XML file.  It is possible that they start out in order and then numerous individual jobs are running synchronously.