This page contains information about PAHMA's data migration of CollectionObjects and related info.
See parent page on PAHMA data migration ETL work.
Collection Objects extract
field mapping
TMS table name |
TMS field name |
CSpace field name |
notes |
---|---|---|---|
Objects |
ObjectID |
shortIdentifier |
|
Components |
ComponentID |
shortIdentifier |
must ensure no duplicates with Objects.ObjectID |
Objects |
ObjectNumber |
objectNumber |
|
Components |
ComponentNumber |
objectNumber |
|
Objects |
CatRais |
pahmaCollection |
NEW field (repeating; controlled list) |
Objects |
DepartmentID |
pahmaTmsLegacyDepartment |
FK to Departments; NEW field (controlled list) |
Objects |
HistAttributions |
objectHistoryNote |
|
Objects |
LoanClass |
pahmaTms2003DataSource |
NEW field (controlled list) |
Objects |
Markings |
distinguishingFeatures |
|
Objects |
Medium |
material |
|
Objects |
MTB_PlaceTermID |
fieldCollectionPlace |
reference to Place Authority |
Objects |
Notes |
comments |
|
Objects |
ObjectStatusID |
pahmaObjectStatus |
FK to ObjectStatuses; NEW field (controlled list) |
Objects |
ObjectTypeID |
collection |
FK to ObjectTypes |
Objects |
PaperFileRef |
nagpraInventoryName |
FK to MTB_NagpraInvNames; controlled list |
Objects |
Portfolio |
portfolioSeries |
FK to MTB_PortfolioSeries; NEW field (controlled list) |
Objects |
Provenance |
fieldLocVerbatim |
|
Objects |
MTB_ResponsibleDept |
responsibleDepartment |
controlled list |
Objects |
Signed |
inscriptionContent |
FK to MTB_NagpraBurialCodes |
Objects |
State |
pahmaNagpraCodeLegacy |
controlled list |
Objects |
Title |
title |
|
ObjContext |
Integer3 |
inventoryCount |
NEW field |
sql query
view: cs_objects
SELECT o.objectID objectID, o.ObjectNumber objectNumber, NULL isComponent, CatRais pahmaCollection, d.Department pahmaTmsLegacyDepartment, HistAttributions objectHistoryNote, (CASE WHEN lower(LoanClass) not in ('', 'catalog card', 'inventory', 'ledger') THEN LoanClass END) pahmaTms2003DataSource, Markings distinguishingFeatures, Medium material, MTB_PlaceTermID fieldCollectionPlace, Notes comments, s.ObjectStatus pahmaObjectStatus, t.ObjectType collection, i.CSpaceID nagpraInventoryName, MTB_Portfolio portfolioSeries, Provenance fieldLocVerbatim, (CASE WHEN MTB_ResponsibleDept = 1 THEN 'Natasha Johnson' WHEN MTB_ResponsibleDept = 2 THEN 'Leslie Freund' WHEN MTB_ResponsibleDept = 3 THEN 'Victoria Bradshaw' WHEN MTB_ResponsibleDept = 4 THEN 'Alicja Egbert' WHEN MTB_ResponsibleDept = 5 THEN 'No collection manager (Registration)' WHEN MTB_ResponsibleDept = 8 THEN 'uncertain' END) responsibleDepartment, Signed inscriptionContent, b.optionID pahmaNagpraCodeLegacy, Title title, c.integer3 inventoryCount FROM Objects o LEFT OUTER JOIN Departments d ON o.departmentID = d.departmentID LEFT OUTER JOIN ObjectStatuses s ON o.objectStatusID = s.objectStatusID LEFT OUTER JOIN ObjectTypes t ON o.objectTypeID = t.objectTypeID LEFT OUTER JOIN MTB_NagpraInvNames i ON o.paperFileRef = i.LongTMSname LEFT OUTER JOIN MTB_NagpraBurialCodes b ON o.state = b.code LEFT OUTER JOIN ObjContext c ON o.objectID = c.objectID WHERE o.IsVirtual = 0 AND o.IsTemplate = 0 UNION ALL SELECT 'c' + CAST(c.componentID AS VARCHAR) objectID, c.componentNumber objectNumber, 'Component' isComponent, NULL pahmaCollection, NULL pahmaTmsLegacyDepartment, NULL objectHistoryNote, NULL pahmaTms2003DataSource, NULL distinguishingFeatures, NULL material, NULL fieldCollectionPlace, NULL comments, NULL pahmaObjectStatus, NULL collection, NULL nagpraInventoryName, NULL portfolioSeries, NULL fieldLocVerbatim, NULL responsibleDepartment, NULL inscriptionContent, NULL pahmaNagpraCodeLegacy, NULL title, NULL inventoryCount FROM ObjComponents c WHERE c.objectID IN (SELECT t.objectID FROM ObjComponents t GROUP BY t.objectID HAVING count(*) > 1) ;
notes
- object components are included as first-class objects if and only if there are multiple components for parent object
- object components must have componentID (objectID) that does not conflict with any parent objectID: we can prepend "c" to the componentID to ensure this, per Michael (1/12/12)
- many object components have a non-unique componentNumber (objectNumber), duplicated by other components and/or parent object; these values are OK to import as-is, per Michael (1/12/12)
- "componentType" field does not need to be imported, per Michael (1/12/12)
- "inscribed" field has only a few entries, which can be updated manually after data migration, per Michael (12/15/11)
- DONE: add 6 new custom fields: pahmaCollection, objectStatus, pahmaTms2003DataSource, pahmaTmsLegacyDepartment, inventoryCount, portfolioSeries
- DONE: add controlled list options for 5 of these new fields (all except inventoryCount)
- DONE: nagpraInventoryName field values need to be transformed to match option ID values in existing controlled list
- TODO: "dimensions" field still needs to be dealt with (John Lowe is working on this as of 1/9/12)
- TODO: add isComponent field for indicating components; field should be hidden in main part of UI but included in display name at the top
Object-Object Relationships extract
sql query
view: cs_objectobjectrelationships
SELECT c.objectID parentObjectID, 'c' + CAST(c.componentID AS VARCHAR) childObjectID FROM ObjComponents c WHERE c.objectID IN (SELECT t.objectID FROM ObjComponents t GROUP BY t.objectID HAVING count(*) > 1) ;
notes
- this should be imported as two-way relationship, so it shows up in UI for both parent and child object
Number of Objects extract
field mapping
TMS table name |
TMS field name |
CSpace field name |
notes |
---|---|---|---|
Objects |
ObjectCount |
numberOfObjects |
|
Components |
CompCount |
numberOfObjects |
|
sql query
view: cs_numberofobjects
SELECT o.objectID, o.objectCount numberOfObjects FROM Objects o WHERE o.IsVirtual = 0 AND o.IsTemplate = 0 AND o.objectID NOT IN (SELECT t.objectID FROM ObjComponents t GROUP BY t.objectID HAVING count(*) = 1) UNION ALL SELECT o.objectID, (CASE WHEN c.compCount > o.objectCount THEN c.compCount ELSE o.objectCount END) numberOfObjects FROM Objects o JOIN ObjComponents c ON o.objectID = c.objectID WHERE o.IsVirtual = 0 AND o.IsTemplate = 0 AND c.objectID IN (SELECT t.objectID FROM ObjComponents t GROUP BY t.objectID HAVING count(*) = 1) UNION ALL SELECT 'c' + CAST(c.componentID AS VARCHAR) objectID, c.compCount numberOfObjects FROM ObjComponents c WHERE c.objectID IN (SELECT t.objectID FROM ObjComponents t GROUP BY t.objectID HAVING count(*) > 1) ;
notes
- for objects with single component, componentCount should be used in lieu of objectCount if and only if it is greater than objectCount, per Michael (1/5/12)
Alternate Numbers extract
field mapping
TMS table name |
TMS field name |
CSpace field name |
notes |
---|---|---|---|
AltNums |
AltNum |
pahmaAltNum |
|
Components |
ComponentNumber |
pahmaAltNum |
|
AltNums |
Description |
pahmaAltNumType |
controlled list |
AltNums |
Remarks |
pahmaAltNumNote |
|
sql query
view: cs_alternatenumbers
SELECT n.ID objectID, n.altnum pahmaAltNum, n.description pahmaAltNumType, n.remarks pahmaAltNumNote FROM AltNums n WHERE n.tableID = 108 UNION ALL SELECT c.objectID, c.componentNumber pahmaAltNum, NULL pahmaAltNumType, NULL pahmaAltNumNote FROM ObjComponents c JOIN Objects o ON c.objectID = o.objectID WHERE o.IsVirtual = 0 AND o.IsTemplate = 0 AND c.objectID IN (SELECT t.objectID FROM ObjComponents t GROUP BY t.objectID HAVING count(*) = 1) AND c.componentNumber IS NOT NULL AND c.componentNumber != o.objectNumber ;
notes
- DONE: 3 new custom fields in repeating field group
- DONE: add controlled list options for pahmaAltNumType field
Object Names extract
field mapping
TMS table name |
TMS field name |
CSpace field name |
notes |
---|---|---|---|
Objects |
ObjectName |
objectName |
|
Components |
ComponentName |
objectName |
|
ObjectNames |
ObjectName |
objectName |
|
ObjectNames |
LanguageID |
objectNameLanguage |
FK to Languages |
ObjectNames |
ObjectNameTypeID |
objectNameNote |
FK to ObjectNameTypes; concatenate with Remarks |
ObjectNames |
Remarks |
objectNameNote |
concatenate with ObjectNameType |
sql query
view: cs_objectnames
SELECT o.objectID, o.objectName, NULL objectNameLanguage, NULL objectNameNote FROM Objects o WHERE o.IsVirtual = 0 AND o.IsTemplate = 0 UNION ALL SELECT 'c' + CAST(c.componentID AS VARCHAR) objectID, c.componentName objectName, NULL objectNameLanguage, NULL objectNameNote FROM ObjComponents c WHERE c.objectID IN (SELECT t.objectID FROM ObjComponents t GROUP BY t.objectID HAVING count(*) > 1) AND c.componentName IS NOT NULL UNION ALL SELECT c.objectID, c.componentName objectName, NULL objectNameLanguage, 'Component name' objectNameNote FROM ObjComponents c JOIN Objects o ON c.objectID = o.objectID WHERE o.IsVirtual = 0 AND o.IsTemplate = 0 AND c.objectID IN (SELECT t.objectID FROM ObjComponents t GROUP BY t.objectID HAVING count(*) = 1) AND c.componentName IS NOT NULL AND coalesce(o.objectName,'') != c.componentName UNION ALL SELECT n.objectID, n.objectName, (CASE WHEN n.languageID != '0' THEN l.MTB_ISOcode END) objectNameLanguage, (CASE WHEN n.objectNameTypeID != '0' THEN t.objectNameType ELSE '' END) + (CASE WHEN n.objectNameTypeID != '0' AND n.remarks is not null THEN '; ' ELSE '' END) + coalesce(n.remarks, '') objectNameNote FROM objectNames n LEFT OUTER JOIN objectNameTypes t ON n.objectNameTypeID = t.objectNameTypeID LEFT OUTER JOIN languages l ON n.languageID = l.languageID WHERE n.objectName IS NOT NULL ;
notes
- TODO: change objectNameLanguage from static term list to vocabulary-controlled list
Brief Descriptions extract
field mapping
TMS table name |
TMS field name |
CSpace field name |
notes |
---|---|---|---|
Objects |
description |
briefDescription |
|
Components |
physDesc |
briefDescription |
|
sql query
view: cs_briefdescriptions
SELECT o.objectID, o.description briefDescription FROM Objects o WHERE o.IsVirtual = 0 AND o.IsTemplate = 0 UNION ALL SELECT 'c' + CAST(c.componentID AS VARCHAR) objectID, c.physDesc briefDescription FROM ObjComponents c WHERE c.objectID IN (SELECT t.objectID FROM ObjComponents t GROUP BY t.objectID HAVING count(*) > 1) AND c.physDesc IS NOT NULL UNION ALL SELECT c.objectID, c.physDesc briefDescription FROM ObjComponents c WHERE c.objectID IN (SELECT t.objectID FROM ObjComponents t GROUP BY t.objectID HAVING count(*) = 1) AND c.physDesc IS NOT NULL ;
Object Titles extract
field mapping
TMS table name |
TMS field name |
CSpace field name |
notes |
---|---|---|---|
ObjTitles |
Title |
title |
|
ObjTitles |
LanguageID |
titleLanguage |
FK to Languages |
ObjTitles |
TitleTypeID |
titleType |
FK to TitleTypes |
sql query
view: cs_objecttitles
SELECT objectID, title, (CASE WHEN n.languageID != '0' THEN l.MTB_ISOcode END) titleLanguage, (CASE WHEN n.titleTypeID != '0' THEN t.titleType END) titleType FROM objTitles n LEFT OUTER JOIN titleTypes t ON n.titleTypeID = t.titleTypeID LEFT OUTER JOIN languages l ON n.languageID = l.languageID ;
notes
- "remarks" field has only a few entries, which can be updated manually after data migration, per Michael (12/15/11)
- TODO: change titleLanguage from static term list to vocabulary-controlled list
Ethnographic File Codes extract
field mapping
TMS table name |
TMS field name |
CSpace field name |
notes |
---|---|---|---|
ObjUserFields |
FieldValue |
pahmaEthnographicFileCode |
|
sql query
view: cs_ethnographicfilecodes
SELECT u.objectID, c.CSoptionID pahmaEthnographicFileCode FROM ObjUserFields u JOIN MTB_EthnographicUseCodes c ON u.FieldValue = c.FieldValue WHERE u.FieldTypeID = 17 ;
notes
- pahmaEthnographicFileCode is currently a static term list, but it will be changed to a Concept Authority reference once that authority is available
Associated Dates extract
this data cannot be imported until v2.1
field mapping
TMS table name |
TMS field name |
CSpace field name |
notes |
---|---|---|---|
ObjDates |
EventType |
assocDateType |
|
ObjDates |
DateText |
assocDate_dateDisplayDate |
|
ObjDates |
DateBegSearch |
assocDate_dateEarliestSingleYear |
|
ObjDates |
DateEndSearch |
assocDate_dateLatestYear |
|
ObjDates |
DayBegSearch |
assocDate_dateEarliestSingleDay |
|
ObjDates |
DayEndSearch |
assocDate_dateLatestDay |
|
ObjDates |
MonthBegSearch |
assocDate_dateEarliestSingleMonth |
|
ObjDates |
MonthEndSearch |
assocDate_dateLatestMonth |
|
ObjDates |
Remarks |
assocDateNote |
|
sql query
view: cs_associateddates
SELECT objectID, lower(eventType) assocDateType, dateText assocDate_dateDisplayDate, (CASE WHEN dateBegSearch != '0' THEN dateBegSearch END) assocDate_dateEarliestSingleYear, (CASE WHEN dateEndSearch != '0' THEN dateEndSearch END) assocDate_dateLatestYear, dayBegSearch assocDate_dateEarliestSingleDay, dayEndSearch assocDate_dateLatestDay, monthBegSearch assocDate_dateEarliestSingleMonth, monthEndSearch assocDate_dateLatestMonth, remarks assocDateNote FROM ObjDates WHERE lower(eventType) not in ('', 'accessioned', 'date collected', 'secondary collection date', 'tertiary collection date', 'date made') AND (dateText is not null OR dateBegSearch != 0) ;
notes
- DONE: populate controlled list options for assocDateType field
- TODO: wait until assocDate field is changed into a structured date (in v2.1; see CSPACE-4775)
Field Collection Dates extract
this data cannot be imported until v2.1
field mapping
TMS table name |
TMS field name |
CSpace field name |
notes |
---|---|---|---|
ObjDates |
DateText |
fieldCollectionDate_dateDisplayDate |
|
ObjDates |
DateBegSearch |
fieldCollectionDate_dateEarliestSingleYear |
|
ObjDates |
DateEndSearch |
fieldCollectionDate_dateLatestYear |
|
ObjDates |
DayBegSearch |
fieldCollectionDate_dateEarliestSingleDay |
|
ObjDates |
DayEndSearch |
fieldCollectionDate_dateLatestDay |
|
ObjDates |
MonthBegSearch |
fieldCollectionDate_dateEarliestSingleMonth |
|
ObjDates |
MonthEndSearch |
fieldCollectionDate_dateLatestMonth |
|
ObjDates |
Remarks |
fieldCollectionNote |
|
Objects |
Dated |
fieldCollectionDate_dateDisplayDate |
|
Objects |
DateBegin |
fieldCollectionDate_dateEarliestSingleYear |
|
Objects |
DateEnd |
fieldCollectionDate_dateLatestYear |
|
sql query
view: cs_fieldcollectiondates
SELECT objectID, dateText fieldCollectionDate_dateDisplayDate, (CASE WHEN dateBegSearch != '0' THEN dateBegSearch END) fieldCollectionDate_dateEarliestSingleYear, (CASE WHEN dateEndSearch != '0' THEN dateEndSearch END) fieldCollectionDate_dateLatestYear, dayBegSearch fieldCollectionDate_dateEarliestSingleDay, dayEndSearch fieldCollectionDate_dateLatestDay, monthBegSearch fieldCollectionDate_dateEarliestSingleMonth, monthEndSearch fieldCollectionDate_dateLatestMonth, remarks fieldCollectionNote FROM ObjDates WHERE lower(eventType) in ('date collected', 'secondary collection date', 'tertiary collection date') UNION SELECT objectID, dated fieldCollectionDate_dateDisplayDate, (CASE WHEN DateBegin != '0' THEN DateBegin END) fieldCollectionDate_dateEarliestSingleYear, (CASE WHEN DateEnd != '0' THEN DateEnd END) fieldCollectionDate_dateLatestYear, null fieldCollectionDate_dateEarliestSingleDay, null fieldCollectionDate_dateLatestDay, null fieldCollectionDate_dateEarliestSingleMonth, null fieldCollectionDate_dateLatestMonth, null fieldCollectionNote FROM Objects WHERE dated is not null OR dateBegin != '0' OR dateEnd != '0' ;
notes
- differences in eventType ('date collected', 'secondary collection date', 'tertiary collection date') are not important and do not need to be migrated, per Michael (12/15/11)
- TODO: wait until fieldCollectionDate field is changed into a structured date (in v2.1; see CSPACE-4775)
- TODO: change fieldCollectionDate field into a repeating field
Production Date extract
field mapping
TMS table name |
TMS field name |
CSpace field name |
notes |
---|---|---|---|
ObjDates |
DateText |
objectProductionDate_dateDisplayDate |
|
ObjDates |
DateBegSearch |
objectProductionDate_dateEarliestSingleYear |
|
ObjDates |
DateEndSearch |
objectProductionDate_dateLatestYear |
|
ObjDates |
DayBegSearch |
objectProductionDate_dateEarliestSingleDay |
|
ObjDates |
DayEndSearch |
objectProductionDate_dateLatestDay |
|
ObjDates |
MonthBegSearch |
objectProductionDate_dateEarliestSingleMonth |
|
ObjDates |
MonthEndSearch |
objectProductionDate_dateLatestMonth |
|
ObjDates |
Remarks |
objectProductionNote |
|
sql query
view: cs_productiondates
SELECT objectID, dateText objectProductionDate_dateDisplayDate, (CASE WHEN dateBegSearch != '0' THEN dateBegSearch END) objectProductionDate_dateEarliestSingleYear, (CASE WHEN dateEndSearch != '0' THEN dateEndSearch END) objectProductionDate_dateLatestYear, dayBegSearch objectProductionDate_dateEarliestSingleDay, dayEndSearch objectProductionDate_dateLatestDay, monthBegSearch objectProductionDate_dateEarliestSingleMonth, monthEndSearch objectProductionDate_dateLatestMonth, remarks objectProductionNote FROM ObjDates WHERE lower(eventType) = 'date made' ;
notes
Bibliographic References extract
field mapping
TMS table name |
TMS field name |
CSpace field name |
notes |
---|---|---|---|
Objects |
Bibliography |
referenceNote |
|
Objects |
PubReferences |
referenceNote |
|
sql query
view: cs_bibliographicreferences
SELECT objectID, bibliography referenceNote FROM Objects WHERE bibliography is not null UNION ALL SELECT objectID, PubReferences referenceNote FROM Objects WHERE PubReferences is not null ;
Annotations extract
field mapping
TMS table name |
TMS field name |
CSpace field name |
notes |
---|---|---|---|
Objects |
Edition |
annotationNote |
|
Objects |
CuratorialRemarks |
annotationNote |
|
Components |
PrepComments |
annotationNote |
|
ObjUserFields |
FieldValue |
annotationNote |
|
sql query
view: cs_annotations
SELECT objectid, edition annotationNote, 'TMS Edition field' annotationType FROM Objects WHERE edition IS NOT NULL UNION ALL SELECT objectid, curatorialRemarks annotationNote, 'TMS Curatorial remarks' annotationType FROM Objects WHERE curatorialRemarks IS NOT NULL UNION ALL SELECT c.objectID objectID, c.prepComments annotationNote, 'Component Prep comments' annotationType FROM ObjComponents c WHERE c.objectID IN (SELECT t.objectID FROM ObjComponents t GROUP BY t.objectID HAVING count(*) = 1) AND c.prepComments IS NOT NULL UNION ALL SELECT 'c' + CAST(c.componentID AS VARCHAR) objectID, c.prepComments annotationNote, 'Component Prep comments' annotationType FROM ObjComponents c WHERE c.objectID IN (SELECT t.objectID FROM ObjComponents t GROUP BY t.objectID HAVING count(*) > 1) AND c.prepComments IS NOT NULL UNION ALL SELECT u.objectID, u.fieldValue annotationNote, t.userFieldType annotationType FROM ObjUserFields u JOIN UserFieldTypes t ON u.fieldTypeID = t.userFieldTypeID WHERE u.FieldTypeID != 17 ;
notes
- some annotationType field values set by query above
- TODO: update controlled list options for annotationType field
- TODO: include some data from TextEntries table in this query...?
Groups extract
to be imported into the Groups Procedure schema
field mapping
TMS table name |
TMS field name |
CSpace field name |
notes |
---|---|---|---|
ObjectPackages |
ObjectPackageID |
shortIdentifier |
|
ObjectPackages |
name |
title |
|
ObjectPackages |
notes |
scopeNote |
|
ObjectPackages |
owner |
owner |
FK to Users; reference to Person Authority |
sql query
view: cs_groups
SELECT objectPackageID groupID, name title, notes scopeNote, (CASE WHEN u.MTB_ConstituentID = -1 THEN NULL ELSE u.MTB_ConstituentID END) owner FROM ObjectPackages p LEFT OUTER JOIN Users u ON p.owner = u.login ;
notes
- "global" field does not need to be imported, per Michael (1/12/12)
Object-Group Relationship extract
field mapping
TMS table name |
TMS field name |
CSpace field name |
notes |
---|---|---|---|
ObjPkgList |
ObjectID |
objectID |
|
ObjPkgList |
ObjectPackageID |
groupID |
|
sql query
view: cs_objectgrouprelationships
SELECT objectID, objectPackageID groupID FROM objPkgList ;