Adding a repeatable group

This is a summary document intended for project team members and early implementers who work closely with the project team. It should be made useful to a broader audience of implementers by adding further structure and more detail, possibly along the lines of How to make a field multivalued (i.e. hold multiple values).

Stage 1 Services

Shut down the CSpace and Nuxeo servers

Edit XML Schema files

Check out a copy of the source code for the CollectionSpace services layer

Unknown macro: {multi-excerpt-include}

Add the repeatable group to the primary (Nuxeo) services schema file

In the directory with the services source code you've downloaded, edit the primary record schema definition file: services/{name of record type}/3rdparty/nuxeo-platform-cs-{name of service}/src/main/resources/schemas/{name of schema file}.xsd

For example, to move three fields into a repeatable group in an Intake record, you would edit the file: services/intake/3rdparty/nuxeo-platform-cs-loanout/src/main/resources/schemas/intakes_common.xsd)

Within that file, move one or more fields within a multivalued group, as follows:

For example, change this:

To this:

These changes to the schema files result in the creation of a multivalued group in the XML payloads for Intake records, sent to and from the Intake service, that looks something like this:

Add the repeatable group to the secondary (JAXB) services schema file

In the directory with the services source code you've downloaded, edit the secondary record schema definition file: services/{name of record type}/jaxb/src/main/resources/{name of schema file}.xsd

In the example for an Intake record, you would edit the file: services/intake/jaxb/src/main/resources/intakes-common.xsd

Within that file, move one or more fields within a multivalued group. Follow the exact set of steps listed above in collectionspace:Add the repeatable group to the primary (Nuxeo) services schema file.

Build and deploy changes to XML Schema files

From the top level of the services module for the record type that you are changing; e.g. services/intake for the Intake service:
Build and install the service's artifacts to your local Maven repository by running mvn clean install -DskipTests
Deploy the changes to the servers by running ant deploy

From the module that builds the JAX-RS application, services/JaxRServiceProvider:
Build and deploy the changes to the servers by running ant deploy.

Start the Nuxeo and CSpace servers

Run tests

Run the relevant services client tests in the module for your record type.
E.g. for Intake records (starting from the services trunk):

cd services/intake/client
mvn test

Revise tests as needed

If there are errors, the source code in some services test classes or the XML data files they may use will need to be changed to reflect the schema changes made by adding repeatable group(s).

Look in services/{recordtype}/client/src/test/java/org/collectionspace/services/client/test/ for test classes like:

{recordtype}ServiceTest.java
{recordtype}AuthRefsTest.java
{recordtype}SearchTest.java

e.g. IntakeServiceTest.java, IntakeAuthRefsTest.java, and so on ...

Special circumstances

In addition to the basic process described above, the following special circumstances may occasionally require additional work be carried out in the services layer:

Fields that appear in list items

Has any list field been moved into the repeatable group?

That is, do of the fields that you've moved into that group appear in summary lists: that is, do they appear in the short summary records, containing only a subset of the fields in the full record, that are returned when you send a 'read list' or search request to the services?

Two ways to check on this:

When there is at least one record of that type present in the database, send a 'read list' request to the relevant service. E.g. for the Intake service, send a GET request to http://\{yourhost}:8180/cspace-services/intakes. View the list of fields returned for each item in the list.
In the services source code tree, view the source code of the relevant recordtype}}DocumentHandler class (e.g. IntakeDocumentHandler.java for intake records). You can find that file within the 'service' sub-module for the relevant {{recordtype module; e.g. services/intakes/service for the Intake service. The extractCommonPartList() method will contain the code used to populate each item in the summary lists.

In most cases, this won't be relevant. If it is, however, you will also need to:

Make an additional edit to the part of the JAXB XSD schema file (mentioned above) that defines the list item schema, to reflect the changes in the fields returned.
Edit the source code of the relevant {{recordtype}}DocumentHandler class (e.g. IntakeDocumentHandler.java for intake records) to reflect this change.
- Generally, the changes you'll need to make are all in the extractCommonPartList() method. These include changes to:
  - The value of the string returned from commonList.setFieldsReturned()
  - The building of each item in the list returned within the loop inside of that method; e.g. within while(iter.hasNext()) or the like.
Remove or change the field from the list of prefetch fields for the record type. In services trunk, in services/{recordtype}/3rdparty/nuxeo-platform-cs-{recordtype}src/main/resources/OSGI-INF/core-types-contrib.xml, edit the list of fields, if any, within the <prefetch> tag.

Computed fields

Has any field that is used in a computation, such as the automatic generation of a display name, been moved into the repeatable group? This could occur if you're adding a repeatable group to records for the items in an authority (e.g. Organization or Person records).

If so, edit the source code of the relevant {{recordtype}}DocumentHandler class (e.g. OrganizationDocumentHandler.java for Organization records) to reflect this change. Generally, the changes you'll need to make are in the handleDisplayNames() method or in code called from that method.

Authority reference fields

Has any field that holds authority references (terms from an authority, such as person or organization terms) been moved into the repeatable group?

If so, you will also need to edit the tenant bindings file, services/common/src/main/config/services/tenant-bindings.xml, to comment out the references to that field in the authRef properties for the relevant record type. Be sure to make this change for each relevant tenant in that file.

(Currently, due to a Nuxeo limitation, it's not possible to search within repeatable groups, and attempting to do so - to perform a so-called "complexType" search - can result in Exceptions.)

Large text fields

Is any field that must hold large amounts of text affected by the change?

If so, you may also need to edit the Nuxeo configuration file that defines 'largetext' fields. (Currently, when using MySQL, Nuxeo defaults to allowing just 500 characters for a field, except for fields explicitly defined as 'largetext'.)

The relevant configuration file is $JBOSS_HOME/templates/collectionspace_mysql/config/default-repository-config.xml. See CSPACE-3053 for details on editing that file.

To specify fields within repeatable groups as 'largetext', see CSPACE-3153.

Stage 2 Application

Add to application layer

Edit the main configuration file

In tomcat-main/src/main/resource/default.xml
surround existing field group with a <repeat> tag

e.g.

<repeat id="currentLocationList/currentLocationGroup">
  <!-- existing group of fields that will repeat as a group goes here -->
</repeat>

In this example, currentLocationGroup is the name this set of fields will be known by in the UI

currentLocationList/currentLocationGroup is the complex structure in the Services

or create an entire new repeat group, e.g.

<repeat id="acquisitionFundingList/acquisitionFunding">
    <field id="acquisitionFundingCurrency" autocomplete="vocab-currency" ui-type="enum">
    <enum>
           <default>USD</default>
    </enum>
    </field>
    <field id="acquisitionFundingValue"></field>
    <field id="acquisitionFundingSource" autocomplete="person-person,organization-organization"></field>
    <field id="acquisitionFundingSourceProvisos"></field>
</repeat>

Now redeploy the app layer. Don't forget the cspace-config.xml

Run XML/JSON tests

The tests in testXMLJSON and testJSONXML verify that the JSON is correctly transformed to the XML and the XML is correctly transformed to the JSON.

You will first need to change the test payloads in:
cspi-services/src/test/resources/org.collectionspace.chain.csp.persistence.services
application/trunk/svcapp/cspi-services/src/test/resources/org/collectionspace/chain/csp/persistence/services/

Usually the payloads are named {recordid}XMLJSON.xml and {recordid}JSON.json

Then, if you have a version of the app in Eclipse or another IDE, test the change in cspi-services/src/test/java/TestService.java

Run services tests

Test the change in tomcat-main/src/test/java/org.collectionspace.chain.csp.persistence.file
tomcat-main/src/test/java/org/collectionspace/chain/csp/persistence/
(is this just TestGeneral, or?)
This tests whether there are any errors when the payloads are sent through the app and into the services for basic CRUD+L functionality

testMultipleStoreTypes and testObjectList()

Payloads are in tomcat-main/src/test/resources/org.collectionspace.chain.csp.persistence

Edit sample data

Edit sample data in .../chain/csp/persistence/services to reflect the schema changes. (This sample data can optionally be used to populate a new or demonstration CollectionSpace system with sample records.)

E.g. for Intake records, edit dummydata-intake.xml

Stage 3 UI

Make the change in the UI

Edit an HTML template file

In the UI source code tree, edit the HTML template file for the relevant record type; e.g. IntakeTemplate.html

Find for the relevant group in that file that will be the target for the repeat behavior. (Generally, most groups of fields that are to be made repeatable are likely already defined as a non-repeatable group of fields.)

Unless you have specified the selector for the repeat in cspace-config.xml then the table row selector <tr class= in the ui should be csc-{recordid}-currentLocationGroup

Place the follow HTML (adapted for your fields) into main/webapp/html/{recordid}Template.html
and then redeploy the UI layer

e.g.

<div class="fl-container-flex fl-fix information-group" id="toggle-location">
  <div class="info-column">
    <div class="fl-container-flex header toggle" onclick="ShowHide('#toggle-location .main', this); return false;">
      <span class="header-toggle"><img src="../images/toggle-less.png" height="12" width="12" alt="toggle icon" /></span>
      Location
    </div>
    <div class="fl-container-flex fl-fix content main">
      <!-- Left-side column -->
      <div class="info-column fl-force-left">
        <div class="info-pair">
          <div class="header">
            <div class="label">Current Location</div>
          </div>
          <div class="content">
            <table>
              <thead>
                <tr>
                  <td>Current Location</td>
                  <td>Current Location Fitness</td>
                  <td>Current Location Note</td>
                </tr>
              </thead>
              <tbody>
                <tr class="csc-intake-currentLocationGroup">
                  <td><input type="text" class="input-alpha-table csc-intake-current-location" /></td>
                  <td><select class="input-select csc-intake-current-location-fitness" /></td>
                  <td><input type="text" class="input-alpha-table csc-intake-current-location-note" /></td>
                </tr>
              </tbody>
            </table>
          </div>
        </div>
      </div>
    </div>
  </div>
</div>

Edit local files used for testing

Update the UISpec file

In your browser, visit:

(e.g. http://localhost:8180/chain/intake/uispec for an Intake record, on a CollectionSpace system running on localhost)

Copy the source of that page, or open any file that was downloaded via your browser as a result of visiting that page and copy the entire file. There will now be some JSON data on your Clipboard.

Paste the contents into a Web-based tool that can "pretty print" this JSON data. (E.g. http://jsonlint.com)
Perform whatever action is required to initiate pretty printing. (E.g. on jsonlint.com, pressing the Validate button)
Copy the pretty printed contents to your Clipboard.

In the UI source code tree, edit the file src/main/webapp/html/uispecs/{name of record type}/uispec.json (e.g. src/main/webapp/html/uispecs/intake/uispec.json for an Intake record)

Paste the contents of your Clipboard into the file you're editing, and save the file.

Update the UI schema file

In your browser, visit:

(e.g. http://localhost:8180/chain/intake/uischema for an Intake record, on a CollectionSpace system running on localhost)

Copy the source of that page, or open any file that was downloaded via your browser as a result of visiting that page and copy the entire file. There will now be some JSON data on your Clipboard.

Paste the contents into a Web-based tool that can "pretty print" this JSON data. (E.g. http://jsonlint.com)
Perform whatever action is required to initiate pretty printing. (E.g. on jsonlint.com, pressing the Validate button)
Copy the pretty printed contents to your Clipboard.

In the UI source code tree, edit the file src/main/webapp/html/uischema/{name of record type}.json (e.g. src/main/webapp/html/uischema/intake.json for an Intake record)

Paste the contents of your Clipboard into the file you're editing, and save the file.

Edit the UI sample data file(s)

In the UI source code tree, edit each of the sample data file(s) in src/main/webapp/html/data/{name of record type} (e.g. src/main/webapp/html/data/intake for an Intake record). These files' names will each end in .json.

Things to check

NB. The UI layer can cache so you will have to do hard refreshes, or explicitly clear your browser's cache. After refreshing or cache clearing, going directly to the page to make sure the selectors etc. are as you think is always a good idea. (To view the generated HTML source, you can use Firebug or equivalent.)

Similarly checking that you remembered to redeploy the app and the app cspace-config.xml, and that the app has correctly restarted by going to http://\{host}:8180/chain/{recordid}/uispec is always a good idea.