Page Comparison

...

• Workflows: System Administration (User management, Group management)

• User Accounts - Accounts, Authentication, Groups, Connections

• Workflows: CSV Import, Validation, and Transfer or Delete
  

Data Flow Sketch (import/update) - IN PROGRESS

...

CSV ingest

1. Options

   1. Set batch-specific import settings

2. Errors/outcomes

   1. File cannot be read/parsed (everything fails)

   2. Invalid character encoding that does not render entire file unreadable (report row/column containing problem)

3. Code actions

   1. Split into separate rows and assign RowID corresponding to row sequence to facilitate errors/warnings that can usefully reference a row from the imported CSV

4. Display

   1. Count of successfully imported rows

5. User actions

   1. Initiate initial validation and record status check

...

Initial validation

1. Code actions, errors, outcomes

   1. Creates Mapper::DataHandler connection/object for use with the batch

   2. FIRST ROW sent to Mapper for validation (DataHandler.validate(data_hash))

      1. If it gets a “Missing Required Field” error

         1. Do not send any more rows

         2. Tell user that CSV is missing a required column. They need to add that and reimport

         3. WORKFLOW STOPS HERE

      2. If it gets no error or a “Required Field Empty” error, continue sending rows for validation

   3. As additional rows are returned from the Mapper with no errors, tag them as “New” or “Existing”

2. Result display

   1. Count of valid and invalid records

   2. Count of new and existing records

   3. Columns - (f) indicates filterable column

      1. RowID

      2. RecordID value

      3. (f) Valid? (y/n)

      4. (f) Status (new/existing)

3. User actions

   1. Filter or select records and…

   2. Export selected to CSV (will do that and return to or remain on this view)

   3. Remove selected from batch

   4. Initiate Data Preparation

Data Preparation

Code actions, errors/warnings, outcomes

...

Multivalued fields are split, transformations are applied to field values, data quality checks are run

User interaction and design

...