Overview of Languages, Tools, and Libraries

Owned by Adaptavist ScriptRunner for Confluence Cloud
Oct 28, 2020
9 min read

This document provides an overview of the languages, tools, and libraries used in CollectionSpace JS development.

Languages

There's only one. The CollectionSpace JS projects are written in JavaScript (JS).

JavaScript

Language features of JavaScript ES2015 (also known as ES6), ES2016, and ES2017 are used throughout. Support for the latest JavaScript features is inconsistent among browsers. Cross-browser compatibility is achieved by compiling to the lowest-common-denominator among the supported browsers at build-time, using Babel and the Babel Env preset.

One unstandardized language feature, currently a Stage 4 (Finished) proposal, is also used: Object Rest/Spread.

Development Tools and Libraries

The following tools and libraries are used for development.

Node.js

Node.js provides a JavaScript interpreter and libraries to run JS programs outside of a web browser. It is required to run the other development tools, all of which are JS programs. Version 8 is recommended.

npm

npm is a package manager for JavaScript. Its primary purpose is to download and keep up-to-date the external libraries used by a project. It also provides some simple build management capabilities.

CollectionSpace JS projects are distributed as npm packages. The source code for each project contains a top-level package.json file. This file contains package metadata, including version numbers of development and runtime dependencies, and scripts to perform development tasks such as linting, compiling, and testing. Since npm acts as the entry point, it's usually the only build tool you need to know. The other build tools normally do not need to be invoked directly.

To begin work on a project, clone it from GitHub, then run npm install. This downloads the project's dependencies, and installs them in the node_modules directory:

$ git clone https://github.com/collectionspace/cspace-ui.js.git
$ cd cspace-ui.js
$ npm install

To see the available scripts in a project, execute npm run with no arguments:

$ npm run
Lifecycle scripts included in cspace-ui:
  prepublish
    ...
  start
    ...
  test
    ...
available via `npm run-script`:
  build
    ...
  build-cjs
    ...
  build-umd
    ...
  build-umd-dev
    ...
  build-umd-prod
    ...
  check
    ...
  clean
    ...
  ...

To run a script, execute npm run followed by the name of the script:

$ npm run clean
...

Some scripts are part of npm's standard lifecycle. For these, the run part of the command may be omitted:

$ npm test
...

Important npm scripts common to all CollectionSpace JS projects include:

  • lint
    npm run lint will check the source code against linting rules. See ESLint.

  • test
    npm run test or npm test will run tests once. See Karma and nyc.

  • start
    npm run start or npm start will run tests continuously. See Karma.

  • coverage
    npm run coverage will generate test coverage reports. See Karma, nyc, and Instanbul.

  • check
    npm run check will run lint, test, and coverage. A pull request must pass check in order to be accepted.

  • build
    npm run build will generate JavaScript bundles for distribution (a development bundle and a minified production bundle). See webpack.

  • clean
    npm run clean will delete all files generated by development scripts, including distribution bundles and coverage data.

Babel

Babel is a JavaScript-to-JavaScript compiler. It is used to compile newer versions of JavaScript down to an older version that works in a wider range of browsers. Even as browser support for the latest version of JS advances, Babel will continue to provide value; language features of the next version of JavaScript may be used in advance of full compatibility across browsers. Additionally, Babel provides a place to plug in other useful static analysis and transformation tools. These include:

  • JSX transformation
    Compile JSX to JS. JSX is an optional XML format for describing React components. React components may be defined directly in JavaScript, but JSX is usually more readable. In CollectionSpace source code, React components are always defined in JSX.

  • Dev expression removal
    Strip certain development-only code from production builds. This is used to enable printing of verbose warning messages when running in development, without impacting the size or performance of the production build.

  • Test coverage instrumentation
    Modify the source code to track the number of times each function, line, and branch are executed. When combined with a compatible test runner and report generator, this makes it easy to spot the parts of the codebase that are insufficiently tested.

  • i18n message extraction
    Generate JSON message definition files. Translatable messages in the CollectionSpace UI are defined inside the source files where they're used. This is convenient for developers, but inconvenient for translators. This finds and extracts the message definitions from the source code, so that files containing only message definitions may be provided to translators.

Babel is configured into various other tools, and never needs to be invoked directly. Many of the npm scripts in CollectionSpace JS projects will invoke Babel in some way.

webpack

webpack is a JavaScript module bundler. It takes source files in a project, including JS files and their dependencies, CSS files, and small images, and generates a single JS file for distribution. This serves two purposes:

  1. Reduce the number of HTTP requests a browser needs to make to bootstrap an application.
  2. Remove the possibility that a browser may have cached some files from an older version of the application that are incompatible with files from the current version. All of the assets for any version of the application may be bundled together as a unit, and only that unit may be cached.

webpack is configured into various other tools, and never needs to be invoked directly. Several of the npm scripts in CollectionSpace JS projects will invoke webpack, the main one being build. npm run build generates two bundles in the top-level dist directory: a development bundle, and a minified production bundle.

Karma

Karma is a JavaScript test runner. Running tests requires a lot of steps: the test code must be bundled with the code being tested and all of its dependencies; a browser must be started; a server must be started; the browser must make a request to the server, and be served the test bundle to run; the browser must run the tests; the browser must report the results back to the server, which can then output a report; and the browser must be closed. Karma handles all of this, and can start multiple browsers, run the tests in all of them, and aggregate their results. By default, only Chrome is used on a local test run. A full suite of browsers is used on a CI server test run.

The test bundles generated by Karma are configured to be instrumented for coverage reporting, and Karma is configured to output raw coverage data to the top-level coverage directory. The coverage directory will contain a subdirectory for each browser in which tests are run. Istanbul may be used to generate reports from this raw data.

Karma generally never needs to be invoked directly. A few of the npm scripts in CollectionSpace JS projects will invoke Karma, the main ones being test and start. npm [run] test will run tests once, opening and closing a browser, and reporting the results. npm [run] start will start Karma in continuous mode. In this mode, a browser will be opened, and tests will be run once; then the browser will remain open, and as source files are edited and saved, the changes will be detected, and the tests will automatically be rerun. During development you'll usually want to keep terminal and browser windows open off to the side, continuously running tests.

ESLint

ESLint is a JavaScript code linter. It is used to catch potential errors and to enforce style. The linting rules used by CollectionSpace JS projects are sourced from Airbnb, mainly because they are a reasonable set of rules that are maintained by people who are not us. The Airbnb style guide should be followed, even where it is not enforced by linting rules.

ESLint never needs to be invoked directly. The npm lint and check scripts will invoke it.

Mocha

Mocha is the JS test framework used for writing tests in cspace.js projects. It is a library that provides functions like describe() and before(), which form the structure of the tests. Karma is configured to make the Mocha library available to the test bundles running in the browser.

Mocha is also itself a test runner, but it never needs to be invoked directly. It is wrapped into Karma, so npm test is all you need. Developers writing tests should be familiar with the Mocha API.

Chai

Chai is the assertion library used to make assertions in tests. It provides a fluent, chainable JavaScript API for describing expected results, for example, result.should.be.an('object').with.property('count', 3). Karma is configured to make the Chai library available to the test bundles running in the browser. Developers writing tests should be familiar with the Chai API.

nyc

nyc is a test runner for Node.js. Certain cspace.js projects – typically, lower level libraries that have no visible UI – such as cspace-api.js and cspace-client.js, are designed to be usable both in browsers and in Node.js. Karma is great for testing code in browsers, but these projects need to be tested in Node.js as well. Like Karma, nyc wraps Mocha. Mocha itself is a test runner that will run tests in Node.js, but it is not aware of test coverage instrumentation. nyc exists in the development stack for one reason: to output test coverage data from Node.js tests, as Karma is doing for browser tests. This coverage data goes into the coverage/Node.js directory, alongside coverage data gathered from browser tests.

nyc never needs to be invoked directly. In the projects that need it, npm test will execute tests in both nyc and Karma.

Istanbul

Istanbul is used to generate test coverage reports from the raw data output by Karma and nyc.

Istanbul never needs to be invoked directly. npm run coverage will use it to generate reports from the data in the top-level coverage directory, which, after running tests, will include data from all browser and Node.js tests. The generated reports will aggregate the data from all the tests, and output a textual report to the terminal, and an HTML report to the coverage/lcov-report directory.

Cloud Services

The following cloud services are used in CollectionSpace JS development.

Travis CI

Travis CI provides a cloud-based continuous integration service that is integrated into GitHub. In any cspace.js project, pushing to master will trigger a Travis CI build. Each Travis CI build executes npm run check, so linting errors or test failures will cause the build to fail. Any pull request made will also trigger a build of that pull request, and the results will be reported on GitHub. Pull requests must build successfully to be accepted.

The output from each build is publicly viewable, and may be checked for error messages when builds fail.

Sauce Labs

Sauce Labs provides a cloud service that runs tests in a large number of browser/OS combinations. In CollectionSpace JS projects, Sauce Labs support is configured into Travis CI and Karma, so that when tests are run in Travis CI, Karma will start remote browsers in the Sauce Labs cloud, and run the tests in those browsers. The browsers tested in Sauce represent all of the browsers supported by CollectionSpace.

Coveralls

Coveralls is a test coverage reporting service that makes coverage reports publicly available on the web. In cspace.js projects, Travis CI is configured to send the raw test coverage data generated by Karma to Coveralls, which then generates and makes available a coverage report for each build. The coverage report generated by Coveralls is not as detailed as the report generated by Istanbul; in particular, it does not show missed branches. Coveralls is mainly intended to provide a quick way for potential users of a project to evaluate its fitness. It is also integrated into GitHub, and will post notifications on pull requests describing how the pull request affects test coverage. A pull request will not be accepted if it decreases test coverage by more than 5%, or to below 90%.

GitHub

GitHub hosts the source code for CollectionSpace JS projects. The repositories available here are intended to be used by developers who are writing code for cspace.js projects. Pull requests are welcome.

All repositories are hosted in the CollectionSpace organization. The cspace.js project repositories have names that end with ".js", such as cspace-client.js and cspace-ui.js. Each project corresponds 1:1 with a GitHub repository.

npm

"npm" refers both to the package management tool described above, and in this context, the repository of JavaScript packages hosted at npmjs.com. The packages available here are intended to be used by developers writing code for their own projects that utilize cspace.js libraries.

Each cspace.js project corresponds 1:1 with an npm package. Packages are published to npm with the same name as the project's GitHub repository, excluding the ".js" suffix.

UNPKG

UNPKG is a (not-for-production) CDN that automatically distributes JavaScript bundles from packages published to npm. The JS bundles available here are intended to be used by CollectionSpace hosters and web developers who want to use CollectionSpace JS libraries on a web page. This is easily done by including a script tag on the page that loads a JS file from the CDN:

<script src="http://unpkg.com/cspace-ui/dist/cspaceUI.min.js"></script>
<script>
	cspaceUI();
</script>

See the CodePen demos for complete examples of using the cspace-ui package on a web page.

Each cspace.js project will have two bundles hosted on UNPKG: a development bundle (named with the ".js" extension), and a minified production bundle (named with the ".min.js" extension). 

For production you'll want to download the JS file you're using, and host it on your own server or another CDN with uptime guarantees that meet your needs.

CodePen

CodePen hosts demos of configuring and displaying the CollectionSpace UI on a web page. There's no particular reason to use CodePen over similar services like JSFiddle or JS Bin. Those other services may be used in the future. Note that in order to access the REST API on nightly.collectionspace.org from a page hosted on another service, the CORS configuration on nightly must be amended with that service's hostname.