mirror of
https://gitlab.com/wgp/dougal/software.git
synced 2025-12-06 09:07:09 +00:00
14a2f57c8d
The results are now saved as follows:
For shot QCs, failing tests result in an event being created in
the event_log table. The text of the event is the QC result message,
while the labels are as set in the QC definition. It is conventionally
expected that these include a `QC` label. The event `meta` contains a
`qc_id` attribute with the ID of the failing QC.
For sequences, failing tests result in a `meta` entry under `qc`, with
the QC ID as the key and the result message as the value.
Finally, the project's `info` table still has a `qc` key, but unlike
with the old code, which stored all the QC results in a huge object
under this key, now only the timestamp of the last time a QC was run on
this project is stored, as `{ "updatedOn": timestamp }`.
The QCs are launched by calling the main() function in /lib/qc/index.js.
This function will first check the timestamp of the files imported into
the project and only run QCs if any of the file timestamps are later
than `info.qc.updatedOn`. Likewise, for each sequence, the timestamp of
the files conforming that sequence is checked against
`info.qc.updatedOn` and only those which are newer are actually
processed. This cuts down the running time very considerably.
The logic now is much easier on memory too, as it doesn't load the
whole project at once into memory. Instead, shotpoint QCs are processed
first, and for this a cursor is used, fetching one shotpoint at a
time. Then the sequence QCs are run, also one sequence at a time
(fetched via an individual query touching the `sequences_summary` view,
rather than via a cursor; we reuse some of the lib/db functions here),
for each sequence all its shotpoints and a list of missing shots are
also fetched (via lib/db function reuse) and passed to the QC functions
as predefined variables.
The logic of the QC functions is also changed. Now they can return:
* If a QC passes, the function MUST return boolean `true`.
* If a QC fails, the function MAY return a string describing the nature
of the failure, or in the case of an `iterate: sequence` type test,
it may return an object with these attributes:
- `remarks`: a string describing the nature of the failure;
- `labels`: a set of labels to associate with this failure;
- `shots`: a object in which each attribute denotes a shotpoint number
and the value consists of either a string or an object with
`remarks` (string), `labels` (array of strings) attributes. This allows
us to add detail about which shotpoints exactly contribute to cause a
sequence-wide test failure (this may not be applicable to every
sequence-wide QC) and it's also a handy way to detect and insert events
for missing shots.
* For QCs which may give false positives, such as missing gun data, a
new QC definition attribute is introduced: if `ignoreAllFailed` is
boolean `true` and all shots fail the test for a sequence, or all
sequences fail the test for a prospect, the results of the QC will be
ignored, as if the test had passed. This is mostly to deal with gun or
any other data that may be temporarily missing.
Dougal web server
Backend server for the user interface
Install
Clone / copy this directory to the destination directory and then run
npm install
Run
In production this should be run as a system service, but for development or a one-off run:
node index.js
or
npm start
Options
The following environment variables affect the behaviour of the application:
NODE_ENV: IfNODE_ENV=testsome output messages are suppressed.HTTP_PORT: Port to listen on. Default is 3000.HTTP_PATH: Path to prepend to the API routes. Default is null.DOUGAL_API_CONFIG: Location of the API configuration file. Default is$HOME/etc/www/config.json.
The server always listens on 127.0.0.1. Use a proxy like Nginx to expose it to the network.
Use
The API specification may be found under ./spec.
Generate the documentation with:
./node_modules/.bin/redoc-cli bundle ./spec/openapi.yaml