This script runs the deferred imports. It is meant to
be called from a cronjob at regular intervals – every
one or two minutes is probably a good setting.
It checks if another instance is already running before
doing its thing.
If anything goes wrong (any of the called processes exits
with non-zero condition) it will send an alert to GitLab,
provided that the authorisation key is known.
In case of errors (or anything else of note), send_alert.py
can be used to push information to a GitLab alerts endpoint.
It is generic enough that it can be used with anything else, though.
If their respective configuration keys are not
defined in a survey configuration, the import
routines will print an informational message
and exit successfully.
Some columns have been renamed:
* ts0 → tstamp
* shotNumber → point
The ts1 column from events_timed has been removed.
Labels attached to a sequence / shot without an event
have been removed. All labels are now associated with events.
Changes to event views.
The endpoint /project/:project/configuration/:path(*)?
returns the contents of the survey configuration YAML
file for a given project.
To retrieve the full configuration:
* /project/:project/configuration
To retrieve a specific subset (e.g., binning parameters):
* /project/:project/configuration/binning
To retrieve a specific value (e.g., inline bin width):
* /project/:project/configuration/binning/I_width
It inserts `LDSP` and `FDSP` labels, if those exist
in the `labels` table, on the last and first shotpoints
of the day when a sequence is shot through midnight.
The server timezone is always set to UTC so the midnight
shot implicitly refers to UTC through this.
Still a work in progress. The recently viewed projects
list is meant to show the last three or so projects that
the user has visited on this computer, probably using localStorage.
It can show:
- all events (this could get slow);
- a single sequence;
- a set of sequences;
- a single date;
- a range between two dates.
It does not (yet) do pagination and filtering is local only.
Labels can be associated with events and can have
display properties such as a description and colour,
this is why we need an endpoint for the client to
retrieve them.
Events may be filtered by sequence(s):
…/event?sequence=1
…/event?sequence=1;3;7
Events may be filtered by date:
…/event?date0=1970-01-01
Events may be filtered by a date interval:
…/event?date0=1970-01-01&date1=1980-01-01
Events may also be paginated.
Events can be associated either with a timestamp *or* with a
sequence + shotpoint (but not both, for data integrity reasons).
Events and shotpoints can also have “labels” associated with them.
The difference between a comment and a label is that the former
is free text while the latter is predefined and has associated
properties (currently only for display, but could also have QC
related properties such as ensuring that there is only one “FSP”
label per sequence and so on).
The `events` view puts everything together into a coherent view.
Note that this view may produce multiple rows for the same
timestamp or shotpoint, for instance when the event has both
a text comment and one or more labels.
