It may happen that the sailline info file has data for more lines
than are actually in the preplot (e.g., if importing a reduced
preplot file). In this case, we don't want a constraint violation
error due to missing corresponding lines in `preplot_lines` so we
check for that and only import lines that do exist in `preplot_lines`
As the survey definitions will no longer be stored in files
under etc/surveys/ but directly on the database, this
function replaces configuration.surveys()
The script bin/daily_tasks.py is intended to be run shortly after
midnight every day (e.g., via cron).
At the moment it inserts any missing LDSP / FDSP events. It can
be extended with other tasks as needed either by expanding
Datastore.run_daily_tasks() or by adding to bin/daily_tasks.py.
Fixes#223.
The INSERT INTO raw_lines / final_lines will not always be executed as
the lines may already exist (particularly in raw_lines because of
*online*), so whether it worked or not we merge the metadata immediately
afterwards (this may cause an extra notification to be fired).
The reason why need to do it like this instead of relying on a trigger
is because the entry in final_lines is created first and the final_shots
are populated. If we first the trigger on final_lines it is not going
to find any shots; if we fire it as a row trigger on final_shots it
would try to label every point in sequence as it is imported; finally if
we fire it as a statement trigger on final_shots we have no idea which
sequence was imported.
If a final sequence file or directory name matches a pattern
which is recognised to indicate a ‘pending acceptance’ status,
the final data (if any exists) for that sequence will be deleted
and a comment added to the effect that the sequence has been
marked as ‘pending’.
To accept the sequence, rename its final file or directory name
accordingly.
Note: it is the *final* data that is searched for a matching
pattern, not the raw.
Closes#91.
Even if it hasn't, the file gets imported anyway
(into the `files` table) but we exit early to avoid
an error when trying to determine shooting direction.
This check is not necessary for final P1/11 or gun data.
Fixes#104.
Doing otherwise will result in the gun data file
appearing has having been read, but no data will
have been saved as there was nowhere to save to.
Fixes#29.
Each of the save_* operations starts a transaction
(which is automatically commited if all goes well).
The main reason for this is to ensure that by the
time raw_lines and final_lines events fire, the
corresponding entries in raw_shots and final_shots
have already been populated.
Provided that the SmartSource headers are being
saved to file, and that the path to those files
is present in the survey configuration, we now
import SmartSource information as metadata in
raw_shots.meta->'smsrc'.
Closes#19.
It will not delete any labels that have been removed
from the configuration, as those may be used, but it
will add new labels and modify existing ones if they
changed.
