Ingest Pelagic Data Systems (PDS) Track Data — ingest_pds

Downloads and stores only new tracks not previously uploaded to cloud storage. Uses parallel processing for improved performance.

Usage

ingest_pds_tracks(
  log_threshold = logger::DEBUG,
  batch_size = NULL,
  package = "coasts"
)

Arguments

log_threshold: The logging threshold to use. Default is logger::DEBUG.
batch_size: Optional number of tracks to process. If NULL, processes all new tracks.
package: Name of the package whose inst/conf.yml to read. Defaults to "coasts".

Value

None (invisible).

Details

Track objects are written as <prefix>_<trip_id>.parquet and are not versioned — they are the one exception to the <prefix>__<timestamp>_<sha>__.<ext> convention every other Peskas artefact follows. Re-ingesting a trip overwrites it in place and leaves no history, which is defensible for a finished GPS trip but has two consequences worth knowing: cloud_object_name() with version = "latest" cannot be used on the tracks bucket at all, and which tracks already exist is determined entirely by object name — see extract_trip_ids_from_filenames().