Pre-process Pelagic Data System tracks — preprocess_pds

Downloads raw structured data from cloud storage services and pre-process into a binary format that is easier to deal with in R.

Usage

preprocess_pds_tracks(log_threshold = logger::DEBUG)

Arguments

log_threshold: The (standard Apache logj4) log level used as a threshold for the logging infrastructure. See logger::log_levels for more details

Value

no outputs. This function is used for it's side effects

Details

The preprocessed file consists of a dataframe containing PDS tracks' diagnostics useful to the following validation step.

The parameters needed are:

pds:
 tracks:
  token: !expr Sys.getenv('PDS_TOKEN')
  secret: !expr Sys.getenv('PDS_SECRET')
  file_prefix:
pds_storage:
  google:
   options:
    bucket: pds-timor

Progress through the function is tracked using the package logger.