Package index • peskas.zanzibar.data.pipeline

Cloud Storage

Functions that interact with cloud storage providers.

cloud_object_name(): Retrieve Full Name of Versioned Cloud Object

cloud_storage_authenticate(): Authenticate to a Cloud Storage Provider

download_cloud_file(): Download Object from Cloud Storage

get_preprocessed_surveys(): Download WCS Preprocessed Surveys

get_validated_surveys(): Download WCS Validated Surveys

upload_cloud_file(): Upload File to Cloud Storage

Workflow

These are arguably the most important functions in the package. Each of these functions executes a step in the data pipeline.

ingest_wcs_surveys(): Ingest WCS Catch Survey Data

preprocess_wcs_surveys(): Pre-process Zanzibar WCS Surveys

validate_wcs_surveys(): Validate WCS Surveys Data

Ingestion

Functions dedicated to data pulling and ingestion from servers

ingest_wcs_surveys(): Ingest WCS Catch Survey Data

retrieve_wcs_surveys(): Retrieve WCS Surveys from Kobotoolbox

Preprocessing

Functions dedicated to the preprocessing module and general data processing

preprocess_wcs_surveys(): Pre-process Zanzibar WCS Surveys

expand_taxa(): Expand Taxonomic Vectors into a Data Frame

Validation

Functions dedicated to the validation module

alert_outlier(): Generate an alert vector based on the univOutl::LocScaleB() function

validate_catch(): Validate Catch Data and Detect Outliers

validate_dates(): Validate Submission Dates in WCS Surveys

validate_length(): Validate Length Data and Detect Outliers

validate_market(): Validate Market Data and Detect Outliers in Price Per Kilogram

validate_wcs_surveys(): Validate WCS Surveys Data

Helper functions

Functions dedicated to data processing.

add_version(): Add timestamp and sha string to a file name

read_config(): Read configuration file

Export

Functions dedicated dissemination of processed and validated data and visualizations

kepler_mapper(): Generate a Kepler.gl map