Package index • peskas.mozambique.data.pipeline

Workflow

These are arguably the most important functions in the package. Each of these functions executes a step in the data pipeline.

export_landings(): Export Processed Landings Data

ingest_landings(): Download and Process Surveys from Kobotoolbox

preprocess_landings(): Preprocess Landings Data

Cloud Storage

Functions that interact with cloud storage providers.

get_metadata(): Get metadata tables

mdb_collection_pull(): Retrieve Data from MongoDB

mdb_collection_push(): Upload Data to MongoDB and Overwrite Existing Content

Ingestion

Functions dedicated to the ingestion module

get_kobo_data(): Retrieve Data from Kobotoolbox API

ingest_landings(): Download and Process Surveys from Kobotoolbox

Preprocessing

Functions dedicated to the preprocessing module

calculate_catch(): Calculate Catch Weights Using Length-Weight Relationships

preprocess_landings(): Preprocess Landings Data

process_species_group(): Process Species Length and Catch Data

Validation

Functions dedicated to the validation module

Export

Functions dedicated dissemination of processed and analysed fisheries data

export_landings(): Export Processed Landings Data

Helper functions

Functions dedicated to data processing.

add_version(): Add timestamp and sha string to a file name

read_config(): Read configuration file