Package index • peskas.malawi.data.pipeline

Workflow

These are arguably the most important functions in the package. Each of these functions executes a step in the data pipeline.

export_landings(): Export Selected Landings Data

export_matched_tracks(): Export Matched Vessel Tracks with Landing Data

ingest_landings(): Ingest Survey landings from Kobotoolbox into MongoDB

merge_trips(): Merge Fishing Trips Data with PDS records

preprocess_landings(): Preprocess Landings Data

validate_landings(): Validate Fisheries Data

Cloud Storage

Functions that interact with cloud storage providers.

get_metadata(): Get metadata tables

mdb_collection_pull(): Retrieve Data from MongoDB

mdb_collection_push(): Upload Data to MongoDB and Overwrite Existing Content

Ingestion

Functions dedicated to the ingestion module

flatten_with_names(): Recursively Flatten a Nested List with Concatenated Names

get_trip_points(): Get Trip Points from Pelagic Data Systems API

get_trips(): Retrieve Trip Details from Pelagic Data API

ingest_landings(): Ingest Survey landings from Kobotoolbox into MongoDB

merge_trips(): Merge Fishing Trips Data with PDS records

process_asset(): Process Data for a Single Asset ID

process_survey(): Process Survey Data into a Tidy Tibble

Preprocessing

Functions dedicated to the preprocessing module

preprocess_landings(): Preprocess Landings Data

standardize_list_column(): Standardize a List-Column in a Data Frame

Validation

Functions dedicated to the validation module

alert_outlier(): Generate an Alert Vector Based on Outlier Detection

get_bounds(): Get Bounds for Outlier Detection

get_catch_bounds(): Get Catch Bounds

get_pricekg_bounds(): Get Catch Price Bounds

validate_catch(): Validate Catch Data

validate_dates(): Validate Landing Dates

validate_landings(): Validate Fisheries Data

validate_nboats(): Validate Number of Boats

validate_nfishers(): Validate Number of Fishers

validate_pricekg(): Validate Catch Price per Kilogram

validate_this_imei(): Validate IMEI Number

Export

Functions dedicated dissemination of processed and analysed fisheries data

export_landings(): Export Selected Landings Data

export_matched_tracks(): Export Matched Vessel Tracks with Landing Data

Helper functions

Functions dedicated to data processing.

add_version(): Add timestamp and sha string to a file name

read_config(): Read configuration file