Package index

Workflow

These are arguably the most important functions in the package. Each of these functions executes a step in the data pipeline.

calculate_weights(): Add weight of species to merged landings

estimate_fishery_indicators(): Estimate fisheries indicators

format_public_data(): Format public data

ingest_landings(): Ingest Landings Survey data

ingest_legacy_landings(): Ingest legacy Landings Survey data

ingest_metadata_tables(): Ingest metadata tables

ingest_pds_tracks(): Ingest Pelagic Data System tracks data

ingest_pds_trips(): Ingest Pelagic Data System trips data

ingest_updated_landings(): Ingest Landings Survey data (Peskas 2)

ingest_validation_tables(): Ingest flags tables

merge_landings(): Merge recent and legacy pre-processed landings

merge_trips(): Merge trips

model_indicators(): Model fisheries indicators

preprocess_landings_step_1(): Pre-process Timor Landings Survey data (step 1)

preprocess_landings_step_2(): Pre-process Timor Landings Survey data (step 2)

preprocess_legacy_landings(): Preprocess Timor legacy Landings Survey data

preprocess_metadata_tables(): Preprocess metadata tables

preprocess_pds_tracks(): Pre-process Pelagic Data System tracks

preprocess_pds_trips(): Pre-process Pelagic Data System trips

preprocess_updated_landings(): Pre-process Timor Updated Landings Survey

validate_landings(): Validate landings

Cloud

Cloud storage

Functions that interact with cloud storage providers.

cloud_object_name(): Get the full name of a versioned cloud object

cloud_storage_authenticate(): Authenticate to a storage cloud provider

download_cloud_file(): Download an object from a cloud storage bucket to a local file

insistent_download_cloud_file(): Insistent version of download_cloud_file()

insistent_upload_cloud_file(): Insistent version of upload_cloud_file()

upload_cloud_file(): Upload a local file to a cloud storage bucket

air_get_records(): Get records from an Airtable table

get_catch_types(): Download metadata catch types

get_distance(): Estimate distance between consecutive trips

get_fao_composition(): Get FAO Food Composition Data

get_fish_length(): Download species body-length parameters

get_full_tracks(): Download tracks data in a single file

get_full_trips(): Get trips from single-file tracks data.

get_host_url(): Select server from kobo

get_merged_landings(): Download merged landings

get_merged_trips(): Download Peskas surveys and PDS data

get_models(): Download models estimates

get_nutrients_table(): Get taxa nutritional values

get_preprocessed_metadata(): Download Peskas metadata

get_preprocessed_sheets(): Download Peskas metadata

get_sync_tracks(): Download and synchronize tracks data in a single file

get_timor_boundaries(): Extract Timor Geographical Boundaries

get_tracks_descriptors(): Extract summaries from pds tracks

get_tracks_ids(): Get pds IDs

get_tracks_map(): Get map of Timor pds tracks.

get_validated_landings(): Download Peskas validated landings

get_validation_sheet(): Get peskas validation sheet

ingest_complete_tracks(): Ingest tracks data as a single file

ingest_kepler_tracks(): Ingest a Kepler.gl map

ingest_landings(): Ingest Landings Survey data

ingest_legacy_landings(): Ingest legacy Landings Survey data

ingest_metadata_tables(): Ingest metadata tables

ingest_pds_map(): Generate and ingest Timor maps

ingest_pds_matched_trips(): Ingest PDS matched trip

ingest_pds_tracks(): Ingest Pelagic Data System tracks data

ingest_pds_trips(): Ingest Pelagic Data System trips data

ingest_rfish_table(): Ingest species weight-length info

ingest_updated_landings(): Ingest Landings Survey data (Peskas 2)

ingest_validation_tables(): Ingest flags tables

add_version(): Add timestamp and sha string to a file name

Google Spreadsheets

Functions to interact with Google Drive and Spreadsheets

authenticate_google_drive(): Authenticate to Google Drive

Airtable

Functions to interact with Airtable

air_get_records(): Get records from an Airtable table

air_records_to_tibble(): Format Airtable records to a data frame (tibble)

air_tibble_to_records(): Format a data frame (tibble) as Airtable records

air_upload_records(): Upload and create Airtable records to a base

Data

Survey

Functions that help retrieving survey data

retrieve_survey(): Download survey

retrieve_survey_data(): Download kobo data in as json using the v1 API

retrieve_survey_metadata(): Download survey metadata

validate_surveys_time(): Validate surveys' temporal parameters

get_host_url(): Select server from kobo

PDS

Functions that help retrieving Pelagic Data System data

ingest_pds_map(): Generate and ingest Timor maps

ingest_pds_matched_trips(): Ingest PDS matched trip

ingest_pds_tracks(): Ingest Pelagic Data System tracks data

ingest_pds_trips(): Ingest Pelagic Data System trips data

preprocess_pds_tracks(): Pre-process Pelagic Data System tracks

preprocess_pds_trips(): Pre-process Pelagic Data System trips

retrieve_pds_tracks(): Download pelagic data system tracks

retrieve_pds_tracks_data(): Download pelagic data system tracks in as csv

retrieve_pds_trips(): Download pelagic data system trips

retrieve_pds_trips_data(): Download pelagic data system trips in as csv or json

validate_pds_data(): Validate pds trips duration and distance

validate_pds_trips(): Validate Pelagic Data System trips

kepler_mapper(): Generate a Kepler.gl map

Data modeling

Functions dedicated to data modeling.

get_models(): Download models estimates

model_indicators(): Model fisheries indicators

model_landings(): Model number of landings per unit

model_value(): Model landing value

Data validation

Functions dedicated to data validation

get_validated_landings(): Download Peskas validated landings

pt_validate_boats(): Parse and validate boats table

pt_validate_catch_types(): Parse and validate catch types table

pt_validate_centro_pescas(): Parse and validate centro de pescas table

pt_validate_devices(): Parse and validate devices table

pt_validate_fao_catch(): Parse and validate fao catch table

pt_validate_flags(): Validate flags from metadata tables

pt_validate_morphometric_table(): Parse and validate morphometric table

pt_validate_vms_installs(): Parse and validate vms_installs table

validate_catch_params(): Validate surveys' catch parameters

validate_catch_price(): Validate surveys' total catch values

validate_landings(): Validate landings

validate_pds_data(): Validate pds trips duration and distance

validate_pds_trips(): Validate Pelagic Data System trips

validate_price_weight(): Outlier identification based on Cook's distance

validate_surveys_time(): Validate surveys' temporal parameters

alert_outlier(): Generate an alert vector based on the univOutl::LocScaleB() function

send_validation_mail(): Send validation summary email

Helper functions

Data preprocessing

Functions dedicated to data processing.

pt_nest_attachments(): Nest attachment columns

pt_nest_species(): Nest species columns

pt_validate_boats(): Parse and validate boats table

pt_validate_catch_types(): Parse and validate catch types table

pt_validate_centro_pescas(): Parse and validate centro de pescas table

pt_validate_devices(): Parse and validate devices table

pt_validate_fao_catch(): Parse and validate fao catch table

pt_validate_flags(): Validate flags from metadata tables

pt_validate_morphometric_table(): Parse and validate morphometric table

pt_validate_vms_installs(): Parse and validate vms_installs table

clean_catches(): Clean SFF landings' catches table

clean_legacy_landings(): Clean SFF landings data

clean_updated_landings(): Clean Updated (Peskas 2) landings data

coalist(): Coalesce vectors

merge_consecutive_trips(): Merge short intervalled trips

join_weights(): Join length-weights and nutritional parameters info to preprocessed landings

retrieve_lengths(): Retrieve weight-length parameters

fill_missing_regions(): Fill missing regions

Taxonomy

convert_taxa_names(): Convert taxa codes to common names

Utils

read_config(): Read configuration file

Publishing

Functions deemed to share data and reports.

delete_dataverse(): Delete a dataverse collection

publish_dataverse(): Publish a Dataverse repository

upload_dataverse(): Upload and publish a dataset on Dataverse

delete_dataset(): Delete a dataset of a dataverse collection

publish_last_dataset(): Publish latest dataset created

upload_files(): Upload files to Dataverse

generate_metadata(): Generate a list of metadata

generate_description(): Generate data description

send_sites_report(): Send sites report email

Utils

read_config(): Read configuration file

Workflow

Cloud

Cloud storage

Google Spreadsheets

Airtable

Data

Survey

PDS

Data modeling

Data validation

Helper functions

Data preprocessing

Taxonomy

Utils

Data sharing

Publishing

Utils