Export Summary Data to MongoDB

Downloads previously summarized WorldFish survey data from cloud storage, incorporates modeled aggregated estimates, and exports everything to MongoDB collections for use in data portals. The function also generates geographic regional summaries.

Usage

export_portal(log_threshold = logger::DEBUG, package = "coasts")

Arguments

log_threshold

The logging level threshold for the logger package (e.g., DEBUG, INFO) See logger::log_levels for available options.

package

Name of the package whose inst/conf.yml to read. Defaults to "coasts". Pass your own package name when calling from a downstream package with a compatible configuration.

Value

NULL (invisible). The function uploads data to MongoDB as a side effect.

Details

The function performs the following operations:

• Downloads nine summary datasets from cloud storage:

  • Monthly summaries: Aggregated catch metrics by district and month

  • All monthly summaries: as above but without the dashboard exclusions

  • Taxa summaries: Species-specific metrics in long format

  • Districts summaries: District-level indicators over time

  • Gear summaries: Performance metrics by gear type

  • Grid summaries: Spatial grid data from vessel tracking

  • Taxa traits: FishBase traits per landed taxon

  • Length summaries: recorded catch per length class

  • Gear taxa summaries: recorded catch per gear and species

• Downloads aggregated catch estimates from the modeling step

• Creates geographic regional summaries from the unfiltered monthly data (all_monthly_summaries), so the coasts portal keeps every survey form while the MongoDB dashboard collections keep the filtered ones

• Joins aggregated estimates (fishing trips, catch tonnage, revenue) and the share of the fleet tracked behind them (sampling_rate) to the monthly and district summaries

• Transforms monthly summaries to long format for portal consumption

• Adds the FAO ARTFISH catch by species to the taxa summaries, as the metric estimated_catch_tn_fao, for a country that has it (fao_portal_taxa())

• Uploads all datasets to specified MongoDB collections

• Writes the district summaries with both methods' estimates and their GAUL codes to the API bucket as <country>/aggregated/district-months, served by peskas-api's /data/district-months (api_district_months())

The function expects the summary files to be named with the pattern: {file_prefix}_{table_name}.parquet where table_name is one of: monthly_summaries, all_monthly_summaries, taxa_summaries, districts_summaries, gear_summaries, grid_summaries, taxa_traits, length_summaries, gear_taxa_summaries

See also

• summarize_data() for generating the summary datasets

• download_parquet_from_cloud() for retrieving data from cloud storage

• mdb_collection_push() for uploading data to MongoDB

Examples

if (FALSE) { # \dontrun{
# Export WF summary data with default debug logging
export_portal()

# Export with info-level logging only
export_portal(logger::INFO)
} # }