
Preprocess Landings Data (Version 1)
preprocess_landings_v1.Rd
This function preprocesses raw landings data from Google Cloud Storage. It performs various data cleaning and transformation operations, including column renaming, data pivoting, and standardization of catch names.
Usage
preprocess_landings_v1(log_threshold = logger::DEBUG)
Value
No return value. Function processes the data and uploads the result as a Parquet file to Google Cloud Storage.
Details
The function performs the following main operations:
Downloads raw data from Google Cloud Storage
Renames columns and selects relevant fields
Generates unique survey IDs
Cleans and standardizes text fields
Pivots catch data from wide to long format
Standardizes catch names and separates size information
Converts data types and handles cases with no catch data
Uploads the processed data as a Parquet file to Google Cloud Storage