| # # # Distribution Statement A. Approved for public release. Distribution unlimited. | # # # | # # # Author: | # # # Naval Research Laboratory, Marine Meteorology Division | # # # | # # # This program is free software: you can redistribute it and/or modify it under | # # # the terms of the NRLMMD License included with this program. This program is | # # # distributed WITHOUT ANY WARRANTY; without even the implied warranty of | # # # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the included license | # # # for more details. If you did not receive the license, for more information see: | # # # https://github.com/U-S-NRL-Marine-Meteorology-Division/ Version 1.6.1 (2023-01-04) ************************** This release included formatting updates to conform to GeoIPS style guides. It also includes a script to test ALL available functionality on github.com, as well as minor bug fixes. Security Updates ================ Resolve bandit suggestions -------------------------- *From issue GEOIPS#126: 2022-12-29, update source for black, bandit and numpy docstrings* * Added “usedforsecurity=False” to sha1 call (unique geolocation files) * Replaced eval with ast.literal_eval * Replaced yaml.load with yaml.safe_load :: modified: geoips/image_utils/colormap_utils.py modified: geoips/interface_modules/readers/utils/geostationary_geolocation.py modified: geoips/sector_utils/utils.py Major New Functionality ======================= Add script for checking code for style compliance ------------------------------------------------- *From issue GEOIPS#126: 2022-12-29, update source for black, bandit and numpy docstrings* * Includes calls to the following packages: :: black bandit -ll -r flake8 (with flake8-rst-docstrings plugin) :: new file: tests/utils/check_code.sh Basic filename module --------------------- *From issue GEOIPS#2: 2022-12-06, update interface families* * Add very basic filename module - just put a straightforwardly named file in a specified directory * Only relying on required geoips xarray attributes, passed product name, and area_def.area_id if defined :: geoips/interface_modules/filename_formats/basic_fname.py Enhancements ============ Separate groups of output and product families in procflow ---------------------------------------------------------- *From issue GEOIPS#2: 2022-12-06, update interface families* * Denote output families that require a specified list of output files in advance separately from those that do NOT require a complete list of files in advance. * Allows dynamically determining output filenames from within an output format. * Do not automatically generate metadata if output filenames not pre-specified * Add support for unsectored products that require area_def :: geoips/interface_modules/procflows/single_source.py Add filename family: xarray_area_product_to_filename ---------------------------------------------------- *From issue GEOIPS#2: 2022-12-06, update interface families* * Only xarray, area_def, and product_name as args, and output_type and basedir as kwargs * Other filename types have a LOT of required kwargs :: geoips/dev/filename.py Add output family: xrdict_area_product_to_outlist ------------------------------------------------- *From issue GEOIPS#2: 2022-12-06, update interface families* * xarray_dict, area_def, product_name args * NOTE: output_fnames is NOT included in argument list, which means resulting file list will NOT be checked for consistency. :: geoips/dev/output.py Add product family: unsectored_xarray_dict_area_to_output_format ---------------------------------------------------------------- *From issue GEOIPS#2: 2022-12-06, update interface families* * This indicates procflow must process this unsectored data type from within the area_def loop. * Ie, unsectored read, but final product may require area information :: geoips/dev/product.py Set zorder using "order" xarray attr ------------------------------------ *From issue GEOIPS#2: 2022-12-06, update interface families* * Allow using “order” attribute on xarray object to set zorder in plot :: geoips/interface_modules/output_formats/imagery_clean.py Allow AWS wildcards ------------------- *From issue GEOIPS#2: 2022-12-06, update interface families* * Allow wildcards in NOAA AWS downloads :: tests/download_noaa_aws.sh Bug fixes ========= Replace numpy aliases --------------------- *From issue GEOIPS#144: 2023-01-04, slight documentation updates* * Replace numpy.float with float, numpy.bool with bool * GEOIPS#145 - numpy no longer supports aliases to builtin types :: geoips/image_utils/mpl_utils.py Remove numpy aliases to builtin types ------------------------------------- *From issue GEOIPS#145: 2023-01-03, remove numpy aliases to builtin types* * Remove numpy aliases to builtin types * Update AHI HSD reader to replace numpy.bool with bool * numpy v1.20.0 no longer allows using numpy aliases to builtin types. :: geoips/interface_modules/readers/ahi_hsd.py Fix file duplicate removal syntax --------------------------------- *From issue GEOIPS#2: 2022-12-06, update interface families* * Fix indentation for if/else duplicate removal SKIP :: geoips/filenames/duplicate_files.py Fix check_continue syntax ------------------------- *From issue GEOIPS#2: 2022-12-06, update interface families* * Add +z to variable checks * Failed with different invocations of calling the script :: setup/bash_setup/check_continue Update YAML metadata storm_start_datetime ----------------------------------------- *From issue GEOIPS#2: 2022-12-06, update interface families* :: tests/outputs/amsr2_ocean.tc.windspeed.imagery_clean/20200518_073601_IO012020_amsr2_gcom-w1_windspeed_140kts_85p45_1p0-clean.png.yaml Code Formatting and Style Updates ================================= Apply numpy docstring and black code style ------------------------------------------ *From issue GEOIPS#126: 2022-12-29, update source for black, bandit and numpy docstrings* * Update all docstrings to numpy formatting * Apply black with default options to all Python code * Begin applying flake8 updates Testing Updates =============== Standardize geoips packages variables ------------------------------------- *From issue GEOIPS#153: 2023-01-04, standardize geoips packages variables* * Replace all instances of $GEOIPS with $GEOIPS_PACKAGES_DIR/geoips * $GEOIPS is an optional environment variable. Standardize geoips test data variables -------------------------------------- *From issue GEOIPS#149: 2023-01-04, standardize testing variables* * Replace GEOIPS BASEDIR/test_data with GEOIPS_TESTDATA_DIR * Replacing all instances of GEOIPS BASEDIR/test_data with GEOIPS_TESTDATA_DIR * Remove tests/README.md * REMOVE tests/README.md - deprecated, info now included in template repos. Add capability to obtain and test all available functionality ------------------------------------------------------------- *From issue GEOIPS#147: 2023-01-03, add capability to test all avaialble functionality* * Add test_full_install.sh * Add script to obtain and test all test repos. * Add “setup_test_repos” to setup.sh, which clones, updates, and uncompresses test repo. :: setup.sh tests/test_full_install.sh Use original AMSR2 filenames ---------------------------- *From issue GEOIPS#11: 2022-12-12, original AMSR2 filenames* * Renamed AMSR2 test datasets to use original filenames, for reference. * Update test scripts/outputs accordingly * Source filename in call and YAML metadata output :: modified: tests/scripts/amsr2.config_based_overlay_output_low_memory.sh modified: tests/scripts/amsr2.config_based_overlay_output.sh modified: tests/scripts/amsr2.tc.89H-Physical.imagery_annotated.sh modified: tests/outputs/amsr2.tc_overlay.37pct.imagery_annotated_over_Infrared-Gray/20200518_073601_IO012020_amsr2_gcom-w1_37pct_140kts_95p89_res1p0-cr100-bgInfrared-Gray.png.yaml modified: tests/outputs/amsr2.tc_overlay.89pct.imagery_annotated_over_Infrared-Gray/20200518_073601_IO012020_amsr2_gcom-w1_89pct_140kts_98p32_res1p0-cr100-bgInfrared-Gray.png.yaml modified: tests/outputs/amsr2.tc_overlay.37pct.imagery_annotated_over_Visible/20200518_073601_IO012020_amsr2_gcom-w1_37pct_140kts_95p89_res1p0-cr100-bgVisible.png.yaml modified: tests/outputs/amsr2.tc_overlay.89pct.imagery_annotated_over_Visible/20200518_073601_IO012020_amsr2_gcom-w1_89pct_140kts_98p32_res1p0-cr100-bgVisible.png.yaml modified: tests/outputs/amsr2.tc.89H-Physical.imagery_annotated/20200518_073601_IO012020_amsr2_gcom-w1_89H-Physical_140kts_100p00_res1p0-cr300.png.yaml Documentation Updates ===================== Use relative links ------------------ *From issue GEOIPS#144: 2023-01-04, slight documentation updates* * Replace full links with relative - will work on different branches Simplify installation.rst ------------------------- *From issue GEOIPS#144: 2023-01-04, slight documentation updates* * Slightly rearrange installation.rst - only include a single “Complete conda-based installation” section to avoid confusion.