# # # Distribution Statement A. Approved for public release. Distribution unlimited.
# # #
# # # Author:
# # # Naval Research Laboratory, Marine Meteorology Division
# # #
# # # This program is free software: you can redistribute it and/or modify it under
# # # the terms of the NRLMMD License included with this program. This program is
# # # distributed WITHOUT ANY WARRANTY; without even the implied warranty of
# # # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the included license
# # # for more details. If you did not receive the license, for more information see:
# # # U-S-NRL-Marine-Meteorology-Division/

Version 1.6.1 (2023-01-04)#

This release included formatting updates to conform to GeoIPS style guides. It also includes a script to test ALL available functionality on github.com, as well as minor bug fixes.

Security Updates#

Resolve bandit suggestions#

From issue GEOIPS#126: 2022-12-29, update source for black, bandit and numpy docstrings

  • Added “usedforsecurity=False” to sha1 call (unique geolocation files)

  • Replaced eval with ast.literal_eval

  • Replaced yaml.load with yaml.safe_load

modified: geoips/image_utils/colormap_utils.py
modified: geoips/interface_modules/readers/utils/geostationary_geolocation.py
modified: geoips/sector_utils/utils.py

Major New Functionality#

Add script for checking code for style compliance#

From issue GEOIPS#126: 2022-12-29, update source for black, bandit and numpy docstrings

  • Includes calls to the following packages:

black
bandit -ll -r
flake8 (with flake8-rst-docstrings plugin)

new file: tests/utils/check_code.sh

Basic filename module#

From issue GEOIPS#2: 2022-12-06, update interface families

  • Add very basic filename module - just put a straightforwardly named file in a specified directory

  • Only relying on required geoips xarray attributes, passed product name, and area_def.area_id if defined

geoips/interface_modules/filename_formats/basic_fname.py

Enhancements#

Separate groups of output and product families in procflow#

From issue GEOIPS#2: 2022-12-06, update interface families

  • Denote output families that require a specified list of output files in advance separately from those that do NOT require a complete list of files in advance.

  • Allows dynamically determining output filenames from within an output format.

  • Do not automatically generate metadata if output filenames not pre-specified

  • Add support for unsectored products that require area_def

geoips/interface_modules/procflows/single_source.py

Add filename family: xarray_area_product_to_filename#

From issue GEOIPS#2: 2022-12-06, update interface families

  • Only xarray, area_def, and product_name as args, and output_type and basedir as kwargs

  • Other filename types have a LOT of required kwargs

geoips/dev/filename.py

Add output family: xrdict_area_product_to_outlist#

From issue GEOIPS#2: 2022-12-06, update interface families

  • xarray_dict, area_def, product_name args

  • NOTE: output_fnames is NOT included in argument list, which means resulting file list will NOT be checked for consistency.

geoips/dev/output.py

Add product family: unsectored_xarray_dict_area_to_output_format#

From issue GEOIPS#2: 2022-12-06, update interface families

  • This indicates procflow must process this unsectored data type from within the area_def loop.

  • Ie, unsectored read, but final product may require area information

geoips/dev/product.py

Set zorder using “order” xarray attr#

From issue GEOIPS#2: 2022-12-06, update interface families

  • Allow using “order” attribute on xarray object to set zorder in plot

geoips/interface_modules/output_formats/imagery_clean.py

Allow AWS wildcards#

From issue GEOIPS#2: 2022-12-06, update interface families

  • Allow wildcards in NOAA AWS downloads

tests/download_noaa_aws.sh

Bug fixes#

Replace numpy aliases#

From issue GEOIPS#144: 2023-01-04, slight documentation updates

  • Replace numpy.float with float, numpy.bool with bool

  • GEOIPS#145 - numpy no longer supports aliases to builtin types

geoips/image_utils/mpl_utils.py

Remove numpy aliases to builtin types#

From issue GEOIPS#145: 2023-01-03, remove numpy aliases to builtin types

  • Remove numpy aliases to builtin types

    • Update AHI HSD reader to replace numpy.bool with bool

    • numpy v1.20.0 no longer allows using numpy aliases to builtin types.

geoips/interface_modules/readers/ahi_hsd.py

Fix file duplicate removal syntax#

From issue GEOIPS#2: 2022-12-06, update interface families

  • Fix indentation for if/else duplicate removal SKIP

geoips/filenames/duplicate_files.py

Fix check_continue syntax#

From issue GEOIPS#2: 2022-12-06, update interface families

  • Add +z to variable checks

  • Failed with different invocations of calling the script

setup/bash_setup/check_continue

Update YAML metadata storm_start_datetime#

From issue GEOIPS#2: 2022-12-06, update interface families

tests/outputs/amsr2_ocean.tc.windspeed.imagery_clean/20200518_073601_IO012020_amsr2_gcom-w1_windspeed_140kts_85p45_1p0-clean.png.yaml

Code Formatting and Style Updates#

Apply numpy docstring and black code style#

From issue GEOIPS#126: 2022-12-29, update source for black, bandit and numpy docstrings

  • Update all docstrings to numpy formatting

  • Apply black with default options to all Python code

  • Begin applying flake8 updates

Testing Updates#

Standardize geoips packages variables#

From issue GEOIPS#153: 2023-01-04, standardize geoips packages variables

  • Replace all instances of $GEOIPS with $GEOIPS_PACKAGES_DIR/geoips

    • $GEOIPS is an optional environment variable.

Standardize geoips test data variables#

From issue GEOIPS#149: 2023-01-04, standardize testing variables

  • Replace GEOIPS BASEDIR/test_data with GEOIPS_TESTDATA_DIR

  • Replacing all instances of GEOIPS BASEDIR/test_data with GEOIPS_TESTDATA_DIR

  • Remove tests/README.md

  • REMOVE tests/README.md - deprecated, info now included in template repos.

Add capability to obtain and test all available functionality#

From issue GEOIPS#147: 2023-01-03, add capability to test all avaialble functionality

  • Add test_full_install.sh

    • Add script to obtain and test all test repos.

    • Add “setup_test_repos” to setup.sh, which clones, updates, and uncompresses test repo.

setup.sh
tests/test_full_install.sh

Use original AMSR2 filenames#

From issue GEOIPS#11: 2022-12-12, original AMSR2 filenames

  • Renamed AMSR2 test datasets to use original filenames, for reference.

  • Update test scripts/outputs accordingly

  • Source filename in call and YAML metadata output

modified: tests/scripts/amsr2.config_based_overlay_output_low_memory.sh
modified: tests/scripts/amsr2.config_based_overlay_output.sh
modified: tests/scripts/amsr2.tc.89H-Physical.imagery_annotated.sh
modified: tests/outputs/amsr2.tc_overlay.37pct.imagery_annotated_over_Infrared-Gray/20200518_073601_IO012020_amsr2_gcom-w1_37pct_140kts_95p89_res1p0-cr100-bgInfrared-Gray.png.yaml
modified: tests/outputs/amsr2.tc_overlay.89pct.imagery_annotated_over_Infrared-Gray/20200518_073601_IO012020_amsr2_gcom-w1_89pct_140kts_98p32_res1p0-cr100-bgInfrared-Gray.png.yaml
modified: tests/outputs/amsr2.tc_overlay.37pct.imagery_annotated_over_Visible/20200518_073601_IO012020_amsr2_gcom-w1_37pct_140kts_95p89_res1p0-cr100-bgVisible.png.yaml
modified: tests/outputs/amsr2.tc_overlay.89pct.imagery_annotated_over_Visible/20200518_073601_IO012020_amsr2_gcom-w1_89pct_140kts_98p32_res1p0-cr100-bgVisible.png.yaml
modified: tests/outputs/amsr2.tc.89H-Physical.imagery_annotated/20200518_073601_IO012020_amsr2_gcom-w1_89H-Physical_140kts_100p00_res1p0-cr300.png.yaml

Documentation Updates#

Simplify installation.rst#

From issue GEOIPS#144: 2023-01-04, slight documentation updates

  • Slightly rearrange installation.rst - only include a single “Complete conda-based installation” section to avoid confusion.