Version latest (2025-07-31)#
Documentation: Enhanced documentation for output formatters and checkers
Documentation: User instructions for installing geoips on linux with conda
Documentation: Copy-edited cli documentation
Documentation: Add pytest testing documentation
Documentation: Add linux help section to testing documentation
Documentation: Document file cache and geoips_cache_dir
Documentation: User instructions for installing geoips on windows with conda
Documentation: User instructions for installing geoips on windows with conda
Documentation: Add test sector documentation
Documentation: Update to non-conda installation documentation
Bug fix: Fix linting errors from flake8 and cspell
Bug fix: Geoips config install bug fix.
Bug fix: Restore non-editable installability.
Bug fix: Use filter=”tar” in tarfile extract commands
Bug fix: Pin python to <3.13.0
Bug fix: Fix error handling in yamlpluginvalidator.validate()
Bug fix: Fixing uninstantiated variables in abi_netcdf reader
Bug fix: Fix unclosed file handle warnings
Bug fix: Change np.fromstring to np.frombuffer
Continuous integration: Update input data and output imagery
Testing: Nextcloud test dataset updates.
Testing: Added test_data_amsub to nextcloud
Deprecation: Update deprecated area_def.x-y_size and area_def.name calls.
Deprecation: Fix ahi_hsd warnings
Enhancement: Reorganize testing documentation structure
Enhancement: Pydantic gridline annotator plugin models
Enhancement: Geoips run order based command
Enhancement: Write cached json files to geoips_cache_dir
Enhancement: Alphabetize command-line arguments in help output
Enhancement: Add support for neutral tests in yaml-based testing
Enhancement: Add –warnings option to the cli
Enhancement: Add support for invalid json in command-line kwargs. eg. –reader-kwargs
Enhancement: Update dockerfile
Removal: Remove outdated integration tests documentation
Installation: Finalize the conda create command for completeness
Installation: Add a troubleshooting page to installation section of docs
Hot fix: Add support for alternative data_fusion package name
Documentation#
Enhanced documentation for output formatters and checkers#
Added comprehensive documentation for output checkers and formatters, including detailed explanations of how they work, their importance in the testing process, and practical examples of their usage.
added: docs/source/concepts/architecture/interfaces/module_based/output-checkers/index.rst
added: docs/source/releases/latest/doc-expand-w-plugins.yaml
modified: docs/source/concepts/architecture/documentation/index.rst
modified: docs/source/concepts/architecture/documentation/where-to-put.rst
modified: docs/source/concepts/architecture/index.rst
modified: docs/source/concepts/architecture/plugin-registries.rst
modified: docs/source/concepts/architecture/structure-of-geoips/index.rst
modified: docs/source/concepts/architecture/tests/index.rst
modified: docs/source/concepts/functionality/command-line/index.rst
modified: docs/source/concepts/functionality/documentation-building.rst
modified: docs/source/concepts/functionality/index.rst
modified: docs/source/concepts/functionality/interfaces/module_based/algorithm.rst
modified: docs/source/concepts/functionality/interfaces/module_based/colormapper.rst
modified: docs/source/concepts/functionality/interfaces/module_based/filename_formats.rst
modified: docs/source/concepts/functionality/interfaces/module_based/interpolators.rst
modified: docs/source/concepts/functionality/interfaces/module_based/output_formats.rst
modified: docs/source/concepts/functionality/interfaces/module_based/procflows.rst
modified: docs/source/concepts/functionality/interfaces/module_based/readers.rst
modified: docs/source/concepts/functionality/interfaces/module_based/title_formats.rst
modified: docs/source/concepts/functionality/interfaces/yaml_based/dynamic_sectors.rst
modified: docs/source/concepts/functionality/interfaces/yaml_based/gridline_annotators.rst
modified: docs/source/concepts/functionality/interfaces/yaml_based/product_defaults.rst
modified: docs/source/concepts/functionality/interfaces/yaml_based/products.rst
modified: docs/source/concepts/functionality/interfaces/yaml_based/static_sectors.rst
modified: docs/source/tutorials/nrtp-examples/using-inotifywait-for-basic-near-real-time-processing.rst
Documentation#
User instructions for installing geoips on linux with conda#
Add installation instructions for external users to install GeoIPS on Linux with Conda
added: docs/source/contribute/dev-install.rst
added: docs/source/contribute/dev_linux_with_conda.rst
added: docs/source/getting-started/installing/user_linux_with_conda.rst
added: docs/source/releases/latest/1089-add-user-install-instructions-Linux.yaml
added: docs/source/releases/latest/update-user-install-docs.yaml
modified: .cspell.json
modified: docs/source/contribute/index.rst
modified: docs/source/getting-started/installing/expert.rst
modified: docs/source/getting-started/installing/index.rst
deleted: docs/source/getting-started/installing/linux_with_conda.rst
Documentation#
Copy-edited cli documentation#
Copy-edited CLI documentation for correct rst, clarity and brevity.
added: docs/source/userguide/command_line_autodoc.rst
deleted: docs/source/new-docs/concepts/functionality/interfaces/module_based/algorithm.rst
modified: docs/source/concepts/functionality/command-line/command_line_autodoc.rst
modified: docs/source/concepts/functionality/command-line/index.rst
moved: docs/source/concepts/functionality/data_units.rst
Documentation#
Add pytest testing documentation#
Created documentation covering pytest usage, markers, output management, and testing features for GeoIPS
added: docs/source/contribute/testing/running_tests.rst
Documentation#
Add linux help section to testing documentation#
Updated running_tests.rst with additional Linux-specific help information
modified: docs/source/contribute/testing/running_tests.rst
Documentation#
Document file cache and geoips_cache_dir#
Provide documentation describing GEOIPS_CACHE_DIR and the files that get cached.
added: docs/source/concepts/architecture/index.rst
added: docs/source/concepts/architecture/cached-files.rst
Documentation#
User instructions for installing geoips on windows with conda#
Add installation instructions for external users to install GeoIPS on Windows with Conda
added: docs/source/contribute/dev_mac_with_conda.rst
added: docs/source/getting-started/installing/linux_with_conda.rst
added: docs/source/getting-started/installing/troubleshooting.rst
added: docs/source/releases/latest/add-user-install-instructions-Mac.yaml
added: docs/source/releases/latest/finalize-conda-create-and-troubleshooting.yaml
modified: .cspell.json
modified: docs/source/contribute/index.rst
modified: docs/source/getting-started/installing/index.rst
modified: docs/source/getting-started/installing/mac_with_conda.rst
deleted: docs/source/getting-started/installing/user_linux_with_conda.rst
Documentation#
User instructions for installing geoips on windows with conda#
Add installation instructions for external users to install GeoIPS on Windows with Conda
added: docs/source/releases/latest/add-user-install-instructions-Windows.yaml
modified: docs/source/releases/latest/add-user-install-instructions-Windows.yaml
modified: docs/source/getting-started/installing/user_linux_with_conda.rst
Documentation#
Add test sector documentation#
This adds documentation for the up-and-coming small test sectors for integration tests.
added: docs/source/concepts/architecture/tests/test-sectors.rst
modified: docs/source/concepts/architecture/tests/index.rst
Documentation#
Update to non-conda installation documentation#
This updates the “expert” install docs to reflect the package in its current state. It splits the documentation into two files: one for devs and one for users.
added: docs/source/contribute/dev-install.rst
modified: docs/source/getting-started/installing/expert.rst
modified: docs/source/contribute/index.rst
modified: .cspell.json
Bug fix#
Fix linting errors from flake8 and cspell#
Fix linting errors from flake8 and cspell
modified: geoips/plugins/modules/readers/amsua_mhs_mirs.py
Bug fix#
Geoips config install bug fix.#
The command ‘geoips config install’ will cause a KeyError if the environment variable $GEOIPS_TESTDATA_DIR is not set. This PR updates this logic and raises an argparse error if this environment variable isn’t set and the –outdir variable was not provided.
Additionally, this PR added new logic to create the specified output directory if it doesn’t already exist.
modified: geoips/commandline/geoips_config.py
Bug fix#
Restore non-editable installability.#
Move test-data-urls file into GeoIPS repo to fix a breaking change to non-editable installs.
added: geoips/commandline/ancillary_info/test-data-urls.yaml
modified: geoips/commandline/ancillary_info/test_data.py
modified: setup/download_test_data.py
Bug fix#
Use filter=”tar” in tarfile extract commands#
Update all uses of tarfile to use the filter=”tar” option in extract commands. This prevents a number of different types of malicious tar files from being extracted. This will become default behavior in Python 3.14. This gets rid of a DeprecationWarning in lower versions of Python.
modified: geoips/commandline/geoips_config.py
modified: setup/download_test_data.py
Bug fix#
Pin python to <3.13.0#
Changes to datetime handling are breaking in python 3.13. This pins the python version below 3.13 for now.
modified: pyproject.toml
Bug fix#
Fix error handling in yamlpluginvalidator.validate()#
Handle the AttributeError that is raised when plugin doesn’t have a
get() method. Narrow the scope of the current error handling to minimize
side effects.
modified: geoips/interfaces/base.py
Bug fix#
Fixing uninstantiated variables in abi_netcdf reader#
Checking for cached data introduced an error in abi_netcdf reader. This fixes that error as well as a downstream error where a datetime was not timezone-aware
added: docs/source/releases/latest/1085-cached-data-check-is-causing-error-in-readers.yaml
modified: geoips/plugins/modules/readers/abi_netcdf.py
modified: geoips/plugins/modules/readers/ahi_hsd.py
Bug fix#
Fix unclosed file handle warnings#
Several file handles were not correctly closed. This closes them.
modified: geoips/create_plugin_registries.py
Bug fix#
Change np.fromstring to np.frombuffer#
np.fromstring was removed from numpy in v2.3.0. The correct command is now np.frombuffer. This PR just makes that update.
modified: geoips/plugins/modules/readers/ahi_hsd.py
modified: geoips/plugins/modules/readers/ssmis_binary.py
Continuous integration#
Update input data and output imagery#
Update input data and output imagery for FCI and SEVIRI. Update output image for ami.tc.WV.geotiff.sh. Move AMSUB tests to limited_dataset_availability tests. Disable viirs.static.visible.imagery_clean.sh until issue 1076 is solved.
deleted:
tests/outputs/fci.static.Visible.imagery_annotated/20240113.115000.MTG-I1.fci.Visible.global_cylindrical.26p09.EUMETSAT.20p0.png
deleted:
tests/outputs/fci.unprojected_image.Infrared/20240113.115000.MTG-I1.fci.Infrared.self_register.71p89.EUMETSAT.50p0.png
deleted:
tests/outputs/seviri.Convective_Storms.imagery_clean/20200404.080000.msg-1.seviri.Convective_Storms.global_cylindrical.22p76.nesdisstar.20p0.png
deleted:
tests/outputs/seviri.Day_Microphys_Summer.imagery_clean/20200404.080000.msg-1.seviri.Day_Microphys_Summer.global_cylindrical.22p76.nesdisstar.20p0.png
deleted:
tests/outputs/seviri.Day_Microphys_Winter.imagery_clean/20200404.080000.msg-1.seviri.Day_Microphys_Winter.global_cylindrical.22p76.nesdisstar.20p0.png
deleted:
tests/outputs/seviri.Day_Solar.imagery_clean/20200404.080000.msg-1.seviri.Day_Solar.global_cylindrical.22p76.nesdisstar.20p0.png
deleted:
tests/outputs/seviri.Dust-RGB.imagery_clean/20200404.080000.msg-1.seviri.Dust-RGB.global_cylindrical.22p76.nesdisstar.20p0.png
deleted:
tests/outputs/seviri.Dust-RGB.imagery_clean/20200404.080000.msg-1.seviri.Dust_RGB.global_cylindrical.22p76.nesdisstar.20p0.png
deleted:
tests/outputs/seviri.Natural-Color.imagery_clean/20200404.080000.msg-1.seviri.Natural-Color.global_cylindrical.22p76.nesdisstar.20p0.png
deleted:
tests/outputs/seviri.Natural-Color.imagery_clean/20200404.080000.msg-1.seviri.Natural_Color.global_cylindrical.22p76.nesdisstar.20p0.png
deleted:
tests/outputs/seviri.Night_Microphys.imagery_clean/20240327.010000.msg-3.seviri.Night_Microphys.global_cylindrical.22p76.nesdisstar.20p0.png
deleted:
tests/outputs/seviri.Volcanic_Ash.imagery_clean/20200404.080000.msg-1.seviri.Volcanic_Ash.global_cylindrical.22p76.nesdisstar.20p0.png
deleted:
tests/outputs/seviri.WV-Upper.no_self_register.unprojected_image/20231211.080000.msg-2.seviri.WV-Upper-No-SR.unk.nesdisstar..png
deleted:
tests/outputs/seviri.WV-Upper.unprojected_image/20200404.080000.msg-1.seviri.WV-Upper.self_register.69p07.nesdisstar.10p0.png
deleted:
tests/outputs/seviri.WV-Upper.unprojected_image/20231211.080000.msg-2.seviri.WV-Upper.self_register.69p07.nesdisstar.10p0.png
deleted:
tests/outputs/seviri.airmass.imagery_clean/20200404.080000.msg-1.seviri.airmass.global_cylindrical.22p76.nesdisstar.20p0.png
deleted: tests/scripts/seviri.Dust-RGB.imagery_clean.sh
deleted: tests/scripts/seviri.Natural-Color.imagery_clean.sh
added:
tests/outputs/fci.static.Visible.imagery_annotated/20250623.120000.MTG-I1.fci.Visible.global_cylindrical.25p71.EUMETSAT.20p0.png
added:
tests/outputs/fci.unprojected_image.Infrared/20250623.120000.MTG-I1.fci.Infrared.self_register.71p89.EUMETSAT.50p0.png
added:
tests/outputs/seviri.Convective_Storms.imagery_clean/20250624.120000.msg-3.seviri.Convective_Storms.global_cylindrical.22p76.nesdisstar.20p0.png
added:
tests/outputs/seviri.Day_Microphys_Summer.imagery_clean/20250624.120000.msg-3.seviri.Day_Microphys_Summer.global_cylindrical.22p76.nesdisstar.20p0.png
added:
tests/outputs/seviri.Day_Microphys_Winter.imagery_clean/20250624.120000.msg-3.seviri.Day_Microphys_Winter.global_cylindrical.22p76.nesdisstar.20p0.png
added:
tests/outputs/seviri.Day_Solar.imagery_clean/20250624.120000.msg-3.seviri.Day_Solar.global_cylindrical.22p76.nesdisstar.20p0.png
added:
tests/outputs/seviri.Dust-RGB.imagery_clean/20250624.120000.msg-3.seviri.Dust_RGB.global_cylindrical.22p76.nesdisstar.20p0.png
added:
tests/outputs/seviri.Natural-Color.imagery_clean/20250624.120000.msg-3.seviri.Natural_Color.global_cylindrical.22p76.nesdisstar.20p0.png
added:
tests/outputs/seviri.Night_Microphys.imagery_clean/20250624.120000.msg-3.seviri.Night_Microphys.global_cylindrical.22p76.nesdisstar.20p0.png
added:
tests/outputs/seviri.Volcanic_Ash.imagery_clean/20250624.120000.msg-3.seviri.Volcanic_Ash.global_cylindrical.22p76.nesdisstar.20p0.png
added:
tests/outputs/seviri.WV-Upper.no_self_register.unprojected_image/20250624.120000.msg-3.seviri.WV-Upper-No-SR.unk.nesdisstar..png
added:
tests/outputs/seviri.WV-Upper.unprojected_image/20250624.120000.msg-3.seviri.WV-Upper.self_register.69p07.nesdisstar.10p0.png
added:
tests/outputs/seviri.airmass.imagery_clean/20250624.120000.msg-3.seviri.airmass.global_cylindrical.22p76.nesdisstar.20p0.png
modified: tests/integration_tests/test_integration.py
modified: tests/scripts/atms.tc.165H.netcdf_geoips.sh
modified: tests/scripts/fci.static.Visible.imagery_annotated.sh
modified: tests/scripts/fci.unprojected_image.Infrared.sh
modified: tests/scripts/seviri.Convective_Storms.imagery_clean.sh
modified: tests/scripts/seviri.Day_Microphys_Summer.imagery_clean.sh
modified: tests/scripts/seviri.Day_Microphys_Winter.imagery_clean.sh
modified: tests/scripts/seviri.Day_Solar.imagery_clean.sh
modified: tests/scripts/seviri.Dust_RGB.imagery_clean.sh
modified: tests/scripts/seviri.Natural_Color.imagery_clean.sh
modified: tests/scripts/seviri.Night_Microphys.imagery_clean.sh
modified: tests/scripts/seviri.Volcanic_Ash.imagery_clean.sh
modified: tests/scripts/seviri.WV-Upper.no_self_register.unprojected_image.sh
modified: tests/scripts/seviri.WV-Upper.unprojected_image.sh
modified: tests/scripts/seviri.airmass.imagery_clean.sh
Testing#
Nextcloud test dataset updates.#
With the recent release of brand new and/or updated test datasets coming from internal NRL, we needed to update our public hosting of datasets to match these changes. The data has since been transferred over, and we need to reflect this within our code base so the CLI can properly install the new datasets. This was a simple update to test-data-urls.yaml, based on a script I wrote to detect unique test datasets being used within plugin packages.
Additionally, this PR fixes a small bug in the ‘geoips config install’ command where ‘all’ wasn’t working properly.
Finally, this PR added test datasets for saphir and cygnss. When calling the test scripts for these datasets, it was found that cygnss’ compare image was out of date and needed updating.
modified: geoips/commandline/geoips_config.yaml
modified: setup/test-data-urls.yaml
modified: tests/integration_tests/test_integration.py
modified:
tests/outputs/cygnss.tc.windspeed.imagery_clean/20240125_060307_SH062024_cygnss_cygnss_windspeed_80kts_6p22_1p0-clean.png
Testing#
Added test_data_amsub to nextcloud#
This PR is part of many updates we’ve made to public test datasets for GeoIPS. We found matching AMSU-B data to what was used internally, and are now making it public so we can expand our test suite.
Additionally, this PR removed a duplicate test-data-urls.yaml file from setup/, which was used before dev-staging got merged into main. test-data-urls.yaml is now located in geoips/commandline/ancillary_info/, and should work whether or not the user is installed in editable mode.
deleted: setup/test-data-urls.yaml
modified: geoips/commandline/ancillary_info/test-data-urls.yaml
modified: tests/integration_tests/test_integration.py
Deprecation#
Update deprecated area_def.x-y_size and area_def.name calls.#
There were a few PendingDeprecationWarnings raised for any area_def.x_size or area_def.y_size attribute accesses in GeoIPS source code. It was recommended we update those calls to area_def.x_size -> area_def.width and area_def.y_size -> area_def.height. This PR does just that.
The same thing goes for area_def.name and area_def.proj4_string. It was recommended we update any call matching area_def.name to area_def.description and area_def.proj4_string to area_def.proj_str instead. The functionality of these attributes are exactly the same.
Lastly, lots of test .yaml files were updated alongside their corresponding imagery (while that had no difference) due to the order of proj-parameters being shifted in the .yaml files. The contents were the same, so no actual functionality has changed.
modified: geoips/plugins/modules/output_formatters/full_disk_image.py
modified: geoips/plugins/modules/output_formatters/imagery_annotated.py
modified: geoips/plugins/modules/output_formatters/imagery_clean.py
modified: geoips/plugins/modules/output_formatters/imagery_windbarbs.py
modified: geoips/plugins/modules/output_formatters/imagery_windbarbs_multi_level.py
modified: geoips/plugins/modules/output_formatters/metadata_default.py
modified: geoips/plugins/modules/procflows/config_based.py
modified: geoips/plugins/modules/procflows/single_source.py
modified: geoips/plugins/modules/readers/abi_l2_netcdf.py
modified: geoips/plugins/modules/readers/seviri_hrit.py
modified: geoips/sector_utils/tc_tracks.py
modified: geoips/sector_utils/utils.py
modified: geoips/sector_utils/yaml_utils.py
modified: geoips/xarray_utils/data.py
modified:
tests/outputs/abi.tc.IR-BD.imagery_annotated.low_memory/20200918_195020_AL202020_abi_goes-16_IR-BD_110kts_100p00_1p0.png.yaml
modified:
tests/outputs/abi.tc.IR-BD.imagery_annotated/20200918_195020_AL202020_abi_goes-16_IR-BD_110kts_100p00_1p0.png.yaml
modified:
tests/outputs/abi.tc.Infrared.imagery_annotated.low_memory/20200918_195020_AL202020_abi_goes-16_Infrared_110kts_100p00_1p0.png.yaml
modified:
tests/outputs/abi.tc.Visible.imagery_annotated/20200918_195020_AL202020_abi_goes-16_Visible_110kts_100p00_1p0.png
modified:
tests/outputs/abi.tc.Visible.imagery_annotated/20200918_195020_AL202020_abi_goes-16_Visible_110kts_100p00_1p0.png.yaml
modified:
tests/outputs/amsr2.tc.89H-Physical.imagery_annotated/20200518_073601_IO012020_amsr2_gcom-w1_89H-Physical_140kts_100p00_res1p0-cr300.png
modified:
tests/outputs/amsr2.tc.89H-Physical.imagery_annotated/20200518_073601_IO012020_amsr2_gcom-w1_89H-Physical_140kts_100p00_res1p0-cr300.png.yaml
modified:
tests/outputs/amsr2.tc_overlay.37pct.imagery_annotated_over_Infrared-Gray/20200518_073601_IO012020_amsr2_gcom-w1_37pct_140kts_95p89_res1p0-cr100-bgInfrared-Gray.png
modified:
tests/outputs/amsr2.tc_overlay.37pct.imagery_annotated_over_Infrared-Gray/20200518_073601_IO012020_amsr2_gcom-w1_37pct_140kts_95p89_res1p0-cr100-bgInfrared-Gray.png.yaml
modified:
tests/outputs/amsr2.tc_overlay.37pct.imagery_annotated_over_Visible/20200518_073601_IO012020_amsr2_gcom-w1_37pct_140kts_95p89_res1p0-cr100-bgVisible.png
modified:
tests/outputs/amsr2.tc_overlay.37pct.imagery_annotated_over_Visible/20200518_073601_IO012020_amsr2_gcom-w1_37pct_140kts_95p89_res1p0-cr100-bgVisible.png.yaml
modified:
tests/outputs/amsr2.tc_overlay.89pct.imagery_annotated_over_Infrared-Gray/20200518_073601_IO012020_amsr2_gcom-w1_89pct_140kts_98p32_res1p0-cr100-bgInfrared-Gray.png
modified:
tests/outputs/amsr2.tc_overlay.89pct.imagery_annotated_over_Infrared-Gray/20200518_073601_IO012020_amsr2_gcom-w1_89pct_140kts_98p32_res1p0-cr100-bgInfrared-Gray.png.yaml
modified:
tests/outputs/amsr2.tc_overlay.89pct.imagery_annotated_over_Visible/20200518_073601_IO012020_amsr2_gcom-w1_89pct_140kts_98p32_res1p0-cr100-bgVisible.png
modified:
tests/outputs/amsr2.tc_overlay.89pct.imagery_annotated_over_Visible/20200518_073601_IO012020_amsr2_gcom-w1_89pct_140kts_98p32_res1p0-cr100-bgVisible.png.yaml
modified:
tests/outputs/amsr2_rss.tc.windspeed.imagery_clean/20200518_073600_IO012020_amsr2rss_gcom-w1_windspeed_140kts_71p87_1p0-clean.png.yaml
modified:
tests/outputs/ascat_knmi.tc.windbarbs.imagery_windbarbs_clean/20210421_014248_WP022021_ascat_metop-c_windbarbs_120kts_78p20_0p5-clean.png.yaml
modified:
tests/outputs/ascat_low_knmi.tc.windbarbs.imagery_windbarbs/20210421_014156_WP022021_ascat_metop-c_windbarbs_120kts_35p17_1p0.png.yaml
modified:
tests/outputs/ascat_noaa_25km.tc.windbarbs.imagery_windbarbs/20230524_235304_WP022023_ascat_metop-c_windbarbs_135kts_39p90_0p7.png.yaml
modified:
tests/outputs/ascat_noaa_50km.tc.wind-ambiguities.imagery_windbarbs/20230524_235200_WP022023_ascat_metop-c_wind-ambiguities_135kts_50p08_1p1.png.yaml
modified:
tests/outputs/ascat_uhr.tc.wind-ambiguities.imagery_windbarbs/20210421_014200_WP022021_ascatuhr_metop-c_wind-ambiguities_120kts_100p00_0p1.png.yaml
modified:
tests/outputs/gmi.tc.89pct.imagery_clean/20200917_171519_AL202020_gmi_GPM_89pct_115kts_78p16_res1p0-cr300-clean.png.yaml
modified:
tests/outputs/oscat_knmi.tc.windbarbs.imagery_windbarbs/20210209_025351_SH192021_oscat_scatsat-1_windbarbs_135kts_75p10_1p0.png.yaml
modified:
tests/outputs/sar.tc.nrcs.imagery_annotated/20181025_203206_WP312018_sar-spd_sentinel-1_nrcs_130kts_58p51_res1p0-cr300.png.yaml
modified:
tests/outputs/viirsday.tc.Night-Vis-IR.imagery_annotated/20210209_074210_SH192021_viirs_noaa-20_Night-Vis-IR_130kts_100p00_1p0.png.yaml
Deprecation#
Fix ahi_hsd warnings#
Simple PR which addresses warnings raised via the ahi_hsd reader. The changes made in this PR address an unclosed file handler and multiple instances where n-dimensional arrays were cast to a scalar value, which will be deprecated in the future. No functional changes were made.
modified: geoips/plugins/modules/readers/ahi_hsd.py
Enhancement#
Reorganize testing documentation structure#
Restructured testing documentation by creating a dedicated testing section under contribute/ and consolidating test running instructions into a comprehensive guide
deleted: docs/dev/integration_tests.rst
added: docs/source/contribute/testing/index.rst
added: docs/source/contribute/testing/running_tests.rst
modified: docs/source/contribute/index.rst
Enhancement#
Pydantic gridline annotator plugin models#
Transformed legacy json schema code for gridline annotators into our newly developed pydantic schema. Currently still support legacy schema validation for operational GeoIPS, however these changes are one of many needed to support the order based procflow.
Additionally, this PR modified unit testing code for sectors and feature annotators in an attempt to generalize the testing that will be applied to these models. Added a test cases file for gridline annotators models as well, used for unit testing.
deleted: tests/unit_tests/pydantic/feature_annotators/test_feature_annotators.py
deleted: tests/unit_tests/pydantic/sectors/test_sectors.py
added: geoips/pydantic/gridline_annotators.py
added: tests/unit_tests/pydantic/gridline_annotators/test_cases.yaml
added: tests/unit_tests/pydantic/test_pydantic_models.py
modified: geoips/plugins/yaml/gridline_annotators/default.yaml
modified: geoips/plugins/yaml/gridline_annotators/default_palegreen.yaml
modified: geoips/plugins/yaml/gridline_annotators/north_pole.yaml
modified: geoips/plugins/yaml/gridline_annotators/tc_0p25degree.yaml
modified: geoips/plugins/yaml/gridline_annotators/tc_pmw.yaml
modified: geoips/plugins/yaml/gridline_annotators/tc_visir.yaml
modified: geoips/plugins/yaml/gridline_annotators/tc_visir_3200km.yaml
modified: geoips/plugins/yaml/gridline_annotators/tc_windspeed.yaml
modified: geoips/pydantic/feature_annotators.py
modified: tests/unit_tests/pydantic/feature_annotators/test_cases.yaml
modified: tests/unit_tests/pydantic/utils.py
Enhancement#
Geoips run order based command#
This update adds a new command, geoips run order_based command, which executes
the order based process workflow (procflow). While there will likely be updates to
this command in the future, these changes allow you to execute an obp procflow from
the command line rather than manually calling the associated python module.
Currently, the order based procflow only supports reader plugins. More to be supported in the future.
modified: geoips/commandline/ancillary_info/alias_mapping.yaml
modified: geoips/commandline/ancillary_info/cmd_instructions.yaml
modified: geoips/commandline/commandline_interface.py
modified: geoips/commandline/geoips_command.py
modified: geoips/commandline/geoips_run.py
modified: tests/unit_tests/commandline/test_geoips_run.py
Enhancement#
Write cached json files to geoips_cache_dir#
The CLI was not usable when installed by a user other than the user running the CLI. This was because the CLI writes cached json files and was writing them into the install directory. This PR introduces GEOIPS_CACHE_DIR which defaults to platformdirs.user_cache_dir(). It also reorganizes a bit to move the caching functions to geoips/utils for use by other code. This will also be used by the plugin registry. Note, this also adds a new formal dependency on platformdirs, however, this was already a dependency of other dependencies.
deleted: tests/unit_tests/commandline/json_missing/cmd_instructions.yaml
added: geoips/utils/cache_files.py
added: tests/unit_tests/utils/cache_files.py
modified: .cspell.json
modified: pyproject.toml
modified: geoips/commandline/ancillary_info/__init__.py
modified: geoips/commandline/commandline_interface.py
modified: geoips/commandline/geoips_command.py
modified: geoips/filenames/base_paths.py
modified: geoips/interfaces/base.py
modified: tests/unit_tests/commandline/cli_top_level_tester.py
modified: tests/unit_tests/commandline/test_get_commandline_instructions.py
Enhancement#
Alphabetize command-line arguments in help output#
Implemented a custom argument formatter that sorts command-line arguments alphabetically when displaying help messages. This improves readability of help text by presenting arguments in a consistent, predictable order.
modified: geoips/commandline/geoips_command.py
modified: docs/source/concepts/architecture/index.rst
added: tests/unit_tests/commandline/test_help_sorting.py
added: docs/source/concepts/architecture/command_line_interface.rst
Enhancement#
Add support for neutral tests in yaml-based testing#
Enhances YAML-based testing by adding support for neutral tests and pydantic validation.
deleted: tests/unit_tests/pydantic/feature_annotators/test_cases.yaml
deleted: tests/unit_tests/pydantic/sectors/test_cases.yaml
added: docs/source/releases/latest/1001-add-support-for-neutral-tests-in-yaml-based-testing.yaml
added: tests/unit_tests/pydantic/feature_annotators/test_cases_bad.yaml
added: tests/unit_tests/pydantic/feature_annotators/test_cases_neutral.yaml
added: tests/unit_tests/pydantic/sectors/test_cases_bad.yaml
added: tests/unit_tests/pydantic/sectors/test_cases_neutral.yaml
modified: tests/unit_tests/pydantic/feature_annotators/test_feature_annotators.py
modified: tests/unit_tests/pydantic/sectors/test_sectors.py
modified: tests/unit_tests/pydantic/utils.py
Enhancement#
Add –warnings option to the cli#
Add a –warnings option to the CLI to allow controlling behavior when warnings are encountered. Can be set to hide, print, or raise an error on warnings.
modified: geoips/commandline/commandline_interface.py
modified: geoips/commandline/geoips_command.py
modified: geoips/commandline/geoips_test.py
modified: tests/unit_tests/commandline/test_geoips_config_create_registries.py
modified: tests/unit_tests/commandline/test_geoips_config_delete_registries.py
modified: tests/unit_tests/commandline/test_geoips_list_registries.py
Enhancement#
Add support for invalid json in command-line kwargs. eg. –reader-kwargs#
‘JSON does not officially support single quotes ( eg. ‘’ ). However, sometimes people include them in command line kwargs. This adds support for single quotes by calling ast.literal_eval instead of json.loads.’
added: docs/source/releases/latest/335-simplify-cli-kwargs-arguments.yaml
modified: geoips/commandline/args.py
Enhancement#
Update dockerfile#
This update adds multi-stage builds for dev, prod and testing.
modified: Dockerfile
Removal#
Remove outdated integration tests documentation#
Removed the old integration_tests.rst file as part of documentation reorganization
deleted: docs/dev/integration_tests.rst
Installation#
Finalize the conda create command for completeness#
Add a thorough list of packages for the conda create command. This is a more complete list than is needed, but may help avoid potential issues with missing libraries for different architectures.
modified: docs/source/getting-started/installing/linux_with_conda.rst
Installation#
Add a troubleshooting page to installation section of docs#
During the workshop installation, this can be used to compile issues during the installation process. After the workshop, this can be used to allow for a simplified installation, with potential issues and resolutions identified.
modified: docs/source/getting-started/installing/troubleshooting.rst
Hot fix#
Add support for alternative data_fusion package name#
data_fusion is not an allowed name for the data_fusion package on pypi. We have temporarily pushed it to pypi as geoips_data_fusion. This change allows geoips run to use the pypi packaged version of the data_fusion file.
modified: geoips/commandline/geoips_run.py