geoips.pydantic_models.v1 package#

Submodules#

geoips.pydantic_models.v1.algorithms module#

Pydantic models used to validate GeoIPS algorithm plugins.

pydantic model geoips.pydantic_models.v1.algorithms.AlgorithmArgumentsModel[source]#

Bases: PermissiveFrozenModel

Algorithm step argument definition.

Pydantic model defining and validating Algorithm step arguments.

Fields:
Validators:

field gamma_list: List[float] | None = None#
field grid_geo: bool | None = None#
field input_units: str = None#

Units of input data, for applying necessary conversions. Defaults to None, resulting in no unit conversions.

field inverse: bool = None#

  • Boolean flag indicating whether to inverse (True) or not (False) * If True, returned data will be inverted * If False, returned data will not be inverted

Constraints:

  • strict = True

field mask_day: bool = None#
field mask_night: bool = None#
field max_day_zen: float | None = None#
field max_night_zen: float | None = None#
field max_outbounds: str | None = None#

Method to use when applying maximum value of’output_data_range’, if specified. Valid values are: * retain: keep all pixels as is * mask: mask all pixels that are out of range * crop: set out of range values to the nearest bound (min_val or max_val)

field min_night_zen: float | None = None#
field min_outbounds: str | None = None#

Method to use when applying minimum value of’output_data_range’, if specified. Valid values are: * retain: keep all pixels as is * mask: mask all pixels that are out of range * crop: set out of range values to the nearest bound (min_val or max_val)

field norm: bool = None#

Boolean flag indicating whether to normalize (True) or not (False)* * If True, returned data will be in the range from 0 to 1: * If False, returned data will be in the range from min_val to max_val

Constraints:

  • strict = True

field output_data_range: tuple[typing.Annotated[float, Strict(strict=True)], typing.Annotated[float, Strict(strict=True)]] | None = None#

list of min and max value for output data product. This is applied LAST after all other corrections/adjustments. If None, use data.min() and data.max()

field output_units: str = None#

Units of input data, for applying necessary conversions. Defaults to None, resulting in no unit conversions.

field pressure_key: str | None = None#
field pressure_level_range: tuple[float, float] | None = None#

list of min and max pressure levels to filter derived motion wind retrievals. Defaults to None, which results in using all wind retrievals.

field satellite_zenith_angle_cutoff: float | None = None#

Cutoff for masking data where satellite zenith angle exceedsthreshold. None, no masking

field scale_factor: float | None = None#
field sun_zen_correction: bool | None = None#

Boolean flag indicating whether to apply solar zenith correction(True) or not (False) * If True, returned data will have solar zenith correction applied (see data_manipulations.corrections.apply_solar_zenith_correction) * If False, returned data will not be modified based on solar zenith angle)

field time_dim: int | None = None (alias 'Time_Dimension')#
field time_fcst: int | None = None#
field time_key: str = None#
field var_map: Dict[str, str] | None = {}#

Dictionary that maps input variables to names used in xobj

field variables: List[str] | None = None#

List of input variables used in algorithm processing

geoips.pydantic_models.v1.bases module#

Pydantic base models for GeoIPS.

Intended for use by other base models.

PluginModel should be used as the parent class of all other plugin models.

Other models defined here validate field types within child plugin models.

pydantic model geoips.pydantic_models.v1.bases.CoreBaseModel[source]#

Bases: BaseModel

CoreBaseModel for GeoIPS Order-Based Procflow data model validation.

This model provides a standardized Pydantic base class with custom configuration and validation logic for all GeoIPS models built using Pydantic library. It consolidates useful configurations, custom validators, and utility methods.

Features#

  • Pretty-printing:

    Make Pydantic models pretty-print by default. Overrides the default string representation of Pydantic models to generate a user-friendly, JSON-formatted output with two-space indentation.

  • Configured Options:

    Includes a customized ConfigDict with the following options set:

    • str_strip_whitespace=True to trim whitespace around input.

    • validate_by_alias=True to populate data using aliased field names.

    • validate_by_name=True to populate an aliased using its model-defined name.

    • loc_by_alias=False to disallow usage of alias field name in error locations.

    • validate_assignment=False Disables model revalidation when data is changed.

    • arbitrary_types_allowed=True to allow custom data types as field types.

    • strict=False to disallow coercion of values to declared type when possible.

    • allow_inf_nan=False to disallow +/-infinity and NaN values in float and

      decimal fields.

  • check_restricted_fields:

    • Model-level validator that disallows user input for restricted fields.

    • Allows defining a list of restricted fields globally or at the class level.

    • Raises a validation error if a restricted field is provided by the user.

  • model_name:

    • Prints the model name along with the data.

    • Useful for debugging and logging purposes(for dev).

    • This method would be further enhanced in future PR

Validators:

  • check_restricted_fields » all fields

property model_name#

Return the model name for logging and end-user interactions.

restricted_fields: ClassVar[Tuple[str, ...]] = ()#
pydantic model geoips.pydantic_models.v1.bases.DynamicModel[source]#

Bases: CoreBaseModel

Inherits all of the configuration from CoreBaseModel.

The following overrides are applied: - extra=”forbid”: Forbids additional fields beyond those defined in the model. - frozen=False: Allows modification of field values after object instantiation.

This model is intended for cases where additional fields are not permitted and the object data remains mutable after initialization.

Validators:

pydantic model geoips.pydantic_models.v1.bases.FrozenModel[source]#

Bases: CoreBaseModel

Inherits all of the configuration from CoreBaseModel.

The following overrides are applied: - extra=”forbid”: Forbids additional fields beyond those defined in the model. - frozen=True: Disallows modification of field values after object instantiation.

This model is intended for cases where additional fields are not permitted and the object data must remain immutable after initialization.

Validators:

pydantic model geoips.pydantic_models.v1.bases.PermissiveDynamicModel[source]#

Bases: CoreBaseModel

Inherits all of the configuration from CoreBaseModel.

The following overrides are applied: - extra=”allow”: Allows additional fields beyond those defined in the model. - frozen=False: Allows modification of field values after object instantiation.

This model is intended for cases where additional fields are permitted and the object data remains immutable after initialization.

Validators:

pydantic model geoips.pydantic_models.v1.bases.PermissiveFrozenModel[source]#

Bases: CoreBaseModel

Inherits all of the configuration from CoreBaseModel.

The following overrides are applied: - extra=”allow”: Allows additional fields beyond those defined in the model. - frozen=True: Disallows modification of field values after object instantiation.

This model is intended for cases where additional fields are permitted and the object data must remain immutable after initialization.

Validators:

pydantic model geoips.pydantic_models.v1.bases.PluginModel[source]#

Bases: FrozenModel

Base Plugin model for all GeoIPS plugins.

This should be used as the base class for all top-level PluginModels. It adds standard plugin attributes for inheritance. It validates YAML plugins for the order based procflow.

See the YAML plugin documentation here for more information about how this is used.

Fields:
Validators:

  • _derive_package_name » all fields

  • _set_description » all fields

  • _validate_apiVersion » apiVersion

  • _validate_interface » interface

  • _validate_one_line_description » description

field abspath: str = None#

Absolute path to the plugin file.

field description: str = None#

A short description or defaults to first line from docstring.

field docstring: str [Required]#

Docstring for the plugin in numpy format.

field family: PythonIdentifier [Required]#

Family of the plugin.

Constraints:

  • func = <function python_identifier at 0x7f882bda4540>

field interface: PythonIdentifier [Required]#

Name of the plugin’s interface. Run geoips list interfaces to see available options.

Constraints:

  • func = <function python_identifier at 0x7f882bda4540>

field is_registered: bool = True#

Whether or not this plugin is registered.

field name: str [Required]#

Plugin name.

field package: PythonIdentifier = (FieldInfo(annotation=NoneType, required=True, description='Package that contains this plugin.'),)#
Constraints:

  • func = <function python_identifier at 0x7f882bda4540>

field relpath: str = None#

Path to the plugin file relative to its parent package.

apiVersion: str = 'geoips/v1'#
class geoips.pydantic_models.v1.bases.PluginModelMetadata(name: str, bases: Tuple[type, ...], namespace: Dict[str, Any], **kwargs: Any)[source]#

Bases: ModelMetaclass

API version and namespace metadata for the corresponding plugin model.

This is used to derive ‘apiVersion’ and ‘namespace’ for any given PluginModel. PluginModel can be instantiated directly or a child class of PluginModel can be instantiated and the functionality will for the same.

Initially attempted to use __init_subclass__ in the PluginModel class itself, but that only supported child classes of PluginModel (i.e. WorkflowPluginModel, …), but not instantiation of PluginModel itself.

NOTE: Need to inherit from ModelMetaclass, otherwise we’ll wind up with this error:

E TypeError: metaclass conflict: the metaclass of a derived class must be a (non-strict) subclass of the metaclasses of all its bases

geoips.pydantic_models.v1.bases.get_interfaces(namespace) → set[str][source]#

Return a set of distinct interfaces.

This function returns all available plugin interfaces. The results are cached for runtime memory optimization.

Returns:

set of interfaces

Return type:

set of str

geoips.pydantic_models.v1.bases.python_identifier(val: str) → str[source]#

Validate if a string is a valid Python identifier.

Validate if a string is a valid Python identifier and not a reserved Python keyword. See for more information on Python identifiers and reserved keywords.

Validation is performed by calling str.isidentifier and keyword.iskeyword.

Parameters:

val (str) – The input string to validate.

Returns:

The input string if it is a valid Python identifier.

Return type:

str

Raises:

ValueError – If the input string is invalid as a Python identifier or a reserved keyword.

geoips.pydantic_models.v1.bases.step_reference(val: str) → str[source]#

Validate a workflow step reference, optionally into a sub-workflow.

A step reference is one or more Python identifiers joined by . (dots). A single segment (e.g. "reader") refers to a top-level step. A dotted reference (e.g. "subwf.algo" or "split.scope.algo") refers to a step nested inside a workflow or split container step; each segment must itself be a valid Python identifier.

Parameters:

val (str) – The input string to validate.

Returns:

The input string if every dot-separated segment is a valid Python identifier.

Return type:

str

Raises:

ValueError – If val is empty or any segment is not a valid Python identifier.

geoips.pydantic_models.v1.colormappers module#

Pydantic models used to validate GeoIPS OBP v1 colormapper plugins.

pydantic model geoips.pydantic_models.v1.colormappers.ColormapperArgumentsModel[source]#

Bases: PermissiveFrozenModel

Colormapper step argument definition.

Pydantic model defining and validating Colormapper step arguments.

Fields:
Validators:

field cbar_full_width: bool = False#

“Extend the colorbar across the full width of the image”

field cbar_label: str = 'plugin_provided'#

Positional parameter passed to cbar.set_label If specified, use cbar_label string as colorbar label.

field cbar_spacing: str = 'proportional'#

“spacing” argument to pass to fig.colorbar; can also specify directly within “colorbar_kwargs”

field cbar_tick_labels: list[str] | None = None#

‘labels’ argument to pass to cbar.set_ticks. can also specify directly within ‘set_ticks_kwarg’

field cbar_ticks: list[float] | None = None#

Positional parameter passed to cbar.set_ticks Specify explicit list of ticks to include for colorbar.None indicates ticks at int(min) and int(max) values

field cmap_name: str = 'plugin_provided'#

Specify the name of the resulting matplotlib colormap. If no ascii_path specified, will use builtin matplotlib colormap of name cmap_name.

field cmap_path: str | None = None#
field cmap_source: str = 'plugin_provided'#
field colorbar_kwargs: dict | None = None#

keyword arguments to pass through directly to ‘fig.colorbar’

field create_colorbar: bool | None = True#

Specify whether the image should contain a colorbar or not.

field data_range: tuple[float, float] | str = 'plugin_provided'#

Min and max value for colormapmatplotlib.colors.Normalize(vmin=min_val, vmax=max_val)

field pressure_range_legend: List[str] | None = None#

List of strings that are used for setting the cbar tick labels

field set_label_kwarg: dict | None = None#

keyword arguments to pass through directly to “cbar.set_label”

field set_ticks_kwargs: dict | None = None#

keyword arguments to pass through directly to “cbar.set_ticks”

geoips.pydantic_models.v1.coverage_checkers module#

Pydantic models used to validate GeoIPS OBP v1 coverage-checker plugins.

pydantic model geoips.pydantic_models.v1.coverage_checkers.CoverageCheckerArgumentsModel[source]#

Bases: PermissiveFrozenModel

Coverage-Checker step argument definition.

Pydantic model defining and validating Coverage Checker step arguments.

Fields:
Validators:

field area_def: str = None#

Area definition identifier.

field radius_km: float | None = 300#

Radius of center disk to check for coverage.

field variable_name: str = None#

Variable name to check percent unmasked.

geoips.pydantic_models.v1.feature_annotators module#

Pydantic models used to validate GeoIPS feature annotator plugins.

pydantic model geoips.pydantic_models.v1.feature_annotators.CartopyFeature[source]#

Bases: PermissiveFrozenModel

Generic model for cartopy features.

Fields:
Validators:

  • validate_enabled_fields » all fields

field edgecolor: Tuple[float, float, float] | Tuple[float, float, float, float] | str = None#

An rgb tuple, matplotlib named color, or hexidecimal string (#XXXXXX).For more info, see: https://matplotlib.org/stable/users/explain/colors/colors.html Used for Cartopy feature edges.

field enabled: bool [Required]#

Whether or not to enable this feature.

Constraints:

  • strict = True

field linewidth: float | None = None#

The width in pixels of the specified feature.

Constraints:

  • ge = 0

pydantic model geoips.pydantic_models.v1.feature_annotators.FeatureAnnotatorPluginModel[source]#

Bases: PluginModel

Feature Annotator plugin format.

Fields:
Validators:

field spec: FeatureAnnotatorSpec [Required]#

Specification of how to apply cartopy features to your annotated imagery. Works alongside matplotlib and cartopy to generate these features. For more information, see: https://scitools.org.uk/cartopy/docs/v0.14/matplotlib/feature_interface.html

pydantic model geoips.pydantic_models.v1.feature_annotators.FeatureAnnotatorSpec[source]#

Bases: FrozenModel

Feature Annotator spec (specification) format.

Fields:
Validators:

field background: Tuple[float, float, float] | Tuple[float, float, float, float] | str | None = None#

An rgb tuple, matplotlib named color, or hexidecimal string (#XXXXXX).For more info, see: https://matplotlib.org/stable/users/explain/colors/colors.html used for the background of the image.

field borders: CartopyFeature [Required]#

A cartopy borders feature.

Constraints:

  • strict = True

field coastline: CartopyFeature [Required]#

A cartopy coastline feature.

Constraints:

  • strict = True

field rivers: CartopyFeature [Required]#

A cartopy rivers feature.

Constraints:

  • strict = True

field states: CartopyFeature [Required]#

A cartopy states feature.

Constraints:

  • strict = True

geoips.pydantic_models.v1.filename_formatters module#

Pydantic models used to validate GeoIPS OBP v1 filename-formatter plugins.

pydantic model geoips.pydantic_models.v1.filename_formatters.FilenameFormatterArgumentsModel[source]#

Bases: PermissiveFrozenModel

Filename-Formatter step argument definition.

Pydantic model defining and validating Filename Formatter step arguments.

Fields:
Validators:

field area_def: AreaDefinition | None = None#

Spatial domain to process.

field basedir: str = 'plugin_provided'#

Full path to base directory of final product

field coverage: float | None = None#

Image coverage, float between 0.0 and 100.0

Constraints:

  • ge = 0.0

  • le = 100.0

field extension: str = None#

Extension of filename

field extra: str | None = None#

String to include in filename ‘extra’ field If None, use fillval of ‘x’

field metadata_dir: str = 'metadata'#
field metadata_type: str = 'sector_information'#
field output_dict: dict[str, Any] | None = None#
field output_type: str = 'plugin_provided'#

Requested output format, ie png, jpg, tif, etc, defaults to None

field output_type_dir: str = None#

If None, default to output_type.

field product_dir: str = None#
field product_filename: str | None = None#
field product_subdir: str = None#
field source_dir: str = None#

geoips.pydantic_models.v1.gridline_annotators module#

Pydantic models used to validate GeoIPS gridline annotator plugins.

pydantic model geoips.pydantic_models.v1.gridline_annotators.GridlineAnnotatorPluginModel[source]#

Bases: PluginModel

Gridline Annotator plugin format.

Fields:
Validators:

field spec: GridlineAnnotatorSpec [Required]#

Specification of how to apply gridlines, labels, and spacing to your annotated imagery. Works alongside matplotlib and cartopy to generate these attributes.

pydantic model geoips.pydantic_models.v1.gridline_annotators.GridlineAnnotatorSpec[source]#

Bases: FrozenModel

Gridline Annotator spec (specification) format.

Fields:
Validators:

field background: Tuple[float, float, float] | Tuple[float, float, float, float] | str | None = None#

A rgb tuple, matplotlib named color, or hexidecimal string (#XXXXXX) to apply to the background of your image frame. For more info, see: https://matplotlib.org/stable/users/explain/colors/colors.html

field labels: Labels [Required]#

Label settings for the plot.

field lines: Lines [Required]#

Line settings for the plot.

field spacing: Spacing [Required]#

Spacing settings for the plot.

pydantic model geoips.pydantic_models.v1.gridline_annotators.Labels[source]#

Bases: PermissiveFrozenModel

Model used to format labels in annotated imagery.

For more information, visit https://matplotlib.org/stable/ to see more context in how to specify these fields.

Fields:
Validators:

field alpha: float | None = None#

Inverse percentage of label transparency (0 = fully transparent, 1 = fullyopaque).

field backgroundcolor: Tuple[float, float, float] | Tuple[float, float, float, float] | str | None = None#

A rgb tuple, matplotlib named color, or hexidecimal string (#XXXXXX) to apply to the background of your label frame. For more info, see: https://matplotlib.org/stable/users/explain/colors/colors.html

field bottom: bool [Required]#

Whether to display the label at the bottom.

field color: Tuple[float, float, float] | Tuple[float, float, float, float] | str | None = None#

A rgb tuple, matplotlib named color, or hexidecimal string (#XXXXXX) to apply to the label text of your image. For more info, see: https://matplotlib.org/stable/users/explain/colors/colors.html

field fontfamily: str | None = None#

Font family for the label text.

field fontsize: float | str | None = None#

Font size for the label text.

field fontstretch: int | str | None = None#

Stretch level of the font.

field fontstyle: str | None = None#

Style of the font (e.g., ‘normal’, ‘italic’, ‘oblique’).

field fontvariant: str | None = None#

Font variant (e.g., ‘normal’, ‘small-caps’).

field fontweight: int | str | None = None#

Weight of the font.

field left: bool [Required]#

Whether to display the label on the left.

field linespacing: float | None = None#

Spacing between lines of text.

field mouseover: bool | None = None#

Whether to enable mouseover effect.

field position: List[float] | None = None#

Absolute (x, y) position of the label.

field right: bool [Required]#

Whether to display the label on the right.

field rotation: float | str | None = None#

Rotation of the label (degrees or ‘vertical’/’horizontal’).

field rotation_mode: str | None = None#

Rotation mode (‘default’ or ‘anchor’).

field snap: bool | None = None#

Whether to snap the label to the pixel grid.

field top: bool [Required]#

Whether to display the label at the top.

field wrap: bool | None = None#

Whether to allow text wrapping.

field xpadding: int | None = None#

Pixel offset for x-axis labels.

field ypadding: int | None = None#

Pixel offset for y-axis labels.

field zorder: float | None = None#

Order of label rendering (lower value = earlier rendering).

pydantic model geoips.pydantic_models.v1.gridline_annotators.Lines[source]#

Bases: FrozenModel

Model used to format gridlines in annotated imagery.

For more information, visit https://matplotlib.org/stable/ to see more context in how to specify these fields.

Fields:
Validators:

field color: Tuple[float, float, float] | Tuple[float, float, float, float] | str [Required]#

Color of the line (named color, hex, or rgb tuple).

field linestyle: List[int] [Required]#

Pattern of dashes and gaps in the line.

Constraints:

  • min_length = 2

  • max_length = 2

field linewidth: float [Required]#

Width of the line.

pydantic model geoips.pydantic_models.v1.gridline_annotators.Spacing[source]#

Bases: FrozenModel

Model used to format the spacing of gridlines in annotated imagery.

Fields:
Validators:

field latitude: float | Literal['auto'] [Required]#

Latitude spacing in degrees, can be a float or a string. If a string, it must be ‘auto’, which represents automatic spacing based on your area_def.

field longitude: float | Literal['auto'] [Required]#

Longitude spacing in degrees, can be a float or a string. If a string, it must be ‘auto’, which represents automatic spacing based on your area_def.

geoips.pydantic_models.v1.interpolators module#

Pydantic models for interpolator plugins validation.

pydantic model geoips.pydantic_models.v1.interpolators.InterpGaussInterpolator[source]#

Bases: PermissiveFrozenModel

Validate InterpGauss Interpolator.

Fields:
Validators:

field drop_nan: bool = False#

Whether to drop the nan values (default:False)

Constraints:

  • strict = True

field sigmaval: int = 10000#

Used for interp_type ‘gauss’ - multiplication factor for sigmas option: * sigmas = [sigmas]*len(list_of_arrays)

pydantic model geoips.pydantic_models.v1.interpolators.InterpGridInterpolator[source]#

Bases: PermissiveFrozenModel

Validate InterpGrid Interpolator.

Fields:
Validators:

field method: str = 'linear'#

Method of interpolation; defaults to linear

pydantic model geoips.pydantic_models.v1.interpolators.InterpolatorArgumentsModel[source]#

Bases: InterpGaussInterpolator, InterpGridInterpolator

Validate common Interpolator arguments.

Fields:
Validators:

field area_def: str | None = None#

Area definition identifier.

field drop_nan: StrictBool = False#

Whether to drop the nan values (default:False)

Constraints:

  • strict = True

field method: str = 'linear'#

Method of interpolation; defaults to linear

field sigmaval: int = 10000#

Used for interp_type ‘gauss’ - multiplication factor for sigmas option: * sigmas = [sigmas]*len(list_of_arrays)

field varlist: List[str] | None = None#

variables required for specific interpolation processing

geoips.pydantic_models.v1.output_checkers module#

Pydantic models used to validate GeoIPS output checker plugins.

pydantic model geoips.pydantic_models.v1.output_checkers.OutputCheckerArgumentsModel[source]#

Bases: FrozenModel

Output Checker spec (specification) format.

Fields:
Validators:

field compare_path: FilePath | str [Required]#

The path to the comparison file.

field output_products: List[FilePath] | List[str] | None = None#

A list of paths to the output file(s).

field threshold: float | None = None#

Threshold for the image comparison. Argument to pixelmatch. Between 0 and 1, with 0 the most strict comparison, and 1 the most lenient.

geoips.pydantic_models.v1.output_formatters module#

Pydantic models used to validate GeoIPS OBP v1 output-formatter plugins.

pydantic model geoips.pydantic_models.v1.output_formatters.OutputFormatterArgumentsModel[source]#

Bases: PermissiveFrozenModel

Output-Formatter step argument definition.

Pydantic model defining and validating Output Formatter step arguments.

Fields:
Validators:

field append: bool = False#

When True, open the output file in append mode (‘a’) instead of write mode(‘w’). Automatically forced to True for every dataset beyond the firstwhen multiple datasets are written to the same output file.

field area_def: AreaDefinition | None = None#

The domain over which to read data.

field basedir: str = 'plugin_provided'#

Used to construct relative product paths in metadata output. Originally used to strip the TCWWW root path before replacing it with a public URL; now superseded by replace_geoips_paths() and no longer used across the entire implementation.

field bg_data: Any | None = None#
field bg_datatype_title: str | None = None#

background data type

field bg_mpl_colors_info: dict[str, Any] | None = None#

Matplotlib colormap configuration for the background dataset

field bg_product_name_title: str | None = None#

Title of background product

field bg_xarray: Any | None = None#
field clean_fname: str | None = None#

Output file path for a overlay-free elements such as no background imagery, coastlines, gridlines, title, or colorbar. When None, the clean image is skipped.

field clobber: bool = False#

whether to overwrite the output file even if it exists

field cog: bool = True#

Whether to produce a Cloud-Optimized GeoTIFF with internal overview levels. Currently this has no effect as overview generation is handled automatically by cog_translate() call.

field existing_image: str | None = None#

File path to a pre-rendered image onto which new data would be composited

field hist_colorbar: bool = 'plugin_provided'#

Whether to display a histogram-enhanced colorbar instead of the standard colorbar. When True, replaces the standard colorbar with a combined colorbar and data-distribution histogram generated by hist_cmap()

field is_3d: bool = False#

When True, interpret the product data as a 3D array; otherwise,interpret it as a 2D array

field metadata_dir: str = 'metadata'#

Subdirectory name for metadata; using non-default allows for non-operational outputs

field metadata_fname_dict: dict | None = None#

Dictionary of filename metadata passed to the metadata_tc_output_yaml.Only the ‘product_name’ key is accessed to look the product plugin while computing coverage for TC metadata YAML output

field mpl_colors_info: dict[str, Any] | None = None#

Matplotlib colormap configuration dict generated by set_matplotlib_colors_standard(). The common keys include ‘cmap’, ‘norm’, ‘cbar_label’, and / or ‘colorbar’ (bool). When None, a default grayscale colormap spanning the data range is automatically generated. Note: If ‘hist_colorbar=True’, the ‘colorbar’ flage is set to False to suppress the standard colorbar.

field output_dict: dict | None = None#

Only ‘product_spec_override’ key is used to override the product specification used for coverage checking

field output_fname_dict: dict | None = None#
field overwrite: bool = True#

When False, skip writing the output file if it already exists.

field pressure_range_dict: dict | None = None#
field product_datatype_title: str | None = None#

Display name for the data type in figure title.

field product_name_title: str | None = None#

Display name for the product shown in figure titles. When None, falls back to the product_name value

field remove_duplicate_minrange: Any | None = None#
field savefig_kwargs: dict | None = None#

Extra keyword arguments to be used by matplotlib’s savefif(). When None, defaults to an empty dict.

copyright string

field title_formatter: str | None = None#

format for title

field use_compression: bool = False#

When True, apply zlib compression (level 5) to all variables when writing netCDF ouput.

field var_name: str | None = None#

Variable name to extract from the xarray Dataset for plotting.When None, falls back to product_name. Useful when the variable to plot differs from the product name.

field working_directory: str = 'plugin_provided'#

Base output directory for generated files, Defaults to GEOIPS_OUTDIRS

field x_size: int = None#

Number of pixels in the x direction of the projected area definition

field y_size: int = None#

Number of pixels in the y direction of the projected area definition

geoips.pydantic_models.v1.products module#

Pydantic PluginModel for GeoIPS Product and Product Default plugins.

Validates Product and product default plugins using pydantic. Intended to be a ‘carryover’ model which will be used until we fully switch over to using workflow plugins.

pydantic model geoips.pydantic_models.v1.products.ModulePluginArgumentsModel[source]#

Bases: PermissiveFrozenModel

Format specifying which module plugin to use and the arguments to provide to it.

Normally one of ‘algorithm’, ‘colormapper’, ‘interpolator’, but can be other types of module plugins as well (such as ‘coverage_checker’).

Fields:
Validators:

field arguments: dict [Required]#

A dictionary of arbitrary arguments applicable to this specific plugin.

field name: PythonIdentifier [Required]#

The name of the module plugin to be used.

Constraints:

  • func = <function python_identifier at 0x7f882bda4540>

pydantic model geoips.pydantic_models.v1.products.ProductDefaultPluginModel[source]#

Bases: PluginModel

Format for product_default plugins.

Validated with pydantic models.

Fields:
Validators:

  • _validate_plugins_if_exist » all fields

field spec: ProductDefaultSpec [Required]#

The specification of a product default plugin.

pydantic model geoips.pydantic_models.v1.products.ProductDefaultSpec[source]#

Bases: PermissiveFrozenModel

Format of the argument specifications for a product default plugin.

Additional fields may be added as needed.

As well, you can add as many arguments to a certain plugin as needed. Keep in mind these arguments must be present in the actual module plugin.

Fields:
Validators:

field algorithm: SpecPlugin = None#

The specification of an algorithm plugin.

field colormapper: SpecPlugin = None#

The specification of an colormapper plugin.

field coverage_checker: SpecPlugin = None#

The specification of an coverage_checker plugin.

field display_name: str = None#

The display name of your product.

field interpolator: SpecPlugin = None#

The specification of an interpolator plugin.

field mtif_type: str = None#

The format of METOC TIFF to output.

field pad_area_definition: bool = None#

Whether or not to pad your area definition if specified.

field windbarb_plotter: SpecPlugin = None#

The specification of an windbarb_plotter plugin.

class geoips.pydantic_models.v1.products.ProductPluginModel(**data)[source]#

Bases: object

The format of a singular product plugin or a list of them.

pydantic model geoips.pydantic_models.v1.products.ProductSpec[source]#

Bases: ProductDefaultSpec

Format of the argument specifications for a product plugin.

Additional fields may be added as you can override a product_defaults’ arguments if referenced. This may change in the future as we solidify what these arguments will look like.

Fields:
Validators:

field variables: List[str] [Required]#

A list of one or more variables derived from one of the ‘source_names’ referenced in the product plugin. For example, ‘B13BT’ from ‘abi’ or ‘ahi’.

pydantic model geoips.pydantic_models.v1.products.ProductsListPluginModel[source]#

Bases: PluginModel

Format for how to specify a list of product plugins.

Fields:
Validators:

field spec: ProductsListSpec [Required]#

The specification format of a plugin list.

pydantic model geoips.pydantic_models.v1.products.ProductsListSpec[source]#

Bases: FrozenModel

Format for the Product ‘spec’ field.

Uses FrozenModel, meaning no additional fields can be added.

Fields:
Validators:

field products: List[SingleProductPluginModel] [Required]#

A list of one or more products that fall under the same source name.For example, Visible and Infrared under the ‘abi’ source name.

pydantic model geoips.pydantic_models.v1.products.SingleProductPluginModel[source]#

Bases: PluginModel

Format for how to specify a singular product plugin.

Fields:
Validators:

  • _validate_interface » interface

  • check_family_pd_xor » all fields

  • load_product_default » all fields

field interface: Literal['products'] = 'products'#
field product_defaults: ProductDefaultPluginModel = None#

The name of the product_default plugin this product inherits from. Doesn’t need to be a valid python identifier at the current time. Cannot be specified alongside the ‘family’ field.

field source_names: List[str] [Required]#

A list of strings representing the source(s) this product is derived from.Currently doesn’t have to be a valid python identifier as we have some cases that don’t adhere to that (such as amsu-a_mhs).

field spec: ProductSpec [Required]#

Arguments to be passed to the product plugin. Will override arguments provided by the product_defaults if applicable.

pydantic model geoips.pydantic_models.v1.products.SpecPlugin[source]#

Bases: FrozenModel

Model containing the name of the module plugin and the arguments to feed it.

Fields:
Validators:

field plugin: ModulePluginArgumentsModel [Required]#

The specification of the module plugin being overridden or implemented directly.

geoips.pydantic_models.v1.readers module#

Pydantic models used to validate GeoIPS OBP v1 reader plugins.

pydantic model geoips.pydantic_models.v1.readers.ReaderArgumentsModel[source]#

Bases: PermissiveFrozenModel

Reader step argument definition.

Pydantic model defining and validating Reader step arguments.

Fields:
Validators:

  • _handle_deprecated_chans » all fields

  • _validate_and_normalize_areadefinition » area_def

  • _validate_and_normalize_fnames » filenames

field area_def: AreaDefinition | None = None#

The domain over which to read data.

field chans: List[str] = None (alias 'variables')#

List of variables to read

field filenames: List[Path] = None#

full path to the file(s) for static dataset inputs.

field metadata_only: bool = False#

Read metadata only.

field resampled_read: bool = False#

Specify whether a resampled read is required, needed for datatypes that will be read within ‘get_alg_xarray’

field sectored_read: bool = False#
field self_register: str = None#

Enable self-registration.

field self_register_dataset: str = None#

Dataset within the source to use for self-registration

field self_register_source: str = None#

Source dataset to use for self-registration

geoips.pydantic_models.v1.sectors module#

Pydantic models used to validate GeoIPS sector plugins.

pydantic model geoips.pydantic_models.v1.sectors.AreaDefinitionSpec[source]#

Bases: FrozenModel

Defines an AreaDefinition for use with pyresample.

The resulting dictionary should be able to just be passed to pyresample.create_area_def().

Fields:
Validators:

  • _valdiate_and_convert_center » center

field area_extent: Tuple[float, float, float, float] | SectorAreaExtent = None#

Sector area extent in projection units. For more information see the pyresample documentation.

field area_id: str = None#

A name for the resulting pyresample AreaDefinition. Defaults to the sector’s name.

field center: XYCoordinate [Optional]#

The center of the sector in projection units. Defaults to (0.0, 0.0). See the pyresample documentation for more information.

field description: str = None#

A description for the resulting pyresample AreaDefinition. Defaults to the sector’s docstring.

field projection: SectorProjection [Required]#

A dictionary providing Proj projection information for the sector. For more information please see the Proj documentation.

field resolution: float | Tuple[float, float] | SectorResolution = None#

The size of the pixels in the sector in projection units. May be specified as a single float or a tuple of two floats describing the resolution in the x and y directions separately. See the pyresample documentation for more information.

field shape: Tuple[int, int] | SectorShape = None#
field units: Literal['m', 'km', 'meters', 'kilometers', 'deg', 'degrees'] = 'm'#

The units used for resolution and area_extent. This takes priority over the units specified in the projection. For more information on this parameter and its priority order, see the pyresample documentation.

pydantic model geoips.pydantic_models.v1.sectors.BoxMetadata[source]#

Bases: FrozenModel

Metadata format for pyroCb sectors.

Fields:
Validators:

field box_resolution_km: float [Required]#

The resolution of each pixel in kilometers.

Constraints:

  • strict = True

  • gt = 0

field max_lat: float [Required]#

Upper right latitude in degrees.

Constraints:

  • strict = True

  • ge = -90

  • le = 90

field max_lon: float [Required]#

Upper right longitude in degrees.

Constraints:

  • strict = True

  • ge = -180

  • le = 180

field min_lat: float [Required]#

Bottom left latitude in degrees.

Constraints:

  • strict = True

  • ge = -90

  • le = 90

field min_lon: float [Required]#

Bottom left longitude in degrees.

Constraints:

  • strict = True

  • ge = -180

  • le = 180

pydantic model geoips.pydantic_models.v1.sectors.DynamicSectorPluginModel[source]#

Bases: PluginModel

Dynamic sector plugin format.

Fields:
Validators:

field spec: DynamicSectorSpec [Required]#

A field demonstrating how to specify / format your dynamic sector plugin.

pydantic model geoips.pydantic_models.v1.sectors.DynamicSectorSpec[source]#

Bases: DynamicModel

The format of a dynamic sector’s ‘spec’ field.

Fields:
Validators:

field area_id: str = None#

A name for the resulting pyresample AreaDefinition. Defaults to the sector’s name.

field description: str = None#

A description for the resulting pyresample AreaDefinition. Defaults to the sector’s docstring.

field sector_spec_generator: SectorSpecGenerator [Required]#

A field containing the name of the sector_spec_generator to use and the arguments to provide to it.

class geoips.pydantic_models.v1.sectors.EarthConstants(value, names=None, *, module=None, qualname=None, type=None, start=1, boundary=None)[source]#

Bases: float, Enum

A class with Earth related geometrical constants.

SEMI_MAJOR_AXIS = 6371228.0#
pydantic model geoips.pydantic_models.v1.sectors.RegionMetadata[source]#

Bases: FrozenModel

Metadata format for standard static sectors.

Fields:
Validators:

field area: str [Required]#

Geographic area of the sector.

field city: str [Required]#

City which the sector resides in.

field continent: str [Required]#

Continent which the sector resides in.

field country: str [Required]#

Country which the sector resides in.

field state: str [Required]#

State which the sector resides in.

field subarea: str [Required]#

Geographic subarea of the sector.

pydantic model geoips.pydantic_models.v1.sectors.SectorAreaExtent[source]#

Bases: FrozenModel

The extent of the sector in projection units.

For more information on how this is used, see the pyresample documentation.

Fields:
Validators:

field lower_left_xy: Tuple[int, int] [Required]#

Lower left corner of the sector in projection units.

field upper_right_xy: Tuple[int, int] [Required]#

Upper right corner of the sector in projection units.

class geoips.pydantic_models.v1.sectors.SectorPluginModel(**data)[source]#

Bases: object

The format of a singular product plugin or a list of them.

pydantic model geoips.pydantic_models.v1.sectors.SectorProjection[source]#

Bases: PermissiveFrozenModel

Projection information for a sector.

This is a dictionary that provides Proj projection information for the sector. For more information on what parameters can be supplied, see the Proj documentation.

Validation has only been implemented for some of the most common options. Additional sector projection parameters are supported but not validated. If you need validation for a parameter that is not currently implemented, please open an issue and, if possible, a pull request on GitHub.

Fields:
Validators:

field R: float = None#

Radius of the sphere, given in meters. If used in conjunction with +ellps, +R takes precedence.See https://proj.org/en/stable/usage/ellipsoids.html#ellipsoid-size-parameters for more information.

Constraints:

  • strict = True

  • ge = 0

field a: EarthConstants = EarthConstants.SEMI_MAJOR_AXIS#

Semimajor axis of the ellipsoid in meters.

field ellipsoid: str = 'GRS80'#

The name of a built-in ellipsoid definition. See https://proj.org/en/stable/usage/ellipsoids.html#built-in-ellipsoid-definitions for more information, or execute proj -le for a list of built-in ellipsoid names. Defaults to ‘GRS80’.

field h: float = None#

Height of the view point above the Earth and must be in the same units as the radius of the sphere or semimajor axis of the ellipsoid.

Constraints:

  • strict = True

  • ge = 0

field k_0: float = 1.0#

Scale factor. Determines scale factor used in the projection. Defaults to 1.0.

Constraints:

  • strict = True

  • ge = 0

field lat_0: float = None#

Latitude of origin in degrees.

Constraints:

  • strict = True

  • ge = -90

  • le = 90

field lat_1: float = 0.0#

First standard parallel. Defaults to 0.0.

Constraints:

  • strict = True

  • ge = -90

  • le = 90

field lat_2: float = 0.0#

Second standard parallel. Defaults to 0.0.

Constraints:

  • strict = True

  • ge = -90

  • le = 90

field lat_ts: float = 0.0#

Latitude of true scale. Defines the latitude where scale is not distorted. Takes precedence over +k_0 if both options are used together. Defaults to 0.0.

Constraints:

  • strict = True

  • ge = 0

field lon_0: float = None#

Longitude of origin in degrees.

Constraints:

  • strict = True

  • ge = -180

  • le = 180

field proj: str [Required]#

Proj projection alias.

field t_epoch: float = None#

Central epoch of the transformation.

field t_final: float = None#

Final epoch that the coordinate will be propagated to after transformation. The special epoch now can be used instead of writing a specific period in time. When now is used, it is replaced internally with the epoch of the transformation. This means that the resulting coordinate will be slightly different if carried out again at a later date.

field units: Literal['m', 'km', 'degrees'] = 'm'#

Units of the projection. This should not need to be changed from the default. This controls the units of the x and y coordinates in the projection but has no impact on the units of the resolution or area_extent because we specify those units separately.

field x_0: float = 0.0#

False easting, easting at false origin or easting at projection centre (naming and meaning depend on the projection method). Always in meters.*Defaults to 0.0.*

field y_0: float = 0.0#

False northing, northing at false origin or northing at projection centre (naming and meaning depend on the projection method). Always in meters.*Defaults to 0.0.*

pydantic model geoips.pydantic_models.v1.sectors.SectorResolution[source]#

Bases: FrozenModel

The resolution of the sector in projection units.

The height and width of pixels in the units specified by the sector’s projection units.

Fields:
Validators:

field dx: float [Required]#

The width of pixels in the units specified by the sector’s projection units.

Constraints:

  • strict = True

  • gt = 0

field dy: float [Required]#

The height of pixels in the units specified by the sector’s projection units.

Constraints:

  • strict = True

  • gt = 0

pydantic model geoips.pydantic_models.v1.sectors.SectorShape[source]#

Bases: FrozenModel

The shape of the sector in pixels.

Fields:
Validators:

field height: int [Required]#

The height of the sector in pixels. Must be greater than 0.

Constraints:

  • strict = True

  • gt = 0

field width: int [Required]#

The width of the sector in pixels. Must be greater than 0.

Constraints:

  • strict = True

  • gt = 0

pydantic model geoips.pydantic_models.v1.sectors.SectorSpecGenerator[source]#

Bases: PermissiveDynamicModel

The format of the name and arguments for a sector spec generator plugin.

Fields:
Validators:

field arguments: dict [Required]#

A dictionary of arguments to provide to the sector_spec_generator plugin. If an empty dictionary is provided, the default arguments for that plugin will be used in place.

field name: str [Required]#

The name of the sector_spec_generator plugin to use.

Constraints:

  • func = <function python_identifier at 0x7f882bda4540>

pydantic model geoips.pydantic_models.v1.sectors.StaticMetadata[source]#

Bases: FrozenModel

Metadata format for standard static sectors.

This is the same as StaticMetadata, just with an additional ‘region’ level. This is a convenience model for specifying static sector plugins in a legacy format.

Fields:
Validators:

field region: RegionMetadata [Required]#

Additional field used to specify metadata in a legacy format.

pydantic model geoips.pydantic_models.v1.sectors.StaticSectorPluginModel[source]#

Bases: PluginModel

Static sector plugin format.

Fields:
Validators:
field metadata: BoxMetadata | StaticMetadata | StitchedMetadata | TCMetadata | VolcanoMetadata [Required]#

Metadata describing the sector (mostly used in FilenameFormatters and MetadataFormatters).

field spec: AreaDefinitionSpec [Required]#

Specification of the sector’s geographical domain. Used to generate a pyresample AreaDefinition.

pydantic model geoips.pydantic_models.v1.sectors.StitchedMetadata[source]#

Bases: StaticMetadata

Metadata for stitched imagery sectors.

Fields:
Validators:

field primary_area_definition: str [Required]#

Name of the area definition to be used.

pydantic model geoips.pydantic_models.v1.sectors.TCMetadata[source]#

Bases: FrozenModel

Metdata format for Tropical Cyclone sectors.

Fields:
Validators:

field aid_type: str [Required]#

Tropical cyclone forecast aid category.

field center_lat: float [Required]#

Center latitude of the storm.

Constraints:

  • strict = True

  • ge = -90

  • le = 90

field center_lon: float [Required]#

Center longitude of the storm.

Constraints:

  • strict = True

  • ge = -180

  • le = 180

field deck_line: str [Required]#

Deck data file used within the weather forecasting system for this storm.

field final_storm_name: str [Required]#

Final name of the storm used within the track file.

field pressure: float [Required]#

Pressure of the storm in millibars.

Constraints:

  • strict = True

  • gt = 0

field source_file: str [Required]#

File in which the storm data came from.

field storm_basin: str [Required]#

Two character representation of the basin in which the storm originated in.

field storm_name: str [Required]#

Name of the storm.

field storm_num: int [Required]#

Two digit storm number in sequential order from the start of the hurricane season.

Constraints:

  • strict = True

  • gt = 0

field storm_year: int [Required]#

Year in which the storm originated, after 1900.

Constraints:

  • strict = True

  • gt = 1900

field synoptic_time: datetime [Required]#

Synoptic time of the storm.

field velocity_max: float [Required]#

Maximum velocity in knots of the storm.

Constraints:

  • strict = True

  • gt = 0

pydantic model geoips.pydantic_models.v1.sectors.VolcanoMetadata[source]#

Bases: FrozenModel

Metadata format for Volcano sectors.

Fields:
Validators:

field clat: float [Required]#

Center latitude of the volcano sector.

Constraints:

  • strict = True

  • ge = -90

  • le = 90

field clon: float [Required]#

Center longitude of the volcano sector.

Constraints:

  • strict = True

  • ge = -180

  • le = 180

field plume_height: float [Required]#

Altitude of the volcanic plume.

field summit_elevation: float [Required]#

Elevation of the volcano’s summit.

field wind_dir: float [Required]#

Angular direction of wind within the volcano sector.

Constraints:

  • strict = True

  • ge = 0

  • le = 360

field wind_speed: float [Required]#

Windspeed of the volcano sector.

Constraints:

  • strict = True

  • gt = 0

pydantic model geoips.pydantic_models.v1.sectors.XYCoordinate[source]#

Bases: FrozenModel

A coordinate in projection units.

Fields:
Validators:

field x: float [Required]#

The x coordinate in projection units.

field y: float [Required]#

The y coordinate in projection units.

geoips.pydantic_models.v1.sectors.validate_lat_lon_coordinate(arg: tuple[float, float]) → tuple[float, float][source]#

Validate a latitude and longitude coordinate.

geoips.pydantic_models.v1.title_formatters module#

Pydantic models used to validate GeoIPS OBP v1 title-formatter plugins.

pydantic model geoips.pydantic_models.v1.title_formatters.TitleFormatterArgumentsModel[source]#

Bases: PermissiveFrozenModel

Title-Formatter step argument definition.

Pydantic model defining and validating Title Formatter step arguments.

Fields:
Validators:

field area_def: str = None#

Area definition identifier.

field bg_datatype_title: str = None#

Background data type label for the background product title.

field bg_product_name_title: str = None#

Background product name title when background layer is provided.

field product_datatype_title: str = None#

Product data type label to include in the title.

field product_name_title: str = None#

Product name title.

Copyright string to append to the generated title

geoips.pydantic_models.v1.workflows module#

Workflow plugin models.

Defines pydantic models related to Workflow plugins, including top-level callable interfaces (eg. Readers, OutputFormatters, etc.).

pydantic model geoips.pydantic_models.v1.workflows.AlgorithmStepValidationModel[source]#

Bases: PermissiveFrozenModel

Validate step-level requirements for algorithm plugins.

Validators:

  • _variables_required_algorithm_plugins » all fields

pydantic model geoips.pydantic_models.v1.workflows.ColormapperArgumentsModel[source]#

Bases: PermissiveFrozenModel

Validate Colormapper arguments.

Validators:

geoips.pydantic_models.v1.workflows.DEFAULT_RETENTION = 'keep_referenced'#

Default retention policy when retention is not specified in the YAML.

pydantic model geoips.pydantic_models.v1.workflows.FeatureAnnotatorArgumentsModel[source]#

Bases: PermissiveFrozenModel

Validate Feature Annotator arguments (YAML plugin).

Validators:

pydantic model geoips.pydantic_models.v1.workflows.GlobalVariablesModel[source]#

Bases: PermissiveFrozenModel

Workflow-level global variables shared across all steps.

Carries fields that apply uniformly to every step of an Order-Based Procflow workflow rather than belonging to a single step’s arguments. (e.g. temporal windowing, product identification, product DB output configuration, and the presectoring toggle)

Fields:
Validators:

  • _validate_product_db_requires_writer » all fields

  • _validate_window_start_requires_end » all fields

field minimum_coverage: float | str = 'plugin_provided'#
field presector: bool = False#

Specify whether to presector the data prior to applying the algorithm

field product_db: bool = False#
field product_db_writer: str | None = None#
field product_db_writer_kwargs: Dict[str, Any] | None = None#
field product_name: str | None = None#
field reader_defined_area_def: bool = False#
field sector_list: List[str] | None = None#
field window_end_time: dt.datetime | None = None#

If specified, sector temporally between window_start_time and window_end_time.

field window_start_time: dt.datetime | None = None#

If specified, sector temporally between window_start_time and window_end_time.

pydantic model geoips.pydantic_models.v1.workflows.GridlineAnnotatorArgumentsModel[source]#

Bases: PermissiveFrozenModel

Validate Gridline Annotator arguments (YAML plugin).

Validators:

geoips.pydantic_models.v1.workflows.INPUT_REF = '_input'#

Magic depends_on token marking a workflow’s data-injection entry step.

A step whose depends_on contains _input receives the data injected into the workflow from the outside: the parent’s upstream tree for a sub-workflow (or split branch), or an empty DataTree for a top-level workflow. It is a virtual source (not a real step), so it is skipped by dependency-reference validation, cycle detection, and topological ordering. If no step declares _input, the first step receives the injected data (backward-compatible fallback).

pydantic model geoips.pydantic_models.v1.workflows.NestedSpecOverride[source]#

Bases: PermissiveFrozenModel

Spec definition allowing for recursive overrides.

Fields:
Validators:

field steps: Dict[str, StepOverrideType] [Optional]#
pydantic model geoips.pydantic_models.v1.workflows.OutputCheckerOverrideModel[source]#

Bases: PermissiveFrozenModel

Model for output checker step definitions / overrides in a workflow test / steps section. # NOQA

Takes the form of:

output_checker:

name: my_oc full_test_policy: “on_token_mismatch” | “always” | “never” arguments: …

Fields:
Validators:

field compare_path: FilePath | str | None = None#

The path to the comparison file.

field full_test_policy: Literal['on_token_mismatch', 'always', 'never'] = 'on_token_mismatch'#

Tells GeoIPS in what circumstances an output checker should run based on the result of the token comparison. Defaults to only running the specified (or detected) output checker on failed token comparison.

field name: str | None = None (alias 'output_checker_name')#

The name of the output checker plugin to use. If None, use a default output checker plugin associated with the produced file type(s).

field threshold: float | None = None (alias 'output_checker_threshold')#

Threshold for the image comparison. Argument to pixelmatch. Between 0 and 1, with 0 the most strict comparison, and 1 the most lenient.

pydantic model geoips.pydantic_models.v1.workflows.OutputFormatterArgumentsModel[source]#

Bases: PermissiveFrozenModel

Validate Output Formatter arguments.

Validators:

pydantic model geoips.pydantic_models.v1.workflows.ProductArgumentsModel[source]#

Bases: PermissiveFrozenModel

Validate product arguments.

Validators:

pydantic model geoips.pydantic_models.v1.workflows.ProductDefaultArgumentsModel[source]#

Bases: PermissiveFrozenModel

Validate product default arguments.

Validators:

geoips.pydantic_models.v1.workflows.SCAFFOLD_KINDS = frozenset({'join', 'split'})#

Kinds reserved as scaffolding markers.

Steps with these kinds are accepted by schema validation and executed by workflow orchestration rather than plugin resolution.

pydantic model geoips.pydantic_models.v1.workflows.SectorArgumentsModel[source]#

Bases: PermissiveFrozenModel

Validate Sector arguments (YAML plugin).

Validators:

pydantic model geoips.pydantic_models.v1.workflows.StepOverrideType[source]#

Bases: PermissiveFrozenModel

A workflow step override.

Either: - arbitrary arguments OR - a nested spec containing additional steps

Fields:
Validators:

field spec: NestedSpecOverride | None = None#
pydantic model geoips.pydantic_models.v1.workflows.WorkflowArgumentsModel[source]#

Bases: PermissiveFrozenModel

Validate Workflow arguments.

Validators:

pydantic model geoips.pydantic_models.v1.workflows.WorkflowPluginModel[source]#

Bases: PluginModel

A plugin that produces a workflow.

Fields:
Validators:

  • _validate_interface » interface

  • _validate_one_line_description » description

  • propagate_context » all fields

field abspath: str = None#

Absolute path to the plugin file.

field description: str = None#

A short description or defaults to first line from docstring.

field docstring: str [Required]#

Docstring for the plugin in numpy format.

field family: PythonIdentifier [Required]#

Family of the plugin.

Constraints:

  • func = <function python_identifier at 0x7f882bda4540>

field interface: PythonIdentifier [Required]#

Name of the plugin’s interface. Run geoips list interfaces to see available options.

Constraints:

  • func = <function python_identifier at 0x7f882bda4540>

field is_registered: bool = True#

Whether or not this plugin is registered.

field name: str [Required]#

Plugin name.

field package: PythonIdentifier = (FieldInfo(annotation=NoneType, required=True, description='Package that contains this plugin.'),)#
Constraints:

  • func = <function python_identifier at 0x7f882bda4540>

field relpath: str = None#

Path to the plugin file relative to its parent package.

field spec: WorkflowSpecModel [Required]#

The workflow specification

field test: WorkflowTestModel = None#

An optional dictionary of parameters used to test this workflow.

pydantic model geoips.pydantic_models.v1.workflows.WorkflowSpecModel[source]#

Bases: FrozenModel

The specification for a workflow.

Fields:
Validators:

  • _inject_defaults » all fields

  • _reject_defaults » all fields

  • _reject_retention_by_kind » all fields

  • _validate_dependencies » all fields

  • expand_steps » all fields

field defaults: Dict[str, Dict[str, Any]] | None = None#

Per-kind argument defaults applied to every step of that kind. Keys are plugin kind names. Step-level arguments override these.

field globals: GlobalVariablesModel | None = None#

Arguments shared across workflow steps

field retention: Literal['keep_all', 'keep_referenced'] | None = 'keep_referenced'#

Workflow-level data retention policy. - keep_all: never GC any step data. - keep_referenced: GC a step’s data when no remaining downstream step references it.

field retention_by_kind: Dict[str, Literal['keep_all', 'keep_referenced']] | None = None#

Per-kind retention overrides. Keys are plugin kind names (e.g. ‘reader’, ‘algorithm’). If None, no per-kind override. This is a v2 feature; field exists but is not wired in v1.

field steps: Dict[PythonIdentifier, WorkflowStepDefinitionModel] [Required]#

Steps to produce the workflow.

classmethod expand_step(step: dict, info: ValidationInfo, _inputs: list[str] | None = None) → dict[dict][source]#

Expand the definition of this step if it is a select plugin type.

Plugin types this function will expand include [‘products’, ‘product_defaults’, ‘workflows’].

This function will fully expand this step if it is one of the mentioned types before any validation occurs.

Parameters:

  • step (dict) – A dictionary representation of a workflow step.

  • info (ValidationInfo) – An object representing the context in which this model was instantiated.

  • _inputs (list[str], optional) – One or more step ids that are providing input data to this step. Optional. If None, assume no input data is available or needed.

Returns:

steps –

  • An ordered dictionary representing the expanded version of the input step.

Return type:

dict[dict]

classmethod extend_dict(base: dict, new: dict) → dict[source]#

Extend a dictionary with the contents of another dictionary.

Do this extension while avoiding key collisions by automatically renaming conflicting keys.

Keys from new are added to base. If a key from new already exists in base, a numeric suffix is appended to the key name (e.g., key1, key2, etc.) until a unique key is found. The original dictionaries are not modified.

Parameters:

  • base (dict) –

    • The original dictionary whose contents will be preserved. Keys from new will be added to a copy of this dictionary.

  • new (dict) –

    • The dictionary whose key-value pairs will be added to base. If a key already exists in base (or was added earlier during the merge), it will be renamed with an incrementing numeric suffix to ensure uniqueness.

Returns:

  • A new dictionary containing all key-value pairs from base and new. Any conflicting keys from new will be renamed with a numeric suffix (key1, key2, etc.) so that no keys are overwritten.

Return type:

dict

classmethod product_to_steps(plugin: dict, _inputs: list[str] | None = None) → tuple[dict[dict], dict][source]#

Define a product or product default plugin as a series of workflow steps.

Parameters:

  • plugin (dict) – A dictionary representation of a product or product default plugin.

  • _inputs (list[str], optional) – One or more step ids that are providing input data to this step. Optional. If None, assume no input data is available or needed.

Returns:

  • steps (dict[dict]) –

    • An ordered dictionary representing the expanded version of the input plugin.

  • global_vars (dict) –

    • A dictionary of global variables found in this plugin.

pydantic model geoips.pydantic_models.v1.workflows.WorkflowStepDefinitionModel[source]#

Bases: FrozenModel

Validate step definition : kind, name, and arguments.

Fields:
Validators:

  • _ensure_xor_name_spec » all fields

  • _reject_scope » scope

  • _reject_when » when

  • _validate_interpolator_step_depends_on_two » all fields

  • _validate_plugin_arguments » all fields

  • _validate_plugin_kind » kind

  • _validate_plugin_name » all fields

field arguments: Dict[str, Any] | None [Optional]#

step args

field depends_on: List[StepReference] | None = None#

Step references this step depends on. Each reference is either a top-level step id (e.g. ‘reader’) or a dot-separated path into a ‘workflow’/’split’ container step (e.g. ‘subwf.algo’ or ‘split.scope.algo’), or the magic token ‘_input’ marking this step as the workflow’s data-injection entry point (parent data for a sub-workflow/branch, or an empty DataTree at top level). ‘_input’ may be combined with real references and may appear on any number of steps (fan-out). If None, defaults to [previous_step_id] for non-first steps, [] for the first step; when no step declares ‘_input’ the first step receives the injected data. The runner validates all references exist (recursing into sub-workflows) and that no cycles exist.

field full_test_policy: Literal['on_token_mismatch', 'always', 'never', None] = None#

Tells GeoIPS in what circumstances an output checker should run based on the result of the token comparison. Defaults to only running the specified (or detected) output checker on failed token comparison.Should only EVER exist for an output checker step.

field keep: bool = False#

If True, this step’s output data dataset survives garbage collection regardless of the workflow-level retention policy. Metadata (attrs, tokens) always survive.

field kind: Lexeme [Required]#

plugin kind

field name: str | tuple[str] | None = None#

Plugin name. Required for plugin-backed steps, but not for scaffold steps such as split or join or steps with kind: workflow that supply an inline spec.

field scope: str | None = None#

For steps following a split: which branch to operate on. When set, this step’s output node nests at /<split_id>/<scope>/<step_id>. Explicit step-level scope routing is not yet implemented.

field spec: WorkflowSpecModel | None = None#

The workflow specification

field when: str | None = None#

Conditional expression. If set and evaluates to false, this step is skipped. Expressions are pandas-style filter expressions. Runtime evaluation is not yet implemented.

pydantic model geoips.pydantic_models.v1.workflows.WorkflowTestModel[source]#

Bases: FrozenModel

Model for the test section of GeoIPS workflow plugins.

Fields:
Validators:

  • generate_filepaths » filenames

  • validate_kind_keys » all fields

field filenames: List[str] [Required]#

A list of one or more filepaths to the data used for this test.

field globals: Dict[str, Any] [Optional]#

Override dictionary for global arguments.

field kinds: Dict[str, Dict[str, Any]] [Optional]#

Override dictionary for plugins matching a certain ‘kind’.

field outputs: Dict[str, OutputCheckerOverrideModel] [Optional]#

Override dictionary for output checker steps or every instance of an output checker.

field steps: Dict[str, StepOverrideType] [Optional]#

Override dictionary for individual steps.

geoips.pydantic_models.v1.workflows.get_plugin_kinds() → set[str][source]#

Return plugin kinds from available interfaces.

Returns:

singular names of distinct plugin kinds

Return type:

set of str

geoips.pydantic_models.v1.workflows.get_plugin_names(plugin_kind: str) → List[str][source]#

Return valid plugin names for passed plugin kind.

Parameters:

plugin_kind (str) – valid plugin interface name

Returns:

A list of plugin names for a valid plugin kind

Return type:

list

Raises:

AttributeError – If the plugin kind is invalid

Module contents#

GeoIPS order-based procflow v1 models init file.

geoips.pydantic_models.v1.collect_classes(modules)[source]#

Extract all classes from the given modules.

Parameters:

modules (dict) –

  • A dictionary of mod_name: module objects found within geoips.pydantic

geoips.pydantic_models.v1.collect_modules()[source]#

Dynamically find and import all submodules within a package.

On this page
Show Source