geoips.plugins.modules.filename_formatters package#

Subpackages#

Submodules#

geoips.plugins.modules.filename_formatters.basic_fname module#

Filename specification using minimal basic attributes, and no subdirs.

geoips.plugins.modules.filename_formatters.basic_fname.call(xarray_obj, area_def, product_name, output_type='.png', basedir='/users/surratt/geoips/outdirs/preprocessed/annotated_imagery', extra_field=None)[source]#

Create basic filename, in a specific directory (with no subdirectories).

This filename format includes only the start time, platform name, source name, product name, sector name, and data provider, and a full path to basedir with no additional subdirectories.

geoips.plugins.modules.filename_formatters.geoips_fname module#

Standard geoips filename production.

geoips.plugins.modules.filename_formatters.geoips_fname.assemble_geoips_fname(basedir, product_name, source_name, platform_name, sector_name, coverage, resolution, product_datetime, output_type='png', data_provider=None, extra=None, product_dir=None, source_dir=None, continent=None, country=None, area=None, subarea=None, state=None, city=None)[source]#

Produce full output product path from product / sensor specifications.

standard web paths are of the format:

<basedir>/<continent>-<country>-<area>/<subarea>-<state>-<city>/ <productname>/<sensorname>``

standard filenames are of the format:

<date{%Y%m%d}>.<time{%H%M%S}>.<satname>.<sensorname>.<productname>. <sectorname>.<coverage>.<dataprovider>.<extra>

Parameters:

  • basedir (str) – Full path to base directory of final product.

  • product_name (str) – Name of product

  • source_name (str) – Name of data source (sensor)

  • platform_name (str) – Name of platform (satellite)

  • coverage (float) – Image coverage, float between 0.0 and 100.0

  • resolution (float) – Image resolution, float greater than 0.0

  • product_datetime (datetime.datetime) – Datetime object - start time of data used to generate product.

  • output_type (str, optional) – file extension type, default is png

  • data_provider (str, optional) – String to include in filename “data_provider” field

  • extra (str, optional) – String to include in filename “extra” field, default is None If None, use fillval of ‘x’

  • continent (str, optional) – String to include in filename “continent” field, default is None If None, use fillval of ‘x’

  • country (str, optional) – String to include in filename “country” field, default is None If None, use fillval of ‘x’

  • area (str, optional) – String to include in filename “area” field, default is None If None, use fillval of ‘x’

  • subarea (str, optional) – String to include in filename “subarea” field, default is None If None, use fillval of ‘x’

  • state (str, optional) – String to include in filename “state” field, default is None If None, use fillval of ‘x’

  • city (str, optional) – String to include in filename “city” field, default is None If None, use fillval of ‘x’

geoips.plugins.modules.filename_formatters.geoips_fname.call(area_def, xarray_obj, product_name, coverage=None, output_type='png', output_type_dir=None, product_dir=None, product_subdir=None, source_dir=None, basedir='/users/surratt/geoips/outdirs/preprocessed/annotated_imagery')[source]#

Create GeoIPS standard filenames, sector-based subdirs.

This uses the sector specification (continent, country, area, subarea, state, city), product name, source name, and platform name to generate a full unique path, as well as additional attributes to create a fully unique file name.

geoips.plugins.modules.filename_formatters.geoips_fname.geoips_fname_remove_duplicates(fname, mins_to_remove=10, remove_files=False)[source]#

remove_duplicates function currently not defined.

If defined, this function will identify duplicate files, and remove appropriately. It should identify duplicates specifically based on the format/location of the geoips_fname formatted files.

Currently returns empty lists. When defined, will return a list of deleted files and a list of saved files.

geoips.plugins.modules.filename_formatters.geoips_netcdf_fname module#

Standard GeoIPS NetCDF filename production.

geoips.plugins.modules.filename_formatters.geoips_netcdf_fname.assemble_geoips_netcdf_fname(basedir, product_name, source_name=None, platform_name=None, sector_name=None, product_datetime=None, set_subpath=None, time_format='%H%M%S')[source]#

Produce full output product path from product / sensor specifications.

netcdf paths are of the format:
<basedir>/<product_name>/<source_name>/<platform_name>/

<sector_name>/date{%Y%m%d}

netcdf filenames are of the format:
<date{%Y%m%d>.<time{%H%M%S}>.<platform_name>.<product_name>.

<sector_name>.nc

Parameters:

  • basedir (str) – Base directory (additional subdirectories assembled below basedir)

  • product_name (str) – Name of product, used in path and filename

  • source_name (str) – Name of data source (sensor), used in path and filename

  • platform_name (str) – Name of platform (satellite), used in path and filename

  • coverage (float) – Image coverage, float between 0.0 and 100.0, used in filename

  • product_datetime (datetime) – Datetime object - start time of data used to generate product, used in filename

geoips.plugins.modules.filename_formatters.geoips_netcdf_fname.call(area_def, xarray_obj, product_names, coverage=None, output_type='nc', output_type_dir=None, product_dir=None, product_subdir=None, source_dir=None, basedir=None)[source]#

Filename formatting for standard GeoIPS-style NetCDF outputs.

This uses the “assemble_geoips_netcdf_fname” function to appropriately assemble the filename from a base directory, product name, source name, platform nae, sector name, and product time, to allow reuse of this basic filename format from multiple filename formatter plugins.

geoips.plugins.modules.filename_formatters.geotiff_fname module#

Standard GeoIPS GEOTIFF filename formatter.

geoips.plugins.modules.filename_formatters.geotiff_fname.call(area_def, xarray_obj, product_name, coverage=None, output_type='tif', output_type_dir=None, product_dir=None, product_subdir=None, source_dir=None, basedir='/users/surratt/geoips/outdirs/preprocessed/annotated_imagery', output_dict=None)[source]#

GEOTIFF filename formatter.

This uses the standard “geoips_fname” formatter, but with a default output type of “tif”.

geoips.plugins.modules.filename_formatters.metadata_default_fname module#

Default TC metadata filename formatter.

geoips.plugins.modules.filename_formatters.metadata_default_fname.call(area_def, xarray_obj, product_filename, metadata_dir='metadata', metadata_type='sector_information', basedir='/users/surratt/geoips/outdirs/preprocessed/tcwww', output_dict=None)[source]#

Generate TC metadata filenames.

This uses attributes on both the xarray and area_def in order to produce the YAML metadata output specifically for TC sectors. Not valid for other sector types.

This uses the “tc_storm_basedir” utility to ensure a consistent path to the storm directory (so products and metadata end up in the same location)

geoips.plugins.modules.filename_formatters.tc_clean_fname module#

Clean TC filename production (no backgrounds or overlays).

geoips.plugins.modules.filename_formatters.tc_clean_fname.call(area_def, xarray_obj, product_name, coverage=None, output_type='png', output_type_dir=None, product_dir=None, product_subdir=None, source_dir=None, basedir='/users/surratt/geoips/outdirs/preprocessed/tcwww', output_dict=None)[source]#

Clean TC product filename formatter (no gridlines, titles, etc).

This ensures output ends up in “png_clean” directory, with “-clean” appended to the extra field, to avoid conflict with tc_fname based annotated imagery. Uses “tc_fname” module as a base.

Parameters:

  • area_def (pyresample AreaDefinition) – Contains metadata regarding sector

  • xarray_obj (xarray Dataset) – Contains metadata regarding dataset

  • product_name (str) – String product_name specification for use in filename

  • coverage (float) – Percent coverage, for use in filename

  • output_type (str, optional) – Requested output format, ie png, jpg, tif, etc, defaults to None.

  • output_type_dir (str, optional) – Directory name for given output type (ie png_clean, png, etc), defaults to None.

  • product_dir (str, optional) – Directory name for given product, defaults to None.

  • product_subdir (str, optional) – Subdir name for given product, if any, defaults to None.

  • source_dir (str, optional) – Directory name for given source, defaults to None.

  • basedir (str, optional) – Base directory, defaults to $TCWWW.

Returns:

Full path to output “clean” filename - with “-clean” appended to extra field, and “_clean” appended to output_type_dir.

Return type:

str

geoips.plugins.modules.filename_formatters.tc_fname module#

Standard TC filename formatter.

geoips.plugins.modules.filename_formatters.tc_fname.assemble_tc_fname(basedir, tc_year, tc_basin, tc_stormnum, output_type, product_name, source_name, platform_name, coverage, product_datetime, intensity=None, extra=None, output_type_dir=None, product_dir=None, product_subdir=None, output_dict=None, sector_info=None)[source]#

Produce full output product path from product / sensor specifications.

tc web paths are of the format:
<basedir>/tc<tc_year>/<tc_basin>/<tc_basin><tc_stormnum><tc_year>/

<output_type>/<product_name>/<platform_name>/

tc web filenames are of the format:
<date{%Y%m%d%H%M>_<tc_basin><tc_stormnum><tc_year>_<source_name>_

<platform_name>_<product_name>_<intensity>_<coverage>_ <extra>.<output_type>

Parameters:

  • basedir (str) – Base directory for output file.

  • tc_year (int) – Full 4 digit storm year

  • tc_basin (str) –

    2 character basin designation

    SH Southern Hemisphere WP West Pacific EP East Pacific CP Central Pacific IO Indian Ocean AL Atlantic

  • tc_stormnum (int) –

    2 digit storm number

    90 through 99 for invests 01 through 69 for named storms

  • output_type (str) – file extension type

  • product_name (str) – Name of product

  • source_name (str) – Name of data source (sensor)

  • platform_name (str) – Name of platform (satellite)

  • coverage (float) – Image coverage, float between 0.0 and 100.0

  • product_datetime (datetime.datetime) – Datetime object - start time of data used to generate product

  • output_type_dir (str) – Default output_type, dir name

  • product_dir (str) – Default product_name, dir name

Returns:

full path to output file

Return type:

str

geoips.plugins.modules.filename_formatters.tc_fname.call(area_def, xarray_obj, product_name, coverage=None, output_type='png', output_type_dir=None, product_dir=None, product_subdir=None, source_dir=None, basedir='/users/surratt/geoips/outdirs/preprocessed/tcwww', extra_field=None, output_dict=None)[source]#

Create standard TC filenames.

See also

geoips.plugins.modules.filename_formatters.tc_fname.assemble_tc_fname

This uses the shared utility “assemble_tc_fname”, such that a common filename can be used by related filename formatters.

geoips.plugins.modules.filename_formatters.tc_fname.tc_fname_remove_duplicates(fname, mins_to_remove=3, remove_files=False)[source]#

Remove tc_fname duplicate files.

Matches storm name, sensor name, platform name, product name, and resolution for all files within “mins_to_remove” minutes of the current file. All other fields are wild carded during the matching process.

geoips.plugins.modules.filename_formatters.text_winds_day_fname module#

Filename formatter for full-day text windspeed products.

geoips.plugins.modules.filename_formatters.text_winds_day_fname.call(xarray_obj, extension='.txt', basedir='/users/surratt/geoips/outdirs/preprocessed/annotated_imagery/text_winds')[source]#

Create full-day text windspeed filenames.

text_winds_day_fname includes only YYYYMMDD in the filename, so all data for a full day is appended into a single file.

See also

geoips.plugins.modules.filename_formatters.text_winds_full_fname.

assemble_text_windspeeds_text_full_fname Shared utility for generating similarly formatted windspeed filenames.

geoips.plugins.modules.filename_formatters.text_winds_full_fname module#

Filename formatter for text windspeed products.

geoips.plugins.modules.filename_formatters.text_winds_full_fname.assemble_windspeeds_text_full_fname(basedir, source_name, platform_name, data_provider, product_datetime, dt_format='%Y%m%d.%H%M', extension='.txt', creation_time=None)[source]#

Produce full output product path using product / sensor specifications.

Parameters:

  • basedir (str) – base directory

  • source_name (str) – Name of source (sensor)

  • platform_name (str) – Name of platform (satellite)

  • data_provider (str) – Name of data provider

  • product_datetime (datetime.datetime) – Start time of data used to generate product

  • dt_format (str, default="%Y%m%d.%H%M") – Format used to display product_datetime within filename

  • extension (str, default=".txt") – File extension, specifying type.

  • creation_time (datetime.datetime, default=None) – Include given creation_time of file in filename If None, do not include creation time.

Returns:

full path of output filename of the format:

<basedir>/<source_name>_<data_provider>_<platform_name>_ surface_winds_<YYYYMMDDHHMN>

Return type:

str

Examples

>>> startdt = datetime.strptime('20200216T001412', '%Y%m%dT%H%M%S')
>>> assemble_windspeeds_text_full_fname(
...     '/outdir',
...     'smap-spd',
...     'smap',
...     'remss',
...     startdt,
...     '%Y%m%d'
...     )
'/outdir/smap-spd_remss_smap_surface_winds_20200216'
geoips.plugins.modules.filename_formatters.text_winds_full_fname.call(xarray_obj, extension='.txt', basedir='/users/surratt/geoips/outdirs/preprocessed/annotated_imagery/text_winds')[source]#

Create a single text winds file for all data in the current xarray.

This text windspeed filename includes YYYYMMDD.HHMN in the filename in order to include only the current datafile in the file.

See also

geoips.plugins.modules.filename_formatters.text_winds_full_fname.

assemble_windspeeds_text_full_fname Shared utility to create filenames with similar formatting requirements.

geoips.plugins.modules.filename_formatters.text_winds_tc_fname module#

Filename formatter for TC-specific text windspeed outputs.

geoips.plugins.modules.filename_formatters.text_winds_tc_fname.assemble_windspeeds_text_tc_fname(basedir, tc_area_id, tc_year, tc_basin, tc_stormnum, source_name, platform_name, product_datetime, data_provider, extension='.txt', output_dict=None, sector_info=None)[source]#

Produce full output product path from product / sensor specifications.

Parameters:

  • basedir (str) – base directory

  • tc_year (int) – Full 4 digit storm year

  • tc_basin (str) –

    2 character basin designation

    SH Southern Hemisphere WP West Pacific EP East Pacific CP Central Pacific IO Indian Ocean AL Atlantic

  • tc_stormnum (int) –

    2 digit storm number

    90 through 99 for invests 01 through 69 for named storms

  • platform_name (str) – Name of platform (satellite)

  • product_datetime (datetime) – Start time of data used to generate product

Returns:

full path of output filename of the format:
<basedir>/tc<tc_year>/<tc_basin>/<tc_basin><tc_stormnum><tc_year>/

txt/<source_name>_<platform_name>_surface_winds_<data_provider>_ <YYYYMMDDHHMN>

Return type:

str

Examples

>>> startdt = datetime.strptime('20200216T001412', '%Y%m%dT%H%M%S')
>>> assemble_windspeeds_text_tc_fname('/outdir',
...     2020,
...     'SH',
...     16,
...     'smap-spd',
...     'smap',
...     startdt,
...     'remss')
'/outdir/tc2020/SH/SH162020/txt/
geoips.plugins.modules.filename_formatters.text_winds_tc_fname.call(xarray_obj, extension='.txt', basedir='/users/surratt/geoips/outdirs/preprocessed/tcwww', output_dict=None)[source]#

Create TC-specific text windspeed filename.

See also

geoips.plugins.modules.filename_formatters.text_winds_tc_fname.

assemble_windspeeds_text_tc_fname Shared utility to facilitate creating multiple similar filenames from the same code.

Module contents#

geoips filename_formatters init file.