geoips.plugins.modules.readers package#

Subpackages#

Submodules#

geoips.plugins.modules.readers.abi_l2_netcdf module#

ABI Level 2 NetCDF reader.

geoips.plugins.modules.readers.abi_l2_netcdf.calculate_abi_geolocation(metadata, area_def)[source]#

Calculate ABI geolocation.

geoips.plugins.modules.readers.abi_l2_netcdf.call(fnames, area_def=None, metadata_only=False, chans=False, self_register=False)[source]#

Read ABI Level 2 NetCDF data from a list of filenames.

Parameters:

• fnames (list) –

  • List of strings, full paths to files

• metadata_only (bool, default=False) –

  • Return before actually reading data if True

• chans (list of str, default=None) –

  • List of desired channels (skip unneeded variables as needed).

  • Include all channels if None.

• area_def (pyresample.AreaDefinition, default=None) –

  • Specify region to read

  • Read all data if None.

• self_register (str or bool, default=False) –

  • register all data to the specified dataset id (as specified in the return dictionary keys).

  • Read multiple resolutions of data if False.

Returns:

• dictionary of xarray.Dataset objects with required Variables and Attributes.

• Dictionary keys can be any descriptive dataset ids.

Return type:

dict of xarray.Datasets

See also

Xarray and NetCDF Metadata Standards

Additional information regarding required attributes and variables for GeoIPS-formatted xarray Datasets.

geoips.plugins.modules.readers.abi_l2_netcdf.get_metadata(fname)[source]#

Get metadata.

geoips.plugins.modules.readers.abi_l2_netcdf.satpy_read(fname, chans)[source]#

Ingest data using satpy.Scene.

satpy.Scene object converted to xarray dataset at the end

Parameters:

• fname (str) – Full file path

• chans (list, optional) – List of variables to load from file, by default None All variables are loaded in None

Returns:

Dataset holding either requested or all variables from file

Return type:

xarray.dataset

geoips.plugins.modules.readers.abi_l2_netcdf.xr_read(fname, chans=None)[source]#

Ingest data using xarray.open_dataset.

Parameters:

• fname (str) – Full file path

• chans (list, optional) – List of variables to load from file, by default None All variables are loaded in None

Returns:

Dataset holding either requested or all variables from file.

Return type:

xarray.dataset

geoips.plugins.modules.readers.abi_netcdf module#

Standard GeoIPS xarray dictionary based ABI NetCDF data reader.

geoips.plugins.modules.readers.abi_netcdf.call(fnames, metadata_only=False, chans=None, area_def=None, self_register=False)[source]#

Read ABI NetCDF data from a list of filenames.

Parameters:

• fnames (list) –

  • List of strings, full paths to files

• metadata_only (bool, default=False) –

  • Return before actually reading data if True

• chans (list of str, default=None) –

  • List of desired channels (skip unneeded variables as needed).

  • Include all channels if None.

• area_def (pyresample.AreaDefinition, default=None) –

  • Specify region to read

  • Read all data if None.

• self_register (str or bool, default=False) –

  • register all data to the specified dataset id (as specified in the return dictionary keys).

  • Read multiple resolutions of data if False.

Returns:

• dictionary of xarray.Dataset objects with required Variables and Attributes.

• Dictionary keys can be any descriptive dataset ids.

Return type:

dict of xarray.Datasets

See also

Xarray and NetCDF Metadata Standards

Additional information regarding required attributes and variables for GeoIPS-formatted xarray Datasets.

geoips.plugins.modules.readers.abi_netcdf.call_single_time(fnames, metadata_only=False, chans=None, area_def=None, self_register=False)[source]#

Read ABI NetCDF data from a list of filenames.

Parameters:

• fnames (list) –

  • List of strings, full paths to files

• metadata_only (bool, default=False) –

  • Return before actually reading data if True

• chans (list of str, default=None) –

  • List of desired channels (skip unneeded variables as needed).

  • Include all channels if None.

• area_def (pyresample.AreaDefinition, default=None) –

  • Specify region to read

  • Read all data if None.

• self_register (str or bool, default=False) –

  • register all data to the specified dataset id (as specified in the return dictionary keys).

  • Read multiple resolutions of data if False.

Returns:

• dictionary of xarray.Dataset objects with required Variables and Attributes.

• Dictionary keys can be any descriptive dataset ids.

Return type:

dict of xarray.Datasets

See also

Xarray and NetCDF Metadata Standards

Additional information regarding required attributes and variables for GeoIPS-formatted xarray Datasets.

geoips.plugins.modules.readers.abi_netcdf.get_band_metadata(all_metadata)[source]#

Get band metadata.

This method basically just reformats the all_metadata dictionary that is set based on the metadata found in the netcdf object itself to reference channel names as opposed to filenames as the dictionary keys.

geoips.plugins.modules.readers.abi_netcdf.get_data(md, gvars, rad=False, ref=False, bt=False)[source]#

Read data for a full channel’s worth of files.

geoips.plugins.modules.readers.abi_netcdf.get_latitude_longitude(metadata, BADVALS, sect=None)[source]#

Get latitudes and longitudes.

This routine accepts a dictionary containing metadata as read from a NCDF4 format file, and returns latitudes and longitudes for a full disk.

geoips.plugins.modules.readers.abi_netcdf.metadata_to_datetime(metadata)[source]#

Use information from the metadata to get the image datetime.

geoips.plugins.modules.readers.ahi_hsd module#

Advanced Himawari Imager Data Reader.

exception geoips.plugins.modules.readers.ahi_hsd.AutoGenError[source]#

Bases: Exception

Raise exception on geolocation autogeneration error.

geoips.plugins.modules.readers.ahi_hsd.call(fnames, metadata_only=False, chans=None, area_def=None, self_register=False, test_arg='AHI Default Test Arg')[source]#

Read AHI HSD data data from a list of filenames.

Parameters:

• fnames (list) –

  • List of strings, full paths to files

• metadata_only (bool, default=False) –

  • Return before actually reading data if True

• chans (list of str, default=None) –

  • List of desired channels (skip unneeded variables as needed).

  • Include all channels if None.

• area_def (pyresample.AreaDefinition, default=None) –

  • Specify region to read

  • Read all data if None.

• self_register (str or bool, default=False) –

  • register all data to the specified dataset id (as specified in the return dictionary keys).

  • Read multiple resolutions of data if False.

Returns:

• dictionary of xarray.Dataset objects with required Variables and Attributes.

• Dictionary keys can be any descriptive dataset ids.

Return type:

dict of xarray.Datasets

See also

Xarray and NetCDF Metadata Standards

Additional information regarding required attributes and variables for GeoIPS-formatted xarray Datasets.

geoips.plugins.modules.readers.ahi_hsd.call_single_time(fnames, metadata_only=False, chans=None, area_def=None, self_register=False, test_arg='AHI Default Test Arg')[source]#

Read AHI HSD data data from a list of filenames.

Parameters:

• fnames (list) –

  • List of strings, full paths to files

• metadata_only (bool, default=False) –

  • Return before actually reading data if True

• chans (list of str, default=None) –

  • List of desired channels (skip unneeded variables as needed).

  • Include all channels if None.

• area_def (pyresample.AreaDefinition, default=None) –

  • Specify region to read

  • Read all data if None.

• self_register (str or bool, default=False) –

  • register all data to the specified dataset id (as specified in the return dictionary keys).

  • Read multiple resolutions of data if False.

Returns:

• dictionary of xarray.Dataset objects with required Variables and Attributes.

• Dictionary keys can be any descriptive dataset ids.

Return type:

dict of xarray.Datasets

See also

Xarray and NetCDF Metadata Standards

Additional information regarding required attributes and variables for GeoIPS-formatted xarray Datasets.

geoips.plugins.modules.readers.ahi_hsd.findDiff(d1, d2, path='')[source]#

Find diff.

geoips.plugins.modules.readers.ahi_hsd.get_band_metadata(all_metadata)[source]#

Get band metadata.

This method basically just reformats the all_metadata dictionary that is set based on the metadata found in the netcdf object itself to reference channel names as opposed to filenames as the dictionary keys.

geoips.plugins.modules.readers.ahi_hsd.get_data(md, gvars, rad=False, ref=False, bt=False, zoom=1.0)[source]#

Read data for a full channel’s worth of files.

geoips.plugins.modules.readers.ahi_hsd.get_latitude_longitude(metadata, BADVALS, area_def=None)[source]#

Get latitudes and longitudes.

This routine accepts a dictionary containing metadata as read from an HSD format file and returns latitudes and longitudes for a full disk image.

Note: This code has been adapted from Dan Lindsey’s Fortran90 code. This was done in three steps that ultimately culminated in faster, but more difficult to understand code. If you plan to edit this, I recommend that you return to Dan’s original code, then explore the commented code here, then finally, look at the single-command statements that are currently being used.

geoips.plugins.modules.readers.ahi_hsd.metadata_to_datetime(metadata, time_var='ob_start_time')[source]#

Use information from block_01 to get the image datetime.

geoips.plugins.modules.readers.ahi_hsd.set_variable_metadata(xobj_attrs, band_metadata, dsname, varname)[source]#

Set variable metadata.

MLS 20180914 Setting xobj_attrs at the variable level for the associated channel metadata pulled from the actual netcdf file. This will now be accessible from the scifile object. Additionally, pull out specifically the band_wavelength and attach it to the _varinfo at the variable level - this is automatically pulled from the xobj_attrs dictionary and set in the variable._varinfo dictionary in scifile/scifile.py and scifile/containers.py (see empty_varinfo at the beginning of containers.py for dictionary fields that are automatically pulled from the appropriate location in the xobj_attrs dictionary and set on the _varinfo dictionary)

geoips.plugins.modules.readers.ahi_hsd.sort_by_band_and_seg(metadata)[source]#

Sort by band and segment.

geoips.plugins.modules.readers.ami_netcdf module#

Standard GeoIPS xarray dictionary based GeoKOMPSAT AMI NetCDF data reader.

geoips.plugins.modules.readers.ami_netcdf.call(fnames, metadata_only=False, chans=None, area_def=None, self_register=False)[source]#

Read Geo-Kompsat NetCDF data from a list of filenames.

Parameters:

• fnames (list) –

  • List of strings, full paths to files

• metadata_only (bool, default=False) –

  • Return before actually reading data if True

• chans (list of str, default=None) –

  • List of desired channels (skip unneeded variables as needed).

  • Include all channels if None.

• area_def (pyresample.AreaDefinition, default=None) –

  • Specify region to read

  • Read all data if None.

• self_register (str or bool, default=False) –

  • register all data to the specified dataset id (as specified in the return dictionary keys).

  • Read multiple resolutions of data if False.

Returns:

• dictionary of xarray.Dataset objects with required Variables and Attributes.

• Dictionary keys can be any descriptive dataset ids.

Return type:

dict of xarray.Datasets

See also

Xarray and NetCDF Metadata Standards

Additional information regarding required attributes and variables for GeoIPS-formatted xarray Datasets.

geoips.plugins.modules.readers.ami_netcdf.call_single_time(fnames, metadata_only=False, chans=None, area_def=None, self_register=False)[source]#

Read Geo-Kompsat NetCDF data from a list of filenames.

Parameters:

• fnames (list) –

  • List of strings, full paths to files

• metadata_only (bool, default=False) –

  • Return before actually reading data if True

• chans (list of str, default=None) –

  • List of desired channels (skip unneeded variables as needed).

  • Include all channels if None.

• area_def (pyresample.AreaDefinition, default=None) –

  • Specify region to read

  • Read all data if None.

• self_register (str or bool, default=False) –

  • register all data to the specified dataset id (as specified in the return dictionary keys).

  • Read multiple resolutions of data if False.

Returns:

• dictionary of xarray.Dataset objects with required Variables and Attributes.

• Dictionary keys can be any descriptive dataset ids.

Return type:

dict of xarray.Datasets

See also

Xarray and NetCDF Metadata Standards

Additional information regarding required attributes and variables for GeoIPS-formatted xarray Datasets.

geoips.plugins.modules.readers.ami_netcdf.get_data(gvars, fname, rad=False, ref=False, bt=False)[source]#

Get actual pixel data from a specific file using geolocation information.

geoips.plugins.modules.readers.ami_netcdf.get_test_files(test_data_dir)[source]#

Generate testing xarray from test data.

geoips.plugins.modules.readers.ami_netcdf.get_test_parameters()[source]#

Generate test data key for unit testing.

geoips.plugins.modules.readers.ami_netcdf.latlon_from_lincol_geos(resolution, line, column, metadata)[source]#

Calculate latitude and longitude from array indices.

Uses geostationary projection (likely won’t work with extended local area files). Equations and code from GEO-KOMPSAT-2A Level 1B Data User Manual.

geoips.plugins.modules.readers.ami_netcdf.metadata_to_datetime(metadata)[source]#

Use information from the metadata to get the image datetime.

geoips.plugins.modules.readers.amsr2_netcdf module#

Read Advanced Microwave Scanning Radiometer (AMSR2) data products.

geoips.plugins.modules.readers.amsr2_netcdf.call(fnames, metadata_only=False, chans=None, area_def=None, self_register=False, test_arg='AMSR2 Default Test Arg')[source]#

Read AMSR2 netcdf data products.

Parameters:

• fnames (list) –

  • List of strings, full paths to files

• metadata_only (bool, default=False) –

  • Return before actually reading data if True

• chans (list of str, default=None) –

  • List of desired channels (skip unneeded variables as needed).

  • Include all channels if None.

• area_def (pyresample.AreaDefinition, default=None) –

  • NOT YET IMPLEMENTED

  • Specify region to read

  • Read all data if None.

• self_register (str or bool, default=False) –

  • NOT YET IMPLEMENTED

  • register all data to the specified dataset id (as specified in the return dictionary keys).

  • Read multiple resolutions of data if False.

Returns:

• dictionary of xarray.Dataset objects with required Variables and Attributes.

• Dictionary keys can be any descriptive dataset ids.

Return type:

dict of xarray.Datasets

See also

Xarray and NetCDF Metadata Standards

Additional information regarding required attributes and variables for GeoIPS-formatted xarray Datasets.

geoips.plugins.modules.readers.amsr2_netcdf.get_test_files(test_data_dir)[source]#

Generate test files for unit testing reader.

geoips.plugins.modules.readers.amsr2_netcdf.get_test_parameters()[source]#

Generate a data key for unit testing.

geoips.plugins.modules.readers.amsr2_netcdf.read_amsr2_data(full_xarray, chans)[source]#

Read non-AMSR2_OCEAN data.

geoips.plugins.modules.readers.amsr2_netcdf.read_amsr2_mbt(full_xarray, varname, time_array=None)[source]#

Reformat AMSR2 xarray object appropriately.

• variables: latitude, longitude, time, brightness temperature variables

• attributes: source_name, platform_name, data_provider, interpolation_radius_of_influence

geoips.plugins.modules.readers.amsr2_netcdf.read_amsr2_winds(wind_xarray)[source]#

Reformat AMSR2 xarray object appropriately.

• variables: latitude, longitude, time, wind_speed_kts

• attributes: source_name, platform_name, data_provider, interpolation_radius_of_influence

geoips.plugins.modules.readers.amsr2_remss_winds_netcdf module#

Read derived surface winds from REMSS AMSR netcdf data.

geoips.plugins.modules.readers.amsr2_remss_winds_netcdf.call(fnames, metadata_only=False, chans=None, area_def=None, self_register=False)[source]#

Read REMSS AMSR2 derived winds from netcdf data.

Parameters:

• fnames (list) –

  • List of strings, full paths to files

• metadata_only (bool, default=False) –

  • NOT YET IMPLEMENTED

  • Return before actually reading data if True

• chans (list of str, default=None) –

  • NOT YET IMPLEMENTED

  • List of desired channels (skip unneeded variables as needed).

  • Include all channels if None.

• area_def (pyresample.AreaDefinition, default=None) –

  • NOT YET IMPLEMENTED

  • Specify region to read

  • Read all data if None.

• self_register (str or bool, default=False) –

  • NOT YET IMPLEMENTED

  • register all data to the specified dataset id (as specified in the return dictionary keys).

  • Read multiple resolutions of data if False.

Returns:

• dictionary of xarray.Dataset objects with required Variables and Attributes.

• Dictionary keys can be any descriptive dataset ids.

Return type:

dict of xarray.Datasets

See also

Xarray and NetCDF Metadata Standards

Additional information regarding required attributes and variables for GeoIPS-formatted xarray Datasets.

geoips.plugins.modules.readers.amsr2_remss_winds_netcdf.get_test_files(test_data_dir)[source]#

Generate test files for unit testing reader.

geoips.plugins.modules.readers.amsr2_remss_winds_netcdf.get_test_parameters()[source]#

Generate data key for unit testing.

geoips.plugins.modules.readers.amsua_mhs_mirs module#

Read AMSU-A and MHS MIRS NetCDF data files.

This reader is desgined for importing the Advanced Microwave Sounding Unit (AMSU)-A and Microwave Humidity Sounder (HMS) files made available from NOAA MIRS. These data files have a different file name convention and and data structure from previous MSPPS. Example of MIRIS files:

• NPR-MIRS-IMG_v11r4_ma1_s202101111916000_e202101112012000_c202101112047200.nc

  • (ma1 is for metop-B)

• NPR-MIRS-IMG_v11r4_ma2_s202101111715000_e202101111857000_c202101111941370.nc

  • (ma2 is for metop-A)

• NPR-MIRS-IMG_v11r4_n19_s202101111730000_e202101111916000_c202101112001590.nc

  • (NOAA-19)

AMSU-B was replaced with MHS in 2005, and now AMSU-A / MHS fly on newer NOAA birds (>17) and METOP-A/B/C (METOP-A dead now). AMSU-A channels are still channels 1-15, and MHS channels (1-5) live in the same 16-20 slots that previously contained AMSU-B.

More information on the AMSU and MHS instruments may be found here https://en.wikipedia.org/wiki/Advanced_microwave_sounding_unit https://en.wikipedia.org/wiki/Microwave_humidity_sounder

AMSU-A channel information:

Chan Frequency Polarization #Bands Sensitivity Primary Function Num (GHz) (at nadir) NEDT (K) 1 23.8 vertical 1 0.30 Water Vapor Burden 2 31.4 vertical 1 0.30 Water Vapor Burden 3 50.3 vertical 1 0.40 Water Vapor Burden 4 52.8 vertical 1 0.25 Water Vapor Burden 5 53.596 ± 0.115 horizontal 2 0.25 Tropospheric Temp 6 54.4 horizontal 1 0.25 Tropospheric Temp 7 54.94 vertical 1 0.25 Tropospheric Temp 8 55.5 horizontal 1 0.25 Tropospheric Temp 9 57.290 horizontal 1 0.25 Stratospheric Temp 10 57.290 ± 0.217 horizontal 2 0.40 Stratospheric Temp 11 57.290 ± 0.3222 ± 0.048 horizontal 4 0.40 Stratospheric Temp 12 57.290 ± 0.3222 ± 0.022 horizontal 4 0.60 Stratospheric Temp 13 57.290 ± 0.3222 ± 0.010 horizontal 4 0.80 Stratospheric Temp 14 57.290 ± 0.3222 ± 0.0045 horizontal 4 1.20 Stratospheric Temp 15 89.0 vertical 1 0.50 Cloud Top/Snow

MHS channel information:

Chan AMSU-B Frequency AMSU-B Bandwidth Sensitivity Polarization Num Chan# (GHz) Freq(GHz) (MHz) NEDT (K) (at nadir) 1 16 89.0 89.0 ± 0.9 2800 0.22 Vertical 2 17 157.0 150 ± 0.9 2800 0.34 Vertical 3 18 183.311 ± 1.00 183.31 ± 1.00 2 x 500 0.51 Horizontal 4 19 183.311 ± 3.00 183.31 ± 3.00 2 x 1000 0.40 Horizontal 5 20 190.311 183.31 ± 7.00 2200 0.46 Vertical

V1.0: Initial version, NRL-MRY, January 26, 2021

Dataset information:

Basic information on AMSU-A MHS MIRS product file
index: 1     2    3    4      5      6    7
freq: 23.8,31.4,50.3,52.799,53.595,54.4,54.941,
55.499,57.29,57.29,57.29,57.29,57.29,57.29,
index:   8     9      10    11   12   13     14
freq: 55.499,57.29,57.29,57.29,57.29,57.29,57.29,
index: 15  16  17    18      19     20
freq: 89.,89.,157.,183.311,183.311,190.311

dimensions:
      Scanline = 2370 ;
      Field_of_view = 90 ;
      P_Layer = 100 ;
      Channel = 20 ;
      Qc_dim = 4 ;
variables:
      Freq(Channel): Central Frequencies (GHz)
      Polo(Channel): Polarizations
      ScanTime_year(Scanline): Calendar Year 20XX
      ScanTime_doy(Scanline: julian day 1-366
      ScanTime_month(Scanline): Calendar month 1-12
      ScanTime_dom(Scanline): Calendar day of the month 1-31
      ScanTime_hour(Scanline: hour of the day 0-23
      ScanTime_minute(Scanline): minute of the hour 0-59
      ScanTime_second(Scanline): second of the minute 0-59
      ScanTime_UTC(Scanline): Number of seconds since 00:00:00 UTC
      Orb_mode(Scanline): 0-ascending,1-descending
      Latitude(Scanline, Field_of_view):Latitude of the view (-90,90),
                                        unit: degree
      Longitude(Scanline, Field_of_view):Longitude of the view (-180,180),
                                         unit: degree
      Sfc_type(Scanline, Field_of_view):type of surface:0-ocean,1-sea ice,
                                        2-land,3-snow
      Atm_type(Scanline, Field_of_view): type of atmosphere:currently missing
                                      ( note: not needed for geoips products)
      Qc(Scanline, Field_of_view, Qc_dim): Qc: 0-good, 1-usable with problem,
                                               2-bad
      ChiSqr(Scanline, Field_of_view): Convergence rate: <3-good,>10-bad
      LZ_angle(Scanline, Field_of_view): Local Zenith Angle degree
      RAzi_angle(Scanline, Field_of_view):Relative Azimuth Angle 0-360 degree
      SZ_angle(Scanline, Field_of_view):Solar Zenith Angle (-90,90) degree
      BT(Scanline, Field_of_view, Channel): Channel Temperature (K)
      YM(Scanline, Field_of_view, Channel): UnCorrected Channel Temperature(K)
      ChanSel(Scanline, Field_of_view, Channel):Channels Selection Used in
                                                Retrieval
      TPW(Scanline, Field_of_view): Total Precipitable Water (mm)
      CLW(Scanline, Field_of_view):Cloud liquid Water (mm)
      RWP(Scanline, Field_of_view): Rain Water Path (mm)
      LWP(Scanline, Field_of_view): Liquid Water Path (mm)
      SWP(Scanline, Field_of_view): Snow Water Path (mm)
      IWP(Scanline, Field_of_view): Ice Water Path (mm)
      GWP(Scanline, Field_of_view): Graupel Water Path (mm)
      RR(Scanline, Field_of_view): Rain Rate (mm/hr)
      Snow(Scanline, Field_of_view): Snow Cover (range: 0-1) i.e., 1 -> 100%
      SWE(Scanline, Field_of_view): Snow Water Equivalent (cm)
      SnowGS(Scanline, Field_of_view):Snow Grain Size (mm)
      SIce(Scanline, Field_of_view):Sea Ice Concentration (%)
      SIce_MY(Scanline, Field_of_view): Multi-Year Sea Ice Concentration (%)
      SIce_FY(Scanline, Field_of_view): First-Year Sea Ice Concentration (%)
      TSkin(Scanline, Field_of_view): Skin Temperature (K)
      SurfP(Scanline, Field_of_view): Surface Pressure (mb)
      Emis(Scanline, Field_of_view, Channel):Channel Emissivity
                                      (unit:1,  Emis:scale_factor = 0.0001)
      SFR(Scanline, Field_of_view): Snow Fall Rate in mm/hr
      CldTop(Scanline, Field_of_view): Cloud Top Pressure (scale_factor = 0.1)
      CldBase(Scanline, Field_of_view): Cloud Base Pressure
                                        (scale_factor = 0.1)
      CldThick(Scanline, Field_of_view): Cloud Thickness (scale_factor = 0.1)
      PrecipType(Scanline, Field_of_view): Precipitation Type (Frozen/Liquid)
      RFlag(Scanline, Field_of_view): Rain Flag
      SurfM(Scanline, Field_of_view): Surface Moisture (scale_factor = 0.1)
      WindSp(Scanline, Field_of_view): Wind Speed (m/s) (scale_factor = 0.01
      WindDir(Scanline, Field_of_view: Wind Direction (scale_factor = 0.01)
      WindU(Scanline, Field_of_view):U-direction Wind Speed (m/s)
                                    (scale_factor = 0.01)
      WindV(Scanline, Field_of_view: V-direction Wind Speed (m/s)
                                    (scale_factor = 0.01)
      Prob_SF(Scanline, Field_of_view): Probability of falling snow (%)

Additional info:

Variables (nscan, npix):  npix=90 pixels per scan; nscan: vary with orbit
chan-1 AT:  89 GHz              as ch16    anttenna temperature at V-pol
                                           FOV 16km at nadir
chan-2 AT:  150 (157) GHz       as ch17 the number in bracket is for MHS
                                       from metops at V-pol, 16km at nadir
chan-3 AT:  183.31 +-1 GHz          as ch18    at H-pol, 16km
chan-4 AT:  183.31 +-3 GHz          as ch19    at H-pol, 16km
chan-5 AT:  183.31 +-7 (190.3) GHz  as ch20    at V-pol, 16km
lat:     -90, 90     deg
lon:     -180, 180   deg
RR:  surface rainrate (mm/hr)
Snow: surafce snow cover
IWP:  ice water path
SWE:  snow water equvelent
Sfc_type:  surface type
Orbit_mode:   -1: ascending, 1: decending, 2: both
SFR:  snowfall rate (unit?)
LZ_angle:  local zinath angle (deg)
SZ_angle:  solar zinath angle (deg)

geoips.plugins.modules.readers.amsua_mhs_mirs.call(fnames, metadata_only=False, chans=None, area_def=None, self_register=False)[source]#

Read AMSU/MHS MIRS data products.

Parameters:

• fnames (list) –

  • List of strings, full paths to files

• metadata_only (bool, default=False) –

  • Return before actually reading data if True

• chans (list of str, default=None) –

  • List of desired channels (skip unneeded variables as needed).

  • Include all channels if None.

• area_def (pyresample.AreaDefinition, default=None) –

  • Specify region to read

  • Read all data if None.

• self_register (str or bool, default=False) –

  • NOT YET IMPLEMENTED

  • register all data to the specified dataset id (as specified in the return dictionary keys).

  • Read multiple resolutions of data if False.

Returns:

• dictionary of xarray.Dataset objects with required Variables and Attributes.

• Dictionary keys can be any descriptive dataset ids.

Return type:

dict of xarray.Datasets

See also

Xarray and NetCDF Metadata Standards

Additional information regarding required attributes and variables for GeoIPS-formatted xarray Datasets.

geoips.plugins.modules.readers.amsub_hdf module#

Read AMSU-B and MHS passive microwave data files.

This reader is desgined for importing NOAA Advanced Microwave Sounding Unit (AMSU)-B/Microwave Humidity Sounder (HMS) and EUMETSAQT MHS data files in hdf format, such as

• NPR.MHOP.NP.D20153.S2046.E2230.B5832021.NS (N19),

• NPR.MHOP.NN.D20153.S1927.E2105.B7748081.NS (N18),

• NPR.MHOP.M1.D20153.S2229.E2318.B3998282.NS (METOP).

V1.0: Initial version, NRL-MRY, June 1, 2020

Basic information on AMSU-B product file:

Input SD Variables
(nscan, npix):
    npix=90 pixels per scan;
    nscan: vary with orbit
chan-1 AT:  89 GHz as ch16 anttenna temperature at V-pol   FOV 16km at nadir
chan-2 AT:  150 (157) GHz as ch17 the number in bracket is for MHS from
                                  metops at V-pol, 16km at nadir
chan-3 AT:  183.31 +-1 GHz as ch18    at H-pol, 16km
chan-4 AT:  183.31 +-3 GHz as ch19    at H-pol, 16km
chan-5 AT:  183.31 +-7 (190.3) GHz as ch20    at V-pol, 16km
lat:     -90, 90     deg
lon:     -180, 180   deg
RR:  surface rainrate (mm/hr)
Snow: surafce snow cover
IWP:  ice water path (unit?)
SWE:  snow water equvelent  (unit)
Sfc_type:  surface type
Orbit_mode:   -1: ascending, 1: decending, 2: both
SFR:  snowfall rate (unit?)
LZ_angle:  local zinath angle (deg)
SZ_angle:  solar zinath angle (deg)

Vdata info (definition of AMSU-B date):
ScanTime_year
ScanTime_month
ScanTime_day
ScanTime_hour
ScanTime_minute
ScanTime_second
ScanTime_Jday
Time

geoips.plugins.modules.readers.amsub_hdf.call(fnames, metadata_only=False, chans=None, area_def=None, self_register=False)[source]#

Read AMSU-B hdf data products.

Parameters:

• fnames (list) –

  • List of strings, full paths to files

• metadata_only (bool, default=False) –

  • NOT YET IMPLEMENTED

  • Return before actually reading data if True

• chans (list of str, default=None) –

  • NOT YET IMPLEMENTED

  • List of desired channels (skip unneeded variables as needed).

  • Include all channels if None.

• area_def (pyresample.AreaDefinition, default=None) –

  • NOT YET IMPLEMENTED

  • Specify region to read

  • Read all data if None.

• self_register (str or bool, default=False) –

  • NOT YET IMPLEMENTED

  • register all data to the specified dataset id (as specified in the return dictionary keys).

  • Read multiple resolutions of data if False.

Returns:

• dictionary of xarray.Dataset objects with required Variables and Attributes.

• Dictionary keys can be any descriptive dataset ids.

Return type:

dict of xarray.Datasets

See also

Xarray and NetCDF Metadata Standards

Additional information regarding required attributes and variables for GeoIPS-formatted xarray Datasets.

geoips.plugins.modules.readers.ascat_uhr_netcdf module#

Read derived surface winds from BYU ASCAT UHR NetCDF data.

geoips.plugins.modules.readers.ascat_uhr_netcdf.call(fnames, metadata_only=False, chans=None, area_def=None, self_register=False)[source]#

Read ASCAT UHR derived winds or normalized radar cross section data.

Parameters:

• fnames (list) –

  • List of strings, full paths to files

• metadata_only (bool, default=False) –

  • NOT YET IMPLEMENTED

  • Return before actually reading data if True

• chans (list of str, default=None) –

  • NOT YET IMPLEMENTED

  • List of desired channels (skip unneeded variables as needed).

  • Include all channels if None.

• area_def (pyresample.AreaDefinition, default=None) –

  • NOT YET IMPLEMENTED

  • Specify region to read

  • Read all data if None.

• self_register (str or bool, default=False) –

  • NOT YET IMPLEMENTED

  • register all data to the specified dataset id (as specified in the return dictionary keys).

  • Read multiple resolutions of data if False.

Returns:

• dictionary of xarray.Dataset objects with required Variables and Attributes.

• Dictionary keys can be any descriptive dataset ids.

Return type:

dict of xarray.Datasets

See also

Xarray and NetCDF Metadata Standards

Additional information regarding required attributes and variables for GeoIPS-formatted xarray Datasets.

geoips.plugins.modules.readers.ascat_uhr_netcdf.read_byu_data(wind_xarray, fname)[source]#

Reformat ascat xarray object appropriately.

• variables: latitude, longitude, time, wind_speed_kts, wind_dir_deg_met

• attributes: source_name, platform_name, data_provider, interpolation_radius_of_influence

geoips.plugins.modules.readers.atms_hdf5 module#

Reader to read a grannual NOAA ATMS SDR TBs in h5 format.

Output variables in xarray object for geoips processing system

V0: August 25, 2021

The date is generated by the NOAA community satellite processing package (CSPP), developed at CIMSS

Example of ATMS file names:

'SATMS_j01_d20210809_t0959306_e1000023_b19296_fnmoc_ops.h5'
    SDR TBs variables
'GATMO_j01_d20210809_t0959306_e1000023_b19296_fnmoc_ops.h5'
    SDR Geolocation variables

Dataset info:

TB[12,96,22]:  for each granuel

CHAN#  Center-Freq(GHz)  POL
1      23.8               V
2      31.4               V
3      50.3               H
4      51.76              H
5      52.8               H
6      53.596+-0.115      H
7      54.4               H
8      54.94              H
9      55.5               H
10     57.290(f0)         H
11     f0 +-0.217         H
12     f0 +-0.322+-0.048  H
13     f0 +-0.322+-0.022  H
14     f0 +-0.322+-0.010  H
15     f0 +-0.322+-0.0045 H

16     88.2               V
17     165.5              H
18     183.1+-7           H
19     183.1+-4.5         H    (FNMOC used this chan for 183 GHz image)
20     183.1+-3.0         H
21     183.1+-1.8         H
22     183.1+-1.0         H

BeamTime[12,96]:  microsecond, i.e., 1*10^-6.  IET "IDPS Epoch Time" is used
                  It is a signed 64-bit integer giving microseconds since
                  00:00:00.000000 Jan 1 1958.
BrightnessTemperatureFactors[2]: 1: scale (unitless); 2: offset (K)
BrightnessTemperature[12,96,22]: [scan,pix,chans]

SDR geolocation Info
Latitude/Longitude[12,96]:   for Chan 17 only (for initial product,
                                               it is used for all channels)
BeamLatitude[12,96,5]:       for chan 1,2,3,16,17. They will be used for
                                        associated TBs at a later date.
BeamLongitude[12,96,5]:      for chan 1,2,3,16,17
SatelliteAzimuthAngle, SatelliteZenithAngle,
SolarAzimuthAngle, SolarZenithAngle[12,96]

Notes

Unix epoch time is defined as the number of seconds that have elapsed since January 1, 1970 (midnight UTC/GMT). Thus, there is a 12 years difference for the JPSS data when datetime.datetime.utcfromtimestamp(epoch) is used to convert the JPSS IDPS Epoch time to the humman-readable date.

This reader is developed to read one granual a time from ATMS npp and jpss-1(n20) data files.

The example files are:

• SATMS_j01_d20210809_t0959306_e1000023_b19296_fnmoc_ops.h5: for TBs. ‘b’: orbit#

• GATMO_j01_d20210809_t0959306_e1000023_b19296_fnmoc_ops.h5: for geolocations

geoips.plugins.modules.readers.atms_hdf5.call(fnames, metadata_only=False, chans=None, area_def=None, self_register=False)[source]#

Read ATMS hdf5 data products.

Parameters:

• fnames (list) –

  • List of strings, full paths to files

• metadata_only (bool, default=False) –

  • NOT YET IMPLEMENTED

  • Return before actually reading data if True

• chans (list of str, default=None) –

  • NOT YET IMPLEMENTED

  • List of desired channels (skip unneeded variables as needed).

  • Include all channels if None.

• area_def (pyresample.AreaDefinition, default=None) –

  • NOT YET IMPLEMENTED

  • Specify region to read

  • Read all data if None.

• self_register (str or bool, default=False) –

  • NOT YET IMPLEMENTED

  • register all data to the specified dataset id (as specified in the return dictionary keys).

  • Read multiple resolutions of data if False.

Returns:

• dictionary of xarray.Dataset objects with required Variables and Attributes.

• Dictionary keys can be any descriptive dataset ids.

Return type:

dict of xarray.Datasets

See also

Xarray and NetCDF Metadata Standards

Additional information regarding required attributes and variables for GeoIPS-formatted xarray Datasets.

geoips.plugins.modules.readers.atms_hdf5.convert_epoch_to_datetime64(time_array, use_shape=None)[source]#

Convert time to datetime object.

Parameters:

• time_array (array) – Array of start time integers (multiplied by 1e-6 in function)

• use_shape (tuple, optional) – desired output shape of time array, by default None

Returns:

array of converted datetime objects

Return type:

array

geoips.plugins.modules.readers.atms_hdf5.read_atms_file(fname, xarray_atms, metadata_only=False)[source]#

Read ATCF data from file fname.

geoips.plugins.modules.readers.cygnss_netcdf module#

Read derived surface winds from CYGNSS netcdf data.

geoips.plugins.modules.readers.cygnss_netcdf.call(fnames, metadata_only=False, chans=None, area_def=None, self_register=False)[source]#

Read CYGNSS derived winds from netcdf data.

Parameters:

• fnames (list) –

  • List of strings, full paths to files

• metadata_only (bool, default=False) –

  • Return before actually reading data if True

• chans (list of str, default=None) –

  • List of desired channels (skip unneeded variables as needed).

  • Include all channels if None.

• area_def (pyresample.AreaDefinition, default=None) –

  • NOT YET IMPLEMENTED

  • Specify region to read

  • Read all data if None.

• self_register (str or bool, default=False) –

  • NOT YET IMPLEMENTED

  • register all data to the specified dataset id (as specified in the return dictionary keys).

  • Read multiple resolutions of data if False.

Returns:

• dictionary of xarray.Dataset objects with required Variables and Attributes.

• Dictionary keys can be any descriptive dataset ids.

Return type:

dict of xarray.Datasets

See also

Xarray and NetCDF Metadata Standards

Additional information regarding required attributes and variables for GeoIPS-formatted xarray Datasets.

geoips.plugins.modules.readers.cygnss_netcdf.get_test_files(test_data_dir)[source]#

Generate a testing xarray dataset from NetCDF files in the test data directory.

Parameters:

test_data_dir (str) – Path to the directory containing test data.

Returns:

The dataset created from the NetCDF test files.

Return type:

xarray.Dataset

Raises:

FileNotFoundError – If no NetCDF files are found in the specified directory.

geoips.plugins.modules.readers.cygnss_netcdf.get_test_parameters()[source]#

Generate test data key for unit testing.

geoips.plugins.modules.readers.ewsg_netcdf module#

Read EWS-G data.

This EWS-G(Electro-Optical Infrared Weather System - Geostationary) reader is designed for reading theEWS-G data files (EWS-G is renamed from GOES-13). The reader is only using the python functions and xarray variables. The reader is based on EWS-G data in netcdf4 format.

V1.0: NRL-Monterey, 02/25/2021

EWS-G file information:

Example of the gvar filename:   2020.1212.0012.goes-13.gvar.nc

Note that channel-3 is not available for EWS-G.
  gvar_Ch3(TIR=5.8-7.3um, ctr=6.48um,4km): unit=temp-deg(C), scale_factor=0.01

variables:
  gvar_Ch1(VIS=0.55-0.75um, ctr=0.65um,1km): unit=albedo*100,  scale_factor=0.01
  gvar_Ch2(MWIR=3.8-4.0um,  ctr=3.9um, 4km): unit=temp-deg(C), scale_factor=0.01
  gvar_Ch4(TIR=10.2-11.2um, ctr=10.7um,4km): unit=temp-deg(C), scale_factor=0.01
  gvar_Ch6(TIR=12.9-13.7um, ctr=13.3um 4km): unit=temp-deg(C), scale_factor=0.01
  latitude: unit=degree
  longitude:unit=degree
  sat_zenith: unit=degree
  sun_zenith: unit=degree
  rel_azimuth:unit=degree

  variable array definition:  var(scan,pix); scan-->lines, pix-->samples

attributes: many

geoips.plugins.modules.readers.ewsg_netcdf.call(fnames, metadata_only=False, chans=None, area_def=None, self_register=False)[source]#

Read EWS-G data in netcdf4 format.

Parameters:

• fnames (list) –

  • List of strings, full paths to files

• metadata_only (bool, default=False) –

  • NOT YET IMPLEMENTED

  • Return before actually reading data if True

• chans (list of str, default=None) –

  • NOT YET IMPLEMENTED

  • List of desired channels (skip unneeded variables as needed).

  • Include all channels if None.

• area_def (pyresample.AreaDefinition, default=None) –

  • NOT YET IMPLEMENTED

  • Specify region to read

  • Read all data if None.

• self_register (str or bool, default=False) –

  • NOT YET IMPLEMENTED

  • register all data to the specified dataset id (as specified in the return dictionary keys).

  • Read multiple resolutions of data if False.

Returns:

• dictionary of xarray.Dataset objects with required Variables and Attributes.

• Dictionary keys can be any descriptive dataset ids.

Return type:

dict of xarray.Datasets

See also

Xarray and NetCDF Metadata Standards

Additional information regarding required attributes and variables for GeoIPS-formatted xarray Datasets.

geoips.plugins.modules.readers.ewsg_netcdf.call_single_time(fnames, metadata_only=False, chans=None, area_def=None, self_register=False)[source]#

Read EWS-G data in netcdf4 format for one or more files.

Parameters:

• fnames (list) –

  • List of strings, full paths to files

• metadata_only (bool, default=False) –

  • NOT YET IMPLEMENTED

  • Return before actually reading data if True

• chans (list of str, default=None) –

  • NOT YET IMPLEMENTED

  • List of desired channels (skip unneeded variables as needed).

  • Include all channels if None.

• area_def (pyresample.AreaDefinition, default=None) –

  • NOT YET IMPLEMENTED

  • Specify region to read

  • Read all data if None.

• self_register (str or bool, default=False) –

  • NOT YET IMPLEMENTED

  • register all data to the specified dataset id (as specified in the return dictionary keys).

  • Read multiple resolutions of data if False.

Returns:

• dictionary of xarray.Dataset objects with required Variables and Attributes.

• Dictionary keys can be any descriptive dataset ids.

Return type:

dict of xarray.Datasets

See also

Xarray and NetCDF Metadata Standards

Additional information regarding required attributes and variables for GeoIPS-formatted xarray Datasets.

geoips.plugins.modules.readers.fci_netcdf module#

Standard GeoIPS xarray dictionary based FCI NetCDF data reader.

geoips.plugins.modules.readers.fci_netcdf.avail_chans_in_file(fci_file)[source]#

Return list of available channels/variables in file.

Parameters:

fci_file (str) – Full file path to FCI file

Returns:

list of strings of available variables

Return type:

list

geoips.plugins.modules.readers.fci_netcdf.call(fnames, metadata_only=False, chans=None, area_def=None, self_register=False)[source]#

Read and calibrate FCI data.

Parameters:

• fci_files (list) – list of files for a given scan time

• metadata_only (bool, optional) – Only return metadata, by default False

• chans (list, optional) – List of channels/variables to load, by default None

• area_def (pyresample area definition, optional) – Read in data for specific area definition, by default None (currently unsupported)

• self_register (bool, optional) – Self register data to given resolution, by default False (currently unsupported)

Returns:

dictionary of xarray datasets

Return type:

dict

geoips.plugins.modules.readers.fci_netcdf.final_calibrated_variable_name(ncvar_name, chan_map={'ir_105': 'B14Rad', 'ir_105_hr': 'HRB14Rad', 'ir_123': 'B15Rad', 'ir_133': 'B16Rad', 'ir_38': 'B09Rad', 'ir_38_hr': 'HRB09Rad', 'ir_87': 'B12Rad', 'ir_97': 'B13Rad', 'nir_13': 'B06Rad', 'nir_16': 'B07Rad', 'nir_22': 'B08Rad', 'vis_04': 'B01Rad', 'vis_05': 'B02Rad', 'vis_06': 'B03Rad', 'vis_06_hr': 'HRB03Rad', 'vis_08': 'B04Rad', 'vis_09': 'B05Rad', 'wv_63': 'B10Rad', 'wv_73': 'B11Rad'})[source]#

Determine the final name of dataset variable that should be used in GeoIPS.

Parameters:

• ncvar_name (str) – Name of variable in source file

• chan_map (dict, optional) – Mapped band name used by GeoIPS, by default CHAN_MAP

Returns:

Variable name used in GeoIPS (e.g. B14BT)

Return type:

str

geoips.plugins.modules.readers.fci_netcdf.get_geolocation_metadata(xarray_attrs)[source]#

Get geolocation metadata.

geoips.plugins.modules.readers.fci_netcdf.get_standard_geoips_attrs(file_attrs, area_def)[source]#

Set the standard geoips attrs.

Parameters:

file_attrs (dict) – Dictionary holding attributes found in source file

Returns:

Dictionary of all source file attrs, along with required GeoIPS attrs

Return type:

dict

geoips.plugins.modules.readers.fci_netcdf.load_metadata_single_file(fci_file, group=None, variable=None)[source]#

Load metadata from an FCI file.

Parameters:

• fci_file (str) – Full file path to file

• group (str, optional) – Name of group to load attributes, by default None

• variable (_type_, optional) – Name of variable to load attributes, by default None

Returns:

All metadata attrs for group/variable

Return type:

dict

geoips.plugins.modules.readers.fci_netcdf.satpy_read_fci_netcdf_files(fci_files, metadata_only=False, chans=None, area_def=None, self_register=False)[source]#

Read and calibrate data using satpy’s fci_l1c_nc reader.

Parameters:

• fci_files (list) – list of files for a given scan time

• metadata_only (bool, optional) – Only return metadata, by default False

• chans (list, optional) – List of channels/variables to load, by default None

• area_def (pyresample.AreaDefinition, optional) – Read in data for specific area definition, by default None

• self_register (bool, optional) – Self register data to given resolution, by default False (currently unsupported)

Returns:

dictionary of xarray datasets

Return type:

dict

geoips.plugins.modules.readers.fci_netcdf.scene_to_xarray(fci_files, geoips_chans, metadata=None, area_def=None)[source]#

Load files into satpy.Scene object then convert to xarray.Dataset.

Parameters:

• fci_files (list) – List of FCI netCDF files on disk

• geoips_chans (list) – Channels/variables denoted in GeoIPS nomenclature to load into memory, where ‘GeoIPS nomenclature’ is the common variable mapping used in our geostationary readers. See ‘BAND_MAP’ variable for an example of these channels. Allows us to read in radiances, reflectances, or brightness temperatures even though the variable in the dataset itself (such as ‘vis_04’) does not provide that information.

• metadata (dict, optional) – Standard geoips attributes, by default None

• area_def (pyresample.AreaDefinition, optional) – Crop data to specific area definition, by default None

Returns:

Calibrated FCI MTG data

Return type:

xarray.Dataset

geoips.plugins.modules.readers.geoips_netcdf module#

Read pre-processed GeoIPS-formatted NetCDF data.

geoips.plugins.modules.readers.geoips_netcdf.call(fnames, metadata_only=False, chans=None, area_def=None, self_register=False)[source]#

Read preprocessed geoips netcdf output.

Parameters:

• fnames (list) –

  • List of strings, full paths to files

• metadata_only (bool, default=False) –

  • NOT YET IMPLEMENTED

  • Return before actually reading data if True

• chans (list of str, default=None) –

  • NOT YET IMPLEMENTED

  • List of desired channels (skip unneeded variables as needed).

  • Include all channels if None.

• area_def (pyresample.AreaDefinition, default=None) –

  • NOT YET IMPLEMENTED

  • Specify region to read

  • Read all data if None.

• self_register (str or bool, default=False) –

  • NOT YET IMPLEMENTED

  • register all data to the specified dataset id (as specified in the return dictionary keys).

  • Read multiple resolutions of data if False.

Returns:

• dictionary of xarray.Dataset objects with required Variables and Attributes.

• Dictionary keys can be any descriptive dataset ids.

Return type:

dict of xarray.Datasets

See also

Xarray and NetCDF Metadata Standards

Additional information regarding required attributes and variables for GeoIPS-formatted xarray Datasets.

geoips.plugins.modules.readers.geoips_netcdf.read_xarray_netcdf(ncdf_fname)[source]#

Read NetCDF file written out using the xarray python package.

geoips.plugins.modules.readers.gfs_grib module#

Reader to read GFS Files in GRIB format.

(NRL-Monterey, Sept. 2024)

Current support includes:

• GFS

• GFS Wave global domain

This reader can take in multiple forcast times at multiple analysis times.

geoips.plugins.modules.readers.gfs_grib.call(fnames, metadata_only=False, chans=None, area_def=None, self_register=False)[source]#

Read GFS GRIB data.

Parameters:

• fnames (list) –

  • List of strings, full paths to files

• metadata_only (bool, default=False) –

  • Return before actually reading data if True

• chans (list of str, default=None) –

  • NOT YET IMPLEMENTED

  • List of desired channels (skip unneeded variables as needed).

  • Include all channels if None.

• area_def (pyresample.AreaDefinition, default=None) –

  • NOT YET IMPLEMENTED

  • Specify region to read

  • Read all data if None.

• self_register (str or bool, default=False) –

  • NOT YET IMPLEMENTED

  • register all data to the specified dataset id (as specified in the return dictionary keys).

  • Read multiple resolutions of data if False.

Returns:

• dictionary of xarray.Dataset objects with required Variables and Attributes.

• Dictionary keys can be any descriptive dataset ids.

• Conforms to geoips xarray standards, see more in geoips documentation.

Return type:

dict of xarray.Datasets

geoips.plugins.modules.readers.gfs_grib.convert_uv_to_kts(xr_df)[source]#

Convert U/V Components to windspeed and winddir.

geoips.plugins.modules.readers.gfs_grib.press_grb_to_xr(grb_obj, custom_pressure_name='unk_pres')[source]#

Convert pressure grb values to xarray.Dataset.

Parameters:

grb_obj (list of pygrib objects) –

• pygrib selected objects (pressure)

Returns:

final_xarray –

• xarray formatted to geoips standards

Return type:

xarray.Dataset

geoips.plugins.modules.readers.gfs_grib.read_atmos(filenames)[source]#

Read gfs files.

Parameters:

filenames (list of filepaths) –

• gfs grib files

Returns:

final_xarray –

• xarray formatted to geoips standards

Return type:

xarray.Dataset

geoips.plugins.modules.readers.gfs_grib.read_wave(filenames)[source]#

Read gfswave products.

geoips.plugins.modules.readers.gfs_grib.surf_grb_to_xr(grb_obj)[source]#

Convert surface grb values to xarray.Dataset.

Parameters:

grb_obj (list of pygrib objects) –

• pygrib selected objects (single/surface layer)

Returns:

final_xarray –

• xarray formatted to geoips standards

Return type:

xarray.Dataset

geoips.plugins.modules.readers.gmi_hdf5 module#

Read NASA GPM GMI hdf5 data files.

Read a grannual NASA GPM GMI TBs in h5 format (each grannual file is about 5 minutes GPM GMI data)

Output variables in xarray object for geoips processing system

V0: August 4, 2020

Dataset information:

variables in original TBs structure format
tb_info = { 'S1': {  'tb10v': 0,
                     'tb10h': 1,
                     'tb19v': 2,
                     'tb19h': 3,
                     'tb23v': 4,
                     'tb37v': 5,
                     'tb37h': 6,
                     'tb89v': 7,
                     'tb89h': 8,},
             'S2': { 'tb166v': 0,
                     'tb166h': 1,
                     'tb183_3v': 2,
                     'tb183_7v': 3}
         }

geoips.plugins.modules.readers.gmi_hdf5.call(fnames, metadata_only=False, chans=None, area_def=None, self_register=False)[source]#

Read GMI hdf5 data products.

Parameters:

• fnames (list) –

  • List of strings, full paths to files

• metadata_only (bool, default=False) –

  • NOT YET IMPLEMENTED

  • Return before actually reading data if True

• chans (list of str, default=None) –

  • NOT YET IMPLEMENTED

  • List of desired channels (skip unneeded variables as needed).

  • Include all channels if None.

• area_def (pyresample.AreaDefinition, default=None) –

  • NOT YET IMPLEMENTED

  • Specify region to read

  • Read all data if None.

• self_register (str or bool, default=False) –

  • NOT YET IMPLEMENTED

  • register all data to the specified dataset id (as specified in the return dictionary keys).

  • Read multiple resolutions of data if False.

Returns:

• dictionary of xarray.Dataset objects with required Variables and Attributes.

• Dictionary keys can be any descriptive dataset ids.

Return type:

dict of xarray.Datasets

See also

Xarray and NetCDF Metadata Standards

Additional information regarding required attributes and variables for GeoIPS-formatted xarray Datasets.

geoips.plugins.modules.readers.gmi_hdf5.get_test_files(test_data_dir)[source]#

Yeilds unit files and test xarray for unit testing.

geoips.plugins.modules.readers.gmi_hdf5.get_test_parameters()[source]#

Yeilds data key for unit testing.

geoips.plugins.modules.readers.gmi_hdf5.read_gmi_file(fname, xarray_gmi)[source]#

Read a single GMI file fname.

geoips.plugins.modules.readers.imerg_hdf5 module#

Read IMERG rainfall data.

A reader is designed to import IMERG rainfall data for GeoIPS using only python libraries

Aug 17, 2020

for a IMERG 30min data file, the time is the start time of a 30min interval:

0000-0030-0100-0130-0200 ….

Dataset information:

Spatial resolution is 0.1 deg.
1st grid is at (-179.95, -89.95).
Last grid is at (175.95, 89.95)
variable array is (3600,1800)

 metadata['top']['dataprovider'] = 'NASA-GPM'

 dataset_info = { 'Grid': {'MWtime': 'HQobservationTime',
                           'MWid': 'HQprecipSource',
                           'MWrr': 'HQprecipitation',
                           'IRweight': 'IRkalmanFilterWeight',
                           'IRrr': 'IRprecipitation',
                           'rain': 'precipitationCal',
                           'rrQC': 'precipitationQualityIndex',
                           'rrUncal': 'precipitationUncal',
                           'rrProb': 'probabilityLiquidPrecipitation',
                           'rrErr': 'randomError',},
          }

geoips.plugins.modules.readers.imerg_hdf5.call(fnames, metadata_only=False, chans=None, area_def=None, self_register=False)[source]#

Read IMERG hdf5 rain rate data products.

Parameters:

• fnames (list) –

  • List of strings, full paths to files

• metadata_only (bool, default=False) –

  • NOT YET IMPLEMENTED

  • Return before actually reading data if True

• chans (list of str, default=None) –

  • NOT YET IMPLEMENTED

  • List of desired channels (skip unneeded variables as needed).

  • Include all channels if None.

• area_def (pyresample.AreaDefinition, default=None) –

  • NOT YET IMPLEMENTED

  • Specify region to read

  • Read all data if None.

• self_register (str or bool, default=False) –

  • NOT YET IMPLEMENTED

  • register all data to the specified dataset id (as specified in the return dictionary keys).

  • Read multiple resolutions of data if False.

Returns:

• dictionary of xarray.Dataset objects with required Variables and Attributes.

• Dictionary keys can be any descriptive dataset ids.

Return type:

dict of xarray.Datasets

See also

Xarray and NetCDF Metadata Standards

Additional information regarding required attributes and variables for GeoIPS-formatted xarray Datasets.

geoips.plugins.modules.readers.mimic_netcdf module#

MIMIC TPW NetCDF reader.

geoips.plugins.modules.readers.mimic_netcdf.call(fnames, metadata_only=False, chans=None, area_def=None, self_register=False)[source]#

Read TPW MIMIC data from a list of filenames.

Dataset information:

<xarray.Dataset>
Dimensions:             (lat: 721, lon: 1440)
Dimensions without coordinates: lat, lon
Data variables:
    lonArr              (lon) float32 ...
    latArr              (lat) float32 ...
    tpwGrid             (lat, lon) float32 ...
    tpwGridPrior        (lat, lon) float32 ...
    tpwGridSubseq       (lat, lon) float32 ...
    timeAwayGridPrior   (lat, lon) timedelta64[ns] ...
    timeAwayGridSubseq  (lat, lon) timedelta64[ns] ...
    footGridPrior       (lat, lon) float32 ...
    footGridSubseq      (lat, lon) float32 ...
    satGridPrior        (lat, lon) uint8 ...
    satGridSubseq       (lat, lon) uint8 ...

Parameters:

• fnames (list) –

  • List of strings, full paths to files

• metadata_only (bool, default=False) –

  • Return before actually reading data if True

• chans (list of str, default=None) –

  • NOT YET IMPLEMENTED

  • List of desired channels (skip unneeded variables as needed).

  • Include all channels if None.

• area_def (pyresample.AreaDefinition, default=None) –

  • NOT YET IMPLEMENTED

  • Specify region to read

  • Read all data if None.

• self_register (str or bool, default=False) –

  • NOT YET IMPLEMENTED

  • register all data to the specified dataset id (as specified in the return dictionary keys).

  • Read multiple resolutions of data if False.

Returns:

• dictionary of xarray.Dataset objects with required Variables and Attributes.

• Dictionary keys can be any descriptive dataset ids.

Return type:

dict of xarray.Datasets

See also

Xarray and NetCDF Metadata Standards

Additional information regarding required attributes and variables for GeoIPS-formatted xarray Datasets.

geoips.plugins.modules.readers.modis_hdf4 module#

MODIS HDF4 reader.

This reader is designed for geoips for importing MODIS data files in hdf4 format Example files are:

AQUA:      MYD files
     MYD021KM.A2021004.2005.061.NRT.hdf
     MYD03.A2021004.2005.061.NRT.hdf
     MYD14.A2021004.2005.006.NRT.hdf
Terra:     MOD files
     MOD021KM.A2021004.2005.061.NRT.hdf
     MOD02HKM.A2021004.2005.061.NRT.hdf
     MOD02QKM.A2021004.2005.061.NRT.hdf
     MOD03.A2021004.2005.061.NRT.hdf
     MOD14.A2021004.2005.006.NRT.hdf

The MOD03 and MOD14 files have the geolocation (lat/lon) and sensor geoometry infomation, while other files have values at each channels.

geoips.plugins.modules.readers.modis_hdf4.add_to_xarray(varname, nparr, xobj, cumulative_mask, data_type)[source]#

Add variable to xarray Dataset.

geoips.plugins.modules.readers.modis_hdf4.call(fnames, metadata_only=False, chans=None, area_def=None, self_register=False)[source]#

Read MODIS Aqua and Terra hdf data files.

Parameters:

• fnames (list) –

  • List of strings, full paths to files

• metadata_only (bool, default=False) –

  • Return before actually reading data if True

• chans (list of str, default=None) –

  • List of desired channels (skip unneeded variables as needed).

  • Include all channels if None.

• area_def (pyresample.AreaDefinition, default=None) –

  • NOT YET IMPLEMENTED

  • Specify region to read

  • Read all data if None.

• self_register (str or bool, default=False) –

  • NOT YET IMPLEMENTED

  • register all data to the specified dataset id (as specified in the return dictionary keys).

  • Read multiple resolutions of data if False.

Returns:

• dictionary of xarray.Dataset objects with required Variables and Attributes.

• Dictionary keys can be any descriptive dataset ids.

Return type:

dict of xarray.Datasets

See also

Xarray and NetCDF Metadata Standards

Additional information regarding required attributes and variables for GeoIPS-formatted xarray Datasets.

geoips.plugins.modules.readers.modis_hdf4.parse_archive_metadata(metadata, metadatastr)[source]#

Parse archive metadata.

geoips.plugins.modules.readers.modis_hdf4.parse_core_metadata(metadata, metadatastr)[source]#

Parse core metadata.

geoips.plugins.modules.readers.modis_hdf4.parse_metadata(metadatadict)[source]#

Parse MODIS metadata dictionary.

geoips.plugins.modules.readers.modis_hdf4.parse_struct_metadata(metadata, metadatastr)[source]#

Parse metadata struct.

geoips.plugins.modules.readers.saphir_hdf5 module#

Read SAPHIR hdf files.

geoips.plugins.modules.readers.saphir_hdf5.call(fnames, metadata_only=False, chans=None, area_def=None, self_register=False)[source]#

Read SAPHIR hdf data products.

Parameters:

• fnames (list) –

  • List of strings, full paths to files

• metadata_only (bool, default=False) –

  • NOT YET IMPLEMENTED

  • Return before actually reading data if True

• chans (list of str, default=None) –

  • NOT YET IMPLEMENTED

  • List of desired channels (skip unneeded variables as needed).

  • Include all channels if None.

• area_def (pyresample.AreaDefinition, default=None) –

  • NOT YET IMPLEMENTED

  • Specify region to read

  • Read all data if None.

• self_register (str or bool, default=False) –

  • NOT YET IMPLEMENTED

  • register all data to the specified dataset id (as specified in the return dictionary keys).

  • Read multiple resolutions of data if False.

Returns:

• dictionary of xarray.Dataset objects with required Variables and Attributes.

• Dictionary keys can be any descriptive dataset ids.

Return type:

dict of xarray.Datasets

See also

Xarray and NetCDF Metadata Standards

Additional information regarding required attributes and variables for GeoIPS-formatted xarray Datasets.

geoips.plugins.modules.readers.sar_winds_netcdf module#

Read derived surface winds from SAR netcdf data.

geoips.plugins.modules.readers.sar_winds_netcdf.call(fnames, metadata_only=False, chans=None, area_def=None, self_register=False)[source]#

Read SAR derived winds from netcdf data.

Parameters:

• fnames (list) –

  • List of strings, full paths to files

• metadata_only (bool, default=False) –

  • NOT YET IMPLEMENTED

  • Return before actually reading data if True

• chans (list of str, default=None) –

  • NOT YET IMPLEMENTED

  • List of desired channels (skip unneeded variables as needed).

  • Include all channels if None.

• area_def (pyresample.AreaDefinition, default=None) –

  • NOT YET IMPLEMENTED

  • Specify region to read

  • Read all data if None.

• self_register (str or bool, default=False) –

  • NOT YET IMPLEMENTED

  • register all data to the specified dataset id (as specified in the return dictionary keys).

  • Read multiple resolutions of data if False.

Returns:

• dictionary of xarray.Dataset objects with required Variables and Attributes.

• Dictionary keys can be any descriptive dataset ids.

Return type:

dict of xarray.Datasets

See also

Xarray and NetCDF Metadata Standards

Additional information regarding required attributes and variables for GeoIPS-formatted xarray Datasets.

geoips.plugins.modules.readers.sar_winds_netcdf.get_test_files(test_data_dir)[source]#

Generate testing xarray from test data.

geoips.plugins.modules.readers.sar_winds_netcdf.get_test_parameters()[source]#

Generate test data key for unit testing.

geoips.plugins.modules.readers.sar_winds_netcdf.read_sar_data(wind_xarray)[source]#

Reformat SAR xarray object appropriately.

• variables: latitude, longitude, time, wind_speed_kts

• attributes: source_name, platform_name, data_provider, interpolation_radius_of_influence

geoips.plugins.modules.readers.scat_knmi_winds_netcdf module#

Read derived surface winds from KNMI scatterometer netcdf data.

geoips.plugins.modules.readers.scat_knmi_winds_netcdf.call(fnames, metadata_only=False, chans=None, area_def=None, self_register=False)[source]#

Read KNMI scatterometer derived winds from netcdf data.

Parameters:

• fnames (list) –

  • List of strings, full paths to files

• metadata_only (bool, default=False) –

  • NOT YET IMPLEMENTED

  • Return before actually reading data if True

• chans (list of str, default=None) –

  • NOT YET IMPLEMENTED

  • List of desired channels (skip unneeded variables as needed).

  • Include all channels if None.

• area_def (pyresample.AreaDefinition, default=None) –

  • NOT YET IMPLEMENTED

  • Specify region to read

  • Read all data if None.

• self_register (str or bool, default=False) –

  • NOT YET IMPLEMENTED

  • register all data to the specified dataset id (as specified in the return dictionary keys).

  • Read multiple resolutions of data if False.

Returns:

• dictionary of xarray.Dataset objects with required Variables and Attributes.

• Dictionary keys can be any descriptive dataset ids.

Return type:

dict of xarray.Datasets

See also

Xarray and NetCDF Metadata Standards

Additional information regarding required attributes and variables for GeoIPS-formatted xarray Datasets.

geoips.plugins.modules.readers.scat_knmi_winds_netcdf.get_test_files(test_data_dir)[source]#

Generate test xarray from test data for unit testing.

geoips.plugins.modules.readers.scat_knmi_winds_netcdf.get_test_parameters()[source]#

Generate test data key for unit testing.

geoips.plugins.modules.readers.scat_knmi_winds_netcdf.read_knmi_data(wind_xarray)[source]#

Reformat ascat xarray object appropriately.

• variables: latitude, longitude, time, wind_speed_kts, wind_dir_deg_met

• attributes: source_name, platform_name, data_provider, interpolation_radius_of_influence

geoips.plugins.modules.readers.scat_noaa_winds_netcdf module#

Read derived surface winds from KNMI scatterometer netcdf data.

geoips.plugins.modules.readers.scat_noaa_winds_netcdf.call(fnames, metadata_only=False, chans=None, area_def=None, self_register=False)[source]#

Read KNMI scatterometer derived winds from netcdf data.

Parameters:

• fnames (list) –

  • List of strings, full paths to files

• metadata_only (bool, default=False) –

  • NOT YET IMPLEMENTED

  • Return before actually reading data if True

• chans (list of str, default=None) –

  • NOT YET IMPLEMENTED

  • List of desired channels (skip unneeded variables as needed).

  • Include all channels if None.

• area_def (pyresample.AreaDefinition, default=None) –

  • NOT YET IMPLEMENTED

  • Specify region to read

  • Read all data if None.

• self_register (str or bool, default=False) –

  • NOT YET IMPLEMENTED

  • register all data to the specified dataset id (as specified in the return dictionary keys).

  • Read multiple resolutions of data if False.

Returns:

• dictionary of xarray.Dataset objects with required Variables and Attributes.

• Dictionary keys can be any descriptive dataset ids.

Return type:

dict of xarray.Datasets

See also

Xarray and NetCDF Metadata Standards

Additional information regarding required attributes and variables for GeoIPS-formatted xarray Datasets.

geoips.plugins.modules.readers.scat_noaa_winds_netcdf.get_test_files(test_data_dir)[source]#

Generate test xarray from test files for unit testing.

geoips.plugins.modules.readers.scat_noaa_winds_netcdf.get_test_parameters()[source]#

Generate test data key for unit testing.

geoips.plugins.modules.readers.scat_noaa_winds_netcdf.read_noaa_data(wind_xarray)[source]#

Reformat ascat xarray object appropriately.

• variables: latitude, longitude, time, wind_speed_kts, wind_dir_deg_met

• attributes: source_name, platform_name, data_provider, interpolation_radius_of_influence

geoips.plugins.modules.readers.seviri_hrit module#

Read SEVIRI hrit data.

Notes

1. At present, this reader does not work for High Resolution Visible data, which is ignored. Additionally, to ease generation of geolocation fields, datasets are assumed to be square and centered at their sub longitude.

20170330 MLS Try to only decompress what we need (VERY filename dependent),

make scifile and hrit channel names match (more filename dependence), don’t try to decompress/open file for import_metadata (more filename dependence for time). satpy requires time to open file, and requires standard (decompressed) filenames, so built in filename dependence by using satpy.

class geoips.plugins.modules.readers.seviri_hrit.Chan(name)[source]#

Bases: object

Channel class.

property band#

Band property.

property band_num#

Band number property.

property name#

Name property.

property type#

Type property.

class geoips.plugins.modules.readers.seviri_hrit.ChanList(chans)[source]#

Bases: object

ChanList Class.

property bands#

Bands property.

property chans#

Chans property.

property names#

Names property.

exception geoips.plugins.modules.readers.seviri_hrit.XritError(msg, code=None)[source]#

Bases: Exception

Raise exception on XritError.

geoips.plugins.modules.readers.seviri_hrit.calculate_chebyshev_polynomial(coefs, start_dt, end_dt, dt)[source]#

Calculate Chebyshev Polynomial.

geoips.plugins.modules.readers.seviri_hrit.call(fnames, metadata_only=False, chans=None, area_def=None, self_register=False)[source]#

Read SEVIRI hrit data products.

Parameters:

• fnames (list) –

  • List of strings, full paths to files

• metadata_only (bool, default=False) –

  • Return before actually reading data if True

• chans (list of str, default=None) –

  • List of desired channels (skip unneeded variables as needed).

  • Include all channels if None.

• area_def (pyresample.AreaDefinition, default=None) –

  • Specify region to read

  • Read all data if None.

• self_register (str or bool, default=False) –

  • register all data to the specified dataset id (as specified in the return dictionary keys).

  • Read multiple resolutions of data if False.

Returns:

• dictionary of xarray.Dataset objects with required Variables and Attributes.

• Dictionary keys can be any descriptive dataset ids.

Return type:

dict of xarray.Datasets

See also

Xarray and NetCDF Metadata Standards

Additional information regarding required attributes and variables for GeoIPS-formatted xarray Datasets.

geoips.plugins.modules.readers.seviri_hrit.call_single_time(fnames, metadata_only=False, chans=None, area_def=None, self_register=False)[source]#

Read SEVIRI hrit data products.

Parameters:

• fnames (list) –

  • List of strings, full paths to files

• metadata_only (bool, default=False) –

  • Return before actually reading data if True

• chans (list of str, default=None) –

  • List of desired channels (skip unneeded variables as needed).

  • Include all channels if None.

• area_def (pyresample.AreaDefinition, default=None) –

  • Specify region to read

  • Read all data if None.

• self_register (str or bool, default=False) –

  • register all data to the specified dataset id (as specified in the return dictionary keys).

  • Read multiple resolutions of data if False.

Returns:

• dictionary of xarray.Dataset objects with required Variables and Attributes.

• Dictionary keys can be any descriptive dataset ids.

Return type:

dict of xarray.Datasets

See also

Xarray and NetCDF Metadata Standards

Additional information regarding required attributes and variables for GeoIPS-formatted xarray Datasets.

geoips.plugins.modules.readers.seviri_hrit.compare_dicts(d1, d2, skip=None)[source]#

Compare the values in two dictionaries.

If they are equal, return True, otherwise False If skip is set and contains one of the keys, skip that key

geoips.plugins.modules.readers.seviri_hrit.countsToRad(counts, slope, offset)[source]#

Convert counts to rad.

geoips.plugins.modules.readers.seviri_hrit.get_latitude_longitude(gmd, BADVALS, area_def)[source]#

Generate full-disk latitudes and longitudes.

geoips.plugins.modules.readers.seviri_hrit.get_top_level_metadata(fnames, sect)[source]#

Get top level metadata.

geoips.plugins.modules.readers.seviri_hrit.radToBT(rad, platform, band)[source]#

Convert rad to BT.

geoips.plugins.modules.readers.seviri_hrit.radToRef(rad, sun_zen, platform, band)[source]#

Convert Rad to Ref.