geoips.plugins.modules.readers package#
Subpackages#
- geoips.plugins.modules.readers.utils package
- Submodules
- geoips.plugins.modules.readers.utils.geostationary_geolocation module
- geoips.plugins.modules.readers.utils.hrit_reader module
HritDtypeHritErrorHritFileHritFile.annotation_metadataHritFile.bandHritFile.basenameHritFile.block_infoHritFile.block_mapHritFile.compressedHritFile.decompress()HritFile.dirnameHritFile.epilogueHritFile.file_typeHritFile.geolocation_metadataHritFile.metadataHritFile.nameHritFile.prologueHritFile.segmentHritFile.start_datetime
read10bit()
- geoips.plugins.modules.readers.utils.remss_reader module
- Module contents
Submodules#
geoips.plugins.modules.readers.abi_l2_netcdf module#
ABI Level 2 NetCDF reader.
- geoips.plugins.modules.readers.abi_l2_netcdf.calculate_abi_geolocation(metadata, area_def)[source]#
Calculate ABI geolocation.
- geoips.plugins.modules.readers.abi_l2_netcdf.call(fnames, area_def=None, metadata_only=False, chans=False, self_register=False)[source]#
Read ABI Level 2 NetCDF data from a list of filenames.
- Parameters:
fnames (list) –
List of strings, full paths to files
metadata_only (bool, default=False) –
Return before actually reading data if True
chans (list of str, default=None) –
List of desired channels (skip unneeded variables as needed).
Include all channels if None.
area_def (pyresample.AreaDefinition, default=None) –
Specify region to read
Read all data if None.
self_register (str or bool, default=False) –
register all data to the specified dataset id (as specified in the return dictionary keys).
Read multiple resolutions of data if False.
- Returns:
dictionary of xarray.Dataset objects with required Variables and Attributes.
Dictionary keys can be any descriptive dataset ids.
- Return type:
dict of xarray.Datasets
See also
- Xarray and NetCDF Metadata Standards
Additional information regarding required attributes and variables for GeoIPS-formatted xarray Datasets.
geoips.plugins.modules.readers.abi_netcdf module#
Standard GeoIPS xarray dictionary based ABI NetCDF data reader.
- geoips.plugins.modules.readers.abi_netcdf.call(fnames, metadata_only=False, chans=None, area_def=None, self_register=False)[source]#
Read ABI NetCDF data from a list of filenames.
- Parameters:
fnames (list) –
List of strings, full paths to files
metadata_only (bool, default=False) –
Return before actually reading data if True
chans (list of str, default=None) –
List of desired channels (skip unneeded variables as needed).
Include all channels if None.
area_def (pyresample.AreaDefinition, default=None) –
Specify region to read
Read all data if None.
self_register (str or bool, default=False) –
register all data to the specified dataset id (as specified in the return dictionary keys).
Read multiple resolutions of data if False.
- Returns:
dictionary of xarray.Dataset objects with required Variables and Attributes.
Dictionary keys can be any descriptive dataset ids.
- Return type:
dict of xarray.Datasets
See also
- Xarray and NetCDF Metadata Standards
Additional information regarding required attributes and variables for GeoIPS-formatted xarray Datasets.
- geoips.plugins.modules.readers.abi_netcdf.get_band_metadata(all_metadata)[source]#
Get band metadata.
This method basically just reformats the all_metadata dictionary that is set based on the metadata found in the netcdf object itself to reference channel names as opposed to filenames as the dictionary keys.
- geoips.plugins.modules.readers.abi_netcdf.get_data(md, gvars, rad=False, ref=False, bt=False)[source]#
Read data for a full channel’s worth of files.
geoips.plugins.modules.readers.ahi_hsd module#
Advanced Himawari Imager Data Reader.
- exception geoips.plugins.modules.readers.ahi_hsd.AutoGenError[source]#
Bases:
ExceptionRaise exception on geolocation autogeneration error.
- geoips.plugins.modules.readers.ahi_hsd.call(fnames, metadata_only=False, chans=None, area_def=None, self_register=False)[source]#
Read AHI HSD data data from a list of filenames.
- Parameters:
fnames (list) –
List of strings, full paths to files
metadata_only (bool, default=False) –
Return before actually reading data if True
chans (list of str, default=None) –
List of desired channels (skip unneeded variables as needed).
Include all channels if None.
area_def (pyresample.AreaDefinition, default=None) –
Specify region to read
Read all data if None.
self_register (str or bool, default=False) –
register all data to the specified dataset id (as specified in the return dictionary keys).
Read multiple resolutions of data if False.
- Returns:
dictionary of xarray.Dataset objects with required Variables and Attributes.
Dictionary keys can be any descriptive dataset ids.
- Return type:
dict of xarray.Datasets
See also
- Xarray and NetCDF Metadata Standards
Additional information regarding required attributes and variables for GeoIPS-formatted xarray Datasets.
- geoips.plugins.modules.readers.ahi_hsd.get_band_metadata(all_metadata)[source]#
Get band metadata.
This method basically just reformats the all_metadata dictionary that is set based on the metadata found in the netcdf object itself to reference channel names as opposed to filenames as the dictionary keys.
- geoips.plugins.modules.readers.ahi_hsd.get_data(md, gvars, rad=False, ref=False, bt=False, zoom=1.0)[source]#
Read data for a full channel’s worth of files.
- geoips.plugins.modules.readers.ahi_hsd.get_latitude_longitude(metadata, BADVALS, area_def=None)[source]#
Get latitudes and longitudes.
This routine accepts a dictionary containing metadata as read from an HSD format file and returns latitudes and longitudes for a full disk image.
Note: This code has been adapted from Dan Lindsey’s Fortran90 code. This was done in three steps that ultimately culminated in faster, but more difficult to understand code. If you plan to edit this, I recommend that you return to Dan’s original code, then explore the commented code here, then finally, look at the single-command statements that are currently being used.
- geoips.plugins.modules.readers.ahi_hsd.metadata_to_datetime(metadata, time_var='ob_start_time')[source]#
Use information from block_01 to get the image datetime.
- geoips.plugins.modules.readers.ahi_hsd.set_variable_metadata(xobj_attrs, band_metadata, dsname, varname)[source]#
Set variable metadata.
MLS 20180914 Setting xobj_attrs at the variable level for the associated channel metadata pulled from the actual netcdf file. This will now be accessible from the scifile object. Additionally, pull out specifically the band_wavelength and attach it to the _varinfo at the variable level - this is automatically pulled from the xobj_attrs dictionary and set in the variable._varinfo dictionary in scifile/scifile.py and scifile/containers.py (see empty_varinfo at the beginning of containers.py for dictionary fields that are automatically pulled from the appropriate location in the xobj_attrs dictionary and set on the _varinfo dictionary)
geoips.plugins.modules.readers.amsr2_netcdf module#
Read AMSR2 data products.
- geoips.plugins.modules.readers.amsr2_netcdf.call(fnames, metadata_only=False, chans=None, area_def=None, self_register=False)[source]#
Read AMSR2 netcdf data products.
- Parameters:
fnames (list) –
List of strings, full paths to files
metadata_only (bool, default=False) –
Return before actually reading data if True
chans (list of str, default=None) –
List of desired channels (skip unneeded variables as needed).
Include all channels if None.
area_def (pyresample.AreaDefinition, default=None) –
NOT YET IMPLEMENTED
Specify region to read
Read all data if None.
self_register (str or bool, default=False) –
NOT YET IMPLEMENTED
register all data to the specified dataset id (as specified in the return dictionary keys).
Read multiple resolutions of data if False.
- Returns:
dictionary of xarray.Dataset objects with required Variables and Attributes.
Dictionary keys can be any descriptive dataset ids.
- Return type:
dict of xarray.Datasets
See also
- Xarray and NetCDF Metadata Standards
Additional information regarding required attributes and variables for GeoIPS-formatted xarray Datasets.
- geoips.plugins.modules.readers.amsr2_netcdf.read_amsr_data(full_xarray, chans)[source]#
Read non-AMSR2_OCEAN data.
- geoips.plugins.modules.readers.amsr2_netcdf.read_amsr_mbt(full_xarray, varname, time_array=None)[source]#
Reformat AMSR xarray object appropriately.
variables: latitude, longitude, time, brightness temperature variables
attributes: source_name, platform_name, data_provider, interpolation_radius_of_influence
geoips.plugins.modules.readers.amsr2_remss_winds_netcdf module#
Read derived surface winds from REMSS AMSR netcdf data.
- geoips.plugins.modules.readers.amsr2_remss_winds_netcdf.call(fnames, metadata_only=False, chans=None, area_def=None, self_register=False)[source]#
Read REMSS AMSR2 derived winds from netcdf data.
- Parameters:
fnames (list) –
List of strings, full paths to files
metadata_only (bool, default=False) –
NOT YET IMPLEMENTED
Return before actually reading data if True
chans (list of str, default=None) –
NOT YET IMPLEMENTED
List of desired channels (skip unneeded variables as needed).
Include all channels if None.
area_def (pyresample.AreaDefinition, default=None) –
NOT YET IMPLEMENTED
Specify region to read
Read all data if None.
self_register (str or bool, default=False) –
NOT YET IMPLEMENTED
register all data to the specified dataset id (as specified in the return dictionary keys).
Read multiple resolutions of data if False.
- Returns:
dictionary of xarray.Dataset objects with required Variables and Attributes.
Dictionary keys can be any descriptive dataset ids.
- Return type:
dict of xarray.Datasets
See also
- Xarray and NetCDF Metadata Standards
Additional information regarding required attributes and variables for GeoIPS-formatted xarray Datasets.
geoips.plugins.modules.readers.amsub_hdf module#
Read AMSU-B and MHS passive microwave data files.
This reader is desgined for importing NOAA Advanced Microwave Sounding Unit (AMSU)-B/Microwave Humidity Sounder (HMS) and EUMETSAQT MHS data files in hdf format, such as
NPR.MHOP.NP.D20153.S2046.E2230.B5832021.NS (N19),
NPR.MHOP.NN.D20153.S1927.E2105.B7748081.NS (N18),
NPR.MHOP.M1.D20153.S2229.E2318.B3998282.NS (METOP).
V1.0: Initial version, NRL-MRY, June 1, 2020
Basic information on AMSU-B product file:
Input SD Variables
(nscan, npix):
npix=90 pixels per scan;
nscan: vary with orbit
chan-1 AT: 89 GHz as ch16 anttenna temperature at V-pol FOV 16km at nadir
chan-2 AT: 150 (157) GHz as ch17 the number in bracket is for MHS from
metops at V-pol, 16km at nadir
chan-3 AT: 183.31 +-1 GHz as ch18 at H-pol, 16km
chan-4 AT: 183.31 +-3 GHz as ch19 at H-pol, 16km
chan-5 AT: 183.31 +-7 (190.3) GHz as ch20 at V-pol, 16km
lat: -90, 90 deg
lon: -180, 180 deg
RR: surface rainrate (mm/hr)
Snow: surafce snow cover
IWP: ice water path (unit?)
SWE: snow water equvelent (unit)
Sfc_type: surface type
Orbit_mode: -1: ascending, 1: decending, 2: both
SFR: snowfall rate (unit?)
LZ_angle: local zinath angle (deg)
SZ_angle: solar zinath angle (deg)
Vdata info (definition of AMSU-B date):
ScanTime_year
ScanTime_month
ScanTime_day
ScanTime_hour
ScanTime_minute
ScanTime_second
ScanTime_Jday
Time
- geoips.plugins.modules.readers.amsub_hdf.call(fnames, metadata_only=False, chans=None, area_def=None, self_register=False)[source]#
Read AMSU-B hdf data products.
- Parameters:
fnames (list) –
List of strings, full paths to files
metadata_only (bool, default=False) –
NOT YET IMPLEMENTED
Return before actually reading data if True
chans (list of str, default=None) –
NOT YET IMPLEMENTED
List of desired channels (skip unneeded variables as needed).
Include all channels if None.
area_def (pyresample.AreaDefinition, default=None) –
NOT YET IMPLEMENTED
Specify region to read
Read all data if None.
self_register (str or bool, default=False) –
NOT YET IMPLEMENTED
register all data to the specified dataset id (as specified in the return dictionary keys).
Read multiple resolutions of data if False.
- Returns:
dictionary of xarray.Dataset objects with required Variables and Attributes.
Dictionary keys can be any descriptive dataset ids.
- Return type:
dict of xarray.Datasets
See also
- Xarray and NetCDF Metadata Standards
Additional information regarding required attributes and variables for GeoIPS-formatted xarray Datasets.
geoips.plugins.modules.readers.amsub_mirs module#
Read AMSU-B and MHS MIRS NetCDF data files.
This reader is desgined for importing the Advanced Microwave Sounding Unit (AMSU)-B/Microwave Humidity Sounder (HMS) and EUMETSAQT MHS data files in hdf5 from NOAA MIRS. These new data files have a different file name convention and and data structure from previous MSPPS. Example of MIRIS files:
NPR-MIRS-IMG_v11r4_ma1_s202101111916000_e202101112012000_c202101112047200.nc
(ma1 is for metop-B)
NPR-MIRS-IMG_v11r4_ma2_s202101111715000_e202101111857000_c202101111941370.nc
(ma2 is for metop-A)
NPR-MIRS-IMG_v11r4_n19_s202101111730000_e202101111916000_c202101112001590.nc
(NOAA-19)
AMSU-A channel information:
Chan# / Freq(GHz) / bands / Bandwidth(MHz) / Beamwidth(deg) / NE#T(K) /
(Spec.) Polarization at nadir / Instrument Component
1 23.800 1 270 3.3 0.30 V A2
2 31.400 1 180 3.3 0.30 V A2
3 50.300 1 180 3.3 0.40 V A1-2
4 52.800 1 400 3.3 0.25 V A1-2
5 53.596 +-115 2 170 3.3 0.25 H A1-2
6 54.400 1 400 3.3 0.25 H A1-1
7 54.940 1 400 3.3 0.25 V A1-1
8 55.500 1 330 3.3 0.25 H A1-2
9 f0=57,290.344 1 330 3.3 0.25 H A1-1
10 f0+-217 2 78 3.3 0.40 H A1-1
11 f0+-322.2+-48 4 36 3.3 0.40 H A1-1
12 f0+-322.2+-22 4 16 3.3 0.60 H A1-1
13 f0+-322.2+-10 4 8 3.3 0.80 H A1-1
14 f0+-322.2+-4.5 4 3 3.3 1.20 H A1-1
15 89,000 1 <6,000 3.3 0.50 V A1-1
AMSU-B/MHS channel information:
Channel / Centre Frequency (GHz) / Bandwidth (MHz) / NeDT (K) /
Calibration Accuracy (K) / pol. angle (degree)
16 89.0 <6000 1.0 1.0 90-q (Vertical pol)
17 150 <4000 1.0 1.0 90-q (Vertical)
18 183.31+-1.00 500 1.1 1.0 nospec (Horizontal)
19 183.31+-3.00 1000 1.0 1.0 nospec (Horizontal)
20 190.31+17.00 2000 1.2 1.0 90-q (Vertical)
Since AMSU-A sensor is no longer available, we select only the AMSU-B/MHS channels. We decide to use the same names of the five channels used for previous NOAA MSPPS data files. i.e., select frequench index 15-19 (start from 0).
V1.0: Initial version, NRL-MRY, January 26, 2021
Dataset information:
Basic information on AMSU-B product file
index: 1 2 3 4 5 6 7
freq: 23.8,31.4,50.3,52.799,53.595,54.4,54.941,
55.499,57.29,57.29,57.29,57.29,57.29,57.29,
index: 8 9 10 11 12 13 14
freq: 55.499,57.29,57.29,57.29,57.29,57.29,57.29,
index: 15 16 17 18 19 20
freq: 89.,89.,157.,183.311,183.311,190.311
dimensions:
Scanline = 2370 ;
Field_of_view = 90 ;
P_Layer = 100 ;
Channel = 20 ;
Qc_dim = 4 ;
variables:
Freq(Channel): Central Frequencies (GHz)
Polo(Channel): Polarizations
ScanTime_year(Scanline): Calendar Year 20XX
ScanTime_doy(Scanline: julian day 1-366
ScanTime_month(Scanline): Calendar month 1-12
ScanTime_dom(Scanline): Calendar day of the month 1-31
ScanTime_hour(Scanline: hour of the day 0-23
ScanTime_minute(Scanline): minute of the hour 0-59
ScanTime_second(Scanline): second of the minute 0-59
ScanTime_UTC(Scanline): Number of seconds since 00:00:00 UTC
Orb_mode(Scanline): 0-ascending,1-descending
Latitude(Scanline, Field_of_view):Latitude of the view (-90,90),
unit: degree
Longitude(Scanline, Field_of_view):Longitude of the view (-180,180),
unit: degree
Sfc_type(Scanline, Field_of_view):type of surface:0-ocean,1-sea ice,
2-land,3-snow
Atm_type(Scanline, Field_of_view): type of atmosphere:currently missing
( note: not needed for geoips products)
Qc(Scanline, Field_of_view, Qc_dim): Qc: 0-good, 1-usable with problem,
2-bad
ChiSqr(Scanline, Field_of_view): Convergence rate: <3-good,>10-bad
LZ_angle(Scanline, Field_of_view): Local Zenith Angle degree
RAzi_angle(Scanline, Field_of_view):Relative Azimuth Angle 0-360 degree
SZ_angle(Scanline, Field_of_view):Solar Zenith Angle (-90,90) degree
BT(Scanline, Field_of_view, Channel): Channel Temperature (K)
YM(Scanline, Field_of_view, Channel): UnCorrected Channel Temperature(K)
ChanSel(Scanline, Field_of_view, Channel):Channels Selection Used in
Retrieval
TPW(Scanline, Field_of_view): Total Precipitable Water (mm)
CLW(Scanline, Field_of_view):Cloud liquid Water (mm)
RWP(Scanline, Field_of_view): Rain Water Path (mm)
LWP(Scanline, Field_of_view): Liquid Water Path (mm)
SWP(Scanline, Field_of_view): Snow Water Path (mm)
IWP(Scanline, Field_of_view): Ice Water Path (mm)
GWP(Scanline, Field_of_view): Graupel Water Path (mm)
RR(Scanline, Field_of_view): Rain Rate (mm/hr)
Snow(Scanline, Field_of_view): Snow Cover (range: 0-1) i.e., 1 -> 100%
SWE(Scanline, Field_of_view): Snow Water Equivalent (cm)
SnowGS(Scanline, Field_of_view):Snow Grain Size (mm)
SIce(Scanline, Field_of_view):Sea Ice Concentration (%)
SIce_MY(Scanline, Field_of_view): Multi-Year Sea Ice Concentration (%)
SIce_FY(Scanline, Field_of_view): First-Year Sea Ice Concentration (%)
TSkin(Scanline, Field_of_view): Skin Temperature (K)
SurfP(Scanline, Field_of_view): Surface Pressure (mb)
Emis(Scanline, Field_of_view, Channel):Channel Emissivity
(unit:1, Emis:scale_factor = 0.0001)
SFR(Scanline, Field_of_view): Snow Fall Rate in mm/hr
CldTop(Scanline, Field_of_view): Cloud Top Pressure (scale_factor = 0.1)
CldBase(Scanline, Field_of_view): Cloud Base Pressure
(scale_factor = 0.1)
CldThick(Scanline, Field_of_view): Cloud Thickness (scale_factor = 0.1)
PrecipType(Scanline, Field_of_view): Precipitation Type (Frozen/Liquid)
RFlag(Scanline, Field_of_view): Rain Flag
SurfM(Scanline, Field_of_view): Surface Moisture (scale_factor = 0.1)
WindSp(Scanline, Field_of_view): Wind Speed (m/s) (scale_factor = 0.01
WindDir(Scanline, Field_of_view: Wind Direction (scale_factor = 0.01)
WindU(Scanline, Field_of_view):U-direction Wind Speed (m/s)
(scale_factor = 0.01)
WindV(Scanline, Field_of_view: V-direction Wind Speed (m/s)
(scale_factor = 0.01)
Prob_SF(Scanline, Field_of_view): Probability of falling snow (%)
Additional info:
Variables (nscan, npix): npix=90 pixels per scan; nscan: vary with orbit
chan-1 AT: 89 GHz as ch16 anttenna temperature at V-pol
FOV 16km at nadir
chan-2 AT: 150 (157) GHz as ch17 the number in bracket is for MHS
from metops at V-pol, 16km at nadir
chan-3 AT: 183.31 +-1 GHz as ch18 at H-pol, 16km
chan-4 AT: 183.31 +-3 GHz as ch19 at H-pol, 16km
chan-5 AT: 183.31 +-7 (190.3) GHz as ch20 at V-pol, 16km
lat: -90, 90 deg
lon: -180, 180 deg
RR: surface rainrate (mm/hr)
Snow: surafce snow cover
IWP: ice water path
SWE: snow water equvelent
Sfc_type: surface type
Orbit_mode: -1: ascending, 1: decending, 2: both
SFR: snowfall rate (unit?)
LZ_angle: local zinath angle (deg)
SZ_angle: solar zinath angle (deg)
- geoips.plugins.modules.readers.amsub_mirs.call(fnames, metadata_only=False, chans=None, area_def=None, self_register=False)[source]#
Read AMSU/MHS MIRS data products.
- Parameters:
fnames (list) –
List of strings, full paths to files
metadata_only (bool, default=False) –
Return before actually reading data if True
chans (list of str, default=None) –
List of desired channels (skip unneeded variables as needed).
Include all channels if None.
area_def (pyresample.AreaDefinition, default=None) –
Specify region to read
Read all data if None.
self_register (str or bool, default=False) –
NOT YET IMPLEMENTED
register all data to the specified dataset id (as specified in the return dictionary keys).
Read multiple resolutions of data if False.
- Returns:
dictionary of xarray.Dataset objects with required Variables and Attributes.
Dictionary keys can be any descriptive dataset ids.
- Return type:
dict of xarray.Datasets
See also
- Xarray and NetCDF Metadata Standards
Additional information regarding required attributes and variables for GeoIPS-formatted xarray Datasets.
geoips.plugins.modules.readers.ascat_uhr_netcdf module#
Read derived surface winds from BYU ASCAT UHR NetCDF data.
- geoips.plugins.modules.readers.ascat_uhr_netcdf.call(fnames, metadata_only=False, chans=None, area_def=None, self_register=False)[source]#
Read ASCAT UHR derived winds or normalized radar cross section data.
- Parameters:
fnames (list) –
List of strings, full paths to files
metadata_only (bool, default=False) –
NOT YET IMPLEMENTED
Return before actually reading data if True
chans (list of str, default=None) –
NOT YET IMPLEMENTED
List of desired channels (skip unneeded variables as needed).
Include all channels if None.
area_def (pyresample.AreaDefinition, default=None) –
NOT YET IMPLEMENTED
Specify region to read
Read all data if None.
self_register (str or bool, default=False) –
NOT YET IMPLEMENTED
register all data to the specified dataset id (as specified in the return dictionary keys).
Read multiple resolutions of data if False.
- Returns:
dictionary of xarray.Dataset objects with required Variables and Attributes.
Dictionary keys can be any descriptive dataset ids.
- Return type:
dict of xarray.Datasets
See also
- Xarray and NetCDF Metadata Standards
Additional information regarding required attributes and variables for GeoIPS-formatted xarray Datasets.
geoips.plugins.modules.readers.atms_hdf5 module#
Reader to read a grannual NOAA ATMS SDR TBs in h5 format.
Output variables in xarray object for geoips processing system
V0: August 25, 2021
The date is generated by the NOAA community satellite processing package (CSPP), developed at CIMSS
Example of ATMS file names:
'SATMS_j01_d20210809_t0959306_e1000023_b19296_fnmoc_ops.h5'
SDR TBs variables
'GATMO_j01_d20210809_t0959306_e1000023_b19296_fnmoc_ops.h5'
SDR Geolocation variables
Dataset info:
TB[12,96,22]: for each granuel
CHAN# Center-Freq(GHz) POL
1 23.8 V
2 31.4 V
3 50.3 H
4 51.76 H
5 52.8 H
6 53.596+-0.115 H
7 54.4 H
8 54.94 H
9 55.5 H
10 57.290(f0) H
11 f0 +-0.217 H
12 f0 +-0.322+-0.048 H
13 f0 +-0.322+-0.022 H
14 f0 +-0.322+-0.010 H
15 f0 +-0.322+-0.0045 H
16 88.2 V
17 165.5 H
18 183.1+-7 H
19 183.1+-4.5 H (FNMOC used this chan for 183 GHz image)
20 183.1+-3.0 H
21 183.1+-1.8 H
22 183.1+-1.0 H
BeamTime[12,96]: microsecond, i.e., 1*10^-6. IET "IDPS Epoch Time" is used
It is a signed 64-bit integer giving microseconds since
00:00:00.000000 Jan 1 1958.
BrightnessTemperatureFactors[2]: 1: scale (unitless); 2: offset (K)
BrightnessTemperature[12,96,22]: [scan,pix,chans]
SDR geolocation Info
Latitude/Longitude[12,96]: for Chan 17 only (for initial product,
it is used for all channels)
BeamLatitude[12,96,5]: for chan 1,2,3,16,17. They will be used for
associated TBs at a later date.
BeamLongitude[12,96,5]: for chan 1,2,3,16,17
SatelliteAzimuthAngle, SatelliteZenithAngle,
SolarAzimuthAngle, SolarZenithAngle[12,96]
Notes
Unix epoch time is defined as the number of seconds that have elapsed since January 1, 1970 (midnight UTC/GMT). Thus, there is a 12 years difference for the JPSS data when datetime.datetime.utcfromtimestamp(epoch) is used to convert the JPSS IDPS Epoch time to the humman-readable date.
This reader is developed to read one granual a time from ATMS npp and jpss-1(n20) data files.
The example files are:
SATMS_j01_d20210809_t0959306_e1000023_b19296_fnmoc_ops.h5: for TBs. ‘b’: orbit#GATMO_j01_d20210809_t0959306_e1000023_b19296_fnmoc_ops.h5: for geolocations
- geoips.plugins.modules.readers.atms_hdf5.call(fnames, metadata_only=False, chans=None, area_def=None, self_register=False)[source]#
Read ATMS hdf5 data products.
- Parameters:
fnames (list) –
List of strings, full paths to files
metadata_only (bool, default=False) –
NOT YET IMPLEMENTED
Return before actually reading data if True
chans (list of str, default=None) –
NOT YET IMPLEMENTED
List of desired channels (skip unneeded variables as needed).
Include all channels if None.
area_def (pyresample.AreaDefinition, default=None) –
NOT YET IMPLEMENTED
Specify region to read
Read all data if None.
self_register (str or bool, default=False) –
NOT YET IMPLEMENTED
register all data to the specified dataset id (as specified in the return dictionary keys).
Read multiple resolutions of data if False.
- Returns:
dictionary of xarray.Dataset objects with required Variables and Attributes.
Dictionary keys can be any descriptive dataset ids.
- Return type:
dict of xarray.Datasets
See also
- Xarray and NetCDF Metadata Standards
Additional information regarding required attributes and variables for GeoIPS-formatted xarray Datasets.
- geoips.plugins.modules.readers.atms_hdf5.convert_epoch_to_datetime64(time_array, use_shape=None)[source]#
Convert time to datetime object.
- Parameters:
time_array (array) – Array of start time integers (multiplied by 1e-6 in function)
use_shape (tuple, optional) – desired output shape of time array, by default None
- Returns:
array of converted datetime objects
- Return type:
array
geoips.plugins.modules.readers.ewsg_netcdf module#
Read EWS-G data.
This EWS-G(Electro-Optical Infrared Weather System - Geostationary) reader is designed for reading theEWS-G data files (EWS-G is renamed from GOES-13). The reader is only using the python functions and xarray variables. The reader is based on EWS-G data in netcdf4 format.
V1.0: NRL-Monterey, 02/25/2021
EWS-G file information:
Example of the gvar filename: 2020.1212.0012.goes-13.gvar.nc
Note that channel-3 is not available for EWS-G.
gvar_Ch3(TIR=5.8-7.3um, ctr=6.48um,4km): unit=temp-deg(C), scale_factor=0.01
variables:
gvar_Ch1(VIS=0.55-0.75um, ctr=0.65um,1km): unit=albedo*100, scale_factor=0.01
gvar_Ch2(MWIR=3.8-4.0um, ctr=3.9um, 4km): unit=temp-deg(C), scale_factor=0.01
gvar_Ch4(TIR=10.2-11.2um, ctr=10.7um,4km): unit=temp-deg(C), scale_factor=0.01
gvar_Ch6(TIR=12.9-13.7um, ctr=13.3um 4km): unit=temp-deg(C), scale_factor=0.01
latitude: unit=degree
longitude:unit=degree
sat_zenith: unit=degree
sun_zenith: unit=degree
rel_azimuth:unit=degree
variable array definition: var(scan,pix); scan-->lines, pix-->samples
attributes: many
- geoips.plugins.modules.readers.ewsg_netcdf.call(fnames, metadata_only=False, chans=None, area_def=None, self_register=False)[source]#
Read EWS-G data in netcdf4 format.
- Parameters:
fnames (list) –
List of strings, full paths to files
metadata_only (bool, default=False) –
NOT YET IMPLEMENTED
Return before actually reading data if True
chans (list of str, default=None) –
NOT YET IMPLEMENTED
List of desired channels (skip unneeded variables as needed).
Include all channels if None.
area_def (pyresample.AreaDefinition, default=None) –
NOT YET IMPLEMENTED
Specify region to read
Read all data if None.
self_register (str or bool, default=False) –
NOT YET IMPLEMENTED
register all data to the specified dataset id (as specified in the return dictionary keys).
Read multiple resolutions of data if False.
- Returns:
dictionary of xarray.Dataset objects with required Variables and Attributes.
Dictionary keys can be any descriptive dataset ids.
- Return type:
dict of xarray.Datasets
See also
- Xarray and NetCDF Metadata Standards
Additional information regarding required attributes and variables for GeoIPS-formatted xarray Datasets.
geoips.plugins.modules.readers.geoips_netcdf module#
Read pre-processed GeoIPS-formatted NetCDF data.
- geoips.plugins.modules.readers.geoips_netcdf.call(fnames, metadata_only=False, chans=None, area_def=None, self_register=False)[source]#
Read preprocessed geoips netcdf output.
- Parameters:
fnames (list) –
List of strings, full paths to files
metadata_only (bool, default=False) –
NOT YET IMPLEMENTED
Return before actually reading data if True
chans (list of str, default=None) –
NOT YET IMPLEMENTED
List of desired channels (skip unneeded variables as needed).
Include all channels if None.
area_def (pyresample.AreaDefinition, default=None) –
NOT YET IMPLEMENTED
Specify region to read
Read all data if None.
self_register (str or bool, default=False) –
NOT YET IMPLEMENTED
register all data to the specified dataset id (as specified in the return dictionary keys).
Read multiple resolutions of data if False.
- Returns:
dictionary of xarray.Dataset objects with required Variables and Attributes.
Dictionary keys can be any descriptive dataset ids.
- Return type:
dict of xarray.Datasets
See also
- Xarray and NetCDF Metadata Standards
Additional information regarding required attributes and variables for GeoIPS-formatted xarray Datasets.
geoips.plugins.modules.readers.gmi_hdf5 module#
Read NASA GPM GMI hdf5 data files.
Read a grannual NASA GPM GMI TBs in h5 format (each grannual file is about 5 minutes GPM GMI data)
Output variables in xarray object for geoips processing system
V0: August 4, 2020
Dataset information:
variables in original TBs structure format
tb_info = { 'S1': { 'tb10v': 0,
'tb10h': 1,
'tb19v': 2,
'tb19h': 3,
'tb23v': 4,
'tb37v': 5,
'tb37h': 6,
'tb89v': 7,
'tb89h': 8,},
'S2': { 'tb166v': 0,
'tb166h': 1,
'tb183_3v': 2,
'tb183_7v': 3}
}
- geoips.plugins.modules.readers.gmi_hdf5.call(fnames, metadata_only=False, chans=None, area_def=None, self_register=False)[source]#
Read GMI hdf5 data products.
- Parameters:
fnames (list) –
List of strings, full paths to files
metadata_only (bool, default=False) –
NOT YET IMPLEMENTED
Return before actually reading data if True
chans (list of str, default=None) –
NOT YET IMPLEMENTED
List of desired channels (skip unneeded variables as needed).
Include all channels if None.
area_def (pyresample.AreaDefinition, default=None) –
NOT YET IMPLEMENTED
Specify region to read
Read all data if None.
self_register (str or bool, default=False) –
NOT YET IMPLEMENTED
register all data to the specified dataset id (as specified in the return dictionary keys).
Read multiple resolutions of data if False.
- Returns:
dictionary of xarray.Dataset objects with required Variables and Attributes.
Dictionary keys can be any descriptive dataset ids.
- Return type:
dict of xarray.Datasets
See also
- Xarray and NetCDF Metadata Standards
Additional information regarding required attributes and variables for GeoIPS-formatted xarray Datasets.
geoips.plugins.modules.readers.imerg_hdf5 module#
Read IMERG rainfall data.
A reader is designed to import IMERG rainfall data for GeoIPS using only python libraries
Aug 17, 2020
- for a IMERG 30min data file, the time is the start time of a 30min interval:
0000-0030-0100-0130-0200 ….
Dataset information:
Spatial resolution is 0.1 deg.
1st grid is at (-179.95, -89.95).
Last grid is at (175.95, 89.95)
variable array is (3600,1800)
metadata['top']['dataprovider'] = 'NASA-GPM'
dataset_info = { 'Grid': {'MWtime': 'HQobservationTime',
'MWid': 'HQprecipSource',
'MWrr': 'HQprecipitation',
'IRweight': 'IRkalmanFilterWeight',
'IRrr': 'IRprecipitation',
'rain': 'precipitationCal',
'rrQC': 'precipitationQualityIndex',
'rrUncal': 'precipitationUncal',
'rrProb': 'probabilityLiquidPrecipitation',
'rrErr': 'randomError',},
}
- geoips.plugins.modules.readers.imerg_hdf5.call(fnames, metadata_only=False, chans=None, area_def=None, self_register=False)[source]#
Read IMERG hdf5 rain rate data products.
- Parameters:
fnames (list) –
List of strings, full paths to files
metadata_only (bool, default=False) –
NOT YET IMPLEMENTED
Return before actually reading data if True
chans (list of str, default=None) –
NOT YET IMPLEMENTED
List of desired channels (skip unneeded variables as needed).
Include all channels if None.
area_def (pyresample.AreaDefinition, default=None) –
NOT YET IMPLEMENTED
Specify region to read
Read all data if None.
self_register (str or bool, default=False) –
NOT YET IMPLEMENTED
register all data to the specified dataset id (as specified in the return dictionary keys).
Read multiple resolutions of data if False.
- Returns:
dictionary of xarray.Dataset objects with required Variables and Attributes.
Dictionary keys can be any descriptive dataset ids.
- Return type:
dict of xarray.Datasets
See also
- Xarray and NetCDF Metadata Standards
Additional information regarding required attributes and variables for GeoIPS-formatted xarray Datasets.
geoips.plugins.modules.readers.mimic_netcdf module#
MIMIC TPW NetCDF reader.
- geoips.plugins.modules.readers.mimic_netcdf.call(fnames, metadata_only=False, chans=None, area_def=None, self_register=False)[source]#
Read TPW MIMIC data from a list of filenames.
Dataset information:
<xarray.Dataset> Dimensions: (lat: 721, lon: 1440) Dimensions without coordinates: lat, lon Data variables: lonArr (lon) float32 ... latArr (lat) float32 ... tpwGrid (lat, lon) float32 ... tpwGridPrior (lat, lon) float32 ... tpwGridSubseq (lat, lon) float32 ... timeAwayGridPrior (lat, lon) timedelta64[ns] ... timeAwayGridSubseq (lat, lon) timedelta64[ns] ... footGridPrior (lat, lon) float32 ... footGridSubseq (lat, lon) float32 ... satGridPrior (lat, lon) uint8 ... satGridSubseq (lat, lon) uint8 ...
- Parameters:
fnames (list) –
List of strings, full paths to files
metadata_only (bool, default=False) –
Return before actually reading data if True
chans (list of str, default=None) –
NOT YET IMPLEMENTED
List of desired channels (skip unneeded variables as needed).
Include all channels if None.
area_def (pyresample.AreaDefinition, default=None) –
NOT YET IMPLEMENTED
Specify region to read
Read all data if None.
self_register (str or bool, default=False) –
NOT YET IMPLEMENTED
register all data to the specified dataset id (as specified in the return dictionary keys).
Read multiple resolutions of data if False.
- Returns:
dictionary of xarray.Dataset objects with required Variables and Attributes.
Dictionary keys can be any descriptive dataset ids.
- Return type:
dict of xarray.Datasets
See also
- Xarray and NetCDF Metadata Standards
Additional information regarding required attributes and variables for GeoIPS-formatted xarray Datasets.
geoips.plugins.modules.readers.modis_hdf4 module#
MODIS HDF4 reader.
This reader is designed for geoips for importing MODIS data files in hdf4 format Example files are:
AQUA: MYD files
MYD021KM.A2021004.2005.061.NRT.hdf
MYD03.A2021004.2005.061.NRT.hdf
MYD14.A2021004.2005.006.NRT.hdf
Terra: MOD files
MOD021KM.A2021004.2005.061.NRT.hdf
MOD02HKM.A2021004.2005.061.NRT.hdf
MOD02QKM.A2021004.2005.061.NRT.hdf
MOD03.A2021004.2005.061.NRT.hdf
MOD14.A2021004.2005.006.NRT.hdf
The MOD03 and MOD14 files have the geolocation (lat/lon) and sensor geoometry infomation, while other files have values at each channels.
- geoips.plugins.modules.readers.modis_hdf4.add_to_xarray(varname, nparr, xobj, cumulative_mask, data_type)[source]#
Add variable to xarray Dataset.
- geoips.plugins.modules.readers.modis_hdf4.call(fnames, metadata_only=False, chans=None, area_def=None, self_register=False)[source]#
Read MODIS Aqua and Terra hdf data files.
- Parameters:
fnames (list) –
List of strings, full paths to files
metadata_only (bool, default=False) –
Return before actually reading data if True
chans (list of str, default=None) –
List of desired channels (skip unneeded variables as needed).
Include all channels if None.
area_def (pyresample.AreaDefinition, default=None) –
NOT YET IMPLEMENTED
Specify region to read
Read all data if None.
self_register (str or bool, default=False) –
NOT YET IMPLEMENTED
register all data to the specified dataset id (as specified in the return dictionary keys).
Read multiple resolutions of data if False.
- Returns:
dictionary of xarray.Dataset objects with required Variables and Attributes.
Dictionary keys can be any descriptive dataset ids.
- Return type:
dict of xarray.Datasets
See also
- Xarray and NetCDF Metadata Standards
Additional information regarding required attributes and variables for GeoIPS-formatted xarray Datasets.
- geoips.plugins.modules.readers.modis_hdf4.parse_archive_metadata(metadata, metadatastr)[source]#
Parse archive metadata.
- geoips.plugins.modules.readers.modis_hdf4.parse_core_metadata(metadata, metadatastr)[source]#
Parse core metadata.
geoips.plugins.modules.readers.saphir_hdf5 module#
Read SAPHIR hdf files.
- geoips.plugins.modules.readers.saphir_hdf5.call(fnames, metadata_only=False, chans=None, area_def=None, self_register=False)[source]#
Read SAPHIR hdf data products.
- Parameters:
fnames (list) –
List of strings, full paths to files
metadata_only (bool, default=False) –
NOT YET IMPLEMENTED
Return before actually reading data if True
chans (list of str, default=None) –
NOT YET IMPLEMENTED
List of desired channels (skip unneeded variables as needed).
Include all channels if None.
area_def (pyresample.AreaDefinition, default=None) –
NOT YET IMPLEMENTED
Specify region to read
Read all data if None.
self_register (str or bool, default=False) –
NOT YET IMPLEMENTED
register all data to the specified dataset id (as specified in the return dictionary keys).
Read multiple resolutions of data if False.
- Returns:
dictionary of xarray.Dataset objects with required Variables and Attributes.
Dictionary keys can be any descriptive dataset ids.
- Return type:
dict of xarray.Datasets
See also
- Xarray and NetCDF Metadata Standards
Additional information regarding required attributes and variables for GeoIPS-formatted xarray Datasets.
geoips.plugins.modules.readers.sar_winds_netcdf module#
Read derived surface winds from SAR netcdf data.
- geoips.plugins.modules.readers.sar_winds_netcdf.call(fnames, metadata_only=False, chans=None, area_def=None, self_register=False)[source]#
Read SAR derived winds from netcdf data.
- Parameters:
fnames (list) –
List of strings, full paths to files
metadata_only (bool, default=False) –
NOT YET IMPLEMENTED
Return before actually reading data if True
chans (list of str, default=None) –
NOT YET IMPLEMENTED
List of desired channels (skip unneeded variables as needed).
Include all channels if None.
area_def (pyresample.AreaDefinition, default=None) –
NOT YET IMPLEMENTED
Specify region to read
Read all data if None.
self_register (str or bool, default=False) –
NOT YET IMPLEMENTED
register all data to the specified dataset id (as specified in the return dictionary keys).
Read multiple resolutions of data if False.
- Returns:
dictionary of xarray.Dataset objects with required Variables and Attributes.
Dictionary keys can be any descriptive dataset ids.
- Return type:
dict of xarray.Datasets
See also
- Xarray and NetCDF Metadata Standards
Additional information regarding required attributes and variables for GeoIPS-formatted xarray Datasets.
geoips.plugins.modules.readers.scat_knmi_winds_netcdf module#
Read derived surface winds from KNMI scatterometer netcdf data.
- geoips.plugins.modules.readers.scat_knmi_winds_netcdf.call(fnames, metadata_only=False, chans=None, area_def=None, self_register=False)[source]#
Read KNMI scatterometer derived winds from netcdf data.
- Parameters:
fnames (list) –
List of strings, full paths to files
metadata_only (bool, default=False) –
NOT YET IMPLEMENTED
Return before actually reading data if True
chans (list of str, default=None) –
NOT YET IMPLEMENTED
List of desired channels (skip unneeded variables as needed).
Include all channels if None.
area_def (pyresample.AreaDefinition, default=None) –
NOT YET IMPLEMENTED
Specify region to read
Read all data if None.
self_register (str or bool, default=False) –
NOT YET IMPLEMENTED
register all data to the specified dataset id (as specified in the return dictionary keys).
Read multiple resolutions of data if False.
- Returns:
dictionary of xarray.Dataset objects with required Variables and Attributes.
Dictionary keys can be any descriptive dataset ids.
- Return type:
dict of xarray.Datasets
See also
- Xarray and NetCDF Metadata Standards
Additional information regarding required attributes and variables for GeoIPS-formatted xarray Datasets.
geoips.plugins.modules.readers.scat_noaa_winds_netcdf module#
Read derived surface winds from KNMI scatterometer netcdf data.
- geoips.plugins.modules.readers.scat_noaa_winds_netcdf.call(fnames, metadata_only=False, chans=None, area_def=None, self_register=False)[source]#
Read KNMI scatterometer derived winds from netcdf data.
- Parameters:
fnames (list) –
List of strings, full paths to files
metadata_only (bool, default=False) –
NOT YET IMPLEMENTED
Return before actually reading data if True
chans (list of str, default=None) –
NOT YET IMPLEMENTED
List of desired channels (skip unneeded variables as needed).
Include all channels if None.
area_def (pyresample.AreaDefinition, default=None) –
NOT YET IMPLEMENTED
Specify region to read
Read all data if None.
self_register (str or bool, default=False) –
NOT YET IMPLEMENTED
register all data to the specified dataset id (as specified in the return dictionary keys).
Read multiple resolutions of data if False.
- Returns:
dictionary of xarray.Dataset objects with required Variables and Attributes.
Dictionary keys can be any descriptive dataset ids.
- Return type:
dict of xarray.Datasets
See also
- Xarray and NetCDF Metadata Standards
Additional information regarding required attributes and variables for GeoIPS-formatted xarray Datasets.
geoips.plugins.modules.readers.seviri_hrit module#
Read SEVIRI hrit data.
Notes
At present, this reader does not work for High Resolution Visible data, which is ignored. Additionally, to ease generation of geolocation fields, datasets are assumed to be square and centered at their sub longitude.
- 20170330 MLS Try to only decompress what we need (VERY filename dependent),
make scifile and hrit channel names match (more filename dependence), don’t try to decompress/open file for import_metadata (more filename dependence for time). satpy requires time to open file, and requires standard (decompressed) filenames, so built in filename dependence by using satpy.
- class geoips.plugins.modules.readers.seviri_hrit.Chan(name)[source]#
Bases:
objectChannel class.
- property band#
Band property.
- property band_num#
Band number property.
- property name#
Name property.
- property type#
Type property.
- class geoips.plugins.modules.readers.seviri_hrit.ChanList(chans)[source]#
Bases:
objectChanList Class.
- property bands#
Bands property.
- property chans#
Chans property.
- property names#
Names property.
- exception geoips.plugins.modules.readers.seviri_hrit.XritError(msg, code=None)[source]#
Bases:
ExceptionRaise exception on XritError.
- geoips.plugins.modules.readers.seviri_hrit.calculate_chebyshev_polynomial(coefs, start_dt, end_dt, dt)[source]#
Calculate Chebyshev Polynomial.
- geoips.plugins.modules.readers.seviri_hrit.call(fnames, metadata_only=False, chans=None, area_def=None, self_register=False)[source]#
Read SEVIRI hrit data products.
- Parameters:
fnames (list) –
List of strings, full paths to files
metadata_only (bool, default=False) –
Return before actually reading data if True
chans (list of str, default=None) –
List of desired channels (skip unneeded variables as needed).
Include all channels if None.
area_def (pyresample.AreaDefinition, default=None) –
Specify region to read
Read all data if None.
self_register (str or bool, default=False) –
register all data to the specified dataset id (as specified in the return dictionary keys).
Read multiple resolutions of data if False.
- Returns:
dictionary of xarray.Dataset objects with required Variables and Attributes.
Dictionary keys can be any descriptive dataset ids.
- Return type:
dict of xarray.Datasets
See also
- Xarray and NetCDF Metadata Standards
Additional information regarding required attributes and variables for GeoIPS-formatted xarray Datasets.
- geoips.plugins.modules.readers.seviri_hrit.compare_dicts(d1, d2, skip=None)[source]#
Compare the values in two dictionaries.
If they are equal, return True, otherwise False If skip is set and contains one of the keys, skip that key
- geoips.plugins.modules.readers.seviri_hrit.countsToRad(counts, slope, offset)[source]#
Convert counts to rad.
- geoips.plugins.modules.readers.seviri_hrit.get_latitude_longitude(gmd, BADVALS, area_def)[source]#
Generate full-disk latitudes and longitudes.
geoips.plugins.modules.readers.sfc_winds_text module#
Read derived surface winds from SAR, SMAP, SMOS, and AMSR text data.
- geoips.plugins.modules.readers.sfc_winds_text.call(fnames, metadata_only=False, chans=None, area_def=None, self_register=False)[source]#
Read one of SAR, SMAP, SMOS, AMSR derived winds from text data.
- Parameters:
fnames (list) –
List of strings, full paths to files
metadata_only (bool, default=False) –
NOT YET IMPLEMENTED
Return before actually reading data if True
chans (list of str, default=None) –
NOT YET IMPLEMENTED
List of desired channels (skip unneeded variables as needed).
Include all channels if None.
area_def (pyresample.AreaDefinition, default=None) –
NOT YET IMPLEMENTED
Specify region to read
Read all data if None.
self_register (str or bool, default=False) –
NOT YET IMPLEMENTED
register all data to the specified dataset id (as specified in the return dictionary keys).
Read multiple resolutions of data if False.
- Returns:
dictionary of xarray.Dataset objects with required Variables and Attributes.
Dictionary keys can be any descriptive dataset ids.
- Return type:
dict of xarray.Datasets
See also
- Xarray and NetCDF Metadata Standards
Additional information regarding required attributes and variables for GeoIPS-formatted xarray Datasets.
geoips.plugins.modules.readers.smap_remss_winds_netcdf module#
Read derived surface winds from REMSS SMAP netcdf data.
- geoips.plugins.modules.readers.smap_remss_winds_netcdf.call(fnames, metadata_only=False, chans=None, area_def=None, self_register=False)[source]#
Read one of SMAP derived winds from netcdf data.
- Parameters:
fnames (list) –
List of strings, full paths to files
metadata_only (bool, default=False) –
NOT YET IMPLEMENTED
Return before actually reading data if True
chans (list of str, default=None) –
NOT YET IMPLEMENTED
List of desired channels (skip unneeded variables as needed).
Include all channels if None.
area_def (pyresample.AreaDefinition, default=None) –
NOT YET IMPLEMENTED
Specify region to read
Read all data if None.
self_register (str or bool, default=False) –
NOT YET IMPLEMENTED
register all data to the specified dataset id (as specified in the return dictionary keys).
Read multiple resolutions of data if False.
- Returns:
dictionary of xarray.Dataset objects with required Variables and Attributes.
Dictionary keys can be any descriptive dataset ids.
- Return type:
dict of xarray.Datasets
See also
- Xarray and NetCDF Metadata Standards
Additional information regarding required attributes and variables for GeoIPS-formatted xarray Datasets.
geoips.plugins.modules.readers.smos_winds_netcdf module#
Read derived surface winds from SAR, SMAP, SMOS, and AMSR netcdf data.
- geoips.plugins.modules.readers.smos_winds_netcdf.call(fnames, metadata_only=False, chans=None, area_def=None, self_register=False)[source]#
Read SMOS derived winds from netcdf data.
- Parameters:
fnames (list) –
List of strings, full paths to files
metadata_only (bool, default=False) –
NOT YET IMPLEMENTED
Return before actually reading data if True
chans (list of str, default=None) –
NOT YET IMPLEMENTED
List of desired channels (skip unneeded variables as needed).
Include all channels if None.
area_def (pyresample.AreaDefinition, default=None) –
NOT YET IMPLEMENTED
Specify region to read
Read all data if None.
self_register (str or bool, default=False) –
NOT YET IMPLEMENTED
register all data to the specified dataset id (as specified in the return dictionary keys).
Read multiple resolutions of data if False.
- Returns:
dictionary of xarray.Dataset objects with required Variables and Attributes.
Dictionary keys can be any descriptive dataset ids.
- Return type:
dict of xarray.Datasets
See also
- Xarray and NetCDF Metadata Standards
Additional information regarding required attributes and variables for GeoIPS-formatted xarray Datasets.
geoips.plugins.modules.readers.ssmi_binary module#
SSMI binary reader.
This SSMI reader is desgined for importing SSMI sdr data files (such as ssmi_orbital_sdrmi_f15_d20200427_s104500_e123100_r05323_cfnoc.def). This reader is created to read in TBs at 19 (V,H),22V, 37(V,H) and 85 (V,H) GHz channels. There are A/B scans for 85 GHz. The combined A/B scans will be used for TC imagery products at 85 GHz.
This GEOIPS python code is based on a SSMI SDR reader in C.
Convert SSMI_HIRES_AB for 85GHz and SSMI_LORES for 19-37GHz into xarray for GEOIPS framework
V1.0: Initial version, NRL-MRY, May 19, 2020
SSMI input data info:
pixels per scan:
LORES=64 for 19, 22, and 37 GHz channels;
HIRES=128 for 85 GHz channels
19V[LORES] FOV: 69km x 43km
19H[LORES]
22V[LORES] 50km x 40km
37V[LORES] 37km x 28km
37H[LORES]
85V[HIRES][2] [][0]: A scans; [][1]: B scans 15km x 13km
85H[HIRES][2]
-----header info-----
int32
cyr, cmon, cday, /* file creation date */
chr, cmin, /* file creation time */
scans, /* number of scans in file (from DataSeq) */
scid, /* spacecraft ID */
rev, /* nominal rev */
bjld, bhr, bmin, bsec, /* begin day of year (julian day), time=hr,min,sec */
ejld, ehr, emin, esec, /* ending day of year, time */
ajld, ahr, amin, asec, /* ascending node DOY, time */
lsat; /* logical satellite ID */
-----scan data-----
int32 scann; /* scan number (from ScanHdr1) */
int32 bst; /* B-scan start time (sec) scaled by 10000 */
double xtime; /* bst as seconds since 0z 1 Jan 1987 */
uint16
v19[LORES], /* TBs */
h19[LORES],
v22[LORES],
v37[LORES],
h37[LORES],
v85[HIRES][2],
h85[HIRES][2],
lon[HIRES][2]; /* longitudes */
int16 lat[HIRES][2]; /* latitudes */
char sft[HIRES][2]; /* surface types */
- geoips.plugins.modules.readers.ssmi_binary.call(fnames, metadata_only=False, chans=False, area_def=None, self_register=False)[source]#
Read SSMI FNMOC Binary Data.
- Parameters:
fnames (list) –
List of strings, full paths to files
metadata_only (bool, default=False) –
NOT YET IMPLEMENTED
Return before actually reading data if True
chans (list of str, default=None) –
NOT YET IMPLEMENTED
List of desired channels (skip unneeded variables as needed).
Include all channels if None.
area_def (pyresample.AreaDefinition, default=None) –
NOT YET IMPLEMENTED
Specify region to read
Read all data if None.
self_register (str or bool, default=False) –
NOT YET IMPLEMENTED
register all data to the specified dataset id (as specified in the return dictionary keys).
Read multiple resolutions of data if False.
- Returns:
dictionary of xarray.Dataset objects with required Variables and Attributes.
Dictionary keys can be any descriptive dataset ids.
- Return type:
dict of xarray.Datasets
See also
- Xarray and NetCDF Metadata Standards
Additional information regarding required attributes and variables for GeoIPS-formatted xarray Datasets.
geoips.plugins.modules.readers.ssmis_binary module#
SSMIS Binary reader.
This code is converted from geoips v1 into geoipd v2 framework. This new version of reader is indepent from the GEOIPS system whose environmental parametrs must be used in V1. Now, only python functipns are used with geoips framework and xarray is utilized to process datasets for product applications.
- Version Histopry:
V1: initial code, July 24, 2020, NRL-MRY
- Input File
SSMIS SDR data
- Output Fields
XARRAY onjectives to hold variables
- geoips.plugins.modules.readers.ssmis_binary.append_xarray_dicts(xobjs_list)[source]#
Append two dictionaries of xarray objects.
- geoips.plugins.modules.readers.ssmis_binary.call(fnames, metadata_only=False, chans=False, area_def=None, self_register=False)[source]#
Read SSMIS binary data products.
- Parameters:
fnames (list) –
List of strings, full paths to files
metadata_only (bool, default=False) –
Return before actually reading data if True
chans (list of str, default=None) –
NOT YET IMPLEMENTED
List of desired channels (skip unneeded variables as needed).
Include all channels if None.
area_def (pyresample.AreaDefinition, default=None) –
NOT YET IMPLEMENTED
Specify region to read
Read all data if None.
self_register (str or bool, default=False) –
NOT YET IMPLEMENTED
register all data to the specified dataset id (as specified in the return dictionary keys).
Read multiple resolutions of data if False.
- Returns:
dictionary of xarray.Dataset objects with required Variables and Attributes.
Dictionary keys can be any descriptive dataset ids.
- Return type:
dict of xarray.Datasets
See also
- Xarray and NetCDF Metadata Standards
Additional information regarding required attributes and variables for GeoIPS-formatted xarray Datasets.
geoips.plugins.modules.readers.viirs_netcdf module#
VIIRS NetCDF reader.
This VIIRS reader is designed for reading the NPP/JPSS VIIRS files geoips. The reader is only using the python functions and xarray variables. Although the file name indicates the data is in netcdf4 format.
Thus, the reader is based on the netcdf4 data format.
The orginal reader (viirs_aotcimss_ncdf4_reader.py) was developed for geoips1, which applied many geoips1 function.
V1.0: NRL-Monterey, 09/17/2020
VIIRS file infOrmation:
There are 6 files for each time of VIIRS data, i.e.,
For NASA NPP VIIRS
20200826.074800.npp.viirs.viirs_npp_nasaearthdata_x.x.VNP02DNB.sdr.x.x.nc
20200826.074800.npp.viirs.viirs_npp_nasaearthdata_x.x.VNP02IMG.sdr.x.x.nc
20200826.074800.npp.viirs.viirs_npp_nasaearthdata_x.x.VNP02MOD.sdr.x.x.nc
20200826.074800.npp.viirs.viirs_npp_nasaearthdata_x.x.VNP03DNB.sdr.x.x.nc
20200826.074800.npp.viirs.viirs_npp_nasaearthdata_x.x.VNP03IMG.sdr.x.x.nc
20200826.074800.npp.viirs.viirs_npp_nasaearthdata_x.x.VNP03MOD.sdr.x.x.nc
For JPSS VIIRS
20200914.150000.npp.viirs.viirs_sips_jpss_uwssec_001.x.VJ102DNB.sdr.x.x.nc
20200914.150000.npp.viirs.viirs_sips_jpss_uwssec_001.x.VJ102IMG.sdr.x.x.nc
20200914.150000.npp.viirs.viirs_sips_jpss_uwssec_001.x.VJ102MOD.sdr.x.x.nc
20200914.150000.npp.viirs.viirs_sips_jpss_uwssec_001.x.VJ103DNB.sdr.x.x.nc
20200914.150000.npp.viirs.viirs_sips_jpss_uwssec_001.x.VJ103IMG.sdr.x.x.nc
20200914.150000.npp.viirs.viirs_sips_jpss_uwssec_001.x.VJ103MOD.sdr.x.x.nc
DNB: VIIRS day-night Band obs
MOD: VIIRS M-Band obs
IMG: VIIRS I-Band obs
The .VNP02 files are for the data records, while the .VNP03 files are for the geolocation data records.
The xarray of geoips reader need both the data and lat/lon info. Thus, this VIIRS reader is designed to read in the paired VNP02 and VNP03 files, depending on any one of DNB or IMG or MOD file. In order to minimize dupilcated excution of VIIRS files, additional adjust of excution of the VIIRS files will be needed (discussion with Mindy on how to do it).
- geoips.plugins.modules.readers.viirs_netcdf.add_to_xarray(varname, nparr, xobj, dataset_masks, data_type, nparr_mask)[source]#
Add variable to xarray Dataset.
- geoips.plugins.modules.readers.viirs_netcdf.call(fnames, metadata_only=False, chans=None, area_def=None, self_register=False)[source]#
Read VIIRS netcdf data products.
- Parameters:
fnames (list) –
List of strings, full paths to files
metadata_only (bool, default=False) –
Return before actually reading data if True
chans (list of str, default=None) –
List of desired channels (skip unneeded variables as needed).
Include all channels if None.
area_def (pyresample.AreaDefinition, default=None) –
NOT YET IMPLEMENTED
Specify region to read
Read all data if None.
self_register (str or bool, default=False) –
NOT YET IMPLEMENTED
register all data to the specified dataset id (as specified in the return dictionary keys).
Read multiple resolutions of data if False.
- Returns:
dictionary of xarray.Dataset objects with required Variables and Attributes.
Dictionary keys can be any descriptive dataset ids.
- Return type:
dict of xarray.Datasets
See also
- Xarray and NetCDF Metadata Standards
Additional information regarding required attributes and variables for GeoIPS-formatted xarray Datasets.
- geoips.plugins.modules.readers.viirs_netcdf.required_chan(chans, varnames)[source]#
Return True if required channel.
geoips.plugins.modules.readers.wfabba_ascii module#
WFABBA ascii data reader.
WFABBA is a geostationary fire product produced by SSEC
- geoips.plugins.modules.readers.wfabba_ascii.call(fnames, metadata_only=False, chans=None, area_def=None, self_register=False)[source]#
Read WFABBA ascii data from a list of filenames.
WFABBA ascii files contain list of fire detects with their latitude, longitude, and scan location
- Parameters:
fnames (list) –
List of strings, full paths to files
metadata_only (bool, default=False) –
Return before actually reading data if True
chans (list of str, default=None) –
NOT YET IMPLEMENTED
List of desired channels (skip unneeded variables as needed).
Include all channels if None.
area_def (pyresample.AreaDefinition, default=None) –
NOT YET IMPLEMENTED
Specify region to read
Read all data if None.
self_register (str or bool, default=False) –
NOT YET IMPLEMENTED
register all data to the specified dataset id (as specified in the return dictionary keys).
Read multiple resolutions of data if False.
- Returns:
dictionary of xarray.Dataset objects with required Variables and Attributes.
Dictionary keys can be any descriptive dataset ids.
- Return type:
dict of xarray.Datasets
See also
- Xarray and NetCDF Metadata Standards
Additional information regarding required attributes and variables for GeoIPS-formatted xarray Datasets.
geoips.plugins.modules.readers.windsat_idr37_binary module#
Windsat binary data reader.
This code is designed to read windsat sdr binary data (idr37) file for windsat 37 GHz products in GEOIPS environments. the input file name is something alike US058SORB-BINspp.wndmi_fws_d20191126_s134102_e153244_r87467_cfnmoc.idr37.
V1.0: initial version. Song Yang, NRL-MRY, 01/08/2020
errflag is the important parameter of the windsat edr dataset. It is a 32-bit integer which describes what is the current data point status. Here are the meaning of each bit:
0-7: Wilheit rain flag
8: forward/aft scan (bit set to 1 for forward part of scan, 0 for aft scan )
9: ascending/descending pass flag (1 for ascending, 0 for descending)
10: Warm load flag
11: Warm load gains applied (1 = gains applied, 0 = gains not applied)
12: Glare angle invalid because no 1 vector or LOS doesn't pierce earth
13-18: Glare angle (0 to 30 represents angles of 0 to 60 degree in increments
of 2 deg; 31 represents angles .gt. 60 deg; 32 represents invalid
glare angle)
19: Cold load flag. If set to 1 the VH channel data had to be corrected due
to interference in the cold load signal, such as the moon or a
geostationary satellite.
20: Gain Saturation flag. Set to 1 when strong RFI causes the gain to change.
This is set if any TDR saturation flag is set at this frequency
23: used to hold the rfi flag so that it may be passed on to other structures
such as the resampling and intermediate structures. Finally RFI is placed
in the sdr structure.
other bits are spare
Here is the original sdr record in Fortran:
type IDRRecord_short
real(double) :: JD2000 8 bytes
real, dimension(4):: stokes 16 bytes
real :: latitude 4 bytes
real :: longitude 4 bytes
real :: EIA 4 bytes: earth incidence angle, the angle
on the ground between vertical and
the satellite look vector
real :: PRA 4 bytes: rotation of the polarization plane
from true
real :: CAA 4 bytes: compass azimuth angle on the ground
real :: tI45, tIcp, pra45 12 bytes
integer :: errflag 4 bytes (32-bits integer):a set of bit
flags for data quality and
conditions (above explanation)
integer :: Scan 4 bytes: scan line number in the orbit.
WindSat scans every 1.9 seconds
integer(int16) :: dcnum 2 bytes: pixel number along the scan,
called 'downcount number',
because the highest pixel number
is measured first.
integer(int16) :: SurfaceType 2 bytes: legacy SSMI surface type
integer(int16) :: scanAngle 2 bytes: angle on the ground between the
flight direction and the look
direction
integer(int8) :: water2land ! copied from IDRL record 1 byte:
integer(int8) :: land2water ! copied from IDRL record 1 byte:
end type IDRRecord_short
Gain saturation is when a sudden, large signal causes the gain to change quickly and make averaged gain unreliable.
Forward/Aft is for sensor view position
The warm load flag indicates that calibration may be unreliable due to solar intrusion into the warm load.
Cold load flags do not mean calibration is unreliable. It’s a way for us to check the cold load correction algorithm.
Sun glare is not something to worry about.
The RFI flag is never set at 37 GHz.
tI45, tIcp, pra45, scanAngleI, water2land, and land2water aren’t commanly used.
The first four are for recreating the 6-element pre-Stokes polarization vector, and the last two measure coastal contamination
The actual idr37 data record (idr_record) in C:
typedef struct {
double jd2000;
float stokes[4];
float plat; lat of earth observation
float plon; lon of earth observation
float eia; radiance= ~53deg
float pra;
float caa;
float slat; latitude of satellite position? not
float slon; longitude of satellite position? not
float salt; altitude of satellite (meter? km?) not
int errflag;
int scan;
short dcnum;
short surf;
float spare;
idr_record;
Its total length of idr_record is 72 bytes
- geoips.plugins.modules.readers.windsat_idr37_binary.call(fnames, metadata_only=False, chans=None, area_def=None, self_register=False)[source]#
Read Windsat binary data products.
- Parameters:
fnames (list) –
List of strings, full paths to files
metadata_only (bool, default=False) –
Return before actually reading data if True
chans (list of str, default=None) –
NOT YET IMPLEMENTED
List of desired channels (skip unneeded variables as needed).
Include all channels if None.
area_def (pyresample.AreaDefinition, default=None) –
NOT YET IMPLEMENTED
Specify region to read
Read all data if None.
self_register (str or bool, default=False) –
NOT YET IMPLEMENTED
register all data to the specified dataset id (as specified in the return dictionary keys).
Read multiple resolutions of data if False.
- Returns:
dictionary of xarray.Dataset objects with required Variables and Attributes.
Dictionary keys can be any descriptive dataset ids.
- Return type:
dict of xarray.Datasets
See also
- Xarray and NetCDF Metadata Standards
Additional information regarding required attributes and variables for GeoIPS-formatted xarray Datasets.
geoips.plugins.modules.readers.windsat_remss_winds_netcdf module#
Read derived surface winds from REMSS WINDSAT netcdf data.
- geoips.plugins.modules.readers.windsat_remss_winds_netcdf.call(fnames, metadata_only=False, chans=None, area_def=None, self_register=False)[source]#
Read Remote Sensing Systems Windsat data.
- Parameters:
fnames (list) –
List of strings, full paths to files
metadata_only (bool, default=False) –
NOT YET IMPLEMENTED
Return before actually reading data if True
chans (list of str, default=None) –
NOT YET IMPLEMENTED
List of desired channels (skip unneeded variables as needed).
Include all channels if None.
area_def (pyresample.AreaDefinition, default=None) –
NOT YET IMPLEMENTED
Specify region to read
Read all data if None.
self_register (str or bool, default=False) –
NOT YET IMPLEMENTED
register all data to the specified dataset id (as specified in the return dictionary keys).
Read multiple resolutions of data if False.
- Returns:
dictionary of xarray.Dataset objects with required Variables and Attributes.
Dictionary keys can be any descriptive dataset ids.
- Return type:
dict of xarray.Datasets
See also
- Xarray and NetCDF Metadata Standards
Additional information regarding required attributes and variables for GeoIPS-formatted xarray Datasets.
Module contents#
Geoips readers init file.