How do I access data stored in Earthdata Cloud in Python?
Direct Access
When you have found the data you want to use, you have two options. You can download the data to work locally, or access the data directly to work in the cloud. This second way of working is called “Direct Cloud Access” or simply, “Direct Access”. Your compute instance needs to be in Amazon Web Services (AWS) Region us-west-2 in order for the code to run successfully. We authenticate using a netrc file and an Earthdata Login, see the appendix for more information on Earthdata Login and netrc setup.
Python
We can use the earthaccess python library to grab the file URLs and then access them with the xarray library.
#Import packagesimport earthaccessimport xarray as xr
#Authentication with Earthdata Loginauth = earthaccess.login()
#Access land ice height from ATLAS/ICESat-2 V007 (10.5067/ATLAS/ATL06.007), searching for data over western Greenland coast over two weeks in July 2022. The data are provided as HDF5 granules (files) that span about 1/14th of an orbit.results = earthaccess.search_data(short_name="ATL06", version="007", cloud_hosted=True, temporal = ("2022-07-17","2022-07-31"), bounding_box = (-51.96423,68.10554,-48.71969,70.70529))
#Use xarray to load the data as a multifile dataset for a single group in the HDF5 file, in this case land ice segments:ds = xr.open_mfdataset(earthaccess.open(results), group='/gt1l/land_ice_segments')ds
<xarray.Dataset> Size: 11MB
Dimensions: (delta_time: 240873)
Coordinates:
* delta_time (delta_time) datetime64[ns] 2MB 2022-07-18T01:00:4...
latitude (delta_time) float64 2MB dask.array<chunksize=(10000,), meta=np.ndarray>
longitude (delta_time) float64 2MB dask.array<chunksize=(10000,), meta=np.ndarray>
Data variables:
atl06_quality_summary (delta_time) int8 241kB dask.array<chunksize=(10000,), meta=np.ndarray>
fpb_warning_flag (delta_time) int8 241kB dask.array<chunksize=(10000,), meta=np.ndarray>
h_li (delta_time) float32 963kB dask.array<chunksize=(10000,), meta=np.ndarray>
h_li_sigma (delta_time) float32 963kB dask.array<chunksize=(10000,), meta=np.ndarray>
segment_id (delta_time) float64 2MB dask.array<chunksize=(10000,), meta=np.ndarray>
sigma_geo_h (delta_time) float32 963kB dask.array<chunksize=(10000,), meta=np.ndarray>
Attributes:
data_rate: Data within this group are sparse. Data values are provide...
description: The land_ice_height group contains the primary set of deriv...
xarray.Dataset
delta_time: 240873
delta_time
(delta_time)
datetime64[ns]
2022-07-18T01:00:40.698453696 .....
contentType :
referenceInformation
description :
Number of GPS seconds since the ATLAS SDP epoch. The ATLAS Standard Data Products (SDP) epoch offset is defined within /ancillary_data/atlas_sdp_gps_epoch as the number of GPS seconds between the GPS epoch (1980-01-06T00:00:00.000000Z UTC) and the ATLAS SDP epoch. By adding the offset contained within atlas_sdp_gps_epoch to delta time parameters, the time in gps_seconds relative to the GPS epoch can be computed.
The ATL06_quality_summary parameter indicates the best-quality subset of all ATL06 data. A zero in this parameter implies that no data-quality tests have found a problem with the segment, a one implies that some potential problem has been found. Users who select only segments with zero values for this flag can be relatively certain of obtaining high-quality data, but will likely miss a significant fraction of usable data, particularly in cloudy, rough, or low-surface-reflectance conditions.
flag_meanings :
best_quality potential_problem
flag_values :
[0 1]
long_name :
ATL06_Quality_Summary
source :
ATL06 ATBD - Top Level Fitting Routine
units :
1
valid_max :
1
valid_min :
0
Array
Chunk
Bytes
235.23 kiB
9.77 kiB
Shape
(240873,)
(10000,)
Dask graph
26 chunks in 11 graph layers
Data type
int8 numpy.ndarray
fpb_warning_flag
(delta_time)
int8
dask.array<chunksize=(10000,), meta=np.ndarray>
contentType :
qualityInformation
description :
To flag that segments have potentially erroneous FPB corrections, the ATL06 algorithm sets fpb_warning_flag to 1 if full_sat_fract is greater than 0.90 (strong beams) or greater than 0.40 (weak beams) for either of the two ATL03 20 m segments that are used as inputs to each ATL06 40 m segment. Otherwise, fpb_warning_flag is set to 0.
flag_meanings :
best_quality potential_problem
flag_values :
[0 1]
long_name :
fpb_warning_flag
source :
ATL06 ATBD - FPB Corrections
units :
1
valid_max :
1
valid_min :
0
Array
Chunk
Bytes
235.23 kiB
9.77 kiB
Shape
(240873,)
(10000,)
Dask graph
26 chunks in 11 graph layers
Data type
int8 numpy.ndarray
h_li
(delta_time)
float32
dask.array<chunksize=(10000,), meta=np.ndarray>
contentType :
physicalMeasurement
description :
Standard land-ice segment height determined by land ice algorithm, corrected for first-photon bias, representing the median- based height of the selected PEs
long_name :
Land Ice height
source :
ATL06 ATBD - Top Level Fitting Routine
units :
meters
Array
Chunk
Bytes
0.92 MiB
39.06 kiB
Shape
(240873,)
(10000,)
Dask graph
26 chunks in 11 graph layers
Data type
float32 numpy.ndarray
h_li_sigma
(delta_time)
float32
dask.array<chunksize=(10000,), meta=np.ndarray>
contentType :
qualityInformation
description :
Propagated error due to sampling error and FPB correction from the land ice algorithm
long_name :
Expected RMS segment misfit
source :
ATL06 ATBD - Top Level Fitting Routine
units :
meters
Array
Chunk
Bytes
0.92 MiB
39.06 kiB
Shape
(240873,)
(10000,)
Dask graph
26 chunks in 11 graph layers
Data type
float32 numpy.ndarray
segment_id
(delta_time)
float64
dask.array<chunksize=(10000,), meta=np.ndarray>
contentType :
referenceInformation
description :
Segment number, counting from the equator. Equal to the segment_id for the second of the two 20m ATL03 segments included in the 40m ATL06 segment
long_name :
Reference Point, m
source :
ATL06 ATBD - Top Level Fitting Routine
units :
1
Array
Chunk
Bytes
1.84 MiB
78.12 kiB
Shape
(240873,)
(10000,)
Dask graph
26 chunks in 11 graph layers
Data type
float64 numpy.ndarray
sigma_geo_h
(delta_time)
float32
dask.array<chunksize=(10000,), meta=np.ndarray>
contentType :
qualityInformation
description :
Total vertical geolocation error due to PPD and POD, including the effects of horizontal geolocation error on the segment vertical error.
Data within this group are sparse. Data values are provided only for those ICESat-2 20m segments where at least one beam has a valid land ice height measurement.
description :
The land_ice_height group contains the primary set of derived ATL06 products. This includes geolocation, height, and standard error and quality measures for each segment. This group is sparse, meaning that parameters are provided only for pairs of segments for which at least one beam has a valid surface-height measurement.
End User License Agreement (EULA)
Sometimes, accessing data in NASA Earthdata Cloud requires an End-User License Agreement (EULA). If you cannot access a dataset, this may be your issue! See these instructions for how to authorize EULAs.
