How do I access Cloud Data from my Local Machine in Python?

Downloading data

When you have found the data you want to use, you have two options. You can download the data to work locally, or access the data directly to work in the cloud. This second way of working is called “Direct Cloud Access” or simply, “Direct Access”.

This page covers downloading data from Earthdata Cloud to your local machine using the earthaccess library.

earthaccess

We can use the earthaccess python library to grab the file URLs and then access them with the xarray library.

#Import packages
import earthaccess
import xarray as xr

#Authentication with Earthdata Login
auth = earthaccess.login(strategy="netrc")

You're now authenticated with NASA Earthdata Login
Using token with expiration date: 10/30/2023
Using .netrc file for EDL

#Access land ice height from ATLAS/ICESat-2 V005 (10.5067/ATLAS/ATL06.005), searching for data over western Greenland coast over two weeks in July 2022. The data are provided as HDF5 granules (files) that span about 1/14th of an orbit.

results = earthaccess.search_data(short_name="ATL06",
                                  version="005",
                                  cloud_hosted=True,
                                  temporal = ("2022-07-17","2022-07-31"),
                                  bounding_box = (-51.96423,68.10554,-48.71969,70.70529))

Granules found: 5

files = earthaccess.download(results, "./local_folder")

downloaded_files = []
for f in files: 
    file = "local_folder/"+ f
    downloaded_files.append(file)

 Getting 5 granules, approx download size: 0.0 GB
File ATL06_20220718010029_03991603_005_01.h5 already downloaded
File ATL06_20220718134522_04071605_005_01.h5 already downloaded
File ATL06_20220722005209_04601603_005_01.h5 already downloaded
File ATL06_20220726004352_05211603_005_01.h5 already downloaded
File ATL06_20220722133704_04681605_005_01.h5 already downloaded

['local_folder/ATL06_20220718010029_03991603_005_01.h5',
 'local_folder/ATL06_20220718134522_04071605_005_01.h5',
 'local_folder/ATL06_20220722005209_04601603_005_01.h5',
 'local_folder/ATL06_20220722133704_04681605_005_01.h5',
 'local_folder/ATL06_20220726004352_05211603_005_01.h5']

#Use xarray to load the data as a multifile dataset for a single group in the HDF5 file, in this case land ice segments:
ds = xr.open_mfdataset(downloaded_files, group='/gt1l/land_ice_segments', engine='h5netcdf')
ds

<xarray.Dataset>
Dimensions:                (delta_time: 241711)
Coordinates:
  * delta_time             (delta_time) datetime64[ns] 2022-07-18T01:00:46.67...
    latitude               (delta_time) float64 dask.array<chunksize=(78325,), meta=np.ndarray>
    longitude              (delta_time) float64 dask.array<chunksize=(78325,), meta=np.ndarray>
Data variables:
    atl06_quality_summary  (delta_time) int8 dask.array<chunksize=(78325,), meta=np.ndarray>
    h_li                   (delta_time) float32 dask.array<chunksize=(78325,), meta=np.ndarray>
    h_li_sigma             (delta_time) float32 dask.array<chunksize=(78325,), meta=np.ndarray>
    segment_id             (delta_time) float64 dask.array<chunksize=(78325,), meta=np.ndarray>
    sigma_geo_h            (delta_time) float32 dask.array<chunksize=(78325,), meta=np.ndarray>
Attributes:
    Description:  The land_ice_height group contains the primary set of deriv...
    data_rate:    Data within this group are sparse.  Data values are provide...

xarray.Dataset

Dimensions:
- delta_time: 241711
Coordinates: (3)
- delta_time
  (delta_time)
  datetime64[ns]
  2022-07-18T01:00:46.678760592 .....
  contentType :
  referenceInformation
  description :
  Number of GPS seconds since the ATLAS SDP epoch. The ATLAS Standard Data Products (SDP) epoch offset is defined within /ancillary_data/atlas_sdp_gps_epoch as the number of GPS seconds between the GPS epoch (1980-01-06T00:00:00.000000Z UTC) and the ATLAS SDP epoch. By adding the offset contained within atlas_sdp_gps_epoch to delta time parameters, the time in gps_seconds relative to the GPS epoch can be computed.
  long_name :
  Elapsed GPS seconds
  source :
  section 4.4
  standard_name :
  time
```
array(['2022-07-18T01:00:46.678760592', '2022-07-18T01:00:46.681322640',
       '2022-07-18T01:00:46.684008720', ..., '2022-07-26T00:49:18.826637808',
       '2022-07-26T00:49:18.829449568', '2022-07-26T00:49:18.832263232'],
      dtype='datetime64[ns]')
```
- latitude
  (delta_time)
  float64
  dask.array<chunksize=(78325,), meta=np.ndarray>
  contentType :
  physicalMeasurement
  description :
  Latitude of segment center, WGS84, North=+,
  long_name :
  Latitude
  source :
  section 3.10
  standard_name :
  latitude
  units :
  degrees_north
  valid_max :
  90.0
  valid_min :
  -90.0
  
  Array Chunk
  
  Bytes 1.84 MiB 611.91 kiB
  
  Shape (241711,) (78325,)
  
  Dask graph 5 chunks in 11 graph layers
  
  Data type float64 numpy.ndarray
  
  longitude
  
  (delta_time)
  
  float64
  
  dask.array<chunksize=(78325,), meta=np.ndarray>
  
  contentType :
  
  physicalMeasurement
  
  description :
  
  Longitude of segment center, , WGS84, East=+
  
  long_name :
  
  Longitude
  
  source :
  
  section 3.10
  
  standard_name :
  
  longitude
  
  units :
  
  degrees_east
  
  valid_max :
  
  180.0
  
  valid_min :
  
  -180.0
  
  Array Chunk
  
  Bytes 1.84 MiB 611.91 kiB
  
  Shape (241711,) (78325,)
  
  Dask graph 5 chunks in 11 graph layers
  
  Data type float64 numpy.ndarray
  
  Data variables: (5)
  
  atl06_quality_summary
  
  (delta_time)
  
  int8
  
  dask.array<chunksize=(78325,), meta=np.ndarray>
  
  contentType :
  
  qualityInformation
  
  description :
  
  The ATL06_quality_summary parameter indicates the best-quality subset of all ATL06 data. A zero in this parameter implies that no data-quality tests have found a problem with the segment, a one implies that some potential problem has been found. Users who select only segments with zero values for this flag can be relatively certain of obtaining high-quality data, but will likely miss a significant fraction of usable data, particularly in cloudy, rough, or low-surface-reflectance conditions.
  
  flag_meanings :
  
  best_quality potential_problem
  
  flag_values :
  
  [0 1]
  
  long_name :
  
  ATL06_Quality_Summary
  
  source :
  
  section 4.3
  
  units :
  
  1
  
  valid_max :
  
  1
  
  valid_min :
  
  0
  
  Array Chunk
  
  Bytes 236.05 kiB 76.49 kiB
  
  Shape (241711,) (78325,)
  
  Dask graph 5 chunks in 11 graph layers
  
  Data type int8 numpy.ndarray
  
  h_li
  
  (delta_time)
  
  float32
  
  dask.array<chunksize=(78325,), meta=np.ndarray>
  
  contentType :
  
  physicalMeasurement
  
  description :
  
  Standard land-ice segment height determined by land ice algorithm, corrected for first-photon bias, representing the median- based height of the selected PEs
  
  long_name :
  
  Land Ice height
  
  source :
  
  section 4.4
  
  units :
  
  meters
  
  Array Chunk
  
  Bytes 0.92 MiB 305.96 kiB
  
  Shape (241711,) (78325,)
  
  Dask graph 5 chunks in 11 graph layers
  
  Data type float32 numpy.ndarray
  
  h_li_sigma
  
  (delta_time)
  
  float32
  
  dask.array<chunksize=(78325,), meta=np.ndarray>
  
  contentType :
  
  qualityInformation
  
  description :
  
  Propagated error due to sampling error and FPB correction from the land ice algorithm
  
  long_name :
  
  Expected RMS segment misfit
  
  source :
  
  section 4.4
  
  units :
  
  meters
  
  Array Chunk
  
  Bytes 0.92 MiB 305.96 kiB
  
  Shape (241711,) (78325,)
  
  Dask graph 5 chunks in 11 graph layers
  
  Data type float32 numpy.ndarray
  
  segment_id
  
  (delta_time)
  
  float64
  
  dask.array<chunksize=(78325,), meta=np.ndarray>
  
  contentType :
  
  referenceInformation
  
  description :
  
  Segment number, counting from the equator. Equal to the segment_id for the second of the two 20m ATL03 segments included in the 40m ATL06 segment
  
  long_name :
  
  Reference Point, m
  
  source :
  
  section 3.1.2.1
  
  units :
  
  1
  
  Array Chunk
  
  Bytes 1.84 MiB 611.91 kiB
  
  Shape (241711,) (78325,)
  
  Dask graph 5 chunks in 11 graph layers
  
  Data type float64 numpy.ndarray
  
  sigma_geo_h
  
  (delta_time)
  
  float32
  
  dask.array<chunksize=(78325,), meta=np.ndarray>
  
  contentType :
  
  qualityInformation
  
  description :
  
  Total vertical geolocation error due to PPD and POD, including the effects of horizontal geolocation error on the segment vertical error.
  
  long_name :
  
  Vertical Geolocation Error
  
  source :
  
  ATBD Section 3.10
  
  units :
  
  meters
  
  Array Chunk
  
  Bytes 0.92 MiB 305.96 kiB
  
  Shape (241711,) (78325,)
  
  Dask graph 5 chunks in 11 graph layers
  
  Data type float32 numpy.ndarray

Indexes: (1)

delta_time

PandasIndex

PandasIndex(DatetimeIndex(['2022-07-18 01:00:46.678760592',
               '2022-07-18 01:00:46.681322640',
               '2022-07-18 01:00:46.684008720',
               '2022-07-18 01:00:46.686753504',
               '2022-07-18 01:00:46.689526560',
               '2022-07-18 01:00:46.692315280',
               '2022-07-18 01:00:46.695049040',
               '2022-07-18 01:00:46.700724096',
               '2022-07-18 01:00:46.703545872',
               '2022-07-18 01:00:46.706366832',
               ...
               '2022-07-26 00:49:18.806914512',
               '2022-07-26 00:49:18.809737328',
               '2022-07-26 00:49:18.812559600',
               '2022-07-26 00:49:18.815380608',
               '2022-07-26 00:49:18.818200224',
               '2022-07-26 00:49:18.821015744',
               '2022-07-26 00:49:18.823827088',
               '2022-07-26 00:49:18.826637808',
               '2022-07-26 00:49:18.829449568',
               '2022-07-26 00:49:18.832263232'],
              dtype='datetime64[ns]', name='delta_time', length=241711, freq=None))

Attributes: (2)
Description :
The land_ice_height group contains the primary set of derived ATL06 products. This includes geolocation, height, and standard error and quality measures for each segment. This group is sparse, meaning that parameters are provided only for pairs of segments for which at least one beam has a valid surface-height measurement.
data_rate :
Data within this group are sparse. Data values are provided only for those ICESat-2 20m segments where at least one beam has a valid land ice height measurement.