Setup for tutorials

This tutorial was originally developed for the 2021 Cloud Hackathon and iterated for the 2021 Cloud Workshop at AGU

This tutorial will help you get oriented and setup to run the following tutorials. This setup involves a JupyterHub (or Hub) managed by 2i2c.

Step 1. Login to the Hub

Please go to the Openscapes Jupyter Hub https://openscapes.2i2c.cloud and Log in with your GitHub Account, and select “Small”.

Alternatively, you can also click this badge to launch the Hub:


Select Small for your server. The default language is Python, and you also have the option to select R and Matlab. Matlab is “bring your own licence” and you’ll be prompted to give your details.

Note: It takes a few minutes for the Hub to load. Please be patient!

While the Hub loads, we’ll:

  • Discuss cloud environments
  • See how my Desktop is setup
  • Discuss python and conda environments

Then, when the Hub is loaded, we’ll get oriented in the Hub.

Discussion: Cloud environment

A brief overview. See NASA Openscapes Cloud Environment in the 2021-Cloud-Hackathon book for more detail.

Cloud infrastructure

Discussion: My desktop setup

I’ll screenshare to show and/or talk through how I have oriented the following software we’re using:

  • Workshop Book (my teaching notes, your reference material)
  • Zoom Chat

Discussion: Python and Conda environments

Why Python?

Default Python Environment:

We’ve set up the Python environment with conda.

name: openscapes
channels:
  - conda-forge
dependencies:
  - python=3.9
  - pangeo-notebook
  - awscli~=1.20
  - boto3~=1.19
  - gdal~=3.3
  - rioxarray~=0.8
  - xarray~=0.19
  - h5netcdf~=0.11
  - netcdf4~=1.5
  - h5py~=2.10
  - geoviews~=1.9
  - matplotlib-base~=3.4
  - hvplot~=0.7
  - pyproj~=3.2
  - bqplot~=0.12
  - geopandas~=0.10
  - zarr~=2.10
  - cartopy~=0.20
  - shapely==1.7.1
  - pyresample~=1.22
  - joblib~=1.1
  - pystac-client~=0.3
  - s3fs~=2021.7
  - ipyleaflet~=0.14
  - sidecar~=0.5
  - jupyterlab-geojson~=3.1
  - jupyterlab-git
  - jupyter-resource-usage
  - ipympl~=0.6
  - conda-lock~=0.12
  - pooch~=1.5
  - pip
  - pip:
    - tqdm
    - harmony-py
    - earthdata
    - zarr-eosdis-store

Bash terminal and installed software

Libraries that are available from the terminal

  • gdal 3.3 commands ( gdalinfo, gdaltransform…)
  • hdf5 commands ( h5dump, h5ls..)
  • netcdf4 commands (ncdump, ncinfo …)
  • jq (parsing json files or streams from curl)
  • curl (fetch resources from the web)
  • awscli (AWS API client, to interact with AWS cloud services)
  • vim (editor)
  • tree ( directory tree)
  • more …

corn 🌽 and updating the environment

Scientific Python is a vast space and we only included libraries that are needed in our tutorials. Our default environment can be updated to include any Python library that’s available on pip or conda.

The project used to create our default environment is called corn 🌽 (as it can include many Python kernels).

If we want to update a library or install a whole new environment we need to open an issue on this repository.

Step 2. JupyterHub orientation

Now that the Hub is loaded, let’s get oriented.

First impressions

  • Launcher & the big blue button
  • “home directory”

Step 3. Navigate to the Workshop folder

The workshop folder 2021-Cloud-Workshop-AGU is in the shared folder on JupyterHub.

Jupyter notebooks

Let’s get oriented to Jupyter notebooks, which we’ll use in all the tutorials.

How do I end my session?

(Also see How do I end my Openscapes session? Will I lose all of my work?)

When you are finished working for the day it is important to explicitly log out of your Openscapes session. The reason for this is it will save us a bit of money! When you keep a session active it uses up AWS resources and keeps a series of virtual machines deployed.

Stopping the server happens automatically when you log out, so navigate to “File -> Log Out” and just click “Log Out”!

!!! NOTE “logging out” - Logging out will NOT cause any of your work to be lost or deleted. It simply shuts down some resources. It would be equivalent to turning off your desktop computer at the end of the day.