PyGeoweaver package is a Python wrapper of the GeoWeaver. This package is designed for Jupyter users to be able to directly use Geoweaver in Jupyter notebook or JupyterLab (JupyterHub). ## 1. Installing PyGeoweaver You can either install pygeoweaver using pip or build from source
Requirement already satisfied: pygeoweaver in /Users/l21-n02609-comm/opt/anaconda3/lib/python3.9/site-packages (0.7.4)
Requirement already satisfied: requests==2.28.2 in /Users/l21-n02609-comm/opt/anaconda3/lib/python3.9/site-packages (from pygeoweaver) (2.28.2)
Requirement already satisfied: setuptools>=61.0 in /Users/l21-n02609-comm/opt/anaconda3/lib/python3.9/site-packages (from pygeoweaver) (68.0.0)
Requirement already satisfied: pydantic==1.10.9 in /Users/l21-n02609-comm/opt/anaconda3/lib/python3.9/site-packages (from pygeoweaver) (1.10.9)
Requirement already satisfied: typing-extensions>=4.2.0 in /Users/l21-n02609-comm/opt/anaconda3/lib/python3.9/site-packages (from pydantic==1.10.9->pygeoweaver) (4.6.3)
Requirement already satisfied: urllib3<1.27,>=1.21.1 in /Users/l21-n02609-comm/opt/anaconda3/lib/python3.9/site-packages (from requests==2.28.2->pygeoweaver) (1.26.16)
Requirement already satisfied: certifi>=2017.4.17 in /Users/l21-n02609-comm/opt/anaconda3/lib/python3.9/site-packages (from requests==2.28.2->pygeoweaver) (2023.5.7)
Requirement already satisfied: idna<4,>=2.5 in /Users/l21-n02609-comm/opt/anaconda3/lib/python3.9/site-packages (from requests==2.28.2->pygeoweaver) (3.4)
Requirement already satisfied: charset-normalizer<4,>=2 in /Users/l21-n02609-comm/opt/anaconda3/lib/python3.9/site-packages (from requests==2.28.2->pygeoweaver) (3.1.0)
The gw.start() function starts the Geoweaver Application on background to serve requests
import pygeoweaver as gw
Stop running Geoweaver if any..
Check Java..
Start Geoweaver..
Success: Geoweaver is up
2. Reset Passwords (optional if you already did)
This password will be used to authenticate the process runs on localhost (a.k.a the default host with id 100001).
Enter password: ········
Re-enter password: ········
3. Import Workflow : Earthdata Cloud Clinic
This tutorial from Cloud cookbook uses two different direct cloud access & subsetting options available in the Earthdata Cloud. * The earthaccess python library for data search and direct cloud access, followed by xarray subsetting * The Harmony-py python library for direct cloud access & data subsetting
The workflow for earthdata cloud clinic for geoweaver is hosted at https://github.com/earth-artificial-intelligence/earth_data_cloud_clinic_geoweaver
import osimport urllib.requestfrom IPython.display import Image= os.path.expanduser('~' )= "https://github.com/earth-artificial-intelligence/earth_data_cloud_clinic_geoweaver/archive/refs/tags/v0.0.6.zip" = os.path.join(home_dir, "kr2novmm4i9i27z5adyj.zip" )
Unzipping /Users/joe/gw-workspace/temp/kr2novmm4i9i27z5adyj.zip to /Users/joe/gw-workspace/temp/kr2novmm4i9i27z5adyj/
/Users/joe/gw-workspace/temp/kr2novmm4i9i27z5adyj - earth_data_cloud_clinic_geoweaver-0.0.6/
/Users/joe/gw-workspace/temp/kr2novmm4i9i27z5adyj - earth_data_cloud_clinic_geoweaver-0.0.6/.DS_Store
/Users/joe/gw-workspace/temp/kr2novmm4i9i27z5adyj - earth_data_cloud_clinic_geoweaver-0.0.6/README.md
/Users/joe/gw-workspace/temp/kr2novmm4i9i27z5adyj - earth_data_cloud_clinic_geoweaver-0.0.6/code/
/Users/joe/gw-workspace/temp/kr2novmm4i9i27z5adyj - earth_data_cloud_clinic_geoweaver-0.0.6/code/earth_access_authenticate.py
/Users/joe/gw-workspace/temp/kr2novmm4i9i27z5adyj - earth_data_cloud_clinic_geoweaver-0.0.6/code/earth_data_utils.py
/Users/joe/gw-workspace/temp/kr2novmm4i9i27z5adyj - earth_data_cloud_clinic_geoweaver-0.0.6/code/install_deps.sh
/Users/joe/gw-workspace/temp/kr2novmm4i9i27z5adyj - earth_data_cloud_clinic_geoweaver-0.0.6/code/install_packages.sh
/Users/joe/gw-workspace/temp/kr2novmm4i9i27z5adyj - earth_data_cloud_clinic_geoweaver-0.0.6/code/process.json
/Users/joe/gw-workspace/temp/kr2novmm4i9i27z5adyj - earth_data_cloud_clinic_geoweaver-0.0.6/code/retrieve_ssh.py
/Users/joe/gw-workspace/temp/kr2novmm4i9i27z5adyj - earth_data_cloud_clinic_geoweaver-0.0.6/code/retrieve_sst.py
/Users/joe/gw-workspace/temp/kr2novmm4i9i27z5adyj - earth_data_cloud_clinic_geoweaver-0.0.6/code/s3_file_storage.py
/Users/joe/gw-workspace/temp/kr2novmm4i9i27z5adyj - earth_data_cloud_clinic_geoweaver-0.0.6/earthdata.ipynb
/Users/joe/gw-workspace/temp/kr2novmm4i9i27z5adyj - earth_data_cloud_clinic_geoweaver-0.0.6/geoweaver_screenshot.png
/Users/joe/gw-workspace/temp/kr2novmm4i9i27z5adyj - earth_data_cloud_clinic_geoweaver-0.0.6/gulf.json
/Users/joe/gw-workspace/temp/kr2novmm4i9i27z5adyj - earth_data_cloud_clinic_geoweaver-0.0.6/history/
/Users/joe/gw-workspace/temp/kr2novmm4i9i27z5adyj - earth_data_cloud_clinic_geoweaver-0.0.6/history/195yXqgSfvPIInJ2gI.json
/Users/joe/gw-workspace/temp/kr2novmm4i9i27z5adyj - earth_data_cloud_clinic_geoweaver-0.0.6/history/197a10n0xbrc4rqwve.json
/Users/joe/gw-workspace/temp/kr2novmm4i9i27z5adyj - earth_data_cloud_clinic_geoweaver-0.0.6/history/1bZvPF5Z1tytBGDqph.json
/Users/joe/gw-workspace/temp/kr2novmm4i9i27z5adyj - earth_data_cloud_clinic_geoweaver-0.0.6/history/39SB7eVc2494ECqik2.json
/Users/joe/gw-workspace/temp/kr2novmm4i9i27z5adyj - earth_data_cloud_clinic_geoweaver-0.0.6/history/3KpWI4XE8npoDNFXTQ.json
/Users/joe/gw-workspace/temp/kr2novmm4i9i27z5adyj - earth_data_cloud_clinic_geoweaver-0.0.6/history/3ZlO4QZpqjhGyKmyXO.json
/Users/joe/gw-workspace/temp/kr2novmm4i9i27z5adyj - earth_data_cloud_clinic_geoweaver-0.0.6/history/43kckrxzh5QbESss0R.json
/Users/joe/gw-workspace/temp/kr2novmm4i9i27z5adyj - earth_data_cloud_clinic_geoweaver-0.0.6/history/54jcaiqoso3iree437.json
/Users/joe/gw-workspace/temp/kr2novmm4i9i27z5adyj - earth_data_cloud_clinic_geoweaver-0.0.6/history/5dixki8g2scz0sjj56.json
/Users/joe/gw-workspace/temp/kr2novmm4i9i27z5adyj - earth_data_cloud_clinic_geoweaver-0.0.6/history/5ou0GVSM1NQnlv7FOB.json
/Users/joe/gw-workspace/temp/kr2novmm4i9i27z5adyj - earth_data_cloud_clinic_geoweaver-0.0.6/history/5ucpoaa3e0bbnwyu1n.json
/Users/joe/gw-workspace/temp/kr2novmm4i9i27z5adyj - earth_data_cloud_clinic_geoweaver-0.0.6/history/5uzqmkhig32ub23cax.json
/Users/joe/gw-workspace/temp/kr2novmm4i9i27z5adyj - earth_data_cloud_clinic_geoweaver-0.0.6/history/6LDOLdZroopQ1UvaWF.json
/Users/joe/gw-workspace/temp/kr2novmm4i9i27z5adyj - earth_data_cloud_clinic_geoweaver-0.0.6/history/6hxs1emptzyrmsi0f3.json
/Users/joe/gw-workspace/temp/kr2novmm4i9i27z5adyj - earth_data_cloud_clinic_geoweaver-0.0.6/history/8CmlmhGPYckTE5VQnh.json
/Users/joe/gw-workspace/temp/kr2novmm4i9i27z5adyj - earth_data_cloud_clinic_geoweaver-0.0.6/history/8nqne2wcyke9t1hy5p.json
/Users/joe/gw-workspace/temp/kr2novmm4i9i27z5adyj - earth_data_cloud_clinic_geoweaver-0.0.6/history/Ay4VYNLteCA4TvHHvt.json
/Users/joe/gw-workspace/temp/kr2novmm4i9i27z5adyj - earth_data_cloud_clinic_geoweaver-0.0.6/history/B2kMBsZQW3Adxgi1uH.json
/Users/joe/gw-workspace/temp/kr2novmm4i9i27z5adyj - earth_data_cloud_clinic_geoweaver-0.0.6/history/BacPAagztwKFogMSWa.json
/Users/joe/gw-workspace/temp/kr2novmm4i9i27z5adyj - earth_data_cloud_clinic_geoweaver-0.0.6/history/IbB3mgAjkrED9qC8zw.json
/Users/joe/gw-workspace/temp/kr2novmm4i9i27z5adyj - earth_data_cloud_clinic_geoweaver-0.0.6/history/JUM7mZ2Clo86FSJQpS.json
/Users/joe/gw-workspace/temp/kr2novmm4i9i27z5adyj - earth_data_cloud_clinic_geoweaver-0.0.6/history/KeJgbxTIefDdZTHTLf.json
/Users/joe/gw-workspace/temp/kr2novmm4i9i27z5adyj - earth_data_cloud_clinic_geoweaver-0.0.6/history/KmoUyTblH4DeCQYfWN.json
/Users/joe/gw-workspace/temp/kr2novmm4i9i27z5adyj - earth_data_cloud_clinic_geoweaver-0.0.6/history/PNFURuqn5MBZk110Fh.json
/Users/joe/gw-workspace/temp/kr2novmm4i9i27z5adyj - earth_data_cloud_clinic_geoweaver-0.0.6/history/QBaTZsjXUzZrYyaSGY.json
/Users/joe/gw-workspace/temp/kr2novmm4i9i27z5adyj - earth_data_cloud_clinic_geoweaver-0.0.6/history/TX1TxAhpPjO6dvySl7.json
/Users/joe/gw-workspace/temp/kr2novmm4i9i27z5adyj - earth_data_cloud_clinic_geoweaver-0.0.6/history/UbPSHj43zEmCk2k5CR.json
/Users/joe/gw-workspace/temp/kr2novmm4i9i27z5adyj - earth_data_cloud_clinic_geoweaver-0.0.6/history/UfT7rXlmUDYmfoP804.json
/Users/joe/gw-workspace/temp/kr2novmm4i9i27z5adyj - earth_data_cloud_clinic_geoweaver-0.0.6/history/VeCiz4t1MvIQX4i2yN.json
/Users/joe/gw-workspace/temp/kr2novmm4i9i27z5adyj - earth_data_cloud_clinic_geoweaver-0.0.6/history/YBoSD2A5E8pi15Utoi.json
/Users/joe/gw-workspace/temp/kr2novmm4i9i27z5adyj - earth_data_cloud_clinic_geoweaver-0.0.6/history/YZGTgiuACdQqM2CObt.json
/Users/joe/gw-workspace/temp/kr2novmm4i9i27z5adyj - earth_data_cloud_clinic_geoweaver-0.0.6/history/Z9cOHlCTrfGR4beaLN.json
/Users/joe/gw-workspace/temp/kr2novmm4i9i27z5adyj - earth_data_cloud_clinic_geoweaver-0.0.6/history/ZYvkLM99T33qWDzU5l.json
/Users/joe/gw-workspace/temp/kr2novmm4i9i27z5adyj - earth_data_cloud_clinic_geoweaver-0.0.6/history/a4t3g4kcmx2j5iggfu.json
/Users/joe/gw-workspace/temp/kr2novmm4i9i27z5adyj - earth_data_cloud_clinic_geoweaver-0.0.6/history/b70aawjt8z4isn1dwy.json
/Users/joe/gw-workspace/temp/kr2novmm4i9i27z5adyj - earth_data_cloud_clinic_geoweaver-0.0.6/history/b8oEcCvvIU6KaKiYoT.json
/Users/joe/gw-workspace/temp/kr2novmm4i9i27z5adyj - earth_data_cloud_clinic_geoweaver-0.0.6/history/bv0wvvajfbrscyj6pm.json
/Users/joe/gw-workspace/temp/kr2novmm4i9i27z5adyj - earth_data_cloud_clinic_geoweaver-0.0.6/history/c3lYCsxly0BDlEg6da.json
/Users/joe/gw-workspace/temp/kr2novmm4i9i27z5adyj - earth_data_cloud_clinic_geoweaver-0.0.6/history/cf2KWtlaFRUAoEIhAS.json
/Users/joe/gw-workspace/temp/kr2novmm4i9i27z5adyj - earth_data_cloud_clinic_geoweaver-0.0.6/history/cn80gdwbo9cmtqfmt4.json
/Users/joe/gw-workspace/temp/kr2novmm4i9i27z5adyj - earth_data_cloud_clinic_geoweaver-0.0.6/history/csltVkAvLqfza0jV0i.json
/Users/joe/gw-workspace/temp/kr2novmm4i9i27z5adyj - earth_data_cloud_clinic_geoweaver-0.0.6/history/drP4XgYr92dISzae7n.json
/Users/joe/gw-workspace/temp/kr2novmm4i9i27z5adyj - earth_data_cloud_clinic_geoweaver-0.0.6/history/e02dq17h7tng220fo1.json
/Users/joe/gw-workspace/temp/kr2novmm4i9i27z5adyj - earth_data_cloud_clinic_geoweaver-0.0.6/history/eiXb07m8PuodHs3y9W.json
/Users/joe/gw-workspace/temp/kr2novmm4i9i27z5adyj - earth_data_cloud_clinic_geoweaver-0.0.6/history/exp8ncff5wdbitkm8o.json
/Users/joe/gw-workspace/temp/kr2novmm4i9i27z5adyj - earth_data_cloud_clinic_geoweaver-0.0.6/history/fbSMCmb9pAMK1bogT0.json
/Users/joe/gw-workspace/temp/kr2novmm4i9i27z5adyj - earth_data_cloud_clinic_geoweaver-0.0.6/history/gh2rmy5cq1bc003ak4.json
/Users/joe/gw-workspace/temp/kr2novmm4i9i27z5adyj - earth_data_cloud_clinic_geoweaver-0.0.6/history/h26w0736xpdftw6zw1.json
/Users/joe/gw-workspace/temp/kr2novmm4i9i27z5adyj - earth_data_cloud_clinic_geoweaver-0.0.6/history/hCz5UKbmmsYoiPeelF.json
/Users/joe/gw-workspace/temp/kr2novmm4i9i27z5adyj - earth_data_cloud_clinic_geoweaver-0.0.6/history/iMtwJlVtkG2jGPEJCm.json
/Users/joe/gw-workspace/temp/kr2novmm4i9i27z5adyj - earth_data_cloud_clinic_geoweaver-0.0.6/history/jid4vegdc9y0pz1oco.json
/Users/joe/gw-workspace/temp/kr2novmm4i9i27z5adyj - earth_data_cloud_clinic_geoweaver-0.0.6/history/ju768g303se38lkt74.json
/Users/joe/gw-workspace/temp/kr2novmm4i9i27z5adyj - earth_data_cloud_clinic_geoweaver-0.0.6/history/kX9THeiXUlECn7qnLK.json
/Users/joe/gw-workspace/temp/kr2novmm4i9i27z5adyj - earth_data_cloud_clinic_geoweaver-0.0.6/history/kialj0m6av03b04sv3.json
/Users/joe/gw-workspace/temp/kr2novmm4i9i27z5adyj - earth_data_cloud_clinic_geoweaver-0.0.6/history/kr2novmm4i9i27z5adyj.json
/Users/joe/gw-workspace/temp/kr2novmm4i9i27z5adyj - earth_data_cloud_clinic_geoweaver-0.0.6/history/kr6AXjoMUHFXn74s6B.json
/Users/joe/gw-workspace/temp/kr2novmm4i9i27z5adyj - earth_data_cloud_clinic_geoweaver-0.0.6/history/l6yVTXJZi87L8UPfDB.json
/Users/joe/gw-workspace/temp/kr2novmm4i9i27z5adyj - earth_data_cloud_clinic_geoweaver-0.0.6/history/l873z9b33henwzrydx.json
/Users/joe/gw-workspace/temp/kr2novmm4i9i27z5adyj - earth_data_cloud_clinic_geoweaver-0.0.6/history/lXakuW2z080MDGe8i1.json
/Users/joe/gw-workspace/temp/kr2novmm4i9i27z5adyj - earth_data_cloud_clinic_geoweaver-0.0.6/history/lbTt2RL9gtcLVP6xKj.json
/Users/joe/gw-workspace/temp/kr2novmm4i9i27z5adyj - earth_data_cloud_clinic_geoweaver-0.0.6/history/lwpewb4927s4e4m49r.json
/Users/joe/gw-workspace/temp/kr2novmm4i9i27z5adyj - earth_data_cloud_clinic_geoweaver-0.0.6/history/lxABhFBxbAvzawPiKs.json
/Users/joe/gw-workspace/temp/kr2novmm4i9i27z5adyj - earth_data_cloud_clinic_geoweaver-0.0.6/history/mczeu09JHbwVkuAAeC.json
/Users/joe/gw-workspace/temp/kr2novmm4i9i27z5adyj - earth_data_cloud_clinic_geoweaver-0.0.6/history/mr9YcFRUnuNyThJb4Q.json
/Users/joe/gw-workspace/temp/kr2novmm4i9i27z5adyj - earth_data_cloud_clinic_geoweaver-0.0.6/history/mx1w8k46bha4zi0q8r.json
/Users/joe/gw-workspace/temp/kr2novmm4i9i27z5adyj - earth_data_cloud_clinic_geoweaver-0.0.6/history/n4tc1hvnq5eamswpkj.json
/Users/joe/gw-workspace/temp/kr2novmm4i9i27z5adyj - earth_data_cloud_clinic_geoweaver-0.0.6/history/nfuxH8eVz896NHRm1m.json
/Users/joe/gw-workspace/temp/kr2novmm4i9i27z5adyj - earth_data_cloud_clinic_geoweaver-0.0.6/history/nkx236238gagh5wycu.json
/Users/joe/gw-workspace/temp/kr2novmm4i9i27z5adyj - earth_data_cloud_clinic_geoweaver-0.0.6/history/nnyv3p5uhplqv43ml3.json
/Users/joe/gw-workspace/temp/kr2novmm4i9i27z5adyj - earth_data_cloud_clinic_geoweaver-0.0.6/history/o0enoj5tgqss95igeb.json
/Users/joe/gw-workspace/temp/kr2novmm4i9i27z5adyj - earth_data_cloud_clinic_geoweaver-0.0.6/history/oU0aQLscHakCzO9BOi.json
/Users/joe/gw-workspace/temp/kr2novmm4i9i27z5adyj - earth_data_cloud_clinic_geoweaver-0.0.6/history/or6xD9foI8celpyyrB.json
/Users/joe/gw-workspace/temp/kr2novmm4i9i27z5adyj - earth_data_cloud_clinic_geoweaver-0.0.6/history/otrr4bjr5vvwob8kzf.json
/Users/joe/gw-workspace/temp/kr2novmm4i9i27z5adyj - earth_data_cloud_clinic_geoweaver-0.0.6/history/p5qpQWqza1Fwv0GbsF.json
/Users/joe/gw-workspace/temp/kr2novmm4i9i27z5adyj - earth_data_cloud_clinic_geoweaver-0.0.6/history/pYWDe5CUpDNQ9g08Uw.json
/Users/joe/gw-workspace/temp/kr2novmm4i9i27z5adyj - earth_data_cloud_clinic_geoweaver-0.0.6/history/process_1b4ko0.json
/Users/joe/gw-workspace/temp/kr2novmm4i9i27z5adyj - earth_data_cloud_clinic_geoweaver-0.0.6/history/process_2a8mok.json
/Users/joe/gw-workspace/temp/kr2novmm4i9i27z5adyj - earth_data_cloud_clinic_geoweaver-0.0.6/history/process_cgvhiy.json
/Users/joe/gw-workspace/temp/kr2novmm4i9i27z5adyj - earth_data_cloud_clinic_geoweaver-0.0.6/history/process_e57twj.json
/Users/joe/gw-workspace/temp/kr2novmm4i9i27z5adyj - earth_data_cloud_clinic_geoweaver-0.0.6/history/process_nlsntt.json
/Users/joe/gw-workspace/temp/kr2novmm4i9i27z5adyj - earth_data_cloud_clinic_geoweaver-0.0.6/history/process_qmc7xi.json
/Users/joe/gw-workspace/temp/kr2novmm4i9i27z5adyj - earth_data_cloud_clinic_geoweaver-0.0.6/history/process_ru79i2.json
/Users/joe/gw-workspace/temp/kr2novmm4i9i27z5adyj - earth_data_cloud_clinic_geoweaver-0.0.6/history/process_v2ow93.json
/Users/joe/gw-workspace/temp/kr2novmm4i9i27z5adyj - earth_data_cloud_clinic_geoweaver-0.0.6/history/process_wttkvt.json
/Users/joe/gw-workspace/temp/kr2novmm4i9i27z5adyj - earth_data_cloud_clinic_geoweaver-0.0.6/history/process_yvo1ds.json
/Users/joe/gw-workspace/temp/kr2novmm4i9i27z5adyj - earth_data_cloud_clinic_geoweaver-0.0.6/history/rRrz0ESWwjNUfd8J4l.json
/Users/joe/gw-workspace/temp/kr2novmm4i9i27z5adyj - earth_data_cloud_clinic_geoweaver-0.0.6/history/sRPxDerwGuaBIYhhAO.json
/Users/joe/gw-workspace/temp/kr2novmm4i9i27z5adyj - earth_data_cloud_clinic_geoweaver-0.0.6/history/tFbeUeqkws6xyNbueE.json
/Users/joe/gw-workspace/temp/kr2novmm4i9i27z5adyj - earth_data_cloud_clinic_geoweaver-0.0.6/history/tXEbbqVKQhjVLLM2f3.json
/Users/joe/gw-workspace/temp/kr2novmm4i9i27z5adyj - earth_data_cloud_clinic_geoweaver-0.0.6/history/u6EibxNEZg5Or4d4gZ.json
/Users/joe/gw-workspace/temp/kr2novmm4i9i27z5adyj - earth_data_cloud_clinic_geoweaver-0.0.6/history/unlenty6p8qrl03z42.json
/Users/joe/gw-workspace/temp/kr2novmm4i9i27z5adyj - earth_data_cloud_clinic_geoweaver-0.0.6/history/v97kvdt3cfum1172y2.json
/Users/joe/gw-workspace/temp/kr2novmm4i9i27z5adyj - earth_data_cloud_clinic_geoweaver-0.0.6/history/vC36VuDH7D6OOMF68w.json
/Users/joe/gw-workspace/temp/kr2novmm4i9i27z5adyj - earth_data_cloud_clinic_geoweaver-0.0.6/history/vx911STRa1BDoIfOd7.json
/Users/joe/gw-workspace/temp/kr2novmm4i9i27z5adyj - earth_data_cloud_clinic_geoweaver-0.0.6/history/wDqF0CmSx2MfoE037z.json
/Users/joe/gw-workspace/temp/kr2novmm4i9i27z5adyj - earth_data_cloud_clinic_geoweaver-0.0.6/history/woFrPp1c1EdP5vQQMr.json
/Users/joe/gw-workspace/temp/kr2novmm4i9i27z5adyj - earth_data_cloud_clinic_geoweaver-0.0.6/history/wq5bex7m1w1t185yyw.json
/Users/joe/gw-workspace/temp/kr2novmm4i9i27z5adyj - earth_data_cloud_clinic_geoweaver-0.0.6/history/wxJMPLZFgS9gqClzF6.json
/Users/joe/gw-workspace/temp/kr2novmm4i9i27z5adyj - earth_data_cloud_clinic_geoweaver-0.0.6/history/xU3NHiBe4cXiTolHSR.json
/Users/joe/gw-workspace/temp/kr2novmm4i9i27z5adyj - earth_data_cloud_clinic_geoweaver-0.0.6/history/xdc6fso7wkybrk71o7.json
/Users/joe/gw-workspace/temp/kr2novmm4i9i27z5adyj - earth_data_cloud_clinic_geoweaver-0.0.6/history/xsUiKqCumWr86iaEFQ.json
/Users/joe/gw-workspace/temp/kr2novmm4i9i27z5adyj - earth_data_cloud_clinic_geoweaver-0.0.6/history/xvrhhmdem4lsrax6io.json
/Users/joe/gw-workspace/temp/kr2novmm4i9i27z5adyj - earth_data_cloud_clinic_geoweaver-0.0.6/history/y2mnxktf5j68wxy68b.json
/Users/joe/gw-workspace/temp/kr2novmm4i9i27z5adyj - earth_data_cloud_clinic_geoweaver-0.0.6/history/yf0JCKUqg9sfzi0Tlq.json
/Users/joe/gw-workspace/temp/kr2novmm4i9i27z5adyj - earth_data_cloud_clinic_geoweaver-0.0.6/history/ywrvvwo20iif52rga6.json
/Users/joe/gw-workspace/temp/kr2novmm4i9i27z5adyj - earth_data_cloud_clinic_geoweaver-0.0.6/history/zkammrkv9md6gesze8.json
/Users/joe/gw-workspace/temp/kr2novmm4i9i27z5adyj - earth_data_cloud_clinic_geoweaver-0.0.6/history/zuwTMUYPw7YXrBjWxj.json
/Users/joe/gw-workspace/temp/kr2novmm4i9i27z5adyj - earth_data_cloud_clinic_geoweaver-0.0.6/workflow.json
Precheck: Done
Workflow ID: kr2novmm4i9i27z5adyj
Successfully saved to database. Complete.
4. Check Workflow Details
The workflow has four processes:
install_packages - install required packages
earth_data_utils - define all the utility functions like auth_earthaccess, get_dataset by xarray, etc
retrieve_ssh - using earthaccess to query EarthDataCloud for SEA_SURFACE_HEIGHT_ALT_GRIDS_L4_2SATS_5DAY_6THDEG_V_JPL2205
retrieve_sst - using earthaccess to query EarthDataCloud for MUR-JPL-L4-GLOB-v4.1
Check workflow id and other metadata using detail_workflow.
"kr2novmm4i9i27z5adyj" )
WorkflowId: kr2novmm4i9i27z5adyj
+----------------------+-------------------------+-------------+--------+---------------+
| Workflow Id | Name | Description | Owner | Confidential? |
+----------------------+-------------------------+-------------+--------+---------------+
| kr2novmm4i9i27z5adyj | earth_data_cloud_clinic | | 111111 | FALSE |
+----------------------+-------------------------+-------------+--------+---------------+
5. Run workflow
The follow line will execute the workflow on the provided host. 100001 means localhost. You can use gw.detail_host("100001") to check the details.
It will prompt a password input field to ask for password of the target host. If you don’t know your default password of localhost, please run gw.reset_password().
= 'kr2novmm4i9i27z5adyj' , host_list= '100001' )
Enter password for host - 100001: ········
Running workflow kr2novmm4i9i27z5adyj
The workflow has been kicked off.
History Id: svx32gpq6x3xmqkorm
Waiting for it to finish
> Start to run process: ru79i2
> Fnished process: ru79i2 - history: zg6v1hy042k - Status: Done
> Start to run process: e57twj
> Fnished process: e57twj - history: o2qf88jyh2f - Status: Done
> Start to run process: nlsntt
> Fnished process: nlsntt - history: fk604fwceda - Status: Done
Total time cost: 104 seconds
Execution is over. Final status: Done.
6. Check workflow history
From the log, we can see the workflow completed successfully after running all the four member processes. The final process run’s history id is ohvdmvpfy1m. We can use show_history to check.
## Get Results by Checking History "fk604fwceda" )
+-------------+--------+---------------------+---------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-------+
| History Id | Status | Begin Time | End Time | Input | Output | Notes |
+-------------+--------+---------------------+---------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-------+
| fk604fwceda | Done | 06/21/2023 03:30:26 | 06/21/2023 03:31:00 | from earth_data_utils import *
def search_and_get_sst():
auth_earthaccess()
results = earthaccess.search_data(
short_name=sst_short_name,
cloud_hosted=True,
temporal=("2021-07-01", "2021-07-02"),
)
print("found results: ", results)
return results
def plot_subset(ds_subset):
ds_subset.plot(figsize=(12,6), x='lon', y='lat')
home_dir = os.path.expanduser('~')
file_name = f"geoweaver_plot_sst.png"
save_file_path = os.path.join(home_dir, file_name)
plt.savefig(save_file_path)
print(f"new figure is saved to {save_file_path}")
with get_dataset(search_and_get_sst()) as ds:
subset = ds["analysed_sst"].sel(lat=slice(27, 37),
lon=slice(-70,-50)).std('time')
plot_subset(subset)
| You're now authenticated with NASA Earthdata Login
Using token with expiration date: 08/19/2023
Using .netrc file for EDL
Granules found: 2
found results: [Collection: {'Version': '4.1', 'ShortName': 'MUR-JPL-L4-GLOB-v4.1'}
Spatial coverage: {'HorizontalSpatialDomain': {'Geometry': {'BoundingRectangles': [{'WestBoundingCoordinate': -180, 'SouthBoundingCoordinate': -90, 'EastBoundingCoordinate': 180, 'NorthBoundingCoordinate': 90}]}}}
Temporal coverage: {'RangeDateTime': {'EndingDateTime': '2021-07-01T21:00:00.000Z', 'BeginningDateTime': '2021-06-30T21:00:00.000Z'}}
Size(MB): 0
Data: ['https://archive.podaac.earthdata.nasa.gov/podaac-ops-cumulus-protected/MUR-JPL-L4-GLOB-v4.1/20210701090000-JPL-L4_GHRSST-SSTfnd-MUR-GLOB-v02.0-fv04.1.nc'], Collection: {'Version': '4.1', 'ShortName': 'MUR-JPL-L4-GLOB-v4.1'}
Spatial coverage: {'HorizontalSpatialDomain': {'Geometry': {'BoundingRectangles': [{'WestBoundingCoordinate': -180, 'SouthBoundingCoordinate': -90, 'EastBoundingCoordinate': 180, 'NorthBoundingCoordinate': 90}]}}}
Temporal coverage: {'RangeDateTime': {'EndingDateTime': '2021-07-02T21:00:00.000Z', 'BeginningDateTime': '2021-07-01T21:00:00.000Z'}}
Size(MB): 0
Data: ['https://archive.podaac.earthdata.nasa.gov/podaac-ops-cumulus-protected/MUR-JPL-L4-GLOB-v4.1/20210702090000-JPL-L4_GHRSST-SSTfnd-MUR-GLOB-v02.0-fv04.1.nc']]
Opening 2 granules, approx size: 0.0 GB
QUEUEING TASKS | : 0%| | 0/2 [00:00<?, ?it/s]
QUEUEING TASKS | : 100%|██████████| 2/2 [00:00<00:00, 1421.56it/s]
PROCESSING TASKS | : 0%| | 0/2 [00:00<?, ?it/s]
PROCESSING TASKS | : 50%|█████ | 1/2 [00:00<00:00, 1.51it/s]
PROCESSING TASKS | : 100%|██████████| 2/2 [00:00<00:00, 2.97it/s]
PROCESSING TASKS | : 100%|██████████| 2/2 [00:00<00:00, 2.59it/s]
COLLECTING RESULTS | : 0%| | 0/2 [00:00<?, ?it/s]
COLLECTING RESULTS | : 100%|██████████| 2/2 [00:00<00:00, 22192.08it/s]
new figure is saved to /Users/joe/geoweaver_plot_sst.png
| - |
+-------------+--------+---------------------+---------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-------+
From the history, we can see the result image location is at /Users/joe/geoweaver_plot_sst.png. Let’s display it here.
7. Display sea surface temperature data
= os.path.join(home_dir, 'geoweaver_plot_sst.png' )))
Bingo! The image is successfully retrieved by the workflow and the data in xarray is rendered and saved to a local png file. The workflow runs with no errors and the replication succeeded.
8. Edit the process to change date range
Replicating is only the start. We always want to change the code and make it work for our purposes. Here we will demonstrate how to change the code in the workflow.
First, let’s list all the processes in the workflow:
= 'kr2novmm4i9i27z5adyj' )
0
earth_data_utils
wttkvt-GJjq8
1
install_packages
ru79i2-VCNCv
2
retrieve_sst
nlsntt-zMUba
3
retrieve_ssh
e57twj-ECmak
You can see the ids are a little different with dashes in the middle of them. That is because we need to distinguish the processes in the workflow as there might be same processes called multiple times in one workflow.
The original process id is just the first part. For example, the process id of retrieve_sst is nlsntt.
Copy that id and use get_code_for_process function to get current code.
'nlsntt' )
'from earth_data_utils import *\n\ndef search_and_get_sst():\n auth_earthaccess()\n results = earthaccess.search_data(\n short_name=sst_short_name,\n cloud_hosted=True,\n temporal=("2021-07-01", "2021-07-02"),\n )\n print("found results: ", results)\n return results\n\ndef plot_subset(ds_subset):\n ds_subset.plot(figsize=(12,6), x=\'lon\', y=\'lat\')\n home_dir = os.path.expanduser(\'~\')\n \n file_name = f"geoweaver_plot_sst.png"\n save_file_path = os.path.join(home_dir, file_name)\n plt.savefig(save_file_path)\n print(f"new figure is saved to {save_file_path}")\n\nwith get_dataset(search_and_get_sst()) as ds:\n subset = ds["analysed_sst"].sel(lat=slice(27, 37), \n lon=slice(-70,-50)).std(\'time\')\n plot_subset(subset)\n\n'
The code is not formatted correctly. So many \n are confusing.
An easy way is to directly save the code into a local file using function sync.
# write the file to disk to make changes = 'nlsntt' , local_path= "." , direction= "download" )
Proceeding with download
Wrote file retrieve_sst.py to .
The step above should create a new python file named retrieve_sst.py in the same folder as this notebook.
Open it using any editor and you should see the code as follows:
from earth_data_utils import *
def search_and_get_sst():
auth_earthaccess()
results = earthaccess.search_data(
short_name=sst_short_name,
cloud_hosted=True,
temporal=("2021-07-01", "2021-07-02"),
)
print("found results: ", results)
return results
def plot_subset(ds_subset):
ds_subset.plot(figsize=(12,6), x='lon', y='lat')
home_dir = os.path.expanduser('~')
file_name = f"geoweaver_plot_{str(uuid.uuid4())}.png"
save_file_path = os.path.join(home_dir, file_name)
plt.savefig(save_file_path)
print(f"new figure is saved to {save_file_path}")
with get_dataset(search_and_get_sst()) as ds:
subset = ds["analysed_sst"].sel(lat=slice(27, 37),
lon=slice(-70,-50)).std('time')
plot_subset(subset)Suppose we want the SST data of another region, for example, change the lat=slice(27, 37) to lat=slice(37, 47). Make the change in retrieve_sst.py, save and close.
Next step is to write the new code back into Geoweaver process using the same sync function. This time, the option direction should be upload meaning we are syncing changes from local file to Geoweaver database.
# sync changes back to the workflow after editing the file = 'nlsntt' , local_path= 'retrieve_sst.py' , direction= "upload" )
Proceeding with upload
Process update was successful
After the step finishes, verify the code change using get_process_code again.
'nlsntt' )
'from earth_data_utils import *\n\ndef search_and_get_sst():\n auth_earthaccess()\n results = earthaccess.search_data(\n short_name=sst_short_name,\n cloud_hosted=True,\n temporal=("2021-07-01", "2021-07-02"),\n )\n print("found results: ", results)\n return results\n\ndef plot_subset(ds_subset):\n ds_subset.plot(figsize=(12,6), x=\'lon\', y=\'lat\')\n home_dir = os.path.expanduser(\'~\')\n \n file_name = f"geoweaver_plot_sst.png"\n save_file_path = os.path.join(home_dir, file_name)\n plt.savefig(save_file_path)\n print(f"new figure is saved to {save_file_path}")\n\nwith get_dataset(search_and_get_sst()) as ds:\n subset = ds["analysed_sst"].sel(lat=slice(37, 47), \n lon=slice(-70,-50)).std(\'time\')\n plot_subset(subset)\n\n'
Ok, great! You can see the changes are made as the lat= has become 37, 47 now. The editing process is complete.
Now, we just need to rerun the workflow again with the new code.
= 'kr2novmm4i9i27z5adyj' , host_list= '100001' )
Enter password for host - 100001: ········
Running workflow kr2novmm4i9i27z5adyj
The workflow has been kicked off.
History Id: cf5xnsjounspuhlk9z
Waiting for it to finish
> Start to run process: ru79i2
> Fnished process: ru79i2 - history: arr51wopzbw - Status: Done
> Start to run process: e57twj
> Fnished process: e57twj - history: ppyfc2he31g - Status: Done
> Start to run process: nlsntt
> Fnished process: nlsntt - history: e1ce5zjdm3y - Status: Done
Total time cost: 105 seconds
Execution is over. Final status: Done.
You can see there is a new history id for process nlsntt: m3t1sdcsl3k. Check its content to find the result png file location:
"e1ce5zjdm3y" )
+-------------+--------+---------------------+---------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-------+
| History Id | Status | Begin Time | End Time | Input | Output | Notes |
+-------------+--------+---------------------+---------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-------+
| e1ce5zjdm3y | Done | 06/21/2023 03:37:55 | 06/21/2023 03:38:33 | from earth_data_utils import *
def search_and_get_sst():
auth_earthaccess()
results = earthaccess.search_data(
short_name=sst_short_name,
cloud_hosted=True,
temporal=("2021-07-01", "2021-07-02"),
)
print("found results: ", results)
return results
def plot_subset(ds_subset):
ds_subset.plot(figsize=(12,6), x='lon', y='lat')
home_dir = os.path.expanduser('~')
file_name = f"geoweaver_plot_sst.png"
save_file_path = os.path.join(home_dir, file_name)
plt.savefig(save_file_path)
print(f"new figure is saved to {save_file_path}")
with get_dataset(search_and_get_sst()) as ds:
subset = ds["analysed_sst"].sel(lat=slice(37, 47),
lon=slice(-70,-50)).std('time')
plot_subset(subset)
| You're now authenticated with NASA Earthdata Login
Using token with expiration date: 08/19/2023
Using .netrc file for EDL
Granules found: 2
found results: [Collection: {'Version': '4.1', 'ShortName': 'MUR-JPL-L4-GLOB-v4.1'}
Spatial coverage: {'HorizontalSpatialDomain': {'Geometry': {'BoundingRectangles': [{'WestBoundingCoordinate': -180, 'SouthBoundingCoordinate': -90, 'EastBoundingCoordinate': 180, 'NorthBoundingCoordinate': 90}]}}}
Temporal coverage: {'RangeDateTime': {'EndingDateTime': '2021-07-01T21:00:00.000Z', 'BeginningDateTime': '2021-06-30T21:00:00.000Z'}}
Size(MB): 0
Data: ['https://archive.podaac.earthdata.nasa.gov/podaac-ops-cumulus-protected/MUR-JPL-L4-GLOB-v4.1/20210701090000-JPL-L4_GHRSST-SSTfnd-MUR-GLOB-v02.0-fv04.1.nc'], Collection: {'Version': '4.1', 'ShortName': 'MUR-JPL-L4-GLOB-v4.1'}
Spatial coverage: {'HorizontalSpatialDomain': {'Geometry': {'BoundingRectangles': [{'WestBoundingCoordinate': -180, 'SouthBoundingCoordinate': -90, 'EastBoundingCoordinate': 180, 'NorthBoundingCoordinate': 90}]}}}
Temporal coverage: {'RangeDateTime': {'EndingDateTime': '2021-07-02T21:00:00.000Z', 'BeginningDateTime': '2021-07-01T21:00:00.000Z'}}
Size(MB): 0
Data: ['https://archive.podaac.earthdata.nasa.gov/podaac-ops-cumulus-protected/MUR-JPL-L4-GLOB-v4.1/20210702090000-JPL-L4_GHRSST-SSTfnd-MUR-GLOB-v02.0-fv04.1.nc']]
Opening 2 granules, approx size: 0.0 GB
QUEUEING TASKS | : 0%| | 0/2 [00:00<?, ?it/s]
QUEUEING TASKS | : 100%|██████████| 2/2 [00:00<00:00, 1568.26it/s]
PROCESSING TASKS | : 0%| | 0/2 [00:00<?, ?it/s]
PROCESSING TASKS | : 50%|█████ | 1/2 [00:00<00:00, 1.21it/s]
PROCESSING TASKS | : 100%|██████████| 2/2 [00:00<00:00, 2.42it/s]
COLLECTING RESULTS | : 0%| | 0/2 [00:00<?, ?it/s]
COLLECTING RESULTS | : 100%|██████████| 2/2 [00:00<00:00, 21454.24it/s]
new figure is saved to /Users/joe/geoweaver_plot_sst.png
| - |
+-------------+--------+---------------------+---------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-------+
The new png is generated. Visualize it in jupyter.
= os.path.join(home_dir, 'geoweaver_plot_sst.png' )))
Nice! The figure looks great with SST texture different from the one displayed above. That means the change of the spatial extent in the process worked great!
The reproducing experiment is done.
9. Check history of specific processes or workflows
# get run history of the process 'nlsntt' )
0
e1ce5zjdm3y
from earth_data_utils import *\n\ndef search_a...
You're now authenticated with NASA Earthdata L...
2023-06-21 07:37:55.840
2023-06-21 07:38:33.137
None
nlsntt
100001
Done
1
afz7j3dzavr
from earth_data_utils import *\n\ndef search_a...
You're now authenticated with NASA Earthdata L...
2023-06-21 07:33:45.739
2023-06-21 07:34:17.617
None
nlsntt
100001
Done
2
fk604fwceda
from earth_data_utils import *\n\ndef search_a...
You're now authenticated with NASA Earthdata L...
2023-06-21 07:30:26.691
2023-06-21 07:31:00.588
None
nlsntt
100001
Done
3
uf7ngwa1cnl
from earth_data_utils import *\n\ndef search_a...
You're now authenticated with NASA Earthdata L...
2023-06-21 07:27:23.479
2023-06-21 07:27:56.093
None
nlsntt
100001
Done
4
jxyo4sjnw7i4dsvt30
from earth_data_utils import *\n\ndef search_a...
You're now authenticated with NASA Earthdata L...
2023-06-21 07:07:44.443
2023-06-21 07:08:22.248
None
nlsntt
None
Done
5
m3t1sdcsl3k
from earth_data_utils import *\n\ndef search_a...
You're now authenticated with NASA Earthdata L...
2023-06-21 07:02:58.529
2023-06-21 07:03:31.363
None
nlsntt
100001
Done
6
ohvdmvpfy1m
from earth_data_utils import *\n\ndef search_a...
You're now authenticated with NASA Earthdata L...
2023-06-21 04:35:07.128
2023-06-21 04:35:41.058
None
nlsntt
100001
Done
7
55ierovrm5x
from earth_data_utils import *\n\ndef search_a...
You're now authenticated with NASA Earthdata L...
2023-06-21 04:21:51.164
2023-06-21 04:22:26.277
None
nlsntt
100001
Done
8
EcOKe1e6zRlz
from earth_data_utils import *\n\ndef search_a...
You're now authenticated with NASA Earthdata L...
2023-06-21 04:19:36.446
2023-06-21 04:20:10.986
None
nlsntt
None
Done
9
Clldd9m98PAt
from earth_data_utils import *\n\ndef search_a...
You're now authenticated with NASA Earthdata L...
2023-06-21 04:15:33.330
2023-06-21 04:16:09.377
None
nlsntt
None
Done
10
sAZF2bohiT4g
from earth_data_utils import *\n\ndef search_a...
You're now authenticated with NASA Earthdata L...
2023-06-21 04:10:58.009
2023-06-21 04:11:29.683
None
nlsntt
None
Done
11
muhcxxqhzna
from earth_data_utils import *\n\ndef search_a...
You're now authenticated with NASA Earthdata L...
2023-06-21 04:08:44.419
2023-06-21 04:09:36.507
None
nlsntt
100001
Done
12
5c4g1wdrzme
from earth_data_utils import *\n\ndef search_a...
Running
2023-06-21 04:07:23.881
2023-06-21 04:07:35.184
None
nlsntt
100001
Done
13
V4V7YE0Q20Qd
from earth_data_utils import *\n\ndef search_a...
You're now authenticated with NASA Earthdata L...
2023-06-21 04:05:26.906
2023-06-21 04:06:20.076
None
nlsntt
None
Done
14
gH0Y1ntFvvJQ
from earth_data_utils import *\n\ndef search_a...
You're now authenticated with NASA Earthdata L...
2023-06-21 04:01:01.589
2023-06-21 04:01:57.329
None
nlsntt
None
Done
15
bc8wLVHggQC2
from earth_data_utils import *\n\ndef search_a...
You're now authenticated with NASA Earthdata L...
2023-06-21 03:55:43.750
2023-06-21 03:56:34.311
None
nlsntt
None
Failed
16
fFuSBuGyhfUh
from earth_data_utils import *\n\ndef search_a...
You're now authenticated with NASA Earthdata L...
2023-06-21 03:44:26.103
2023-06-21 03:55:39.992
None
nlsntt
None
Done
17
Mt9qs1sz3ypu
from earth_data_utils import *\n\ndef search_a...
File "/Users/joe/gw-workspace/Mt9qs1sz3ypu/r...
2023-06-21 03:44:13.505
2023-06-21 03:44:13.553
None
nlsntt
None
Failed
18
g5wgNPIMKvt6
from earth_data_utils import *\n\ndef search_a...
File "/Users/joe/gw-workspace/g5wgNPIMKvt6/r...
2023-06-21 03:43:54.668
2023-06-21 03:43:54.724
None
nlsntt
None
Failed
19
R1pmcdWqLp6V
from earth_data_utils import *\n\ndef search_a...
You're now authenticated with NASA Earthdata L...
2023-06-21 03:40:48.248
2023-06-21 03:41:17.846
None
nlsntt
None
Failed
20
D5poheZLB3st
from earth_data_utils import *\n\ndef search_a...
Traceback (most recent call last):\n File "/U...
2023-06-21 03:40:35.390
2023-06-21 03:40:41.692
None
nlsntt
None
Failed
21
5lqwvos3wb6
from earth_data_utils import *\n\ndef search_a...
You're now authenticated with NASA Earthdata L...
2023-06-21 03:33:59.859
2023-06-21 03:34:27.842
None
nlsntt
100001
Failed
22
XYWhBOUtZcPG
from earth_data_utils import *\n\ndef search_a...
You're now authenticated with NASA Earthdata L...
2023-06-21 03:25:02.150
2023-06-21 03:25:36.884
None
nlsntt
None
Failed
23
4NoU0dDwnwmb
from earth_data_utils import *\n\ndef search_a...
Traceback (most recent call last):\n File "/U...
2023-06-21 03:24:10.798
2023-06-21 03:24:17.863
None
nlsntt
None
Failed
24
4AheIx9gmzVw
from earth_data_utils import *\n\ndef search_a...
You're now authenticated with NASA Earthdata L...
2023-06-21 03:15:01.301
2023-06-21 03:15:33.319
None
nlsntt
None
Failed
25
GNDa18SBhJTV
from earth_data_utils import *\n\ndef search_a...
You're now authenticated with NASA Earthdata L...
2023-06-21 03:11:51.951
2023-06-21 03:12:27.523
None
nlsntt
None
Failed
'kr2novmm4i9i27z5adyj' )
0
cf5xnsjounspuhlk9z
wttkvt-GJjq8;ru79i2-VCNCv;nlsntt-zMUba;e57twj-...
fwu7qad3kq6;arr51wopzbw;e1ce5zjdm3y;ppyfc2he31g;
2023-06-21 07:36:46.153
2023-06-21 07:37:55.840000000
None
kr2novmm4i9i27z5adyj
100001;
Done
1
dad8g3gkb76stetwix
wttkvt-GJjq8;ru79i2-VCNCv;nlsntt-zMUba;e57twj-...
1c5sjmcrgnj;fx19g9buif9;afz7j3dzavr;jlmxs448gc7;
2023-06-21 07:32:27.062
2023-06-21 07:33:45.739000064
None
kr2novmm4i9i27z5adyj
100001;
Done
2
svx32gpq6x3xmqkorm
wttkvt-GJjq8;ru79i2-VCNCv;nlsntt-zMUba;e57twj-...
zle2vcmqtkh;zg6v1hy042k;fk604fwceda;o2qf88jyh2f;
2023-06-21 07:29:18.002
2023-06-21 07:30:26.691000064
None
kr2novmm4i9i27z5adyj
100001;
Done
3
b70aawjt8z4isn1dwy
wttkvt-GJjq8;ru79i2-VCNCv;nlsntt-zMUba;e57twj-...
617pj1d8nis;zzcxtngjbdu;uf7ngwa1cnl;agb7rrec49u;
2023-06-21 07:26:09.740
2023-06-21 07:26:14.760999936
None
kr2novmm4i9i27z5adyj
100001;
Running
4
bv0wvvajfbrscyj6pm
wttkvt-GJjq8;ru79i2-VCNCv;nlsntt-zMUba;e57twj-...
1h6mwhjgffr;ced80h6vr3r;m3t1sdcsl3k;ed1lczm7cgh;
2023-06-21 07:01:45.800
2023-06-21 07:02:58.528999936
None
kr2novmm4i9i27z5adyj
100001;
Done
...
...
...
...
...
...
...
...
...
...
94
nfuxH8eVz896NHRm1m
wttkvt-GJjq8;yvo1ds-96vns;cgvhiy-BgnXM;v2ow93-...
rm0v06cmvuc;8nq18ogdwdi;jhzwkhohmz7;6o2p0699ri...
2023-06-08 16:08:37.776
2023-06-08 16:08:43.665999872
None
kr2novmm4i9i27z5adyj
jf7wuu;
Failed
95
VeCiz4t1MvIQX4i2yN
wttkvt-GJjq8;yvo1ds-96vns;cgvhiy-BgnXM;v2ow93-...
15jtnmmliz8;sj2llldbybq;alm4h4h5irw;fs47znp6yg...
2023-06-08 16:08:06.441
2023-06-08 16:08:14.723000064
None
kr2novmm4i9i27z5adyj
jf7wuu;
Failed
96
tFbeUeqkws6xyNbueE
wttkvt-GJjq8;yvo1ds-96vns;cgvhiy-BgnXM;v2ow93-...
5gi96g5psgp;7ikdfl3nkyj;zftnvonktj6;aipjqmvzj5...
2023-06-08 16:06:07.658
NaT
None
kr2novmm4i9i27z5adyj
jf7wuu;
Stopped
97
PNFURuqn5MBZk110Fh
wttkvt-GJjq8;yvo1ds-96vns;cgvhiy-BgnXM;v2ow93-...
ro72gckii7a;dnqed48moze;xb7v9iap72l;8heewks755...
2023-06-08 16:05:50.800
NaT
None
kr2novmm4i9i27z5adyj
jf7wuu;
Stopped
98
wxJMPLZFgS9gqClzF6
wttkvt-GJjq8;yvo1ds-96vns;cgvhiy-BgnXM;v2ow93-...
9a06gtkzws7;4z8l2w2pq5n;2z9k699i4su;w8f5ftv7f9...
2023-06-08 16:05:15.441
NaT
None
kr2novmm4i9i27z5adyj
jf7wuu;
Stopped
99 rows × 9 columns
10. Directly run local python process using PyGeoweaver
= '100001' , process_id= 'nlsntt' , sync_path= 'retrieve_sst.py' )
Enter password for host - 100001: ········
Updating code on workflow with the given file path.
Staring process nlsntt
Execution finished
Total time cost: 41 seconds
+--------------------+--------+---------------------+---------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-------+
| History Id | Status | Begin Time | End Time | Input | Output | Notes |
+--------------------+--------+---------------------+---------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-------+
| orznud2v2ugsyqfjlj | Done | 06/21/2023 03:40:26 | 06/21/2023 03:41:00 | from earth_data_utils import *
def search_and_get_sst():
auth_earthaccess()
results = earthaccess.search_data(
short_name=sst_short_name,
cloud_hosted=True,
temporal=("2021-07-01", "2021-07-02"),
)
print("found results: ", results)
return results
def plot_subset(ds_subset):
ds_subset.plot(figsize=(12,6), x='lon', y='lat')
home_dir = os.path.expanduser('~')
file_name = f"geoweaver_plot_sst.png"
save_file_path = os.path.join(home_dir, file_name)
plt.savefig(save_file_path)
print(f"new figure is saved to {save_file_path}")
with get_dataset(search_and_get_sst()) as ds:
subset = ds["analysed_sst"].sel(lat=slice(37, 47),
lon=slice(-70,-50)).std('time')
plot_subset(subset)
| You're now authenticated with NASA Earthdata Login
Using token with expiration date: 08/19/2023
Using .netrc file for EDL
Granules found: 2
found results: [Collection: {'Version': '4.1', 'ShortName': 'MUR-JPL-L4-GLOB-v4.1'}
Spatial coverage: {'HorizontalSpatialDomain': {'Geometry': {'BoundingRectangles': [{'WestBoundingCoordinate': -180, 'SouthBoundingCoordinate': -90, 'EastBoundingCoordinate': 180, 'NorthBoundingCoordinate': 90}]}}}
Temporal coverage: {'RangeDateTime': {'EndingDateTime': '2021-07-01T21:00:00.000Z', 'BeginningDateTime': '2021-06-30T21:00:00.000Z'}}
Size(MB): 0
Data: ['https://archive.podaac.earthdata.nasa.gov/podaac-ops-cumulus-protected/MUR-JPL-L4-GLOB-v4.1/20210701090000-JPL-L4_GHRSST-SSTfnd-MUR-GLOB-v02.0-fv04.1.nc'], Collection: {'Version': '4.1', 'ShortName': 'MUR-JPL-L4-GLOB-v4.1'}
Spatial coverage: {'HorizontalSpatialDomain': {'Geometry': {'BoundingRectangles': [{'WestBoundingCoordinate': -180, 'SouthBoundingCoordinate': -90, 'EastBoundingCoordinate': 180, 'NorthBoundingCoordinate': 90}]}}}
Temporal coverage: {'RangeDateTime': {'EndingDateTime': '2021-07-02T21:00:00.000Z', 'BeginningDateTime': '2021-07-01T21:00:00.000Z'}}
Size(MB): 0
Data: ['https://archive.podaac.earthdata.nasa.gov/podaac-ops-cumulus-protected/MUR-JPL-L4-GLOB-v4.1/20210702090000-JPL-L4_GHRSST-SSTfnd-MUR-GLOB-v02.0-fv04.1.nc']]
Opening 2 granules, approx size: 0.0 GB
QUEUEING TASKS | : 0%| | 0/2 [00:00<?, ?it/s]
QUEUEING TASKS | : 100%|██████████| 2/2 [00:00<00:00, 2267.80it/s]
PROCESSING TASKS | : 0%| | 0/2 [00:00<?, ?it/s]
PROCESSING TASKS | : 50%|█████ | 1/2 [00:00<00:00, 1.53it/s]
PROCESSING TASKS | : 100%|██████████| 2/2 [00:00<00:00, 2.71it/s]
COLLECTING RESULTS | : 0%| | 0/2 [00:00<?, ?it/s]
COLLECTING RESULTS | : 100%|██████████| 2/2 [00:00<00:00, 16131.94it/s]
new figure is saved to /Users/joe/geoweaver_plot_sst.png
| - |
+--------------------+--------+---------------------+---------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-------+
Use the same code to visualize the result in a plot.
= os.path.join(home_dir, 'geoweaver_plot_sst.png' )))
This image has the same texture as we got in the previous workflow run, and it means running local python directly in pygeoweaver will have the same outputs and won’t impact the consistency.
PyGeoWeaver offers significant benefits over manual management of geospatial processes. It ensures that result images remain consistent regardless of file location changes, enables seamless integration with your local Python file, and provides a repeatable and automated process. By utilizing PyGeoWeaver, you can achieve consistent, reproducible, and efficient geospatial analysis, enhancing your productivity and reducing the burden of manual workflow management.
11. Geoweaver GUI for creating and editing workflows
There is something not covered in this tutorial about how to create a new workflow in Geoweaver. We usually do that in the Geoweaver GUI.
Check if you can see Geoweaver GUI using the show method like the picture below.
If you’re running Jupyter on your localhost, accessing Geoweaver is like a breeze. If you’re on your localhost, simply type gw.show in a Jupyter cell block or navigate to http://localhost:8070/Geoweaver in your browser, and you’re good to go. However, if you’re running Jupyter on a remote host, a little extra setup is needed to access Geoweaver.
To access Geoweaver on a remote host, you’ll need to proxy the URL to a public address. For instance, if you’re using an AWS EC2 instance, replace “localhost” in the URL with the public IP address of your instance. Then enter the modified URL in your browser. Don’t forget to add port 8070 to the inbound rule in your EC2 instance’s security group settings to ensure a smooth connection.
If you’re eager to explore Geoweaver’s GUI and dive into detailed tutorials, you can find them at https://esipfed.github.io/Geoweaver/. However, in this notebook, our focus is on demonstrating the incredible capabilities of Geoweaver for replicating and reproducing workflows.
2023-06-21 03:17:27,586 - INFO - enter ipython block
2023-06-21 03:17:27,587 - WARNING - This only works when the Jupyter is visited from localhost!
