Jupyter Meets the Earth#

This is the project home for the EarthCube-funded project (Awards 1928406 and 1928374): Jupyter meets the Earth: Enabling discovery in geoscience through interactive computing at scale.

Project description#

The Jupyter meets the Earth project combines research use cases in geosciences with technical developments within the Jupyter and Pangeo ecosystems. In this model of equal partners, scientific questions help drive software infrastructure development, and new technologies expand the horizons of viable research.

The full project description is available in the original grant proposal.

What you can find here#

In addition to the project updates and our event archive, here you can find onboarding resources for Jupyter and the other related tools, tailored for Earth science researchers. Jupyter ecosystem helps advance Earth science research in many ways, including data exploration, interactive analysis, documentation, and reproducibility of work. To give you an idea of how Jupyter achieves this, some of these tutorials come along with a motivating example that focuses on real science problems using real data.

Demos#

Our YouTube channel and video resources#

Our YouTube channel hosts the meeting records and learning resources delivered by the team members. You can also find several playlists introducing our project or Jupyter tools for geoscientists.

Blog Posts#

Past presentations and events#

2022#

EarthCube annual meeting#

June 14-16, 2022

  • We gave a talk entitled “Let supplemental material be FAIR: using narrative and reusable Jupyter Book to complement your journal paper.” It highlighted Jupyter Book as an ideal tool for making supplemental information of an academic article in terms of accessibility, reproducibility, and the FAIR concept.

  • We hosted a workshop entitled “Keeping your head in the clouds: reproducible, collaborative science with open cloud infrastructure” to reach out to the community about the concept of living-in-the-cloud science. We reviewed the recent development of the Jupyter tools and introduced our recommended cloud-native research practices to the audience.

    • Workshop slides (to be updated)

  • Poster presentation: Moges et al. (2022). HydroBench: Jupyter Supported Reproducible Hydrological Model Benchmarking and Diagnostic tool. Our hydro team also presented HydroBench: Jupyter Supported Reproducible Hydrological Model Benchmarking and Diagnostic tool.

Project Tyra Seminar Series#

January 22, 2022

Whyjay Zheng gave a talk introducing the JMTE project and the Jupyter ecosystem to researchers in/from Taiwan.

2021#

American Geophysical Union Fall meeting#

December 13-17, 2021

We delivered the following presentations in the 2021 AGU Fall meeting:

  • Sapienza et al.: Spectral Unmixing of Antarctic Snow Grain Size Distribution: A Data-Driven Perspective

  • Sundell et al.: How a cloud based JupyterHub can scaffold researchers’ scientific workflows and teaching activities

  • Zheng et al.: Gftt: an open-source tool for evaluating remotely sensed glacier velocity products

  • Snow et al.: Basal channel outflow inferred from persistent polynya variability at the Eastern Thwaites Ice Shelf

  • Grigsby et al.: Mission in a minute: Complex Spatial Query and Data Access in the Cloud for the ICESat-2 Mission

  • Sapienza et al.: From MAVEN to Bayes: A Data-Driven Approach to Mars’ Space Environment

  • Azari et al.: Mars’ Global Magnetic Field Response To Localized Crustal Fields and Interplanetary Magnetic Field Direction

  • Bolibar et al.: Optimizing and Discovering Models of Glacier Processes with Neural Networks Embedded in Differential Equations

  • Pérez et al.: Scientific Communication and Reproducible Publishing in the Jupyter Ecosystem and Beyond (Invited)

  • Badman et al.: Pandemic Programming for an REU: Developing a Virtual Bootcamp and Professional Development Series for the Advancing Space Sciences through Undergraduate Research Experiences (ASSURE) Program

  • Moges et al.: HydroBench: Jupyter Supported Reproducible Hydrological Model Benchmarking and Diagnostic tool.

Seminars in Taiwan#

July 12, July 15, and August 4-6, 2021

Team member Whyjay Zheng was invited to host three virtual and in-person workshops in Taiwan (at Taiwan University, Taiwan Normal University, and Cheng Kung University, respectively), aiming to engage the international Earth science community with the Jupyter ecosystem and tools.

EarthCube annual meeting#

June 15-17, 2021

In addition to the update about a Cryoscience use case (EZ-FeatureTrack), the JMTE project team hosted a workshop entitled “The Jupyter Landscape: a high-level map for (geo)scientists”, aiming to provide onboarding resources and tutorials for earth scientists to use Jupyter tools.

  • EZ-FeatureTrack project notebook: an interactive, Jopyter notebook-based interface that deploys an entire glacier feature tracking workflow. This open-source tool aims to provide researchers, educators, and other users an accessible and reproducible method for mapping glacier velocity over time.

  • Jupyter workshop:

    • slides (to be updated)

    • recordings and videos (to be updated)

2nd Eddy Cross Disciplinary Symposium#

June 7-10, 2021

Fernando Pérez participated in a planery session.

Big Data from Space 2021#

May 18-20, 2021

Fernando Pérez gave a keynote talk entitled “From Interactive Computing To Collaborative Science:Opportunities In The Cloud With Open Infrastructure.”

CSDMS 2021#

May 17-21, 2021

Fernando Pérez gave a keynote talk focusing on the Jupyter technology at the Community Surface Dynamics Modeling System (CSDMS) annual meeting.

The 50th Arctic workshop#

April 15-17, 2021

We gave a virtual presentation about the Cryoscience use case entitled “Mapping ice flow velocity using an interactive, cloud-based feature tracking workflow”:

CIG developers meeting#

February 25, 2021

We were invited to present a brief overview of the Pangeo ecosystem at the 2021 Computational Infrastructure for Geodynamics developers workshop

ESIP winter meeting#

January 26, 2021

As a part of the ESIP winter meeting, we were invited to present in the session: Jupyter Notebooks: Harnessing the full potential

2020#

American Geophysical Union Fall meeting#

December 1-17, 2020

As a part of the AGU Annual meeting, we presented:

  • IN002-05 Jupyter meets the Earth: advancing an open ecosystem that supports science

  • GP005-04 Advances in Magnetotelluric modelling and inversion with SimPEG

  • C004-0010 Multimodal Dataset Integration for Cloud Masking of ICESat-2

  • IN002-07 Fusion of ICESat-2 and complementary remote sensing data for interactive visualization in Jupyter

  • H141-0025 Spatio-Temporal Interpolation of Cloud Data

  • H140-0003 Jupyer Supported Interactive Data Processing Workflow for Intensively Monitored Watersheds across the US

  • TH086 Scripts to Software Frameworks: How Do We Share Our Research Code?

  • SCIWS13 Best Practices for Developing and Sustaining Your Open-Source Research Software

Supercomputing Conference (SC)#

November 11-19, 2020

As a part of the Fourth Workshop on Interactive High-Performance Computing @ SC20, Anderson Banihirwe spoke on a panel about Interactive High Performance Computing.

EarthCube Webinar#

November 6, 2020

We gave a presentation “Jupyter and Pangeo: Building robust, open ecosystems in EarthCube” at the EarthCube Webinar Joining the EarthCube Community as a Funded Project - a Discussion on the 2021 NSF EarthCube Solicitation.

Next Generation Ecosystems Experiments (NGEE) Seminars#

October 19, 2020

As a part of the NGEE-Tropics seminar series, we gave a presentation entitled “Jupyter meets the Earth: partnering domain research with open-source development”:

JupyterCon Presentation#

October 15, 2020

JupyterCon was held online from October 5-17. We presented a poster on the Jupyter meets the Earth project:

Discussion with Coiled#

September 24, 2020

As a part of the Science Thursdays livestream events hosted by Hugo Bowne-Anderson and Matthew Rocklin, Lindsey Heagy joined for a conversation on using dask for inverse problems in geophysics

EarthCube Council of Funded projects#

September 10, 2020

We gave a project-overview presentation as a part of the EarthCube Council of Funded Projects regular meeting series:

EarthCube community meeting#

July 27, 2020

As a part of the EarthCube 2020 meeting, we held a two hour session that included short presentations about the Jupyter meets the Earth project, lightning talks from members of the community and time for Q&A.

IceSat2 Hackweek#

June 8-18, 2020

Several team members participated as instructors and joing in the project-work at the 2020 ICESat-2 Hackweek. The tutorials that project team members Fernando Pérez and Lindsey Heagy contributed to were on:

Transform 2020#

June 6-14, 2020

As a part of the Transform 2020 virtual conference, Lindsey Heagy gave a tutorial on geophysical inversions with SimPEG.