Accessing Pangeo Climate Data
Accessing Pangeo Climate Data¶
In this notebook, we select and pull CESM data from Pangeo using our pip-installable library, cesm.
# Import libraries
import os
import pandas as pd
## After pip install,
from cesm import cesm_pull as cpull
# Read in available datasets, filtered for NCAR's CESM
ncar_models = cpull.find_available_data()
ncar_models.drop(columns=['zstore']) # dropping link for cleaner ouput
| activity_id | institution_id | source_id | experiment_id | member_id | table_id | variable_id | grid_label | dcpp_init_year | version | |
|---|---|---|---|---|---|---|---|---|---|---|
| 198051 | CMIP | NCAR | CESM2 | historical | r11i1p1f1 | Amon | hurs | gn | NaN | 20190514 |
| 198054 | CMIP | NCAR | CESM2 | historical | r11i1p1f1 | Amon | huss | gn | NaN | 20190514 |
| 198057 | CMIP | NCAR | CESM2 | historical | r11i1p1f1 | Amon | pr | gn | NaN | 20190514 |
| 198172 | CMIP | NCAR | CESM2 | historical | r11i1p1f1 | Amon | tas | gn | NaN | 20190514 |
| 437199 | ScenarioMIP | NCAR | CESM2 | ssp126 | r11i1p1f1 | Amon | huss | gn | NaN | 20200528 |
| 437549 | ScenarioMIP | NCAR | CESM2 | ssp126 | r11i1p1f1 | Amon | tasmax | gn | NaN | 20200528 |
| 437550 | ScenarioMIP | NCAR | CESM2 | ssp126 | r11i1p1f1 | Amon | tasmin | gn | NaN | 20200528 |
| 437555 | ScenarioMIP | NCAR | CESM2 | ssp126 | r11i1p1f1 | Amon | tas | gn | NaN | 20200528 |
| 437687 | ScenarioMIP | NCAR | CESM2 | ssp126 | r11i1p1f1 | Amon | hurs | gn | NaN | 20200528 |
| 437792 | ScenarioMIP | NCAR | CESM2 | ssp126 | r11i1p1f1 | Amon | pr | gn | NaN | 20200528 |
| 437906 | ScenarioMIP | NCAR | CESM2 | ssp585 | r11i1p1f1 | Amon | pr | gn | NaN | 20200528 |
| 437907 | ScenarioMIP | NCAR | CESM2 | ssp585 | r11i1p1f1 | Amon | huss | gn | NaN | 20200528 |
| 437918 | ScenarioMIP | NCAR | CESM2 | ssp585 | r11i1p1f1 | Amon | hurs | gn | NaN | 20200528 |
| 437977 | ScenarioMIP | NCAR | CESM2 | ssp585 | r11i1p1f1 | Amon | tasmin | gn | NaN | 20200528 |
| 437978 | ScenarioMIP | NCAR | CESM2 | ssp585 | r11i1p1f1 | Amon | tasmax | gn | NaN | 20200528 |
| 437979 | ScenarioMIP | NCAR | CESM2 | ssp585 | r11i1p1f1 | Amon | tas | gn | NaN | 20200528 |
| 438988 | ScenarioMIP | NCAR | CESM2 | ssp245 | r11i1p1f1 | Amon | tas | gn | NaN | 20200528 |
| 439008 | ScenarioMIP | NCAR | CESM2 | ssp370 | r11i1p1f1 | Amon | tas | gn | NaN | 20200528 |
| 439033 | ScenarioMIP | NCAR | CESM2 | ssp370 | r11i1p1f1 | Amon | tasmax | gn | NaN | 20200528 |
| 439034 | ScenarioMIP | NCAR | CESM2 | ssp370 | r11i1p1f1 | Amon | tasmin | gn | NaN | 20200528 |
| 439072 | ScenarioMIP | NCAR | CESM2 | ssp370 | r11i1p1f1 | Amon | pr | gn | NaN | 20200528 |
| 439073 | ScenarioMIP | NCAR | CESM2 | ssp370 | r11i1p1f1 | Amon | huss | gn | NaN | 20200528 |
| 439075 | ScenarioMIP | NCAR | CESM2 | ssp370 | r11i1p1f1 | Amon | hurs | gn | NaN | 20200528 |
| 439080 | ScenarioMIP | NCAR | CESM2 | ssp245 | r11i1p1f1 | Amon | tasmax | gn | NaN | 20200528 |
| 439227 | ScenarioMIP | NCAR | CESM2 | ssp245 | r11i1p1f1 | Amon | hurs | gn | NaN | 20200528 |
| 439229 | ScenarioMIP | NCAR | CESM2 | ssp245 | r11i1p1f1 | Amon | huss | gn | NaN | 20200528 |
| 439230 | ScenarioMIP | NCAR | CESM2 | ssp245 | r11i1p1f1 | Amon | pr | gn | NaN | 20200528 |
| 439241 | ScenarioMIP | NCAR | CESM2 | ssp245 | r11i1p1f1 | Amon | tasmin | gn | NaN | 20200528 |
# Assess unique values of ncar_models
meval = cpull.model_eval(ncar_models)
(28, 11)
activity_id : ['ScenarioMIP', 'CMIP']
institution_id : ['NCAR']
source_id : ['CESM2']
experiment_id : ['ssp370', 'historical', 'ssp245', 'ssp126', 'ssp585']
member_id : ['r11i1p1f1']
table_id : ['Amon']
variable_id : ['tasmin', 'huss', 'tas', 'hurs', 'pr', 'tasmax']
grid_label : ['gn']
version : [20200528, 20190514]
| activity_id | institution_id | source_id | experiment_id | member_id | table_id | variable_id | grid_label | zstore | dcpp_init_year | version | |
|---|---|---|---|---|---|---|---|---|---|---|---|
| 198051 | CMIP | NCAR | CESM2 | historical | r11i1p1f1 | Amon | hurs | gn | gs://cmip6/CMIP6/CMIP/NCAR/CESM2/historical/r1... | NaN | 20190514 |
| 437550 | ScenarioMIP | NCAR | CESM2 | ssp126 | r11i1p1f1 | Amon | tasmin | gn | gs://cmip6/CMIP6/ScenarioMIP/NCAR/CESM2/ssp126... | NaN | 20200528 |
| 437918 | ScenarioMIP | NCAR | CESM2 | ssp585 | r11i1p1f1 | Amon | hurs | gn | gs://cmip6/CMIP6/ScenarioMIP/NCAR/CESM2/ssp585... | NaN | 20200528 |
| 439033 | ScenarioMIP | NCAR | CESM2 | ssp370 | r11i1p1f1 | Amon | tasmax | gn | gs://cmip6/CMIP6/ScenarioMIP/NCAR/CESM2/ssp370... | NaN | 20200528 |
| 439227 | ScenarioMIP | NCAR | CESM2 | ssp245 | r11i1p1f1 | Amon | hurs | gn | gs://cmip6/CMIP6/ScenarioMIP/NCAR/CESM2/ssp245... | NaN | 20200528 |
# Use the grab_data() function to check the output of one model variable
ds = cpull.grab_data(ncar_models, 439230)
ds
<xarray.Dataset>
Dimensions: (lat: 192, nbnd: 2, lon: 288, time: 1032)
Coordinates:
* lat (lat) float64 -90.0 -89.06 -88.12 -87.17 ... 88.12 89.06 90.0
lat_bnds (lat, nbnd) float64 dask.array<chunksize=(192, 2), meta=np.ndarray>
* lon (lon) float64 0.0 1.25 2.5 3.75 5.0 ... 355.0 356.2 357.5 358.8
lon_bnds (lon, nbnd) float64 dask.array<chunksize=(288, 2), meta=np.ndarray>
* time (time) object 2015-01-15 12:00:00 ... 2100-12-15 12:00:00
time_bnds (time, nbnd) object dask.array<chunksize=(1032, 2), meta=np.ndarray>
Dimensions without coordinates: nbnd
Data variables:
pr (time, lat, lon) float32 dask.array<chunksize=(288, 192, 288), meta=np.ndarray>
Attributes: (12/48)
Conventions: CF-1.7 CMIP-6.2
activity_id: ScenarioMIP
branch_method: standard
branch_time_in_child: 735110.0
branch_time_in_parent: 735110.0
case_id: 1731
... ...
tracking_id: hdl:21.14100/077f7679-2555-431e-a864-29011759e8e7...
variable_id: pr
variant_info: CMIP6 SSP2-4.5 experiments (2015-2100) with CAM6,...
variant_label: r11i1p1f1
netcdf_tracking_ids: hdl:21.14100/077f7679-2555-431e-a864-29011759e8e7...
version_id: v20200528# Plot the model variable (precipitation) at one timestamp
ds.sel(time='2015-01-15')['pr'].plot(figsize=(14, 10));
# Assess the selected model variables to check for consistency in dims/coords
## All CMIP data have same coords, and all ScenarioMIP data have same coords
cpull.data_eval(ncar_models)
| activity_id | experiment_id | variable_id | lat_size | lat_min | lat_max | lon_size | lon_min | lon_max | time_size | time_min | time_max | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 198051 | CMIP | historical | hurs | 192 | -90.0 | 90.0 | 288 | 0.0 | 358.75 | 1980 | 1850-01-15 12:00:00 | 2014-12-15 12:00:00 |
| 198054 | CMIP | historical | huss | 192 | -90.0 | 90.0 | 288 | 0.0 | 358.75 | 1980 | 1850-01-15 12:00:00 | 2014-12-15 12:00:00 |
| 198057 | CMIP | historical | pr | 192 | -90.0 | 90.0 | 288 | 0.0 | 358.75 | 1980 | 1850-01-15 12:00:00 | 2014-12-15 12:00:00 |
| 198172 | CMIP | historical | tas | 192 | -90.0 | 90.0 | 288 | 0.0 | 358.75 | 1980 | 1850-01-15 12:00:00 | 2014-12-15 12:00:00 |
| 437687 | ScenarioMIP | ssp126 | hurs | 192 | -90.0 | 90.0 | 288 | 0.0 | 358.75 | 1032 | 2015-01-15 12:00:00 | 2100-12-15 12:00:00 |
| 437199 | ScenarioMIP | ssp126 | huss | 192 | -90.0 | 90.0 | 288 | 0.0 | 358.75 | 1032 | 2015-01-15 12:00:00 | 2100-12-15 12:00:00 |
| 437792 | ScenarioMIP | ssp126 | pr | 192 | -90.0 | 90.0 | 288 | 0.0 | 358.75 | 1032 | 2015-01-15 12:00:00 | 2100-12-15 12:00:00 |
| 437555 | ScenarioMIP | ssp126 | tas | 192 | -90.0 | 90.0 | 288 | 0.0 | 358.75 | 1032 | 2015-01-15 12:00:00 | 2100-12-15 12:00:00 |
| 437549 | ScenarioMIP | ssp126 | tasmax | 192 | -90.0 | 90.0 | 288 | 0.0 | 358.75 | 1032 | 2015-01-15 12:00:00 | 2100-12-15 12:00:00 |
| 437550 | ScenarioMIP | ssp126 | tasmin | 192 | -90.0 | 90.0 | 288 | 0.0 | 358.75 | 1032 | 2015-01-15 12:00:00 | 2100-12-15 12:00:00 |
| 439227 | ScenarioMIP | ssp245 | hurs | 192 | -90.0 | 90.0 | 288 | 0.0 | 358.75 | 1032 | 2015-01-15 12:00:00 | 2100-12-15 12:00:00 |
| 439229 | ScenarioMIP | ssp245 | huss | 192 | -90.0 | 90.0 | 288 | 0.0 | 358.75 | 1032 | 2015-01-15 12:00:00 | 2100-12-15 12:00:00 |
| 439230 | ScenarioMIP | ssp245 | pr | 192 | -90.0 | 90.0 | 288 | 0.0 | 358.75 | 1032 | 2015-01-15 12:00:00 | 2100-12-15 12:00:00 |
| 438988 | ScenarioMIP | ssp245 | tas | 192 | -90.0 | 90.0 | 288 | 0.0 | 358.75 | 1032 | 2015-01-15 12:00:00 | 2100-12-15 12:00:00 |
| 439080 | ScenarioMIP | ssp245 | tasmax | 192 | -90.0 | 90.0 | 288 | 0.0 | 358.75 | 1032 | 2015-01-15 12:00:00 | 2100-12-15 12:00:00 |
| 439241 | ScenarioMIP | ssp245 | tasmin | 192 | -90.0 | 90.0 | 288 | 0.0 | 358.75 | 1032 | 2015-01-15 12:00:00 | 2100-12-15 12:00:00 |
| 439075 | ScenarioMIP | ssp370 | hurs | 192 | -90.0 | 90.0 | 288 | 0.0 | 358.75 | 1032 | 2015-01-15 12:00:00 | 2100-12-15 12:00:00 |
| 439073 | ScenarioMIP | ssp370 | huss | 192 | -90.0 | 90.0 | 288 | 0.0 | 358.75 | 1032 | 2015-01-15 12:00:00 | 2100-12-15 12:00:00 |
| 439072 | ScenarioMIP | ssp370 | pr | 192 | -90.0 | 90.0 | 288 | 0.0 | 358.75 | 1032 | 2015-01-15 12:00:00 | 2100-12-15 12:00:00 |
| 439008 | ScenarioMIP | ssp370 | tas | 192 | -90.0 | 90.0 | 288 | 0.0 | 358.75 | 1032 | 2015-01-15 12:00:00 | 2100-12-15 12:00:00 |
| 439033 | ScenarioMIP | ssp370 | tasmax | 192 | -90.0 | 90.0 | 288 | 0.0 | 358.75 | 1032 | 2015-01-15 12:00:00 | 2100-12-15 12:00:00 |
| 439034 | ScenarioMIP | ssp370 | tasmin | 192 | -90.0 | 90.0 | 288 | 0.0 | 358.75 | 1032 | 2015-01-15 12:00:00 | 2100-12-15 12:00:00 |
| 437918 | ScenarioMIP | ssp585 | hurs | 192 | -90.0 | 90.0 | 288 | 0.0 | 358.75 | 1032 | 2015-01-15 12:00:00 | 2100-12-15 12:00:00 |
| 437907 | ScenarioMIP | ssp585 | huss | 192 | -90.0 | 90.0 | 288 | 0.0 | 358.75 | 1032 | 2015-01-15 12:00:00 | 2100-12-15 12:00:00 |
| 437906 | ScenarioMIP | ssp585 | pr | 192 | -90.0 | 90.0 | 288 | 0.0 | 358.75 | 1032 | 2015-01-15 12:00:00 | 2100-12-15 12:00:00 |
| 437979 | ScenarioMIP | ssp585 | tas | 192 | -90.0 | 90.0 | 288 | 0.0 | 358.75 | 1032 | 2015-01-15 12:00:00 | 2100-12-15 12:00:00 |
| 437978 | ScenarioMIP | ssp585 | tasmax | 192 | -90.0 | 90.0 | 288 | 0.0 | 358.75 | 1032 | 2015-01-15 12:00:00 | 2100-12-15 12:00:00 |
| 437977 | ScenarioMIP | ssp585 | tasmin | 192 | -90.0 | 90.0 | 288 | 0.0 | 358.75 | 1032 | 2015-01-15 12:00:00 | 2100-12-15 12:00:00 |
# Check how many variables each model has
## The 2 extra in ScenarioMIP are for 'tasmin' and 'tasmax'
model_groups = pd.DataFrame({
'# variables': ncar_models.groupby(['activity_id', 'experiment_id']).size()
})
model_groups
| # variables | ||
|---|---|---|
| activity_id | experiment_id | |
| CMIP | historical | 4 |
| ScenarioMIP | ssp126 | 6 |
| ssp245 | 6 | |
| ssp370 | 6 | |
| ssp585 | 6 |
# Read in the selected model data!
if os.path.exists('../data/historical.zarr'):
print('Data already downloaded!')
else:
cpull.model_pull()
Completed downloading ('CMIP', 'historical')!
Completed downloading ('ScenarioMIP', 'ssp126')!
Completed downloading ('ScenarioMIP', 'ssp245')!
Completed downloading ('ScenarioMIP', 'ssp370')!
Completed downloading ('ScenarioMIP', 'ssp585')!