.. DO NOT EDIT. .. THIS FILE WAS AUTOMATICALLY GENERATED BY SPHINX-GALLERY. .. TO MAKE CHANGES, EDIT THE SOURCE PYTHON FILE: .. "tutorials/chain.py" .. LINE NUMBERS ARE GIVEN BELOW. .. only:: html .. note:: :class: sphx-glr-download-link-note Click :ref:`here ` to download the full example code .. rst-class:: sphx-glr-example-title .. _sphx_glr_tutorials_chain.py: Chaining Operations =================== Often, a data processing pipeline looks like the following: #. Apply a blocked mean or median to the data #. Remove a trend from the blocked data #. Fit a spline to the residual of the trend #. Grid using the spline and restore the trend The :class:`verde.Chain` class allows us to created gridders that perform multiple operations on data. Each step in the chain filters the input and passes the result along to the next step. For gridders and trend estimators, filtering means fitting the model and passing along the residuals (input data minus predicted data). When predicting data, the predictions of each step are added together. Other operations, like :class:`verde.BlockReduce` and :class:`verde.BlockMean` change the input data values and the coordinates but don't impact the predictions because they don't implement the :meth:`~verde.base.BaseGridder.predict` method. .. note:: The :class:`~verde.Chain` class was inspired by the :class:`sklearn.pipeline.Pipeline` class, which doesn't serve our purposes because it only affects the feature matrix, not what we would call *data* (the target vector). For example, let's create a pipeline to grid our sample bathymetry data. .. GENERATED FROM PYTHON SOURCE LINES 39-69 .. code-block:: default import matplotlib.pyplot as plt import numpy as np import pygmt import pyproj import verde as vd data = vd.datasets.fetch_baja_bathymetry() region = vd.get_region((data.longitude, data.latitude)) # The desired grid spacing in degrees spacing = 10 / 60 # Use Mercator projection because Spline is a Cartesian gridder projection = pyproj.Proj(proj="merc", lat_ts=data.latitude.mean()) proj_coords = projection(data.longitude.values, data.latitude.values) fig = pygmt.Figure() fig.coast(region=region, projection="M20c", land="#666666") pygmt.makecpt(cmap="viridis", series=[data.bathymetry_m.min(), data.bathymetry_m.max()]) fig.plot( x=data.longitude, y=data.latitude, color=data.bathymetry_m, cmap=True, style="c0.05c", ) fig.basemap(frame=True) fig.colorbar(frame='af+l"bathymetric depth [m]"') fig.show() .. image-sg:: /tutorials/images/sphx_glr_chain_001.png :alt: chain :srcset: /tutorials/images/sphx_glr_chain_001.png :class: sphx-glr-single-img .. rst-class:: sphx-glr-script-out Out: .. code-block:: none .. GENERATED FROM PYTHON SOURCE LINES 70-72 We'll create a chain that applies a blocked median to the data, fits a polynomial trend, and then fits a standard gridder to the trend residuals. .. GENERATED FROM PYTHON SOURCE LINES 72-82 .. code-block:: default chain = vd.Chain( [ ("reduce", vd.BlockReduce(np.median, spacing * 111e3)), ("trend", vd.Trend(degree=1)), ("spline", vd.Spline()), ] ) print(chain) .. rst-class:: sphx-glr-script-out Out: .. code-block:: none Chain(steps=[('reduce', BlockReduce(reduction=, spacing=18500.0)), ('trend', Trend(degree=1)), ('spline', Spline(mindist=0))]) .. GENERATED FROM PYTHON SOURCE LINES 83-89 Calling :meth:`verde.Chain.fit` will automatically run the data through the chain: #. Apply the blocked median to the input data #. Fit a trend to the blocked data and output the residuals #. Fit the spline to the trend residuals .. GENERATED FROM PYTHON SOURCE LINES 89-92 .. code-block:: default chain.fit(proj_coords, data.bathymetry_m) .. raw:: html 
Chain(steps=[('reduce',
                  BlockReduce(reduction=<function median at 0x7fab4c1501f0>,
                              spacing=18500.0)),
                 ('trend', Trend(degree=1)), ('spline', Spline(mindist=0))])
In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook.
On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.


.. GENERATED FROM PYTHON SOURCE LINES 93-102 Now that the data has been through the chain, calling :meth:`verde.Chain.predict` will sum the results of every step in the chain that has a ``predict`` method. In our case, that will be only the :class:`~verde.Trend` and :class:`~verde.Spline`. We can verify the quality of the fit by inspecting a histogram of the residuals with respect to the original data. Remember that our spline and trend were fit on decimated data, not the original data, so the fit won't be perfect. .. GENERATED FROM PYTHON SOURCE LINES 102-112 .. code-block:: default residuals = data.bathymetry_m - chain.predict(proj_coords) plt.figure() plt.title("Histogram of fit residuals") plt.hist(residuals, bins="auto", density=True) plt.xlabel("residuals (m)") plt.xlim(-1500, 1500) plt.show() .. image-sg:: /tutorials/images/sphx_glr_chain_002.png :alt: Histogram of fit residuals :srcset: /tutorials/images/sphx_glr_chain_002.png :class: sphx-glr-single-img .. GENERATED FROM PYTHON SOURCE LINES 113-116 Likewise, :meth:`verde.Chain.grid` creates a grid of the combined trend and spline predictions. This is equivalent to a *remove-compute-restore* procedure that should be familiar to the geodesists among us. .. GENERATED FROM PYTHON SOURCE LINES 116-132 .. code-block:: default grid = chain.grid( region=region, spacing=spacing, projection=projection, dims=["latitude", "longitude"], data_names="bathymetry", ) grid = vd.distance_mask( data_coordinates=(data.longitude, data.latitude), maxdist=spacing * 111e3, grid=grid, projection=projection, ) grid .. raw:: html 
<xarray.Dataset>
    Dimensions:     (latitude: 61, longitude: 59)
    Coordinates:
      * longitude   (longitude) float64 245.0 245.2 245.3 ... 254.4 254.5 254.7
      * latitude    (latitude) float64 20.0 20.17 20.33 20.5 ... 29.66 29.82 29.99
    Data variables:
        bathymetry  (latitude, longitude) float64 -3.621e+03 -3.709e+03 ... nan nan
    Attributes:
        metadata:  Generated by Chain(steps=[('reduce',\n              BlockReduc...


.. GENERATED FROM PYTHON SOURCE LINES 133-134 Finally, we can plot the resulting grid: .. GENERATED FROM PYTHON SOURCE LINES 134-146 .. code-block:: default fig = pygmt.Figure() fig.coast(region=region, projection="M20c", land="#666666") fig.grdimage( grid=grid.bathymetry, cmap="viridis", nan_transparent=True, ) fig.basemap(frame=True) fig.colorbar(frame='af+l"bathymetric depth [m]"') fig.show() .. image-sg:: /tutorials/images/sphx_glr_chain_003.png :alt: chain :srcset: /tutorials/images/sphx_glr_chain_003.png :class: sphx-glr-single-img .. rst-class:: sphx-glr-script-out Out: .. code-block:: none .. GENERATED FROM PYTHON SOURCE LINES 147-150 Each component of the chain can be accessed separately using the ``named_steps`` attribute. It's a dictionary with keys and values matching the inputs given to the :class:`~verde.Chain`. .. GENERATED FROM PYTHON SOURCE LINES 150-154 .. code-block:: default print(chain.named_steps["trend"]) print(chain.named_steps["spline"]) .. rst-class:: sphx-glr-script-out Out: .. code-block:: none Trend(degree=1) Spline(mindist=0) .. GENERATED FROM PYTHON SOURCE LINES 155-158 All gridders and estimators in the chain have been fitted and can be used to generate grids and predictions. For example, we can get a grid of the estimated trend: .. GENERATED FROM PYTHON SOURCE LINES 158-174 .. code-block:: default grid_trend = chain.named_steps["trend"].grid( region=region, spacing=spacing, projection=projection, dims=["latitude", "longitude"], data_names="bathymetry", ) grid_trend = vd.distance_mask( data_coordinates=(data.longitude, data.latitude), maxdist=spacing * 111e3, grid=grid_trend, projection=projection, ) grid_trend .. raw:: html 
<xarray.Dataset>
    Dimensions:     (latitude: 61, longitude: 59)
    Coordinates:
      * longitude   (longitude) float64 245.0 245.2 245.3 ... 254.4 254.5 254.7
      * latitude    (latitude) float64 20.0 20.17 20.33 20.5 ... 29.66 29.82 29.99
    Data variables:
        bathymetry  (latitude, longitude) float64 -4.911e+03 -4.864e+03 ... nan nan
    Attributes:
        metadata:  Generated by Trend(degree=1)


.. GENERATED FROM PYTHON SOURCE LINES 176-187 .. code-block:: default fig = pygmt.Figure() fig.coast(region=region, projection="M20c", land="#666666") fig.grdimage( grid=grid_trend.bathymetry, cmap="viridis", nan_transparent=True, ) fig.basemap(frame=True) fig.colorbar(frame='af+l"bathymetric depth [m]"') fig.show() .. image-sg:: /tutorials/images/sphx_glr_chain_004.png :alt: chain :srcset: /tutorials/images/sphx_glr_chain_004.png :class: sphx-glr-single-img .. rst-class:: sphx-glr-script-out Out: .. code-block:: none .. rst-class:: sphx-glr-timing **Total running time of the script:** ( 0 minutes 13.391 seconds) .. _sphx_glr_download_tutorials_chain.py: .. only :: html .. container:: sphx-glr-footer :class: sphx-glr-footer-example .. container:: sphx-glr-download sphx-glr-download-python :download:`Download Python source code: chain.py ` .. container:: sphx-glr-download sphx-glr-download-jupyter :download:`Download Jupyter notebook: chain.ipynb ` .. only:: html .. rst-class:: sphx-glr-signature `Gallery generated by Sphinx-Gallery `_