.. DO NOT EDIT.
.. THIS FILE WAS AUTOMATICALLY GENERATED BY SPHINX-GALLERY.
.. TO MAKE CHANGES, EDIT THE SOURCE PYTHON FILE:
.. "tutorials/model_selection.py"
.. LINE NUMBERS ARE GIVEN BELOW.

.. only:: html

    .. note::
        :class: sphx-glr-download-link-note

        Click :ref:`here <sphx_glr_download_tutorials_model_selection.py>`
        to download the full example code

.. rst-class:: sphx-glr-example-title

.. _sphx_glr_tutorials_model_selection.py:


.. _model_selection:

Model Selection
===============

In :ref:`model_evaluation`, we saw how to check the performance of an
interpolator using cross-validation. We found that the default parameters for
:class:`verde.Spline` are not good for predicting our sample air temperature
data. Now, let's see how we can tune the :class:`~verde.Spline` to improve the
cross-validation performance.

Once again, we'll start by importing the required packages and loading our
sample data.

.. GENERATED FROM PYTHON SOURCE LINES 22-39

.. code-block:: default

    import cartopy.crs as ccrs
    import matplotlib.pyplot as plt
    import numpy as np
    import pyproj

    import verde as vd

    data = vd.datasets.fetch_texas_wind()

    # Use Mercator projection because Spline is a Cartesian gridder
    projection = pyproj.Proj(proj="merc", lat_ts=data.latitude.mean())
    proj_coords = projection(data.longitude.values, data.latitude.values)

    region = vd.get_region((data.longitude, data.latitude))
    # The desired grid spacing in degrees
    spacing = 15 / 60








.. GENERATED FROM PYTHON SOURCE LINES 40-42

Before we begin tuning, let's reiterate what the results were with the
default parameters.

.. GENERATED FROM PYTHON SOURCE LINES 42-51

.. code-block:: default


    spline_default = vd.Spline()
    score_default = np.mean(
        vd.cross_val_score(spline_default, proj_coords, data.air_temperature_c)
    )
    spline_default.fit(proj_coords, data.air_temperature_c)
    print("R² with defaults:", score_default)






.. rst-class:: sphx-glr-script-out

 Out:

 .. code-block:: none

    /usr/share/miniconda3/envs/test/lib/python3.9/site-packages/sklearn/base.py:88: FutureWarning: The mindist parameter of verde.Spline is no longer required and will be removed in Verde 2.0.0. Use the default value to obtain the future behavior.
      new_object = klass(**new_object_params)
    /usr/share/miniconda3/envs/test/lib/python3.9/site-packages/verde/model_selection.py:784: FutureWarning: The default scoring will change from R² to negative root mean squared error (RMSE) in Verde 2.0.0. This may change model selection results slightly.
      score = estimator.score(*test_data)
    /usr/share/miniconda3/envs/test/lib/python3.9/site-packages/sklearn/base.py:88: FutureWarning: The mindist parameter of verde.Spline is no longer required and will be removed in Verde 2.0.0. Use the default value to obtain the future behavior.
      new_object = klass(**new_object_params)
    /usr/share/miniconda3/envs/test/lib/python3.9/site-packages/verde/model_selection.py:784: FutureWarning: The default scoring will change from R² to negative root mean squared error (RMSE) in Verde 2.0.0. This may change model selection results slightly.
      score = estimator.score(*test_data)
    /usr/share/miniconda3/envs/test/lib/python3.9/site-packages/sklearn/base.py:88: FutureWarning: The mindist parameter of verde.Spline is no longer required and will be removed in Verde 2.0.0. Use the default value to obtain the future behavior.
      new_object = klass(**new_object_params)
    /usr/share/miniconda3/envs/test/lib/python3.9/site-packages/verde/model_selection.py:784: FutureWarning: The default scoring will change from R² to negative root mean squared error (RMSE) in Verde 2.0.0. This may change model selection results slightly.
      score = estimator.score(*test_data)
    /usr/share/miniconda3/envs/test/lib/python3.9/site-packages/sklearn/base.py:88: FutureWarning: The mindist parameter of verde.Spline is no longer required and will be removed in Verde 2.0.0. Use the default value to obtain the future behavior.
      new_object = klass(**new_object_params)
    /usr/share/miniconda3/envs/test/lib/python3.9/site-packages/verde/model_selection.py:784: FutureWarning: The default scoring will change from R² to negative root mean squared error (RMSE) in Verde 2.0.0. This may change model selection results slightly.
      score = estimator.score(*test_data)
    /usr/share/miniconda3/envs/test/lib/python3.9/site-packages/sklearn/base.py:88: FutureWarning: The mindist parameter of verde.Spline is no longer required and will be removed in Verde 2.0.0. Use the default value to obtain the future behavior.
      new_object = klass(**new_object_params)
    /usr/share/miniconda3/envs/test/lib/python3.9/site-packages/verde/model_selection.py:784: FutureWarning: The default scoring will change from R² to negative root mean squared error (RMSE) in Verde 2.0.0. This may change model selection results slightly.
      score = estimator.score(*test_data)
    R² with defaults: 0.7960368857061437




.. GENERATED FROM PYTHON SOURCE LINES 52-64

Tuning
------

:class:`~verde.Spline` has the ``damping`` regularization parameter which
smooths the solution and provides a least-squares fit to the data instead of
an exact fit at the observation points. This is often desirable to mitigate
data errors and provide better results when points are widely spaced. Would
changing the default values give us a better score?

We can answer this question by changing the values in our ``spline`` and
re-evaluating the model score repeatedly for different values of this
parameter. Let's test the following values:

.. GENERATED FROM PYTHON SOURCE LINES 64-67

.. code-block:: default


    dampings = [None, 1e-4, 1e-3, 1e-2]








.. GENERATED FROM PYTHON SOURCE LINES 68-70

Now we can loop over each value and collect the scores for each parameter
choice.

.. GENERATED FROM PYTHON SOURCE LINES 70-79

.. code-block:: default


    spline = vd.Spline()
    scores = []
    for damping in dampings:
        spline.set_params(damping=damping)
        score = np.mean(vd.cross_val_score(spline, proj_coords, data.air_temperature_c))
        scores.append(score)
    print(scores)





.. rst-class:: sphx-glr-script-out

 Out:

 .. code-block:: none

    /usr/share/miniconda3/envs/test/lib/python3.9/site-packages/sklearn/base.py:88: FutureWarning: The mindist parameter of verde.Spline is no longer required and will be removed in Verde 2.0.0. Use the default value to obtain the future behavior.
      new_object = klass(**new_object_params)
    /usr/share/miniconda3/envs/test/lib/python3.9/site-packages/verde/model_selection.py:784: FutureWarning: The default scoring will change from R² to negative root mean squared error (RMSE) in Verde 2.0.0. This may change model selection results slightly.
      score = estimator.score(*test_data)
    /usr/share/miniconda3/envs/test/lib/python3.9/site-packages/sklearn/base.py:88: FutureWarning: The mindist parameter of verde.Spline is no longer required and will be removed in Verde 2.0.0. Use the default value to obtain the future behavior.
      new_object = klass(**new_object_params)
    /usr/share/miniconda3/envs/test/lib/python3.9/site-packages/verde/model_selection.py:784: FutureWarning: The default scoring will change from R² to negative root mean squared error (RMSE) in Verde 2.0.0. This may change model selection results slightly.
      score = estimator.score(*test_data)
    /usr/share/miniconda3/envs/test/lib/python3.9/site-packages/sklearn/base.py:88: FutureWarning: The mindist parameter of verde.Spline is no longer required and will be removed in Verde 2.0.0. Use the default value to obtain the future behavior.
      new_object = klass(**new_object_params)
    /usr/share/miniconda3/envs/test/lib/python3.9/site-packages/verde/model_selection.py:784: FutureWarning: The default scoring will change from R² to negative root mean squared error (RMSE) in Verde 2.0.0. This may change model selection results slightly.
      score = estimator.score(*test_data)
    /usr/share/miniconda3/envs/test/lib/python3.9/site-packages/sklearn/base.py:88: FutureWarning: The mindist parameter of verde.Spline is no longer required and will be removed in Verde 2.0.0. Use the default value to obtain the future behavior.
      new_object = klass(**new_object_params)
    /usr/share/miniconda3/envs/test/lib/python3.9/site-packages/verde/model_selection.py:784: FutureWarning: The default scoring will change from R² to negative root mean squared error (RMSE) in Verde 2.0.0. This may change model selection results slightly.
      score = estimator.score(*test_data)
    /usr/share/miniconda3/envs/test/lib/python3.9/site-packages/sklearn/base.py:88: FutureWarning: The mindist parameter of verde.Spline is no longer required and will be removed in Verde 2.0.0. Use the default value to obtain the future behavior.
      new_object = klass(**new_object_params)
    /usr/share/miniconda3/envs/test/lib/python3.9/site-packages/verde/model_selection.py:784: FutureWarning: The default scoring will change from R² to negative root mean squared error (RMSE) in Verde 2.0.0. This may change model selection results slightly.
      score = estimator.score(*test_data)
    /usr/share/miniconda3/envs/test/lib/python3.9/site-packages/sklearn/base.py:88: FutureWarning: The mindist parameter of verde.Spline is no longer required and will be removed in Verde 2.0.0. Use the default value to obtain the future behavior.
      new_object = klass(**new_object_params)
    /usr/share/miniconda3/envs/test/lib/python3.9/site-packages/verde/model_selection.py:784: FutureWarning: The default scoring will change from R² to negative root mean squared error (RMSE) in Verde 2.0.0. This may change model selection results slightly.
      score = estimator.score(*test_data)
    /usr/share/miniconda3/envs/test/lib/python3.9/site-packages/sklearn/base.py:88: FutureWarning: The mindist parameter of verde.Spline is no longer required and will be removed in Verde 2.0.0. Use the default value to obtain the future behavior.
      new_object = klass(**new_object_params)
    /usr/share/miniconda3/envs/test/lib/python3.9/site-packages/verde/model_selection.py:784: FutureWarning: The default scoring will change from R² to negative root mean squared error (RMSE) in Verde 2.0.0. This may change model selection results slightly.
      score = estimator.score(*test_data)
    /usr/share/miniconda3/envs/test/lib/python3.9/site-packages/sklearn/base.py:88: FutureWarning: The mindist parameter of verde.Spline is no longer required and will be removed in Verde 2.0.0. Use the default value to obtain the future behavior.
      new_object = klass(**new_object_params)
    /usr/share/miniconda3/envs/test/lib/python3.9/site-packages/verde/model_selection.py:784: FutureWarning: The default scoring will change from R² to negative root mean squared error (RMSE) in Verde 2.0.0. This may change model selection results slightly.
      score = estimator.score(*test_data)
    /usr/share/miniconda3/envs/test/lib/python3.9/site-packages/sklearn/base.py:88: FutureWarning: The mindist parameter of verde.Spline is no longer required and will be removed in Verde 2.0.0. Use the default value to obtain the future behavior.
      new_object = klass(**new_object_params)
    /usr/share/miniconda3/envs/test/lib/python3.9/site-packages/verde/model_selection.py:784: FutureWarning: The default scoring will change from R² to negative root mean squared error (RMSE) in Verde 2.0.0. This may change model selection results slightly.
      score = estimator.score(*test_data)
    /usr/share/miniconda3/envs/test/lib/python3.9/site-packages/sklearn/base.py:88: FutureWarning: The mindist parameter of verde.Spline is no longer required and will be removed in Verde 2.0.0. Use the default value to obtain the future behavior.
      new_object = klass(**new_object_params)
    /usr/share/miniconda3/envs/test/lib/python3.9/site-packages/verde/model_selection.py:784: FutureWarning: The default scoring will change from R² to negative root mean squared error (RMSE) in Verde 2.0.0. This may change model selection results slightly.
      score = estimator.score(*test_data)
    /usr/share/miniconda3/envs/test/lib/python3.9/site-packages/sklearn/base.py:88: FutureWarning: The mindist parameter of verde.Spline is no longer required and will be removed in Verde 2.0.0. Use the default value to obtain the future behavior.
      new_object = klass(**new_object_params)
    /usr/share/miniconda3/envs/test/lib/python3.9/site-packages/verde/model_selection.py:784: FutureWarning: The default scoring will change from R² to negative root mean squared error (RMSE) in Verde 2.0.0. This may change model selection results slightly.
      score = estimator.score(*test_data)
    /usr/share/miniconda3/envs/test/lib/python3.9/site-packages/sklearn/base.py:88: FutureWarning: The mindist parameter of verde.Spline is no longer required and will be removed in Verde 2.0.0. Use the default value to obtain the future behavior.
      new_object = klass(**new_object_params)
    /usr/share/miniconda3/envs/test/lib/python3.9/site-packages/verde/model_selection.py:784: FutureWarning: The default scoring will change from R² to negative root mean squared error (RMSE) in Verde 2.0.0. This may change model selection results slightly.
      score = estimator.score(*test_data)
    /usr/share/miniconda3/envs/test/lib/python3.9/site-packages/sklearn/base.py:88: FutureWarning: The mindist parameter of verde.Spline is no longer required and will be removed in Verde 2.0.0. Use the default value to obtain the future behavior.
      new_object = klass(**new_object_params)
    /usr/share/miniconda3/envs/test/lib/python3.9/site-packages/verde/model_selection.py:784: FutureWarning: The default scoring will change from R² to negative root mean squared error (RMSE) in Verde 2.0.0. This may change model selection results slightly.
      score = estimator.score(*test_data)
    /usr/share/miniconda3/envs/test/lib/python3.9/site-packages/sklearn/base.py:88: FutureWarning: The mindist parameter of verde.Spline is no longer required and will be removed in Verde 2.0.0. Use the default value to obtain the future behavior.
      new_object = klass(**new_object_params)
    /usr/share/miniconda3/envs/test/lib/python3.9/site-packages/verde/model_selection.py:784: FutureWarning: The default scoring will change from R² to negative root mean squared error (RMSE) in Verde 2.0.0. This may change model selection results slightly.
      score = estimator.score(*test_data)
    /usr/share/miniconda3/envs/test/lib/python3.9/site-packages/sklearn/base.py:88: FutureWarning: The mindist parameter of verde.Spline is no longer required and will be removed in Verde 2.0.0. Use the default value to obtain the future behavior.
      new_object = klass(**new_object_params)
    /usr/share/miniconda3/envs/test/lib/python3.9/site-packages/verde/model_selection.py:784: FutureWarning: The default scoring will change from R² to negative root mean squared error (RMSE) in Verde 2.0.0. This may change model selection results slightly.
      score = estimator.score(*test_data)
    /usr/share/miniconda3/envs/test/lib/python3.9/site-packages/sklearn/base.py:88: FutureWarning: The mindist parameter of verde.Spline is no longer required and will be removed in Verde 2.0.0. Use the default value to obtain the future behavior.
      new_object = klass(**new_object_params)
    /usr/share/miniconda3/envs/test/lib/python3.9/site-packages/verde/model_selection.py:784: FutureWarning: The default scoring will change from R² to negative root mean squared error (RMSE) in Verde 2.0.0. This may change model selection results slightly.
      score = estimator.score(*test_data)
    /usr/share/miniconda3/envs/test/lib/python3.9/site-packages/sklearn/base.py:88: FutureWarning: The mindist parameter of verde.Spline is no longer required and will be removed in Verde 2.0.0. Use the default value to obtain the future behavior.
      new_object = klass(**new_object_params)
    /usr/share/miniconda3/envs/test/lib/python3.9/site-packages/verde/model_selection.py:784: FutureWarning: The default scoring will change from R² to negative root mean squared error (RMSE) in Verde 2.0.0. This may change model selection results slightly.
      score = estimator.score(*test_data)
    /usr/share/miniconda3/envs/test/lib/python3.9/site-packages/sklearn/base.py:88: FutureWarning: The mindist parameter of verde.Spline is no longer required and will be removed in Verde 2.0.0. Use the default value to obtain the future behavior.
      new_object = klass(**new_object_params)
    /usr/share/miniconda3/envs/test/lib/python3.9/site-packages/verde/model_selection.py:784: FutureWarning: The default scoring will change from R² to negative root mean squared error (RMSE) in Verde 2.0.0. This may change model selection results slightly.
      score = estimator.score(*test_data)
    /usr/share/miniconda3/envs/test/lib/python3.9/site-packages/sklearn/base.py:88: FutureWarning: The mindist parameter of verde.Spline is no longer required and will be removed in Verde 2.0.0. Use the default value to obtain the future behavior.
      new_object = klass(**new_object_params)
    /usr/share/miniconda3/envs/test/lib/python3.9/site-packages/verde/model_selection.py:784: FutureWarning: The default scoring will change from R² to negative root mean squared error (RMSE) in Verde 2.0.0. This may change model selection results slightly.
      score = estimator.score(*test_data)
    /usr/share/miniconda3/envs/test/lib/python3.9/site-packages/sklearn/base.py:88: FutureWarning: The mindist parameter of verde.Spline is no longer required and will be removed in Verde 2.0.0. Use the default value to obtain the future behavior.
      new_object = klass(**new_object_params)
    /usr/share/miniconda3/envs/test/lib/python3.9/site-packages/verde/model_selection.py:784: FutureWarning: The default scoring will change from R² to negative root mean squared error (RMSE) in Verde 2.0.0. This may change model selection results slightly.
      score = estimator.score(*test_data)
    [0.7960368857061437, 0.8447749626357742, 0.8382287942980732, 0.8409658539038973]




.. GENERATED FROM PYTHON SOURCE LINES 80-81

The largest score will yield the best parameter combination.

.. GENERATED FROM PYTHON SOURCE LINES 81-87

.. code-block:: default


    best = np.argmax(scores)
    print("Best score:", scores[best])
    print("Score with defaults:", score_default)
    print("Best damping:", dampings[best])





.. rst-class:: sphx-glr-script-out

 Out:

 .. code-block:: none

    Best score: 0.8447749626357742
    Score with defaults: 0.7960368857061437
    Best damping: 0.0001




.. GENERATED FROM PYTHON SOURCE LINES 88-94

**That is a nice improvement over our previous score!**

This type of tuning is important and should always be performed when using a
new gridder or a new dataset. However, the above implementation requires a
lot of coding. Fortunately, Verde provides convenience classes that perform
the cross-validation and tuning automatically when fitting a dataset.

.. GENERATED FROM PYTHON SOURCE LINES 97-105

Cross-validated gridders
------------------------

The :class:`verde.SplineCV` class provides a cross-validated version of
:class:`verde.Spline`. It has almost the same interface but does all of the
above automatically when fitting a dataset. The only difference is that you
must provide a list of ``damping`` values to try instead of only a single
value:

.. GENERATED FROM PYTHON SOURCE LINES 105-108

.. code-block:: default


    spline = vd.SplineCV(dampings=dampings)








.. GENERATED FROM PYTHON SOURCE LINES 109-111

Calling :meth:`~verde.SplineCV.fit` will run a grid search over all parameter
values to find the one that maximizes the cross-validation score.

.. GENERATED FROM PYTHON SOURCE LINES 111-114

.. code-block:: default


    spline.fit(proj_coords, data.air_temperature_c)





.. rst-class:: sphx-glr-script-out

 Out:

 .. code-block:: none

    /usr/share/miniconda3/envs/test/lib/python3.9/site-packages/verde/spline.py:245: FutureWarning: The mindist parameter of verde.Spline is no longer required and will be removed in Verde 2.0.0. Use the default value to obtain the future behavior.
      spline = Spline(**params)
    /usr/share/miniconda3/envs/test/lib/python3.9/site-packages/sklearn/base.py:88: FutureWarning: The mindist parameter of verde.Spline is no longer required and will be removed in Verde 2.0.0. Use the default value to obtain the future behavior.
      new_object = klass(**new_object_params)
    /usr/share/miniconda3/envs/test/lib/python3.9/site-packages/verde/model_selection.py:784: FutureWarning: The default scoring will change from R² to negative root mean squared error (RMSE) in Verde 2.0.0. This may change model selection results slightly.
      score = estimator.score(*test_data)
    /usr/share/miniconda3/envs/test/lib/python3.9/site-packages/sklearn/base.py:88: FutureWarning: The mindist parameter of verde.Spline is no longer required and will be removed in Verde 2.0.0. Use the default value to obtain the future behavior.
      new_object = klass(**new_object_params)
    /usr/share/miniconda3/envs/test/lib/python3.9/site-packages/verde/model_selection.py:784: FutureWarning: The default scoring will change from R² to negative root mean squared error (RMSE) in Verde 2.0.0. This may change model selection results slightly.
      score = estimator.score(*test_data)
    /usr/share/miniconda3/envs/test/lib/python3.9/site-packages/sklearn/base.py:88: FutureWarning: The mindist parameter of verde.Spline is no longer required and will be removed in Verde 2.0.0. Use the default value to obtain the future behavior.
      new_object = klass(**new_object_params)
    /usr/share/miniconda3/envs/test/lib/python3.9/site-packages/verde/model_selection.py:784: FutureWarning: The default scoring will change from R² to negative root mean squared error (RMSE) in Verde 2.0.0. This may change model selection results slightly.
      score = estimator.score(*test_data)
    /usr/share/miniconda3/envs/test/lib/python3.9/site-packages/sklearn/base.py:88: FutureWarning: The mindist parameter of verde.Spline is no longer required and will be removed in Verde 2.0.0. Use the default value to obtain the future behavior.
      new_object = klass(**new_object_params)
    /usr/share/miniconda3/envs/test/lib/python3.9/site-packages/verde/model_selection.py:784: FutureWarning: The default scoring will change from R² to negative root mean squared error (RMSE) in Verde 2.0.0. This may change model selection results slightly.
      score = estimator.score(*test_data)
    /usr/share/miniconda3/envs/test/lib/python3.9/site-packages/sklearn/base.py:88: FutureWarning: The mindist parameter of verde.Spline is no longer required and will be removed in Verde 2.0.0. Use the default value to obtain the future behavior.
      new_object = klass(**new_object_params)
    /usr/share/miniconda3/envs/test/lib/python3.9/site-packages/verde/model_selection.py:784: FutureWarning: The default scoring will change from R² to negative root mean squared error (RMSE) in Verde 2.0.0. This may change model selection results slightly.
      score = estimator.score(*test_data)
    /usr/share/miniconda3/envs/test/lib/python3.9/site-packages/verde/spline.py:245: FutureWarning: The mindist parameter of verde.Spline is no longer required and will be removed in Verde 2.0.0. Use the default value to obtain the future behavior.
      spline = Spline(**params)
    /usr/share/miniconda3/envs/test/lib/python3.9/site-packages/sklearn/base.py:88: FutureWarning: The mindist parameter of verde.Spline is no longer required and will be removed in Verde 2.0.0. Use the default value to obtain the future behavior.
      new_object = klass(**new_object_params)
    /usr/share/miniconda3/envs/test/lib/python3.9/site-packages/verde/model_selection.py:784: FutureWarning: The default scoring will change from R² to negative root mean squared error (RMSE) in Verde 2.0.0. This may change model selection results slightly.
      score = estimator.score(*test_data)
    /usr/share/miniconda3/envs/test/lib/python3.9/site-packages/sklearn/base.py:88: FutureWarning: The mindist parameter of verde.Spline is no longer required and will be removed in Verde 2.0.0. Use the default value to obtain the future behavior.
      new_object = klass(**new_object_params)
    /usr/share/miniconda3/envs/test/lib/python3.9/site-packages/verde/model_selection.py:784: FutureWarning: The default scoring will change from R² to negative root mean squared error (RMSE) in Verde 2.0.0. This may change model selection results slightly.
      score = estimator.score(*test_data)
    /usr/share/miniconda3/envs/test/lib/python3.9/site-packages/sklearn/base.py:88: FutureWarning: The mindist parameter of verde.Spline is no longer required and will be removed in Verde 2.0.0. Use the default value to obtain the future behavior.
      new_object = klass(**new_object_params)
    /usr/share/miniconda3/envs/test/lib/python3.9/site-packages/verde/model_selection.py:784: FutureWarning: The default scoring will change from R² to negative root mean squared error (RMSE) in Verde 2.0.0. This may change model selection results slightly.
      score = estimator.score(*test_data)
    /usr/share/miniconda3/envs/test/lib/python3.9/site-packages/sklearn/base.py:88: FutureWarning: The mindist parameter of verde.Spline is no longer required and will be removed in Verde 2.0.0. Use the default value to obtain the future behavior.
      new_object = klass(**new_object_params)
    /usr/share/miniconda3/envs/test/lib/python3.9/site-packages/verde/model_selection.py:784: FutureWarning: The default scoring will change from R² to negative root mean squared error (RMSE) in Verde 2.0.0. This may change model selection results slightly.
      score = estimator.score(*test_data)
    /usr/share/miniconda3/envs/test/lib/python3.9/site-packages/sklearn/base.py:88: FutureWarning: The mindist parameter of verde.Spline is no longer required and will be removed in Verde 2.0.0. Use the default value to obtain the future behavior.
      new_object = klass(**new_object_params)
    /usr/share/miniconda3/envs/test/lib/python3.9/site-packages/verde/model_selection.py:784: FutureWarning: The default scoring will change from R² to negative root mean squared error (RMSE) in Verde 2.0.0. This may change model selection results slightly.
      score = estimator.score(*test_data)
    /usr/share/miniconda3/envs/test/lib/python3.9/site-packages/verde/spline.py:245: FutureWarning: The mindist parameter of verde.Spline is no longer required and will be removed in Verde 2.0.0. Use the default value to obtain the future behavior.
      spline = Spline(**params)
    /usr/share/miniconda3/envs/test/lib/python3.9/site-packages/sklearn/base.py:88: FutureWarning: The mindist parameter of verde.Spline is no longer required and will be removed in Verde 2.0.0. Use the default value to obtain the future behavior.
      new_object = klass(**new_object_params)
    /usr/share/miniconda3/envs/test/lib/python3.9/site-packages/verde/model_selection.py:784: FutureWarning: The default scoring will change from R² to negative root mean squared error (RMSE) in Verde 2.0.0. This may change model selection results slightly.
      score = estimator.score(*test_data)
    /usr/share/miniconda3/envs/test/lib/python3.9/site-packages/sklearn/base.py:88: FutureWarning: The mindist parameter of verde.Spline is no longer required and will be removed in Verde 2.0.0. Use the default value to obtain the future behavior.
      new_object = klass(**new_object_params)
    /usr/share/miniconda3/envs/test/lib/python3.9/site-packages/verde/model_selection.py:784: FutureWarning: The default scoring will change from R² to negative root mean squared error (RMSE) in Verde 2.0.0. This may change model selection results slightly.
      score = estimator.score(*test_data)
    /usr/share/miniconda3/envs/test/lib/python3.9/site-packages/sklearn/base.py:88: FutureWarning: The mindist parameter of verde.Spline is no longer required and will be removed in Verde 2.0.0. Use the default value to obtain the future behavior.
      new_object = klass(**new_object_params)
    /usr/share/miniconda3/envs/test/lib/python3.9/site-packages/verde/model_selection.py:784: FutureWarning: The default scoring will change from R² to negative root mean squared error (RMSE) in Verde 2.0.0. This may change model selection results slightly.
      score = estimator.score(*test_data)
    /usr/share/miniconda3/envs/test/lib/python3.9/site-packages/sklearn/base.py:88: FutureWarning: The mindist parameter of verde.Spline is no longer required and will be removed in Verde 2.0.0. Use the default value to obtain the future behavior.
      new_object = klass(**new_object_params)
    /usr/share/miniconda3/envs/test/lib/python3.9/site-packages/verde/model_selection.py:784: FutureWarning: The default scoring will change from R² to negative root mean squared error (RMSE) in Verde 2.0.0. This may change model selection results slightly.
      score = estimator.score(*test_data)
    /usr/share/miniconda3/envs/test/lib/python3.9/site-packages/sklearn/base.py:88: FutureWarning: The mindist parameter of verde.Spline is no longer required and will be removed in Verde 2.0.0. Use the default value to obtain the future behavior.
      new_object = klass(**new_object_params)
    /usr/share/miniconda3/envs/test/lib/python3.9/site-packages/verde/model_selection.py:784: FutureWarning: The default scoring will change from R² to negative root mean squared error (RMSE) in Verde 2.0.0. This may change model selection results slightly.
      score = estimator.score(*test_data)
    /usr/share/miniconda3/envs/test/lib/python3.9/site-packages/verde/spline.py:245: FutureWarning: The mindist parameter of verde.Spline is no longer required and will be removed in Verde 2.0.0. Use the default value to obtain the future behavior.
      spline = Spline(**params)
    /usr/share/miniconda3/envs/test/lib/python3.9/site-packages/sklearn/base.py:88: FutureWarning: The mindist parameter of verde.Spline is no longer required and will be removed in Verde 2.0.0. Use the default value to obtain the future behavior.
      new_object = klass(**new_object_params)
    /usr/share/miniconda3/envs/test/lib/python3.9/site-packages/verde/model_selection.py:784: FutureWarning: The default scoring will change from R² to negative root mean squared error (RMSE) in Verde 2.0.0. This may change model selection results slightly.
      score = estimator.score(*test_data)
    /usr/share/miniconda3/envs/test/lib/python3.9/site-packages/sklearn/base.py:88: FutureWarning: The mindist parameter of verde.Spline is no longer required and will be removed in Verde 2.0.0. Use the default value to obtain the future behavior.
      new_object = klass(**new_object_params)
    /usr/share/miniconda3/envs/test/lib/python3.9/site-packages/verde/model_selection.py:784: FutureWarning: The default scoring will change from R² to negative root mean squared error (RMSE) in Verde 2.0.0. This may change model selection results slightly.
      score = estimator.score(*test_data)
    /usr/share/miniconda3/envs/test/lib/python3.9/site-packages/sklearn/base.py:88: FutureWarning: The mindist parameter of verde.Spline is no longer required and will be removed in Verde 2.0.0. Use the default value to obtain the future behavior.
      new_object = klass(**new_object_params)
    /usr/share/miniconda3/envs/test/lib/python3.9/site-packages/verde/model_selection.py:784: FutureWarning: The default scoring will change from R² to negative root mean squared error (RMSE) in Verde 2.0.0. This may change model selection results slightly.
      score = estimator.score(*test_data)
    /usr/share/miniconda3/envs/test/lib/python3.9/site-packages/sklearn/base.py:88: FutureWarning: The mindist parameter of verde.Spline is no longer required and will be removed in Verde 2.0.0. Use the default value to obtain the future behavior.
      new_object = klass(**new_object_params)
    /usr/share/miniconda3/envs/test/lib/python3.9/site-packages/verde/model_selection.py:784: FutureWarning: The default scoring will change from R² to negative root mean squared error (RMSE) in Verde 2.0.0. This may change model selection results slightly.
      score = estimator.score(*test_data)
    /usr/share/miniconda3/envs/test/lib/python3.9/site-packages/sklearn/base.py:88: FutureWarning: The mindist parameter of verde.Spline is no longer required and will be removed in Verde 2.0.0. Use the default value to obtain the future behavior.
      new_object = klass(**new_object_params)
    /usr/share/miniconda3/envs/test/lib/python3.9/site-packages/verde/model_selection.py:784: FutureWarning: The default scoring will change from R² to negative root mean squared error (RMSE) in Verde 2.0.0. This may change model selection results slightly.
      score = estimator.score(*test_data)
    /usr/share/miniconda3/envs/test/lib/python3.9/site-packages/verde/spline.py:261: FutureWarning: The mindist parameter of verde.Spline is no longer required and will be removed in Verde 2.0.0. Use the default value to obtain the future behavior.
      self.spline_ = Spline(**parameter_sets[best])


.. raw:: html

    <div class="output_subarea output_html rendered_html output_result">
    <style>#sk-container-id-3 {color: black;background-color: white;}#sk-container-id-3 pre{padding: 0;}#sk-container-id-3 div.sk-toggleable {background-color: white;}#sk-container-id-3 label.sk-toggleable__label {cursor: pointer;display: block;width: 100%;margin-bottom: 0;padding: 0.3em;box-sizing: border-box;text-align: center;}#sk-container-id-3 label.sk-toggleable__label-arrow:before {content: "▸";float: left;margin-right: 0.25em;color: #696969;}#sk-container-id-3 label.sk-toggleable__label-arrow:hover:before {color: black;}#sk-container-id-3 div.sk-estimator:hover label.sk-toggleable__label-arrow:before {color: black;}#sk-container-id-3 div.sk-toggleable__content {max-height: 0;max-width: 0;overflow: hidden;text-align: left;background-color: #f0f8ff;}#sk-container-id-3 div.sk-toggleable__content pre {margin: 0.2em;color: black;border-radius: 0.25em;background-color: #f0f8ff;}#sk-container-id-3 input.sk-toggleable__control:checked~div.sk-toggleable__content {max-height: 200px;max-width: 100%;overflow: auto;}#sk-container-id-3 input.sk-toggleable__control:checked~label.sk-toggleable__label-arrow:before {content: "▾";}#sk-container-id-3 div.sk-estimator input.sk-toggleable__control:checked~label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-3 div.sk-label input.sk-toggleable__control:checked~label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-3 input.sk-hidden--visually {border: 0;clip: rect(1px 1px 1px 1px);clip: rect(1px, 1px, 1px, 1px);height: 1px;margin: -1px;overflow: hidden;padding: 0;position: absolute;width: 1px;}#sk-container-id-3 div.sk-estimator {font-family: monospace;background-color: #f0f8ff;border: 1px dotted black;border-radius: 0.25em;box-sizing: border-box;margin-bottom: 0.5em;}#sk-container-id-3 div.sk-estimator:hover {background-color: #d4ebff;}#sk-container-id-3 div.sk-parallel-item::after {content: "";width: 100%;border-bottom: 1px solid gray;flex-grow: 1;}#sk-container-id-3 div.sk-label:hover label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-3 div.sk-serial::before {content: "";position: absolute;border-left: 1px solid gray;box-sizing: border-box;top: 0;bottom: 0;left: 50%;z-index: 0;}#sk-container-id-3 div.sk-serial {display: flex;flex-direction: column;align-items: center;background-color: white;padding-right: 0.2em;padding-left: 0.2em;position: relative;}#sk-container-id-3 div.sk-item {position: relative;z-index: 1;}#sk-container-id-3 div.sk-parallel {display: flex;align-items: stretch;justify-content: center;background-color: white;position: relative;}#sk-container-id-3 div.sk-item::before, #sk-container-id-3 div.sk-parallel-item::before {content: "";position: absolute;border-left: 1px solid gray;box-sizing: border-box;top: 0;bottom: 0;left: 50%;z-index: -1;}#sk-container-id-3 div.sk-parallel-item {display: flex;flex-direction: column;z-index: 1;position: relative;background-color: white;}#sk-container-id-3 div.sk-parallel-item:first-child::after {align-self: flex-end;width: 50%;}#sk-container-id-3 div.sk-parallel-item:last-child::after {align-self: flex-start;width: 50%;}#sk-container-id-3 div.sk-parallel-item:only-child::after {width: 0;}#sk-container-id-3 div.sk-dashed-wrapped {border: 1px dashed gray;margin: 0 0.4em 0.5em 0.4em;box-sizing: border-box;padding-bottom: 0.4em;background-color: white;}#sk-container-id-3 div.sk-label label {font-family: monospace;font-weight: bold;display: inline-block;line-height: 1.2em;}#sk-container-id-3 div.sk-label-container {text-align: center;}#sk-container-id-3 div.sk-container {/* jupyter's `normalize.less` sets `[hidden] { display: none; }` but bootstrap.min.css set `[hidden] { display: none !important; }` so we also need the `!important` here to be able to override the default hidden behavior on the sphinx rendered scikit-learn.org. See: https://github.com/scikit-learn/scikit-learn/issues/21755 */display: inline-block !important;position: relative;}#sk-container-id-3 div.sk-text-repr-fallback {display: none;}</style><div id="sk-container-id-3" class="sk-top-container"><div class="sk-text-repr-fallback"><pre>SplineCV(dampings=[None, 0.0001, 0.001, 0.01], mindists=[0])</pre><b>In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook. <br />On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.</b></div><div class="sk-container" hidden><div class="sk-item"><div class="sk-estimator sk-toggleable"><input class="sk-toggleable__control sk-hidden--visually" id="sk-estimator-id-3" type="checkbox" checked><label for="sk-estimator-id-3" class="sk-toggleable__label sk-toggleable__label-arrow">SplineCV</label><div class="sk-toggleable__content"><pre>SplineCV(dampings=[None, 0.0001, 0.001, 0.01], mindists=[0])</pre></div></div></div></div></div>
    </div>
    <br />
    <br />

.. GENERATED FROM PYTHON SOURCE LINES 115-117

The estimated best ``damping``, as well as the cross-validation scores, are
stored in class attributes:

.. GENERATED FROM PYTHON SOURCE LINES 117-121

.. code-block:: default


    print("Highest score:", spline.scores_.max())
    print("Best damping:", spline.damping_)





.. rst-class:: sphx-glr-script-out

 Out:

 .. code-block:: none

    Highest score: 0.8447749626357742
    Best damping: 0.0001




.. GENERATED FROM PYTHON SOURCE LINES 122-124

The cross-validated gridder can be used like any other gridder (including in
:class:`verde.Chain` and :class:`verde.Vector`):

.. GENERATED FROM PYTHON SOURCE LINES 124-134

.. code-block:: default


    grid = spline.grid(
        region=region,
        spacing=spacing,
        projection=projection,
        dims=["latitude", "longitude"],
        data_names="temperature",
    )
    print(grid)





.. rst-class:: sphx-glr-script-out

 Out:

 .. code-block:: none

    <xarray.Dataset>
    Dimensions:      (latitude: 43, longitude: 51)
    Coordinates:
      * longitude    (longitude) float64 -106.4 -106.1 -105.9 ... -94.06 -93.8
      * latitude     (latitude) float64 25.91 26.16 26.41 ... 35.91 36.16 36.41
    Data variables:
        temperature  (latitude, longitude) float64 24.7 24.56 24.42 ... 7.542 7.639
    Attributes:
        metadata:  Generated by SplineCV(dampings=[None, 0.0001, 0.001, 0.01], mi...




.. GENERATED FROM PYTHON SOURCE LINES 135-138

Like :func:`verde.cross_val_score`, :class:`~verde.SplineCV` can also run the
grid search in parallel using `Dask <https://dask.org/>`__ by specifying the
``delayed`` attribute:

.. GENERATED FROM PYTHON SOURCE LINES 138-141

.. code-block:: default


    spline = vd.SplineCV(dampings=dampings, delayed=True)








.. GENERATED FROM PYTHON SOURCE LINES 142-145

Unlike :func:`verde.cross_val_score`, calling :meth:`~verde.SplineCV.fit`
does **not** result in :func:`dask.delayed` objects. The full grid search is
executed and the optimal parameters are found immediately.

.. GENERATED FROM PYTHON SOURCE LINES 145-150

.. code-block:: default


    spline.fit(proj_coords, data.air_temperature_c)

    print("Best damping:", spline.damping_)





.. rst-class:: sphx-glr-script-out

 Out:

 .. code-block:: none

    /usr/share/miniconda3/envs/test/lib/python3.9/site-packages/verde/spline.py:245: FutureWarning: The mindist parameter of verde.Spline is no longer required and will be removed in Verde 2.0.0. Use the default value to obtain the future behavior.
      spline = Spline(**params)
    /usr/share/miniconda3/envs/test/lib/python3.9/site-packages/sklearn/base.py:88: FutureWarning: The mindist parameter of verde.Spline is no longer required and will be removed in Verde 2.0.0. Use the default value to obtain the future behavior.
      new_object = klass(**new_object_params)
    /usr/share/miniconda3/envs/test/lib/python3.9/site-packages/verde/model_selection.py:784: FutureWarning: The default scoring will change from R² to negative root mean squared error (RMSE) in Verde 2.0.0. This may change model selection results slightly.
      score = estimator.score(*test_data)
    /usr/share/miniconda3/envs/test/lib/python3.9/site-packages/verde/model_selection.py:784: FutureWarning: The default scoring will change from R² to negative root mean squared error (RMSE) in Verde 2.0.0. This may change model selection results slightly.
      score = estimator.score(*test_data)
    /usr/share/miniconda3/envs/test/lib/python3.9/site-packages/verde/model_selection.py:784: FutureWarning: The default scoring will change from R² to negative root mean squared error (RMSE) in Verde 2.0.0. This may change model selection results slightly.
      score = estimator.score(*test_data)
    /usr/share/miniconda3/envs/test/lib/python3.9/site-packages/verde/model_selection.py:784: FutureWarning: The default scoring will change from R² to negative root mean squared error (RMSE) in Verde 2.0.0. This may change model selection results slightly.
      score = estimator.score(*test_data)
    /usr/share/miniconda3/envs/test/lib/python3.9/site-packages/verde/model_selection.py:784: FutureWarning: The default scoring will change from R² to negative root mean squared error (RMSE) in Verde 2.0.0. This may change model selection results slightly.
      score = estimator.score(*test_data)
    /usr/share/miniconda3/envs/test/lib/python3.9/site-packages/verde/model_selection.py:784: FutureWarning: The default scoring will change from R² to negative root mean squared error (RMSE) in Verde 2.0.0. This may change model selection results slightly.
      score = estimator.score(*test_data)
    /usr/share/miniconda3/envs/test/lib/python3.9/site-packages/verde/model_selection.py:784: FutureWarning: The default scoring will change from R² to negative root mean squared error (RMSE) in Verde 2.0.0. This may change model selection results slightly.
      score = estimator.score(*test_data)
    /usr/share/miniconda3/envs/test/lib/python3.9/site-packages/verde/model_selection.py:784: FutureWarning: The default scoring will change from R² to negative root mean squared error (RMSE) in Verde 2.0.0. This may change model selection results slightly.
      score = estimator.score(*test_data)
    /usr/share/miniconda3/envs/test/lib/python3.9/site-packages/verde/model_selection.py:784: FutureWarning: The default scoring will change from R² to negative root mean squared error (RMSE) in Verde 2.0.0. This may change model selection results slightly.
      score = estimator.score(*test_data)
    /usr/share/miniconda3/envs/test/lib/python3.9/site-packages/verde/model_selection.py:784: FutureWarning: The default scoring will change from R² to negative root mean squared error (RMSE) in Verde 2.0.0. This may change model selection results slightly.
      score = estimator.score(*test_data)
    /usr/share/miniconda3/envs/test/lib/python3.9/site-packages/verde/model_selection.py:784: FutureWarning: The default scoring will change from R² to negative root mean squared error (RMSE) in Verde 2.0.0. This may change model selection results slightly.
      score = estimator.score(*test_data)
    /usr/share/miniconda3/envs/test/lib/python3.9/site-packages/verde/model_selection.py:784: FutureWarning: The default scoring will change from R² to negative root mean squared error (RMSE) in Verde 2.0.0. This may change model selection results slightly.
      score = estimator.score(*test_data)
    /usr/share/miniconda3/envs/test/lib/python3.9/site-packages/verde/model_selection.py:784: FutureWarning: The default scoring will change from R² to negative root mean squared error (RMSE) in Verde 2.0.0. This may change model selection results slightly.
      score = estimator.score(*test_data)
    /usr/share/miniconda3/envs/test/lib/python3.9/site-packages/verde/model_selection.py:784: FutureWarning: The default scoring will change from R² to negative root mean squared error (RMSE) in Verde 2.0.0. This may change model selection results slightly.
      score = estimator.score(*test_data)
    /usr/share/miniconda3/envs/test/lib/python3.9/site-packages/verde/spline.py:261: FutureWarning: The mindist parameter of verde.Spline is no longer required and will be removed in Verde 2.0.0. Use the default value to obtain the future behavior.
      self.spline_ = Spline(**parameter_sets[best])
    Best damping: 0.0001




.. GENERATED FROM PYTHON SOURCE LINES 151-154

The one caveat is the that the ``scores_`` attribute will be a list of
:func:`dask.delayed` objects instead because the scores are only computed as
intermediate values in the scheduled computations.

.. GENERATED FROM PYTHON SOURCE LINES 154-157

.. code-block:: default


    print("Delayed scores:", spline.scores_)





.. rst-class:: sphx-glr-script-out

 Out:

 .. code-block:: none

    Delayed scores: [Delayed('mean-9a9814d2-10c0-40b7-aa56-19f5252c483e'), Delayed('mean-71ec853e-3ba4-4d64-9bf3-10f8658b8c68'), Delayed('mean-8f70d964-b72e-4fbd-a3a2-8cefbb1d898c'), Delayed('mean-d6772dac-6041-48b2-bfb5-6f8490137a86')]




.. GENERATED FROM PYTHON SOURCE LINES 158-162

Calling :func:`dask.compute` on the scores will calculate their values but
will unfortunately run the entire grid search again. So using
``delayed=True`` is not recommended if you need the scores of each parameter
combination.

.. GENERATED FROM PYTHON SOURCE LINES 164-169

The importance of tuning
------------------------

To see the difference that tuning has on the results, we can make a grid
with the best configuration and see how it compares to the default result.

.. GENERATED FROM PYTHON SOURCE LINES 169-178

.. code-block:: default


    grid_default = spline_default.grid(
        region=region,
        spacing=spacing,
        projection=projection,
        dims=["latitude", "longitude"],
        data_names="temperature",
    )








.. GENERATED FROM PYTHON SOURCE LINES 179-180

Let's plot our grids side-by-side:

.. GENERATED FROM PYTHON SOURCE LINES 180-211

.. code-block:: default


    mask = vd.distance_mask(
        (data.longitude, data.latitude),
        maxdist=3 * spacing * 111e3,
        coordinates=vd.grid_coordinates(region, spacing=spacing),
        projection=projection,
    )

    grid = grid.where(mask)
    grid_default = grid_default.where(mask)

    plt.figure(figsize=(14, 8))
    for i, title, grd in zip(range(2), ["Defaults", "Tuned"], [grid_default, grid]):
        ax = plt.subplot(1, 2, i + 1, projection=ccrs.Mercator())
        ax.set_title(title)
        pc = grd.temperature.plot.pcolormesh(
            ax=ax,
            cmap="plasma",
            transform=ccrs.PlateCarree(),
            vmin=data.air_temperature_c.min(),
            vmax=data.air_temperature_c.max(),
            add_colorbar=False,
            add_labels=False,
        )
        plt.colorbar(pc, orientation="horizontal", aspect=50, pad=0.05).set_label("C")
        ax.plot(
            data.longitude, data.latitude, ".k", markersize=1, transform=ccrs.PlateCarree()
        )
        vd.datasets.setup_texas_wind_map(ax)
    plt.show()




.. image-sg:: /tutorials/images/sphx_glr_model_selection_001.png
   :alt: Defaults, Tuned
   :srcset: /tutorials/images/sphx_glr_model_selection_001.png
   :class: sphx-glr-single-img





.. GENERATED FROM PYTHON SOURCE LINES 212-215

Notice that, for sparse data like these, **smoother models tend to be better
predictors**. This is a sign that you should probably not trust many of the
short wavelength features that we get from the defaults.


.. rst-class:: sphx-glr-timing

   **Total running time of the script:** ( 0 minutes  0.676 seconds)


.. _sphx_glr_download_tutorials_model_selection.py:


.. only :: html

 .. container:: sphx-glr-footer
    :class: sphx-glr-footer-example



  .. container:: sphx-glr-download sphx-glr-download-python

     :download:`Download Python source code: model_selection.py <model_selection.py>`



  .. container:: sphx-glr-download sphx-glr-download-jupyter

     :download:`Download Jupyter notebook: model_selection.ipynb <model_selection.ipynb>`


.. only:: html

 .. rst-class:: sphx-glr-signature

    `Gallery generated by Sphinx-Gallery <https://sphinx-gallery.github.io>`_