diff --git a/00_Introduction/00_Introduction.ipynb b/00_Introduction/00_Introduction.ipynb index 3c30845..e37d020 100644 --- a/00_Introduction/00_Introduction.ipynb +++ b/00_Introduction/00_Introduction.ipynb @@ -789,7 +789,7 @@ " 2.54\n", " \n", " \n", - " Simplex Method\n", + " Simplex Method (pending validation)\n", " 1.53\n", " \n", " \n", @@ -824,7 +824,7 @@ "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", - "version": "3.8.6" + "version": "3.7.9" }, "latex_envs": { "LaTeX_envs_menu_present": true, diff --git a/_build/.doctrees/00_Introduction/00_Introduction.doctree b/_build/.doctrees/00_Introduction/00_Introduction.doctree index 2697c6f..adeef74 100644 Binary files a/_build/.doctrees/00_Introduction/00_Introduction.doctree and b/_build/.doctrees/00_Introduction/00_Introduction.doctree differ diff --git a/_build/.doctrees/environment.pickle b/_build/.doctrees/environment.pickle index 81bb5ce..a1f6723 100644 Binary files a/_build/.doctrees/environment.pickle and b/_build/.doctrees/environment.pickle differ diff --git a/_build/html/00_Introduction/00_Introduction.html b/_build/html/00_Introduction/00_Introduction.html index 2551838..631b968 100644 --- a/_build/html/00_Introduction/00_Introduction.html +++ b/_build/html/00_Introduction/00_Introduction.html @@ -318,9 +318,26 @@ +
  • + + 4. Benchmark Methods + + +
  • - 3. Evaluation Metrics for Forecast Accuracy + 5. Evaluation Metrics for Forecast Accuracy
    • +
  • + + 6. Summary of Forecast Accuracy for Jena Climate Dataset + +
    • @@ -381,6 +403,8 @@ import matplotlib.pyplot as plt import statsmodels.graphics.tsaplots as tg %matplotlib inline + +plt.rcParams['figure.figsize'] = [15, 2] @@ -709,8 +733,31 @@

    with the characteristic polynomials as defined above.

    +
    +

    4. Benchmark Methods

    +

    In order to properly measure the performance of a forecasting model, we first need to establish several baselines. This section introduces several methods that will serve as benchmarks. Obviously, any forecasting method we develop must beat these benchmarks. Otherwise, the new method is not even worth considering.

    +

    In the notation below, \(T\) refers to the length of the time series and \(h\) refers to the prediction horizon.

    +
    +

    Naïve Method

    +

    Forecasts of all future values are equal to the last observation.

    +
    +\[\begin{align*} + \hat{y}_{T+h} &= y_T +\end{align*}\]
    +
    +
    +

    Seasonal Naïve Method

    +

    Forecasts are equal to the last observed value from the same season of the year (e.g. the same month of the previous year).

    +
    +\[\begin{align*} + \hat{y}_{T+h} &= y_{T+h-m(k+1)} +\end{align*}\]
    +

    where \(m\) is the seasonal period and \(k\) is the integer part of \((h-1)/m\) (i.e. the number of complete years in the forecast period prior to time \(T+h\)).

    +

    As an example, if we were forecasting a monthly time series, the forecast for all future February values is simply equal to the last observed February value. With weekly data, the forecast of all future Friday values is equal to the last observed Friday value. And so on.

    +
    +
    -

    3. Evaluation Metrics for Forecast Accuracy

    +

    5. Evaluation Metrics for Forecast Accuracy

    Forecasting is one of the most common inference tasks in time series analysis. In order to properly gauge the performance of a time series model, it is common practice to divide the dataset into two parts: training and test data. Model parameters are estimated using training data, then the models are used to generate forecasts that are evaluated against the test data.

    Error statistics come in different flavors, each with their own advantages and disadvantages.

    @@ -746,6 +793,47 @@ \end{align*}\]
    +
    +

    6. Summary of Forecast Accuracy for Jena Climate Dataset

    +

    The handbook goes over several time series forecasting methods and compares performance of said models on the Jena Climate Dataset. Specifically, each method attempts to forecast the temperature variable (in Celsius). A summary of the forecast accuracy for each model is shown below.

    + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
    MethodAverage MAE (Celsius)
    Naive3.18
    Seasonal Naive2.61
    Linear Regression2.86
    ARIMA3.19
    VAR2.54
    Simplex Method (pending validation)1.53
    LightGBM2.08