Forecasting with foundation models: Lag-Llama, Chronos, TimesFM, and Moirai

A forecasting foundation model is a network trained once on a very large, very mixed collection of time series, then used on series it has never seen, often with no further training at all. You hand it the recent history of your series and it hands back a forecast distribution. That is a different contract from the one in the statistical guide or the neural guide, where every model is fitted to your data before it forecasts anything.

The promise is attractive: no model selection, no training loop, sensible uncertainty out of the box, and something useful for short or new series. The risks are less obvious. A pretrained model may have already seen your benchmark, it may be slower and costlier than the method it replaces, and on a clean seasonal series it may not beat a forty-millisecond exponential-smoothing fit. This guide explains how Lag-Llama, Chronos, TimesFM and Moirai work inside, then shows how to evaluate them fairly against seasonal naive and AutoETS on the same AirPassengers holdout used throughout this series. If forecast origins, rolling backtests and prediction intervals are new to you, read the mathematical intuition guide first.

What "foundation model" means for time series

In language modelling, a foundation model is pretrained on a broad corpus with a generic objective (predict the next token) and then reused for many downstream tasks. The time series version follows the same recipe. Collect thousands to millions of series from many domains (energy, traffic, retail, weather, web traffic, finance, often plus synthetic series), train one network to predict the future of a series from its past, and hope that the patterns it learns (trend, seasonality, level shifts, noise shapes) transfer to new series.

Three ways of using such a model come up repeatedly.

• Zero-shot. Load the pretrained weights and forecast your series directly. Nothing is trained. The only thing the model learns about your series is what it reads from the context window at inference time.
• Few-shot or in-context. Still no gradient updates, but you give the model more relevant context, such as a longer history or related series, and let it condition on that.
• Fine-tuning. Continue training the pretrained weights on your own training split for a small number of steps, usually with a low learning rate. This is ordinary supervised training that starts from a good initialisation instead of random weights.

Write the context as $x_{1:C}$, a window of $C$ past values, and the forecast horizon as $H$. Every model in this guide approximates the same conditional distribution:

where $\theta$ are network weights learned during pretraining. The key difference from a fitted ARIMA or ETS model is where $\theta$ comes from. For ETS, the smoothing parameters are estimated on your series. For a foundation model, $\theta$ was estimated on everyone else's series, and your data enters only through the conditioning on $x_{1:C}$.

That one fact explains both the strength and the weakness. A pretrained model can produce reasonable forecasts from 30 observations where AutoARIMA would struggle to identify anything. But it cannot know business facts that are absent from the context (a promotion calendar, a store closure, a change in how a metric is counted) unless the specific model accepts covariates and you supply them.

The honest evaluation questions

Before looking at architectures, fix the questions you will ask of any pretrained forecaster. They matter more than the architecture.

Has the model already seen the test data? Pretraining corpora are assembled from public archives. Many of those archives contain exactly the benchmark series people then use to evaluate the model: M4, M3, tourism, electricity, traffic, and small textbook series such as AirPassengers are widely mirrored. If a series, or a near copy, was in the pretraining corpus, a zero-shot score on it is partly a memory test. The Chronos authors handle this explicitly by reporting an in-domain benchmark (datasets that were part of training) separately from a zero-shot benchmark (datasets held out from training). Read the data card for any model you evaluate and treat overlap as a leakage risk, not a detail. For AirPassengers in particular, assume a large public corpus may contain it, which is one reason the numbers in this guide are a teaching exercise rather than evidence.

Does it beat cheap baselines on my data, with my protocol? A model card's benchmark average says little about your series. Compare against seasonal naive and AutoETS on a rolling-origin backtest built from your own data, using the same horizon, origins and metrics for every method. If the foundation model cannot beat AutoETS there, the leaderboard does not matter.

Are the probabilistic forecasts calibrated? Most of these models output distributions. Check interval coverage and a quantile loss, not only point error.

What does it cost? A pretrained model may need a GPU to hit your latency budget, may be hundreds of megabytes on disk, and may have licence terms that matter for commercial use.

flowchart TD
    A["Your own series<br/>not a public benchmark"] --> B["Define horizon H, season m,<br/>origins, metrics"]
    B --> C["Check model data card<br/>for corpus overlap"]
    C --> D["Rolling origins:<br/>cut history at each cutoff"]
    D --> E["Seasonal naive"]
    D --> F["AutoETS or AutoARIMA"]
    D --> G["Foundation model zero-shot<br/>same context, same H"]
    D --> H2["Foundation model fine-tuned<br/>on pre-cutoff data only"]
    E --> I["Score each origin:<br/>MAE, RMSE, MASE,<br/>coverage, pinball"]
    F --> I
    G --> I
    H2 --> I
    I --> J{"Beats AutoETS across<br/>most origins and segments?"}
    J -->|"yes"| K["Weigh cost, latency, licence"]
    J -->|"no"| L["Keep the fitted baseline"]

The fine-tuned arm in that diagram has a subtle rule: when you fine-tune for a given origin, only data before that origin may be used. Fine-tuning once on the full history and then backtesting is the same leakage as fitting a scaler on the whole dataset.

Lag-Llama: lag features as tokens

Lag-Llama is a decoder-only transformer in the style of LLaMA, trained for univariate probabilistic forecasting. Its distinctive idea is how it builds one input token per time step.

Lags as a token

A language model reads one token per word. Lag-Llama reads one token per time step, but that token is a vector, not a single value. For a set of lag indices $\mathcal{L} = \lbrace l_1, \dots, l_K \rbrace$, the lag part of the token at time $t$ is

Take a toy monthly example with $\mathcal{L} = \lbrace 1, 2, 3, 12, 24 \rbrace$. The token for January 1960 in AirPassengers contains the values for December, November and October 1959, January 1959, and January 1958. One token therefore already contains "last month" and "same month last year" side by side. The attention layers do not have to discover the yearly pattern by searching back twelve positions; it is placed directly in the input.

That is why the lag set contains seasonal lags. The released model builds its lag set as the union of typical lags for several frequencies (quarterly, monthly, weekly, daily, hourly, minutely, secondly), so one model can serve all of them. In practice this means the token for a monthly series contains short lags plus multiples of 12, and also some lags that only make sense at other frequencies. The exact list lives in the repository code; read it there rather than trusting a summary. A consequence worth knowing: the largest lags reach much further back than the context window, so the model needs history of length roughly context length plus maximum lag. For a short series such as AirPassengers, the far lags fall before the start of the data and are padded, which is one reason short monthly series are not where this design shines.

Date features and scaling statistics

Each token also gets calendar features $\mathbf{f}_t$ derived from the timestamp, such as month of year or day of week, scaled to a small range. Again the model covers many frequencies at once, so it uses features from second-of-minute up to quarter-of-year.

Values are normalised with robust scaling computed on the context window:

where IQR is the interquartile range, the 75th percentile minus the 25th. A tiny example: for the context $100, 110, 120, 130, 200$ the median is $120$ and, with a simple quartile convention, the IQR is $130 - 110 = 20$. The spike $200$ becomes $(200 - 120)/20 = 4$, while a mean-and-standard-deviation scaling would let that one spike inflate the scale for every other point. Because scaling removes the level, the model also receives the scaling statistics themselves (the location and scale, in transformed form) as extra inputs, so it can still tell a large series from a small one.

The full token is the concatenation $\mathbf{k}_t = \left[\mathbf{v}_t, \mathbf{f}_t, \text{scale stats}\right]$, which a linear layer projects to the model width. From there it is a standard causal transformer stack with LLaMA-style components: pre-normalisation with RMSNorm, rotary position embeddings, and a causal mask so position $t$ only attends to earlier positions.

flowchart LR
    A["Raw series x"] --> B["Robust scaling<br/>median and IQR"]
    B --> C["Lag vector at t<br/>x at t-1, t-2, t-12, t-24 ..."]
    A --> D["Date features<br/>month, week, day ..."]
    B --> E["Scale statistics"]
    C --> F["Concatenate<br/>one token per step"]
    D --> F
    E --> F
    F --> G["Linear projection"]
    G --> H["Causal decoder<br/>RMSNorm, RoPE, attention"]
    H --> I["Student-t head<br/>df, mean, scale"]
    I --> J["Sample next value"]
    J -.->|"append and rebuild lags"| C

Lag-Llama: Student-t head, sampling and context

The output at each position is not a point value. A small head maps the hidden state to the three parameters of a Student-t distribution: degrees of freedom $\nu$, location $\mu$ and scale $\sigma$. The density is

Read it piece by piece. $\mu$ is the centre. $\sigma$ stretches the curve. $\nu$ controls the tails: small $\nu$ gives heavy tails, so occasional large surprises are not treated as impossible, and as $\nu$ grows the curve approaches a normal distribution. Positivity of $\sigma$ and $\nu$ is enforced with a softplus, and in the GluonTS Student-t output the degrees of freedom are kept above 2 so the variance exists. Training minimises the negative log-likelihood of the observed next values:

The pretraining corpus is a collection of public datasets from several domains, and the paper uses augmentations (Freq-Mix and Freq-Mask) to reduce overfitting.

At inference time the model forecasts autoregressively, exactly like DeepAR. Draw one value from the predicted Student-t for $t = C+1$, append it to the series, rebuild the lag token for $C+2$, draw again, and so on to $C+H$. That gives one sample path. Repeat for $S$ paths (100 is common) and read quantiles at each step from the $S$ samples. Because each path feeds its own samples back in, uncertainty compounds over the horizon in a natural way. The cost is $S \times H$ forward passes, batched across samples.

Context length is the number of tokens the transformer attends over. The released checkpoint was pretrained with a short context (32 in the paper). Lags already reach beyond that window, so a short context sees more history than it seems. The repository README discusses trying longer contexts, using RoPE scaling when you exceed the pretrained length, and suggests that context length is one of the first hyperparameters to tune; follow its current guidance.

Zero-shot versus fine-tuning. The repository wraps the model in a GluonTS estimator, LagLlamaEstimator. For zero-shot you load the checkpoint, build a predictor, and call it. For fine-tuning you create the same estimator with the checkpoint path, a learning rate and Lightning trainer settings, and call estimator.train(...) on your training data, which returns a predictor. The paper reports that fine-tuning on a target dataset improved over zero-shot in its experiments; on your data, it is a hypothesis to test in the backtest.

Chronos: forecasting as language modelling

Chronos, from Amazon, takes a more radical position: do not design a time series architecture at all. Turn real values into tokens from a fixed vocabulary and train an off-the-shelf language model (T5) on them with ordinary cross-entropy.

Mean scaling

Given a context $x_{1:C}$, Chronos divides by the mean absolute value of the context:

If $s$ is zero (an all-zero context), the implementation sets it to 1. For AirPassengers, the mean absolute value of the 132 training months is about 262.5, so the last training value, 405 passengers in December 1959, maps to roughly $405 / 262.5 \approx 1.54$. Unlike Lag-Llama, there is no centring: zero stays at zero, which helps intermittent series with many zeros.

Uniform quantization

Next, choose $B$ bin centres $c_1 < c_2 < \dots < c_B$ evenly spaced on an interval $[l, r]$, with edges halfway between neighbouring centres:

The quantizer and dequantizer are

The released models use $[l, r] = [-15, 15]$ and a vocabulary of 4096 tokens, a few of which are reserved for special tokens such as padding and end-of-sequence. So a bin is about $30/4093 \approx 0.0073$ wide in scaled units. On the passenger scale, the quantization error is at most half a bin width times $s$, roughly $0.0037 \times 262.5 \approx 1$ passenger. The tradeoff is the range: any value more than 15 times the context's mean absolute level is clipped to the edge bin, so a forecast cannot grow past that.

Training and sampling

The token sequence $z_{1:C}$ goes into a T5 encoder, and the decoder is trained to emit the future tokens $z_{C+1:C+H}$ followed by an end token. The loss is plain categorical cross-entropy over the vocabulary $V$:

Notice what is unusual. Cross-entropy treats bin 1000 and bin 1001 as unrelated classes; the model is never told they are close numbers. It learns that from data. The upside is that the output is a free-form categorical distribution, so it can represent skewed or multimodal futures without assuming a Student-t or Gaussian shape.

The training data combines public datasets with synthetic series. The paper introduces TSMixup, which creates new training series by mixing several real ones, and KernelSynth, which samples series from Gaussian processes built from randomly combined kernels.

At inference, Chronos samples tokens autoregressively from the decoder, dequantizes each one with $d(j)$, and multiplies by $s$ to return to the original scale. Repeating gives $S$ sample paths, and the forecast quantile at step $h$ is the empirical quantile of the $S$ samples:

The package's predict_quantiles method does exactly this, returning both the quantiles and the sample mean.

flowchart LR
    A["Context x1 to xC"] --> B["Mean scaling<br/>divide by mean abs value s"]
    B --> C["Uniform bins on -15 to 15<br/>value to token id"]
    C --> D["T5 encoder"]
    D --> E["T5 decoder<br/>softmax over vocabulary"]
    E --> F["Sample token"]
    F -.->|"feed back"| E
    F --> G["Dequantize<br/>bin centre times s"]
    G --> H["S sample paths"]
    H --> I["Empirical quantiles<br/>and mean"]

The original models come in sizes from tiny (about 8M parameters) to large (about 710M), all with a 512-step context and a default prediction length of 64.

Chronos-Bolt is a later variant in the same package. Instead of one token per value and sampling, it splits the scaled context into patches, encodes them with a T5 encoder, and has the decoder output several quantiles for many future steps at once, trained with a quantile loss. There is no sampling loop, so it is much faster; the authors report large speedups over the original models of the same size. Current releases of the package also include a newer Chronos-2 pipeline with covariate support, which this guide does not cover.

TimesFM, Moirai and TimeGPT in brief

TimesFM

TimesFM, from Google, is a patched decoder-only transformer. The series is cut into non-overlapping input patches of length $p$ (32 in the paper). Each patch, together with a mask of which points are padding, goes through a small residual MLP to become one token, and a causal transformer runs over the patch tokens. The output from each position is a whole output patch of length $p_{\text{out}}$ (128 in the paper), longer than the input patch:

Making the output patch longer than the input patch means a 128-step forecast needs one decoding step instead of four, which cuts the autoregressive error build-up and the latency. During training, random masking of the start of the context teaches the model to handle every context length. The paper describes a roughly 200M parameter model pretrained on a large corpus that includes Google Trends, Wikipedia page views and synthetic data, trained with a point forecasting loss. Later releases added larger checkpoints and quantile outputs; check the model card for the version you load.

Moirai

Moirai, from Salesforce, is a masked-encoder transformer aimed at being universal across frequencies and numbers of variables. Three ideas matter.

• Multiple patch sizes. It has several input projection layers, one per patch size, and picks a larger patch for high-frequency data (minutely, hourly) and a smaller one for low-frequency data (monthly, yearly). This keeps the token count manageable without throwing away detail on short series.
• Any-variate attention. A multivariate series is flattened into one long sequence of patch tokens from all variables. Rotary embeddings encode time position, and a learned attention bias tells attention whether two tokens belong to the same variable or different ones. Because the bias depends only on same-or-different, the model is invariant to the order of the variables and can accept any number of them, including covariates.
• Mixture distribution head. The output is a mixture of distributions, combining a Student-t, a negative binomial, a log-normal and a low-variance normal, with learned weights: $p(y) = \sum_k w_k \, p_k(y)$. Different components suit count data, positive skewed data, heavy-tailed data and near-deterministic data.

It was pretrained on LOTSA, a large open collection of time series assembled by the authors.

TimeGPT

TimeGPT, from Nixtla, is offered as a hosted API. The weights are not public, you need an API key, and your data is sent to a third-party service. It is mentioned here for completeness only; this guide does not call it, and the same evaluation protocol applies if you do: same origins, same baselines, same metrics, plus a careful reading of the data handling terms.

Tutorial part 1: the baselines you must beat

Everything starts from numbers you trust. This block was executed in this environment with statsforecast 2.1.1, pandas 2.3.3 and numpy 2.5.4 on CPU.

python -m venv .venv && source .venv/bin/activate
pip install statsforecast

import numpy as np
import pandas as pd
from statsforecast import StatsForecast
from statsforecast.models import Naive, SeasonalNaive, AutoETS
from statsforecast.utils import AirPassengersDF

H, M = 12, 12                      # horizon and season length
df = AirPassengersDF.copy()        # columns: unique_id, ds (month-end), y
train, test = df.iloc[:-H], df.iloc[-H:]

def score(y_true, y_pred, y_train, m=M):
    """MAE, RMSE and MASE (scaled by in-sample seasonal-naive MAE)."""
    y_true, y_pred, y_train = map(np.asarray, (y_true, y_pred, y_train))
    err = y_true - y_pred
    scale = np.mean(np.abs(y_train[m:] - y_train[:-m]))
    return {
        "MAE": np.mean(np.abs(err)),
        "RMSE": np.sqrt(np.mean(err ** 2)),
        "MASE": np.mean(np.abs(err)) / scale,
    }

sf = StatsForecast(
    models=[Naive(), SeasonalNaive(season_length=M), AutoETS(season_length=M)],
    freq="ME",       # AirPassengersDF uses month-end timestamps
    n_jobs=1,
)
fc = sf.forecast(df=train, h=H, level=[80]).merge(test, on=["unique_id", "ds"])

rows = {name: score(fc["y"], fc[name], train["y"])
        for name in ["Naive", "SeasonalNaive", "AutoETS"]}
print(pd.DataFrame(rows).T.round(2))

inside = (fc["y"] >= fc["AutoETS-lo-80"]) & (fc["y"] <= fc["AutoETS-hi-80"])
print("AutoETS 80% interval coverage on the test year:", inside.mean().round(2))

# Rolling-origin backtest: three yearly origins, refit at each one.
cv = sf.cross_validation(df=df, h=H, n_windows=3, step_size=H)
mae = (cv.assign(**{c: (cv["y"] - cv[c]).abs() for c in ["SeasonalNaive", "AutoETS"]})
         .groupby("cutoff")[["SeasonalNaive", "AutoETS"]].mean())
print(mae.round(2))
print("mean over origins:", mae.mean().round(2).to_dict())

Output from the run:

                 MAE    RMSE  MASE
Naive          76.00  102.98  2.50
SeasonalNaive  47.83   50.71  1.57
AutoETS        35.61   40.08  1.17
AutoETS 80% interval coverage on the test year: 0.58
            SeasonalNaive  AutoETS
cutoff
1957-12-31          12.58    18.44
1958-12-31          47.33    44.41
1959-12-31          47.83    35.61
mean over origins: {'SeasonalNaive': 35.92, 'AutoETS': 32.82}

The MASE denominator is the in-sample seasonal-naive MAE on the 132 training months, about 30.45 passengers:

A MASE of 1.17 means AutoETS's test error is 17% larger than the typical one-season-ago error inside the training data. That sounds bad until you notice that 1960 grew faster than earlier years, so every method finds the test year harder than the training years. AutoETS selected an ETS(M,N,M) model: multiplicative errors and seasonality, no trend component. Its 80% interval covered only 7 of 12 test months, so it is overconfident on this year.

The rolling backtest is the more important output. Over three yearly origins, AutoETS wins on average, but seasonal naive wins clearly on the 1958 forecast year. A single holdout would have hidden that. This is the bar a foundation model has to clear: not one number, but a pattern across origins.

Tutorial part 2: Chronos zero-shot (not executed here)

The code in this section was not executed in this environment. It is written against the chronos-forecasting package API (the BaseChronosPipeline.from_pretrained and predict_quantiles methods as they appear in release 2.x of the package). Running it downloads pretrained weights from the Hugging Face Hub. Results will depend on the checkpoint, the package version and the random seed, so this guide reports no numbers for it.

pip install "chronos-forecasting>=2.0" statsforecast

import numpy as np
import pandas as pd
import torch
from chronos import BaseChronosPipeline
from statsforecast.utils import AirPassengersDF

H, M = 12, 12
df = AirPassengersDF.copy()
train, test = df.iloc[:-H], df.iloc[-H:]

pipeline = BaseChronosPipeline.from_pretrained(
    "amazon/chronos-t5-small",     # or "amazon/chronos-bolt-small"
    device_map="cpu",              # "cuda" if you have a GPU
    torch_dtype=torch.float32,
)

torch.manual_seed(0)               # Chronos-T5 samples; fix the seed
context = torch.tensor(train["y"].to_numpy(), dtype=torch.float32)
quantiles, mean = pipeline.predict_quantiles(
    context,                       # 1D tensor: one series
    prediction_length=H,
    quantile_levels=[0.1, 0.5, 0.9],
)
# quantiles: (batch=1, H, 3)   mean: (batch=1, H)
q10, q50, q90 = (quantiles[0, :, i].numpy() for i in range(3))

For the original Chronos-T5 models, predict_quantiles also accepts num_samples= (passed through to sampling); more samples give smoother quantiles at a linear cost in compute. Chronos-Bolt models return quantiles directly and do not accept sampling arguments, so drop num_samples when you switch to them.

Score it with the same score function from part 1, using the median as the point forecast, and add two probabilistic checks:

def pinball(y, q, tau):
    u = np.asarray(y) - np.asarray(q)
    return np.mean(np.maximum(tau * u, (tau - 1) * u))

y_test = test["y"].to_numpy()
point = score(y_test, q50, train["y"])
coverage_80 = np.mean((y_test >= q10) & (y_test <= q90))
mean_pinball = np.mean([pinball(y_test, q, t)
                        for q, t in [(q10, 0.1), (q50, 0.5), (q90, 0.9)]])
print({k: round(v, 2) for k, v in point.items()},
      "coverage80:", round(coverage_80, 2), "pinball:", round(mean_pinball, 2))

To put Chronos into the same rolling backtest as the baselines, loop over the cutoffs printed in part 1, slice the history up to each cutoff as the context, forecast 12 steps, and score each window separately. No model state carries over between origins because nothing is trained, which makes zero-shot backtesting simple and honest.

Tutorial part 3: Lag-Llama zero-shot (not executed here)

This code was also not executed in this environment. It follows the usage pattern in the official Lag-Llama repository README and demo notebooks: install the repository's requirements, download the checkpoint from the Hugging Face Hub, read the hyperparameters stored in the checkpoint, and build a GluonTS predictor. The repository pins specific versions of GluonTS and its dependencies, so follow its README for installation and check it for any API changes before running.

git clone https://github.com/time-series-foundation-models/lag-llama/
cd lag-llama
pip install -r requirements.txt
huggingface-cli download time-series-foundation-models/Lag-Llama lag-llama.ckpt --local-dir .

import numpy as np
import torch
from gluonts.dataset.pandas import PandasDataset
from gluonts.evaluation import make_evaluation_predictions
from lag_llama.gluon.estimator import LagLlamaEstimator
from statsforecast.utils import AirPassengersDF

H, CONTEXT = 12, 32
df = AirPassengersDF.copy()
df["y"] = df["y"].astype("float32")
df["unique_id"] = df["unique_id"].astype(str)

# Full series: make_evaluation_predictions holds out the last H points itself.
dataset = PandasDataset.from_long_dataframe(
    df.set_index("ds"), target="y", item_id="unique_id", freq="M"
)

ckpt = torch.load("lag-llama.ckpt", map_location="cpu", weights_only=False)
args = ckpt["hyper_parameters"]["model_kwargs"]

estimator = LagLlamaEstimator(
    ckpt_path="lag-llama.ckpt",
    prediction_length=H,
    context_length=CONTEXT,
    input_size=args["input_size"],
    n_layer=args["n_layer"],
    n_embd_per_head=args["n_embd_per_head"],
    n_head=args["n_head"],
    scaling=args["scaling"],
    time_feat=args["time_feat"],
    batch_size=1,
    num_parallel_samples=100,
    device=torch.device("cpu"),
)
lightning_module = estimator.create_lightning_module()
transformation = estimator.create_transformation()
predictor = estimator.create_predictor(transformation, lightning_module)

forecast_it, ts_it = make_evaluation_predictions(
    dataset=dataset, predictor=predictor, num_samples=100
)
forecast = list(forecast_it)[0]          # a GluonTS SampleForecast
q10, q50, q90 = (forecast.quantile(t) for t in (0.1, 0.5, 0.9))

Score q10, q50 and q90 with exactly the same score, pinball and coverage code as in part 2. The README also shows how to pass a RoPE scaling argument when you use a context longer than the pretrained one, and how to fine-tune: create the estimator with a learning rate and trainer_kwargs such as a maximum number of epochs, then call estimator.train(train_dataset, cache_data=True, shuffle_buffer_length=1000), which returns a predictor. If you fine-tune inside a backtest, train separately for each origin on data before that origin.

Reading the comparison table

AirPassengers, last 12 months held out, $H = 12$, $m = 12$. Baseline numbers come from the executed run above (statsforecast 2.1.1, Python 3.14, CPU); they are deterministic for these models but can shift slightly across library versions. Foundation-model rows were not run here.

When you fill in the last two rows on your own machine, read them the following way.

• Compare against the AutoETS row first, not seasonal naive. Beating seasonal naive on a strongly seasonal, trending series is a low bar. AutoETS is the honest opponent.
• Look at the per-origin numbers. A model that wins one year by a lot and loses two years narrowly has a different risk profile from one that wins all three by a little.
• Remember the leakage caveat. AirPassengers is one of the most copied series in existence. A strong zero-shot score here cannot rule out that the model has seen it. Repeat the exercise on a private series before drawing any conclusions.
• Check intervals. AutoETS covered 7 of 12 months at 80%. If a foundation model covers 10 or 11 with intervals of similar width, that is a real advantage for a decision that depends on the tails.
• Twelve points is tiny. Differences of a few passengers in MAE on one year are well within noise. Large conclusions need many series and many origins.

Decision guide: zero-shot, fine-tune, or fit a model

There is no universal winner. The choice depends on how much history you have, how many series, what you know about the domain, and what the forecast costs to produce.

flowchart TD
    A["New forecasting task"] --> B{"Enough history per series?<br/>several seasons"}
    B -->|"yes"| C{"Few series, clear<br/>trend and seasonality?"}
    C -->|"yes"| D["Fit AutoETS or AutoARIMA<br/>cheap, explainable"]
    C -->|"no, many related series"| E["Try global neural model<br/>and zero-shot foundation model"]
    B -->|"no, short or new series"| F["Zero-shot foundation model<br/>vs seasonal naive"]
    E --> G{"Zero-shot beats baselines<br/>in rolling backtest?"}
    F --> G
    G -->|"yes"| H{"Latency and cost acceptable?"}
    G -->|"no"| I{"Enough data to fine-tune<br/>per origin?"}
    I -->|"yes"| J["Fine-tune, re-run backtest"]
    I -->|"no"| D
    J --> G
    H -->|"yes"| K["Deploy with monitoring<br/>keep baseline as fallback"]
    H -->|"no"| L["Smaller or faster variant<br/>such as Chronos-Bolt"]
    L --> G

Fit a statistical model when you have a handful of series with several seasons of history and stable structure. You get an explainable model, fast refits, and a well-understood interval. The statistical guide covers this in depth.

Use a zero-shot foundation model when series are short, new, or numerous and heterogeneous, when you cannot afford per-series modelling effort, or as a strong second opinion. Cold-start products, new sensors and quick exploratory forecasts are natural fits.

Fine-tune when zero-shot is close to but not better than your baselines, you have a reasonable amount of in-domain data, and you can afford a proper per-origin backtest. Fine-tuning also adds the overfitting risks of any trained model, so it needs early stopping and a validation split that respects time.

Train a task-specific network (the neural and transformer guides) when you have many related series and rich covariates that the pretrained model cannot use.

Cost and latency

On the machine used here (Apple Silicon CPU), fitting AutoETS to the 132 training months and producing a 12-step forecast took about 40 milliseconds per series. Foundation models trade that for a fixed inference cost that does not depend on fitting but does depend on model size and decoding style.

• Chronos-T5 samples token by token, so cost grows with number of samples times horizon times model size. The released sizes range from about 8M to about 710M parameters; the larger ones are slow on CPU for long horizons.
• Chronos-Bolt and TimesFM produce many steps per forward pass and no sampling loop, which makes them much cheaper per forecast.
• Lag-Llama is small, but forecasts autoregressively with many sample paths, and fine-tuning adds a training job per retrain.
• TimeGPT moves the compute to a hosted service, which trades local hardware for network latency, per-call pricing and data governance questions.

Also count the hidden costs: model files of tens to hundreds of megabytes per checkpoint, pinned dependency stacks (Lag-Llama in particular), GPU availability, and licences that differ between checkpoints.

Pitfalls and a checklist for fair evaluation

Common ways a foundation-model comparison goes wrong:

• Benchmark overlap. Evaluating on a public dataset that sits inside the pretraining corpus.
• Unequal context. Giving the foundation model the full history but refitting the baseline on a shorter window, or the reverse. Every method should see exactly the history available at each cutoff.
• Point versus median versus mean. Sample-based models give a mean and quantiles. Using the mean for one model and the median for another changes MAE comparisons. State which you used.
• Frequency and timestamp handling. AirPassengers stamps months at month-end; mixing month-start and month-end conventions silently shifts calendar features or breaks joins.
• Default context lengths. Chronos truncates to its context length, and Lag-Llama's pretrained context is short. Defaults may not suit your series; tune them on validation origins, not the test year.
• Scale limits. Chronos clips scaled values to its bin range, so a series that is about to grow far beyond its recent mean absolute level cannot be forecast past that limit.
• Unseeded sampling. Sampled forecasts vary from run to run. Fix seeds and, better, report variation across several seeds.

Checklist before you trust a result:

1. The evaluation series are private or provably absent from the pretraining corpus, or the leakage risk is stated next to the result.
2. Seasonal naive and AutoETS (or AutoARIMA) are in the same table, same origins, same horizon.
3. At least three rolling origins, with per-origin results reported, not only the average.
4. MAE, RMSE and MASE for point accuracy, plus interval coverage and a pinball or CRPS score for the distribution.
5. Context, horizon, model checkpoint name, package versions and seeds are all recorded.
6. Fine-tuning, if any, uses only data before each origin.
7. Latency and memory are measured on the hardware you will deploy on.
8. A fallback baseline stays in production, with monitoring that compares the two.

Practice exercises

1. Run the Chronos code from part 2 locally with amazon/chronos-t5-small and amazon/chronos-bolt-small. Fill in the table above for both, including 80% coverage, and time each call.
2. Extend the Chronos code to the three rolling origins from part 1. Does the ranking against AutoETS stay the same at every origin?
3. Run Chronos-T5 with five different seeds and 20 versus 200 samples. How much does MAE move? Is that variation larger than the gap to AutoETS?
4. Compute Chronos's mean scale $s$ for the AirPassengers training context by hand and check which bin the value 622 (July 1960) would fall into. How far above the context mean could the series go before clipping?
5. Write down the Lag-Llama lag set for a monthly series from the repository code. Which lags are seasonal, which are irrelevant for monthly data, and how many months of history does the largest lag need?
6. Pick a private or self-generated series (for example, a simulated series with a level shift in the test year) where corpus overlap is impossible, and repeat the whole comparison. Does the foundation model's advantage over AutoETS change?

References

• Rasul et al., Lag-Llama: Towards Foundation Models for Probabilistic Time Series Forecasting, and the official repository.
• Ansari et al., Chronos: Learning the Language of Time Series.
• Das et al., A decoder-only foundation model for time-series forecasting (TimesFM).
• Woo et al., Unified Training of Universal Time Series Forecasting Transformers (Moirai).
• Garza and Mergenthaler-Canseco, TimeGPT-1.
• Salinas et al., DeepAR: Probabilistic Forecasting with Autoregressive Recurrent Networks.
• Nixtla, statsforecast documentation.
• Hyndman and Athanasopoulos, Forecasting: Principles and Practice.