Week 7: Combining the models
Goal of today: a combined model that uses the linear regression, the random forest and the neural net together. It is saved to a file, and the web page gets a Train now button and shows our best forecast for tomorrow:
Tomorrow (2026-10-07), max temperature
ecmwf: 22.2 °C gfs: 24.2 °C icon: 24.6 °C
Our best forecast: 23.0 °C [Train now]
How to work: do the steps in order. After each step, run the code and compare with the expected output. Go on only when yours looks the same. (Your numbers will differ a little, because you have more days of data.)
Start from your week 6 project (or the week 6 download), with the venv active.
What you need to know
Ensemble: combining models. Each model makes different mistakes. The random forest is good at some unusual situations, the linear model at the typical days, the neural net is somewhere in between. If we combine their answers wisely, some of their mistakes cancel out. You already saw this idea work: our models beat the single weather forecasts by combining them. A group of models working together is called an ensemble.
The simplest ensemble is the average:
forecast = (linear_answer + forest_answer + neural_answer) / 3
But the average trusts every model equally. It is better to learn how much to trust each one.
Stacking does exactly that. A second model learns from the answers of the first models:
┌─▶ linear regression ─┐
ecmwf, gfs, icon├─▶ random forest ─┼─▶ final linear regression ─▶ forecast
└─▶ neural net ─┘
(level 1) (level 2: learns how much
to trust each model)
Cross-validation: honest answers for level 2. The level-2 model must learn from honest answers: answers on days the level-1 models did not train on. On their own training days every model looks good (it has seen the answers), so level 2 would trust the model that memorized the most, not the one that forecasts best. So the training days are cut into 5 parts:
part: 1 2 3 4 5
round 1: [test][train][train][train][train] -> honest answers for part 1
round 2: [train][test][train][train][train] -> honest answers for part 2
... (5 rounds)
After 5 rounds we have an honest answer for every training day. This is called cross-validation.
StackingRegressor from scikit-learn does all of this for us.
Saving a model to a file. Training takes seconds; predicting takes milliseconds. So we train once, save the trained model
to a file, and the web page just loads it. Turning a Python object into a file is called serialization;
we use the joblib package (it comes with scikit-learn):
joblib.dump(model, "model.joblib") # save
model = joblib.load("model.joblib") # load (also in another program, days later)
Train on everything before using it. For measuring the error we keep the newest 20% aside, as in week 5. But the model we really use is trained once more on all days: the newest days are the most valuable for tomorrow.
GET and POST. The browser talks to the server with requests. The two most common kinds: - GET: "give me a page". Typing an address or clicking a link is a GET. A GET must never change anything. - POST: "do something". Sending a form is a POST. Training (which saves a new model file) changes something, so it is a POST.
HTML forms. A form with a button sends a POST request when you click the button:
<form method="post" action="/train/eger/models">
<button>Train now</button>
</form>
In Flask, a route accepts POST only if you say so: @app.route("/train/...", methods=["POST"]).
Step 1: Try stacking in the interactive Python
Before changing any file, try the idea. Start python in your project folder and type
(the ... lines are continuation lines, Python shows them by itself):
python
>>> from sklearn.ensemble import StackingRegressor, RandomForestRegressor
>>> from sklearn.linear_model import LinearRegression
>>> import ml
>>> sources = ml.SETS["models"]
>>> table = ml.build_dataset("eger", sources)
>>> train, test = ml.split_by_date(table)
>>> combined = StackingRegressor(
... estimators=[("linear", LinearRegression()),
... ("random_forest", RandomForestRegressor(n_estimators=200, min_samples_leaf=5, random_state=0)),
... ("neural_net", ml.make_neural_net())],
... final_estimator=LinearRegression())
>>> combined.fit(train[sources], train["measured"])
>>> print(ml.mae(combined.predict(test[sources]), test["measured"]))
0.6735155427909901
>>> print(combined.final_estimator_.coef_)
[ 1.01939435 0.17898281 -0.19674706]
>>> exit()
fit takes some seconds: it trains every level-1 model 5 times (cross-validation) and once more on all training days.
What you see:
- The combined model's error on the test days is 0.67 °C. In week 6, the best single model in Eger was linear regression with 0.68 °C.
- final_estimator_ is the level-2 linear regression. Its coef_ are the weights of the three level-1 models:
linear 1.02, random forest 0.18, neural net −0.20. A negative weight looks strange, but it is fine:
the linear model and the neural net give almost the same answers, so level 2 takes "a bit more linear, a bit less
neural net". The weights add up to about 1, so the result stays a temperature.
Step 2: Add the combined model to ml.py
In ml.py, change the sklearn.ensemble import at the top to:
from sklearn.ensemble import RandomForestRegressor, StackingRegressor
Then change make_models(): the combined model is the 4th model in the dictionary:
def make_models():
linear = LinearRegression()
forest = RandomForestRegressor(n_estimators=200, min_samples_leaf=5, random_state=0)
neural = make_neural_net()
combined = StackingRegressor(
estimators=[("linear", LinearRegression()),
("random_forest", RandomForestRegressor(n_estimators=200, min_samples_leaf=5, random_state=0)),
("neural_net", make_neural_net())],
final_estimator=LinearRegression(),
)
return {"linear": linear, "random_forest": forest, "neural_net": neural, "combined": combined}
The combined model gets its own, fresh level-1 models. (It must not share them with the other three entries, because training one would change the other.)
Run it:
python ml.py
You should see (it takes a few seconds longer than last week):
Budapest / models: 974 complete days
ecmwf 0.64 °C
gfs 1.18 °C
icon 0.70 °C
average of sources 0.67 °C
linear 0.58 °C <-- best
random_forest 0.73 °C
neural_net 0.59 °C
combined 0.59 °C
Budapest / providers: 0 complete days
not enough data yet (need 30)
Eger / models: 974 complete days
ecmwf 0.82 °C
gfs 1.56 °C
icon 1.28 °C
average of sources 0.93 °C
linear 0.68 °C
random_forest 0.82 °C
neural_net 0.69 °C
combined 0.67 °C <-- best
Eger / providers: 0 complete days
not enough data yet (need 30)
In Eger the combined model is the best. In Budapest it is very close to the best. It is never the worst: that is why we will use it on the web page.
Step 3: Save and load a model: a tiny example
First try joblib on a very small model, so you see what happens. Start python:
python
>>> import joblib
>>> from sklearn.linear_model import LinearRegression
>>> model = LinearRegression()
>>> model.fit([[1], [2], [3]], [2, 4, 6])
>>> print(model.predict([[10]]))
[20.]
>>> joblib.dump(model, "test_model.joblib")
>>> exit()
The model learned "y = 2·x", so for 10 it says 20. Now there is a file test_model.joblib in your folder.
Start a new python (the old model is gone from memory) and load it:
python
>>> import joblib
>>> loaded = joblib.load("test_model.joblib")
>>> print(loaded.predict([[10]]))
[20.]
>>> exit()
The loaded model gives the same answer without training again. You can delete test_model.joblib now.
Step 4: config.py: a folder for the saved models
Add one line after DB_PATH:
MODELS_DIR = BASE_DIR / "models"
The folder does not exist yet. The code creates it the first time it saves a model.
Step 5: Train and save the combined model
In ml.py, add import joblib at the top (first line of the imports), and add MODELS_DIR to the config import:
import joblib
import numpy as np
...
from config import CITIES, MODELS_DIR, SETS
Add these two functions before print_scores():
def model_path(city, data_set):
return MODELS_DIR / f"{city}_{data_set}.joblib"
def train_and_save(city, data_set):
"""Measure the scores, then train the combined model on ALL days and save it to a file."""
scores, days = evaluate(city, data_set)
if scores is None:
return None, days
sources = SETS[data_set]
table = build_dataset(city, sources)
model = make_models()["combined"]
model.fit(table[sources], table["measured"])
MODELS_DIR.mkdir(exist_ok=True)
joblib.dump({"model": model, "sources": sources, "scores": scores, "days": days},
model_path(city, data_set))
return scores, days
model_path()gives the file name of a model, e.g.models/eger_models.joblib. One file per city and data set.train_and_save()first callsevaluate(): the honest error on the newest 20%, for the web page. Then it trains the combined model again, now on all days, and saves it.- We save a dictionary, not only the model: the list of sources (which columns the model needs), the scores and the number of days. Whoever loads the file later gets everything in one piece.
- If there is not enough data, it returns
Noneand saves nothing.
Try it:
python
>>> from ml import train_and_save
>>> scores, days = train_and_save("eger", "models")
>>> print(days)
974
>>> print(scores["combined"])
0.6735155427909901
>>> print(train_and_save("eger", "providers"))
(None, 0)
>>> exit()
Now look into your project folder: there is a new folder models with the file eger_models.joblib.
There is no file for providers: not enough data yet.
Step 6: Forecast tomorrow with the saved model
First look at the data we will use. The forecasts for tomorrow are already in the database (collect.py saves them every morning),
only the measured value is missing:
python
>>> from db import load_table
>>> print(load_table("eger", ["ecmwf", "gfs", "icon"]).tail(3))
ecmwf gfs icon measured
day
2026-10-05 23.2 23.9 25.0 23.0
2026-10-06 23.2 24.3 25.2 NaN
2026-10-07 22.2 24.2 24.6 NaN
>>> exit()
The last row is tomorrow. Its forecasts are the inputs for our model.
In ml.py, add from sources import tomorrow to the imports (after from db import load_table), and add this function
after train_and_save():
def predict_tomorrow(city, data_set):
"""Use the saved model and today's forecasts to predict tomorrow's max temperature."""
path = model_path(city, data_set)
if not path.exists():
return None
saved = joblib.load(path)
sources = saved["sources"]
table = load_table(city, sources)
if tomorrow() not in table.index:
return None
row = table.loc[[tomorrow()], sources]
if row.isna().values.any():
return None # a source is missing for tomorrow
return round(float(saved["model"].predict(row)[0]), 1)
- It loads the file saved in step 5 and takes tomorrow's row from the table.
table.loc[[tomorrow()], sources]with double brackets gives a one-row table (DataFrame). With single brackets you would get a single row (Series), andpredict()needs a table.- If anything is missing (no model file, no forecasts for tomorrow, one source missing), it returns
Noneinstead of crashing. The web page will then say "no trained model yet". round(..., 1): one decimal is enough for a temperature.
Try it:
python
>>> from ml import predict_tomorrow
>>> print(predict_tomorrow("eger", "models"))
23.0
>>> print(predict_tomorrow("budapest", "models"))
None
>>> exit()
Budapest gives None: we have not trained a Budapest model yet (step 5 trained only Eger). The web page will fix that with a button.
Step 7: Show the best forecast on the web page
In app.py:
- Add this import (after
from db import load_table):python from ml import mae, predict_tomorrow - Delete the
mae()function fromapp.py. It is inml.pynow, and we import it from there. (Two copies of the same function would sooner or later become different.) - In
city_page(), add one line to therender_template(...)call, after thetomorrow_forecasts=...line:python best_forecast=predict_tomorrow(city, data_set),
In templates/city.html, add two lines in the box at the top, just before the closing </div> of the box:
<br>
<b>Our best forecast: {{ best_forecast if best_forecast is not none else "no trained model yet" }}{% if best_forecast is not none %} °C{% endif %}</b>
The Jinja if ... else prints the forecast if there is one, otherwise the text "no trained model yet".
Run the app:
python app.py
Open http://127.0.0.1:5000/city/eger. You should see in the box at the top:
Our best forecast: 23.0 °C
Now click Budapest in the menu:
Our best forecast: no trained model yet
Correct: there is no Budapest model file yet. Stop the server with Ctrl+C.
Step 8: A route that trains
In app.py, extend the ml import:
from ml import mae, predict_tomorrow, print_scores, train_and_save
and add this route before the if __name__ == "__main__": line at the end:
@app.route("/train/<city>/<data_set>", methods=["POST"])
def train(city, data_set):
if city not in CITIES or data_set not in SETS:
abort(404)
scores, days = train_and_save(city, data_set)
if scores:
print_scores(scores)
best = min(scores, key=scores.get) if scores else None
return render_template("train.html", cities=CITIES, city=city, data_set=data_set, sets=SETS,
scores=scores, days=days, best=best)
methods=["POST"]: this address works only with POST, because it changes something (it saves a new model file).- An unknown city or data set gives "Not Found" (404).
print_scores()prints the results into the terminal too, so you see them there as well.bestis the name of the model with the smallest error; the result page shows it in green.
Create the result page, templates/train.html:
templates/train.html
{% extends "base.html" %}
{% block content %}
<h2>Training: {{ cities[city].name }} – {{ data_set }}</h2>
{% if scores %}
<p>Trained on {{ days }} complete days. Error on the newest 20% of the days (lower is better):</p>
<table>
<tr><th>model</th><th>average error</th></tr>
{% for name, value in scores.items() %}
<tr {% if name == best %}class="best"{% endif %}><td>{{ name }}</td><td>{{ "%.2f" | format(value) }} °C</td></tr>
{% endfor %}
</table>
<p>The <b>combined</b> model was saved and is used for "our best forecast".</p>
{% else %}
<p>Not enough data: only {{ days }} complete days. Keep collecting.</p>
{% endif %}
<p><a href="{{ url_for('city_page', city=city, set=data_set) }}">Back</a></p>
{% endblock %}
"%.2f" | format(value) prints the number with 2 decimals.
Start the app again (python app.py) and open this address in the browser: http://127.0.0.1:5000/train/budapest/models
You should see:
Method Not Allowed
The method is not allowed for the requested URL.
This is correct! Typing an address is a GET request, and the route accepts only POST. So nobody can start a training just by opening a link. In the next step, the button sends the POST.
Step 9: The Train now button
In templates/city.html, add the form right after the Our best forecast line from step 7 (still inside the box):
<form method="post" action="{{ url_for('train', city=city, data_set=data_set) }}" style="display:inline">
<button>Train now</button>
</form>
url_for('train', city=city, data_set=data_set)builds the address of thetrain()route, e.g./train/budapest/models. If you rename the route later, the link still works.style="display:inline"keeps the button on the same line as the text.
Start the app (python app.py), open http://127.0.0.1:5000/city/budapest and click Train now.
It takes 5–30 seconds (the button seems to do nothing; wait). You should see:
Training: Budapest – models
Trained on 974 complete days. Error on the newest 20% of the days (lower is better):
model average error
ecmwf 0.64 °C
gfs 1.18 °C
icon 0.70 °C
average of sources 0.67 °C
linear 0.58 °C <- green
random_forest 0.73 °C
neural_net 0.59 °C
combined 0.59 °C
The combined model was saved and is used for "our best forecast".
In the terminal you see the request: "POST /train/budapest/models HTTP/1.1" 200.
Click Back. The box now shows:
Our best forecast: 23.8 °C
Now switch to providers in the menu and click Train now:
Training: Budapest – providers
Not enough data: only 0 complete days. Keep collecting.
(If you have collected for some weeks already, you may see a result instead. You need at least 30 complete days.)
Done!
Your complete changed files
Compare with yours. If something does not work, copy these.
config.py
"""Settings shared by all scripts."""
from pathlib import Path
# Files are always created next to this file, no matter where you start Python from.
BASE_DIR = Path(__file__).resolve().parent
DB_PATH = BASE_DIR / "weather.db"
MODELS_DIR = BASE_DIR / "models"
TIMEZONE = "Europe/Budapest"
CITIES = {
"budapest": {"name": "Budapest", "lat": 47.4979, "lon": 19.0402},
"eger": {"name": "Eger", "lat": 47.9025, "lon": 20.3772},
}
# MET Norway wants to know who is asking. You may add your own e-mail address,
# but NOT a fake one like ...@example.com (that is blocked with error 403).
USER_AGENT = "weather-lab/1.0 (university lab project)"
# Live set: three independent forecast providers, collected by us every day.
PROVIDER_SOURCES = ["open_meteo", "met_norway", "wttr"]
# History set: three weather models whose old day-ahead forecasts can be downloaded.
MODEL_SOURCES = ["ecmwf", "gfs", "icon"]
OPEN_METEO_MODELS = {"ecmwf": "ecmwf_ifs025", "gfs": "gfs_seamless", "icon": "icon_seamless"}
HISTORY_START = "2024-02-05"
SETS = {"models": MODEL_SOURCES, "providers": PROVIDER_SOURCES}
ml.py
"""Machine learning: learn how to combine several forecasts into a better one.
Run it directly to compare all models: python ml.py
"""
import joblib
import numpy as np
from sklearn.ensemble import RandomForestRegressor, StackingRegressor
from sklearn.linear_model import LinearRegression
from sklearn.neural_network import MLPRegressor
from sklearn.pipeline import make_pipeline
from sklearn.preprocessing import StandardScaler
from config import CITIES, MODELS_DIR, SETS
from db import load_table
from sources import tomorrow
MIN_DAYS = 30 # do not train on fewer complete days than this
def build_dataset(city, sources):
"""Complete days only: every forecast AND the measured value must be present."""
return load_table(city, sources).dropna()
def split_by_date(table, test_share=0.2):
"""Older days for training, the newest days for testing. Never shuffle time series!"""
cut = int(len(table) * (1 - test_share))
return table.iloc[:cut], table.iloc[cut:]
def mae(predicted, measured):
"""Mean absolute error: on average, how many degrees we are wrong."""
return float(np.mean(np.abs(np.asarray(predicted) - np.asarray(measured))))
def baseline_scores(test, sources):
"""How good are the forecasts without any machine learning?"""
scores = {source: mae(test[source], test["measured"]) for source in sources}
scores["average of sources"] = mae(test[sources].mean(axis=1), test["measured"])
return scores
def make_neural_net():
# The scaler moves every input to a similar range; neural nets learn badly without it.
return make_pipeline(
StandardScaler(),
MLPRegressor(hidden_layer_sizes=(16, 8), alpha=0.01, max_iter=3000, random_state=0),
)
def make_models():
linear = LinearRegression()
forest = RandomForestRegressor(n_estimators=200, min_samples_leaf=5, random_state=0)
neural = make_neural_net()
combined = StackingRegressor(
estimators=[("linear", LinearRegression()),
("random_forest", RandomForestRegressor(n_estimators=200, min_samples_leaf=5, random_state=0)),
("neural_net", make_neural_net())],
final_estimator=LinearRegression(),
)
return {"linear": linear, "random_forest": forest, "neural_net": neural, "combined": combined}
def evaluate(city, data_set):
"""Train every model on the older days, measure the error on the newest days."""
sources = SETS[data_set]
table = build_dataset(city, sources)
if len(table) < MIN_DAYS:
return None, len(table)
train, test = split_by_date(table)
scores = baseline_scores(test, sources)
for name, model in make_models().items():
model.fit(train[sources], train["measured"])
scores[name] = mae(model.predict(test[sources]), test["measured"])
return scores, len(table)
def model_path(city, data_set):
return MODELS_DIR / f"{city}_{data_set}.joblib"
def train_and_save(city, data_set):
"""Measure the scores, then train the combined model on ALL days and save it to a file."""
scores, days = evaluate(city, data_set)
if scores is None:
return None, days
sources = SETS[data_set]
table = build_dataset(city, sources)
model = make_models()["combined"]
model.fit(table[sources], table["measured"])
MODELS_DIR.mkdir(exist_ok=True)
joblib.dump({"model": model, "sources": sources, "scores": scores, "days": days},
model_path(city, data_set))
return scores, days
def predict_tomorrow(city, data_set):
"""Use the saved model and today's forecasts to predict tomorrow's max temperature."""
path = model_path(city, data_set)
if not path.exists():
return None
saved = joblib.load(path)
sources = saved["sources"]
table = load_table(city, sources)
if tomorrow() not in table.index:
return None
row = table.loc[[tomorrow()], sources]
if row.isna().values.any():
return None # a source is missing for tomorrow
return round(float(saved["model"].predict(row)[0]), 1)
def print_scores(scores):
best = min(scores, key=scores.get)
for name, value in scores.items():
mark = " <-- best" if name == best else ""
print(f" {name:20s} {value:5.2f} °C{mark}")
if __name__ == "__main__":
for city in CITIES:
for data_set in SETS:
scores, days = evaluate(city, data_set)
print(f"\n{CITIES[city]['name']} / {data_set}: {days} complete days")
if scores is None:
print(f" not enough data yet (need {MIN_DAYS})")
else:
print_scores(scores)
app.py
"""The web app. python app.py then open http://127.0.0.1:5000"""
from flask import Flask, abort, redirect, render_template, request, url_for
from config import CITIES, SETS
from db import load_table
from ml import mae, predict_tomorrow, print_scores, train_and_save
from sources import tomorrow
app = Flask(__name__)
def source_errors(table, sources):
"""Average error of each source on the days where we know the measured value."""
known = table.dropna()
if known.empty:
return {}
return {source: round(mae(known[source], known["measured"]), 2) for source in sources}
@app.route("/")
def home():
return redirect(url_for("city_page", city="budapest"))
@app.route("/city/<city>")
def city_page(city):
if city not in CITIES:
abort(404)
data_set = request.args.get("set", "models")
if data_set not in SETS:
abort(404)
sources = SETS[data_set]
table = load_table(city, sources).tail(60)
chart = {
"days": list(table.index),
"measured": [None if v != v else v for v in table["measured"]], # NaN -> None (empty in the chart)
"sources": {s: [None if v != v else v for v in table[s]] for s in sources},
}
tomorrow_forecasts = table.loc[tomorrow()].drop("measured").to_dict() if tomorrow() in table.index else {}
return render_template(
"city.html", cities=CITIES, city=city, data_set=data_set, sets=SETS,
chart=chart, errors=source_errors(table, sources),
tomorrow=tomorrow(), tomorrow_forecasts=tomorrow_forecasts,
best_forecast=predict_tomorrow(city, data_set),
rows=table.iloc[::-1].head(15).round(1).to_dict("index"),
)
@app.route("/train/<city>/<data_set>", methods=["POST"])
def train(city, data_set):
if city not in CITIES or data_set not in SETS:
abort(404)
scores, days = train_and_save(city, data_set)
if scores:
print_scores(scores)
best = min(scores, key=scores.get) if scores else None
return render_template("train.html", cities=CITIES, city=city, data_set=data_set, sets=SETS,
scores=scores, days=days, best=best)
if __name__ == "__main__":
app.run(debug=True)
templates/city.html
{% extends "base.html" %}
{% block content %}
<h2>{{ cities[city].name }} – {{ data_set }}</h2>
<div class="box">
<b>Tomorrow ({{ tomorrow }}), max temperature</b><br>
{% for source, value in tomorrow_forecasts.items() %}
{{ source }}: {{ value }} °C
{% else %}
No forecasts for tomorrow yet. Run collect.py.
{% endfor %}
<br>
<b>Our best forecast: {{ best_forecast if best_forecast is not none else "no trained model yet" }}{% if best_forecast is not none %} °C{% endif %}</b>
<form method="post" action="{{ url_for('train', city=city, data_set=data_set) }}" style="display:inline">
<button>Train now</button>
</form>
</div>
<canvas id="chart" height="110"></canvas>
<h3>Average error of each source (last 60 days)</h3>
<table>
<tr>{% for source in errors %}<th>{{ source }}</th>{% endfor %}</tr>
<tr>{% for value in errors.values() %}<td>{{ value }} °C</td>{% endfor %}</tr>
</table>
<h3>Latest days</h3>
<table>
<tr><th>day</th>{% for source in sets[data_set] %}<th>{{ source }}</th>{% endfor %}<th>measured</th></tr>
{% for day, row in rows.items() %}
<tr><td>{{ day }}</td>
{% for source in sets[data_set] %}<td>{{ row[source] if row[source] == row[source] else "" }}</td>{% endfor %}
<td><b>{{ row.measured if row.measured == row.measured else "" }}</b></td></tr>
{% endfor %}
</table>
<script>
const chart = {{ chart | tojson }};
const datasets = [{ label: "measured", data: chart.measured, borderColor: "black", borderWidth: 3 }];
const colors = ["#1f77b4", "#ff7f0e", "#2ca02c"];
Object.entries(chart.sources).forEach(([name, values], i) => {
datasets.push({ label: name, data: values, borderColor: colors[i], borderWidth: 1 });
});
new Chart(document.getElementById("chart"), {
type: "line",
data: { labels: chart.days, datasets: datasets },
options: { spanGaps: false, pointRadius: 1 },
});
</script>
{% endblock %}
templates/train.html is complete in step 8. db.py, sources.py, collect.py, load_history.py, check_data.py and
templates/base.html did not change.
Check
- ☐
python ml.pyshows acombinedline for both cities. It is better than every single model and close to the best of the three regressors. - ☐ After Train now, the files
models/budapest_models.joblibandmodels/eger_models.joblibexist. - ☐ Our best forecast is a sensible number, near the three forecasts above it.
- ☐ Opening
/train/eger/modelsin the address bar gives Method Not Allowed; the button works. - ☐ Press Train now on the
providersset: "Not enough data" (until you have 30 complete days).
Results so far
Error on the test days (newest 20%), in °C. Lower is better. Your numbers will differ a little.
| ecmwf | gfs | icon | average | linear | random forest | neural net | combined | |
|---|---|---|---|---|---|---|---|---|
| Budapest | 0.64 | 1.18 | 0.70 | 0.67 | 0.58 | 0.73 | 0.59 | 0.59 |
| Eger | 0.82 | 1.56 | 1.28 | 0.93 | 0.68 | 0.82 | 0.69 | 0.67 |
The models cut the error of the best weather model by 10–20%, without any physics, just by learning from the past. The combined model is never the worst and often the best: that is why we use it on the page.
Extra
- Look inside the saved model: load
models/eger_models.joblibwithjoblib.load()and printsaved["model"].final_estimator_.coef_. Is it the same as in step 1? (It was trained on all days now, not only on 80%.) - Replace the final
LinearRegression()withRidge()or with a smallMLPRegressor. Better or worse? - Show also the date and time of the last training on the page (save it in the joblib file).
If something goes wrong
| Problem | Reason |
|---|---|
Method Not Allowed |
You opened /train/... in the address bar (GET). Use the button. |
NameError: name 'mae' is not defined in app.py |
You deleted mae() but did not import it from ml. See step 7. |
BuildError: Could not build url for endpoint 'train' |
The template has the button (step 9), but app.py has no train() route yet (step 8). |
| Our best forecast: no trained model yet | Press Train now first, or no forecasts for tomorrow: run collect.py. |
| The button "does nothing" | Training takes 5–30 seconds. Wait, and look at the terminal. |
The feature names should match those that were passed during fit |
The saved model was trained with other inputs (you changed the sources). Press Train now again, or delete the models folder. |