How to Build Your First ML Model — Step-by-Step Tutorial

Training a model takes three lines of Python. Knowing whether the model is any good takes the rest of the afternoon, and that part is what separates a tutorial copy-paste from actual machine learning. This walkthrough builds a house price predictor with pandas and scikit-learn, from a blank file to a saved model, and spends as much time on honest evaluation as on training. If you want the theory first, the explainer on what machine learning is covers it in plain language.

What you are building, and why house prices

The project is a regression model: it takes a row of numbers describing a neighborhood and returns one more number, the typical house value there. Regression predicts a quantity on a continuous scale. Its sibling, classification, predicts a label such as "spam" or "not spam."

House prices are a good first problem for two reasons. The inputs are things you can reason about (income, house age, location), so you can tell when a result looks wrong. And the target is a single number, which keeps the evaluation arithmetic simple enough to do by hand.

Think of the model as a student preparing for an exam. The training data is the stack of practice problems with answer keys. The test data is the real exam, with questions the student has never seen. A student who memorizes the answer key scores perfectly on practice and badly on the exam, and a model can do exactly the same thing. Everything in this tutorial is built around not fooling yourself in that way.

Why this tutorial does not use the Boston dataset

Older versions of this tutorial, including this one, used the Boston Housing dataset. It no longer belongs in a beginner course. scikit-learn deprecated load_boston in version 1.0 and scheduled its removal for 1.2. The library's own documentation explains why: the dataset's authors engineered a variable called "B" on the assumption that racial self-segregation had a positive effect on house prices, and the research behind it did not adequately demonstrate that assumption. The maintainers say they strongly discourage using it except to study ethical problems in data science.

The documentation points to two replacements, the California housing dataset and the Ames housing dataset. This tutorial uses California housing because it ships with scikit-learn and has no missing values, so you can focus on the modeling workflow. Code written for load_boston will fail on any current release.

The California dataset holds 20,640 rows and eight numeric features. Each row is a census block group from the 1990 U.S. census, the smallest geographic unit for which the Census Bureau publishes sample data, typically 600 to 3,000 people. The target is the median house value in that block group, expressed in units of $100,000 and ranging from 0.15 to 5. The data comes from StatLib and was described by Pace and Barry in a 1997 paper on sparse spatial autoregressions.

Feature	What it measures
MedInc	Median income in the block group
HouseAge	Median house age in the block group
AveRooms	Average rooms per household
AveBedrms	Average bedrooms per household
Population	Block group population
AveOccup	Average household members
Latitude	Block group latitude
Longitude	Block group longitude

Feature definitions from the scikit-learn documentation for the California housing dataset.

Step 1: Set up a clean environment

Install the libraries inside a virtual environment so this project cannot break another one. A virtual environment is a private folder of packages tied to a single project.

python -m venv ml-env
source ml-env/bin/activate        # Windows: ml-env\Scripts\activate
pip install pandas numpy scikit-learn matplotlib joblib
python -c "import sklearn; print(sklearn.__version__)"

Note the version that prints. A few functions used below, such as root_mean_squared_error, were added in scikit-learn 1.4. If your version is older, upgrade it with pip install --upgrade scikit-learn, or compute the square root of the mean squared error yourself with np.sqrt. The model-persistence section later explains why version numbers matter even more once you save a model.

Step 2: Load the data

import pandas as pd
from sklearn.datasets import fetch_california_housing

housing = fetch_california_housing(as_frame=True)
df = housing.frame            # features plus the target column

print(df.shape)               # (20640, 9): eight features and one target
print(df.head())
print(df.isna().sum())        # missing values per column

With as_frame=True, scikit-learn hands back a pandas DataFrame whose last column, MedHouseVal, is the target. The first call downloads the file and caches it in ~/scikit_learn_data, so it needs an internet connection once.

Real projects rarely start this clean. Most of your own datasets will arrive with gaps, and the guide to handling missing data covers the standard imputation choices. Here the check simply confirms that the documentation was right and every column is complete.

Step 3: Look at the data before you model it

Spend ten minutes on exploration before training anything. Models do not warn you when the input is nonsense, so you have to.

print(df.describe().T)                       # count, mean, min, max per column

import matplotlib.pyplot as plt
df["MedHouseVal"].hist(bins=50)
plt.xlabel("Median house value ($100,000)")
plt.ylabel("Block groups")
plt.show()

print(df.corr()["MedHouseVal"].sort_values(ascending=False))

Three things are worth looking for. First, the range of the target: it tops out at 5.0, or $500,000, so a model trained here cannot learn about anything pricier. A ceiling in the data becomes a ceiling in the predictions. Second, extreme values: the dataset documentation warns that AveRooms and AveBedrms can be unusually large in areas with few households and many empty houses, such as vacation resorts. Odd values like these pull averages around, a problem the explainer on why outliers can change averages walks through. Third, the correlations.

A correlation table tells you which columns move together with the target, and nothing more. If median income correlates strongly with house value, the model will lean on it, but that does not mean raising incomes raises prices. The distinction is the subject of how correlation differs from causation, and it matters the moment someone wants to act on a model's output.

Step 4: Split the data into training and test sets

from sklearn.model_selection import train_test_split

X = df.drop(columns="MedHouseVal")    # features: what the model sees
y = df["MedHouseVal"]                 # target: what it must predict

X_train, X_test, y_train, y_test = train_test_split(
    X, y, test_size=0.2, random_state=42
)

print(len(X_train), len(X_test))      # 16512 4128

The 20% test share of 20,640 rows leaves 4,128 for testing and 16,512 for training. The random_state argument fixes the shuffle so that you and anyone running your code get the same split. scikit-learn's guidance on randomness is that an integer seed gives identical results on every call, while omitting it gives a different split each run, which makes results impossible to compare. Do not try seeds until one produces a flattering score. That is a quiet way of training on your test set.

The reason for splitting at all comes straight from the scikit-learn documentation: learning the parameters of a prediction function and testing it on the same data is a methodological mistake, because a model that merely repeated the labels it had seen would score perfectly and predict nothing useful. The full argument is in why training and test data must be separated.

Data set	Used for	How often you may look at it
Training	Fitting the model's parameters	Constantly
Validation or cross-validation folds	Comparing models and tuning settings	Often
Test	One final, unbiased score	Once, at the end

Step 5: Build a baseline before building a model

Before any real algorithm, build the dumbest possible predictor and measure it. A baseline tells you what "no skill" looks like, so you can recognize skill when you see it.

from sklearn.dummy import DummyRegressor
from sklearn.metrics import root_mean_squared_error, mean_absolute_error, r2_score

def report(name, model, X_eval, y_eval):
    pred = model.predict(X_eval)
    print(f"{name:<22} RMSE={root_mean_squared_error(y_eval, pred):.3f}  "
          f"MAE={mean_absolute_error(y_eval, pred):.3f}  "
          f"R2={r2_score(y_eval, pred):.3f}")

baseline = DummyRegressor(strategy="mean").fit(X_train, y_train)
report("Baseline (mean)", baseline, X_test, y_test)

DummyRegressor with the default strategy="mean" ignores every feature and always predicts the average target it saw in training. Its documentation carries a blunt warning, "Do not use it for real problems," because it exists only as a yardstick. Guessing the average is like estimating the height of every stranger you meet as the national average: right on average, useless for any individual.

A constant model that predicts the average gets an R² of exactly 0.0 on the data it was fitted to. On a held-out test set the score is usually close to zero and can dip slightly below it, since the test set's average will not match the training average exactly. Anything your real model scores must be judged against this floor.

Step 6: Train a linear regression

from sklearn.linear_model import LinearRegression

model = LinearRegression()
model.fit(X_train, y_train)

for name, coef in zip(X.columns, model.coef_):
    print(f"{name:<12} {coef:>9.4f}")
print("Intercept", round(model.intercept_, 4))

That fit call is the entire "training" step. Linear regression looks for one weight per feature, plus a constant, such that the weighted sum of the features lands as close as possible to the true price across the training rows. "As close as possible" means it minimizes the sum of squared errors. Picture a mixing desk with eight sliders: fitting is the process of nudging each slider until the combined output best matches the recording.

The printed coefficients are readable. Each one says how much the predicted value (in units of $100,000) changes when that feature rises by one unit and the others stay fixed. A positive coefficient on MedInc means richer block groups get higher predicted prices. Check that the signs make sense. A coefficient that contradicts common sense is often your first clue about a data problem, or about two features carrying overlapping information.

Other algorithms learn differently. Decision trees split the data into regions, and gradient methods adjust parameters step by step to shrink the error, a process described in how gradient descent finds better answers. The broader family tree is mapped in ML algorithms explained.

Step 7: Evaluate on data the model has not seen

report("Linear regression", model, X_test, y_test)

You now have the model's honest score next to the baseline's. Each metric answers a slightly different question.

Metric	What it measures	Units	Notes
MSE (mean squared error)	Average of squared errors	Target units squared	Punishes large misses heavily
RMSE (root mean squared error)	Square root of MSE	Same as target	Readable as a typical miss, weighted toward big ones
MAE (mean absolute error)	Average of absolute errors	Same as target	Treats all misses equally
R² (coefficient of determination)	Share of variance explained relative to the mean-only model	Unitless	1.0 is perfect, 0.0 matches the baseline, negative is worse than the baseline

Because the target is in units of $100,000, an RMSE of 0.7 would mean a typical miss on the order of $70,000. That arithmetic is the reason to translate every error back into dollars before deciding whether a model is good enough.

R² deserves a careful reading. scikit-learn's documentation says the best possible score is 1.0 and that it can be negative, "because the model can be arbitrarily worse." For example, r2_score([1, 2, 3], [3, 2, 1]) returns -3.0. So R² is not a percentage accuracy, and a value of 0.6 does not mean 60% of predictions are right. It says that the model removes about 60% of the squared error that the mean-only guess would have made. That is why you always report it beside a baseline.

No single number settles whether a result is acceptable. What counts as a good R² or RMSE depends on the stakes, the cost of errors, and what alternatives exist. An error of $70,000 is tolerable for a rough neighborhood comparison and unacceptable for pricing a mortgage. Decide what level of error your use case can bear before looking at the score, not after.

Step 8: Look at the errors, not just the score

import numpy as np
import matplotlib.pyplot as plt

pred = model.predict(X_test)

fig, axes = plt.subplots(1, 2, figsize=(11, 4))

axes[0].scatter(y_test, pred, s=4, alpha=0.3)
lim = [0, max(y_test.max(), pred.max())]
axes[0].plot(lim, lim, "r--")                 # perfect predictions lie on this line
axes[0].set_xlabel("Actual ($100,000)")
axes[0].set_ylabel("Predicted ($100,000)")

residuals = y_test - pred
axes[1].scatter(pred, residuals, s=4, alpha=0.3)
axes[1].axhline(0, color="r", linestyle="--")
axes[1].set_xlabel("Predicted ($100,000)")
axes[1].set_ylabel("Residual (actual minus predicted)")

plt.tight_layout()
plt.show()

A single score hides where the model fails. The left panel plots predictions against truth; points hugging the red line are good predictions. The right panel plots the residuals, which are the actual values minus the predictions. In a healthy model these scatter randomly around zero. A visible curve, a funnel that widens at high prices, or a hard edge at the top of the range signals structure the model is missing.

With this dataset you will likely see a horizontal band of points along the top of the actual-value axis, where the target hits its 5.0 ceiling. The model has no way to represent "more than $500,000," so those rows pile up as systematic errors. Reading plots like this is a habit worth building early. It is how you find out what to fix, which is more useful than knowing only that something is wrong.

Step 9: Try stronger models and compare fairly

A straight line cannot capture the fact that location matters in a curved, patchy way. Two blocks with identical incomes can sit on opposite sides of a city with very different prices. Tree-based models handle this naturally, because they split the map into regions.

from sklearn.ensemble import RandomForestRegressor, HistGradientBoostingRegressor

forest = RandomForestRegressor(
    n_estimators=200, min_samples_leaf=5, random_state=42, n_jobs=-1
).fit(X_train, y_train)

hgb = HistGradientBoostingRegressor(random_state=42).fit(X_train, y_train)

report("Random forest", forest, X_test, y_test)
report("Hist. gradient boosting", hgb, X_test, y_test)

A random forest trains many decision trees on random subsets of the rows and features, then averages their answers. Averaging cancels out the quirks of individual trees, the way polling many people beats asking one. How random forests reduce error covers the mechanism, and a deeper treatment of the gradient-boosting family, including the popular library, is in XGBoost explained.

scikit-learn publishes a benchmark of exactly this comparison on exactly this dataset. In its example gallery, a random forest with min_samples_leaf=5 and a histogram gradient boosting model with max_leaf_nodes=15 are evaluated on California housing with four-fold cross-validation. Both improve as trees are added until they plateau, and the random forest plateaus earlier without reaching the best boosted model's test score. The page itself notes that results vary across runs and machines, so treat this as a pattern to check on your own hardware.

Model	Strengths	Weaknesses
Linear regression	Fast, transparent coefficients	Cannot bend; struggles with interactions such as location
Random forest	Handles nonlinearity, little tuning needed	Larger files, slower predictions
Histogram gradient boosting	Strong accuracy on tabular data, fast on large rows	More settings to tune

Notice that the table recommends no winner outright. A transparent model that is slightly less accurate is sometimes the right choice, for example when a person has to explain each prediction to a customer. Accuracy is one requirement among several.

Step 10: Tune and compare with cross-validation, keeping the test set sealed

Now comes the trap. If you try ten settings and keep whichever scores best on the test set, the test score stops measuring generalization. The scikit-learn documentation describes the risk directly: there is still a risk of overfitting on the test set because parameters can be tweaked until the estimator performs optimally, and then knowledge about the test set leaks into the model.

The fix is cross-validation on the training data. In k-fold cross-validation the training set is split into k parts. The model trains on k-1 of them, is validated on the remaining part, and the process repeats so that every part serves once as the validation set. The reported performance is the average across folds.

from sklearn.model_selection import KFold, cross_val_score, GridSearchCV

cv = KFold(n_splits=5, shuffle=True, random_state=0)

scores = cross_val_score(
    hgb, X_train, y_train, cv=cv, scoring="neg_root_mean_squared_error"
)
rmse_folds = -scores                          # scorers return negated errors
print(rmse_folds.round(3), rmse_folds.mean().round(3), rmse_folds.std().round(3))

param_grid = {
    "learning_rate": [0.05, 0.1, 0.2],
    "max_leaf_nodes": [15, 31, 63],
    "max_iter": [100, 300],
}
search = GridSearchCV(
    HistGradientBoostingRegressor(random_state=42),
    param_grid, cv=cv, scoring="neg_root_mean_squared_error", n_jobs=-1,
)
search.fit(X_train, y_train)
print(search.best_params_, -search.best_score_)

best = search.best_estimator_
report("Tuned HGB (test, once)", best, X_test, y_test)   # the single final look

The minus signs appear because scikit-learn scorers follow the convention that higher is better, so error metrics are exposed as negated values, such as neg_root_mean_squared_error. Flip the sign to recover a normal RMSE.

The spread across folds is as informative as the average. If five folds give RMSE values of 0.48, 0.49, 0.47, 0.50, and 0.48 (hypothetical numbers), the estimate is stable. If they range from 0.3 to 0.9, the model is very sensitive to which rows it sees, and you should distrust any single split.

Grid search tries every combination, here 3 × 3 × 2 = 18 settings, each fit five times, for 90 fits in total. That multiplication is why grids get expensive fast. Once the grid grows beyond a few dozen cells, random sampling of settings is the usual compromise.

Step 11: Keep preprocessing inside a pipeline

Some models need their inputs on a common scale. Linear models with regularization are one example. The tempting shortcut is to scale the whole dataset first and split afterwards, and it is wrong. This is data leakage: information from rows the model should not have seen, such as the test set's mean and spread, contaminates training.

scikit-learn's documentation on common pitfalls defines leakage as using information that would not be available at prediction time when building the model, and gives a rule of thumb: never call fit on the test data. Its demonstration is striking. With 200 samples, 10,000 random features, and random labels, accuracy should sit near 0.5 because there is nothing to learn. Selecting features on all the data before splitting produced 0.76. Doing it correctly gave 0.5. The model had found a signal that was not there, purely through leakage.

from sklearn.pipeline import make_pipeline
from sklearn.preprocessing import StandardScaler
from sklearn.linear_model import Ridge

pipe = make_pipeline(StandardScaler(), Ridge(alpha=1.0))

scores = cross_val_score(pipe, X_train, y_train, cv=cv,
                         scoring="neg_root_mean_squared_error")
print((-scores).mean().round(3))

pipe.fit(X_train, y_train)
report("Scaled ridge", pipe, X_test, y_test)

A pipeline chains the steps so that, inside every cross-validation fold, the scaler learns its mean and spread from the training portion only and merely applies them to the validation portion. The documentation notes that the leakage risk applies to almost all transformers, including StandardScaler, SimpleImputer, and PCA. Wrapping every transformation in a pipeline is the simplest habit that prevents it. The broader consequences for published results are covered in why data leakage breaks AI evaluations, and the cleanup work that feeds such steps is the subject of what feature engineering means.

Step 12: Spot overfitting by comparing train and test scores

report("Forest on TRAIN", forest, X_train, y_train)
report("Forest on TEST", forest, X_test, y_test)

Flexible models can fit training data far more closely than they fit new data. If the train score is excellent and the test score is much worse, the model has memorized quirks of the training rows. That is overfitting, and the trade-off behind it is laid out in overfitting and underfitting explained. If both scores are poor, the model is underfitting: too simple, or fed uninformative features.

For a forest, the usual levers are min_samples_leaf (larger values smooth the model), max_depth (shallower trees are simpler), and the number of trees. For a linear model, regularization strength is the lever. Move one lever at a time and watch the cross-validated score, never the test score.

Step 13: Save the model and make predictions

import joblib

joblib.dump(best, "housing_model.joblib")

# Later, in another script or session
loaded = joblib.load("housing_model.joblib")

new_block = pd.DataFrame([{
    "MedInc": 5.0,        # $50,000 median income (the units are tens of thousands)
    "HouseAge": 20.0,
    "AveRooms": 6.0,
    "AveBedrms": 1.0,
    "Population": 1200.0,
    "AveOccup": 3.0,
    "Latitude": 37.8,
    "Longitude": -122.3,
}])
print(loaded.predict(new_block)[0] * 100_000)

Passing a DataFrame with the same column names and order as the training data avoids the most common prediction bug, a silent column mix-up. The values above are made up for illustration, and the output is a prediction for a 1990 block group, not a valuation of any real property today.

Saving carries two warnings that most tutorials skip. The scikit-learn documentation states that pickle, and joblib and cloudpickle by extension, have documented security vulnerabilities by design: never load a file from an untrusted source, just as you would never run untrusted code. Its alternatives are ONNX, described as the most secure option though not every model is supported, and the skops format, which loads only trusted types.

Second, the documentation says there is no supported way to load a model trained with a different version of scikit-learn. Results may be unexpected or the process may crash. The practical advice is to pin dependency versions, use containers, and store metadata next to the model: the training data snapshot, the training code, the library versions, and the cross-validation score. The same discipline underlies production machine learning generally, introduced in MLOps for beginners.

Step 14: Serve predictions through a small API

from flask import Flask, request, jsonify
import joblib
import pandas as pd

app = Flask(__name__)
model = joblib.load("housing_model.joblib")       # a file you created yourself
FEATURES = ["MedInc", "HouseAge", "AveRooms", "AveBedrms",
            "Population", "AveOccup", "Latitude", "Longitude"]

@app.post("/predict")
def predict():
    payload = request.get_json(silent=True) or {}
    missing = [f for f in FEATURES if f not in payload]
    if missing:
        return jsonify(error=f"missing fields: {missing}"), 400
    row = pd.DataFrame([{f: float(payload[f]) for f in FEATURES}])
    value = float(model.predict(row)[0])
    return jsonify(predicted_value_usd=round(value * 100_000))

if __name__ == "__main__":
    app.run(port=5000)                             # development server only

Test it from a terminal:

curl -X POST http://localhost:5000/predict \
  -H "Content-Type: application/json" \
  -d '{"MedInc":5.0,"HouseAge":20,"AveRooms":6,"AveBedrms":1,"Population":1200,"AveOccup":3,"Latitude":37.8,"Longitude":-122.3}'

The built-in Flask server is for development. A real deployment sits behind a production server, validates ranges as well as presence, and logs inputs and outputs so you can notice when the world drifts away from the training data, a failure mode described in what model drift means in production.

When the score is bad: a debugging checklist

A disappointing score has a short list of usual causes. Work through them in order, because the early items are cheap to check and common.

Symptom	Likely cause	What to check
Test score near the baseline	Weak features or a bug	Print a few predictions; confirm the target and features line up row for row
Train score high, test score low	Overfitting	Simpler model, regularization, more data, cross-validation
Both scores low	Underfitting	More expressive model, better features
Score too good to be true	Leakage	Anything derived from the target; preprocessing fit before the split
Score drops sharply on new data	Distribution shift	Compare feature ranges between training and new data
Folds disagree wildly	Small or uneven data	More rows, repeated cross-validation, stratified or grouped splits

The "too good to be true" row is the one beginners meet least often and regret most. If a first model scores near perfection, assume a mistake until proven otherwise. Typical culprits are a column computed from the target, duplicated rows landing on both sides of the split, or a time-ordered dataset shuffled randomly so that the model peeks at the future.

For that last case, the order of the rows matters. If you are predicting next month from past months, a random split lets the model train on data from after the period it is tested on. Split by time instead: train on the earlier rows and test on the later ones.

What this model cannot do

It helps to be clear about the limits before showing the result to anyone. This model was trained on 1990 census data for California. It knows nothing about the 2020s housing market, other states, interest rates, or the condition of any individual house. Its unit of prediction is a neighborhood, so applying it to one specific home asks it to answer a question it was never trained on.

It also inherits whatever biases the data holds. The Boston dataset is the cautionary tale: a variable that looked like a helpful feature encoded a discriminatory assumption, and generations of students trained on it without asking. Before using any dataset, ask who collected it, for what purpose, and which columns could act as stand-ins for sensitive attributes such as race. Location features can do this, since neighborhoods and demographics are entangled.

Finally, a model that predicts well does not explain why prices are what they are. Treat its outputs as estimates with error bars, not as facts.

Common misconceptions about first models

"Training is the hard part." In this tutorial it was one line. Data preparation, honest evaluation, and monitoring consume most of the time in real projects.

"A higher R² always means a better model." Not if it came from leakage, a lucky split, or a test set you tuned against. A score is only as trustworthy as the procedure that produced it.

"Accuracy is the metric for everything." Accuracy is a classification metric, and even there it can mislead. A model that always predicts the majority class can look excellent on imbalanced data, which is explained in why 99% accuracy can be useless. Regression uses error measures like RMSE and MAE.

"I should jump straight to deep learning." For tables of numbers, tree-based models are a strong, quick default, and they are what you just built. Deep learning frameworks earn their complexity on images, audio, and text. When you get there, the comparison of PyTorch, TensorFlow, and JAX is a reasonable next stop, and the overview of Python ML libraries places scikit-learn in that wider landscape.

"The numbers in a tutorial are the numbers I should get." Your scores depend on the library version, the split, and the random seed. Compare your model to your baseline, not to a screenshot.

The complete script

import numpy as np
import pandas as pd
import joblib
from sklearn.datasets import fetch_california_housing
from sklearn.model_selection import train_test_split, KFold, GridSearchCV
from sklearn.dummy import DummyRegressor
from sklearn.ensemble import HistGradientBoostingRegressor
from sklearn.metrics import root_mean_squared_error, mean_absolute_error, r2_score

# 1. Load
housing = fetch_california_housing(as_frame=True)
df = housing.frame
X = df.drop(columns="MedHouseVal")
y = df["MedHouseVal"]

# 2. Split once; keep the test set sealed
X_train, X_test, y_train, y_test = train_test_split(
    X, y, test_size=0.2, random_state=42
)

def report(name, model, X_eval, y_eval):
    pred = model.predict(X_eval)
    print(f"{name:<24} RMSE={root_mean_squared_error(y_eval, pred):.3f}  "
          f"MAE={mean_absolute_error(y_eval, pred):.3f}  "
          f"R2={r2_score(y_eval, pred):.3f}")

# 3. Baseline
baseline = DummyRegressor(strategy="mean").fit(X_train, y_train)
report("Baseline (mean)", baseline, X_test, y_test)

# 4. Tune with cross-validation on the training data only
cv = KFold(n_splits=5, shuffle=True, random_state=0)
search = GridSearchCV(
    HistGradientBoostingRegressor(random_state=42),
    {"learning_rate": [0.05, 0.1, 0.2],
     "max_leaf_nodes": [15, 31, 63],
     "max_iter": [100, 300]},
    cv=cv, scoring="neg_root_mean_squared_error", n_jobs=-1,
).fit(X_train, y_train)
print("Best settings:", search.best_params_)

# 5. One final look at the test set
best = search.best_estimator_
report("Tuned model (test)", best, X_test, y_test)

# 6. Save
joblib.dump(best, "housing_model.joblib")

FAQ

Q: Why not use the Boston Housing dataset, as many older tutorials do? A: scikit-learn deprecated load_boston in version 1.0 and scheduled its removal for 1.2, so the code no longer runs on current releases. The documentation also describes an ethical problem: the dataset includes a variable built on the assumption that racial self-segregation raised house prices. The California housing dataset is the library's suggested replacement.

Q: What is a good R² score? A: It depends on the problem, so there is no universal threshold. R² compares your model with a mean-only guess: 1.0 is perfect, 0.0 matches the baseline, and negative values are worse than the baseline. Decide what error is tolerable for your use case, translate RMSE back into real units, and judge the model against a baseline and against the cost of being wrong.

Q: Why split the data instead of testing on the training data? A: A model scored on the rows it learned from can look perfect while having memorized them. The scikit-learn documentation calls testing on the training data a methodological mistake. A held-out test set shows how the model performs on rows it has never seen, which is what matters in practice.

Q: What is the difference between the test set and cross-validation? A: Cross-validation repeatedly splits the training data to compare models and tune settings, so you can iterate freely. The test set is held back for one final evaluation after all choices are made. Looking at it repeatedly while tuning leaks information and makes the final score optimistic.

Q: Can I use my own dataset instead? A: Yes. Load it with pd.read_csv, choose a numeric target column, and use the same workflow. Before modeling, check for missing values, outliers, duplicated rows, and columns that could leak the answer. If your rows are ordered in time, split by time rather than randomly.

Q: Should I try deep learning next? A: Not necessarily. For tabular data, tree-based models are a strong default and are quick to train and inspect. Deep learning tends to pay off on images, audio, and text. Get comfortable with evaluation, baselines, and pipelines first, since those skills transfer to every model type.

Q: Is it safe to share a saved joblib file? A: Treat these files like executable code. scikit-learn's documentation warns never to load a pickle-based file from an untrusted source, and recommends options such as ONNX or skops when you need to exchange models. Also keep the library versions the same between training and loading.

The takeaway

The first model is the easy part. Anyone can call fit and print a number, and the number will always look like an answer. What makes it an answer is the scaffolding around it: a baseline to beat, a test set nobody touched, cross-validation for the choices in between, and a pipeline that keeps preprocessing from leaking the future into the past.

If you remember one habit from this walkthrough, make it this one: before celebrating any score, ask what the model would have scored if it had known nothing, and ask whether anything in your process could have let it peek. The rest of machine learning, from simple regressions to the largest neural networks, is a longer version of those two questions.

Sources

• scikit-learn documentation, Real world datasets: California housing
• scikit-learn documentation, fetch_california_housing
• scikit-learn 1.1 documentation, load_boston (deprecated) and its ethical warning
• scikit-learn documentation, Cross-validation: evaluating estimator performance
• scikit-learn documentation, Common pitfalls and recommended practices
• scikit-learn documentation, Model persistence
• scikit-learn documentation, r2_score, root_mean_squared_error, and DummyRegressor
• scikit-learn example gallery, Comparing random forests and histogram gradient boosting models

Last verified: October 2026

---

Read next:

• How Decision Trees Split Data
• Random Forest Explained
• Best Machine Learning Courses 2026