Install and Set Up Scikit-Learn

Install Scikit-Learn and the essential Python data science libraries, create an isolated virtual environment, verify that everything works correctly, understand common installation problems, and learn the import patterns used throughout this course.

01. Installing the Full Stack

Scikit-Learn works as part of the wider Python data science ecosystem. Although sklearn provides the machine learning algorithms, real machine learning projects commonly use additional libraries for numerical computation, data manipulation, and visualization.

For this course, we will primarily work with Scikit-Learn, NumPy, Pandas, and Matplotlib.

pip install scikit-learn numpy pandas matplotlib

The command tells pip to download and install the required packages. If the packages are already installed, pip will normally report that the requirements are already satisfied or update them when appropriate.

02. Checking Python and pip

Before installing packages, it is useful to verify that Python and pip are available from your terminal. This can help identify problems where Python is not installed correctly or where the wrong Python installation is being used.

python --version
pip --version

You should see the installed Python version and information about the pip package manager. On some systems, the Python command may be python3 instead.

python3 --version
pip3 --version

03. Verifying the Installation

After installation, the next step is to verify that Python can successfully import the required libraries.

import sklearn
import numpy as np
import pandas as pd
import matplotlib

print("sklearn:", sklearn.**version**)
print("numpy:", np.**version**)
print("pandas:", pd.**version**)
print("matplotlib:", matplotlib.**version**)

print("All installed!")

If this program runs without an import error, the basic environment is working correctly.

04. Virtual Environment (Recommended)

A virtual environment creates an isolated Python environment for a project. Packages installed inside the environment are separated from packages installed in other projects.

This is particularly useful for machine learning because different projects may require different versions of libraries.

python -m venv venv

The command creates a directory named venv containing an isolated Python environment.

On Windows, activate it using:

venv\Scripts\activate

On macOS or Linux, use:

source venv/bin/activate

Once the environment is activated, install the project dependencies inside it.

pip install scikit-learn numpy pandas matplotlib

05. Activating and Deactivating the Environment

When you work on a project, activate its virtual environment before running or installing project dependencies.

On Windows:

venv\Scripts\activate

On macOS and Linux:

source venv/bin/activate

When you are finished working on the project, you can leave the virtual environment with:

deactivate

06. Standard Import Pattern

Once the environment is ready, machine learning programs typically import the specific classes and functions required for the task.

import numpy as np
import pandas as pd

from sklearn.model_selection import train_test_split
from sklearn.preprocessing import StandardScaler
from sklearn.linear_model import LinearRegression
from sklearn.metrics import mean_squared_error, accuracy_score

Notice that NumPy and Pandas use common aliases, np and pd. Scikit-Learn classes are usually imported directly from their specific modules.

07. Understanding sklearn Modules

Scikit-Learn is organized into different modules. Each module contains tools designed for a particular part of the machine learning workflow.

Understanding this organization makes it easier to search the documentation and find the appropriate tool for a particular machine learning task.

08. Testing a Simple sklearn Model

A successful import is useful, but we can perform a stronger test by actually creating and training a model.

import numpy as np
from sklearn.linear_model import LinearRegression

X = np.array([
[1],
[2],
[3],
[4]
])

y = np.array([
2,
4,
6,
8
])

model = LinearRegression()

model.fit(X, y)

prediction = model.predict([[5]])

print("Prediction:", prediction[0])

If the program runs successfully and produces a prediction, your environment is capable of executing a basic Scikit-Learn machine learning workflow.

09. Installing Packages with python -m pip

On systems with multiple Python installations, using python -m pip can make it clearer which Python interpreter is being used to run pip.

python -m pip install scikit-learn numpy pandas matplotlib

This tells Python to execute pip as a module. It is often useful when the command pip points to a different Python installation than the one you are using to run your program.

10. Checking Installed Packages

pip can display the packages installed in the current environment.

pip list

You can also inspect information about a particular package.

pip show scikit-learn

The command can display information such as the installed version, package location, and dependencies.

11. Common Installation Errors

Installation problems are common when setting up Python environments. Most errors can be diagnosed by checking the Python version, active environment, pip configuration, and package names.

If you receive an import error after installation, first check whether the environment where the package was installed is the same environment used to run the Python program.

12. Updating pip

Keeping pip reasonably up to date can help with package installation and dependency management.

python -m pip install --upgrade pip

After updating pip, you can install or update your project dependencies normally.

13. Creating a Requirements File

When a project uses several Python packages, it is useful to record its dependencies in a requirements.txt file. This makes it easier to recreate the environment on another computer.

numpy
pandas
matplotlib
scikit-learn

Another environment can install these dependencies using:

pip install -r requirements.txt

Requirements files are particularly useful when sharing projects with classmates, teammates, servers, or other development machines.

14. Complete Environment Verification

We can combine the checks from this lecture into one script that verifies the important libraries and performs a simple machine learning operation.

import sklearn
import numpy as np
import pandas as pd
import matplotlib

from sklearn.linear_model import LinearRegression

print("Scikit-Learn:", sklearn.**version**)
print("NumPy:", np.**version**)
print("Pandas:", pd.**version**)
print("Matplotlib:", matplotlib.**version**)

X = np.array([[1], [2], [3]])
y = np.array([2, 4, 6])

model = LinearRegression()
model.fit(X, y)

print("Test prediction:", model.predict([[4]])[0])
print("Environment is working!")

This test checks both package imports and a basic sklearn operation. If it runs without errors, your environment is ready for the upcoming machine learning lessons.

15. Best Practices for the Course

A clean environment saves time later. Many machine learning errors are caused not by the algorithm itself but by incorrect packages, incompatible versions, or running code in the wrong Python environment.

16. Key Points

import sklearn
import numpy as np
import pandas as pd
import matplotlib

print("sklearn:", sklearn.**version**)
print("numpy:", np.**version**)
print("pandas:", pd.**version**)
print("matplotlib:", matplotlib.**version**)

# Your code here

Understand the Machine Learning Workflow

A machine learning model is not created by simply calling one function. Scikit-Learn provides tools that allow you to follow a complete workflow: collect or load data, prepare it, split it into training and testing sets, choose a model, train it, make predictions, evaluate its performance, and improve it when necessary.

01. Understand the Machine Learning Workflow

A typical machine learning project follows a series of steps. Each step has a specific purpose and contributes to building a reliable model.

The general workflow can be represented as:

1. Collect or load data
2. Explore the data
3. Prepare the data
4. Split the data
5. Choose a model
6. Train the model
7. Make predictions
8. Evaluate the model
9. Improve the model

Scikit-Learn provides functions and classes for most of these steps. For example, train_test_split() can divide the data, while classes such as LinearRegression, KNeighborsClassifier, and DecisionTreeClassifier can be used to create machine learning models.

The important idea is that training a model is only one part of the machine learning workflow. A model must also be tested on data that it did not see during training.

02. Load the Required Libraries

Before building a machine learning model, import the libraries and tools that you need. Scikit-Learn contains many machine learning algorithms and utilities.

import numpy as np
import pandas as pd

from sklearn.model_selection import train_test_split
from sklearn.linear_model import LinearRegression
from sklearn.metrics import mean_squared_error, r2_score

Here, NumPy is useful for numerical operations, while Pandas is commonly used to load and work with tabular data.

From Scikit-Learn, we import train_test_split() for dividing the dataset, LinearRegression for creating a regression model, and evaluation functions for measuring how well the model performs.

You do not need to import every Scikit-Learn module in every project. Import only the tools required for the particular machine learning task.

03. Prepare Features and Targets

Machine learning models usually work with two main types of data: features and a target.

Features are the input values used by the model to make a prediction. The target is the value that the model is trying to predict.

For example, suppose we want to predict a student's exam score from the number of hours they studied.

hours = [[1], [2], [3], [4], [5], [6]]
scores = [35, 42, 50, 58, 67, 75]

Here, hours is the feature because it is used as the input to the model. The scores are the target values because they are what we want the model to predict.

Scikit-Learn normally represents features as a two-dimensional structure, even when there is only one feature. That is why the values are written as [[1], [2], [3]] rather than simply [1, 2, 3].

04. Split the Data

A model should not be evaluated only on the same data that it used for learning. Instead, divide the dataset into a training set and a testing set.

The training data is used to teach the model. The testing data is kept separate and is used later to check how well the model performs on unseen examples.

Scikit-Learn provides the train_test_split() function for this purpose.

from sklearn.model_selection import train_test_split

X_train, X_test, y_train, y_test = train_test_split(
hours,
scores,
test_size=0.2,
random_state=42
)

The function returns four values. X_train and y_train contain the training data, while X_test and y_test contain the testing data.

The test_size=0.2 argument means that approximately 20% of the available data is reserved for testing, while the remaining 80% is used for training.

The random_state=42 argument makes the split reproducible. This means that running the same code again will produce the same split.

05. Choose a Machine Learning Model

After preparing and splitting the data, choose an algorithm that matches the problem you are solving.

Different machine learning problems require different types of models.

Choosing the correct model depends on the type of problem, the dataset, the number of features, the amount of data, and the expected output.

For this example, we are predicting a numerical exam score, so regression is appropriate. We can use LinearRegression.

06. Create and Train the Model

Once a model has been selected, create an instance of the model and use the fit() method to train it.

from sklearn.linear_model import LinearRegression

model = LinearRegression()

model.fit(X_train, y_train)

The first line creates a LinearRegression object. The second line calls fit(), which allows the model to learn the relationship between the input features and the target values.

The model examines the training data and learns parameters that can later be used to make predictions.

In Scikit-Learn, the fit() method is one of the most important methods you will encounter. Many Scikit-Learn models follow the same general pattern:

model = SomeModel()

model.fit(X_train, y_train)

07. Make Predictions

After training, use the predict() method to generate predictions.

predictions = model.predict(X_test)

print(predictions)

The model receives the test features stored in X_test and produces predicted values.

For example, if X_test contains the number of hours studied by students that the model has never seen before, the model will use what it learned during training to estimate their exam scores.

The predicted values are stored in the predictions variable.

Prediction can also be performed on completely new data.

new_student = [[7]]

predicted_score = model.predict(new_student)

print(predicted_score)

Here, the model receives a new input representing seven hours of study and estimates the corresponding exam score.

08. Evaluate the Model

A prediction alone does not tell us whether a model is good. We need evaluation metrics to measure its performance.

For regression problems, common metrics include Mean Squared Error (MSE) and R² score.

from sklearn.metrics import mean_squared_error, r2_score

mse = mean_squared_error(y_test, predictions)
r2 = r2_score(y_test, predictions)

print(“MSE:”, mse)
print(“R²:”, r2)

mean_squared_error() measures the average squared difference between the actual values and predicted values. A lower MSE generally indicates that the predictions are closer to the actual values.

r2_score() measures how well the model explains the variation in the target values. A value closer to 1 generally indicates a stronger fit for a regression model.

The correct evaluation metric depends on the type of machine learning problem. Classification models use metrics such as accuracy, precision, recall, and F1-score, while regression models commonly use metrics such as MSE, MAE, and R².

09. Put the Complete Workflow Together

Now we can combine the major steps into one complete Scikit-Learn program.

from sklearn.model_selection import train_test_split
from sklearn.linear_model import LinearRegression
from sklearn.metrics import mean_squared_error, r2_score

hours = [[1], [2], [3], [4], [5], [6]]
scores = [35, 42, 50, 58, 67, 75]

X_train, X_test, y_train, y_test = train_test_split(
hours,
scores,
test_size=0.2,
random_state=42
)

model = LinearRegression()

model.fit(X_train, y_train)

predictions = model.predict(X_test)

mse = mean_squared_error(y_test, predictions)
r2 = r2_score(y_test, predictions)

print(“Predictions:”, predictions)
print(“MSE:”, mse)
print(“R²:”, r2)

This program demonstrates the complete basic workflow. First, the data is created. Then it is divided into training and testing sets. A regression model is created and trained. The trained model makes predictions on the test data, and the predictions are finally evaluated.

Learning this workflow is more important than memorizing individual algorithms because many Scikit-Learn models use a similar interface.

10. Improve the Model

If the model does not perform well, the workflow does not simply end. Machine learning is usually an iterative process.

You may need to improve the data, select different features, preprocess the values, choose another algorithm, tune hyperparameters, or collect more training data.

A simplified improvement cycle looks like this:

Prepare data
      ↓
Train model
      ↓
Evaluate model
      ↓
Analyze results
      ↓
Improve data or model
      ↓
Train again
      ↓
Evaluate again

For example, a model may perform poorly because the features are not useful, the data contains missing values, the model is too simple, or the model is overfitting the training data.

Scikit-Learn provides additional tools for preprocessing, feature selection, cross-validation, and hyperparameter tuning that can be introduced as you progress through machine learning.

11. The Scikit-Learn Estimator Pattern

One of the most useful things to understand about Scikit-Learn is that many algorithms follow a consistent interface.

A typical model follows this pattern:

model = Model()

model.fit(X_train, y_train)

predictions = model.predict(X_test)

This pattern makes Scikit-Learn easier to learn. Once you understand how fit() and predict() work, you can apply the same basic idea to many different algorithms.

Some models also provide methods such as score(), transform(), or fit_transform(). These methods become especially important when working with preprocessing tools and machine learning pipelines.

Work with Datasets in Scikit-Learn

Machine learning models need data to learn from. Before training a model, you need to know how to obtain a dataset, load it into Python, understand its structure, identify features and targets, and prepare it for the machine learning workflow. Scikit-Learn provides several built-in datasets for learning and experimentation, while Pandas makes it easy to load real-world datasets from files such as CSV.

01. What Is a Dataset?

A dataset is a collection of data that can be used to analyze a problem or train a machine learning model. A dataset usually contains multiple observations, where each observation represents one example.

For example, a student dataset might contain information about study hours, attendance, assignments, and exam scores.

Hours  Attendance  Assignments  Score
2      75          6            48
4      82          8            61
6      90          9            74
8      95          10           88

Each row represents one student, while each column represents a variable or feature.

If our goal is to predict the exam score, Hours, Attendance, and Assignments could be features, while Score would be the target.

02. Features and Target

Before training a machine learning model, separate the dataset into input features and the target value.

Features are the information that the model uses to make a prediction. The target is the value that the model is trying to predict.

import pandas as pd

data = pd.DataFrame({
“Hours”: [2, 4, 6, 8],
“Attendance”: [75, 82, 90, 95],
“Score”: [48, 61, 74, 88]
})

X = data[[“Hours”, “Attendance”]]
y = data[“Score”]

print(X)
print(y)

Here, X contains the input features, while y contains the target values.

Notice that X uses double square brackets because it contains multiple columns and therefore remains a two-dimensional DataFrame. The target y is a single Pandas Series.

This separation is an important step because Scikit-Learn expects the model to receive the features separately from the values it should learn to predict.

03. Use Built-in Scikit-Learn Datasets

Scikit-Learn provides several built-in datasets that are useful for learning machine learning concepts without downloading external files.

Some commonly used datasets include:

These datasets are particularly useful when learning because they allow you to practice machine learning without first finding and downloading a dataset yourself.

04. Load the Iris Dataset

The Iris dataset is one of the most commonly used beginner datasets in machine learning. It contains measurements of iris flowers and information about their species.

Scikit-Learn provides it through the load_iris() function.

from sklearn.datasets import load_iris

iris = load_iris()

print(iris)

The returned object contains several pieces of information about the dataset, including the input data, target values, feature names, and target names.

Instead of printing the entire dataset, you can access individual parts of it.

from sklearn.datasets import load_iris

iris = load_iris()

print(iris.data)
print(iris.target)
print(iris.feature_names)
print(iris.target_names)

iris.data contains the input features, while iris.target contains the numerical class labels.

iris.feature_names provides the names of the features, and iris.target_names tells us which classes the numerical target values represent.

05. Understand the Dataset Shape

Before training a model, it is useful to know how much data you have and how many features are available.

NumPy arrays and Pandas DataFrames provide the shape attribute for this purpose.

from sklearn.datasets import load_iris

iris = load_iris()

X = iris.data
y = iris.target

print(“Features shape:”, X.shape)
print(“Target shape:”, y.shape)

The shape is usually displayed as a pair such as (150, 4). The first number represents the number of observations, while the second represents the number of features.

For the Iris dataset, there are 150 examples and 4 input features.

06. Inspect the Data

Do not immediately train a model after loading a dataset. First inspect the data so that you understand what you are working with.

When using Pandas, common inspection methods include head(), info(), describe(), and shape.

import pandas as pd

data = pd.read_csv(“students.csv”)

print(data.head())
print(data.shape)
print(data.info())
print(data.describe())

head() displays the first few rows, which gives you a quick look at the data.

shape tells you how many rows and columns are present.

info() provides information about the columns, data types, and non-null values.

describe() generates statistical information for numerical columns, such as the mean, minimum, maximum, and quartiles.

These checks are useful because real-world datasets are rarely perfectly prepared for machine learning.

07. Load a CSV Dataset

Real machine learning projects often use datasets stored in files. One of the most common formats is CSV, which stands for Comma-Separated Values.

Pandas provides the read_csv() function for loading CSV files.

import pandas as pd

data = pd.read_csv(“students.csv”)

print(data.head())

The string "students.csv" represents the location of the CSV file. If the file is in the same folder as your Python program, you can use just the filename.

If the file is stored somewhere else, you can provide a file path.

data = pd.read_csv("data/students.csv")

After loading the file, data becomes a Pandas DataFrame that can be inspected and prepared for machine learning.

08. Select Features and Target from a CSV

Once the CSV file has been loaded, select the columns that will be used as features and identify the target column.

Suppose the CSV contains the following columns:

Hours,Attendance,Assignments,Score
2,75,6,48
4,82,8,61
6,90,9,74
8,95,10,88

If we want to predict Score, the other columns can be used as input features.

import pandas as pd

data = pd.read_csv(“students.csv”)

X = data[[“Hours”, “Attendance”, “Assignments”]]
y = data[“Score”]

print(X)
print(y)

The variable X now contains the information used to make predictions, while y contains the values the model should learn to predict.

09. Convert Data into NumPy Arrays

Scikit-Learn works naturally with NumPy arrays and Pandas DataFrames. In many situations, you can pass a DataFrame directly to a Scikit-Learn model.

For example:

import pandas as pd

data = pd.read_csv(“students.csv”)

X = data[[“Hours”, “Attendance”]]
y = data[“Score”]

print(type(X))
print(type(y))

You can also convert Pandas data into NumPy arrays when needed.

X_array = X.to_numpy()
y_array = y.to_numpy()

print(X_array)
print(y_array)

However, conversion is not always necessary. Most Scikit-Learn estimators can work directly with Pandas DataFrames and Series.

10. Load Different Built-in Datasets

Once you understand the Iris dataset, you can experiment with other datasets provided by Scikit-Learn.

For example, the digits dataset contains images represented as numerical features.

from sklearn.datasets import load_digits

digits = load_digits()

X = digits.data
y = digits.target

print(“Features:”, X.shape)
print(“Targets:”, y.shape)

You can also load the wine dataset:

from sklearn.datasets import load_wine

wine = load_wine()

X = wine.data
y = wine.target

print(“Features:”, X.shape)
print(“Targets:”, y.shape)

Experimenting with different datasets helps you understand that machine learning models can work with many types of numerical data and different prediction problems.

11. Loading Data Is Only the Beginning

Loading a dataset does not automatically make it ready for a machine learning model. Real datasets may contain missing values, duplicate rows, text values, categorical variables, extreme values, or features with very different scales.

A typical preparation process may look like this:

Load dataset
      ↓
Inspect dataset
      ↓
Handle missing values
      ↓
Select features
      ↓
Select target
      ↓
Encode categorical data
      ↓
Scale features when necessary
      ↓
Split into training and testing data
      ↓
Train the model

Each dataset may require a different preparation process. The goal is to convert raw data into a form that a machine learning algorithm can use effectively.

12. Build a Simple Dataset-to-Model Workflow

We can now combine data loading, feature selection, splitting, training, and prediction into one simple example.

import pandas as pd

from sklearn.model_selection import train_test_split
from sklearn.linear_model import LinearRegression

data = pd.read_csv(“students.csv”)

X = data[[“Hours”, “Attendance”]]
y = data[“Score”]

X_train, X_test, y_train, y_test = train_test_split(
X,
y,
test_size=0.2,
random_state=42
)

model = LinearRegression()

model.fit(X_train, y_train)

predictions = model.predict(X_test)

print(“Predictions:”, predictions)

This example demonstrates an important connection between data loading and the machine learning workflow. The dataset is first loaded using Pandas, then divided into features and a target, and finally passed into the Scikit-Learn training process.

Understanding this connection will make it much easier to work with larger real-world datasets later.

Train / Test Split

Learn why splitting your data is essential, how training and testing data work, and how to use train_test_split correctly to evaluate machine learning models and avoid misleading results.

01. Why Split Data?

If you train and evaluate a model using exactly the same data, the model may appear to perform extremely well because it has already seen those examples. This does not tell us how well the model will perform on completely new data.

This problem is closely related to overfitting. An overfitted model learns the training examples too closely, including patterns that may not generalize to unseen data.

By separating the dataset into training and testing portions, we can train the model on one portion and evaluate it on data that was kept unseen during training.

from sklearn.model_selection import train_test_split
from sklearn.datasets import load_iris

X, y = load_iris(return_X_y=True)

X_train, X_test, y_train, y_test = train_test_split(
X, y,
test_size=0.2,
random_state=42,
stratify=y
)

print("Train:", X_train.shape)
print("Test:", X_test.shape)

02. Training Data vs Testing Data

The training set is the portion of the dataset used to learn the relationships between features and targets. The testing set is kept separate until evaluation.

For example, if a dataset contains 1,000 samples and we use an 80/20 split, approximately 800 samples are used for training and 200 samples are reserved for testing.

03. Using train_test_split()

Scikit-Learn provides the train_test_split() function in sklearn.model_selection. It randomly divides one or more arrays into training and testing subsets.

from sklearn.model_selection import train_test_split

X_train, X_test, y_train, y_test = train_test_split(
X,
y,
test_size=0.2
)

print(X_train.shape)
print(X_test.shape)
print(y_train.shape)
print(y_test.shape)

When both X and y are supplied, Scikit-Learn keeps their corresponding rows aligned. This is extremely important because each feature row must remain associated with the correct target.

04. Understanding test_size

The test_size parameter controls how much data is placed into the testing set.

# 20% of the data for testing
X_train, X_test, y_train, y_test = train_test_split(
    X, y, test_size=0.2
)

# 30% of the data for testing

X_train, X_test, y_train, y_test = train_test_split(
X, y, test_size=0.3
)

# Exactly 100 samples for testing

X_train, X_test, y_train, y_test = train_test_split(
X, y, test_size=100
)

There is no universal test size that is correct for every project. The appropriate choice depends on the dataset size, problem, and evaluation strategy.

05. Understanding random_state

By default, the split is random. Running the same program multiple times can therefore produce different training and testing samples.

The random_state parameter allows you to control the randomness. Using the same value produces the same split each time, which makes experiments easier to reproduce.

from sklearn.model_selection import train_test_split

X_train1, X_test1, y_train1, y_test1 = train_test_split(
X, y,
test_size=0.2,
random_state=42
)

X_train2, X_test2, y_train2, y_test2 = train_test_split(
X, y,
test_size=0.2,
random_state=42
)

print((X_train1 == X_train2).all())

06. Understanding stratify

For classification problems, different classes may have different numbers of examples. A random split can sometimes produce slightly different class proportions between training and testing sets.

The stratify parameter helps preserve the class distribution when splitting the data.

from sklearn.datasets import load_iris
from sklearn.model_selection import train_test_split

X, y = load_iris(return_X_y=True)

X_train, X_test, y_train, y_test = train_test_split(
X,
y,
test_size=0.2,
random_state=42,
stratify=y
)

print("Training classes:", y_train)
print("Testing classes:", y_test)

07. Checking the Split

After splitting the data, it is good practice to check the shapes of all four resulting arrays. This confirms that the split happened as expected.

print("X_train:", X_train.shape)
print("X_test :", X_test.shape)
print("y_train:", y_train.shape)
print("y_test :", y_test.shape)

print("Training samples:", len(X_train))
print("Testing samples:", len(X_test))

For the Iris dataset, there are 150 samples. With test_size=0.2, the test set contains 30 samples and the training set contains 120 samples.

08. Splitting Multiple Arrays

One important advantage of train_test_split() is that it can split multiple related arrays at the same time. This is how we keep features and targets correctly matched.

from sklearn.model_selection import train_test_split

X_train, X_test, y_train, y_test = train_test_split(
X,
y,
test_size=0.2,
random_state=42
)

print("Training features:", X_train.shape)
print("Training targets:", y_train.shape)
print("Testing features:", X_test.shape)
print("Testing targets:", y_test.shape)

09. Train / Test Split with a CSV Dataset

The same technique works when the data comes from your own CSV file. First load the data using Pandas, separate the features and target, and then split them.

import pandas as pd
from sklearn.model_selection import train_test_split

df = pd.read_csv("data.csv")

X = df.drop("target", axis=1)
y = df["target"]

X_train, X_test, y_train, y_test = train_test_split(
X,
y,
test_size=0.2,
random_state=42,
stratify=y
)

print("Training data:", X_train.shape)
print("Testing data:", X_test.shape)

10. Why We Should Not Train on Test Data

The test set should represent data that the model has never seen during training. If test data is accidentally used during training, the evaluation can become overly optimistic.

This is one form of data leakage. Data leakage occurs when information that should be unavailable to the model during training finds its way into the training process.

11. Train / Test Split and Preprocessing

When preprocessing data, you should generally split the data first and then fit preprocessing transformations using the training data. The transformation can then be applied to the test data.

from sklearn.model_selection import train_test_split
from sklearn.preprocessing import StandardScaler

X_train, X_test, y_train, y_test = train_test_split(
X,
y,
test_size=0.2,
random_state=42
)

scaler = StandardScaler()

X_train = scaler.fit_transform(X_train)
X_test = scaler.transform(X_test)

12. Train / Test Split vs Validation Data

For simple projects, a train/test split may be enough. However, larger machine learning workflows often divide data into three sets: training, validation, and testing.

We will study validation and cross-validation in more detail later in the course.

13. Complete Train / Test Example

The following example combines the main concepts from this lecture into a complete workflow.

from sklearn.datasets import load_iris
from sklearn.model_selection import train_test_split

# Load data

X, y = load_iris(return_X_y=True)

# Split into training and testing sets

X_train, X_test, y_train, y_test = train_test_split(
X,
y,
test_size=0.2,
random_state=42,
stratify=y
)

# Display sizes

print("Training features:", X_train.shape)
print("Testing features:", X_test.shape)
print("Training targets:", y_train.shape)
print("Testing targets:", y_test.shape)

14. Common Mistakes

15. Key Points

from sklearn.datasets import load_iris
from sklearn.model_selection import train_test_split

X, y = load_iris(return_X_y=True)

# Split the data

# Your code here

# Print all four shapes

# X_train

# X_test

# y_train

# y_test

Feature Scaling and Normalization

Learn how to scale your data so all features contribute appropriately to model training, understand why different numerical ranges can affect machine learning algorithms, and apply common Scikit-Learn scaling techniques correctly.

01. Why Feature Scaling?

Machine learning datasets often contain features measured on completely different scales. For example, one feature might represent age from 0 to 100, while another feature represents salary from 10,000 to 1,000,000.

If an algorithm depends on distances, magnitudes, or numerical optimization, a feature with much larger values can have a disproportionate influence on the model.

Feature scaling transforms numerical columns to a more comparable scale while preserving the useful information contained in the data.

02. Standardization vs Normalization

Two common approaches to feature scaling are standardization and normalization. Although these terms are sometimes used interchangeably, they describe different transformations.

03. StandardScaler

StandardScaler is one of the most commonly used preprocessing tools in Scikit-Learn. It standardizes each feature using statistics calculated from the training data.

The transformation is based on the formula:

from sklearn.preprocessing import StandardScaler
from sklearn.datasets import load_iris
from sklearn.model_selection import train_test_split

X, y = load_iris(return_X_y=True)

X_train, X_test, y_train, y_test = train_test_split(
X, y,
test_size=0.2,
random_state=42
)

scaler = StandardScaler()

X_train_s = scaler.fit_transform(X_train)
X_test_s  = scaler.transform(X_test)

print("Mean:", X_train_s.mean(axis=0).round(2))
print("Std: ", X_train_s.std(axis=0).round(2))

Notice that the training data uses fit_transform(), while the test data uses only transform(). This distinction is extremely important.

04. Understanding fit(), transform(), and fit_transform()

Scikit-Learn preprocessing objects follow the same API pattern used throughout the library.

scaler = StandardScaler()

# Learn statistics from training data

scaler.fit(X_train)

# Transform the training data

X_train_s = scaler.transform(X_train)

# Transform new data using the same statistics

X_test_s = scaler.transform(X_test)

The common training pattern is therefore:

scaler.fit(X_train)

X_train_scaled = scaler.transform(X_train)
X_test_scaled = scaler.transform(X_test)

05. MinMaxScaler

MinMaxScaler transforms features into a specified range. The default range is from 0 to 1.

This can be useful when you want your features to have a predictable minimum and maximum value.

from sklearn.preprocessing import MinMaxScaler

scaler = MinMaxScaler()

X_train_s = scaler.fit_transform(X_train)
X_test_s  = scaler.transform(X_test)

print("Min:", X_train_s.min(axis=0))
print("Max:", X_train_s.max(axis=0))

The transformation can be represented conceptually as:

06. Choosing a Feature Range

MinMaxScaler allows you to choose a custom output range instead of using 0 to 1.

from sklearn.preprocessing import MinMaxScaler

scaler = MinMaxScaler(feature_range=(-1, 1))

X_train_s = scaler.fit_transform(X_train)
X_test_s = scaler.transform(X_test)

print(X_train_s.min(axis=0))
print(X_train_s.max(axis=0))

This can be useful for algorithms or applications where a particular numerical range is desirable.

07. Why Scaling Helps Some Algorithms

Not every machine learning algorithm is equally affected by feature scale. Algorithms that use distances, gradients, or regularization are often particularly sensitive to differently scaled features.

For example, KNN calculates distances between observations. If one feature has much larger numerical values than another, it can dominate the distance calculation.

08. Algorithms That Often Do Not Require Scaling

Some algorithms are much less sensitive to the numerical scale of individual features. Tree-based algorithms typically make decisions using thresholds rather than distance calculations.

This does not mean scaling is harmful in every situation. It means that scaling is generally not necessary for the core decision-making mechanism of these models.

09. Critical Rule — No Data Leakage

scaler = StandardScaler()

# Correct

X_train_scaled = scaler.fit_transform(X_train)

# Correct

X_test_scaled = scaler.transform(X_test)

The scaler learns its mean and standard deviation from X_train. The same learned values are then used to transform X_test.

scaler = StandardScaler()

X_train_scaled = scaler.fit_transform(X_train)

# Do NOT do this

X_test_scaled = scaler.fit_transform(X_test)

10. Scaling After Train/Test Split

The correct order of operations is important. You should normally split the data before fitting the scaler.

from sklearn.model_selection import train_test_split
from sklearn.preprocessing import StandardScaler

# 1. Split the data

X_train, X_test, y_train, y_test = train_test_split(
X, y,
test_size=0.2,
random_state=42
)

# 2. Create scaler

scaler = StandardScaler()

# 3. Fit only on training data

X_train = scaler.fit_transform(X_train)

# 4. Transform test data

X_test = scaler.transform(X_test)

11. Comparing StandardScaler and MinMaxScaler

Both scalers change the numerical scale of features, but they do so differently.

12. Inspecting the Scaler

After fitting a scaler, Scikit-Learn stores the learned parameters inside the scaler object. These values can be inspected to understand what the preprocessing step learned from the training data.

from sklearn.preprocessing import StandardScaler

scaler = StandardScaler()

X_train_scaled = scaler.fit_transform(X_train)

print("Means:", scaler.mean_)
print("Scales:", scaler.scale_)

These values are learned from the training data and are then reused whenever new data is transformed using the same scaler.

13. Scaling a New Sample

Once a scaler has been fitted, the same scaler can be used to transform new observations before sending them to a trained model.

from sklearn.preprocessing import StandardScaler

scaler = StandardScaler()

X_train_scaled = scaler.fit_transform(X_train)

new_sample = X_test[:1]

new_sample_scaled = scaler.transform(new_sample)

print("Original:", new_sample)
print("Scaled:", new_sample_scaled)

14. Complete Scaling Example

The following example combines dataset loading, train/test splitting, scaling, and a simple model into one workflow.

from sklearn.datasets import load_iris
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import StandardScaler
from sklearn.neighbors import KNeighborsClassifier
from sklearn.metrics import accuracy_score

X, y = load_iris(return_X_y=True)

X_train, X_test, y_train, y_test = train_test_split(
X, y,
test_size=0.2,
random_state=42,
stratify=y
)

scaler = StandardScaler()

X_train = scaler.fit_transform(X_train)
X_test = scaler.transform(X_test)

model = KNeighborsClassifier(n_neighbors=3)

model.fit(X_train, y_train)

predictions = model.predict(X_test)

print("Accuracy:", accuracy_score(y_test, predictions))

15. Common Scaling Mistakes

16. Key Points

from sklearn.datasets import load_iris
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import StandardScaler

X, y = load_iris(return_X_y=True)

# Split the data

# Your code here

# Create the scaler

# Your code here

# Scale training and testing data

# Your code here

# Print the mean and standard deviation

# Your code here

Project 1: Iris Flower Classifier

Apply everything from Unit I to build your first complete classification model with Scikit-Learn. You will load real data, explore it, split it correctly, scale the features, train a KNN model, make predictions, and evaluate its performance.

01. Project Overview

In this project, we will build a machine learning model that can identify the species of an Iris flower from measurements of its physical characteristics.

The Iris dataset contains measurements of iris flowers belonging to three species: setosa, versicolor, and virginica.

These four measurements become our features, while the flower species becomes our target.

02. The Machine Learning Pipeline

This project combines the main concepts covered in Unit I into one complete workflow.

Each step has a specific purpose. Keeping these steps separate makes machine learning projects easier to understand, debug, and maintain.

03. Loading the Iris Dataset

Scikit-Learn provides the Iris dataset through load_iris(). We can load it directly without downloading a CSV file.

from sklearn.datasets import load_iris

iris = load_iris()

print("Feature names:", iris.feature_names)
print("Target names:", iris.target_names)
print("Data shape:", iris.data.shape)
print("Target shape:", iris.target.shape)

The dataset contains 150 samples and 4 numerical features. The target contains three possible flower classes.

04. Separating X and y

Machine learning models generally use X to represent input features and y to represent target values.

iris = load_iris()

X = iris.data
y = iris.target

print("X shape:", X.shape)
print("y shape:", y.shape)

print("First X row:", X[0])
print("First y value:", y[0])

05. Splitting the Dataset

We should not train and evaluate our model using exactly the same examples. Instead, we reserve part of the dataset for testing.

from sklearn.model_selection import train_test_split

X_train, X_test, y_train, y_test = train_test_split(
X,
y,
test_size=0.2,
random_state=42,
stratify=y
)

print("Training samples:", len(X_train))
print("Testing samples:", len(X_test))

With 150 total samples and a 20% test size, approximately 120 samples are used for training and 30 samples are reserved for testing.

06. Scaling the Features

KNN is a distance-based algorithm. This means the numerical scale of the features can affect how distances between flowers are calculated.

We therefore use StandardScaler to standardize the features.

from sklearn.preprocessing import StandardScaler

scaler = StandardScaler()

X_train = scaler.fit_transform(X_train)
X_test = scaler.transform(X_test)

print("Scaled training data:")
print(X_train[:5])

07. Understanding K-Nearest Neighbors

K-Nearest Neighbors (KNN) is a supervised machine learning algorithm that classifies a new observation based on nearby training observations.

When a new flower is given to the model, KNN looks at the closest training examples and uses their classes to determine the predicted class.

08. Choosing n_neighbors

The n_neighbors parameter determines how many nearby training samples KNN considers when making a prediction.

from sklearn.neighbors import KNeighborsClassifier

model_3 = KNeighborsClassifier(n_neighbors=3)
model_5 = KNeighborsClassifier(n_neighbors=5)
model_10 = KNeighborsClassifier(n_neighbors=10)

Using a very small value can make the model highly sensitive to individual training examples, while a larger value considers a broader neighborhood.

09. Creating and Training the Model

Once our data has been prepared, we create a KNN classifier and train it using the training data.

from sklearn.neighbors import KNeighborsClassifier

model = KNeighborsClassifier(n_neighbors=5)

model.fit(X_train, y_train)

print("Model trained successfully!")

The fit() method gives the model access to the training features and their corresponding target labels.

10. Making Predictions

After training, we can use the model to predict the classes of flowers in the test set.

y_pred = model.predict(X_test)

print("Predicted:", y_pred)
print("Actual:   ", y_test)

The predicted values are compared with the actual target values to determine how well the model performed.

11. Measuring Accuracy

Accuracy measures the proportion of predictions that were correct.

from sklearn.metrics import accuracy_score

accuracy = accuracy_score(y_test, y_pred)

print("Accuracy:", accuracy)
print("Accuracy percentage:", accuracy * 100, "%")

12. Classification Report

Accuracy gives us one overall measurement, but classification problems often require more detailed evaluation. Scikit-Learn provides classification_report() for this purpose.

from sklearn.metrics import classification_report

print(
classification_report(
y_test,
y_pred,
target_names=iris.target_names
)
)

The report includes metrics such as precision, recall, and F1-score for each class.

13. Complete Project Code

Now we can combine everything into one complete Python program.

from sklearn.datasets import load_iris
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import StandardScaler
from sklearn.neighbors import KNeighborsClassifier
from sklearn.metrics import accuracy_score, classification_report

# Load dataset

iris = load_iris()

X, y = iris.data, iris.target

print("Shape:", X.shape)
print("Classes:", iris.target_names)

# Split data

X_train, X_test, y_train, y_test = train_test_split(
X,
y,
test_size=0.2,
random_state=42,
stratify=y
)

# Scale features

scaler = StandardScaler()

X_train = scaler.fit_transform(X_train)
X_test = scaler.transform(X_test)

# Create model

model = KNeighborsClassifier(n_neighbors=5)

# Train model

model.fit(X_train, y_train)

# Predict

y_pred = model.predict(X_test)

# Evaluate

accuracy = accuracy_score(y_test, y_pred)

print("Accuracy:", accuracy)

print(classification_report(
y_test,
y_pred,
target_names=iris.target_names
))

14. Making a Prediction for a New Flower

One of the most useful parts of machine learning is making predictions on new data. Suppose we receive measurements for a flower that was not part of the original training dataset.

new_flower = [[5.1, 3.5, 1.4, 0.2]]

new_flower_scaled = scaler.transform(new_flower)

prediction = model.predict(new_flower_scaled)

print("Predicted class:", prediction[0])
print("Predicted species:", iris.target_names[prediction[0]])

15. Understanding the Complete Pipeline

At this point, the entire project can be understood as a sequence of connected steps.

16. Expected Result

A high accuracy on this small, well-known dataset does not mean that every real-world classification problem will be equally easy. Real datasets can contain missing values, noise, class imbalance, outliers, irrelevant features, and much more complex relationships.

17. Common Mistakes

18. Key Points

from sklearn.neighbors import KNeighborsClassifier
from sklearn.metrics import accuracy_score

for k in [3, 5, 7, 9]:

```
model = KNeighborsClassifier(n_neighbors=k)

# Train the model
model.fit(X_train, y_train)

# Make predictions
y_pred = model.predict(X_test)

# Calculate accuracy
accuracy = accuracy_score(y_test, y_pred)

print("K =", k, "| Accuracy =", accuracy)

Linear Regression

Learn how to predict continuous values using Linear Regression, the most fundamental supervised learning algorithm. You will understand the mathematical idea behind a regression line, train a model with scikit-learn, evaluate its performance, and interpret what the model has learned.

01. What Is Linear Regression?

Linear Regression is a supervised machine learning algorithm used to predict a continuous numerical value.

For example, we can use Linear Regression to predict:

• House price from house size
• Salary from years of experience
• Temperature from time of day
• Sales from advertising expenditure
• Student marks from study hours
• Fuel consumption from vehicle characteristics

The word linear means that the model attempts to describe the relationship between input variables and the target using a straight-line relationship.

02. The Basic Linear Regression Equation

For a single input feature, Linear Regression uses the equation:

y = mx + b

• y = predicted value
• x = input feature
• m = slope or coefficient
• b = intercept

In machine learning, this can also be written as:

ŷ = β₀ + β₁x

Here, ŷ represents the predicted value, β₀ is the intercept, and β₁ is the coefficient of the feature.

x = 5
m = 10
b = 5

y = m * x + b

print(y)

The result is 55 because:

y = (10 × 5) + 5 = 55

The trained Linear Regression model automatically learns suitable values for the coefficient and intercept from the training data.

03. Understanding the Coefficient

The coefficient tells us how much the predicted target changes when the input feature increases by one unit.

Suppose our model learns:

y = 8x + 20

The coefficient is 8.

This means that for every one-unit increase in x, the predicted value of y increases by approximately 8 units.

04. Understanding the Intercept

The intercept is the predicted value of the target when all input features are zero.

For example:

y = 10x + 15

The intercept is 15.

When x = 0:

y = (10 × 0) + 15 = 15

In real-world datasets, the intercept may not always have a meaningful practical interpretation if a value of zero for the feature is outside the realistic range of the data.

05. How Linear Regression Learns

During training, Linear Regression searches for coefficients that produce predictions as close as possible to the actual target values.

The difference between an actual value and a predicted value is called the residual or error.

The model commonly uses the ordinary least squares approach, which finds parameters that minimize the sum of squared residuals.

Conceptually:

Residual = Actual Value − Predicted Value

Squaring the errors prevents positive and negative errors from cancelling each other out and gives larger errors more influence during optimization.

06. Creating Training Data

Before training a regression model, we need two important pieces of information:

• X — the input features
• y — the target values we want to predict

For example, imagine that we want to predict a person's score based on the number of hours they studied.

import numpy as np

X = np.array([[1], [2], [3], [4], [5]])
y = np.array([20, 35, 45, 55, 70])

print(X)
print(y)

Here, X contains the number of study hours, while y contains the corresponding scores.

07. Importing LinearRegression

Scikit-learn provides Linear Regression through the sklearn.linear_model module.

from sklearn.linear_model import LinearRegression

We can then create a Linear Regression model:

model = LinearRegression()

At this point, the model has been created but it has not learned anything yet. Learning happens when we call fit().

08. Splitting the Dataset

Just like the Iris classification project, we should separate our data into training and testing sets.

from sklearn.model_selection import train_test_split

X_train, X_test, y_train, y_test = train_test_split(
X,
y,
test_size=0.2,
random_state=42
)

The training data is used to learn the relationship between X and y. The testing data is kept separate so that we can evaluate how well the model performs on unseen examples.

09. Training the Model

Once the data has been split, we can train the Linear Regression model using fit().

model = LinearRegression()

model.fit(X_train, y_train)

During fit(), scikit-learn calculates the model parameters based on the training data.

After training, we can inspect the values learned by the model:

print("Coefficient:", model.coef_)
print("Intercept:", model.intercept_)

10. Making Predictions

Once the model has been trained, we can use predict() to generate predictions for new data.

y_pred = model.predict(X_test)

print(y_pred)

We can also predict the value for a completely new input.

new_data = np.array([[11]])

prediction = model.predict(new_data)

print("Prediction:", prediction[0])

The model uses the equation it learned from the training data to calculate the predicted value.

11. Mean Squared Error (MSE)

Mean Squared Error, commonly called MSE, measures the average squared difference between actual values and predicted values.

The formula is:

MSE = (1/n) × Σ(actual − predicted)²

A lower MSE generally indicates that the predictions are closer to the actual values.

from sklearn.metrics import mean_squared_error

mse = mean_squared_error(y_test, y_pred)

print("MSE:", mse)

12. R² Score

R², or the coefficient of determination, measures how much of the variation in the target is explained by the regression model relative to a baseline that always predicts the mean target value.

A commonly used interpretation is:

• R² = 1.0 — perfect predictions on the evaluated data
• R² = 0 — equivalent to the mean-prediction baseline
• R² < 0 — worse than that baseline on the evaluated data

from sklearn.metrics import r2_score

score = r2_score(y_test, y_pred)

print("R2 Score:", score)

13. Complete Linear Regression Example

Now let's put the complete workflow together: create data, split it, train the model, make predictions, and evaluate the results.

import numpy as np
from sklearn.linear_model import LinearRegression
from sklearn.model_selection import train_test_split
from sklearn.metrics import mean_squared_error, r2_score

X = np.array([[1],[2],[3],[4],[5],[6],[7],[8],[9],[10]])
y = np.array([15, 25, 35, 45, 55, 62, 70, 79, 88, 95])

X_train, X_test, y_train, y_test = train_test_split(
X,
y,
test_size=0.2,
random_state=42
)

model = LinearRegression()

model.fit(X_train, y_train)

y_pred = model.predict(X_test)

print("Coefficient:", model.coef_[0])
print("Intercept:", model.intercept_)

print("Predictions:", y_pred)
print("Actual:", y_test)

print("MSE:", mean_squared_error(y_test, y_pred))
print("R2 Score:", r2_score(y_test, y_pred))

This example follows the standard supervised learning workflow:

Data → Split → Train → Predict → Evaluate

14. Interpreting the Model

Suppose the model produces:

Coefficient: 8.9
Intercept: 6.2
MSE: 4.5
R2 Score: 0.97

The coefficient tells us that an increase of one unit in X is associated with an increase of approximately 8.9 units in the predicted target, according to this fitted model.

The intercept is approximately 6.2, meaning the model predicts 6.2 when X is zero.

The R² value of 0.97 indicates that the fitted model explains a large proportion of the target variation relative to the mean baseline on this evaluated dataset. It does not mean that every prediction is exactly 97% correct.

15. Multiple Linear Regression

Linear Regression is not limited to one feature. We can use multiple features to predict a single continuous target.

For example, house price could depend on:

• House size
• Number of bedrooms
• Number of bathrooms
• Age of the house

The equation becomes:

y = β₀ + β₁x₁ + β₂x₂ + β₃x₃ + ... + βₙxₙ

import numpy as np
from sklearn.linear_model import LinearRegression

X = np.array([
[1000, 2],
[1500, 3],
[2000, 3],
[2500, 4],
[3000, 5]
])

y = np.array([150000, 220000, 300000, 380000, 450000])

model = LinearRegression()
model.fit(X, y)

print("Coefficients:", model.coef_)
print("Intercept:", model.intercept_)

new_house = np.array([[2200, 3]])

prediction = model.predict(new_house)

print("Predicted price:", prediction[0])

Here, the first feature represents house size and the second feature represents the number of bedrooms. The model learns a separate coefficient for each feature.

16. Visualizing a Regression Line

For a dataset with one feature, visualization is an excellent way to understand Linear Regression. We can plot the original data points and then draw the line learned by the model.

import matplotlib.pyplot as plt

plt.scatter(X, y, label="Actual Data")
plt.plot(X, model.predict(X), label="Regression Line")

plt.xlabel("X")
plt.ylabel("y")
plt.title("Linear Regression")

plt.legend()
plt.show()

The dots represent the observed training examples, while the line represents the predictions made by the model.

17. Common Linear Regression Mistakes

Beginners often make several mistakes when building regression models.

• Using classification metrics: Accuracy is generally not the appropriate primary metric for continuous regression targets.
• Testing on training data: This can give an overly optimistic evaluation.
• Ignoring data leakage: Information from the test set should not influence training or preprocessing decisions.
• Assuming high R² means a perfect model: R² must be interpreted together with the prediction errors and application context.
• Ignoring outliers: Extreme observations can strongly influence ordinary least squares regression.
• Assuming correlation proves causation: A fitted relationship does not by itself establish that one variable causes another.
• Using linear regression for every problem: Some relationships are not well represented by a straight-line model.

18. Key Points to Remember