🧠 AI with Python - 🔍 Grid Search using GridSearchCV

Posted On: August 28, 2025

Description:

Choosing the right hyperparameters can make or break a model’s performance. Instead of manually guessing values (and getting lucky or unlucky), Grid Search provides a systematic way to search across combinations and pick the best based on cross-validated performance.

Why Grid Search?

Systematic exploration: Evaluate a predefined grid of hyperparameter combinations.
Cross-validated scoring: Each combination is scored using k-fold CV for reliability.
Refit best model: The best configuration is refit on the full training set for final evaluation.

Minimal Implementation

We’ll demonstrate grid search on an SVM classifier (RBF kernel) using the Iris dataset. The core idea is to (1) define a parameter grid, and (2) run GridSearchCV.

# Define parameter grid for SVC
param_grid = {
    "C": [0.1, 1, 10],
    "gamma": ["scale", 0.1, 0.01, 0.001]
}

# Run grid search (cv=5) and refit best model
from sklearn.model_selection import GridSearchCV
grid = GridSearchCV(estimator=SVC(kernel="rbf"), param_grid=param_grid, cv=5, scoring="accuracy", refit=True)
grid.fit(X_train, y_train)

Interpreting Results

After fitting, use best_params_ and best_score_ to understand what worked best. Then evaluate on the held-out test set.

print("Best Params:", grid.best_params_)
print("Best CV Accuracy:", round(grid.best_score_, 4))

best_model = grid.best_estimator_
print("Test Accuracy:", round(best_model.score(X_test, y_test), 4))

Sample Output:

Best CV Accuracy: 0.9727
Best Params: {'C': 10, 'gamma': 0.1}

Test Accuracy with Best Params: 0.9474

Values will vary slightly based on random splits and your environment.

Key Takeaways

Grid Search + CV provides a robust, unbiased way to choose hyperparameters.
Always evaluate the best-found model on a separate test set.
Start with a small grid, then expand around promising regions to save time.

Code Snippet:

# Import data utilities, model, and grid search tools
from sklearn.datasets import load_iris
from sklearn.model_selection import train_test_split, GridSearchCV
from sklearn.svm import SVC
import numpy as np


# Load the Iris dataset
iris = load_iris()
X, y = iris.data, iris.target

# Create a hold-out test set for final evaluation
X_train, X_test, y_train, y_test = train_test_split(
    X, y, test_size=0.25, random_state=42, stratify=y
)


# Base model: Support Vector Classifier with RBF kernel
svc = SVC(kernel="rbf", probability=False)

# Parameter grid to search over
param_grid = {
    "C": [0.1, 1, 10],
    "gamma": ["scale", 0.1, 0.01, 0.001]
}


# Grid search with cross-validation
grid = GridSearchCV(
    estimator=svc,
    param_grid=param_grid,
    scoring="accuracy",
    cv=5,
    n_jobs=-1,
    refit=True,          # Refit best model on the whole training set
    return_train_score=False
)


# Execute the grid search
grid.fit(X_train, y_train)

# Show the best mean CV score and the corresponding parameters
print("Best CV Accuracy:", round(grid.best_score_, 4))
print("Best Params:", grid.best_params_)


# Evaluate the best model on the held-out test set
best_model = grid.best_estimator_
test_acc = best_model.score(X_test, y_test)
print("Test Accuracy with Best Params:", round(test_acc, 4))


# Optional: peek at all candidates and their mean CV score
mean_scores = grid.cv_results_["mean_test_score"]
params_list = grid.cv_results_["params"]

for p, s in zip(params_list, mean_scores):
    print(f"{p} -> {s:.4f}")

← →	move
↑	rotate
↓	soft drop
Space	hard drop
P	pause / resume

🧠 AI with Python - 🔍 Grid Search using GridSearchCV

Description:

Why Grid Search?

Minimal Implementation

Interpreting Results

Sample Output:

Key Takeaways

Code Snippet:

Comments