⚡️ Saturday ML Spark – 📊 Experiment Tracking with MLflow

Posted on: August 1, 2026

Description:

Building a machine learning model is rarely a one-time process. Data scientists and ML engineers continuously experiment with different algorithms, hyperparameters, preprocessing techniques, and datasets to improve performance.

Imagine training dozens of models over several days. After some time, questions naturally arise:

Which model achieved the highest accuracy?
What hyperparameters produced the best results?
Which dataset was used?
Which model was eventually deployed?

Without proper tracking, answering these questions becomes difficult.

This is why Experiment Tracking is one of the most important practices in modern MLOps, and MLflow has become one of the industry's most popular tools for managing it.

In this project, we'll build an experiment tracking workflow using MLflow and learn how production ML teams organize, compare, and manage machine learning experiments.

What Is Experiment Tracking?

Experiment Tracking is the process of recording everything about a machine learning experiment.

Instead of manually saving results in notebooks or spreadsheets, every training run automatically records information such as:

model type
hyperparameters
evaluation metrics
datasets
trained models
execution time

This creates a complete history of every experiment, making it easy to compare results and reproduce successful models.

Why Is It Important?

Consider training the same model with different numbers of trees.

Run 1

Random Forest
Trees = 50
Accuracy = 95.4%

Run 2

Random Forest
Trees = 100
Accuracy = 96.1%

A week later, you may remember that one model performed better—but not why.

Experiment tracking eliminates this uncertainty by storing every run automatically.

Getting Started with MLflow

MLflow makes experiment tracking remarkably simple.

import mlflow

mlflow.set_experiment(
    "Saturday_ML_Spark"
)

If the experiment doesn't already exist, MLflow creates it automatically.

Every future training run will now be organized under this experiment.

Logging Hyperparameters

Hyperparameters define how a model is trained.

For example:

mlflow.log_param(
    "n_estimators",
    100
)

MLflow stores these values so every experiment can be reproduced later.

Logging Performance Metrics

Once training finishes, important metrics can be recorded.

mlflow.log_metric(
    "accuracy",
    accuracy
)

You can log multiple metrics such as:

accuracy
precision
recall
F1-score
ROC-AUC
inference latency

This makes comparing different experiments much easier.

Saving the Trained Model

MLflow also stores trained models as artifacts.

mlflow.sklearn.log_model(
    model,
    "random_forest_model"
)

Instead of searching through folders full of model files, every model is linked directly to the experiment that created it.

Comparing Multiple Runs

One of MLflow's biggest strengths is comparing experiments.

Instead of asking:

"Which model did I train last Tuesday?"

you can simply open the MLflow UI and compare runs side by side.

You can instantly identify:

highest accuracy
fastest training time
best hyperparameters
latest experiment
deployed model candidate

This dramatically improves the experimentation process.

How Experiment Tracking Works

A typical MLflow workflow looks like this:

Dataset
      │
      ▼
Train Model
      │
      ▼
Log Parameters
      │
      ▼
Log Metrics
      │
      ▼
Save Model
      │
      ▼
Store Experiment
      │
      ▼
Compare Runs
      │
      ▼
Deploy Best Model

This creates a complete and reproducible history of every machine learning experiment.

Benefits of MLflow

Using MLflow offers several advantages:

centralized experiment management
automatic parameter logging
metric tracking
model artifact storage
reproducible workflows
easier collaboration across teams
faster model selection for deployment

These capabilities become increasingly valuable as the number of experiments grows.

Where MLflow Is Used

MLflow is widely adopted in production ML environments, including:

recommendation systems
fraud detection
customer churn prediction
healthcare AI
financial forecasting
computer vision
natural language processing
enterprise MLOps platforms

It integrates seamlessly with popular machine learning frameworks such as Scikit-learn, TensorFlow, PyTorch, and XGBoost.

Key Takeaways

Experiment tracking records every machine learning training run.
MLflow logs parameters, metrics, models, and artifacts in one place.
Tracking experiments improves reproducibility and collaboration.
Comparing multiple runs helps identify the best-performing model for deployment.
MLflow is one of the most widely used open-source tools in modern MLOps workflows.

Conclusion

Developing a successful machine learning model involves far more than writing training code. Managing experiments, comparing results, and maintaining reproducibility are essential for building reliable production AI systems. MLflow simplifies this entire process by providing a centralized platform for tracking every experiment from training to deployment.

This begins the Production AI & MLOps track in Saturday ML Spark. Next, we'll build a Model Monitoring Dashboard, where we'll monitor prediction quality, latency, throughput, and operational metrics to ensure deployed models remain accurate, reliable, and production-ready over time.

Code Snippet:

# =========================================================
# 📦 Install Required Libraries
# =========================================================

# Run this in terminal if not installed:
# pip install mlflow scikit-learn pandas matplotlib

# =========================================================
# 📦 Import Required Libraries
# =========================================================

import mlflow
import mlflow.sklearn
import pandas as pd
from sklearn.datasets import load_breast_cancer
from sklearn.ensemble import RandomForestClassifier
from sklearn.metrics import accuracy_score
from sklearn.model_selection import train_test_split

# =========================================================
# 📊 Load Dataset
# =========================================================

data = load_breast_cancer()
X = pd.DataFrame(data.data, columns=data.feature_names)
y = pd.Series(data.target, name="target")

# =========================================================
# ✂️ Split Dataset
# =========================================================

X_train, X_test, y_train, y_test = train_test_split(
    X,
    y,
    test_size=0.2,
    random_state=42
)

# =========================================================
# 🏗️ Create MLflow Experiment
# =========================================================

mlflow.set_experiment("Saturday_ML_Spark_Experiment_Tracking")

# =========================================================
# 🚀 Define Hyperparameter Values
# =========================================================

tree_values = [10, 50, 100, 200]
results = []

# =========================================================
# 🤖 Train Models and Track Experiments
# =========================================================

for trees in tree_values:

    with mlflow.start_run(run_name=f"RandomForest_{trees}_Trees"):

        model = RandomForestClassifier(
            n_estimators=trees,
            random_state=42
        )

        model.fit(X_train, y_train)
        predictions = model.predict(X_test)
        accuracy = accuracy_score(y_test, predictions)

        mlflow.log_param("algorithm", "RandomForestClassifier")
        mlflow.log_param("n_estimators", trees)
        mlflow.log_param("random_state", 42)

        mlflow.log_metric("accuracy", accuracy)

        mlflow.sklearn.log_model(
            model,
            artifact_path="random_forest_model"
        )

        results.append({
            "Trees": trees,
            "Accuracy": round(accuracy, 4)
        })

        print("=" * 70)
        print(f"Experiment : {trees} Trees")
        print(f"Accuracy   : {accuracy:.4f}")

# =========================================================
# 📊 Display Experiment Summary
# =========================================================

results_df = pd.DataFrame(results)

print("\n" + "=" * 70)
print("EXPERIMENT SUMMARY")
print("=" * 70)
print(results_df)

# =========================================================
# 🏆 Find Best Model
# =========================================================

best_model = results_df.loc[
    results_df["Accuracy"].idxmax()
]

print("\n" + "=" * 70)
print("BEST MODEL")
print("=" * 70)
print(f"Trees    : {best_model['Trees']}")
print(f"Accuracy : {best_model['Accuracy']:.4f}")

# =========================================================
# 💾 Save Experiment Results
# =========================================================

results_df.to_csv(
    "mlflow_experiment_results.csv",
    index=False
)

print("\nExperiment results saved successfully.")

# =========================================================
# 📂 Load Saved Results
# =========================================================

loaded_results = pd.read_csv(
    "mlflow_experiment_results.csv"
)

print("\n" + "=" * 70)
print("LOADED RESULTS")
print("=" * 70)
print(loaded_results)

# =========================================================
# 📈 Rank Experiments
# =========================================================

ranking = loaded_results.sort_values(
    by="Accuracy",
    ascending=False
)

print("\n" + "=" * 70)
print("MODEL RANKING")
print("=" * 70)
print(ranking)

# =========================================================
# 📋 Display Experiment Statistics
# =========================================================

print("\n" + "=" * 70)
print("EXPERIMENT STATISTICS")
print("=" * 70)
print(f"Total Runs       : {len(results_df)}")
print(f"Highest Accuracy : {results_df['Accuracy'].max():.4f}")
print(f"Lowest Accuracy  : {results_df['Accuracy'].min():.4f}")
print(f"Average Accuracy : {results_df['Accuracy'].mean():.4f}")

# =========================================================
# 🏛️ MLflow Workflow
# =========================================================

print("\n" + "=" * 70)
print("MLFLOW WORKFLOW")
print("=" * 70)

print("""
Dataset
    │
    ▼
Train Model
    │
    ▼
Log Parameters
    │
    ▼
Log Metrics
    │
    ▼
Save Model
    │
    ▼
Compare Runs
    │
    ▼
Deploy Best Model
""")

# =========================================================
# ✅ Final Note
# =========================================================

print("Experiment Tracking completed successfully!")
print("MLflow helps track parameters, metrics, artifacts, and trained models, making ML experiments reproducible and easy to compare.")

← →	move
↑	rotate
↓	soft drop
Space	hard drop
P	pause / resume

⚡️ Saturday ML Spark – 📊 Experiment Tracking with MLflow

Description:

What Is Experiment Tracking?

Why Is It Important?

Getting Started with MLflow

Logging Hyperparameters

Logging Performance Metrics

Saving the Trained Model

Comparing Multiple Runs

How Experiment Tracking Works

Benefits of MLflow

Where MLflow Is Used

Key Takeaways

Conclusion

Code Snippet:

Comments

Add Your Comment

⚡️ Saturday ML Spark – 📊 Experiment Tracking with MLflow

Description:

What Is Experiment Tracking?

Why Is It Important?

Getting Started with MLflow

Logging Hyperparameters

Logging Performance Metrics

Saving the Trained Model

Comparing Multiple Runs

How Experiment Tracking Works

Benefits of MLflow

Where MLflow Is Used

Key Takeaways

Conclusion

Code Snippet:

Comments Show Comments

Add Your Comment

Related Posts

⚡️ Saturday ML Spark – 🏪 Feature Store Concepts for Production ML

⚡️ Saturday ML Spark – 📦 Model Versioning & Rollback Strategy

⚡️ Saturday ML Spark – 🐤 Canary Deployment for Machine Learning Models

7-Day AI Crash Course

Comments