AW Dev Rethought

Truth can only be found in one place: the code - Robert C. Martin

🧩 Python Automation Recipes – 📦 Auto ZIP & Upload


Description:

📌 Introduction

In many workflows, files are not just generated — they also need to be compressed and uploaded somewhere for storage, transfer, or backup.

This automation recipe shows how to zip a folder and upload the archive to cloud storage.

To keep it simple and reusable, this version uses an S3-style upload flow with boto3, which is a common real-world pattern for backups, exports, and deployment artifacts.

You can adapt the same structure later for:

  • Google Drive
  • Dropbox
  • internal storage APIs
  • remote servers

🔎 Explanation

  • The script first compresses a target folder into a .zip archive using shutil.make_archive().
  • It adds a timestamp to the archive name so each run produces a unique file.
  • Then it uploads that zip file to an S3 bucket using boto3.
  • This creates a reusable automation pattern for:
    • project backups
    • log archives
    • export deliveries
    • scheduled storage jobs

If you do not want to upload immediately, you can still use the zip part alone as a local backup step.


✅ Key Takeaways

  • 📦 Compress folders automatically into zip archives.
  • ☁️ Upload generated backups to cloud storage.
  • ⚙️ Useful for backup pipelines, archival workflows, and automated exports.

Code Snippet:

from pathlib import Path
from datetime import datetime
import shutil
import boto3

# --- Step 1: Configuration ---

# Folder you want to compress
SOURCE_FOLDER = Path("/my_project")  # Change this

# Temporary folder to store generated ZIP files
OUTPUT_FOLDER = Path("archives")
OUTPUT_FOLDER.mkdir(exist_ok=True)

# AWS S3 configuration
BUCKET_NAME = "your-s3-bucket-name"   # Change this
S3_FOLDER = "backups"                 # Folder path inside S3 bucket

# --- Step 2: Generate timestamped ZIP filename ---
timestamp = datetime.now().strftime("%Y-%m-%d_%H-%M-%S")
zip_name = f"{SOURCE_FOLDER.name}_{timestamp}"
zip_path_without_ext = OUTPUT_FOLDER / zip_name

# --- Step 3: Create ZIP archive ---
archive_path = shutil.make_archive(
    base_name=str(zip_path_without_ext),
    format="zip",
    root_dir=SOURCE_FOLDER
)

print(f"📦 ZIP archive created: {archive_path}")

# --- Step 4: Upload ZIP to S3 ---
try:
    s3_client = boto3.client("s3")

    s3_key = f"{S3_FOLDER}/{Path(archive_path).name}"
    s3_client.upload_file(archive_path, BUCKET_NAME, s3_key)

    print(f"   Uploaded to S3 successfully:")
    print(f"   Bucket: {BUCKET_NAME}")
    print(f"   Key   : {s3_key}")

except Exception as e:
    print(f"Upload failed.")
    print(f"Error: {e}")

Link copied!

Comments

Add Your Comment

Comment Added!