South Broward Football Stats Pipeline

A real story about building a data system for an entire varsity football season.

🔗 Project Repositories

• Backend & Lambda Functions:

https://github.com/lamechbob/sideline-back-end

• Streamlit Dashboard App:

https://github.com/lamechbob/sideline-app

🎥 Video Walkthrough (Coming Soon)

Embed your YouTube link here once recorded.

🧰 Tech Used

Core Technologies

• Python – Lambda functions, ETL logic
• PostgreSQL (AWS RDS) – Secure relational database (private subnet)
• Streamlit – Dashboard UI and visualizations

AWS Services

• Amazon S3 – File storage & folder-based triggers
• AWS Lambda – Serverless ingestion for schedule, roster, and game stats
• Amazon RDS (PostgreSQL) – Central relational database
• Amazon EC2 (Bastion Host) – Secure DBeaver access
• AWS VPC (Private Subnets) – Network isolation and security controls

Tools & Workflow

• Google Sheets – Data collection templates (schedule, roster, stats)
• DBeaver – SQL validation & troubleshooting
• PyCharm – Development environment
• GitHub – Version control & repo organization

1. The Spark

This project started with a simple moment: my cousin became the Offensive Coordinator at South Broward High School. I grew up around the area, and I knew how chaotic high school football operations can be — especially when it comes to stats.

Most teams track stats on paper, group chats, or last-minute messages. It’s messy, inconsistent, and almost impossible to use for real insight.

At the same time, I was sharpening my data engineering skills and looking for a real project that wasn’t just a tutorial or a portfolio exercise. So I asked my cousin:

“Let me handle your stats. I’ll build you a system you can actually use.”

He said yes.

That yes turned into a weekly pipeline — rain, crowds, pressure, and all — that had to work every single Friday night.

This was my chance to build something real… and I took it.

2. The Build

I didn’t begin with AWS diagrams or overengineering.

I began with a simple question:

What’s the simplest system that won’t break on game night?

Starting With Structure

I built three simple but powerful Google Sheets templates:

• Roster (with auto-generated Player IDs)
• Season Schedule
• Game Stats, broken into atomic events (one row per action)

This “every row is an event” rule became the backbone of the entire pipeline.

It kept the data clean, consistent, and predictable week after week.

Turning It Into a Pipeline

Once the templates were ready, I built a lightweight AWS ingestion system:

• One S3 bucket with folders for schedule, roster, and game stats
• Three AWS Lambda functions triggered by file uploads
• An RDS Postgres database secured inside a private subnet
• A SQL view that calculates weekly and season totals
• A Streamlit dashboard that displays the data in real time

Nothing overbuilt — just clean, reliable components that worked together smoothly.

The Weekly Rhythm

Every Friday night, I stood on the sidelines with my iPad in a weatherproof case and tracked every play, row by row.

After each game:

1. Upload the game stats file to S3
2. Let the Lambda process the data
3. Validate everything in DBeaver via an EC2 endpoint
4. Export the updated season summary view
5. Upload the summary CSV back into S3
6. Watch the Streamlit dashboard refresh
7. Push final numbers to MaxPreps

By mid-season, coaches checked the dashboard regularly, players followed their stats, and the entire pipeline became part of the weekly routine.

And every week, it held up.

3. The Season

What made this project special wasn’t the tech — it was the responsibility.

Coaches needed stats to plan.

Players cared about their performance.

Parents watched the leaderboard.

And every Friday night, the data started with me.

There was pressure:

• Real deadlines
• Weather
• Fast-paced games
• Moments where something didn’t look right and needed quick verification

There were also limitations — especially the manual season-summary export — but I still delivered results consistently.

And the system worked.

It delivered:

• Weekly player performance summaries
• Player-level dashboards
• Faster MaxPreps submissions
• A predictable, reliable reporting rhythm

More importantly, it taught me something no certification can:

Production is real when people are waiting on you.

This project strengthened my data engineering discipline and my ability to communicate, troubleshoot under pressure, and deliver for real users.

And it didn’t come from a tech job.

It came from family, community, and showing up every week to keep the lights on.