Building a Data Platform on a Startup Budget
A production data platform serving 3 domains, 12 pipelines, and 25 dashboards was built using open-source tools for $487 per month total infrastructure cost.
Portfolio
Systems built, documented, and reflected upon.
A production data platform serving 3 domains, 12 pipelines, and 25 dashboards was built using open-source tools for $487 per month total infrastructure cost.
SQL training for 35 non-technical stakeholders achieved 71% query independence in 8 weeks, reducing ad-hoc data requests from 23 to 9 per week.
A data quality project quantified dirty data costs at $2.4 million annually across labor waste, decision delays, and direct errors reaching clients. Clean data is a measurable business expense.
A cross-cloud architecture achieved portability across AWS, Azure, and GCP with 12% cost overhead while saving $1.4 million annually by restoring credible vendor negotiation leverage.
A production system migration moved 4.7 million records over 16 weeks with zero data loss and zero unplanned downtime using dual-write patterns and automated reconciliation.
A consent management system reduced support tickets by 78% and increased opt-in rates from 34% to 67% by treating consent as a first-class data entity with its own service and event stream.
Automated ethics auditing on every model update caught 94% of fairness regressions within 15 minutes. The previous quarterly manual audit had a 47-day average detection gap.
Integrating fairness checks and bias audits into a production ML pipeline reduced bias-related incidents by 82% and added only 14 minutes to the training cycle.
8 of 11 knowledge bases failed within 6 months. The 3 survivors embedded knowledge creation into existing workflows instead of adding separate systems.
Query pattern optimization across 3 Snowflake deployments reduced annual costs from $412,000 to $187,000 without changing business logic or removing features.