Problem. 500+ fitness-studio clients needed fast, reliable analytics; reporting on the legacy stack was slow (SLA at 70%) and BI licensing costs were high.

Architecture. AWS Data Lakehouse: DMS CDC → S3 Bronze → Apache Iceberg Silver (S3 Tables) → Athena Gold. Cube Cloud semantic layer — 18 cubes, 11 pre-aggregated views, 58 KPIs across Finance, Memberships, Classes, and Marketing — with JWT-based multi-tenant row-level security for 500+ operators. 20-table Gold analytics layer replacing GoodData.

My role. Senior Data Engineer — architected and built the lakehouse end-to-end and designed the Cube Cloud LDM.

Outcome. Query latency −45%, SLA 70%→98%, BI licensing costs −35%.

Problem. 5+ TB of CTMS clinical trial data sat in a legacy IMPACT system, slowing onboarding across 6+ clinical trial projects, and 200K+ clinical assets had no searchable catalog.

Architecture. PySpark/Airflow migration pipeline moving CTMS data from IMPACT to Veeva Vault; Axon Data Catalog indexing 200K+ clinical assets; 20+ standardised, reusable Data Products and DWS APIs shared across 5+ teams.

My role. Engineer II, Software Configuration and Development — led the migration engineering and built the catalog and reusable Data Products.

Outcome. Onboarding 60% faster across 6+ trials, discoverability +50%, validation efficiency +30%, incident response time −40%.

Problem. Growing multi-tenant data volume across 500+ clients risked silent data quality issues undermining trust in the Gold analytics layer.

Architecture. A 9-check PySpark DQ framework with a quarantine path for failing records and SNS alerting, wired into the Bronze→Silver pipeline.

My role. Designed and built the framework as part of the Kenko AI lakehouse build.

Outcome. Data incidents −60%, data confidence 82%→97%.

# One of 9 checks in the quarantine-and-alert DQ pipeline
def run(self, df, quarantine_path=None):
    total = df.count(); bad_mask = F.lit(False)
    for rule in self.rules:
        fail_cond = self._build_condition(rule)
        bad_mask = bad_mask | fail_cond
    clean_df = df.filter(~bad_mask)
    if quarantine_path:
        df.filter(bad_mask).write.format("delta") \
            .mode("append").save(quarantine_path)
    return clean_df

Problem. Wanted a hands-on way to prove out lakehouse and data-quality patterns learned during a self-directed upskilling period, outside production constraints.

Architecture. Bronze/Silver/Gold medallion architecture with Kafka streaming ingestion, a PySpark DQ engine with quarantine, SCD Type 2 dimensions, and Delta Lake storage.

My role. Independent creator and maintainer — architecture, pipeline code, and CI/CD.

Outcome. CI green, v0.1.0 released. github.com/nikuamit/astra-data-platform ↗