Retail SME: Faster Pipelines, Lower Costs
Stabilising nightly loads and cutting the cloud bill with pragmatic SQL/ADF/Databricks optimisation and better observability.
Industry
Retail (e-commerce)
Company size
~150 staff
Duration
4 weeks (tune-up)
Stack
Azure SQL, ADF, Databricks (Delta), Power BI
Challenge
- Nightly jobs missing SLA; frequent retries and inconsistent freshness.
- Cloud spend trending up without clear drivers.
- Power BI reports slow during morning peak.
Approach
- SQL tune-up: plan cache analysis, parameter sniffing fixes, indexing & statistics overhaul.
- ADF/Spark optimisation: skewed join fixes, partitioning, Delta maintenance & compaction.
- Cost controls: autoscale and schedule adjustments; tiering & retention policy.
- Observability: wired Azure Monitor/Log Analytics; failure + freshness dashboards and runbooks.
Results
| Metric | Before | After |
|---|---|---|
| Nightly pipeline duration | ~6h+ | ~2h |
| Report refresh window | Missed SLA | Met by 7:00am |
| Cluster/runtime cost | Baseline | ↓ ~30–40%* |
| Failure rate | Intermittent | Near-zero (with alerts) |
“Quick wins we could see—and a clear backlog for the next increments.”
Want measurable performance wins?