Analytics stacks are going hybrid. GitLab pairs Postgres with ClickHouse to power analytics on top of an operational workload [1].
That combo preserves OLTP robustness in Postgres while enabling fast, columnar analytics with ClickHouse. The pattern points to a broader shift: run live writes on one engine and crunch queries on another, tuned for the workload [1].
On the optimization front, Databricks's Databricks SQL optimizer—described by espresso.ai—acts like Kubernetes for data warehousing: autoscaling and cluster selection are automated to boost utilization without added latency [2]. The scheduler uses ML models that predict runtime and capacity, letting operators run machines hotter than cloud providers permit and still cut costs [2].
The takeaway: teams are embracing multi-engine setups to balance robustness, speed, and cost, and tooling around automation is becoming a core part of the analytics stack.
References
How GitLab uses Postgres and ClickHouse to build their data stack?
GitLab data stack mixture of Postgres and ClickHouse for analytics; discusses architecture and performance tradeoffs scaling analytical workloads in practice
View sourceShow HN: Optimize Databricks SQL
Show HN: Databricks SQL optimizer; autoscaling, cluster selection, ML scheduling to improve utilization and reduce costs.
View source