In the evolving landscape of cloud analytics, the demand for business intelligence tools to support both operational reporting and in-depth analysis has become a common challenge. This often results in a single environment taking on multiple roles – serving as a presentation layer, a modelling engine, and an ad-hoc compute system simultaneously.
A recent shift in architecture at Carousell, a Southeast Asian marketplace, sheds light on how analytics teams are addressing this issue. The company’s analytics engineers have moved away from a single overloaded BI instance towards a split design that separates critical reporting tasks from exploratory workloads. While this case study reflects Carousell’s experience, it mirrors a broader trend seen in cloud data stacks.
**When BI becomes a compute bottleneck:**
Modern BI tools offer the flexibility to define logic within the reporting layer, which can expedite initial development but also shift computational pressure away from optimised databases to the visualisation tier. At Carousell, heavy analytical tasks were burdening the system, with large datasets causing slow execution paths. This resulted in high query latency, disrupting business operations and stakeholder meetings.
**Separating stability from experimentation:**
In response to these challenges, Carousell engineers opted to reevaluate the distribution of computational workloads. They transferred heavy transformations upstream to BigQuery pipelines, where database engines are better equipped to handle large joins. The BI layer was then reoriented towards metric definition and presentation. This shift led to the creation of two distinct BI environments – one for pre-aggregated executive dashboards and weekly reporting, and the other for exploratory analysis. This segregation helps maintain performance and reliability in both areas.
**Governance as part of infrastructure:**
To ensure stability, Carousell implemented stronger release controls in the new environment. Automated checks through tools like Looker CI and Look At Me Sideways (LAMS) were introduced to validate modelling rules before deployment. This not only catches SQL syntax errors but also enforces documentation and schema discipline, reducing the risk of errors and maintaining clear data definitions.
**Performance gains – and fewer firefights:**
Following the redesign, Carousell’s analytics team saw significant improvements in query times, with the 98th percentile query times dropping from over 40 seconds to under 10 seconds. This has transformed the way business reviews are conducted, allowing stakeholders to focus on evaluating real-time data rather than troubleshooting dashboards. By separating presentation, transformation, and experimentation, the analytics team has been able to reduce fragility and ensure predictable reporting outcomes.
In conclusion, the key lesson for teams scaling their analytics stacks is to establish clear architectural boundaries and determine which workloads belong in the warehouse versus the BI environment. By adopting a structured approach to data governance and performance optimisation, companies can enhance their analytics capabilities and drive better business outcomes.
