Writing ยท May 2018

Goldilocks Criteria: Business Intelligence Platforms

The 'just right' feature set for a BI platform, from integrated warehousing to governance, and why most tools still miss it.

From the archive. The tools have changed since 2018, and the "AI/ML ready" section reads quaint now, but the criteria still hold up.

This is the first in a series of posts on selecting data tools and infrastructure. The format: I list the "perfect" feature set for a dream product. These features rarely exist in a single solution, but if they did, I'd use it. First up: business intelligence platforms.

For this discussion, BI platforms are data platforms that let non-technical business users explore, prepare, and present data germane to their work. They're crucial for informed decisions, and it turns out this is ancient stuff: the term dates to the 1860s, when Richard Devens used it to describe how the banker Sir Henry Furnese gained an advantage over competitors by acting on the information around him.

We've come a long way, from cumbersome on-premise solutions to nimble cloud platforms designed so professionals without a technical background can glean insights easily. BI tools serve a specific purpose: analyzing and reporting on data consolidated from various sources. Some sit on top of separate warehouses; some modern platforms serve as the data store too. They pack a ton of functionality but are typically narrow-scoped: they inform actions taken on other platforms rather than executing anything themselves.

A great use case: OKR dashboards for your company, teams, and individuals, where anyone can pull up a live view of progress toward their goals, anytime, on their phones, with no help from analysts or IT.

Here are the Goldilocks ("just right") criteria I look for:

Integrated data warehouse

Traditionally, BI tools sit on top of separate platforms managed by engineering. A newer class of products lets you upload or connect your data without engineering support, which is a huge advantage: moderately technical users get running without distracting external resources. (Self-service creates data governance challenges, but that's another story.) Imagine joining all your Google Drive spreadsheets with live connections to Google Analytics, Meta, and financial data, then exploring freely. That's what these platforms do.

Data engineering for dummies

The best data scientists I've worked with estimate they spend 80 to 90 percent of their time on data hygiene before analysis can begin. So any functionality that makes data manipulation easy is a huge value add: joining data via drag and drop, changing types with a click, deduplicating rows without SQL. Each one extends the range of users who can go deep without assistance.

Live data, from the cloud, on your phone

Data that arrives attached to an email is DOA. This is an absolute pet peeve: once people start offline discussion and editing, multiple inaccurate versions of the same data set are inevitable. BI tools need a live backend at all times, with dashboards that are (pseudo) real-time and clearly timestamped. And the platform should be mobile-centric. There's nothing more powerful than pulling up live data on your phone mid-conversation.

AI / ML ready

We're in the earliest innings, but your platform should have the foundation for automated machine-learning-driven insights. They're rarely valuable out of the box, but there's no sense investing in a platform that isn't actively working on them. As a start, I want basic statistics on onboarded data: distribution and correlation reports, then simple predictive analytics. Separately, invest in your team's data fluency. Leveling up the people is always more valuable than standing up a whiz-bang technology solution.

Narrative and collaboration focused

A perfect platform supports metrics-backed storytelling, not just shared pie charts. As a product owner, I want to explore a data set and build a coherent, sharable narrative around it: an online presentation with live data at different altitudes, supported by text, images, and annotation drawn directly on the data. It should also host the conversation: unlimited named users, threaded comments, action items, @ mentions.

Governance gone wild

Sad to say, this is critical. Supercritical. The moment you create your second dashboard, you need governance or you'll never find it again, or know whether its data is current, approved, and official. The smart approaches center on clear labeling of data, origins, and duplicates, easy validation of "official" data sets, and an integrated data catalog showing breadth, lineage, approvals, and errors.

User-level data FTW

BI typically plays at the aggregated, anonymous altitude: content, page, campaign, location. Rarely the user level. In a perfect world, a graph model at the atomic data layer would allow pivots at all those altitudes plus the individual user. A newer breed of system, the Customer Data Platform, is jumping into this gap with the promise of a single view of the user. Marketing and sales teams use CDPs today, but the application of that granular view to BI use cases is immense. Perhaps CDPs are the subject of the next post in this series.