The AWS Data Ecosystem Grand Tour - Business Intelligence
Written by Alex Rasmussen on April 2, 2020
This article is part of a series. Here are the rest of the articles in that series:
- Introduction
- Where Your AWS Data Lives
- Block Storage
- Object Storage
- Relational Databases
- Data Warehouses
- Data Lakes
- Key/Value and "NoSQL" Stores
- Graph Databases
- Time-Series Databases
- Ledger Databases
- SQL on S3 and Federated Queries
- Search
- Streaming Data
- File Systems
- Data Ingestion
- ETL
- Processing
- Data Interfaces
- Training Data for Machine Learning
- Data Security
- Business Intelligence
We've spent the past 21(!) articles in this series looking at all the different ways you can store and manage data in AWS. In this final article, we'll look at one of the most important things organizations do with their data - analyzing it and using it to make decisions.
The practice of using software to analyze and present data in order to drive business decisions goes by the somewhat unfortunate name business intelligence (BI for short). Data visualization, statistical analysis, report generation, data preparation, and dashboard creation all take shelter under the broad umbrella that is BI, although which tools qualify as BI and which don't often depends on who you ask. These same tools are also often called data analytics tools, in case that wasn't confusing enough. Naming things is hard.
There are a number of popular BI tools out there, all of which have some degree of integration with AWS. Of course, AWS wants to keep its customers snugly nestled within its ecosystem, so it created a BI tool of its own, called Amazon QuickSight.
In QuickSight, as with any BI tool, users can generate dashboards and run ad-hoc analyses. QuickSight uses a custom in-memory columnar engine called SPICE (Super-fast, Parallel, In-memory Calculation Engine) to do its calculations when aggregating and visualizing data. All data that doesn't reside in databases must be loaded into SPICE before QuickSight can visualize it. QuickSight can operate on database-resident data directly as well, but it's a lot faster if that data is loaded into SPICE first.
As you might expect, QuickSight integrates pretty seamlessly with the rest of the AWS ecosystem. It has built-in integration with RDS, Redshift, and Athena (which opens up querying data in DynamoDB and S3). It's also got a tool called ML insights that can perform anomaly detection and forecasting on metrics and generate natural-language summaries of charts. Once you've created a dashboard in QuickSight, you can share it with colleagues or embed it in a webpage.
And that's ... pretty much it. While QuickSight has been seeing some development attention recently, such as its upcoming integration with Amazon SageMaker, it's still missing a lot of more advanced features present in more established BI tools like Power BI and Tableau.
QuickSight has two editions, standard and enterprise. In standard edition, you pay per user per month, and all users can create and publish dashboards. Enterprise edition splits users into two roles: authors and readers. Authors, like standard edition's users, can create and publish dashboards and are billed by the month. Readers can only view and interact with dashboards, but they're billed by the session (roughly equivalent to 30 minutes of activity) instead of by the month. If you're in an organization where a few people author dashboards that are viewed by the rest of the organization, this pricing model will likely save you some serious cash. Additionally, a bunch of QuickSight's more advanced features - like ML insights and dashboard embedding - are only available in enterprise edition.
Both editions include 10 GB per author of SPICE capacity, with the option to purchase more SPICE capacity (which is shared between your organization's users) for an additional fee per GB-month. The anomaly detection portion of ML insights charges an additional fee per thousand metrics processed.
Thank You for Reading
This concludes the AWS data ecosystem grand tour! This was a huge undertaking: it took over 24,000 words and 22 articles, but we covered (pretty much) every data-related service in the AWS ecosystem. I've learned a lot from it, and I hope you have too.
If you'd like to get notified when I write new articles, please subscribe to the newsletter by entering your e-mail address in the form below, or subscribe to the blog's RSS feed. Whether you're just getting started with AWS or have an existing deployment, if you have questions you'd like answered please reach out.