The purpose of business intelligence (BI), as conceived in the early 1990s, was to give business users direct access to information instead of having to go through the IT department to obtain custom reports or views of information. The idea was to empower business users by allowing them to query a repository of integrated data (i.e., a data warehouse or data mart) and create their own reports. Giving business executives, managers and staff self-service access to information would enable them to avoid the backlog in the IT department and enhance their ability to make decisions, create effective plans, and optimize processes and performance.

In reality, however, self-service BI proved overwhelming for all but the most sophisticated power users. Most users found the original query, reporting and analysis tools too complex or time-consuming to use. Because they didn't use the tools extensively, they would forget how to log on or use various functions and features. They often spent too much time looking for the right report among hundreds or got lost in an endless series of drill-downs and dimension lists. As a result, BI tools replaced the IT department as the major barrier between users and the information they needed to make sound, timely decisions.

In the past five years, many BI tools vendors have responded to these complaints by making their product lines more friendly to the majority of users in the enterprise, not just power users. Consequently, they've rediscovered the value of standard and parameterized reports, and added dashboards and analytic applications that provide a more intuitive interface, dynamic customization, guided analysis and collaboration features.¹ The result is that many organizations can now confidently extend the benefits of BI tools to a majority of their employees throughout the enterprise.

Analytical Silos

Scattershot Deployments. However, despite the popularity of business intelligence, most organizations have not deployed BI tools and solutions in a systematic or consistent manner. They have allowed individual workgroups, departments and divisions to build their own data warehouses and data marts, purchase their own BI tools and define key metrics, data elements and business views in unique, non-standard ways. Thus, although BI usage has increased overall, BI deployments remain small and disconnected. Many organizations today are riddled with these "analytical silos."

Even when an organization has resisted delivering BI solutions in a scattershot manner, business events have intervened making it impossible to deliver consistent information using a standard set of BI technologies and processes for collecting and integrating data for analysis purposes. Mergers, acquisitions, reorganizations, executive turnover and other forces perpetually undermine organizations' attempts to deliver a universal BI solution and avoid analytical silos.

BI Vendors. In addition, BI tools vendors sometimes exert tremendous influence over organizations, especially the uninitiated, who believe that purchasing a BI tool is the only thing they need to do to implement a BI solution. Without strong central controls, many organizations now find themselves in possession of a half-dozen or more BI tools that are associated with different data marts and data warehouses, which contain overlapping information and redundant staff. This not only makes BI a target of cost-conscious CFOs, but it undermines an organization's ability to deliver a consistent view of information for reporting and analysis.

Data Redundancy. In short, the proliferation of analytical silos with their redundant BI tools and data structures threatens to undermine the promise of BI to deliver business insight and value to the enterprise. TDWI research shows that, on average, organizations have 2.1 data warehouses, six independent data marts, 4.5 operational data stores and 28.5 spreadmarts (i.e., spreadsheets that function as independent data marts).² Many organizations report that they have many more analytical silos than this, especially spreadmarts, which in many cases are too numerous to count.

Tool Redundancy. Invariably, each analytical silo uses a different set of BI tools, leading many BI professionals to complain that their organizations "have one of every kind of BI tool imaginable." TDWI research shows that organizations have an average of 3.2 BI tools from different vendors. Not surprisingly, the average number of BI tools increases with company size, from 2.3 BI tools for organizations with less than $500 million in revenues to 3.7 BI tools for organizations with $5 billion or more in revenues. Obviously, bigger companies have more departments and business units that purchase BI tools separately (see Figure 1).

Effect of Industry Consolidation. At first blush, 3.2 BI tools from different vendors doesn't seem like an unmanageable number, but this doesn't necessarily reflect the total number of BI tools a company has. Due to the consolidation in the BI industry, many best of breed BI tools that organizations purchased several years ago have been acquired by leading BI vendors and wrapped into a comprehensive BI suite. Thus, organizations often have two or three times as many BI tools as they have BI vendors.

The Excel Effect. These numbers also don't necessarily count Excel as a BI tool, even though it is the query, reporting, analysis and planning tool of choice for most power users and managers. The numbers would be higher if Excel was counted. Unlike Excel-based spreadmarts, Excel can be a legitimate BI tool when used as a front end to an analytical server.³ After a long period of resistance, many BI vendors are now embracing Excel, as well as other Microsoft Office tools, making them full-fledged BI clients in packaged suites. This greatly aids BI standardization efforts.

BI Tools By Category. The proliferation of BI tools is more obvious if we count BI tools by category instead of by vendor (see sidebar: Standardizing on Categories of BI Tools). The TDWI survey* shows that organizations average almost three production reporting tools, three OLAP tools, two dashboard applications, two end-user query and reporting tools, 1.5 data mining tools and 1.5 planning/modeling tools. Combining these figures with the previous chart, organizations have an average of 13 BI tools (see Figure 2).

Category Consolidation. The good news is that organizations plan to standardize on one BI tool per category within three years (see Figure 2). Organizations are most keen to standardize OLAP tools, which will decline to less than one tool per organization (.99) in three years, suggesting that some organizations are abandoning OLAP tools altogether! Organizations will have less success standardizing production reporting tools (1.4 tools in three years) and planning/modeling tools (1.3 tools in three years). Production reporting tools are traditionally purchased by the IT department and may escape corporate standardization efforts. Planning/modeling tools are currently dominated by Excel and Access, and it is difficult to curb usage of these tools.

Expanding the BI User Base

While organizations are trying to reduce the number of BI tools they possess, they are also looking to expand the number of users who leverage BI tools to make decisions. Expanding the use of BI from power users to all users is a big priority for organizations that want to empower knowledge-workers with relevant and timely information to make quality decisions and improve performance. On average, organizations plan to boost the percentage of employees with BI tools from less than half (41 percent) today to almost two-thirds (60 percent) in three years, a 50 percent jump (see Figure 3).

While trying to expand the base of BI users, organizations also want to increase adoption rates among existing BI users. As mentioned, many employees found earlier generations of BI tools difficult to use and abandoned them. Organizations have compounded the problem by not providing adequate training, erroneously assuming that users "could figure it out." Consequently, many organizations report having lots of BI shelfware - BI licenses that go unused or underused.

Today, only 45 percent of users assigned a BI tool use it on a regular basis (i.e., weekly). In three years, organizations hope to expand regular BI usage to two-thirds (65 percent) of users with access to a BI tool. Doing the math, this means that, on average, 39 percent of all employees will be active BI users in three years, a 111 percent increase from the current rate (18.5 percent). Even if our respondents are overly optimistic (which is usually the case), this is still significant growth.

Characteristics of Enterprise BI Tools. The key to increasing the penetration of BI tools is multifaceted. However, it's imperative that organizations deploy BI tools that are fast, intuitive and customized to a user's role in the organization. The BI tools must also provide access to timely, relevant, and accurate information and be able to reach into operational systems, if required. To date, most BI solutions have fallen short of this ideal.

Push Approach. In scaling BI environments, organizations often try a "push" approach in which they convert a standard report to a PDF or Excel document and e-mail it to multiple recipients in either an automated or manual fashion. This lets organizations deliver the output of BI tools to additional employees without increasing their BI licenses. Not surprisingly, BI vendors are now charging "recipient" licenses for these indirect users, which is slowing adoption of this approach. Additionally, recipients sometimes think these "pushed" reports are spam. According to the TDWI survey, the percentage of recipient BI users will decrease by 14 percent during the next three years.

Pull Approach. Instead, the trend is to empower users by giving them interactive, Web-based BI reports via portals, dashboards and scorecards that minimize information overload and guide users to the most relevant information. The Web has been a boon to BI because it expands BI's reach to anyone with a browser, including customers and suppliers. Also, Web-based BI tools eliminate the need to install software on users' desktops, speeding implementation times and reducing support costs.

These trends are reflected in TDWI data. The percentage of direct BI users will increase from 42 percent to 48 percent in the next three years while the percentage of recipients will decline inversely. In addition, the percentage of Web BI users will increase from 55 percent today to 70 percent in three years while the percentage of desktop users will drop from 45 percent to 30 percent in three years.

Taking Action

Migrating to Enterprise BI. TDWI research shows that two-thirds of organizations (66 percent) are now trying to transform business intelligence from a departmental solution to an enterprise one. Only 17 percent of organizations have completed the task, while the remaining 17 percent will continue to deploy BI departmentally. This data shows that business intelligence has not reached maturity in most organizations (see Figure 4).

What is Enterprise? Of course, the term "enterprise" is a tad ambiguous, which you need to keep in mind when interpreting the survey data. An enterprise solution is an ever-widening circle that depends on one's perspective. For instance, people who work in a worldwide business unit or a functional area that spans all business units might consider their group's data warehouse and BI tools to be an "enterprise" solution. However, the CEO of the corporation might see these "enterprise solutions" as analytical silos if they don't deliver a consistent view across the entire business.

In effect, one organization might have multiple "enterprise" BI solutions and no single version of the truth. Ultimately, an enterprise BI solution needs to span the needs of the entire organization across various functional areas and user segments, addressing both front-end and back-end architecture and delivery methods. This is necessary because the CEO will eventually want an integrated view of the entire organization. This is the real definition of "enterprise."

Indicators of Success. One of the major indicators that organizations are on the right track toward delivering an enterprise BI solution is their ability to standardize on BI tools. Using BI tools as a gauge, some organizations have made progress toward enterprise BI, but a majority have a long way to go.

Today, one-third of organizations have standardized on OLAP and production reporting tools, and only approximately one-quarter have standardized on end-user query and reporting tools. An even smaller percentage has standardized on dashboards/scorecards, data mining tools and planning/modeling tools (see Figure 5).

Within two years, a majority of organizations will have established standards for OLAP, production reporting, end-user query/reporting tools and dashboards/scorecards, which are currently experiencing explosive growth. Because data mining tools are only used by a fraction of end users, few organizations are in a rush to standardize these tools. Planning and scenario modeling applications are dominated by Excel and Access, which accounts for the slow rate of standardization in this category.

Delivering the Full Potential

Today, many organizations desire to provide BI tools that enable all workers to unlock the full potential of information to enhance their decisions, plans and responsiveness to events.

Conform to Users. The key to standardizing BI tools is make them conform to the way users want to work and not vice versa. First, this requires knowing your users - what information they need, how they like to receive it, and how much they want to interact with it. Interestingly, users' requirements change based on what role they are playing at the moment. Sometimes users require lots of information and unfettered access to explore data, and other times they simply need to review summary data on a weekly or monthly basis. For these reasons, we see lots of potential in Web-based dashboards and scorecards; they seem to be the "new face" of business intelligence that meet most users' requirements perfectly.

Second, organizations must fit BI tools to each user and the different roles they play. One size does not fit all. Different users require different BI tools and often different modes within a single tool (i.e., author, navigator, recipient). Thus, to deploy BI to the enterprise, organizations must select BI tools that can be dynamically customized to users' requirements using security profiles. The tools must also support a zero-client environment and provide adequate scalability, performance and extensibility to meet enterprise processing requirements.

Ironically, enterprise BI tools, if deployed properly, begin to fade into the background. Users "wear" them like a favorite old shirt. They incorporate the information into the processes they manage on a daily basis. With the advent of service-oriented architectures and Web services that finally offer a universal way to interconnect application components, we will see BI technology merge into the core applications that users use to manage the processes for which they are responsible on a day-to-day basis. The day that BI becomes invisible is the day it finally succeeds!

*Note: This is an excerpt of a 40-page report published by TDWI in July, 2005. All figures in this article represent results of the survey that accompanied the report. The report is available from TDWI.

1. See Cindi Howson, "Top 10 Mistakes To Avoid When Selecting and Deploying BI Tools." TDWI 10 Mistakes Series, 2004.
2. See Wayne Eckerson, "In Search of a Single Version of Truth: Strategies for Consolidating Analytic Silos." TDWI Research Reports, November, 2004 (www.tdwi.org/research).
3. See Wayne Eckerson, "Getting Control of Runaway Spreadsheets." SearchCIO.com, May 20, 2005. This article presents strategies for curing spreadmarts, including techniques for using Excel as an analytical client.