History of Business Intelligence 

Historically availability and access to data was a significant issue. 

These days, we produce such a massive amount of data, that the current problem is what to do with this data and how to use it to gain valuable insights for businesses.  

The Business Intelligence (BI) concept has progressed enormously over time due to the exponentially growing amount of data since the beginning of Digital Revolution and gradual introduction of new tools to digest and visualize data in data visualization software.  

Let’s go through the history of Business Intelligence. We will see how the concept of BI evolved through the years to come to the shape it has today.  

Pre-digital times 

The history of Business Intelligence dates back to the invention of the term “Business Intelligence” itself. It was first used in 1865 in Cyclopædia of Commercial and Business Anecdotes by Richard Millar Devens.  

At the time, Sir Henry Furnese, a banker, used the information he gathered to make better-informed business decisions. As a result, he gained a competitive advantage. Devens also quoted other examples where merchants tapped into available information to support their business strategies.  

The importance of using the term “business intelligence” in the publication lies in the fact that, for the first time, it described the use of data and empirical evidence to make a business decision instead of relying on gut instincts. As a result, such approach opened a way for a more scientific approach to businesses.  

The Digital Revolution of the 1950’s 

Despite the early apparition of the term “business intelligence”, it wasn’t until the 1950’s, the beginning of the Digital Revolution, that it became an independent scientific process supporting entrepreneurs in making business decisions.  

The 1950’s brought two milestones for BI development: 

• 1956 – IBM invented the hard disk with 5MB of memory storage that opened the door to replacing physical filling systems for digital ones, 
• 1958 – an IBM computer scientist, Hans Peter Luhn, in his journal “A Business Intelligence System”, first described the potential of using BI to generate valuable insights. 

In his essay, Luhn emphasized the ability of such systems to identify available information, find out who needs to know it, and how to distribute it efficiently. It has planted another seed for the concept of BI we know today. 

Early computers and databases of the 1960’s 

In the 1960’s there was a significant spike in the use of computers. We started to be able to collect enormous quantities of data, but we still lacked the tools or technology. There were also issues with data storage and data management.  

Nevertheless, the primary problem was the lack of a repository to bring together all the available data. Such integration is essential because dispersed data by itself doesn’t generate valuable insights.  

Information Management System (IMS) of IBM 

IBM decided to act upon it and introduced their Information Management System (IMS), a hierarchical Database Management System (DBMS). It was based on binary trees, where data was arranged in a hierarchical tree structure of a parent and two child records. 

The solution featured data integrity, security and independence. This approach enabled more effective searches and was a milestone in the direction of higher data organization and development of Business Intelligence.  

Relational model of Ted Codd 

Also, in the 1960’s, a computer scientist, Ted Codd, invented the relational model for database management. It was the theoretical basis for Relational Databases (RDB) and Relational Database Management Systems (RDBMS). 

Codd improved the way databases were created. Rather than being a simple means of organizing data they were transformed to an advanced tool for querying data and finding valuable information. His proposal of developing a “relational database model” gained popularity and was adapted worldwide. 

First vendors of the 1970’s and the concept of Decision Support Systems (DSS) 

The 1970’s were also crucial in the history of Business Intelligence. They welcomed the first BI vendors, such as SAP, Siebel and JD Edwards (both were later acquired by Oracle). Their apparition increased the availability of tools to assist in accessing and organizing the data more effectively.  

IBM and Siebel managed to develop the first comprehensive Business Intelligence systems. They started to provide structure for the vast amount of data collected during the previous years. Unfortunately, the lack of infrastructure for data exchange and incompatible systems still proved to be a big challenge.  

The late 1970’s also coined the concept of Decision Support Systems (DSS). P. G. W. Keen, a British academic, described it as computer system impacting business decisions. The idea was to support the manager’s judgement with valuable insights coming from data stored in the information systems. 

Data warehouses in the 1980’s 

Finally, the 1980’s saw the advent of Data Warehouses (DW), which also was important for the history of Business Intelligence. DWs are systems for data analysis and reporting and they were developed as businesses started to use in-house data analysis solutions regularly. 

DWs started to be used as central repositories of integrated data from one or more sources. They store present and historical data in one place allowing it to be used for crafting analytical reports and preliminary business analytics. These reports could be customized to suit the needs of individual departments. They also cut the amount of time needed to access data. Currently, DWs are still a core component of Business Intelligence.  

At the time, there appeared two different approaches to Enterprise Data Warehouses (EDW): 

• top-down design that stated that DWs should be one part of the overall BI system. Therefore, there should be one Data Warehouse and data marts could source information from it, 
• bottom-up design stating that a Data Warehouse is the conglomerate of all data marts in an enterprise and data is stored in a dimensional model. 

Despite the fact that both approaches were different, both emphasized the importance of data organization and integration from multiple locations.  

By the late 1980s, Business Intelligence tools (BI tools) were already effective tools for analyzing and reporting based on data. Consequently, the 1980s were the starting point for the 1st generation of BI. 

Business Intelligence of the 1990’s 

Development of Business Intelligence tools 

In the 90s, there was a proliferation of BI tools and related technologies. One of the most reputable was Enterprise Resource Planning (ERP). It is management software system integrating applications to manage and automate different areas of business.  

The 1990’s were also the time when Business Intelligence entered the mainstream business area. The tools gained more market and got new features such as batch-processing reporting.  

At this time, some BI services started to provide simplified tools that allowed decision makers to act more independently. Tools were easy-to-use, efficient, and had all the essential functionalities. Consequently, analysts could easily gather data and insights by working directly with BI tools.  

Business Intelligence of the 2000’s 

The 2000’s were a major highlight in the history of Business Intelligence. They were times of a huge BI development and a concentration of BI in the hands of IBM, SAP, Oracle and Microsoft.

The growing popularity of predictive analysis provided a new method of exploiting data algorithms and forecasting future business changes.  

What is more, developing cloud technologies and Internet-based software came to the fore as real-time feeds and enhanced visualization techniques changed the way data was viewed and analyzed.  

Also, Business Intelligence got a boost of brand-new possibilities and types of data to analyze. These came from the birth and rapid development of e-commerce and social media channels, like Facebook, Twitter, or LinkedIn.  

Business Intelligence of today 

After 2010, Business Intelligence has become a standard tool for large and medium businesses in many industries, from banking, through IT and finance to communications.  

Present BI tools enable business users to:

• work across various devices,
• tap into visual analytics to apply analytical reasoning to data via interactive visual interfaces.  

These days, Business Intelligence is understood as a discipline using technology to gather and analyze data, translate it into valuable information, and act on it to gain competitive edge. Consequently, modern Business Intelligence solutions allow us to make better-informed decisions in a quick and effective way.  

Data Lake and Data Lakehouse  

Nevertheless, while DWs were great for structured or historical data, a lot of modern companies need to deal with large volumes of unstructured data with high variety, velocity and volume.  

It has forced the creation of Data Lakes, which serve as repositories for raw data in a variety of formats. What is more, there was also needed the possibility to analyse this data, which, in turn, forced the creation of a metadata layer – Data Lakehouse. 

Data Lakehouses are open architectures combining the best elements of Data Lakes and DWs. They have enabled implementing similar data structures and data management features to those is DWs, but on top of low-cost cloud storage in open formats. Consequently, they are more in line with the Machine Learning and Data Science flow than the classic split into Data Warehouse and Data Lake.  

The examples of Data Lakehouses are Databricks and Azure Synapse. Moreover, if you think about how the data in Power BI service is stored – for example Power BI dataflows can be stored in a Data Lake, one could argue that Power BI utilizes a Data Lakehouse architecture. 

Business Intelligence vs. Data Analytics 

There is a common belief that Business Intelligence is the same as Data Analytics. Nevertheless, Data Analytics focuses on the process of analyzing data to draw conclusions while BI is based around the strategic decision-making processes based on data.  

Consequently, modern BI encompasses the infrastructure, tools, applications, and best practices. All of them facilitate accessing and analyzing information that is later used by managers and other business users to make key operational decisions and gain insights. 

Business Intelligence challenges 

Today we see efforts made to make BI tools and applications even more intuitive. At the same time companies invest in acquiring the skills needed to successfully utilize these tools and implement meaningful business analytics.

We observe a growing demand from enterprises for BI experts, such as:

• Data Engineers,
• Data Analysts,
• Machine Learning Engineers.

They are needed to assist top management and business users in making business decisions based on data from multiple systems.  

One of the most significant challenges for Business Intelligence today is data quality. The amount of data produced is growing at an accelerated pace. Unfortunately, in many cases the data is not organized and is of poor quality.  

Nevertheless, BI solutions and innovations, such as Data Fabric and Data Mesh concepts, are already making BI tools more collaborative and accessible. They will generate even more opportunities for companies and can improve business analytics.  

Data Fabric is a metadata-driven way of connecting a disparate collection to build a single, virtual management layer atop distributed data. On the other hand, Data Mesh stimulates disparate groups of teams to manage data as they see fit but considering common governance rules.  

The evolution of systems will surely result in more simplified and accessible reports. We will surely also see an increase in the quantity of complex data. As a result, although the history of Business Intelligence until today is known, there is still a lot to come. Let’s stay tight and see what the future holds for Business Intelligence tools, BI technologies and what Business Intelligence trends are yet to emerge.