Overview
If there is one trend common to businesses over the past one hundred plus years, it has been the acceleration in the pace of business. I remember a time in the not too distant past when an organization’s annual results were greatly anticipated.
Today we look at quarterly results for many businesses and in some cases monthly or even weekly results. Only a few decades ago, if travelling for business and someone was attempting to contact you, they knew to wait until your return to the “office”.
With the advent of cell phones and laptops now you are expected to be always available and able to respond within a few hours. As a colleague recently said, “If I give you a cell phone and a laptop, you are your ‘office’.”
Much of this increase in the pace of business can be tied directly to the use of technology. Information systems have enabled the real time entry of data from operations that was once captured via paper documents and manually transcribed or entered via a separate data entry process.
Consider business activities such as the stock market indices, in the recent past you would get those results via the evening news or the next day’s newspaper, now these results are available to anyone anywhere at any time via an app on your cell phone.
Perhaps the clearest example of this increase in the pace of business to which everyone can relate is the rise of Google. Google offered the ability of any ordinary person to find information about anything existing in the world of the Internet without any need to understand the source of the data or how it was derived.
Google surpassed rivals such as Yahoo! who required users to navigate a series of menus. Those menus required a user to know where to look and were more time consuming as you navigated through their levels of webpages.
In the world of business many organizations have exploited the capabilities enabled by information technology and implemented software as a core function of their business. One of the primary means by which this occurs is through the use of Enterprise Resource Planning (ERP) systems.
ERP systems collect all the information about activities in a business in much the same way Google does for the Internet. What an organization needs to leverage its ERP system is a Google-like means to understand the ERP data and present it in a decision enabling format without the user of that data needing to know what goes on behind the curtain. These types of systems are available and typically referred to as Business Intelligence Systems.
Business Intelligence Systems for ERP
CIO magazine describes Business Intelligence as a combination of Decision Support, Data Warehousing, and Data Mining. We will look at each of these areas and the skills and capabilities needed by each. We are going to specifically explore the challenges that arise when an organization attempts to roll its own Business Intelligence System versus acquiring the expertise and content from a specialist (i.e. Create your own Google or leverage what Google has already created?)
Decision Support
CIO Magazine defines Decision Support as a capability that is purpose built for support specific decisions. This could be data arranged in a tabular form, sorted by values from highest to lowest. It is often best presented graphically such as in line chart, bar chart or pie chart.
Figure 1. above illustrates how this works in a line chart format. Actual sales (green line) are plotted against a sales target (orange line). Very quickly one can see visually how sales are occurring by month and the orange line gives a reference point as to what sales are expected to be. The green line drops to zero for future months as sales have yet to be recorded for those months.
Figure 2. above illustrates a similar view but shows sales for the current year (blue line) versus sales for a prior year (orange line). As in Figure 1, the comparison is very quick and intuitive. It provides actionable insight and answers the question on how are we doing in sales this year versus last year.
These two extremely simple examples mask a great deal of complexity that underlies a Decision Support System that is not at first obvious. Consider that the following parameters and attributes need to be examined and thought through before even simple charts like these can be created. Those include:
- What are Sales? – Is it Orders received? Is it Products shipped? Does it include returns?
- What time period is important? – Weeks, Months, Years?
- What measure is important? Dollars? Quantities?
- How do we aggregate these? By Geography? Product Line? Salesperson?
- Where and how do we measure and set targets?
- How do we display the data? Via line charts, bar charts, pie charts, etc.
The reality is that many decisions need to be made up-front for an appropriate Decision Support System to provide actionable insights about the business. These decisions if not thought out in advance can result in false starts, reports or graphs that don’t tie back to the questions the business needs to answer and result in multiple iterations in trying to get it right.
This is essentially what often happens when an organization acquires a BI toolkit and expects the users to create their own reports or Decision Support Systems. Even simple reports as illustrated above can take one or two days to create. More complex reports that allow filtering of data can take a week or more. The result is that time is lost and users often simply give up after they realize the time and expertise involved in creating a valuable Decision Support System is beyond their means.
Data Warehousing
A data warehouse is defined as a large, centralized collection of digital data gathered from various units within an organization: The intent of a data warehouse in a Business Intelligence System is to capture the data from the ERP system, and any other relevant system, and organize it to support the Business Intelligence needs of the organization.
To do this effectively is beyond the expertise and skill set of the typical business user. To effectively create a data warehouse to enable a Decision Support System the following competencies are required:
- Understanding the ERP’s database model. All ERP systems are built on top of a database system. In much the same way a gasoline-powered automobile relies on a petroleum company to refine and provide gasoline of a certain type and quality, so an ERP system relies on a database to provide and manage the data for it.
There are multiple database vendors and providers, but in an ERP system, the database system takes care of a variety of back-end functions for the ERP such as back-up, referential integrity, data access language, table structures, etc. To create a data warehouse from the ERP database requires an understanding of the tables the ERP system accesses, how those tables are related, the data elements in those tables, and how to access that data via a data access language. - Understanding the ERP system. It is also necessary to understand the ERP system and how it records data that is entered into it from the different modules. One of the key benefits of ERP systems is that they collect data from multiple functional areas of the business and consolidate and track this data in a consistent manner. But each ERP system is different and how they record this data, especially in situations where standard transactions are only partially complete.
Some typical examples of this including shop floor jobs that are not able to complete within the duration of a shift, customer returns, partially shipped orders, rework jobs, quality defect issues, etc. How the ERP records these types of transactions and how to extract this data out to populate a data warehouse requires an in-depth knowledge of the ERP System. - Understanding Point in Time and Summarized Data. A data warehouse can be designed to not only capture the transactional data coming from the ERP system but to organize it in a way that the ERP system cannot. ERP systems are often very transaction centric and can show what orders were placed today, what is the status of inventory today, what items are being shipped today, etc.
However, if an organization wants to look backwards and understand something that may have occurred in the past at a given point in time, it becomes almost impossible for an ERP system to reconstruct such information. Whereas a data warehouse can be designed to capture this type of data at a given point in time and store it for future access. Typically, such things as inventory value on a specific date, cash on hand at a point in time, jobs completed on specific work centers at specific dates, etc. can all be captured and saved for future reporting and access in the data warehouse.
As illustrated above, a capability and knowledge of data warehousing are essential to creating an effective Business Intelligence System for your ERP. This capability is beyond that of an average business user and is time-consuming and costly even for the IT resources in many organizations.
Data Mining
Data mining sounds hard and complex but in actuality is fairly straightforward. The definition of data mining is the process of collecting, searching through, and analyzing a large amount of data, as to discover patterns or relationships. There are really two ways to accomplish this 1) Use a sophisticated set of analytics tools to comb through data and discover patterns or connections that are not obvious, and 2) Have a knowledge of the typical actionable insights that the business wishes to gather and pre-build reports or graphs to specifically provide those insights.
To support the first approach tools such as IBM’s Watson Artificial Intelligence (AI) capability have been built and are quite robust and capable for this task. One typical example of this type of AI technology at work was illustrated in a study done in retail, where a retailer wanted to know what product was most often also bought with diapers, the answer being beer, was surprising to some, but perhaps not to parents with infants.
These types of data mining systems often require deep expertise in AI, data mining, and data science. In addition, extremely large databases are also needed to support this capability. Users of IBM’s Watson AI system report that multiple hours are spent organizing the data in the system, determining what data should be in the system, and populating that data into the system. The resulting size of the supporting databases are often in the mega terabyte range.
To support the second approach a knowledge of what the organization really needs to know from its data is important. If one has worked across multiple similar organizations, then this type of request is often repeated, because most companies are focused on the same business metrics and are looking for the same insights from their data. Such questions as “What was my inventory on this date?”, “What items did we ship to this customer this month?”, “What Purchase Orders are past due?”, etc. are the types of operational questions that arise everyday and the answers to them are critical to the ongoing operation of the business. If these types of reports can be prebuilt and are robust enough to allow a level of filtering and sorting of the underlying data, then this second or prebuilt approach to answering analytical questions is often sufficient to meet the needs of the business.
Having a Business Intelligence System that has been prebuilt to provide these types of actionable insights that can be leveraged across multiple similar businesses can be a rapid and cost-effective way to “mine” your existing ERP system’s data.
Summary
We have looked here at the various components and requirements of a Business Intelligence System to support an ERP system. The net is that we have discovered that it requires a depth of expertise and a variety of resources to successfully implement such a capability. Some of the typical types of capabilities and resources needed are as follows:
- Knowing what questions to ask
- Knowing what presentation format best suits the question being asked
- Knowing how the data is organized
- Creating a data warehouse separate from the ERP database
- Knowing how the ERP system records events, transactions, and activities especially when those activities don’t complete in a normal daily cycle of events
- Knowing how to “mine” the data warehouse to gain appropriate actionable insights
Many organizations do not have the resources in people, time and expertise to implement a Business Intelligence System based on the above set of requirements. To acquire those skills takes time and an investment in the technology and expertise.
As discussed at the beginning of this article, what made Google the ultimate winner in its space was the speed with which it delivered results. That Speed to Value, eventually won out over a very large base of competition. In every organization there is a clock running and that clock tallies up costs for an organization in a relentless fashion. The employees are collecting a paycheck, the machines that run are using electricity, the rent is accruing on the building and on and on. All of these costs continue to accumulate whether a company is producing and accruing revenue itself.
Having a Business Intelligence System that is prebuilt by an organization with knowledge of the ERP system, with knowledge of data warehousing, and with a portfolio of prebuilt dashboards and reports to mine that data based on the needs generated by other similar businesses, is the most cost effective and rapid path to value.
Come talk to us, because the above paragraph describes exactly what we do.
