Delivering Business Intelligence

In my, I talked about the importance of Business Intelligence and Analytics in organizations and its place in executive decision making. However, I also did mention that in organizations a big challenge is to integrate fragmented pieces of data together to produce a coherent picture which makes sense. With the increased focus around data insights and sudden proliferation of many data sources, analytics teams find themselves pushed to innovate to manage data creatively.

Transactional data is what ultimately metamorphoses into Big Data leading companies to another challenging problem - how to structure and store tons of information flowing in every day. Rather than thinking about handling big data when it becomes big data, we need to be thinking more about handling transactional data while it is becoming big data. This leads us to think about what are our different data sources – are they online, offline, social etc. Do the data sources allow you to directly link to them or do you need to re-build data connections every time there is a data refresh? Consider a problem where we need to forecast hiring spend for next quarter based on new skills needed in the team and continue to do the same for the next 2 years. The ideal solution for any executive will be if reporting analytics can directly lead them to the answer in this ever changing market landscape. However, the reality is far from this. Analytics teams struggle with everyday problems of data refresh from offline and online sources, challenges of working with very large data sets, combining different structural data, making sense of it and ensuring they are delivered with high data quality within timelines of business rhythm.

Large data sets and varying data structures lead to problems with analytics design, automation and possibly in most cases issues with input into tools that help generate insights. I call this a problem of ‘Engineering Excellence’ in analytics teams. This is the part were the largest amount of time is spent – designing and creating actionable insights from myriad data sources of varying forms and usability.

The second issue is the issue of operationalizing business insights on a regular basis to decision making bodies. It includes repetitive delivery of insights with updated information, maintaining data quality and continually adding value. Once the problem of ‘Engineering Excellence’ is solved, this problem of ‘Operational Excellence’ should hopefully merely remain a discipline issue but in the absence of the first, this is where your resources struggle the most.