Failure in data Warehouse

Want to Seriously Throw Good Money After Bad? Start a BI Project! On a group thread on LinkedIn recently I witnessed several consultants arguing over which ETL tool is best for building data warehouses. They all discussed the same, tired old approach they’ve been doing for years, completely oblivious to a few key stats:- 1. According to Gartner, 76% of the budget of any BI project is sunk into the data warehouse. 2. Also, according to Gartner, 70-80% of BI projects fail. 3. Of those that succeed, many are over time or over budget. And yet these same consultants are continuing to use those same ‘best’ ETL tools and approaches to ‘deliver’ BI projects, just like lemmings jumping off a cliff. So what’s going wrong? Let’s look at what’s currently happening... You decide to start your BI project. It is deemed that a Business Analyst (1st consultant) talks to the business to ‘get an understanding’ of their requirements. The BA give her results to a Solution Architect (2nd consultant) who creates the design and gets a Data Modeller (3rd Consultant, but possibly the same guy) to design a data model. Now the likelihood is, he will design a set of star-schemas - because that’s what you’ll need for the reports, right? Also, they can be delivered straight away for a quick win with the business. So the Data Modeller gives his design to a DBA (4th Consultant) to create the schema in the data warehouse server, and to the ETL developer (5th Consultant) to write the ETL code to populate the schema in the data warehouse. Once the schema is present, a report is written by the BI Developer (6th Consultant). Of course, all of this is overseen by the Project Manager (7th Consultant). Now, fast forward a few months to the first deliverable being revealed to the business. More often than not, the solution isn’t what the user expected, wanted or signed-for. Changes are required. Why? Usually, a business user doesn’t understand what is possible or what is required in a data warehouse. Only when they see a result do the pennies drop, and they realise they really needed something else. Either that, or during the time it took to deliver the report, the world moved on and focus changed. Whatever the reason, you must go back through that convoluted process above to make the changes. Now, did I mention that when the ETL Developer is writing his code, he has a problem. If he needs to get data from lots of data sources, he must do some further ‘manipulation’ in order to shoehorn it into a star-schema (a star-schema, by the way, is a table design made up of facts (things we want to measure: Invoice amount, quantity etc) surrounded by dimensions (how we want to sort and slice that data: Customer, Region, Product, Date, Sales Rep). Put simply, you cannot take data from multiple sources and just put it into a star-schema. A lot things must happen first. Bill Inmon, the Father of Data Warehousing, has written numerous books on this science, but in a nutshell he says you should make the data generic and put it into its Third Normal Form. Few people know how to do this properly, and this is where, I believe, that 76% of money and 80% failure rate begins. Your ETL Developer writes a lot of convoluted code on he understands which becomes a nightmare when changes are required. So how do we fix this? Well, I should be clear that Software Development Life Cycle (SDLC) as a development approach does NOT work for BI projects, since they tend to be fluid in nature and more...

  • If your data warehouse failure involves a RAID array and you need RAID data recovery you should contact a professional data recovery service like Data Medics.

You might also like

7 SQLiteAdmin - Summary of Data Definition Language Functions
7 SQLiteAdmin - Summary of Data Definition Language Functions
File-Only vs In-Database MapReduce
File-Only vs In-Database MapReduce
McGraw-Hill Professional Cardiac Surgery in the Adult, Fourth Edition
Book (McGraw-Hill Professional)
  • An update of the most recent surgical perspectives and techniques in ischemic and valvular heart disease, diseases of the great vessels, cardiac arrhythmias, and...
  • New and refreshed content on pivotal trends and topics, including the use of robotic surgery, minimally invasive valve and coronary artery bypass surgery, stem-cell...
  • New full-color design--with many illustrations enhanced expressly for this edition--facilitates the comprehension of surgical procedures throughout
  • Important opening section on the fundamentals of cardiac surgery, from its origins to the surgical anatomy of the heart, cardiac surgical physiology, risk stratification...
  • Key chapters on the perioperative and postoperative management of cardiac patients from internationally recognized experts in the field

Survey: IT and business at odds on defining big data success

— TechTarget
Data warehouse vendors will feel the burn. Hadoop and NoSQL products probably won't replace the enterprise data warehouse, but they are competing for workloads.

Leviton Leviton 51120-3R 120/240 Volt Panel Protector 4-Mode Protection, Light Commercial/Residential Grade, In NEMA 3R Enclosure
Home Improvement (Leviton)
  • 120/240 Volt Panel Protector 4-Mode Protection
  • Real time diagnostic visual indicator shows power and suppression status for each protected phase
  • Standard J-Box metal enclosures
  • Compatible with Decora Home Controls
  • Limited Lifetime Warranty
Western Digital WD Red 3 TB NAS Hard Drive: 3.5 Inch, SATA III, 64 MB Cache - WD30EFRX
Personal Computer (Western Digital)
  • Capacity: 3 TB
  • Rotational Speed (RPM): IntelliPower
  • Microsoft operating systems prior to Vista, 32-bit operating systems, and Mac systems prior to OSX 10.4 may not support volumes greater than 2TB. To recognize the...
  • Interface: SATA3
  • Data Transfer Rate: 6 Gb/s Buffer to Host (max); 145 MB/s Host to/from drive (typical)
  • Form Factor: 3.5 inch
  • Read/Write: 4.4 Watts
  • Standby: 0.6 Watt
CRC Press Root Cause Analysis: Improving Performance for Bottom-Line Results, Fourth Edition
Book (CRC Press)
  • Used Book in Good Condition
Omron Omron 10 Plus Series Upper Arm Blood Pressure Monitor with ComFit Cuff
Health and Beauty (Omron)
  • Omron Healthcare sincerely apologizes for any inconvenience caused with connecting the BP791IT with Microsoft HealthVault.The drivers are now available for download...
  • Calibration Check System
  • Memory Storage: 200
  • Irregular Heartbeat Detector
  • Comes as Two User Mode