Data Warehouse requires disk storage

Data warehouse applications have been around since the late 1980s, and their main purpose was simply to take raw data and turn it into a sensible format for business or financial reporting. Over the years, however, the demands on reporting systems have changed dramatically. To name a few,

• We’ve moved toward real-time data processing
• Data footprints have increased
• The number of users has exploded
• Queries are ad hoc, not always planned in advance
• Updates occur over the course of the business day, not just after hours
• Usage is 24x7x365

These demands place massive performance pressures on the IT infrastructure to support it, and traditional spinning-disk storage can no longer keep up. Disks are made of moving parts, and this configuration greatly inhibits the speed at which they can process data, leading to massive performance problems when any physical I/O is required—especially under heavy load. Batch processes have started to reach unacceptable levels, leading to missed SLAs. Parallel reporting is starting to bring systems to their knees, and the demand of moving toward real-time processing is constrained by an inability to process data fast enough.

All-Silicon Array to the Rescue

All-silicon architecture is designed specifically to address the speed, scale, administration, concurrency and TCO issues plaguing modern infrastructures. By allowing every memory address to be equally accessible at the same great speed, flash fixes the problems that disks are creating. With little to no administration or tuning, all data will flow at flash memory speeds regardless of the locality or the number of LUNs, databases or users. Here are the top five reasons to fast-track your data warehouse applications on flash:

1. Reduce data imports from hours to minutes.

As data sets grow in size, imports have started to take longer and longer to run, breaking SLAs or causing overnight jobs to breach production traffic the next day and thus affecting user experience. Flash storage has three major attributes that can reduce data imports from hours to minutes: super low microsecond latency, high throughput and massive parallelism. With flash storage, a data warehouse can ingest data from 2x, 3x or 4x more sources all in parallel and write data in microsecond latency to reduce batch jobs. For example, data imports have fallen from eight hours to 60 minutes; the best I have seen so far was one week to one day! This change reduces risk for a business and makes data available earlier.