Enabling superior business decisions
DATA WAREHOUSING IT
PRESSURE INCREASES FOR QUALITY DATA REQUIREMENTS
As the business world becomes more competitive and information-intense, large corporations are depending more and more on being able to process data reliably and quickly.
Carleton’s data warehousing products MetaSuite and MetaSuite Parallel Edition offer large companies the capability of bringing data from IT environments that are often quite complex and heterogeneous, and then delivering it to the end user in a uniform integrated format. This provides a sound basis for the company’s decision making.
The key to its system is metadata, which allows users of the database to keep track of the history of the data. This way, they can be sure that the numbers in front of them are reliable and have been processed correctly. As Franz Bolk, part of Carleton’s management team explains: ‘You get the data in the right format, at the right time, and you get the right information.
Its biggest client to date is a large national bank, who use a Sun Environment with 48 parallel processors, and an Oracle 8 database to process 6.4 terabytes of data. The system is not only useful for large clients, though. Bolk estimates the minimum size at which MetaSuite breaks even at 150 function points.
On the other hand its client Mercedes Benz uses the system on a smaller 10Gb Data Mart.
MetaSuite can handle complexity as well as size. Another bank needs to extract data from over 600 varied systems distributed across a country, load them onto an IBM MVS mainframe to get it together and clean it, and then load it on to a Sun Data Warehouse to prepare it for analysis. It is then sampled on another IBM mainframe Data Mart and downloaded onto a Windows NT and UNIX consuming layer.
The main benefit of MetaSuite is that it makes the most of parallel systems. ‘If I have a system with 50 processors,’ says Bolk, ‘in theory the system runs 50 times faster. We lose 10% for the interprocessing communications, but that is normal. In fact 10% is excellent.’
Speed is a major advantage of MetaSuite. It can process in half an hour as much data as a mainframe can handle in seven.
Another major advantage is quality assurance. The user can ensure the quality of the data, as is possible on most systems. However, as Bolk explains, ‘We look at data warehousing as a process, so you have quality assurance about the process as well. ’This means that an auditor can check the history of the data warehouse, and be certain that it has been correctly processed throughout, by consulting the metadata. ‘This is one very unique feature,’ adds Bolk.
The MetaSuite Parallel Edition brings a new dimension to quality assurance. It enables the user to set up a ‘flow’, instructing the processors to perform several tasks one after the other. Also, half the processors can be instructed to perform one operation, and the others to perform another. This means that quality control algorithms can be built into the applications, to ensure quality in real time.
MetaSuite also offers considerable ease of installation and use. For a known system, the company usually aims to install the system in one day. Earlier generations of data warehousing software required considerable amounts of COBOL programming by hand, or didn’t offer the levels of functionality that companies require. MetaSuite generates applications which are ready to run, and require no COBOL expertise.
This contributes to the most significant of MetaSuite’s advantages – its cost-effectiveness. It needs no code to be added by hand, so Bolk estimates the effectiveness as two hours per function point, which keeps cost-of-ownership down. According to Bolk the nearest competitor takes six hours. Maintenance is usually estimated as a fixed proportion of cost-of-ownership, and MetaSuite offers proportionate savings in this department too.
With faster, more reliable data warehousing at lower cost, the MetaSuite and MetaSuite Parallel Edition offer packages that no organisation looking for large-scale information processing capacity can afford to ignore, particularly as in this area, as Bolk says, ‘A bad decision can cost millions.’