MetaSuite PE 7.1

Analyst: Gary Cooper

Product Category: Data Warehousing

Date: July 2000

Company: MINERVA SoftCare

 

 

 

Abstract:

 

The increased global competitive marketplace has placed additional pressures on most businesses, not only to raise their competitive profile, but in many cases to maintain the position they currently hold. The effective use of corporate data is making a marked impact on the organisations that are wise enough to exploit this unique source of competitive information. Organisations who do not utilise their data wisely are driving their business forward whilst being blindfolded. This can have only one consequence, the business will crash, whilst organisations who are moving forward with both eyes open, will succeed.

 

 

 

Information as a Competitive Enabler

A company’s data is the DNA of the organisation, with its capacity to uniquely identify each individual organisation. It is the one aspect of business that is totally unique to each company. It is for this fact alone that organisations that do not exploit what is often referred to as its ‘crown jewels’ are in danger of losing ground in terms of their competitiveness. It is now a widely accepted fact that organisations must use their information effectively if they are to compete in the modern business world. The intelligent use of information can dramatically transform the business fortunes of a company, where restructuring, deregulation and industry consolidations have had such a major impact. The amount of data available on which to base such important business decisions has literally exploded exponentially. Information relevant to the competitive profile of the business must therefore be discerned from the deluge of data to which the business manager is subjected. Butler Group believes that corporate agility and competitive innovation can only be achieved through knowledge empowerment of all employees in the organisation, to give an accurate view of current business dynamics on a day-to-day basis.

The need for corporate agility means that organisations have to be more flexible and adaptable than previous business environments warranted or required. It requires both strong market awareness and the ability to provide solutions to customer’s specific needs. Companies therefore have to focus on market segmentation, and servicing different types of customers who are now, more than at any time previously, very sophisticated and demanding of good service, in order to ensure competitiveness. These core issues are fundamental in enabling organisations to work smarter and be more aware of both internal and external influences. The key is corporate data.

To utilise this priceless asset it is necessary to integrate data from all parts of the organisation into a homogeneous whole, only then can the power of the knowledge contained be fully realised. To do this requires a corporate data warehouse, which can be supported where necessary by smaller, more focussed data marts.

 

The data warehouse solution provides a flexible framework for enabling more people to make more informed decisions. The ultimate goal of data warehousing is the creation of a single logical view of data that may reside in many disparate physical databases. A Data warehouse offers ready access to information relating to a company’s business, products, and customers. This provides developers and business users with a single working model of the enterprise’s data; something that is absent in virtually every organisation. All too often business information is isolated into individual islands, making it impossible to see the big picture and therefore plan business strategy with all the relevant facts to hand. From this single corporate repository individual data markets, for example, at department level, can be created.

Operational data resides in a variety of repositories, and consists of both structured and unstructured data that lacks any semblance of integration. Within the data warehouse however, all data must be contained as a single integrated source if it is to be of value to the organisation, where the level of granularity will profoundly affect the volume of detail residing in the warehouse. The more detail there is the lower the level of granularity, the less detail, the higher the level of granularity.

In many large enterprise organisations much of the data is mainframe-based involving several types of hardware and software environments. Consequently the collecting of data can be infinitely more complex where organisations have expanded through mergers and takeovers rather than by organic growth. A critical factor in the growth of the total amount of data held in today’s organisations has been the ability to store massive amounts in very large, fast data stores. This situation has been greatly exacerbated by the intrusion of the Internet, with vast amounts of data being made available to organisations of all sizes and markets. It could be said that organisations are in danger of being drowned in a sea of information.

The value of the data warehouse lies in its ability to help users make more informed, faster decisions, through the analysis of key trends and events that directly impact on the business. However, the data warehouse is, as it name suggests, a repository for the collection of data, in order to extract the ‘gold nuggets’, the data must be clean, normalised, aggregated, enriched, transformed and verified in order to ensure a consistent and accurate format can be presented. A number of tools therefore have to be employed to ensure that this transformation can take place, tools, which must then be supported by analytical tools to enable users to accumulate, aggregate, filter or recalculate data held in the warehouse. Other important issues include scalability and performance, which must be addressed if the data warehouse is to efficiently fulfil the functions it was originally intended for. Minerva SoftCare’s MetaSuite PE (Parallel Edition) product set provides all the essential requirements for building, and managing the data warehouse, with support for third-party analysis and Business Intelligence tools from vendors such as Brio, Cognos, and Business Objects.

 

MetaSuite Solution

Minerva’s MetaSuite PE is a four-layered, hierarchical framework comprising six major components, the four layers are:

  1. Operational Environment – including hardware, data, and meta data.
  2. Integration Environment – including hardware, data staging for the temporary storage of containers of un-processed internal and external data, data warehouse, and meta store.
  3. Distribution Environment – including data mart and meta data.
  4. Consumption Environment – including hardware, process data, and meta data.

 

The four-layered framework is supported by the following components:

  1. MetaStore
  2. MetaCollect – integrates with MetaStore
  3. MetaMap
  4. MetaGenerate – integrates with MetaMap
  5. MetaTrace
  6. MetaScape.

MetaSuite PE has been designed to take full advantage of the total use of parallel systems and databases to provide customers with a maximum return on their investment and to minimise their Total Cost of Ownership (TCO). Data can be extracted and processed in parallel, plus the storing of large data sets in parallel to maximise access and eliminate storage restrictions, providing what Minerva calls ‘data scalability’. The Orchestrate Application Framework and Runtime system, which is at the heart of MetaSuite PE, provides the basic services required to execute a data integration process in parallel. These services include: creating parallel operations; pipelining between operators; managing internode communications; establishing parallel connections between the applications and stored data; provide a single program view for management of the application, including centralised error handling and event logging; and performing an orderly cleanup in the event of program failure on the parallel device. Developers program sequential application logic with MetaSuite PE managing the complexities involved in providing scalability and portability for deploying in Symmetric MultiProcessing (SMP), Massively Parallel Processing (MPP) and clustered environments.

MetaStore

The key to collecting and integrating data is through its operational meta data, which must travel with the data. It provides the means to identify where the data originated, if it has been subsequently changed, who its owner is, and its age. MetaSuite PE enables the automatic collection and storage of meta data, from source or target databases, in the MetaStore. It provides a central point of reference that is vital towards identifying the data warehouse and the source data held within it.

The MetaStore is an open meta database that can be implemented on any mainframe, UNIX, or NT-based Relational DataBase Management System (RDBMS) such as Oracle, Sybase SQL Server, Microsoft vSQL Server, DB2, or DB2/2.

MetaCollect

MetaCollect integrates with MetaStore for the collection of data. It provides access to existing data definitions held, for example, in the DB2 catalogue or the IDMS integrated dictionary. It also supports the loading of source code files, for example, COBOL file descriptions or copybooks, into the MetaStore.

MetaMap

The transformation of the operational data to a format suitable for holding in the data warehouse for analytical purposes is achieved via the MetaMap component. MetaMap facilitates the specification of non-procedural rules to describe the required format upon completion of the transformation stage.

Included in MetaMap is a Mapping Manager, which guides users through the creation of the business mapping and technical rules, which define the cleaning, filtering, transformation, and consolidation processes. The modification of the business map and technical rules and the subsequent code generation is processed in MetaGenerate.

MetaGenerate

Data extract programs are created using MetaGenerate. It utilises the technical rules defined in the MetaMap stage, which are imported into MetaGenerate where a Program Manager provides the facilities for extracting code generation and translating it into COBOL source code, although COBOL skills are not a requirement for customers. To ensure performance optimisation or cater for specific conditions not accounted for in the rules, the complete process can be customised.

MetaTrace

The building of quality checking routines and the tracing of data is achieved through the MetaTrace component. Pre-defined reports are generated for checking all extractions, plus the generation of an impact analysis report to identify affected applications where changes in the source or target database have occurred.

MetaScape

MetaScape is a browser tool developed as an application, it can be run on any system that supports a browser. It provides users with a view of the meta data held in the MetaStore, providing both analysts and business users with a map for navigating the data warehouse. Metascape provides the features for interrogating data to identify its source, quality, transformation rules and owner.

MetaSuite PE – The Competitive Enabler

Minerva SoftCare has specifically targeted MetaSuite PE at the larger global enterprise market, where the accumulation of vast quantities of data, for example, multi-terabytes of transaction data is not uncommon. It is these markets in particular that require the features and functionality that MetaSuite PE provides in order to maintain their competitiveness. Partnerships with many organisations that have the requisite expertise and knowledge to support MetaSuite PE are well established, for example leading database vendors, Value Added Resellers (VARs) and distributors.

Minerva SoftCare is one of the leading vendors in the data warehousing market, where they have been established since 1993, the year they were founded, MetaSuite was introduced in late 1997. Privately owned the company has its headquarters in Elewijt, near Brussels, plus offices in Asia, England, France, Germany, Holland, Pacific (Sydney) and South America (Columbia).

 

Minerva SoftCare

Tervuursesteenweg 609

B-1982 Elewijt

Belgium

Tel No. +32 (0) 15 62 71 00

Fax No. +32 (0) 15 62 76 10

e-mail Address: minerva_s@metasuite.com

Web Address: www.metasuite.com