We are yet to find a client with multiple sources of data where the format and structure of their data is identical and facilitates easy joining. Achieving a single view of ‘the customer’ or a business operation yields valuable insights that otherwise would be very difficult to achieve.

Extract Transform and Load Solutions:

We have worked with most of the available ETL tools which are present in our client base. In 2000, most of the ETL work that we did was with PL/SQL or DataStage, now owned by IBM, however, our extensive work in the Siebel client base has extended our Informatica experience and now that many more clients are using Microsoft SQL Server we are working much more SSIS (SQL Server Integration Services, formally DTS). We also have experience with Ab Initio , used where data volumes are particularly challenging and BizTalk, from Microsoft, which we are seeing much more of now. Sometimes it may be possible to create view across heterogeneous databases by using MetaMatrix and potentially deliver a joined up view much quicker and at considerably less cost.

Architectures:

There are a number of recognised architectures and the appropriate one should be adopted based upon the business requirement. There are a number of choices that you can make over logical and physical design and whether you opt for relational, OLAP or Bitmapped software for your repository. There are a number of important factors to reflect on; The quality and timeliness of the data are of critical importance, you will have quality issues and it’s better to fix them in the transactional systems than through an ETL process, implement your business rules in your repository and not in your reports except as a last resort.

Master Data Management:

Another issue that has dogged organisations is keeping their business hierarchies synchronised across different business applications. A relatively new sector, now called Master Data Management, is emerging which allow business users to re-define geography, product hierarchies, or any other business dimension and have it dynamically ripple the changes throughout the enterprise. The leader amongst these is MDM from Hyperion.

This is a big subject and we would be happy to share our experiences with you.