Data Transformation in the Real World

Be warned: In real-world data warehouse solutions that involve disparate data sources, you almost never find relatively clean data like that in the Northwind database. I've extracted legacy system data that has no domain, entity, or referential integrity. I once found parts of people's names in a column that was supposed to contain dates. In another case, I couldn't find any way in the operational system to relate individual billing accounts to customers, even though these items are clearly related in a business sense.

The ease of this fictitious project can help introduce you to some concepts, but it's in no way representative of the data quality, integration, and management challenges you'll likely face in most data warehousing efforts. A data mart solution and the OLAP tools you might connect to it for reporting and analysis represent the tip of the iceberg in a data warehouse. Lurking below the surface are data quality and life-cycle analyses; data integration and extraction processes; data extraction, transformation, and loading (ETL) processes; design and implementation considerations; and meta data management. Don't let these underlying pieces catch you off-guard.

Please or Register to post comments.

IT/Dev Connections

Las Vegas
September 30th - October 4th

Paul ThurottOur Experts will show you:
• Common SQL Server
Problems
• Best Practices for T-SQL
• SQL Server Integration
Services
• Database Development

Come See Michael Otey & Tim Ford in Person!

Early Registration Now Open

From the Blogs
May 21, 2013
blog

A Common Misconception about MAXDOP

Out of the box, SQL Server is (and has been) able to take advantage of multiple processors/cores without any effort on behalf of administrators....More
May 9, 2013
blog

My ISO 8601-Compliant Signature 2

My family recently just "officially" announced that we're in the process of adopting a child from South Africa. We're quite excited, of course, but there's a ton of paperwork to do—along with the need for gobs of signatures....More
May 8, 2013
blog

Use SSIS for ETL from Hadoop

In this blog post, Mark Kromer walks you through using SSIS as a way to use ETL techniques using Microsoft's Hadoop on Windows (HDInsight) as a source using Hive connectors...More
SQL Server Pro Forums

Get answers to questions, share tips, and engage with the SQL Server community in our Forums.