DATA WAREHOUSING FROM THE TRENCHES 

Your business strategy is only as good as its execution

By Stephen Brobst
sbrobst@alum.mit.edu
April 29 - May 05, 2002

A traditional data warehouse focuses on strategic decision support. Online analytical processing (OLAP) and data mining techniques are used to explore historical data for trends and insights to develop business strategies. Applications with high returns on investment include customer segmentation, strategic price management, retail category management, asset management and channel management. The business value associated with transforming an organization to quantitatively driven strategy development has been a tremendous success for data warehousing.

As organizations realize the value of data for strategic decision making, the lack of information available for front-line decision making has become dramatically apparent. A well- developed strategy is critical, but ultimate value to an organization is only as good as its execution. As a result, deployment of data warehouse solutions for operational, tactical decision making is becoming an increasingly important use of information assets. A variety of architectural frameworks such as Active Data Warehousing, the Corporate Information Factory and Zero-Latency Enterprise have emerged in recognition of the importance of tactical decision support as an extension of traditional data warehouse capabilities.

Consider, for example, the customer relationship management (CRM) strategy for an airline. Traditional uses of a data warehouse by an airline allow for customer segmentation, profitability analysis, scheduling and route design, and development of pricing strategies for maximizing revenue yield. An enterprise data warehouse allows for analysis of the impact of scheduling/route changes by customer segment. It also allows for price- elasticity studies and assessment of the impact of different pricing policies by customer segment. It is the integration of data across the multiple subject areas such as customer, revenue and scheduling that yields maximum value from a data warehouse solution (in contrast to data mart deployments for each application or subject area).

The value of a single, integrated source of truth for operational aspects of CRM has also been recognized within most organizations. It is in the data warehouse that the total travel experience for a customer comes together into a single repository: baggage handling, channel utilization, frequent flyer status, and so on. Direct marketing operations often make use of the data warehouse for purposes such as list pulls for targeted promotions, identification of at-risk customers for retention programmes and execution of win-back campaigns. While these uses of a data warehouse are clearly more operational than purely strategic decision support, they are still back-office activities.

The demand upon next-generation data warehouse deployments is for delivery of information to the front lines of an organization. Today, an airline can assess how profitable you are and send free gifts in the mail; however, travellers are most interested in the quality of the actual travel experience. Identification of at-risk customers for retention programmes following bad travel experiences has less impact than proactive decision making that minimizes the impact of any disruption.

For example, imagine that your flight into Chicago is more than an hour late and you have missed your connecting flight to La Guardia. There is one last flight into La Guardia that evening, but there are only three available seats for the 10 people on your flight who missed the same connection. How will those three remaining seats be allocated? It is most likely the seats will be given to those who run the fastest, push the hardest or get on their cell phones most quickly. This is an example of a clear disconnect between the airline's CRM strategy and its execution.

A sophisticated airline knows from its data warehouse who its best customers are not just its frequent flyers, but its highly profitable frequent flyers. From its operational systems, an airline also knows travellers who have missed their connections well before the flight lands. Unfortunately, the two pieces of information are not brought together to allow optimal allocation of the remaining three seats. Additionally, the front-line organization does not know which customers missed similar connections last week, thus putting them at risk for defection. A traditional data warehouse will eventually have all of the required information; however, it is not updated frequently enough to make a difference in the travel experience as it unfolds.

Data warehouse architects have some new challenges to address with the emerging demands for tactical decision support. The need for data freshness escalates significantly. Data acquisition on a trickle-feed basis using enterprise application integration (EAI) tools supplants batch-loading strategies for certain classes of data. The number of users and performance requirements for the data warehouse will increase by orders of magnitude with deployment of analytic applications to the front lines of an organization. Data access paths become much more narrow and structured in a tactical decision support environment as compared to traditional data warehousing. Finally, 100 per cent uptime is a requirement for execution in today's competitive environment.

Get ready to evolve traditional data warehouse deployments to architectures that support extreme service levels in terms of performance, availability and data freshness.

Stephen Brobst is an internationally known expert on high-end data warehouse implementations. He performed his masters and Ph.D. research in parallel processing architectures at the Massachusetts Institute of Technology and teaches the high-performance data warehouse design course at TDWI. Brobst can be reached at