Each year I look forward to attending Microsoft’s Tech Ed to learn about and explore new technologies currently under development. It’s all the more exciting when some of our client needs will be directly met with the release of these new technologies. This year, apart from the excitement surrounding the release of Windows 7, the highlight for me was the announced entrance of Microsoft into the CEP (Complex Event Processing) domain. CEP will be released as an analytical component of SQL Server 2008.
What is CEP? CEP describes the processing of low-latency events with the goal of deriving meaning and arriving at the ability to take action in response to these events in real-time. This technology allows for event correlation and abstraction to aid in, for instance, anomaly and production-quality detection and KPI monitoring. It also facilitates real-time root cause analysis. CEP is currently used in utilities, web analytics, manufacturing, and financial services and is an excellent fit for processing data in the cloud. Events could originate in a flow of transactions from a system or from instrumented devices, and analysis may incorporate static data or other high-latency data in a database along with complex event data. Analysis based in CEP can be used to refine and enhance performance throughout a broad array of systems, both physical and virtual.
How does CEP differ from database applications? As a comparison, in the traditional world of database applications we see the landscape as:
- Ad-hoc queries and requests
- Latency in seconds, hours, days
- Data rate of perhaps hundreds of events per second
When we move to the arena of low-latency complex event processing, we turn the use of our database engine, in effect, upside-down. We’re now processing:
- Continuous standing queries
- Latency in milliseconds or less
- Data rate of tens of thousands or more events per second
- Potentially out-of-order timed events
Microsoft will provide a Community Technology Preview of CEP in mid-2009, and the production release will follow in 2010. See this white paper for additional details, and look for more information from KMA on this topic in the future!
Comments