Over the past few years, there has been a noticeable increase in awareness of event processing. This week the Financial Times (FT), one of the most widely read papers in the world, did a nice article on event processing called Event processing: A monitor that can set alarms ringing, quoting StreamBase and our friends at Apama. Algorithmic trading, real-time P&L, and real-time logistics systems done at Dell Computer were all cited as examples of applications that benefit from CEP today.
The article was great, but one flaw jumped out to me, probably due to the nature of the Dell Computer case study cited - the fact that these applications are simply monitoring for changes in event streams. In my first month of travel talking to StreamBase customers, I'm finding the same kind of applications that we had with Apama that do much more than simply monitor streaming data and firing off advisory signals to humans.
It is true that many of the patterns detected by a CEP engine are used as decision-support - to inform a human. Real-time decision support is an important business requirement in most CEP applications. But, at the same time, CEP can automate action that might not even be possible if left to manual intervention. For example, last week I met with a hedge fund who uses CEP monitor a firm's intraday P&L. The system monitors trading activity during the day. When P&L moves too quickly toward its internal limits, the system alerts the head of the trading desk. This is the decision support element of the system. But, at the same time, StreamBase rules can automate action, depending on the nature of the risk. For example, orders above a given size (specified by data that drives the CEP rules) can have their order state automatically altered by the CEP rules so that they can be reviewed by the head of desk. In this way, decisions can be "automated," yet guided by, human decision-making. Small orders, that wouldn't significantly change the position of the firm, and that would be distracting and time consuming to dequeue, are allowed to go through.
And, other "automated actions" include the spawning of new rules based on changing conditions. For example, when risk profiles change suddenly, the system spawns a new rule based on that can evaluate individual order books that are contributing the most to the change. This kind of chained rules - rules that trigger other rules, and maintain complex state, are often the central reason the industry still calls forms of event processing "complex" - it's the ability to identify rich, complex patterns of events that makes the technology so powerful.
I was on a panel this week at the DWT show in London where one of the participants claimed that "robotraders" were making traders obsolete. I have seen no evidence of this. Sure, there are less traders than there were 5 years ago - much of what used to be done manually is now automated. But almost every CEP decision making system is a textured one - some elements of decision making are automated, some are completely left to humans, and some are guided by data and rules - rules that are altered and improved as business conditions change over time.
Applications that do more than just monitor and filter streams unlock the deeper business value in event processing technology: by helping automate intelligent action they can help firms seize opportunity and avert risk before it's too late.