Blogs are dangerous, especially when theoretical arguments are used to make an ostensibly valid point. Such is the case in a recent posting that concludes that Smart Order Routing (SOR) is not a CEP application. This is a misinformed posting, and I'd like to clarify why.
First a little context: at StreamBase, we've been applying CEP to a wide variety of SOR applications with dozens of investment banks - bulge bracket firms, tier 2 firms, exchange / liquidity / ATS providers, and hedge funds: our team builds SOR applications every day and have years of experience.
Let's examime the basis for the conclusion that SOR is not a CEP application, (excerpts from this CEP blog post):
This is a laymen's view of SOR, and lacks understanding of the complexity that a real system exhibits. Sure, SOR applications make some simple routing decisions. But the reality is that "order routing based on simple rules" is not modern SOR at all. Moreover, it's light years away from a good BPM application. Sure, "rules" are important (James Taylor's post points this out), but let's look deeper beyond a surface-level understanding of SOR.
Here's the next part of the argument: "SOR is not a CEP because CEP deals with evaluating a confluence of events and then taking action." SOR doesn't "evaluate a "confluence of events?" This argument is now officially absurd, and dead wrong.
Let's back up and look the motivation for SOR. The reason banks are adopting SOR is because the very structure of trading is changing dramatically, and has been for years. Regulatory changes in the equities market have spurred the rise of alternatives - ATSs, ECNs - dark pools of liquidity. In the U.S. market there are as many as 30 ECNs today. And in the unregulated FX market, dozens of banks trade electronically with each other, and there will be 15 electronic FX venues by the end of the year. Moreover, the rise in interest in dark pools of liquidity add yet another complex set of events that must be detected, aggregated, combined, matched, and manipulated in real time. It's not uncommon for a single SOR system to connect to 10 or more markets and multiple asset classes. Not only is this a confluence of events, it's a stunningly complicated environment in which to create a complex, real-time model in which to apply "simple" routing decisions. On this basis alone, SOR needs CEP.
Now let's look at processing complexity. "Dumb" order routing, the kind of routing the blogger describes, would make a route decision and emit a message. But there is little value in that. "Real" order routing is much more sophisticated. For example, orders in many liquidity pools can be partially filled, and those partial fills are communicated by FIX messages and are returned to the SOR engine. SOR operates by analyzing the confluence of events from market data feeds, order flows from OMS systems, and executions, aggregating and analyzing those events in real time, and adjust routing decisions on the fly. For example, an order for 10,000 shares of IBM might have 10 partial fills, from multiple markets, in a 3 second time window, and make several routing decisions in that time frame. In other words, even just ONE order can spawn "a confluence" of related messages by itself! These executions are often reconciled with the original parent order, and the SOR may alter its execution strategy (governed by CEP rules) as the fills are returned and the markets move.
And there are many more areas of SOR that require even more sophisticated complex event processing - one example is liquidity detection and optimization. Bill Harts and Rob Almgren, former head of quantitative research with Bank of America, published a paper on liquidity detection and estimation that is predicated on CEP as an implementation engine.
Another critical factor that is ignored is the requirement for performance, scalability, and latency. Add all these requirements to process multiple streams of event data, and the complex interactions created by the order flow process, and you get thousands of events a second against SOR logic, the requirement for millisecond-level latency. A multi-threaded CEP engine becomes even more critical for this kind of application. I've never heard any SOR practitioner ever utter the letters "BPM" in this context, it's simply a non-starter.
So SOR is a great application for CEP because it combines events from multiple streams, looks for patterns, and takes action - complex action that spawns more events a deeper confluence of events, and more action. It does this in a high-perforance, low-latency, complex environment.
Not only is SOR a great example of a CEP application, it's arguable that CEP is mandatory for today's SOR.