On the Requirements of Smart Order Routing (SOR) and CEP

The requirements of Smart Order Routing (SOR) applications are a good fit for complex event processing (CEP) because SOR applications need to evaluate a confluence of events, detect complex patterns of market events and manage feedback loops of trade execution state in real-time.  And they have to do all of this in a very low latency environment, with a lot of events.

Let's back up and look at the motivation for SOR.  The reason banks use SOR is because the very structure of trading is changing dramatically, and has been for years.  Regulatory changes in the equities market have spurred the rise of trading venue alternatives - ATSs, ECNs - dark pools of liquidity.  Traditional trading exchanges are shutting down because of these cheaper electronic options.  In the U.S. market there are as many as 30 ECNs today.  And in the unregulated FX market, dozens of banks trade electronically with each other, and there will be 15 electronic FX venues by the end of the year.  Moreover, the rise in interest in dark pools of liquidity add yet another complex set of events that must be detected, aggregated, combined, matched, and manipulated in real time.  It's not uncommon for a single SOR system to connect to 10 or more markets and multiple asset classes.  Not only is this a confluence of events, it's a stunningly complicated environment in which to create a complex, real-time model in which to apply "simple" routing decisions. On this basis alone, SOR needs CEP.

Now let's look at processing complexity.  One example of complex sequences of events that can be managed by CEP is in the area of order and execution state management.  Orders in many trading liquidity pools can be partially filled, and those partial fills are communicated by FIX messages and are returned to the SOR engine.  The FIX messages themselves are events.  SOR operates by analyzing the confluence of events from market data feeds, order flows from OMS systems, and executions, aggregating and analyzing those events in real time, and adjust routing decisions on the fly.  For example, an order for 10,000 shares of IBM might have 10 partial fills, from multiple markets, in a 3 second time window, and make several routing decisions in that time frame.  In other words, even just ONE order can spawn "a confluence" of related messages by itself!  These executions are often reconciled with the original parent order, and the SOR may alter its execution strategy (governed by CEP rules) as the fills are returned and the markets move.

Other SOR requirements that require sophisticated event pattern detection is for liquidity detection and optimization.  Bill Harts and Rob Almgren, former head of quantitative research with Bank of America, published a paper on liquidity detection and estimation that is predicated on CEP as an implementation engine.

Another critical requirement of SOR systems is high performance, scalability, and low latency.  Addressing the event processing challenges described above in the face of multiple streams of event data, the complex interactions created by the order flow process, and you get thousands of events a second against SOR logic, the requirement for millisecond-level latency.  A multi-threaded CEP engine is critical for this kind of application, because it's SOR typically monitor markets in their entirety, and not just for the best bid and offer, but even for level 2 quote data (which shows all the public quotes of market makers wishing to sell or buy stock and recently executed orders).

So the requirements of SOR are a great fit for CEP because it combines events from multiple streams, looks for patterns, and takes action - complex action that spawns more events a deeper confluence of events, and more action.  It does this in a high-performance, low-latency, complex environment. 

Not only is SOR a great example of a CEP application, it's arguable that CEP is mandatory for today's SOR.