The StreamBase Event Processing Blog

Guangfa Building The Fastest Trading System in China with StreamBase CEP and Thomson Reuters

Guangfa (GF) Securities, one of the largest securities brokers in China, announced this week that they are building out a StreamBase CEP-based trading system, which will be the fastest in China, and includes innovative rapid pairs trading and order execution algorithms.

The GF system will be an end-to-end trading system, but what's remarkable about their system is that it's not being built with legions of programmers and consultants.  GF has a small specialist team that have re-used open-source components from the StreamBase Component Exchange and extended them with their own "secret sauce" and knowledge of the Chinese micro-market structure. 

GF's rapid time to market is made possible due to the real differences in how the leading products are built. Esper is a simple, "roll your own" toolkit, and lacks enterprise features and connectivity. Apama is a low-level scripting language with lots of pre-built applications for FX and surveillance, but doesn't have the robust development tools or visual programming support offered by StreamBase CEP.

StreamBase CEP is built for the professional developer and quantitative analyst. It's a comprehensive, event-based system platform, featuring a visual CEP development platform, seamless integration to event data, Java, Matlab, and over 150 environments typically found on the trading floor. This makes it possible for a small team of professional staff to build a complete system in months, not years.

StreamBase CEP is designed for speed. Speed of development - when measured, usually 40%-75% faster than other approaches. Speed of deployment - out of the box, applications that have sub-millisecond latency. Speed to work with Java - with the ability to quickly snap in existing Javacode, or extend event-based logic with Java. And finally, speed of integration, with an ecosystem of over 150 external integration points including our partners in China, Thomson Reuters.

Learning about this difference is repaid to developers, since time to market has a tangible impact on the business bottom line. We recently produced a 20 minute CEP programming tutorial by StreamBase Solutions Architect Mark Hudson that gives developers a good introduction to the power of visual programming with StreamBase Studio. Watch it and learn how you can get systems to market more quickly and with higher quality, leveraging what you have and the development practices you already use.

Do more with less and do it quickly.

 

Guest Blog by Peter Rossi on NETIQ, Aegis and CEP integration

In the increasingly complicated world of IT – Big Data, Cloud, Super Datacentres, DevOps and Agile Development legacy IT support teams are overwhelmed with the new world that they live in.

Traditional server architectures on a 3 or 5 year refresh cycle are being replaced with cloud solutions which may only exist for a matter of hours, but are equally important to the success of the organisations they underpin.

In most organisations IT monitoring systems are designed in from the ground up, as most CIO’s and CTO’s have been caught out by an unexplained outage at some point. However, many IT departments still live blissfully unaware of what is happening in their infrastructure…. Until it’s too late!

Regardless of the actual monitoring technology you will generally see a team of IT professionals attempting to understand this raft of new data. They have to assess multiple streams of monitoring data, understand its context, point the finger at the potential problem and then perform some form of remediation to resolve it. 

This has led to the rise of Complex Event Processing (CEP) engines coupled with IT work flow automation tools, such as NetIQ Aegis which utilises StreamBase as its CEP engine.

NetIQ Aegis allows you to listen to multiple streams of data from your IT monitoring systems simultaneously, for example:

• Server Alerts
• Storage Systems
• Network Switches
• Network Intrusion Detection Systems
• Firewalls
• Audit Logs

You can then create “triggers” using the StreamBase CEP to perform data correlation in real time; this is referred to as a temporal query. In the old world of querying static data, imagine this to be a static query that the data washes over.

This kind of “data in flight” analysis has changed the financial markets – you can now analyse events as they are happening – this type of processing allows you to watch for events across all available data streams in real time – e.g.

• If you see a missing ping alert within any 5 minute window:
  • AND the URL www.myurl.com stops responding
  • AND there is an alert from the storage subsystem
  • THEN Trigger

Every time the CEP engine sees a group of matching events it raises a new alert or trigger…. This is where it gets interesting….

NetIQ Aegis can use event triggers to kick off workflow processes. These workflow’s can contain any number of steps to remediate known faults or raise service desk tickets or make any number of changes to your systems to resolve the problem, automatically, with no human input.

This isn’t to say that we don’t need skilled IT professionals anymore, far from it, but why have a highly skilled worker repeating very standard and similar fixes all day long, when they could be put to use making your business more successful? Complex Event Processing can help you achieve this.

About the author

Peter Rossi is an IT automation expert in the UK currently working on exciting new highly automated cloud project Skyscape Cloud Services (www.skyscapecloud.com). Independently of this, Peter is the author of a well known Microsoft scripting blog www.poshpete.com which focusses on the use of PowerShell in automation. 

Follow Peter on Twitter 

 

Betfair Adopts StreamBase for Real-Time Analytics

(By Richard Tibbetts, StreamBase CTO)

Today we announced that Betfair has selected StreamBase for Real Time Analytics. I’m excited about this news as it demonstrates the importance of real-time data and analytics for driving the best possible consumer user experience for streaming, online services, and the role of StreamBase as the ideal platform for event-driven environments like Betfair.

American readers of this blog may be unfamiliar with Betfair.  Betfair is the world’s largest betting community and one of the world’s leading online betting and gaming operators. At the heart of Betfair is its pioneering Betting Exchange, where customers come together to bet on odds set by Betfair or offered by other customers, instead of with a traditional bookmaker. The Betting Exchange provides better pricing, flexibility, and a level playing field in a massive market. In April of this year, Betfair had over 4 million registered customers worldwide and processed, on average, more than 7 million transactions per day on the Betting Exchange. 

Betfair provides this service via Internet, mobile, and tablet devices that maximize the playing experience, and the Betfair infrastructure implements a service that implements its sophisticated betting market, real-time data processing, and real time clearing of bets.

Betfair is well known for its technically ambitious support for betting during a match. Players can bet while watching a live match, at the last possible moment. When I first met Alex Kozlenkov (Head of Research at Betfair) he described his challenge of executing 100,000 transactions per second against a single product, each transaction (or event) potentially impacts the price of that product, and must be processed in order.

And it’s not just the transactions that are streaming in the Betfair environment, they are also a leader in using real-time data to make the customer experience more engaging, fun, and unique. We’re in the midst of a renaissance of innovation in data management due to “Big Data” – innovative tools and infrastructure designed to manage increasing volume, velocity, and variety of information.  The betting world is no different - new real-time data feeds provide streaming play-by-play data from cricket matches, and real-time social media data brings an opportunity to crowd-source context about the events from fans watching live (in related news, StreamBase recently announced integration with GNIP and connectivity to Hadoop via Flume that enables connectivity to Big Data)

By investing in real-time analytics, Betfair can tap into streaming information and from the betting ecosystem and leverage it to provide a better experience for users live on the site, on mobile devices, and on tablets, providing innovative context and experience to the Betfair experience.

Consumers are now conditioned by social media to expect information and data to be real–time, whether it be pictures of a night out, stock twits, or a location update. Betfair represents a step in the march of all businesses to become more social media aware, real-time, and web-scale. Customers expect more than just a scrolling Twitter feed in the sidebar – they expect real-time context and relevance. The winning firms will integrate a variety of data sources - structured and unstructured, historical and real-time - and distill them down to insight that matters, when it matters. The real-time business of the future requires all of this, and requires real-time analytics and event processing as the new IT computing fabric.

StreamBase LiveView Demonstration and Animated Short

In the first week since the "worldwide premier" of our two StreamBase LiveView videos at the Securities Industry and Finanical Markets Association shown in NYC, we've had almost 1,000 views on Vimeo and YouTube. This product is bringing the power of CEP to entirely new markets and users, and it's very exciting to see interest in it grow so rapidly.

Here's the StreamBase LiveView demonstration video:

 

And our animated short, produced by Grumo Media (thanks, Pedram!)

 

Execution Consulting Services 2.0 Defined

AITE Group’s most recent survey of Wall Street reveals that the vast majority of broker-dealers feel that the importance of their customer service, or "Execution Consulting Services" is more important than quality of their trading algorithms. 

87% of respondents view the manageability and intelligence regarding their trading systems as more important than the actual algorithms behind the strategies themselves.

The majority of broker-dealers in the AITE Group survey now feel that "low touch service models" are no longer acceptable. Low touch (see figure) means the traditional, reactive customer support model – the customer calls and eventually the organization reacts. According to AITE Group’s survey, broker-dealer respondents reported an average 3-5 day turnaround time to resolve buy-side inquiries.

AITE Group’s report defines a next generation approach to client support that they call "Execution Consulting Services 2.0" which attempts to leap frog the traditional approach and drastically reduce the average 3-5 day response to close to real-time.

AITE Group defines ECS 2.0 as having 3 main elements:

1. High-touch advisory services concerning the optimal use of electronic trading service

2. Analytics tools for trade execution quality, risk, and transaction cost

3. Real-time tools that provide live insight, not just rear-view mirror historical reports

ECS 2.0 though isn't just the job of just one person. AITE Group sees the ECS group consisting of a cross-functional team including relationship managers, market structure experts, traders, IT, quants, and research analysts within the Electronic Trading Group.  In order to provide high-touch ECS 2.0 services there is growing need for support tools that can provide a single, unified view of customer order flow and standing with the firm.

The AITE Group report goes on to identify the key technologies that broker-dealers need to better support their customers:

1. Active alerting (79%)

2. Pre-trade and post-trade trade analysis (79%)

3. Single view into client data (57%)

4. Real-time trade analysis (50%)

5. Real-time network monitoring (43%)

6. Ad-hoc query of real-time data flow (36%) 

We built StreamBase LiveView to address these requirements, even though at the time we hadn't heard the term "Execution Consulting Services" used widely. But this is exactly what our development partner, and our freshman class customers for StreamBase LiveView 1.0 asked for and use StreamBase LiveView for today.

As a result, LiveView is beginning to bring event processing to the masses. Traditionally, CEP has been at the heart of apps that need machine-to-machine automated processing, like algorithmic trading. But LiveView brings that model to apps that require machine-to-human interaction, like ECS. LiveView processes streaming, real-time data and harnesses it in a way that humans can use to manage automated infrastructure like automated trading. These applications require real-time analytics, the aggregation of streaming data that can change millions of times a day, continuous query processing and in-memory data warehousing. ECS2.0 teams require ad-hoc, interactive visualization of that warehouse, the ability to slice and dice data at will, and an alerting system that pushes events to ECS staff.

For more information, read the AITE Group report or sign up for the roundtable on ECS taking place next week, or watch the StreamBase LiveView Demonstration at www.streambase.com/liveview where you will also find technical whitepapers, case studies, and lots more.

 

SunGard Wins Innovation Award for Real-Time Risk Management

We're always proud of our partners when they do innovative things with StreamBase, so today I'm thrilled to congratulate SunGard, who won The Banker’s “2012 Innovation in Banking Technology” award for their StreamBase-based risk management application, Validi Order Validator, their new ultra-low latency order validation engine. 

The Banker is one of the world's leading banking and finance resources and is read in over 180 countries around the world. A panel of 21 senior banking technology executives selected the winners from a shortlist created by The Banker’s editorial team.

John Beck, trading and technology editor at The Banker said: “When it comes to trading technology, the arms race continues unabated. And for banks, catering to client requirements for access to fast and profitable algorithmic strategies, best execution, ultra-low latency connections and new exchanges and asset classes means that standing still is not an option. Meanwhile, the influx of post-crisis legislation has meant that the importance of post-trade processing has increased.

Congratulations to our friends at SunGard!

Gain Capital: A Glimpse into the Future of Analytics?

Imagine trying to drive your car while ONLY looking in the rear-view mirror? If you're like most businesses, you make your decisions by looking at reports, which is like driving while only looking in the rear-view mirror.

First generation analytics and BI tools - Excel, Tableau, Qliktech - provide a map of where you've been. They're so essential that they are a $10 billion software market. But using old data to drive your business only tells you what potholes you already hit and what turns you could have made.

StreamBase LiveView provides a live dashboard of where you should go next. It analyzes live data; not old data. LiveView absorbs data streams with just milliseconds of latency - sensor data, high-speed market data, client orders, streaming news, tweets, FIX messages, network messages, emails, log data. LiveView gives you a live view of data - before everyone else sees the report tomorrow!

Live insight helps you see potholes before you run over them, and decide where to turn next. For a more complete overview of LiveView, watch the 4-minute intro presentation here. 

Today, GAIN Capital went public about their use of LiveView and how they use it to steer their business differently. Rob Perez, VP of Trading and Risk Management Systems at GAIN Capital, shared a terrific example of the transformation live analytics can bring. GAIN's business is trading, and success is linked to the quality of connectivity to the electronic markets. Gain services over 140 countries, with an average monthly transaction volume of over $275 billion. 

Previously, GAIN managed their trading operations with traditional tools - databases, log files and business intelligence. At the end of the day or week, reports are generated, problems are discovered, and addressed.

Now, LiveView helps GAIN understand and optimize their trading operations in real-time. Perez said in EuromoneyFXNews: "We an see if any metrics are moving outside of our comfort zone, and we can immediately reach out to our liquidity providers to understand what's going on."

At any time, GAIN clients can trade on any of 10 or more markets. It's essential to GAIN that problems with prices, connectivity, or quality are elimintated. Now, GAIN can now determine:

• What deals are being rejected now?
• Which liquidity providers are giving good prices now?
• Which liquidity providers are having problems now?

Live insight helps GAIN decide to call a poorly performing partner the moment there's a problem. Finding a problem as it's happening makes it easier to describe and easier to fix. And finding problems in real-time gives GAIN the opportunity to correct it before their clients notice. If the issue is with a partner who can't fix it on the spot, Gain can adjust and route around that market. 

StreamBase LiveView is the first platform of its kind to provide a complete analytics solution for live data and real-time action, which we believe could be a new disruptive physics that allows companies to broaden the decisions they make by looking ahead, not only behind. 

So, what decisions would YOU make differently if you stopped looking only in the rear-view mirror?

 

Three Stories from StreamBase LiveView Customers

A year and a half in the making, we're thrilled to release StreamBase LiveView 1.0 (press release). Since we released the concept video last November (which drew almost 4,000 views on YouTube), our early access program has been a huge success. We'll announce an exciting set of customers in the coming weeks from that program.  

What's all the excitement about? While the technology is awesome (download this great StreamBase LiveView Technical Paper by CTO Richard Tibbets, Steve Yang, Rob MacNeil, and Dave Rydzewski), it's really all about the value our customers get from the stuff we build.

Here were three "aha" moments I had with those visionary adopters that, for me, crystallize why LiveView is so special.

1. Real-Time Service Experience Management. Recently the head of trading at a big bank told me that when a big buy side client like Fidelity calls to ask about the performance of a trade, they:

• Issue a request to IT
• IT searches log files, databases, and reports to gather raw data
• IT analyzes the data with one or more of: a quant, a risk expert, an execution expert, and a market structure expert
• The team writes a report
• The team briefs the sales trader on the account
• Finally, after 3-5 DAYS - they explain the situation to the client
• They decide on a course of action
• Adjustments are made - algorithms or strategies are changed

No more. Now, LiveView helps them make many of these decisions in seconds. LiveView creates a live, 360 degree view of order flow, executions, risk, transaction cost, risk, and operations. LiveView lets sales staff see what's happening before the data is lost in an IT black hole of log files, databases, and reports. 

Now, the moment the client calls, a salesman, risk manager, or trade execution specialist can answer if orders are being received correctly? Is the risk profile in line? Are the trades in line with the cost benchmarks? Are my 5 EMSs up and running? Are connections to the exchanges fast? Does the client want to adjust their strategies? 

The transformation is exciting: a typical reactive support function becomes proactive, opportunities lead to bigger opportunities, and problems become sales opportunities.

2. Real-Time Selling. In January, at a meeting at another bank, a sales trader asked: "Could I ask LiveView in the morning to alert me whenever a big order comes in from an important client using the new algorithm we released last week so I can bird-dog it?"

This was the perfect question, because it strikes at the heart of what LiveView does: it's like setting hundreds or thousands of business probes, and being alerted proactively when one is tripped. The probes can be turned on, turned off, and adjusted on the fly. Any system, internal or external, packaged app or custom built.

Firms that provide a highly automated service to clients need a real-time intelligence system for sales people. The ability to sell more depends on the ability to anticipate that a key client is headed for trouble, or when the time is right to encourage more sales.

And by building real-time intelligence into the system, everyone becomes a sales person, because the conditions that encourage more sales are right in their face.

3. Real-Time Fraud Detection. My last story took 3 months to unfold. Another LiveView early adopter wanted to implement a "fraud detection" analytic with LiveView. The firm needs to monitor machine-driven algorithms, and look for suspicious patterns. Sounds simple, right? Not so simple!

In this case, the patterns of interest required correlation of 5 streams of real-time data. Patterns happen within 15-30 second windows, during which thousands of dollars could be lost. Attacks come in bursts.

The data required to find these patterns was loaded into a data warehouse and reports were checked each day. Decisions to act were made every day.

LiveView now intercepts the data before it hit the warehouse by connecting LiveView to the source of data. It took 3 days to integrate these sources because it took that long to find someone who knew where 3 of the data streams came from!

StreamBase detects fraud patterns in milliseconds. But the really interesting part came next.

Once this firm could see patterns of fraud, they were faced with a new challenge: what to DO about it? How many times did the pattern need to be repeated until active surveillance is started?  Should the action be quarantined for a period, or halted immediately? All these questions were new, and the answers to them keeps changing.

The fact that the answers keep changing highlights the importance of ease of use. Analytics must be changed quicky and be made available to fraud experts - in some cases, in hours - as understanding deepens, and as the bad guys change their tactics.

Better, higher value-add customer service for highly automated industries. Knowledge workers who anticipate sales opportunities. Spotting fraud in high-speed transactions streams and taking action. These stories are the stories of the future for the real-time enterprise. These stories tell the story of why StreamBase LiveView is so special.

Chasing an Ambulance Full of BATS

The high-profile trading failure at BATS this week sparked a rash of negative publicity, gracing the front page of media outlets from the Wall Street Journal to Forbes. In the software industry, more software companies have proclaimed their techology would have sheltered BATS from their storm than cheap umbrella sidewalk vendors on a rainy day in New York City.

An especially troubling rash of "thought leadership" came from my own industry - real-time analytics and complex event processing. Forbes, for example, published an article by SAP (who sells a CEP product) BATS hits the rocks, which may have lifted its sales that proclaims that the BATS incident portends that "CEP technology is about to move from the backwaters of data management technology to center stage."

In Wall Street and Technology, John Bates, the CTO of Progress Software was quoted in the artice Flash Crash at BATS renews Market Concerns to say firms "clearly lacked adequate risk controls such as real-time surveillance and monitoring. Hopefully this incident serves as a wake up call." What kind of software do you think Bates sells?  Drum roll please... real-time surveillence and monitoring!"

My company sells these kinds of solutions too. And I know BATS well (disclosure: BATS is not a StreamBase client), and they are as technologically advanced as any firm in the world. So this kind of ambulance chasing is not only unbecoming, it's also dead wrong. It's like suggesting Formula One racing will be safer if the cars can be made to go faster (CEP) and we do a better job of filming them as they go careening into a wall (monitoring). Indeed, BATS quickly reported that they had a bug in their system, so comments about risk management and surveillance are irrelevent.

That said, there's an issue here for the capital markets - but the issue is cultural, not technological. 

What's happening in the capital markets is that we are in the midst of real-time automation renaissance. 10 years ago, less than 5% of stock trading in the U.S. markets was done algorithmically; today, as much as 70% is done on computer-implemented, and human designed, algorithms.

Larry Tabb, the founder and chief executive of the Tabb Group, said in the Times that “The markets basically gutted their high-cost, nonstop infrastructures for very fast, low-cost infrastructures." Now we are talking - Larry is on the right path!

In software, mistakes happen frequently. Professor Neil Johnson at the University of Miami analyzed 18,520 "ultrafast Black Swan [trading] events" from 2005-2011. In Stock Market Flaws Not So Rare, the New York Times that just one of the 13 U.S. stock exchanges - NASDAQ - declared 139 breakdowns at other exchanges last year alone, up from 69 the year before.

Software breaks. We have a word for it: bugs. But that's not new. What is new is that the errors can do damage so much more quickly, and, because the world is now so hyperconnected, everyone knows about it more quickly than ever before. 

So what is the magic elixir to making the capital markets fast, fair, and safe?  

There are still voices out there that seem to think that the answer is to discontinue the use of computers for automation. Ha! The whining about the evils of capitalism and sophisticated technologies continues to rain down from dissenters posting their admonations from inside their pup tents via shiny new iPad 3's. These dissenters need to Get Over It - technology is here to stay, and its use is expanding quickly from Wall Street to Main Street.

Making automation safe and reliable requires complex, cultural change:

1. Continue to refine reporting requirements. You can't manage what you can't measure. In the United States, exchanges didn't report outages on other exchanges until 2007. Nasdaq reports that there have been 110 outages in the last year like the one that triggered the BATS mishap. Continued measurement and transparency will lead the industry toward greater reliability. With more analysis and reporting, more accountability is possible, and regulators could focus on correlating events among exchanges and brokerages to look for systemic flaws.

2. Refine and lighten regulation; don't just pile it on. Dodd Frank and MIFID 2 have put regulation on the must-do agenda of all firms. But it's still largely up to the reader to separate what's essential from what's fluff, and that's not effective. Exchanges like BATS are extrodinarily regulated - so more, heavier regulation is not the answer.

3. More systemic attention to software quality.  Moore's Law - the idea that the power of computing doubles approximately every two years, does not apply to software quality. The BATS crash shines a light on the issue that is inbred in innovation; the rising importance of software testing, quality, and simulation. Most firms have ad-hoc processes for quality and should have continuous testing, regression testing, and real world simulation. CEP can make systems go faster, and surveillence helps spot problems more quicky, but when more attention is paid to testing and simulation, less errors occur. 

These changes are complex and cultural, not technological. Tools will help, but not solve, the challenge of ensuring a safe, reliable, and fair market.

 

Tech Q&A with CEP Developers, and StreamBase

Today StreamBase hosted an online roundtable entitled “How to Empower Your Development Team with CEP and Visual Programming” featuring developers from J.P. Morgan, GAIN Capital, and Datasports. Combined, the panelists have 35+ years of development experience, and years of CEP experience, and they were kind enough to share those experiences with us, discussing the adoption of CEP technology, developing applications with CEP, and the best practices they’ve learned along the way.

For developers, development managers, architects, and business stakeholders this roundtable tells an interesting story about how CEP technology can truly transform the role of the developer. Take a look at some of the highlights below, watch the replay, then comment below and tell us about your experiences with CEP. 

Talk about your experience with CEP

Phil Martin, Datasports: I like using StreamBase because I like tools that let the computers do stuff that the computers are good at, and let me do stuff that requires thought. As a developer, you wind up doing some tasks over and over again; the structure of the application, domain specific plumbing, etc. Everytime feels like wasted effort. StreamBase lets me focus on the stuff that requires thought, not the details of implementation, connectivity, processing events, and so on. Another comment is around the aggregate operator. It is analogous to another API to learn. It’s an incredibly rich and flexible operator that I think almost qualifies as its own sub language or sub paradigm.  There’s a lot there and a lot of subtlety in its behavior, learn about and study it!

Mike Grandilone, J.P. Morgan Asset Management: The shift to a message passing structure is sometimes a shift in thinking. It's key to understanding how StreamBase handles the life of a tuple; we gained that through experience.  

Talk about the applications you've built with StreamBase

Joseph Chua, GAIN Capital: After 4 days of training and going through the examples, I built my first StreamBase application in just three hours, and it was fully deployed in a week. It connects to market data, manipulates the data, and stores it in a high-speed database. We've also built market making and real-time surveillence applications, and many more to come.

Phil: I've built several applications with StreamBase. One is an "indication of interest" (IOI) app that monitors trading interest and compares it against a CRM system and sends alerts to individual sales traders because they have a customer who either has traded on the symbol, transacted above a given threshold, or has a real-time order for it. The correlation of these factors in real-time also includes real-time news sources. Also, I've worked on two algorithmic trading systems that host their strategies in StreamBase and perform real-time risk checks on the algos, and an ETF arbitrage system where there was only one strategy, but it was a very involved strategy.

Mike: We brought a smart order routing system up and running in 6 months with 2 developers, which we thought was very fast. We also got the business people involved in the canvas and the development - this is a very important factor in getting applications done quickly.

Talk about the benefits of StreamBase and visual programming

Phil: StreamBase helps facilitate communication. Whenever you want to convey information about a system, you always draw a picture. And they almost always look like StreamBase Event Flow. If you have taken the necessary care to clearly delineate modules, then, depending on the technical sophistication of who you are talking to, you can choose how far to drill down. It's extremely helpful in faciliating technical discourse; with the business users, it saves time.

Mike: Visual development has really improved the speed of our development. Especially when you're doing a proof of concept. It helps you understand the application's logic, and also communicate with the business; in typical programming environments like Java, you're not going to show many people the code, but you can do it with StreamBase.

Joseph: The big value is that all the technical details are abstracted away in terms of connectivity and low-level code; we can then communicate with the business owners, so development cycles are shorter and more efficient. The other value we have found is they we can very quickly build a prototype solution so the business can make decisions more quickly with respect to requirements and decisions to develop the applications.

Phil: One thing that's helpful: with StreamBase, you can tell at a glance that spaghetti code is spaghetti code. You can tell if an application was thoughtfully designed without hours and hours and hours of code review.

What best practices have you learned from your experiences with CEP?

Mike: You need to shift your thinking, to think in terms of events, rather than traditional static applications. And, just like any application, build a testing environment at the same time as your application logic.

Joseph: Identify the connection points quickly, be open minded and willing to unlearn the more traditional ways of development, use StreamBase interfaces, and make sure you continuously refactor the application.

Phil: There are lot of resources for ramping up StreamBase skills - invest the effort to develop your skills. The speed of putting together something quickly is a great asset. The thing to keep in mind is that although the development speed of StreamBase is a benefit in the short term, it can, if you're not careful, work against you in the long term.

Phil: While the principles aren't different, the practice IS different. The first thing that sold me is seeing the ease with which you can change and tune your threading model, take things in and out of the threading part of your system, with multiplicity and dispatch methods. Scalability and performance is about taking more advantage of the cores in a system, and careful management of it becomes more and more important.

What was your experience integrating other code into CEP?

Mike: We have separate routing logic written in Java for routing logic that we integrated with our StreamBase application. We've done a lot of work with custom Java operators in StreamBase - it's very straight forward - no problem in doing it. There's lots of power in developing in Java for code that isn't event driven that allowed us to bring in the power of algorithmic trading as a drop-in to the StreamBase environment. Also, we have a C# front end GUI, which uses the .NET interface from StreamBase - this was easy, although the challenge was how to take a GUI that isn't event-based and make it work with an event-driven application. We're in production with that now.

The StreamBase community

Phil: I've put two components onto the StreamBase Component Exchange - mostly adapters. The publishing process was straightforward. One is an adapter that watches for events on the file system and creates StreamBase events when files are created, renamed, modified, or deleted, and another one listens for calls on a web service interface and creates events for the applciation. I suggest other developers look at the utility classes – see what are other developers are doing, and pick up some tips and techniques and helpful hands-on examples of how they are building their applications. 

StreamBase Resources for Developers

Below are the StreamBase resources mentioned in this roundtable:

• Trial version of StreamBase: www.streambase.com/download
• Product documentation: docs.streambase.com
• Personalized demonstrations: www.streambase.com/forms/request-a-demonstration
• “How To” developer webinars: www.streambase.com/webinars
• StreamBase Component Exchange: www.streambase.com/sbx
• StreamBase University: www.streambase.com/sbu
• StreamBase Training and Certification: www.streambase.com/community/training-and-certification
• Datasports blog for StreamBase CEP developers: http://datasports.ca