The StreamBase Event Processing Blog

Ease of Use: The Next Frontier for Event Processing

We're back! The acquisition of StreamBase by TIBCO was a big change, but as we near our "First 100 day" milestone, we're starting to come back to our regular conversation on the blog, and in our various social media outlets, including, of course, our new home @TIBCO. 

We kick it off with an article I wrote for IT Briefcase called Ease of Use: The Next Frontier for Event Processing: 

The future of computing will be event-driven, and recently, we’ve reached a new frontier in its emergence.

In June, the two leading “pure play” event-processing vendors, StreamBase and Apama, were acquired in the same week. These acquisitions prompted industry analysts to ponder who will be the winners in general-purpose analytic platforms for high velocity data. Philip Howard from Bloor Research wondered about the future of StreamBase, about IBM, SAS, and open-source projects like Cassandra, Impala, and Twitter Storm. Each has vastly different capabilities, focus, and ecosystems. Indeed, it’s still very early days for real-time technologies, but there is broad agreement that the next revolution in enterprise technology stacks will be driven by the need for real time.

But before picking a winner, shouldn’t we examine a more basic question: What do Main Street firms need to become more real time?

Continued at IT Briefcase...

Best Practice #3: Plan for Spikes, Not Aggregate Flow

This post is a part of a series of best practices from leading StreamBase enterprise customers who have effectively deployed "real-time" business systems. 

"Can StreamBase handle 300 million messages a day?" asked a senior IT manager the other day.

"Well, it's common for a single StreamBase server to handle 100,000 messages a second for applications like yours," said our field architect. 

"OK, but can you handle 300 million messages a day?" 

Why this bizarre miscommunication? 100,000 messages per second is 8.7 BILLION messages per day, yet the IT manager was worried StreamBase wouldn't handle his volumes.

Experienced event processing architects think about scalability in a different way. So, for this best practice blog entry, I teamed up with one of our most senior field architects, Hayden Schultz, to describe the art of scalability design and how to think about scalability in real-time systems. 

THE SITUATION: AGGREGATE DATA CAPACITY PLANNING IS COMMON PRACTICE

Many technologists think about capacity planning in terms of aggregate data volumes. That is, they predict how much data they need to process in a day and provision their network, database, and hardware accordingly.

THE PROBLEM: REAL-TIME DEMANDS DIFFERENT METRICS

Aggregate volume isn't the right way to plan real-time systems.  Real-time systems often have a 100-to-one, 1,000-to-one, or even a million-to-one ratio of event-computation-to-action ratio. Technologists need to think in terms of stream burst rates, and what the right behavior is during bursts.  

THE SOLUTION: THINK IN THREE SCALABILITY DIMENSIONS 

There are three dimensions to scalability in real-time systems: 

1)  Bursts behavior. It's not uncommon for bursts to be so narrow that you don't really need to design around them because while performance will degrade during the burst, the burst is fairly brief so the system performance degrades so briefly that it's not noticeable. A stable real-time system is designed so that the maximum data rate that it's capable of processing is greater than the average data rate, so if a burst is reasonably narrow, the latency of the responses increases during the burst while incoming data is buffered; when the burst is over, the backlog of data is quickly processed and the system resumes its normal performance.

2) Consider Conflation. Increasing the response latency during bursts can be the wrong approach for some systems, but the cost of designing for spikes 3-5X higher than the average can be expensive. A technique called conflation helps mitigate this issue. Processing events that are buffered during bursts has the unfortunate tendency to process events that have already been superseded by more recent events, so the system is using its overtaxed resources to compute results that are obsolete while letting the most current results age and perhaps become obsolete in turn. Conflation means that new events overwrite processed buffered events so that only the most current data is processed. The data rate is reduced and obsolete events are not used. Conflation may slightly increase average response time, but can dramatically reduce latency during traffic spikes. 

3) Computation reduction. Some systems have computational loads so that a new result cannot be produced for each input event. These systems typically produce results at a fixed frequency. This is common when the result is sent to a graphical user interface (GUI). A typical GUI can't update nearly as quickly as many CEP applications can produce results, and even if it could the updates would overwhelm the users trying to use the GUI. Sending results at a fixed rate solves this problem. If the data rate of events entering the CEP system should become too large a common strategy is to reduce the computation frequency. When the burst is over, the computation frequency can be increased to its normal rate.  

THE IMPLICATION: CORRECT SYSTEMS AT THE RIGHT COST

Planning system capacity in terms of burst rates, data reduction, and computational reduction helps balance correct system behavior and cost, and turns handling a billion messages a day into child's play.

The 3 Principles of Analyzing Data in Motion



 

When I first saw an event processing language 10 years ago, as a C++ developer, it opened my eyes to a new development "physics" that made dealing with information in motion simple. Today, it seems like all modern data is in motion: streaming social media data, market data in financial services, and the increasing adoption of the internet of things and sensor-driven systems. Clearly, the technology needed to manage data in motion is different than that needed to manage data at rest; and that's where StreamBase complex event processing, or CEP comes in - its the ultimate platform to monitor, analyze, and act on data that's in motion.

Complex event processing doesn't mean the technology is complicated to use; it's quite the opposite.... the term refers to the ability to identify complex sequences of conditions - as the data moves through an enterprise.

Three CEP capabilities, aggregation, correlation, and temporal analytics, provide the foundation for dealing with data in motion.

For example, in the gift card market, let's say you wanted to detect a common fraud condition, such as: "Tell me when the number of gift card redemptions at any POS machine is above $2,000 in any 1-hour period." - this situation can be easily detected by the CEP's aggregation facility, which continuously calculates metrics across sliding time-bound windows of moving data which is used to understand real-time trends. Continuous aggregation is hard to do with traditional tools; but with CEP, it's built in - simply drag the "aggregation" operator onto your application template, and you're off.

The second key new principle of CEP is Correlation. Correlation means connecting to multiple streams of data in motion and, over a period of time that may be seconds or days, identify that condition A was followed by B, then C. For example, if we connect to streams of gift card redemptions from 1,000 point of sale terminals, CEP would allow me to continuously identify conditions that compare the POS terminals to each other like: "Alert me if the gift card redemptions in one store are above 150% the average of the other stores."

Finally, CEP is designed for temporal analytics, which is the idea of having time as a first-class computing element. Temporal analytics are critical for processing data where the rate and momentum of change matters. For example, in fraud, sudden surges of activity are common indications of fraudulent activity - so detecting "If the number of gift card sale and card activations within 4 hours is greater than the average number of daily activations of that store in the previous week, stop approving activations." Unlike computing models designed to summarize and roll up historical data, CEP is designed to ask and answer these questions on data as it changes.

These three elements: aggregation, correlation, and temporal analytics, set CEP apart from all other computing platforms by helping technologists look at what's happening now, not just what happened in the past more quickly than ever imagined before. Data in motion, and the use of CEP to analyze it, is the future of computing.

90 Seconds on How to Develop with Java and CEP

Not long ago I sat down with Phil Martin, a senior consultant at Datasports, for a 30-minute online Q&A and demo on “How to Develop with Java and CEP”. Phil’s been developing applications with Java and StreamBase CEP for over 3 years now and we wanted to pick his brain about his experiences developing applications with these two languages.

If you’re a Java developer looking to build real-time event based applications you should definitely check it out in its entirety here. Then again, if you’re a developer you probably don’t have 30 minutes of free time to watch the video...unless of course you're developing with StreamBase!

So instead I'll make it easier for you. Below you’ll find the highlights from this conversation, and the best part is you should be able to digest it all in about 90 seconds.

Q: Phil, what kind of use cases and applications are a good fit for CEP development?

A: It should be a real-time system where performance is an important concern. And it should be unsurprisingly an event or message based type of processing model where those events or messages are clearly defined, where the content and format is known at design time.  And I think that you get increasing benefit from a platform like StreamBase if you have a large number of disparate kinds of systems that you’re looking to integrate with. 

Q: What aspects of my application should be developed in CEP and which should be developed in Java?

A: Anything that implements a low level protocol type of logic probably belongs in the Java domain, for example things that are implementing interactions with vendor provided client libraries or protocol libraries or doing low level network disk or database IO. In general other than that I find that the CEP programming model works the best for almost all business logic.

Q: What are the benefits of developing with CEP?

A: When I’m developing code I want to concentrate on the things the human being does well and have the computer do things that machines do well.  And for me that means I want to be focused as much as possible on business logic.  And what I find is that using a platform like StreamBase lets me stay focused on business logic.  The other benefit that I see is visual programming. I find the day-to-day model for doing development more productive. I also find that the resultant applications tend to be fast and stable.

Q: What have been your experiences around time-to-market gains with CEP?

A: My general rule of thumb is developing with CEP is three to five times faster than using a traditional programming model like Java or .NET development.

Q: How does CEP address some of the biggest frustrations Java developers face?

A: My biggest frustration also relates to the biggest strength in Java, which is that for every level of your application infrastructure there are choices to be made.  There are choices around what version of the JRE, what kind of framework you’re going to use to write your application, the list goes on and on. By having so many of those choices already made for me and made well by the StreamBase platform I can do what I want to do, which is to stay focused on solving business problems.

Q: There are some objections that we sometimes hear from Java developers around having to learn a new language and concerns about visual programming and its effect on performance. Why shouldn’t Java developers be concerned about that?

A: In terms of being able to actually get ramped up, get hands on and start doing productive work I find that curve with StreamBase is actually very fast compared with other environments. I would normally estimate that someone who is never touched StreamBase before can actually be productive in a matter of weeks.

The StreamBase environment is compiled the same as any Java applications, the components and infrastructure are developed with performance in mind.

I found that the visual environment makes the processes of refactoring and reusing code and tuning the threading model much simpler, much clearer, and much, much faster than what it would be in a traditional programming environment.

Thanks to Phil for doing this webinar with us. You can check out his blog at http://blog.datasports.ca/ where he blogs about financial systems engineering, including StreamBase, Java, and the Canadian markets.

Betfair Adopts StreamBase for Real-Time Analytics

(By Richard Tibbetts, StreamBase CTO)

Today we announced that Betfair has selected StreamBase for Real Time Analytics. I’m excited about this news as it demonstrates the importance of real-time data and analytics for driving the best possible consumer user experience for streaming, online services, and the role of StreamBase as the ideal platform for event-driven environments like Betfair.

American readers of this blog may be unfamiliar with Betfair.  Betfair is the world’s largest betting community and one of the world’s leading online betting and gaming operators. At the heart of Betfair is its pioneering Betting Exchange, where customers come together to bet on odds set by Betfair or offered by other customers, instead of with a traditional bookmaker. The Betting Exchange provides better pricing, flexibility, and a level playing field in a massive market. In April of this year, Betfair had over 4 million registered customers worldwide and processed, on average, more than 7 million transactions per day on the Betting Exchange. 

Betfair provides this service via Internet, mobile, and tablet devices that maximize the playing experience, and the Betfair infrastructure implements a service that implements its sophisticated betting market, real-time data processing, and real time clearing of bets.

Betfair is well known for its technically ambitious support for betting during a match. Players can bet while watching a live match, at the last possible moment. When I first met Alex Kozlenkov (Head of Research at Betfair) he described his challenge of executing 100,000 transactions per second against a single product, each transaction (or event) potentially impacts the price of that product, and must be processed in order.

And it’s not just the transactions that are streaming in the Betfair environment, they are also a leader in using real-time data to make the customer experience more engaging, fun, and unique. We’re in the midst of a renaissance of innovation in data management due to “Big Data” – innovative tools and infrastructure designed to manage increasing volume, velocity, and variety of information.  The betting world is no different - new real-time data feeds provide streaming play-by-play data from cricket matches, and real-time social media data brings an opportunity to crowd-source context about the events from fans watching live (in related news, StreamBase recently announced integration with GNIP and connectivity to Hadoop via Flume that enables connectivity to Big Data)

By investing in real-time analytics, Betfair can tap into streaming information and from the betting ecosystem and leverage it to provide a better experience for users live on the site, on mobile devices, and on tablets, providing innovative context and experience to the Betfair experience.

Consumers are now conditioned by social media to expect information and data to be real–time, whether it be pictures of a night out, stock twits, or a location update. Betfair represents a step in the march of all businesses to become more social media aware, real-time, and web-scale. Customers expect more than just a scrolling Twitter feed in the sidebar – they expect real-time context and relevance. The winning firms will integrate a variety of data sources - structured and unstructured, historical and real-time - and distill them down to insight that matters, when it matters. The real-time business of the future requires all of this, and requires real-time analytics and event processing as the new IT computing fabric.

StreamBase LiveView Demonstration and Animated Short

In the first week since the "worldwide premier" of our two StreamBase LiveView videos at the Securities Industry and Finanical Markets Association shown in NYC, we've had almost 1,000 views on Vimeo and YouTube. This product is bringing the power of CEP to entirely new markets and users, and it's very exciting to see interest in it grow so rapidly.

Here's the StreamBase LiveView demonstration video:

 

And our animated short, produced by Grumo Media (thanks, Pedram!)

 

Tech Q&A with CEP Developers, and StreamBase

Today StreamBase hosted an online roundtable entitled “How to Empower Your Development Team with CEP and Visual Programming” featuring developers from J.P. Morgan, GAIN Capital, and Datasports. Combined, the panelists have 35+ years of development experience, and years of CEP experience, and they were kind enough to share those experiences with us, discussing the adoption of CEP technology, developing applications with CEP, and the best practices they’ve learned along the way.

For developers, development managers, architects, and business stakeholders this roundtable tells an interesting story about how CEP technology can truly transform the role of the developer. Take a look at some of the highlights below, watch the replay, then comment below and tell us about your experiences with CEP. 

Talk about your experience with CEP

Phil Martin, Datasports: I like using StreamBase because I like tools that let the computers do stuff that the computers are good at, and let me do stuff that requires thought. As a developer, you wind up doing some tasks over and over again; the structure of the application, domain specific plumbing, etc. Everytime feels like wasted effort. StreamBase lets me focus on the stuff that requires thought, not the details of implementation, connectivity, processing events, and so on. Another comment is around the aggregate operator. It is analogous to another API to learn. It’s an incredibly rich and flexible operator that I think almost qualifies as its own sub language or sub paradigm.  There’s a lot there and a lot of subtlety in its behavior, learn about and study it!

Mike Grandilone, J.P. Morgan Asset Management: The shift to a message passing structure is sometimes a shift in thinking. It's key to understanding how StreamBase handles the life of a tuple; we gained that through experience.  

Talk about the applications you've built with StreamBase

Joseph Chua, GAIN Capital: After 4 days of training and going through the examples, I built my first StreamBase application in just three hours, and it was fully deployed in a week. It connects to market data, manipulates the data, and stores it in a high-speed database. We've also built market making and real-time surveillence applications, and many more to come.

Phil: I've built several applications with StreamBase. One is an "indication of interest" (IOI) app that monitors trading interest and compares it against a CRM system and sends alerts to individual sales traders because they have a customer who either has traded on the symbol, transacted above a given threshold, or has a real-time order for it. The correlation of these factors in real-time also includes real-time news sources. Also, I've worked on two algorithmic trading systems that host their strategies in StreamBase and perform real-time risk checks on the algos, and an ETF arbitrage system where there was only one strategy, but it was a very involved strategy.

Mike: We brought a smart order routing system up and running in 6 months with 2 developers, which we thought was very fast. We also got the business people involved in the canvas and the development - this is a very important factor in getting applications done quickly.

Talk about the benefits of StreamBase and visual programming

Phil: StreamBase helps facilitate communication. Whenever you want to convey information about a system, you always draw a picture. And they almost always look like StreamBase Event Flow. If you have taken the necessary care to clearly delineate modules, then, depending on the technical sophistication of who you are talking to, you can choose how far to drill down. It's extremely helpful in faciliating technical discourse; with the business users, it saves time.

Mike: Visual development has really improved the speed of our development. Especially when you're doing a proof of concept. It helps you understand the application's logic, and also communicate with the business; in typical programming environments like Java, you're not going to show many people the code, but you can do it with StreamBase.

Joseph: The big value is that all the technical details are abstracted away in terms of connectivity and low-level code; we can then communicate with the business owners, so development cycles are shorter and more efficient. The other value we have found is they we can very quickly build a prototype solution so the business can make decisions more quickly with respect to requirements and decisions to develop the applications.

Phil: One thing that's helpful: with StreamBase, you can tell at a glance that spaghetti code is spaghetti code. You can tell if an application was thoughtfully designed without hours and hours and hours of code review.

What best practices have you learned from your experiences with CEP?

Mike: You need to shift your thinking, to think in terms of events, rather than traditional static applications. And, just like any application, build a testing environment at the same time as your application logic.

Joseph: Identify the connection points quickly, be open minded and willing to unlearn the more traditional ways of development, use StreamBase interfaces, and make sure you continuously refactor the application.

Phil: There are lot of resources for ramping up StreamBase skills - invest the effort to develop your skills. The speed of putting together something quickly is a great asset. The thing to keep in mind is that although the development speed of StreamBase is a benefit in the short term, it can, if you're not careful, work against you in the long term.

Phil: While the principles aren't different, the practice IS different. The first thing that sold me is seeing the ease with which you can change and tune your threading model, take things in and out of the threading part of your system, with multiplicity and dispatch methods. Scalability and performance is about taking more advantage of the cores in a system, and careful management of it becomes more and more important.

What was your experience integrating other code into CEP?

Mike: We have separate routing logic written in Java for routing logic that we integrated with our StreamBase application. We've done a lot of work with custom Java operators in StreamBase - it's very straight forward - no problem in doing it. There's lots of power in developing in Java for code that isn't event driven that allowed us to bring in the power of algorithmic trading as a drop-in to the StreamBase environment. Also, we have a C# front end GUI, which uses the .NET interface from StreamBase - this was easy, although the challenge was how to take a GUI that isn't event-based and make it work with an event-driven application. We're in production with that now.

The StreamBase community

Phil: I've put two components onto the StreamBase Component Exchange - mostly adapters. The publishing process was straightforward. One is an adapter that watches for events on the file system and creates StreamBase events when files are created, renamed, modified, or deleted, and another one listens for calls on a web service interface and creates events for the applciation. I suggest other developers look at the utility classes – see what are other developers are doing, and pick up some tips and techniques and helpful hands-on examples of how they are building their applications. 

StreamBase Resources for Developers

Below are the StreamBase resources mentioned in this roundtable:

• Trial version of StreamBase: www.streambase.com/download
• Product documentation: docs.streambase.com
• Personalized demonstrations: www.streambase.com/forms/request-a-demonstration
• “How To” developer webinars: www.streambase.com/webinars
• StreamBase Component Exchange: www.streambase.com/sbx
• StreamBase University: www.streambase.com/sbu
• StreamBase Training and Certification: www.streambase.com/community/training-and-certification
• Datasports blog for StreamBase CEP developers: http://datasports.ca

Introducing Live BI and StreamBase LiveView

For some time now, StreamBase has been working on a new product concept that's a little bit CEP, a little bit data warehouse, and a little bit business intelligence. Today we've unveiled it as StreamBase LiveView. Our customers have called it a "game changer," and a product that "fills a massive hole in the BI space." Hyperbole like that needs to be backed up, so here we go.

Complex event processing (introduction on Google+) was envisioned by its inventors as a "database for moving data" as firms increasingly deal with data in motion from mobile devices, social media and digitized data of all types, and in all industries. In its first decade of commercial use, CEP has come to power automated trading on Wall Street. A who's who list of firms have been public about their adoption of CEP, so let's examine the live BI problem in the context of Wall Street.

Industry analysts estimate that over 60% of U.S. equities trading is automated. The problem is, with so much automation, it's impossible to see and act on what's going on, because a typical trading environment can generate 10's of millions of events a day. Existing business intelligence and data warehouse tools aren't used in the front office of Wall Street, because they're designed to summarize what's already happened, not what's happening now. 

StreamBase LiveView brings BI and streaming data together, for the first time. Applications that need Live BI are everywhere on Wall Street - trading risk management, position management, P&L management, surveillance, compliance, and trading itself, as Waters Technology magazine describes in the article that StreamBase Seeks 'Radical Transparancy' in Business Intelligence.

And applications in other markets can be found in any environment that has big data in motion: telecom (location aware insight and live network operations), security (live fraud detection, live surveillance), and e-commerce (live social media analytics, ad placement, big streaming analytics for Hadoop and Flume). These apps can be deployed on premise or distributed in the cloud. 

StreamBase LiveView is built with StreamBase CEP, which provides connectivity to over 125 event-based sources of data and the extreme scalability required by live BI. The server-based architecture is designed to process hundreds of millions of events per node, absorbed from a network of event sources such as an enterprise message bus or scalable event absorption layer like Flume or IBM Streams. At the front end, users can slice and dice streams of data with ad-hoc query tools in the StreamBase LiveView Desktop (shown above). But here's the key: once users ask the questions they want answered, StreamBase LiveView keeps answering their questions - as conditions change in real-time; users see an always-live view of streaming data.

For the user, all this technology means just one thing: they see business intelligence that's constantly fresh and relevant to their business operations. It's not a view of 5 days ago, or even 5 minutes ago. The data is live, the insight is live, and live insight can lead to live action.

Facebook, Twitter, Google+ give us a live view of what's happening in our social networks; StreamBase LiveVew gives firms a live view of their enterprise data.

To learn more, watch the StreamBase LiveView presentation, follow new materials, articles, and resources on Google+ (+StreamBase and CEO +Mark Palmer), read industry analyst Curt Monash, DMBS2.0, or follow StreamBase Systems (@streambase), CEO Mark Palmer (@mrkwpalmer ), and CTO Richard Tibbetts (@tibbets) on Twitter.

 

Real-Time Big Data and the 11 Principles of Modern Surveillance Systems

(This article is also posted in the TABB Group's TABBForum, and generated a broad discussion in the comments there. To participate, visit the original post here.)

In his TabbFORUM article, Dave Tolladay eloquently argues that real-time surveillance is crucial in today's high-frequency trading landscape, and I agree. Ten years ago, real-time computing began to influence trading; today, HFT is pervasive and existing surveillance techniques, designed to run at the speed of click trading, should be considered to be legacy architectures in need of a radically new, modern approach. 

But managing big data in real-time is a new problem of physics, and those physics come center stage when it comes to surveillance. A recent panel on modern surveillance systems, Miranda Mizen from TABB Group, Justin Amos from Redkite Financial Systems and Richard Tibbetts from StreamBase compared modern surveillance systems to “anti-virus software for trading.” 

In the course of their conversation, Justin, Miranda and Richard touched on 11 key points about the “next generation” architectural requirements for surveillance, including: 

1. Real-time analytics. Surveillance systems must analyze market data at the speed of automated trading. That means real-time computing and complex event processing (CEP) as one of the critical technology enablers of modern surveillance systems. 

2. Historical “big data” analytics. Surveillance systems must put activity in context – suspicious activity can be an aberration or a pattern, based on historical actions. The big data movement has spurred a renaissance in the database industry with firms like Netezza (now IBM), Vertica (now HP), MongoDB (open source) as the flag bearers for the NoSQL (Not Only SQL) database revolution that aims to supplant legacy RDBMS systems to deal with the rising deluge of data described in the Economist piece. In surveillance, “big data analytics” allow on-line access to trade-by-trade history for investigation, trending, and discovery to be mashed up with real-time data to provide a real-time and historical context to behavior. 

3. Inclusion of new types of data sources. Justin suggested that the inclusion of real-time news – and even social media data sources – can be a benefit in correlating, for example, suspicious trading on news. 

4. Data fusion. Real-time, historical and new data sources must be fused for a complete picture of intra-day trading behavior, and a 360 degree view of market data, activity and sentiment. 

5. Multi-asset class support. Since HFT is becoming increasingly cross-asset class, so must modern surveillance systems. 

6. Pre-built algorithms. Todayʼs surveillance systems must support standard algorithms like front running, spiking, tailgating, and ramping that can mistakenly or purposefully occur in computer-based HFT systems. 

7. Adaptive algorithms. The ability to adapt algorithms on the fly to reduce false positives is important, due to the increasingly dynamic capability of modern HFT systems to adapt their behavior. 

8. Algorithm extensibility. The ability to quickly add new algorithms to the system as market dynamics evolve is also a key modern requirement for systems – they must keep pace with the rate of change of modern trading algorithms. 

9. Flexible workflow management and case management. Surveillance systems generally donʼt make automated decisions to change trading behavior or stop trading on their own; instead, they tend to guide a series of workflow steps to alert compliance officers, portfolio managers or traders and manage the process of alert analysis, resolution and tracking. 

10. Accessibility for the business analyst. With regulation changing so quickly, more firms want their own control over surveillance behavior, so some provide tools that allow compliance personnel to discover and implement surveillance algorithms on their own. 

11. Cloud-based or on-premise deployment. Surveillance is not typically the core expertise of trading groups, so the ability to deploy systems either in the cloud or on premise allows for more rapid and global rollout of surveillance tools. 

Real-time big data is a change in computing physics on Wall Street. New requirements motivate new solutions, and the new solutions and innovation coming from the surveillance community continues to prove that innovation never sleeps on Wall Street.

 

StreamBase News at SIFMA

The SIFMA show in New York is always a big week for news for firms that do business in the capital markets. This year, we had so much new news  that we issued a series of announcements over the last several weeks. In case you missed any of them, here's a summary of what we're talking about at SIFMA this year.

1. Thomson Reuters Selects StreamBase CEP. This is a huge endorsement from a leader in the capital markets. Thomson Reuters will utilize StreamBase in their open, global, ultra-high speed and resilient network for the financial industry, Elektron, as an integrated element in their Enterprise Platform for Velocity Analytics, as an embedded CEP engine in Thomson Reuters applications, and in their Thomson Reuters Dealing Aggregator FX Trading Solution. We're very honored to be selected by Thomson Reuters!

2. BMF&BOVESPA Selects StreamBase CEP. The largest exchange operator in South America selected StreamBase as their enterprise-wide deployment platform for event-based applications - yet another huge endorsement and evidence of the increased use of CEP for enterprise-wide deployments. 

3. StreamBase in Asia: Complex Event Processing in Trading with SunGard. Our partner SunGard recently held a online panel discussion about algorithmic trading, market data management, risk management, and other other applications in Hong Kong with Benjamin Becar, algorithmic trading product management for SunGard in Hong Kong, and Richard Tibbetts, StreamBase CTO. You can watch the replay of the session here.

4. New product release: StreamBase 7. Fantastic new usability, and performance improvements in four different dimensions.

5. StreamBase University Program. Our new program to provide StreamBase software and certification resources to universities to help students learn about the growing field of CEP in school. The StreamBase University Program is integrated with our StreamBase Developer Certification Program and StreamBase Component Exchange community for sharing StreamBase integrations, code examples, frameworks, and adapters. 

6. Erjang / Erlang integration and the growing StreamBase Component Exchange Community. The latest contribution from Darach Ennis (@darachennis) has been a big hit in the Erlang community and on SlideShare. Watch it here: Complex Erjang / Erlang Processing with @StreamBase. 

7. CEP in Risk Management and Surveillance. CEP isn't just for algo trading anymore! Broad adoption by firms like Thomson Reuters and BMF&BOVESPA, and increasing use in applications such as risk management and surveillance illustrate that CEP is becoming a strategic platform for firms on Wall Street, and for third party developed applications with "CEP Inside."

8. "Faster than a Flash" - MIT Technology Review on CEP for market data risk management. MIT Technology Review published a story on how proprietary trading firm PhaseCapital used CEP to have ZERO bad trades on the day of the flash crash. While other pundits promote fear of a looming "splash crash" - bigger and worse than the "flash crash," we're simply trying to promote firms that are actually doing something about it with innovative technology approaches to make trading safer.

I'll be at the show in New York this week and addressing questions on Twitter about these developments and others at @mrkwpalmer.