IBM Streams

Key features

Real-time Stream Processing

Continuously and rapidly analyzes massive volumes of moving data.

High Data Throughput

Can handle very high data throughput rates (millions of messages or events per second).

Low Latency

Provides sub-millisecond response times for real-time analysis.

Programming Model & Language (SPL)

Offers a specialized programming language (Streams Processing Language) for developing streaming applications.

Multi-language Support

Supports development in other languages like Java, Scala, and Python.

Integrated Development Environment (IDE)

Provides IBM Streams Studio for simplified application development (for on-premises installations).

Runtime System

Executes applications on a single or distributed set of resources.

Cloud-Native Architecture

Can be installed and deployed on Red Hat OpenShift or Kubernetes environments using Kubernetes operators and Docker images.

Deployment as a Service

Available as a service for IBM Cloud Pak for Data environments.

Automated Deployment

Automatically deploys stream processing applications on configured hardware.

Incremental Deployment

Allows extending stream processing applications without restarting.

Secure & Auditable Runtime Environment

Provides security and auditing features for the runtime environment.

Dynamic Monitoring

Continuously monitors the state and utilization of computing resources, with dynamic monitoring of running applications via Streams Console, streamtool commands, and REST APIs.

Unstructured Data Handling

Capable of processing virtually any data, whether structured or unstructured (texts, audio, images, voice, video, web traffic, GPS data, email, financial transactions, satellite data, sensor logs).

Integration Capabilities

Integrates with other data infrastructure like Hadoop, Spark, and external data sources via edge adapters. Can also integrate with Business Rules software like Operational Decision Manager (ODM).

Data Lifecycle Stages

Supports common stages in streaming applications:

Ingest

Consumes live data from disparate sources (sensors, social media, databases, file systems).

Prepare

Parses, transforms, filters, cleans, aggregates, or enriches data in memory for real-time analytics.

Detect & Predict

Performs real-time analysis, gaining insights using built-in toolkits (timeseries, R toolkit) or custom operators.

Decide

Creates logic to act on insights, potentially integrating with business rules engines.

Act

Sends analysis results to visualization servers, sends alerts, or publishes results to subscribers.

Toolkits

Provides a set of toolkits for data analysis (e.g., timeseries for modeling, anomaly detection, forecasting; R toolkit for R scripts).

Fault Tolerance & High Availability

Designed to manage high availability, automated fault tolerance, and recovery.

Reduced Data Storage

Helps reduce data storage by filtering and extracting only relevant information in real-time.

Dynamic Resource Allocation

Determines where best to deploy operators to meet resource requirements of new and existing workloads.