Welcome to SPBench’s documentation!

Note

This project is under active development.

SPBench (Stream Processing Benchmark) is a framework for benchmarking C++ stream processing applications. The main goal of SPBench is to enable users to easily create custom benchmarks from real-world stream processing applications and evaluate multiple parallel programming interfaces (PPIs).

The SPBench is intended for three main audiences:

• Users who want to run performance tests with the SPBench benchmarks. It implements different real-world stream processing applications using different parallel programming interfaces (PPIs) and parallelism patterns, and makes available the main metrics used in this domain. The framework also enables users to create custom benchmarks with new PPIs.

• Researchers and developers who want to test and evaluate new technologies and solutions. The SPBench benchmarks are also highly parameterizable and its API allows for easy and fast code reuse across all applications.

• Students and teachers who want to learn/teach stream parallelism. The benchmarks implemented with the SPBench API abstract the low-level code and expose to users only the stream core of each application (in a few lines of code). Therefore, it allows users to easily identify each individual operator and data dependencies, to understand what the stream flow looks like. Then users can build parallelism on top of this highly simplified code.

---

SPBench benchmark suite

The SPBench suite comprises the following stream processing applications and will be more in the future:

• Ferret (PARSEC)

• Lane Detection

• Bzip2

• Face Recognizer (Person Recognition)

See more details in Applications.

For each of them, there are parallel benchmarks implemented using the following parallel programming interfaces and will be more in the future:

• Intel TBB

• FastFlow

• OpenMP

• ISO C++ Threads

• SPar

• GrPPI (with OpenMP, FastFlow, TBB, and ISO C++ threads backends)

---

SPBench framework

One of the aspects that most differentiates SPBench from related solutions is how users interact with it. The kernel of SPBench is its API that allows users to access simplified versions of the applications included in our suite. Based on the sequential versions, users can implement parallelism, create new custom benchmarks with different parallelism strategies, or even explore new parallel PPIs. Users can also configure and modify parameters, such as build dependencies and metrics, through a command-line interface (CLI). This CLI accesses a database containing all benchmarks from the SPBench suite. Other secondary parameters can be tuned via the CLI with simple commands. This allows users to entirely focus on writing and tuning the parallelism rather than spending time with the non-relevant low-level aspects of each application. The figure below illustrates the SPBench framework.

---

You can download the slides here.

---

Contents

• Download Links
• Basic Concepts
• Getting Started
  • SPBench Dependencies
  • Command-Line Interface
  • Installing SPBench
  • How to use
  • Inputs
• SPBench Applications
  • Ferret (PARSEC)
  • Lane Detection
  • Person Recognition
  • Bzip2
• SPBench Benchmarks
  • Single source benchmarks
  • Multiple source benchmarks
• How to install
  • SPBench Dependencies
  • Applications Installation
  • Workload installation
• Command-Line Interface
  • SPBench install
  • Global configuration file
  • Operations with benchmarks
    • Update benchmarks
    • Add a new benchmark
    • Edit a benchmark
    • Configure a benchmark
    • Compile benchmarks
    • Execute benchmarks
    • List benchmarks
    • Delete a benchmark
    • Rename a benchmark
    • Clean a benchmark
  • Operations with input workloads
    • Download input workloads
    • List inputs
    • Add new input ID
    • Delete a input ID
  • Operations with applications
    • List applications
    • Add a new application
    • Delete a application
  • Operations with operators
    • List operators
    • Edit an operator
    • Reset operators
• Compiling Configuration
  • JSON Configuration File
  • Example
  • Global Compiling Configuration
  • UPL
• SPBench workloads
  • Workload Classes
    • Lane Detection
    • Bzip2
    • Person Recognition
    • Ferret
  • Using External Inputs
    • Adding Result Checking
  • Workload Characterization
• Performance Metrics
  • Statically Selecting Performance Metrics
  • Dinamically Selecting Performance Metrics
  • Performance Metrics for Multi-source Benchmarks
• Management Options
  • General Management
  • Batching Management
  • Frequency Management
  • Management Options for Multi-source Benchmarks
• Moving SPBench
• Adding New Applications
• Publications
  • How To Cite
  • Other papers using SPBench
• Copying
• About
  • Design Collaborators
  • Dev Collaborators