How to run SPBench

Command-Line Interface

The SPBench framework works with a command-line interface (CLI).

The CLI offers some commands. You can check them running:

./spbench --help

For each command there is also a “help” guide. You can access it using:

./spbench [command] -h

How to use

In this framework you can simply run the benchmarks that are already provided or implement your own specific benchmark.

To run the benchmarks you first need to have installed the application dependencies and also the workloads. See pages How to install and SPBench workloads for more details.

After installing the workloads and dependencies (make sure you set the environment variables source setup_vars.sh), you can run the ‘list’ command to see all the pre-available benchmarks in SPBench.

./spbench list

Then you can select one of the benchmarks from the list and use the command below to compile it.

./spbench compile -bench <selected_bench_name_from_list>

If the compilation was successful, just run it via the ‘exec’ command.

./spbench exec -bench <selected_bench_name_from_list> -input <input_id> ... [optional_commands] <optional_args>

Check Command-Line Interface for other options.

Inputs

In the exec step you must choose a given input for the benchmark.

These inputs can be downloaded and will be stored in the inputs/ directory.

You can run the ./spbench download-inputs to automatically download them.

You can use your own input or choose a given one.

You can check the given inputs through:

./spbench list-inputs

See the SPBench workloads page for more details about workloads.

How to run

To do a simple benchmark run, you can use the command below.

./spbench exec -bench <benchmark_id> -input <input_id>

A practical execution example: .. code-block:: bash

./spbench exec -bench ferret_sequential -input medium

Performance metrics

With SPBench you can evaluate four performance metrics in different ways.

  • Latency

  • Throughput

  • CPU usage (%)

  • Memory usage (KB)

Here are four arguments you can use when running the ‘exec’ command:

  • -monitor <time_interval>

    It monitors latency, throughput, CPU and memory usage. Here you can indicate a monitoring time interval in milliseconds (e.g. 250.Optional). It will generate a log file inside a log folder (spbench/log/). This file will contain the results got for each metric at each time interval.

  • -latency

    It prints the average latency results for each application operator on the screen after the execution.

  • -throughput

    It prints the average throughput results after the execution.

  • -latency-monitor

    It monitors the latency per stage and per item and generates a log file inside the log folder.

  • -resource-usage

    It prints the global memory and CPU usage for the selected application.

Management arguments

All of these are also OPTIONAL arguments that you can use within the ‘exec’ command.

  • -in-memory

    It runs the application in-memory, which means that all the input is first loaded into the memory before start processing it. The opposite is true for the writing phase, where the result stays in memory and is only writen to the disk after all the input is proccessed.

  • -nthreads <number_of_threads>

    Here you can define a number of threads to run the parallel versions. To be more preciselly, the degree of parallelism of the parallel stages (Obs.: some PPIs can create more threads than parallel stages). It shows up as a global variable that can be used inside the parallel versions as spb::SPBench::nthreads. The default value is 1.

  • -batch <batch_size>

    Here you can change the size of the batches, so each item can carry more data. By default, the size of each batch is 1.

  • -d

    This a specific argument for bzip2. You can use it to run this application in decompress mode. Observe that for de decompression mode it is required to select a compressed workload class version, e.g. small_d, medium_d, or large_d.

Itens attributes

All itens implements some attributes that you can use to build customizations. (forthcoming)

Tip

  • You can run source setup_vars.sh to load the libraries.

  • You must load the libraries before using SPBench every time you star a new session.

  • To make FastFlow/SPar properly recognize and map the cores of the processor you must run at SPBench root bash ./ppis/SPar/libraries/ff/mapping_string.sh and bash ./ppis/fastflow/ff/mapping_string.sh. Only required once. However, this FastFlow feature may not be able to automatically set this value in the .../ff/config.sh file. In this case, you must do manually update the config.hpp file with the result given by the mapping_string.sh script.