Command-Line Interface

The SPBench framework works with a command-line interface (CLI).

The CLI offers some commands. You can check them running:

./spbench --help

For each command there is also a “help” guide. You can access it using:

./spbench [command] -h

Install

./spbench install -h

Description:

It install the dependencies for the SPBench benchmarks. See (INSTALL LINK) for more information.

Optional arguments:
  • -app <some_application>

    Install dependencies only for the seleted application.

Update

Description:

Use it for setting up the paths and for generating new makefiles for the benchamarks benchmarks.

You must run it everytime you change the SPBench path or after running the ‘configure’ command.

Usage examples:

For all applications:

  • ./spbench update

For a specific benchmark:

  • ./spbench update -bench <benchmark_name>

For all benchmarks based on a specific application:

  • ./spbench update -app <application_name>

For all benchmarks implemented with a specific PPI:

  • ./spbench update -ppi <ppi_name>

For more info:

  • ./spbench update -h

New

./spbench new -h

Description:

It allows for creating a new benchmark based on a SPBench application. This command will create a fresh copy of the selected application and add it to the database.

Required arguments:
  • -bench <new_benchmark_id>

    Set a name for your new benchmark.

  • -app <application_name>

    You must insert an application ID. See (applications page link) for more details.

  • -ppi <any_ppi>

    We recommend inserting a name for the PPI you will use here. But you are free to insert any identification word.

Optional arguments:
  • -from <other_benchmark_id> (Optional)

    Use this option to create your new benchmark as a copy of an existent benchmark. Insert the name of the benchmark you want to copy from

  • -nsources

    Use this option to create a new benchmark with support for multiple sources

Usage example:

  • ./spbench new -bench <new_bench_name> -app <application_name> -ppi <ppi_name>

Edit

./spbench edit -h

Description:

It opens a choosen benchmark for editing the source code.

Required arguments:

  • -bench <benchmark_id>

Optional arguments:
  • -editor <editor_name> (Default: nano)

    You can select a different text editor available in your machine.

Usage example:

  • ./spbench edit -bench <benchmark_name> -editor <text_editor_name>

Configure

./spbench configure -h

Description:

It allows users to add compiling dependencies for a benchmark. It opens a JSON configuration file for users to write the required dependencies.

Required arguments:

  • -bench <benchmark_id>

Optional arguments:
  • -editor <editor_name> (Default: nano)

    You can select a different text editor available in your machine.

Usage example:

  • ./spbench configure -bench <benchmark_name> -editor <text_editor_name>

Exec

./spbench exec -h

Description:

This command runs a given benchmark.

Required arguments:
  • -bench <benchmark_id>

    Name of the benchmark to execute.

  • -input <input_id>

    Insert an input ID (run ./spbench list-inputs). You can also insert multiple inputs for multiple source benchmarks.

Optional arguments:

  • -batch <batch_size>

    It can be used to change the batch size (default: 1).

  • -frequency <item_delay> (microsecond)

    Items will be generated in a constant frequency at source. You must indicate a time interval in MICROseconds, e.g. 1000 (= 1 millisecond). The minimum frequency you can set is 0 microsecond. However, the actual minimum limit will be defined by your system’s memory access speed. E.g., the command -frequency 0 is equivalent to -in-memory.

  • -nthreads <number_of_threads>

    A global variable that can be used to change the number of threads for the parallel benchmarks.

  • -in-memory

    It runs the application in in-memory mode.

  • -latency

    Print average latency results at the end of the execution.

  • -throughput

    Print the average throughput at the end of the execution.

  • -resource-usage

    It prints the global memory and CPU usage at the end of the execution. To use it is required to run the benchmark as root or adjust paranoid value.

  • -monitor <time_interval> (millisecond)

    Use this option to monitor latency, throughput, CPU and memory usage. You must indicate a monitoring time interval in milliseconds (e.g. 250).

  • -latency-monitor

    It monitors latency per stage and per item and generates a log file.

  • -user-arg <user_custom_argument>

    User custom argument. Allow users to pass one or more arguments to use inside the benchmarks. Single argument: [-user-arg single_argument] or [-user-arg “single argument”]. Multiple arguments: [-user-arg two arguments] or [-user-arg two -user-args arguments].

  • -test-result

    This option will run a correctness test on the output (when available) after the benchmark execution.

  • -d (Bzip2 only)

    Enables decompress mode for Bzip2 benchmarks.

Usage example:

  • ./spbench exec -bench ferret_tbb_farm -input small -nthreads 8

    Simple run.

  • ./spbench exec -bench lane_sequential -input medium -latency -throughput -resource-usage -test-result

    Running with basic metrics and result testing.

  • ./spbench exec -bench bzip2_ff_farm -d -input large -nth 6 -latency-monitor -monitor 300

    Running Bzip2 in decompress mode with monitoring metrics enabled.

  • ./spbench exec -bench person_seq_ns -input large -input my_custom_input -in-memory

    Running multiple source Person Recognition benchmark with two inputs and in-memory enabled.

  • ./spbench exec -bench my_custom_bench -input small -batch 4 -frequency 50000 -user-arg "My awesome argument"

    Running a custom benchmark with batch size = 4 and inter-item delay of 50 ms, plus a custom argument.

List

./spbench list -h

Description:

This command lists all SPBench benchmarks, including user’s custom benchmarks.

Optional arguments:
  • -app <application_name>

    Lists only benchmarks based on the given application.

Usage example:

  • ./spbench list

Delete

./spbench delete -h

Description:

It completely deletes a given benchmark.

Required arguments:
  • -bench <benchmark_id>

    Name of the benchmark to delete.

Usage example:

  • ./spbench delete -bench my_not_so_awesome_bench

New-input

./spbench new-input -h

Description:

It allows for adding an alias for a custom input. The chosen alias will be registered for future uses. You must use this option to run the benchmarks using other workloads than the ones provided by SPBench.

Required arguments:
  • -id <new_input_id>

    Set a name for your new input (existent entries will be replaced)

  • -app <application_name>

    Insert an application from the SPBench suite.

  • -input "<input_string>"

    You must insert the input string (e.g., “/home/user/my_input_file …”)

Optional arguments:
  • -md5 <md5_hash_for_result_testing>

    SPBench uses the associated md5 hash to check the correctness of the result for this input (Optional). ATTENTION: do not insert the input file resulting md5 hash here. You must insert the md5 hash regarding the resulting output file. We strongly recommend that you run your new input with a sequential benchmark to compute this hash of the resulting output file.

Usage example:

  • ./spbench new-input -id immensely_large -app bzip2 -input "/home/me/awesome_songs/never_gonna_give_you_up-remix.mp3"

Delete-input

./spbench delete-input -h

Description:

Delete a given input ID from the SPBench inputs registry. It does not delete the associated files.

Required arguments:
  • -id <new_input_id>

    Insert the ID of the input to delete.

  • -app <application_name>

    Insert the associated app for the given input.

Usage example:

  • ./spbench delete-input -id my_super_extra_medium_input -app lane_detection

Rename

./spbench rename -h

Description:

This command can be used to change the name of a given benchmark inside SPBench.

Required arguments:
  • -old-name <old_bench_id>

    Insert the old and existent benchmark ID.

  • -new-name <new_bench_id>

    Insert the new ID.

Usage example:

  • ./spbench -old my_certainly_working_bench -new my_bench-v0.1-alpha

List-inputs

./spbench list-inputs -h

Description:

This command lists all application’s inputs provided by SPBench, including users custom ones.

Optional arguments:
  • -app <application_name>

    Lists only inputs based on the given application.

Usage example:

  • ./spbench list-inputs

Download-inputs

./spbench download-inputs -h

Description:

Download the inputs for the SPBench benchmarks.

Optional arguments:
  • -app <application_name>

    Use this option for downloading inputs for a specific application.

  • -class <workload_class> (It applies only for Bzip2)

    You can insert the class of workload you want to download (leave it empty to download all classes). Possible options: all, small, medium, large.

  • -force

    Use this option to delete any respective input file and download it again.

Usage example:
  • ./spbench download-inputs (recommended)

    It will download everything.

  • ./spbench download-inputs -app bzip2 -class small -force

    Bzip2 has some large files to download, so users can use this option to avoid downloading everything.