Unit tests overview

As we have seen in a previous post, we can launch the unit test suite through the C emulator using:

$ cd $ROCKET_ROOT/rocket-chip/emulator
$ make -jN run-asm-tests
$ make -jN run-bmark-tests
$ make -jN run-asm-tests-debug # With waveforms

It is also possible to launch individual tests:

$ make output/rv64ui-p-add.out # Emulation
$ make output/rv64ui-p-add.vcd # Waveforms

Each run produces the Verilog code of the chip (through Chisel):

$ ls $ROCKET_ROOT/rocket-chip/emulator/generated-src
freechips.rocketchip.system.DefaultConfig
freechips.rocketchip.system.DefaultConfig.0x0.0.regmap.json
freechips.rocketchip.system.DefaultConfig.0x0.1.regmap.json
freechips.rocketchip.system.DefaultConfig.0x2000000.0.regmap.json
freechips.rocketchip.system.DefaultConfig.0x40.0.regmap.json
freechips.rocketchip.system.DefaultConfig.0xc000000.0.regmap.json
freechips.rocketchip.system.DefaultConfig.anno.json
freechips.rocketchip.system.DefaultConfig.behav_srams.v
freechips.rocketchip.system.DefaultConfig.conf
freechips.rocketchip.system.DefaultConfig.d
freechips.rocketchip.system.DefaultConfig.dts
freechips.rocketchip.system.DefaultConfig.fir
freechips.rocketchip.system.DefaultConfig.graphml
freechips.rocketchip.system.DefaultConfig.json
freechips.rocketchip.system.DefaultConfig.memmap.json
freechips.rocketchip.system.DefaultConfig.plusArgs
freechips.rocketchip.system.DefaultConfig.rom.conf
freechips.rocketchip.system.DefaultConfig.v
TestHarness.anno.json

Along with the chip, the associated C++ code generated by Verilator is also produced:

$ ls $ROCKET_ROOT/rocket-chip/emulator/generated-src/freechips.rocketchip.system.DefaultConfig
VTestHarness__1.cpp
VTestHarness__2.cpp
VTestHarness__3.cpp
...

Note: For tests with waveforms, obtained specifying either make output/<test_name>.vcd or globally with make <suite_name>-debug, the generated files are in generated-src-debug.

In the end, the .out file (or .vcd) is generated from its corresponding test in the riscv-tests repository. The .out file contains a cycle-by-cycle dump of write-back stage of the pipeline.

C0: 483 [1] pc=[00000002138] W[r 3=000000007fff7fff][1] R[r 1=000000007fffffff] R[r 2=ffffffffffff8000] inst=[002081b3] add s1, ra, s0
C0: 484 [1] pc=[0000000213c] W[r29=000000007fff8000][1] R[r31=ffffffff80007ffe] R[r31=0000000000000005] inst=[7fff8eb7] lui t3, 0x7fff8
C0: 485 [0] pc=[00000002140] W[r 0=0000000000000000][0] R[r 0=0000000000000000] R[r 0=0000000000000000] inst=[00000000] unknown

RISC-V unit tests

Rocket uses the riscv-tests through its rocket-tools as its test suite.

Those tests use Tests Virtual Machines (TVM), a set of definitions to ensure the reproducibility of each test program. Each TVM is defined over an instruction set and execution privilege. For example, rv32ui covers RV32I at user-level while rv64sv covers RV64IV at supervisor level. Each TVM defines:

  • registers
  • instructions
  • portions of memory
  • execution
  • test data
  • test results

A test program for RISC-V is written within a single assembly language file, which is passed through the C preprocessor, and all regular assembly directives can be used. Each test program should first include the riscv_test.h header file, which defines the macros used by the TVM. Each TVM runs on a specific environment from one of the following:

  • p: virtual memory is disabled, only core 0 is available
  • pm: virtual memory is disabled, all cores are available
  • pt: virtual memory is disabled, timer interrupt fires every 100 cycles
  • v: virtual memory is enabled

Unit tests workflow

Dissecting the emulator Makefile and co., we can look at what “exactly” is happening when we launch a unit test.

The emulator’s Makefile defines flags and directories and launches the final commands to run the tests and benchmarks. It includes the root Makefrag and the Makefrag-verilator.

The root Makefrag builds Chisel3 and FIRRTL from source, it sets several jars and sbt configuration (memory size, timeout cycles, etc.).

The Makefrag-verilator rules:

  • generate the FIRRTL intermediate representation of the core
  • generate the Verilog files from the core as well
  • build and install Verilator directly
  • run Verilator to emulate the circuit

The test itself is gathered from the installed riscv-tests at and a symbolic link is setup in the output.

$ ln -fs /opt/riscv-rocket/riscv64-unknown-elf/share/riscv-tests/isa/rv64ui-p-addi output/rv64ui-p-addi

The test itself is ran on top of the emulator with the command:

./emulator-freechips.rocketchip.system-freechips.rocketchip.system.DefaultConfig \
    +max-cycles=100000000 \
    +verbose output/rv64ui-p-addi \
    3>&1 1>&2 2>&3\
    | /bin/spike-dasm  > output/rv64ui-p-addi.out\
    && [ $PIPESTATUS -eq 0 ]