Under the hood, the tests use tb_cpu_program Verilog testbench.
Python script called run.py is used for testing.
This will use makefiles under the hood,
one from the project root, and other from tests/official subdirectory.
There are two types of tests, official riscv tests that work by selftesting the processor. That means that if there is a problem with more instructions like branches, store words, it's possible that the tests will pass even though the processor is not working properly.
The other type are custom tests. These do not depend on functionality of processor instructions. There is just a few of these. For each program, it's possible to specify different memory to load, so multiple parameters are passed to the same program.
To validate test success, memory is dumped to a file at the end of the test - upon getting to ebreak instruction. This memory is then validated against file with expected memory values. The memory file format is Verilogs hex format.
Selftests are checked the same way, but upon success, on address zero, there should be 0xAA. On fail, 0xFF. Here are the macros used for test pass and fail
#define RVTEST_PASS \
addi x1, zero, 0xAA; \
sw x1, 0(zero); \
nop; \
ebreak;
#define TESTNUM gp
#define RVTEST_FAIL \
addi x1, zero, 0xFF; \
sw x1, 0(zero); \
sw x1, 4(TESTNUM); \
nop; \
ebreak;
These macros are defined in tests/official/env/p/riscv_test.h.
To run the tests, run.py can be used.
By default, it will run the custom tests.
It must be supplied one of run or list subcommands.
To list all the tests, use ./run.py list. To run the tests,
supply ./run.py run.
Every test has a group. For custom tests the group specifies the program to run.
Name of the test specifies the arguments.
For official tests, there is only one group so far, rv32ui.
It's possible to filter one or more of the tests,
by using --filter flag. For example ./run.py run --filter gcd
to run only tests from the gcd group. For running only one test
from the group, specify it after a dot. For example ./run.py --filter gcd.1071-462-21.
usage: run.py [-h] [-f [FILTER ...]] [-t {custom,official}] [--trace] [--print-registers] {run,list}
Test simple RISC-V processor written in Verilog.
positional arguments:
{run,list} What to do. Either run the test, or run testcases based on filter.
options:
-h, --help show this help message and exit
-f [FILTER ...], --filter [FILTER ...]
filter, should be in group.test format
-t {custom,official}, --type {custom,official}
type of the testcases, either custom testcases or official riscv selftests
--trace trace, produce vcd file
--print-registers dump registers on ebreak to stdout
To create a custom test, two things are needed.
First, a C program has to be made. This program is expected to be located in
../programs/. It should have a main function. This function will be called.
It should ideally load parameters from the beginning of the memory, and save
the result to the memory afterwards, again, at the beginning of the memory.
Second, arguments have to be specified. To do this, first create a directory
custom/$program. In this directory, files with parameters should be placed.
For each test, two files are needed, input memory and expected memory.
Name of the files should be $test-input.dat and $test-expected.dat.
They should be in Verilog hex format (input will be read by readmemh, expected
data will be compared against file written with writememh). Only first few
words have to be specified, no need to put in full memory image.
I have chosen to name the files by parameters passed in, every parameter separated by a dash. And at the end, the expected result is listed. For example, gcd.1071-462-21 will pass in 1071 and 462 as arguments to the gcd program. The expected result (gcd(1071, 462)) is 21.