Dear reader,
This blog is moving to substack: https://fprox.substack.com/
You will find past articles (with corrections and extensions) and new ones as time goes on.
For example, a list of posts related to the RISC-V vector extensions can be found here: https://fprox.substack.com/p/risc-v-vector-in-a-nutshell
If you'd like you can subscribe to the substack and receive an email when new posts are published.
See you there.
RISC-V Vector extension has defined multiple ways to organize vector data over one or multiple vector registers. In this previous blog post we presented the concept of vector register groups defined by RVV 1.0: the capacity to group together multiple vector registers into a meta (larger) vector register which can be operated on as a single operand by most instructions and thus extend the size of vector operands and results. Recently a new concept was introduced: vector element groups: considering multiple contiguous elements as a single larger element and operate on a group as if it was a single element. The concept was suggested by Krste Asanovic in this email; and later specified in a standalone document of the vector spec: https://github.com/riscv/riscv-v-spec/blob/master/element_groups.adoc.
A vector element group is defined by an effective element width (EEW) and an element group size (EGS), it is a group of EGS elements, each EEW-bit wide. The total group width (in bits) is called the Element Group Width (or EGW, EGW = EEW * EGS).
NOTE: the single element width parameter implies that all elements in an element group have the same width.
The element group is useful to manipulate multiple data elements which make sense as a block (e.g. a 128-bit ciphertext for the AES cipher algorithm) without the need to define large element widths and implement their support in hardware.
An element group can be fully contained in one vector register or can overlap multiple registers. In the former case, a single vector register can contain multiple element groups.
An element group can also have EGW larger than an implementation VLEN; in this case a multi-register group is required to fit a single element group. The same constraints as any vector register group apply: the register group is encoded by its first register whose index must be a multiple of the group size.
EEW can either be specific to the opcode or defined through SEW. For example most vector crypto instructions defines EEW from SEW (even if only a small subset of values are legal): it is required to define vtype properly before executing a vector crypto instruction. This is in particular useful to reuse the same instruction for different algorithm: e.g. setting SEW=32 bits, vl=4 and executing a vsha2c will perform a SHA-256 message compression, while setting SEW=64 bits and vl=4 and executing a vsha2c will perform a SHA-512 message compression . We will provide more detail on the new vector crypto extension in a future post.
Contrary to EEW, EGS is always defined by the opcode: it is not a new vtype field. For example vsha2ms (SHA-2 message schedule) statically defines EGS as 4.
The following diagram illustrates two examples of element groups. The top element group has EGW half as wide as VLEN and so two element groups fit in a single vector registers. The bottom example has EGW twice as wide as VLEN and so a 2-register group is required to fit a single element group.
The element group concept has been first used by the vector cryptography extension proposal (draft under architectural review at the time of writing). Different element groups configurations are used:
In this second blog post on RISC-V compressed extension we will review the Zc* family of extension designed to improve code size for embedded platforms (if you have not read the first part, it is accessible here).
Instructions of the Zcmp and Zcmt extensions reuse encodings of compressed double precision floating-point loads and stores and are thus incompatible with these instructions. This is in line with the philosophy of those new extensions: improve code size for embedded platforms where floating-point instructions are less critical (and may not even be implemented in their uncompressed formats).
RISC-V base ISAs (RV32I and RV64I) define 32-bit wide instructions. Those instructions follow the standard RISC instruction set architecture pattern: they fall into a limited set of possible encodings with opcode, register indices and immediate fields always being located in the same bit fields, making it easier to decode. RISC-V instructions can have up to 2 source registers and 1 destination register, with the possibility to use the destination as an extra source (e.g. fmadd). Each register is encoded on a 5-bit index: there are a total of 32 general purpose registers which can be used or written by an instruction (you can find more information on RISC-V register file in this post). This means that up to 15 bits of the instruction word may be dedicated to source/destination encoding, leaving the other 17 bits for immediate and function encodings.
Most programs will often rely heavily on a small subset of instructions which will have a very large static and dynamic frequency compared to all other instructions: static frequency counts the occurrences in the program binary (e.g. from an objdump) while dynamic frequency counts the occurrences during the execution (e.g. from an execution trace). Code structure (loop, functions, ...) and input values affect differently dynamic and static frequencies.
The fact that instructions appear with different frequencies can be exploited by ISA design to reduce code size by encoding the most used instructions on less bits than a general instruction. This can be beneficial for both low power and high performance targets: less memory required to store instructions, more instructions fit in caches, fetching N bytes provides more uops ... with the cost of a more complex decoding and with the caveat that instruction addresses are not aligned to a 32-bit boundary in memory.
Early on RISC-V was extended with compressed instruction extensions. And now multiple standard extensions offer compressed instructions (with encoding less than the initial 32 bits and/or fusing operations from multiple standard instructions). We have divided the survey of RISC-V compressed instructions in two posts: this post reviews some aspects of the first compressed extension, the C extension, while the second one will review the newer Zc* extensions.
To allow code size improvements RISC-V base ISA was rapidly complemented with the Compressed extension, a.k.a the C extension. This extension is defined in Chapter 16 of the unprivileged specification (see pdf).
The C extension defines 9 new instruction formats, each of them fitting on 16-bit. It is not a standalone extensions but is built on top of RV32I or RV64I. It defines a little over 40 new instructions, with some of them only accessible for larger XLEN (RV64 or RV128) or limited to smaller XLEN (RV32FC) and some of them requiring the floating-point extension (F) or the double precision extension (D). Each instruction in the C-extension has an uncompressed counterpart (and can be expanded to this counterpart by the micro-architecture to simplify instruction decoding and execution).
In an assembly program the compressed instructions are easily distinguishable: they start with the prefix "c.", e.g. c.addi is the compressed add with immediate with a non-zero 6-bit immediate. We will review some of the compressed instruction formats and how they compare with their expanded counterparts: arithmetic operations with immediate, arithmetic operations with registers, load/store with focus on load from the stack.
The following diagram illustrates the differences between the encoding of addi in the base ISA (top, 32-bit wide) and in the C extension (bottom, 16-bit wide): the immediate has been reduced from 12 to 6 bits, the same register index is used for both source and destination and the instruction operation is encoded on 5 bits rather than 10. One thing to note is that the register index is still 5-bit wide, all the registers can be addressed in this opcode. For c.addi as for other instances of the CI-type (e.g. compressed load word from stack pointer c.lwsp) the destination must be non-zero (else encoding is reserved). For c.addi, the sign-extended immediate must also be non-zero.
Obviously c.addi is less expressive than addi: smaller immediate range, source and destination have to be the same register. This affects all compressed instructions: they constitutes a subset of their expanded counterparts, hopefully the most useful one.
Another compressed format is the CA-type (compressed arithmetic) illustrated by the following diagram:
In the CA-type, one of the source register index (rs1) is fused with destination (rd), and the register indices are now 3-bit wide. Furthermore they do not encode directly a register index, the indirection is presented in the following table (copied from the RVC standard): for example 0b100 encodes x12 in compressed instructions working on general purpose / integer registers. This is why the register indices are labelled with rd', rs1', and rs2' (rather than rd, rs1, and rs2). The encoded registers were selected because they are the most frequently used ones (the specification notes that the RISC-V ABI was even changed to ensure the 8 registers mapped in RVC were among the most used). The following table (copied from RISC-V unprivileged specification chapter 16 section 16.2 on the C extension instruction formats, page 100) provides the mapping between the 8 possible values of encoded register indices in the CA-type and the actual general purpose or floating-point registers, alongside the ABI register names.
The second type of compressed load/store uses a register-base address: 2 registers are encoded the base address rs1' and the load destination rd' (respectively store source rs2'). Each of those 2 registers is encoded on 3 bits and uses the reduced RVC Register Number encoding listed previously.
Both types of compressed loads and stores uses specific immediate encoding where bit order might look a bit scrambled. According to the specification, the immediate split into various bit fields and their ordering was chosen to ensure that most immediate bits could be found in the same location across various families of instructions.
The following diagram illustrates the difference between the encoding of the lw rd, offset(x2) (I-type, top part) and the c.lwsp rd, offset(sp) (CI type, bottom part). In the compressed encoding, the 5-bit offset is used to encode a 4-byte offset and is implicitly multiplied by 4 before being added to the stack pointer sp/x2. The 5th bit is stored in opcode position 12 as other instructions of type CI and the lower 5-bit immediate field stores the offset [4:2] has its most 3 significant bits and offset [7:6] has its 2 least significant bits. rd cannot be equal to 0 in the compressed instruction encoding.
Let us now consider an illustration of the impact of the C extension on a toy program: a vector add function.
Using godbolt.org compiler explorer (a wonderful tool) we can compile a RV64 program assuming no support for C extension, or assuming it is supported: compiler explorer result.
The example program is simply:
int vector_add(float* r, float* a, float* b, unsigned n) { unsigned i = 0; for (;i < n; ++i) r[i] = a [i] + b[i]; return n; }
The first assembly was compiled by clang 15.0.0 for RV64 with options: "-O2 -march=rv64imfd".
(it also contains a 2-instruction main function)
vector_add: beqz a3,6b4 <vector_add+0x30> slli a4,a3,0x20 srli a4,a4,0x20 flw ft0,0(a1) flw ft1,0(a2) fadd.s ft0,ft0,ft1 fsw ft0,0(a0) addi a1,a1,4 addi a2,a2,4 addi a4,a4,-1 addi a0,a0,4 bnez a4,690 <vector_add+0xc> mv a0,a3 ret
The second assembly was compiled with the same compiler and options: "-O2 -march=rv64imfdc" (the only difference is the addition of the C extension).
vector_add: beqz a3,6a6 <vector_add+0x22> slli a4,a3,0x20 srli a4,a4,0x20 flw ft0,0(a1) flw ft1,0(a2) fadd.s ft0,ft0,ft1 fsw ft0,0(a0) addi a1,a1,4 addi a2,a2,4 addi a4,a4,-1 addi a0,a0,4 bnez a4,68c <vector_add+0x8> mv a0,a3 ret
The two assembly results can look very similar (and they mostly are) but one can notice a few differences. Let's start with the similarities: the instructions mnemonic and the registers used are identical: there are as many instructions in both listings (14) and they use the same registers (same sources and destinations).
On the front of differences: the immediate offset differs (on beqz and bnez): this is because the instruction encoding lengths differ. Let's look at the second compilation result annotated with the program addresses (left, in hexadecimal) and the opcodes (in grey, above the instruction):
One may ask why the compressed version of flw (c.flw) has not been used by the compiler; the reason is simple: this instruction only exists for RV32FC, not for RV64FC.
Another fact is that the 2-byte code alignment requirement for branch target can be witnessed for the target of the first branch beqz which, when taken, branches to address 0x6a6, a 2-byte aligned, but not 4-byte aligned, address.
The C extension provides a reduced encoding for a subset of the base instructions. The subset was selected to cover the most used instructions in their most used form. The reduced encoding allows for code size improvements which benefit both low power and high performance implementations.
For a more in depth overview of the C -extension we invite you to read Andrew Waterman's master thesis report which contains many insights on the impact of compressed instructions on RISC-V program (link to pdf).
The RISC-V Vector Extension (RVV) defines and exploits the concept of vector register groups. This post clarifies what a register group is, how it is useful and the caveat associated with it.
A single vector instruction can operate on operands each consisting of more than one register and can produce similarly extended results.
A group is a bundle of multiple registers. RVV 1.0 supports groups of 1, 2, 4 or 8 registers. The indices of registers in a group span a contiguous range and the lowest index must be a multiple of the group size. For example the vector registers [4, 5, 6, 7] form a valid group for size 4, but [5, 6] is not a valid group for size 2. The lowest index is used to specify the vector register group in an instruction opcode: e.g. when LMUL=4, v4 stands for v4v5v6v7. Thus the actual sizes of the operands (and destination) of an instruction depends on the context (the vtype configuration as we will see later): vadd.vv v4, v0, v12 may operate on 1, 2 or 4 wide register groups depending on the context (8 is not a legal size as neither v4 nor v12 are aligned on an index multiple of 8).
The size of a vector register group is called the vector length multiplier (a.k.a LMUL) in RVV specification: https://github.com/riscv/riscv-v-spec/blob/master/v-spec.adoc#342-vector-register-grouping-vlmul20.
For an operation, LMUL is set by the vlmul field in vtype (so it is usually modified through vset instructions). The following RISC-V assembly sequence first defines LMUL as 8 and then executes a double precision vector addition.
vsetvli x1, zero, e64, m8, ta, mu
vfadd.vv v16, v8, v24Since the LMUL value is 8, the actual sequence of operation is equivalent to the following (assuming each instruction operates on a single-register wide group):
vfadd.vv v16, v8, v24 vfadd.vv v17, v9, v25 vfadd.vv v18, v10, v26 vfadd.vv v19, v11, v27 vfadd.vv v20, v12, v28 vfadd.vv v21, v13, v29 vfadd.vv v22, v14, v30 vfadd.vv v23, v15, v31
In our 5 and a half blog series, RVV in a Nutshell, we presented the basics of RISC-V Vector extension (RVV 1.0), but even after this overview some aspects of this large extension can still seem difficult to apprehend. In this sub-series, we will review in more details some of those aspects. For example, it is difficult to interpret a RVV assembly instruction. Let's review the main component of such instructions and study various examples.
The mnemonic often describes the destination type: for example vadd is a vector add with a single-width destination, while vwadd is a widening vector add and vmadc is a vector addition with carry returning a mask of output carries.
The second component is the operand type(s). It describes on what type of operand(s) the operation is performed. The most common is .vv (vector-vector) which often means that the instruction admits at least two inputs and both are single-width vectors.
The list of various possibilities includes:
The conversions constitutes a category of its own for the operand types, because the mnemonic suffix describes: the destination format, the source format, and the type of operand. For example vfcvt.x.f.v is a vector (.v) conversion from floating-point element (.f) to signed integer (.x) result elements. .xu is used to indicate unsigned integers, .rtz is used to indicate a static round-towards-zero rounding mode.
There can be one or two sources: vs2 and vs1 for vector-vector instructions. If the operations admits a scalar operand, or an immediate operand then vs1 is replaced by rs1 (respectively imm), e.g. vfadd.vf v4, v2, ft3. Vector loads have a memory source, e.g. vloxei8.v vd, (rs1), vs2 [, vm] which has a scalar register as address source and a vector register as destination source.
RVV defines 3-operand instruction, e.g. vmacc.vv. For those operations the destination register vd is both a source and a destination: the operation is destructive: one of the source operand is going to be overwritten by the result.
Most RVV operation can be masked: in such case the v0 register is used as a mask to determine which elements are active and whose elements of the result will be copied from the old destination value or filled with a pre-determined pattern. RVV 1.0 only supports true bit as active masks: the element is considered active if the bit at the corresponding index is set to 1 in v0, and inactive if it is 0. This is what is encoded by the last operand of our example: v0.t (for v0 "true"). If this last operand is missing, then the operation is unmasked (all body elements are considered active).
More information can be found in this post of the original series: RVV in a nutshell (part 3): operations with and on masks.
RISC-V does not mandate the support of any floating-point operations as part of the base ISA (RV32I and RV64I) but single precision and double-precision extensions (e.g. RV64F and RV64D) were among the first to be specified. The F extension added 32 floating-point registers, floating-point arithmetic operations, moves and conversions instructions. The D extension (which requires the F extension) extended this set of operation to also support double precision. There also exists a Q extension for quad-precision.
In reality, there are multiple F and D extensions: RV32F (to extend the 32-bit base ISA) and RV64F (for the 64-bit base ISA), similarly there are RV32D and RV64D. RV32D has the particularity of adding 64-bit wide floating-point registers when the associated general purpose registers are only 32-bit wide. This flexibility of RISC-V ISA is reviewed in the post RISC-V Register Files.
Initially, smaller floaitng-point formats, such as half precision, were not supported.
Half precision used to be lagging behind in term of ISA and hardware support. It was only specified has a storage format in the 2008 revision of the IEEE-754 standard (the IEEE standard specifying floating-point formats and operations). The momentum of deep learning and convolutional neural networks has kick-started a renewed interest for small number formats and in particular half precision (among many others).
Note: In the IEEE-754 2019 revision, half precision is still not defined has a basic format but as an interchange format. Although the standard is more permissive in terms of which formats can admit arithmetic operations.
RISC-V International (the association behind the RISC-V specification(s)) recently ratified two extensions specifying sets of instructions for half-precision support: Zfh and Zfhmin. The later being defined as a subset of the former, we will review Zfh first.
The RISC-V consortium defines profiles. These profiles aim at defining a common set of mandatory extensions and a reduced set of optional extensions which can be used by hardware and software providers to build a compatible ecosystem without having to deal with more specialized ISA extensions. Profile descriptions can be found on RISC-V github.
RVA22 is the most recent profile, it is dedicated for 64-bit application processors
Zfhmin is part of the mandatory extensions of the RVA22 profile, while Zfh is an optional extension (which supersede Zfhmin when selected). This means that all application processors targeting compatibility with the RISC-V ecosystem must have a minimal support for half precision, and than extended support is part of the extended profile. Neither of the vector extensions Zvfh nor Zvfhmin are required in the RVA22 profile.