Showing posts with label registers. Show all posts
Showing posts with label registers. Show all posts

RISC-V Vector Register Groups

 The RISC-V Vector Extension (RVV) defines and exploits the concept of vector register groups. This post clarifies what a register group is, how it is useful and the caveat associated with it.

A single vector instruction can operate on operands each consisting of more than one register and can  produce similarly extended results.

 A group is a bundle of multiple registers. RVV 1.0 supports groups of 1, 2, 4 or 8 registers. The indices of registers in a group span a contiguous range and the lowest index must be a multiple of the group size. For example the vector registers [4, 5, 6, 7] form a valid group for size 4, but [5, 6] is not a valid group for size 2. The lowest index is used to specify the vector register group in an instruction opcode: e.g. when LMUL=4, v4 stands for v4v5v6v7. Thus the actual sizes of the operands (and destination) of an instruction depends on the context (the vtype configuration as we will see later): vadd.vv v4, v0, v12 may operate on 1, 2 or 4 wide register groups depending on the context (8 is not a legal size as neither v4 nor v12 are aligned on an index multiple of 8).

The size of a vector register group is called the vector length multiplier (a.k.a LMUL) in RVV specification: https://github.com/riscv/riscv-v-spec/blob/master/v-spec.adoc#342-vector-register-grouping-vlmul20

For an operation, LMUL is set by the vlmul field in vtype (so it is usually modified through vset instructions). The following RISC-V assembly sequence first defines LMUL as 8 and then executes a double precision vector addition.

	vsetvli	x1, zero, e64, m8, ta, mu
	vfadd.vv	v16, v8, v24

Since the LMUL value is 8, the actual sequence of operation is equivalent to the following (assuming each instruction operates on a single-register wide group):

	vfadd.vv	v16, v8,  v24
	vfadd.vv	v17, v9,  v25
	vfadd.vv	v18, v10, v26
	vfadd.vv	v19, v11, v27
	vfadd.vv	v20, v12, v28
	vfadd.vv	v21, v13, v29
	vfadd.vv	v22, v14, v30
	vfadd.vv	v23, v15, v31

LMUL encoding in vtype

In RVV 1.0, vlmul can take 4 values for integral LMUL: 0b000 (LMUL=1), 0b001 (LMUL=2), 0b010 (LMUL=4), 0b011 (LMUL=8), it can also take 3 fractional values: 0b111 (LMUL=1/2), 0b110 (LMUL=1/4), 0b101 (LMUL=1/8), but more on that in a future post.

LMUL vs EMUL

The actual length multiplier used for an operation operand or destination is called EMUL, for effetctive LMUL. EMUL depends on vlmul, but also on the operation. For example EMUL=LMUL for any source or destination of a vadd.vv, but EMUL=2*LMUL for both vd and vs2 of vwadd.wv (because of its widening characteristic). EMUL can never be larger than 8 (it is not possible for a vector register group to exceed 8 registers) so LMUL=8 is reserved for widening instructions: it is not supported in RVV 1.0.

Some operations which embed the effective element width (EEW) in their opcode define EMUL to keep the following ratio equality EEW/EMUL=SEW/LMUL. For example if LMUL=1, SEW=8-bit, then vle32 admits EEW=32-bit and EMUL=EEW*LMUL/SEW=4, this also mean the register group destination will have to be described by a register whose index is a multiple of 4 (EMUL).

Since EMUL can differ between an instructions operands and it results, partial or complete overlap between register groups are a possibility. RVV 1.0 authorizes 3 types of overlap between source and destination register groups. Those overlaps are specified here https://github.com/riscv/riscv-v-spec/blob/v1.0/v-spec.adoc#52-vector-operands and illustrated by the Figure below (e.g. of v0v1v2v3 register group in various source/destination EEW configurations).


Benefits of register grouping

There are several benefits associated to the availability of register groups:
  • even if architectural registers are "only" VLEN-bit wide, single instructions can perform operation on larger operands and produce larger results:
    • for widening operation results or narrowing operation source, having the possibility of building a register groups enables the use of a full register source or enables populating a full register destination
    • providing large input sets: for example for a vrgather the input data set contains VLMAX elements, where VLMAX=VLEN/SEW*LMUL increases with LMUL: one can build larger lookup table by using larger register groups.
    • allowing reuse of operands between extended registers (e.g. immediate or scalar operand)
    • reducing code size: for data parallel section with vector lengths larger than what can fit in a single vector register, less instructions are required to operate on the same amount of data. 
    • reducing number of loop iterations: since VLMAX increases with LMUL, during loop stripmining, less iterations are required when LMUL increases.
  • exposing more opportunity to micro-architecture
    • even if a micro-architecture can not operate directly on register groups larger than VLEN-bit wide, it can still exploit micro-op parallelism and chaining opportunities exposed by larger register groups.

Constraints induced by register grouping

One of the caveat of register grouping is that the number of architectural registers decreases with the length multiplier value. For example there are only 4 8-wide register groups: v0, v8, v16, v24. This means that register allocation pressure can be a limiting factor when using larger register groups: it makes it more difficult to maintain in register all variables simultaneously alive in a sequence and memory spilling may be required.

This pressure can even be augmented by the fact that v0 is used as the mask operand in RVV 1.0, this may also limit the number of register groups: when using masked operations with large register groups, v0 often prevents a program from using the first register group. For example, when LMUL=4, if one wants to use v0 as the mask input, then one cannot use the v0,v1,v2,v3 group. This group can still be partially exploited with LMUL=1 for example to build the v0 mask or partially for LMUL=2 (v2,v3).

The availability of larger inputs may not be well supported by the implementation: for example a LMUL=8 input data set for a vrgather may not be supported with the same latency and/or throughput as a vrgather with LMUL=1, since each result element may originate from a larger pool. Thus even if legal the instruction configuration may not be efficient but the implementation still needs to support it one way or the other.

Conclusion

Vector register groups and the associated length multiplier is an interesting concept which can be leveraged thanks to the vector length agnostic characteristic of RVV where the software build around the vector length parameter can be agnostic of VLEN and thus easily extended to VLEN*LMUL register groups. If you want to learn more on the RISC-V Vector Extension you may consider reading our RVV in a Nutshell blog series starting with part 1.

Resources




RISC-V Register Files

RISC-V ISA defines several register files. There are at least 3 in the main set of extensions: the general purpose register file (XRF) introduced in the base integer extensions, the floating-point register file (FRF) introduced in the floating-point extensions and the vector register file (you guessed it, it was introduced in the vector extension a.k.a. RVV). We will not consider the control and status registers (CSR file) which have their own specificity (it is more common to split system registers from general purpose registers, although commonality could be debated).

Diagram of RISC-V register files and the operations between them


Register files characteristics

Each register file contains 32 architectural registers. The first register of the general purpose register file (x0) is a bit specific since its value is a hardwired constant: 0. The first register of the vector register file (v0) is the only one which can be used as the mask operand in RVV 1.0 (more information on RVV masked operation can be found in RISC-V Vector Extension in a Nutshell: part 3). 

The size of the registers in each file is an architecture parameter: XLEN for the general purpose register file, FLEN for the floating-point register file and VLEN for the vector register file. 

The general purpose register file is sized to fit a virtual address value. For example for the base 32-bit RV32I ISA, the general purpose registers are 32-bit wide, while they are 64-bit wide for RV64I (and 128-bit wide for RV128I, although this architecture is seldom used). For the vectore registers, VLEN must be a power of 2, greater or equal to ELEN (maximum element width supported by the implementation) and must not exceed 65536 (2^16). 

Moving data between register files

The diagram at the beginning of this post illustrates the base characteristics of each register file and the basic move operations between them and from/to memory. For the XRF and FRF, only operations on 32-bit values are drawn. There exists similar operation for double precisions: e.g. in RV64D when XLEN=FLEN=64 bits, which adds fldfsdfmv.d.xfmv.x.d and numerous conversions from integer format to double precision and reverse (e.g. in RV64IFD fcvt.d.wu f3, x2 corresponds to converting from an unsigned 32-bit integer in the bottom 32-bit of the x2 general purpose register to a double precision number stored in the f3 floating-point register).

For the vector register file, the scalar data size and the vector element size are not encoded as part of the opcode but are configured in the vsew field of the vtype configuration register, so there are no need for type specific vector moves. The diagram only represents explicit data moves between FRF/XRF and VRF but most vector instructions admit a vector-scalar variant which reads one of its operand directly from XRF or FRF (e.g. vfmadd.vf splats a scalar floating-point register as the multiplier). For more details on the vector registers and the vector extension in general you can refer to the series: RISC-V Vector extension in a Nutshell published on this blog.

Why multiple register files ?

This discussion focuses on the general purpose and floating-point register files. It is much more understandable to have a different vector register files (vector register tends to be larger than the other types of registers) although some ISAs (e.g. x86 SSE and AVX extensions) reuse small registers as the low parts of larger registers, this is not the case in RISC-V, where general purpose, floating-point and vector registers do not overlap. 

NOTE: The option of overlapping FRF and VRF was considered and dropped during the specification process of RVV. See https://github.com/riscv/riscv-v-spec/blob/v1.0/v-spec.adoc#51-scalar-operands

Having multiple register files has some advantages and some drawbacks. Let's start with a benefit.

The first benefit is that the architecture can expose more architectural registers without extending the size of a register index in the instruction encoding. The type of instruction (.e.g. floating-point addition) is encoded in the opcode, and all (most) floating-point instructions implicitly operate on floating-point registers, thus there is no need to distinguish floating-point and general purpose registers in the opcode encoding of the register itself. The number of available architectural registers impact the register allocation pressure when writing (in assembly) or compiling a program: more registers often means less flexible ABI, less spilling.

In general, RISC-V ISA uses 5 bits to encode the register index (for operands and results), which provides 32 registers, since the type of register is part of the opcode specification rather than the register index, RISC-V architecture can in fact have 96 registers: 32 general purpose registers, 32 floating-point registers and 32 vector registers (you are authorized to say only 95 and exclude the specific x0, although having a hardwired 0-value operand is quite useful and certainly outweighs the benefit of an extra general purpose register in most cases).

This benefit can also be considered a drawback since more registers also means higher hardware cost for implementations and more registers to save in case of context switch. This was solved for RISC-V by introducing the Zfinx/Zdinx/Zhinx extensions which define floating-point operations working on general purpose registers (thus limiting the cost of floating-point support for the more constrained implementations: high performance out-of-order implementation with very large physical register files are generally less concerned by limiting the size of the architectural register files, although it can impact mapping tables, context sizes ...).

Another benefit of separate register files is that register can be encoded to optimize the data they store: for example the RISC-V open-source processor rocket-chip uses hardfloat's specific encoding of floating-point numbers (see Recoded Format section here) to makes floating-point operations more efficient (simplifying the detection of special values and reducing the encoding difference between normal and subnormal numbers). The use of a format specific encoding is facilitated by the fact that data moves are explicit (you need to execute a fmv.f.x to move the content of a general purpose register to a floating-point register before performing any operation on it): the recoding of value can be performed during the explicit data move (including from/to memory) and the recoding can exploit the fact that the value type is determined by the operation acting on it. Such recoding can only leave within an internal register and values must be converted to canonical formats when moved to another register file or to memory.

Once again this can also be considered a drawback since you have to explicitly move data from one register file to the next (which may have a non-zero latency and uses encoding space) and that you need to define separate memory operations for each register files: loading a 32-bit single precision number is not the same operation as loading a 32-bit integer value, since the destination registers differ. This was surveyed by the diagram and the section Moving data between register files. This drawback is more easily alleviated by wide out-of-order implementations which can extract more ILP and cover the cost of those explicit moves (although this cost still impacts latency dominated chains of instructions). It is also removed in the Z(f/d/h)inx extensions.

Another advantage of multiple register files is that you can tune the file's architecture characteristics to the domain you want to support. RISC-V does not expose a configurable number of registers but you XLEN, FLEN and VLEN are defined separately (the first two depending on which extensions are enabled): you can have a 32-bit ISA with 64-bit double precision registers: the architecture does not have to extend its integer registers to 64 bits, while still keeping 64 registers with adapted sizes: general purpose/integer instructions can be efficient and low power while the core only activates the FRF for workloads which require the extra activity.

Finally an implementation advantage, which was pointed out by a colleague (Alex S.): having several register files reduces the number of read / write ports per register file while being able to serve as many execution pipelines more efficiently. High performance cores often have a lot of ports on each register files (more than ten read ports may not be uncommon) that serve a lot of different execution pipelines in parallel. With the same total number of execution pipelines multiplying the number of architectural register file helps to split physical register files accordingly, decreasing the number of port per file. This is a good benefit as the complexity of a register file increases rapidly with the number of ports (in particular read ports). Even for low performance implementations, limiting the number of ports per file provide more efficient register files.

Conclusion

In this post we reviewed the main register files specified by RISC-V ISA, their basic characteristics and how they interact together. We listed some of the reasons for this choice alongside some of the drawbacks of specializing register files.

Initially published Oct 17th 2022, updated Oct 19th 2022.

Thanks

Thank you to Alex S. for pointing out a key advantage (decreasing the number of ports per register file) I missed in the first version of this blog post.

References: