Floating Point Optimization

Introduction

In the past it was rare for an embedded processor to have dedicated floating point hardware, this usually limited you to either using fixed point math (which can be very tricky to write) or very slow software floating point emulation. Fortunately the ARM Cortex A8 found in the OMAP3 has 2 Floating Point Units, a non-pipelined VFP-lite conforming to the IEEE754 standard for floating point arithmetic and a pipelined SIMD NEON coprocessor. The VFP-lite can handle both single and double precession arithmetic, as well as properly handling exceptions and subnormal numbers. However, This full spec compliance and due to the presence of the NEON has resulted in a relatively slow implementation, it usually takes from 18 to 21 cycles to perform a single precision multiply accumulate. The NEON unit on the other hand is designed for very fast single precision vector math, it can sustain multiply accumulates at a rate of two per cycle. Efficiently utilizing these coprocessors in GCC will be the focus of this article.

Note: In this article I refer to the A8's integer pipeline as the "ARM" , the VFP-lite as simply the "VFP" and the NEON unit as the "NFP".

Compiler Support

The NEON + VFP-lite is a new design from ARM and hence does not yet have very mature compiler support. At present the CodeSourcery toolchain has the best support since the mainline GCCs do not support NEON yet. Compiler versions:

• CS2007q3: Working NEON
• CS2008q1: Broken NEON
• CS2009q1: Partially Working NEON.

Generally the CS2007q3 release is recommended. The newest release does have better support for vectorizing integer and floating point computations on the NEON, but this has proven to be buggy.

Inorder to instruct the compiler to produce NEON / VFP code you should use the following compile flags: -mfpu=neon -mfloat-abi=softfp, or -mfpu=vfp -mfloat-abi=softfp. Unfortunately the prebuilt toolchains do not support the passing / returning of values in floating point registers (i talk about this some more in the Transfers section), there is however a patch being tested that supports hardfp and may be included in future toolchains. Additionally, If you want the compiler to attempt to vectorize your integer / floating point code for the NEON you should add: -ftree-vectorize to your flags.

Therefore i recommend the following flags: -O3 -mcpu=cortex-a8 -mfpu=neon -ftree-vectorize -mfloat-abi=softfp -ffast-math -fsingle-precision-constant

VFP-Lite RunFast

Under the correct circumstances some of The VFPs instructions will be executed in the NEON coprocessor. Unfortunately this does not gain the full benefit of the NEON, it still takes 7 cycles for an FMAC / FMUL / FADD. Due to this quirk you will likely get better scalar performance by accessing the NEON directly via Intrinsics or ASM.

Inorder for VFP instruction to execute in the NFP the following constraints must be met:

• RunFast mode must be enabled
• Must be single precision floating point operands
• Must not be a vector instruction (GCC doesn't appear to use this feature, so don't worry about it)

Runfast mode is enabled when the following conditions are present:

• subnormal numbers are being flushed to zero
• default NaN mode is active
• no floating point exceptions are enabled

At the present time, it is unsure to me whether Runfast mode will be enabled by default in the Angstrom distribution. If it is not you can use the following C code to enforce it:

void enable_runfast()
{
	static const unsigned int x = 0x04086060;
	static const unsigned int y = 0x03000000;
	int r;
	asm volatile (
		"fmrx	%0, fpscr			\n\t"	//r0 = FPSCR
		"and	%0, %0, %1			\n\t"	//r0 = r0 & 0x04086060
		"orr	%0, %0, %2			\n\t"	//r0 = r0 | 0x03000000
		"fmxr	fpscr, %0			\n\t"	//FPSCR = r0
		: "=r"(r)
		: "r"(x), "r"(y)
	);
}

The instructions that are executed on the NFP are: FADDS, FSUBS, FABSS, FNEGS, FMULS, FNMULS, FMACS, FNMACS, FMSCS, FNMSCS, FCMPS, FCMPES, FCMPZS, FCMPEZS, FUITOS, FSITOS, FTOUIS, FTOSIS, FTOUIZS, FTOSIZS, FSHTOS, FSLTOS, FUHTOS, FULTOS, FTOSHS, FTOSLS, FTOUHS, FTOULS.

Single Precision Constants

One important and easy optimization is to make sure that single precision constants are being used. By default this is not the case, instead a double precision constant is being used, so all related operations involving that constant require double precision instructions and cannot be executed on the NEON. eg

float foo(float x)
{ 
	return (2.123 * x); 
}

might end up the same as:

float foo(float x)
{
	double dx = (double) x;
	double dy = (double) 2.123; 
	double dr = dx * dy;
	float r = (float) dr;
	return r;
}

You can enforce single precision constants by including the compiler flag: -fsingle-precision-constant, alternatively you can append an 'f' to the end of each constant. ie 2.123f

NFP / VFP to ARM Transfers

Probably the biggest bottleneck in the architecture is that inorder to transfer a number from the VFP / NFP registers onto the ARM you must stall both the ARM and NFP / VFP for >20 cycles. This is particularly troublesome because this is how GCC supplies arguments and recieves returns from functions. Possibly The best way to minimize operand passing stalls is to make the floating point functions inline.

Another source of NFP / VFP - ARM transfers are conditional branches that depend on floating point numbers. You can do the condition on the VFP but inorder to branch the flags must be sent from the VFP to the ARM. For very simple branches your best bet is to not branch at all and instead use arithmetic. ie

if (x < 0) {x += 1.1244;}

Is the same as:

x = x + (x < 0) * 1.1244

However you might want to keep a close eye on what the compiler actually produces with the above code.

One interesting fact is that using stores and loads do not cause a stall. So aslong as you don't need the result straight away you can hide the 20 cycle latency. Instead of doing a transfer you; store your NFP / VFP result to memory, do some work on the ARM, then load the result back onto the ARM without penalty. ie

void foo(float *x, float *r)
{
	*r = 123 + *x;
}

void bar(float *x, float *r)
{
	*r = 546 + *x;
}

void main()
{
	float x = 10;
	float y, z;
	foo(&x, &y)
	
	//do ~20 cycles of ARM work
	
	bar(&y, &z);

}

NEON SIMD

The NEON unit is similar to the MMX and SSE extensions found on X86 processors, it is optimized for Single Instruction Multiple Data (SIMD) operations. The NEON unit has 2 floating point pipelines, an integer pipeline and a 128bit load/store/permute pipeline. When properly utilized it is a very powerful coprocessor. Unfortunately GCC does a rather poor job of vectorizing code for the NEON unit. To get the best performance you should use either the intrinsics provided in the "arm_neon.h" header or hand written assembly.

Summary

Therefore, Inorder to achieve the best floating point performance on the Pandora:

• Use the CodeSourcery 2007q3 release and these flags: -O3 -mcpu=cortex-a8 -mfpu=neon -ftree-vectorize -mfloat-abi=softfp -ffast-math -fsingle-precision-constant
• Only use single precision floating point
• Enable RunFast mode
• Use NEON intrinsics / ASM for vector, or even scalar, code.
• Inline floating point code (unless its very large)
• Minimize Conditional Branches
• Pass Arguments via pointers instead of by value and do integer work in between function calls.