Cross-compiler

To compile your applications to the ARM architecture you're going to need a cross-compiler.

Using the SDK-Installer

Cpasjuste on the GP32X forums has made a nice script to get a build environment and libraries installed on Ubuntu/Debian systems. Download links and a brief explanation can be found in this thread.

Manual Installation

If you want a bit more control, Code-Sourcery hosts a bunch of cross-compiling suites for various platforms on their site, some editions have to be paid for, but they do have "lite" editions without support that are completely free.

Head up onto here. The recommended release at time of writing is 2009q3.

Installers are available for Linux and Windows. Alternatively, download and unpack the tarball, then add the bin directory to your path.

Example: Setting up a cross compiler using Code Sourcery, and doing so inside a VM

Skeezix makes some notes in his blog about setting up CodeSourcery's prebuilt toolchain in a VM very quickly.