Cross-compiler
One of the ways to compile your applications to the ARM architecture is through a cross-compiler.
Using the SDK-Installer
Cpasjuste on the GP32X forums has made a nice script to get a build environment and libraries installed on Ubuntu/Debian systems. Download links and a brief explanation can be found in this thread.
Manual Installation
If you want a bit more control, Code-Sourcery hosts a bunch of cross-compiling suites for various platforms on their site, some editions have to be paid for, but they do have "lite" editions without support that are completely free.
Head up onto here. The recommended release at time of writing is 2009q3.
Installers are available for Linux and Windows. Alternatively, download and unpack the tarball, then add the bin directory to your path.
Example: Setting up a cross compiler using Code Sourcery, and doing so inside a VM
Skeezix makes some notes in his blog about setting up CodeSourcery's prebuilt toolchain in a VM very quickly.