Extend Utils
Contents
Overview
Extend Utils provide a variety of tools to manipulate Extend, Overlay and Swap files. These are generally loop-back files stored on SD card for various purposes which include, but is not limited to:
- OS Extends - running another distribution on top of Angstrom in a chroot environment.
- Dev Extends - a method of providing dev tool support without installing directly to NAND.
- File system Overlays - redirect writes from NAND to a file on an SD card.
- Swap - use a swap file on SD card to provide additional memory for the Pandora when needed.
This page used to only document OS Extends... it will now document all facets of the Extend Utils.
Extend Utils have been written by Stuckie and while very much work-in-progress, the scripts are very useful in day to day tasks.
Available Files
The Extend Utils suite is updated regularly, and will always be available here, as part of the Simple Dev system: Extend Utils
For those that prefer the "classic" version which doesn't have as much options available to it, you can grab that here: Classic Extend Utils
For support, please direct questions to the relevant threads on the forum - see Links at the bottom of this page.
Bug Reports
In order for a bug report to be useful, I need the following information from you:
Output from "dmesg | tail", "mount" and your preMount script.
A description of the error.
Just PM me them on the GP32X boards, or post in the relevant threads - Links at the bottom of this page.
Empty Extends
The following files are pre-formatted ext2 files. These are included in New Extend Utils, but are available here as well as they're just loop files and can therefore be used with other things:
Dev Extends
The last Dev Extend is very very old and may not work on recent firmwares.
A newer one is in progress as of 24th Feb 2011.
Old Releases
The current release of the Dev Extend is available here:
The original release is here:
Upgrade functionality is being worked on.
OS Extends
Arch Linux
Debian
New "Current" Release
Alternatively, check the Debian On SD project for booting Debian directly from SD card.
Gentoo
None, yet.
Ubuntu
Ubuntu is now officially 100% not supported. Upstart is an arse.
Extend Utils Help
Extend Utils ( version 1.0a ) now facilitates the creation of "PreMount" scripts. For users of previous Extend Utils script sets, these separate scripts are essentially a PreMount script - the Ubuntu specific variant being a prime example.
This allows us to create customised mounting procedures - such as being able to mount multiple Extends and Overlays at once; for example, a compressed Base OS Extend with perhaps an Overlay on top to "catch" installed programs.
Unfortunately, as is always the case with more customization, the complexity has shot up. This guide will help you to create a PreMount script given four common scenarios: a Swap mount, a Home or Root Overlay, an OS Extend and a Dev Extend.
Creating a Swap PreMount
Perhaps the simplest PreMount script is a Swap Mount. For this, I shall assume you have not created a .swap file.
- Start up Extend Utils.
- Select New Extend/Overlay/Swap.
- Select New Swap File.
- Select 256MB - this uses one of the pre-created zipped Extend files, whereas Custom will create a new one on the card.
- Save the file anywhere you like.
- It will extract the appropriate zip to where you selected.
- Back at the Main Menu, select New Pre-Mount.
- Select New Swap Mount Script.
- Find your Swap File you just created.
- Select a location for your Pre-Mount script to be saved to.
- Finally, back at the Main Menu, you can select Mount Existing Pre-Mount.
- Find where your Pre-Mount script was saved to, and that's it.. you have swap mounted.
Obviously, now that the script is saved, you only need to repeat the last two steps, or just run the script direct from the Terminal.
Creating a Root or Home Overlay PreMount
Again, I shall assume that you do not have any pre-made .overlay files.
- Start up Extend Utils.
- Select New Overlay.
- Select New Overlay File.
- Select 1GB - or whatever you like.. again be aware that Custom may very well chew through your SD card as it DD's direct to it!
- Save the file anywhere you like.
- It will extract the appropriate zip to where you selected.
- Back at the Main Menu, select New Pre-Mount.
- Select New Overlay Mount Script.
- Select either Standard Home Overlay or Standard Root Overlay.
- Find your Overlay that you created.
- Select where you want to save the Pre-Mount script.
- Finally, back at the Main Menu, you can select Mount Existing Pre-Mount.
- Find where your Pre-Mount script was saved to, and that's it.. either the majority of / or your /home is bound to the overlay.
Creating an OS Extend PreMount
Generally, these will have been pre-made downloads, or you'll have created a chroot environment and wrapped it in an extend already. It's also important to note that there are essentially TWO types of OS Extends - compressed and uncompressed. Compressed OS Extends should be affixed with compressed; base-debian-sid-compressed.extend for example, which donates it's a base install of debian sid, in a compressed extend. Compressed extends can only be mounted as read-only, and will likely fail if you try to mount them read-write. This means they require an Overlay in order to be much use. As such, I will describe a "classic" uncompressed OS Extend procedure, and a "modern" compressed OS Extend procedure.
Compressed OS Extends
- Start Extend Utils.
- Select New Pre-Mount.
- Select New Extend Mount.
- Select OS Extend.
This is where you need to start paying attention as to what you're doing. For our purposes, we want a basic configuration for say a Base Debian Sid Extend, and one Overlay. The Base Debian Sid Extend we would have downloaded already, and the Overlay we should have created as well.
- Move the slider to select 1 Read-Only Extend.
- Select our base-debian-sid-compressed.extend ( assuming you've downloaded it, and it's been released! )
- We don't want any Read-Write Extends, so Cancel or select 0.
- We do want an Overlay, so select 1 Read-Write Overlay.
- Choose the Overlay you want to use.
- Now you must name the mount point. This can be anything you like, but it must be unique and not the name of the Extend or Overlay you've chosen.
- You'll then be asked where you want to save the PreMount script.
Again, we need to pay attention as to what we want here. For our purposes, we just want a Terminal, so that we can call apt-get and install whatever we feel like, and run it from the shell to interact with the program in Angstrom. You can, of course, re-do this procedure later after having installed a window manager and desktop manager ( IE: something like Fluxbox and GDM ) so that you can jump fully into Debian if you so wish.
- Select the Terminal option.
- You'll now be sent back to the Main Menu, where you can now select Mount Existing Pre-Mount.
- Select your Pre-Mount Script you've just created.
- After the gksudo checks, you should have a Terminal open inside your OS Extend, where all writes go to your Overlay.
Uncompressed/Classic OS Extends
These ones are much easier to deal with, and are generally what have been released up till now. These don't require an Overlay ( though you can still use one if you like. ) The procedure for setting up a mount script for one of these runs as follows:
- Start Extend Utils.
- Select New Pre-Mount.
- Select New Extend Mount.
- Select OS Extend.
As said, we don't need an Overlay, and Uncompressed/Classic Extends can be mounted Read-Write without any issues. Therefore, we only need to select 1 Read-Write Extend, and 0 everything else.
- We don't want any Read-Only Extends, so Cancel or select 0.
- We don want a Read-Write Extend...so move the slider to 1 and find your Extend file.
- Again, we don't want an Overlay, so select 0 or Cancel it.
- Now you must name the mount point. This can be anything you like, but it must be unique and not the name of the Extend.
- You'll then be asked where you want to save the PreMount script.
Now we need to know roughly what's in our Extend.. to be safe, we can just choose a Terminal as that's guaranteed to work. If, however, you know that GDM is installed and setup ( I'll ensure that any of my Extends that are, are documented in saying so ) then you can choose the GDM option.
- We'll assume we've got GDM present, so select GDM.
- You'll now be sent back to the Main Menu, where you can now select Mount Existing Pre-Mount.
- Select your Pre-Mount Script you've just created.
- After the gksudo checks, GDM will start up alongside Angstrom and ask you to login. You should have a login/password already - either from you manually setting it up, or the default that whoever set the .Extend up performed.
Creating a Dev Extend Pre-Mount
Dev Extends are essentially Root Extends these days.. however that's not to say there won't eventually be packs so that there's a base Dev Extend with just the bare essentials, then addons for GTK, QT, wxWidgets, etc.. so having them classed as an Extend seems like the best option.
The current Dev Extends are created exactly like the Uncompressed/Classic OS Extends, but for completeness sakes:
- Start Extend Utils.
- Select New Pre-Mount.
- Select New Extend Mount.
- Select Dev Extend.
- Select 0 Read-Only Extends.
- Select 1 Read-Write Extend.
- Find your Dev Extend.
- Select 0 Read-Write Overlays.
- Come up with a mount point.
- Save your Pre-Mount somewhere.
- You may now load it up through the Mount Existing Pre-Mount script if you so wish.
OS Extends
OS Extend is a method of running another distribution on top of Angstrom. This is currently very much a hacky WORK IN PROGRESS method of doing so, and this page describes what it does, how to get started, and any major pitfalls in using it.
Currently, I've been successful in getting Debian, Ubuntu and Arch running to varying degrees. This page will be split up into these distributions so you should be able to jump to the correct section for any oddities that I've uncovered, and any workarounds presented.
Essentially, what I'm doing is using a loop back file, formatted as Ext2, and performing a first stage install within it for Arm. This is called by either debootstrap, rootstock, or whichever system the target distribution uses. These "Extend" files are then copied over to SD card/Memory Stick/etc... and chrooted into.
However, this brings some very important caveats:
- System Services don't always tend to work, and starting them can clobber Angstrom's services ( see the HAL/DBus issues in Ubuntu below. )
- Limited resources when running TWO XServers and TWO Desktop Managers does start to show, but the fact that it's still running fairly well is a testament to how powerful the Pandora actually is.
- Logging out of XFCE will speed up the chrooted environment, but tends to shut down NetworkManager with it, disabling WiFi for the environment unless it knows how to handle it.
Hopefully, these can be dealt with in due time.
That said, there are benefits of using a system such as this:
- It's (almost) the exact same distribution as you'd be installing on SD Card, so why not test it out first before sacrificing an SD card fully?
- Perhaps you only have the one SD card? It may be a gigantic SD card but partitioning scares you, so this allows you to try a chosen distribution first without messing about!
- Some distributions have a HUGE library of packages ready and waiting to be played with.. why not see which ones you'd like ported over to Angstrom natively?
- We can be sneaky and take advantage of all Pandora specific updates to Angstrom, and have our Extended OS install reap the benefits rather than having to wait for someone to port it over ( kernel drivers, for example. )
- Why not show off just how powerful the Pandora actually is - you are running TWO X Servers and TWO Desktop Managers at once!
- Development between multiple distributions can be done much more easily, as you could bind your development distribution, and check driver support in Angstrom.
- And my absolute favourite - it's a fun hack!
So, there is good reason to try and pursue a system such as this, if perhaps not generally geared to "End User Consumption."
Arch OS Extend
A proof of concept is available ( see Files above ) Not much work has been taking in furthering the install as of this time.
Android OS Extend
This might be tricky due to dalvik.. doesn't seem to be any fast way of "borrowing" an existing rootfs, will have to just try compiling manually for the Pandora.
Links
Debian OS Extend
Debian was the first distribution to work in this environment.
Various proof of concepts exist, and an "official" compressed version of Sid with Synaptic. See Files above.
See Debian On SD for a better solution ( Debian booting from SD ) for the moment.
Ubuntu OS Extend
Ubuntu caused a great deal of stress and hacking about to get it to work in any shape or form - this is mostly due to Upstart.
Eventually, only Jaunty was able to be loadable much.
With the recent progres with Debian On SD I think that Ubuntu is a lost cause, so I'm removing all Ubuntu files from my server as they were utterly hacky to begin with.
Dev Extend
A Dev Extend is a special kind of Overlay which only binds enough of the file system to allow development tools to work. As such, most information on Overlays - particularly Root Overlays - match Dev Extends.
Dev Extends are routinely updated and match Stuckie's current development environment. Support for updating between revisions is planned, but not currently implemented.
Unfortunately, a Dev Extend hasn't been released for a while, and may currently BREAK some applications on the Pandora with the latest hot fix releases.. use with caution, or create an Overlay and install dev tools into that ( which is effectively all a Dev Extend is. )
Overlays
Overlays allow the base system to be kept unmodified, and all changes to be propagated to the Overlay file. This is immensely useful for development work where you can install a development environment "normally" and compile what is needed, then unbind the overlay and run it on a "vanilla" install, to see if any libraries are missing.
Stuckie's general development environment uses a Dev Extend and an Overlay - so that all compiled code sits in the Home Overlay, and the development environment is in the Dev Extend. This allows any programs that have not been configured fully to keep clear of the NAND when running, should they create config files in the user's Home directory, as well as separating the compiled programs from the development environment for easier testing on the "vanilla" state.
Overlays essentially come in two flavours - Home and Root - where the latter covers everything written to / and the former only covering /home, however an Overlay could be made for any combination of directories.
Extend Utils GUI
On the forums, SomeGuy99 has suggested the creation of a GUI-based app to wrap around the Extend Utils suite. This would greatly simplify what is going on by being able to provide on-screen help, as well as tying the many scripts into one app and not cluttering up the System menu any further.
This also needs to be a cross-platform application, if only for the creation of Extend files. As such, it needs a cross-platform widget toolkit; preferably a light-weight one to run on Pandora as well. Currently, this looks to be FLTK
For the time being, the Extend Utils has been rewritten ( version 1.0a ) to further use Zenity and create customised mount scripts, in preparation for the work on the FLTK-based GUI.
