Difference between revisions of "Extend Utils"

From Pandora Wiki
Jump to: navigation, search
(formating)
(Step-by-step guide)
 
(16 intermediate revisions by 3 users not shown)
Line 1: Line 1:
 
= Overview =
 
= Overview =
  
{{split sections}}
 
 
[[Extend Utils]] provide a variety of tools to manipulate Extend, Overlay and Swap files. These are generally loop-back files stored on SD card for various purposes which include, but is not limited to:
 
[[Extend Utils]] provide a variety of tools to manipulate Extend, Overlay and Swap files. These are generally loop-back files stored on SD card for various purposes which include, but is not limited to:
 
* OS Extends - running another distribution on top of Angstrom in a chroot environment.
 
* OS Extends - running another distribution on top of Angstrom in a chroot environment.
Line 11: Line 10:
  
 
Extend Utils have been written by Stuckie and while very much work-in-progress, the scripts are very useful in day to day tasks.
 
Extend Utils have been written by Stuckie and while very much work-in-progress, the scripts are very useful in day to day tasks.
 
= Available Files =
 
 
The Extend Utils suite is updated regularly, and will always be available here, as part of the [[SimpleDev]] system:
 
[http://www.stuckiegamez.co.uk/apps/pandora/SimpleDev/extendutils.pnd Extend Utils]
 
 
For those that prefer the "classic" version which doesn't have as much options available to it, you can grab that here:
 
[http://www.stuckiegamez.co.uk/apps/pandora/SimpleDev/OldExtendUtils.pnd Classic Extend Utils]
 
 
For support, please direct questions to the relevant threads on the forum - see Links at the bottom of this page.
 
  
 
= Bug Reports =
 
= Bug Reports =
Line 28: Line 17:
  
 
Just PM me them on the GP32X boards, or post in the relevant threads - Links at the bottom of this page.
 
Just PM me them on the GP32X boards, or post in the relevant threads - Links at the bottom of this page.
 +
 +
= Available Files =
 +
 +
The current ExtendUtils will always be on my website here:
 +
[http://www.stuckiegamez.co.uk/apps/pandora/ExtendUtils/ExtendUtils.pnd Extend Utils]
 +
 +
For support, please direct questions to this forum thread: [http://www.gp32x.com/board/index.php?/topic/59740-extend-utils/ Extend Utils on GP32X]<br />
 +
Older threads where all this came from are at the bottom of this page for the interested.
  
 
== Empty Extends ==
 
== Empty Extends ==
The following files are pre-formatted ext2 files. These are included in New Extend Utils, but are available here as well as they're just loop files and can therefore be used with other things:
+
The following files are pre-formatted ext2 files. These are included in Extend Utils, but are available here as well as they're just loop files and can therefore be used with other things:
* [http://www.stuckiegamez.co.uk/apps/pandora/SimpleDev/256MBExtend.zip 256MB Extend]
+
* [http://www.stuckiegamez.co.uk/apps/pandora/ExtendUtils/256MBExtend.zip 256MB Extend]
* [http://www.stuckiegamez.co.uk/apps/pandora/SimpleDev/512MBExtend.zip 512MB Extend]
+
* [http://www.stuckiegamez.co.uk/apps/pandora/ExtendUtils/512MBExtend.zip 512MB Extend]
* [http://www.stuckiegamez.co.uk/apps/pandora/SimpleDev/1GBExtend.zip 1GB Extend]
+
* [http://www.stuckiegamez.co.uk/apps/pandora/ExtendUtils/1GBExtend.zip 1GB Extend]
* [http://www.stuckiegamez.co.uk/apps/pandora/SimpleDev/2GBExtend.zip 2GB Extend]
+
* [http://www.stuckiegamez.co.uk/apps/pandora/ExtendUtils/2GBExtend.zip 2GB Extend]
* [http://www.stuckiegamez.co.uk/apps/pandora/SimpleDev/4GBExtend.zip 4GB Extend]
+
* [http://www.stuckiegamez.co.uk/apps/pandora/ExtendUtils/4GBExtend.zip 4GB Extend]
  
 
== Dev Extends ==
 
== Dev Extends ==
The last Dev Extend is very very old and may not work on recent firmwares.<br />
+
All Angstrom-based Dev Extends are currently deprecated as of 26th June 2011.
A newer one is in progress as of 24th Feb 2011.
+
 
 +
Please see the Lenny Debian OS Extend instead.
 +
 
 +
== OS Extends ==
 +
=== Debian ===
 +
Lenny Dev Extend
 +
* [http://www.stuckiegamez.co.uk/apps/pandora/ExtendUtils/debian.lennydev.20110626.zip 512mb Lenny Extend]
 +
* [http://stuckiegamez.co.uk/apps/pandora/ExtendUtils/debian.lennydev.20110629.1gb.zip 1GB Lenny Extend]
  
=== Old Releases ===
+
The Lenny Dev Extend has a Debian bootstrap with dev tools ( enough for the Wakebreaker GL ES example plus gtk 1.2 dev )<br />
The current release of the Dev Extend is available here:
+
About 400mb is taken, so choose the 1GB extend if you think you'll need lots of additional libraries.
* [http://www.stuckiegamez.co.uk/apps/pandora/SimpleDev/Dev.Extend.20thJune2010.zip Dev Extend - 20th June 2010]
 
  
The original release is here:
+
Alternatively, check the [[Debian On SD]] project for booting Debian directly from SD card.
* [http://www.stuckiegamez.co.uk/apps/pandora/SimpleDev/Dev.Extend.18thJune2010.zip Dev Extend - 18th June 2010]
 
  
Upgrade functionality is being worked on.
+
====Step-by-step guide====
 +
In this guide, you will mount a Lenny Dev Extend (to give you a protected dev environment), in conjunction with a 1gb overlay home extend (to give you room in which to place your files).
  
== OS Extends ==
+
=====Part 1: the 1GB home extend=====
=== Arch Linux ===
+
#Download [http://stuckiegamez.co.uk/apps/pandora/ExtendUtils/ExtendUtils.pnd ExtendUtils.pnd] to your Pandora (see [[User_manual#Installing_a_PND_file_.28an_application.29|here]] for instructions on where to place PNDs)
* [http://www.stuckiegamez.co.uk/apps/pandora/SimpleDev/archmobile.1GbExtend.zip 1Gb Arch Extend]
+
#Run it. It's called "Extend Utils" and is in "System" category.
 +
#Select "New Extend/Overlay/Swap"
 +
#Select "New Overlay File"
 +
#Select a size (for example, 1gb) .. if you select "Custom", prepare for a long wait.
 +
#Place the file somewhere on your SD card (the default folder it suggests is a good choice, because it always looks there first) and give it a name. For example, "1gb"... you don't need to add .overlay or .extend, as it does it for you.
 +
#It'll now either extract a premade empty file, or if you selected "Custom" it'll DD a new file, taking forever, and probably raping your SD in the process.. you have been warned ;)
 +
#You'll be pushed back to the main menu now.
 +
#Select "New Pre-Mount"
 +
#Select "New Overlay Mount Script"
 +
#Select "Standard Home Overlay"
 +
#Choose your file ("1gb.overlay" in our example) that you saved on your SD card in the previous stage
 +
#Save the mount script anywhere you like, calling it anything you like... for our purposes, save it in the same folder as before and name it "1gb"
 +
#Back to the main menu.
 +
#Select "Mount Existing Pre-Mount". Note: If your SD card is formatted to anything other than ext/2/3/4 (for example, if it uses FAT32 or NTFS), you can't run it directly from the Terminal. You'll need to run it through Extend Utils or type <code>/bin/bash /my/mount/script.preMount</code> instead.
 +
#Choose the script you saved in the previous part ("1gb.premount")
 +
#Type your sudo password (your password to log in to your Pandora at startup)
 +
#That should be it.
  
=== Debian ===
+
'''For checking purposes:'''
New "Current" Release
 
* [http://www.stuckiegamez.co.uk/apps/pandora/SimpleDev/debian-sid-synaptic-compressed.7z ~250mb Compressed Debian Sid 28th July 2010 with Synaptic]
 
  
Alternatively, check the [[Debian On SD]] project for booting Debian directly from SD card.
+
#Launch a Terminal (start menu -> Terminal)
 +
#Type <code>mount</code>
 +
#Look for a line at the bottom reading something like "aufs on /home type aufs (rw, si=xxxxxxxx)"
 +
#Type <code>cd /tmp/NAMEOFOVERLAY.overlay/YOURUSERNAME</code> (for our example, if we were stuckie: <code>cd /tmp/1gb.overlay/stuckie</code>)
 +
#We'll put a random file here to ensure it works. Type in <code>touch testfile</code>. This will create an empty file named "testfile". Type <code>ls</code> and it should be among the files listed. Now type <code>ls ~</code>, and it should also show up.
  
=== Gentoo ===
+
=====Part 2: Debian Dev Extend=====
None, yet.
+
#Download [http://stuckiegamez.co.uk/apps/pandora/ExtendUtils/DebianDevExtend.pnd DebianDevExtend.pnd] to your Pandora.
 +
#Download the [http://www.stuckiegamez.co.uk/apps/pandora/ExtendUtils/debian.lennydev.20110626.zip 512mb] or the [http://stuckiegamez.co.uk/apps/pandora/ExtendUtils/debian.lennydev.20110629.1gb.zip 1GB] Lenny Dev Extend, unzip it (for example, using Squeeze), place the file ("debian.lennydev.20110626.extend" for 512mb, "debian.lennydev.20110629.1gb.extend" for 1GB) into the directory <code>/media/yourSDcard/pandora/appdata/DebianDevExtend</code> (it doesn't matter where you put it, but this is easier because it's the default directory it will look in)
 +
#Launch Debian Dev Extend. It's in the "System" category.
 +
#Click "Ok" past the notifications.
 +
#Select the file (named "debian.lennydev.20110626.extend" or "debian.lennydev.20110629.1gb.extend").
 +
#Type your sudo password if prompted (you may be quick enough to still be in the sudo cache that it doesn't ask). Once you're done, a Debian Dev Extend terminal will open.
 +
#In the Debian Dev Extend terminal, type <code>gcc</code> to ensure that it works. If it says <code>gcc: no input files</code>, all is well.
 +
#Type <code>cd</code> to jump directly to your home folder. Type <code>ls</code> to ensure our test file is there. If it is, you're good to go.
 +
#Your new password while you're in the Debian Dev Extend terminal is <code>pandora</code>, so remember that if you're planning to do any <code>sudo</code> commands! For example, to add new libraries you would type in <code>sudo apt-get install nameofpackage</code>. You can search which packages are available at [http://www.debian.org/distrib/packages this page]
  
=== Ubuntu ===
+
=====How it all links together=====
Ubuntu is now officially 100% not supported. Upstart is an arse.
+
You can open a file manager (for example Thunar, or Xfe) and verify that everything is as it should be. Here's how it should look:
 +
*The contents of <code>/home</code> and <code>/tmp/1gb.overlay</code> should be almost identical
 +
*The contents of the following three directories should be identical:
 +
**<code>/home/username</code>
 +
**<code>/tmp/debextend/home/username</code>
 +
**<code>/tmp/1gb.overlay/username</code>
 +
("username" represents your username)
  
 
= Extend Utils Help =
 
= Extend Utils Help =
Line 75: Line 110:
 
== Creating a Swap PreMount ==
 
== Creating a Swap PreMount ==
 
Perhaps the simplest PreMount script is a Swap Mount. For this, I shall assume you have not created a .swap file.
 
Perhaps the simplest PreMount script is a Swap Mount. For this, I shall assume you have not created a .swap file.
* Start up Extend Utils.
+
# Start up Extend Utils.
* Select New Extend/Overlay/Swap.
+
# Select New Extend/Overlay/Swap.
* Select New Swap File.
+
# Select New Swap File.
* Select 256MB - this uses one of the pre-created zipped Extend files, whereas Custom will create a new one on the card.
+
# Select 256MB - this uses one of the pre-created zipped Extend files, whereas Custom will create a new one on the card.
* Save the file anywhere you like.
+
# Save the file anywhere you like.
* It will extract the appropriate zip to where you selected.
+
# It will extract the appropriate zip to where you selected.
* Back at the Main Menu, select New Pre-Mount.
+
# Back at the Main Menu, select New Pre-Mount.
* Select New Swap Mount Script.
+
# Select New Swap Mount Script.
* Find your Swap File you just created.
+
# Find your Swap File you just created.
* Select a location for your Pre-Mount script to be saved to.
+
# Select a location for your Pre-Mount script to be saved to.
* Finally, back at the Main Menu, you can select Mount Existing Pre-Mount.
+
# Finally, back at the Main Menu, you can select Mount Existing Pre-Mount.
* Find where your Pre-Mount script was saved to, and that's it.. you have swap mounted.
+
# Find where your Pre-Mount script was saved to, and that's it.. you have swap mounted.
  
 
Obviously, now that the script is saved, you only need to repeat the last two steps, or just run the script direct from the Terminal.
 
Obviously, now that the script is saved, you only need to repeat the last two steps, or just run the script direct from the Terminal.
Line 92: Line 127:
 
== Creating a Root or Home Overlay PreMount ==
 
== Creating a Root or Home Overlay PreMount ==
 
Again, I shall assume that you do not have any pre-made .overlay files.
 
Again, I shall assume that you do not have any pre-made .overlay files.
* Start up Extend Utils.
+
# Start up Extend Utils.
* Select New Overlay.
+
# Select New Overlay.
* Select New Overlay File.
+
# Select New Overlay File.
* Select 1GB - or whatever you like.. again be aware that Custom may very well chew through your SD card as it DD's direct to it!
+
# Select 1GB - or whatever you like.. again be aware that Custom may very well chew through your SD card as it DD's direct to it!
* Save the file anywhere you like.
+
# Save the file anywhere you like.
* It will extract the appropriate zip to where you selected.
+
# It will extract the appropriate zip to where you selected.
* Back at the Main Menu, select New Pre-Mount.
+
# Back at the Main Menu, select New Pre-Mount.
* Select New Overlay Mount Script.
+
# Select New Overlay Mount Script.
* Select either Standard Home Overlay or Standard Root Overlay.
+
# Select either Standard Home Overlay or Standard Root Overlay.
* Find your Overlay that you created.
+
# Find your Overlay that you created.
* Select where you want to save the Pre-Mount script.
+
# Select where you want to save the Pre-Mount script.
* Finally, back at the Main Menu, you can select Mount Existing Pre-Mount.
+
# Finally, back at the Main Menu, you can select Mount Existing Pre-Mount.
* Find where your Pre-Mount script was saved to, and that's it.. either the majority of / or your /home is bound to the overlay.
+
# Find where your Pre-Mount script was saved to, and that's it.. either the majority of / or your /home is bound to the overlay.
  
 
== Creating an OS Extend PreMount ==
 
== Creating an OS Extend PreMount ==
Line 111: Line 146:
  
 
=== Compressed OS Extends ===
 
=== Compressed OS Extends ===
* Start Extend Utils.
+
# Start Extend Utils.
* Select New Pre-Mount.
+
# Select New Pre-Mount.
* Select New Extend Mount.
+
# Select New Extend Mount.
* Select OS Extend.
+
# Select OS Extend.<br/>This is where you need to start paying attention as to what you're doing.<br/>For our purposes, we want a basic configuration for say a Base Debian Sid Extend, and one Overlay.<br/>The Base Debian Sid Extend we would have downloaded already, and the Overlay we should have created as well.
 
+
# Move the slider to select 1 Read-Only Extend.
This is where you need to start paying attention as to what you're doing.
+
# Select our base-debian-sid-compressed.extend ( assuming you've downloaded it, and it's been released! )
For our purposes, we want a basic configuration for say a Base Debian Sid Extend, and one Overlay.
+
# We don't want any Read-Write Extends, so Cancel or select 0.
The Base Debian Sid Extend we would have downloaded already, and the Overlay we should have created as well.
+
# We do want an Overlay, so select 1 Read-Write Overlay.
 
+
# Choose the Overlay you want to use.
* Move the slider to select 1 Read-Only Extend.
+
# Now you must name the mount point. This can be anything you like, but it must be unique and not the name of the Extend or Overlay you've chosen.
* Select our base-debian-sid-compressed.extend ( assuming you've downloaded it, and it's been released! )
+
# You'll then be asked where you want to save the PreMount script.<br/>Again, we need to pay attention as to what we want here.<br/>For our purposes, we just want a Terminal, so that we can call apt-get and install whatever we feel like, and run it from the shell to interact with the program in Angstrom.<br/>You can, of course, re-do this procedure later after having installed a window manager and desktop manager ( IE: something like Fluxbox and GDM ) so that you can jump fully into Debian if you so wish.
* We don't want any Read-Write Extends, so Cancel or select 0.
+
# Select the Terminal option.
* We do want an Overlay, so select 1 Read-Write Overlay.
+
# You'll now be sent back to the Main Menu, where you can now select Mount Existing Pre-Mount.
* Choose the Overlay you want to use.
+
# Select your Pre-Mount Script you've just created.
* Now you must name the mount point. This can be anything you like, but it must be unique and not the name of the Extend or Overlay you've chosen.
+
# After the gksudo checks, you should have a Terminal open inside your OS Extend, where all writes go to your Overlay.
* You'll then be asked where you want to save the PreMount script.
 
 
Again, we need to pay attention as to what we want here.
 
For our purposes, we just want a Terminal, so that we can call apt-get and install whatever we feel like, and run it from the shell to interact with the program in Angstrom.
 
You can, of course, re-do this procedure later after having installed a window manager and desktop manager ( IE: something like Fluxbox and GDM ) so that you can jump fully into Debian if you so wish.
 
 
 
* Select the Terminal option.
 
* You'll now be sent back to the Main Menu, where you can now select Mount Existing Pre-Mount.
 
* Select your Pre-Mount Script you've just created.
 
* After the gksudo checks, you should have a Terminal open inside your OS Extend, where all writes go to your Overlay.
 
  
 
=== Uncompressed/Classic OS Extends ===
 
=== Uncompressed/Classic OS Extends ===
Line 141: Line 166:
 
These don't require an Overlay ( though you can still use one if you like. )
 
These don't require an Overlay ( though you can still use one if you like. )
 
The procedure for setting up a mount script for one of these runs as follows:
 
The procedure for setting up a mount script for one of these runs as follows:
* Start Extend Utils.
+
# Start Extend Utils.
* Select New Pre-Mount.
+
# Select New Pre-Mount.
* Select New Extend Mount.
+
# Select New Extend Mount.
* Select OS Extend.
+
# Select OS Extend. <br/>As said, we don't need an Overlay, and Uncompressed/Classic Extends can be mounted Read-Write without any issues.<br/>Therefore, we only need to select 1 Read-Write Extend, and 0 everything else.
 
+
# We don't want any Read-Only Extends, so Cancel or select 0.
As said, we don't need an Overlay, and Uncompressed/Classic Extends can be mounted Read-Write without any issues.
+
# We do want a Read-Write Extend...so move the slider to 1 and find your Extend file.
Therefore, we only need to select 1 Read-Write Extend, and 0 everything else.
+
# Again, we don't want an Overlay, so select 0 or Cancel it.
 
+
# Now you must name the mount point. This can be anything you like, but it must be unique and not the name of the Extend.
* We don't want any Read-Only Extends, so Cancel or select 0.
+
# You'll then be asked where you want to save the PreMount script. <br/>Now we need to know roughly what's in our Extend.. to be safe, we can just choose a Terminal as that's guaranteed to work. If, however, you know that GDM is installed and setup ( I'll ensure that any of my Extends that are, are documented in saying so ) then you can choose the GDM option.
* We don want a Read-Write Extend...so move the slider to 1 and find your Extend file.
+
# We'll assume we've got GDM present, so select GDM.
* Again, we don't want an Overlay, so select 0 or Cancel it.
+
# You'll now be sent back to the Main Menu, where you can now select Mount Existing Pre-Mount.
* Now you must name the mount point. This can be anything you like, but it must be unique and not the name of the Extend.
+
# Select your Pre-Mount Script you've just created.
* You'll then be asked where you want to save the PreMount script.
+
# After the gksudo checks, GDM will start up alongside Angstrom and ask you to login. You should have a login/password already - either from you manually setting it up, or the default that whoever set the .Extend up performed.
 
 
Now we need to know roughly what's in our Extend.. to be safe, we can just choose a Terminal as that's guaranteed to work. If, however, you know that GDM is installed and setup ( I'll ensure that any of my Extends that are, are documented in saying so ) then you can choose the GDM option.
 
 
 
* We'll assume we've got GDM present, so select GDM.
 
* You'll now be sent back to the Main Menu, where you can now select Mount Existing Pre-Mount.
 
* Select your Pre-Mount Script you've just created.
 
* After the gksudo checks, GDM will start up alongside Angstrom and ask you to login. You should have a login/password already - either from you manually setting it up, or the default that whoever set the .Extend up performed.
 
  
 
== Creating a Dev Extend Pre-Mount ==
 
== Creating a Dev Extend Pre-Mount ==
Line 166: Line 184:
  
 
The current Dev Extends are created exactly like the Uncompressed/Classic OS Extends, but for completeness sakes:
 
The current Dev Extends are created exactly like the Uncompressed/Classic OS Extends, but for completeness sakes:
* Start Extend Utils.
+
# Start Extend Utils.
* Select New Pre-Mount.
+
# Select New Pre-Mount.
* Select New Extend Mount.
+
# Select New Extend Mount.
* Select Dev Extend.
+
# Select Dev Extend.
* Select 0 Read-Only Extends.
+
# Select 0 Read-Only Extends.
* Select 1 Read-Write Extend.
+
# Select 1 Read-Write Extend.
* Find your Dev Extend.
+
# Find your Dev Extend.
* Select 0 Read-Write Overlays.
+
# Select 0 Read-Write Overlays.
* Come up with a mount point.
+
# Come up with a mount point.
* Save your Pre-Mount somewhere.
+
# Save your Pre-Mount somewhere.
* You may now load it up through the Mount Existing Pre-Mount script if you so wish.
+
# You may now load it up through the Mount Existing Pre-Mount script if you so wish.
 
 
= OS Extends =
 
OS Extend is a method of running another distribution on top of Angstrom. This is currently very much a hacky WORK IN PROGRESS method of doing so, and this page describes what it does, how to get started, and any major pitfalls in using it.
 
 
 
Currently, I've been successful in getting Debian, Ubuntu and Arch running to varying degrees. This page will be split up into these distributions so you should be able to jump to the correct section for any oddities that I've uncovered, and any workarounds presented.
 
 
 
Essentially, what I'm doing is using a loop back file, formatted as Ext2, and performing a first stage install within it for Arm. This is called by either debootstrap, rootstock, or whichever system the target distribution uses. These "Extend" files are then copied over to SD card/Memory Stick/etc... and chrooted into.
 
 
 
However, this brings some very important caveats:
 
* System Services don't always tend to work, and starting them can clobber Angstrom's services ( see the HAL/DBus issues in Ubuntu below. )
 
* Limited resources when running TWO XServers and TWO Desktop Managers does start to show, but the fact that it's still running fairly well is a testament to how powerful the Pandora actually is.
 
* Logging out of XFCE will speed up the chrooted environment, but tends to shut down NetworkManager with it, disabling WiFi for the environment unless it knows how to handle it.
 
 
 
Hopefully, these can be dealt with in due time.
 
 
 
That said, there are benefits of using a system such as this:
 
* It's (almost) the exact same distribution as you'd be installing on SD Card, so why not test it out first before sacrificing an SD card fully?
 
* Perhaps you only have the one SD card? It may be a gigantic SD card but partitioning scares you, so this allows you to try a chosen distribution first without messing about!
 
* Some distributions have a HUGE library of packages ready and waiting to be played with.. why not see which ones you'd like ported over to Angstrom natively?
 
* We can be sneaky and take advantage of all Pandora specific updates to Angstrom, and have our Extended OS install reap the benefits rather than having to wait for someone to port it over ( kernel drivers, for example. )
 
* Why not show off just how powerful the Pandora actually is - you are running TWO X Servers and TWO Desktop Managers at once!
 
* Development between multiple distributions can be done much more easily, as you could bind your development distribution, and check driver support in Angstrom.
 
* And my absolute favourite - it's a fun hack!
 
 
 
So, there is good reason to try and pursue a system such as this, if perhaps not generally geared to "End User Consumption."
 
 
 
== Arch OS Extend ==
 
A proof of concept is available ( see Files above )
 
Not much work has been taking in furthering the install as of this time.
 
 
 
== Android OS Extend ==
 
This might be tricky due to dalvik.. doesn't seem to be any fast way of "borrowing" an existing rootfs, will have to just try compiling manually for the Pandora.
 
 
 
=== Links ===
 
* http://elinux.org/Android_on_OMAP
 
* http://omappedia.org/wiki/Android_Getting_Started
 
 
 
== Debian OS Extend ==
 
Debian was the first distribution to work in this environment.
 
 
 
Various proof of concepts exist, and an "official" compressed version of Sid with Synaptic.
 
See Files above.
 
 
 
See [[Debian On SD]] for a better solution ( Debian booting from SD ) for the moment.
 
 
 
== Ubuntu OS Extend ==
 
Ubuntu caused a great deal of stress and hacking about to get it to work in any shape or form - this is mostly due to Upstart.<br />
 
Eventually, only Jaunty was able to be loadable much.<br />
 
With the recent progres with [[Debian On SD]] I think that Ubuntu is a lost cause, so I'm removing all Ubuntu files from my server as they were utterly hacky to begin with.
 
 
 
= Dev Extend =
 
A Dev Extend is a special kind of Overlay which only binds enough of the file system to allow development tools to work. As such, most information on Overlays - particularly Root Overlays - match Dev Extends.
 
 
 
Dev Extends are routinely updated and match Stuckie's current development environment.
 
Support for updating between revisions is planned, but not currently implemented.
 
 
 
Unfortunately, a Dev Extend hasn't been released for a while, and may currently BREAK some applications on the Pandora with the latest hot fix releases.. use with caution, or create an Overlay and install dev tools into that ( which is effectively all a Dev Extend is. )
 
 
 
= Overlays =
 
Overlays allow the base system to be kept unmodified, and all changes to be propagated to the Overlay file. This is immensely useful for development work where you can install a development environment "normally" and compile what is needed, then unbind the overlay and run it on a "vanilla" install, to see if any libraries are missing.
 
 
 
Stuckie's general development environment uses a Dev Extend and an Overlay - so that all compiled code sits in the Home Overlay, and the development environment is in the Dev Extend. This allows any programs that have not been configured fully to keep clear of the NAND when running, should they create config files in the user's Home directory, as well as separating the compiled programs from the development environment for easier testing on the "vanilla" state.
 
 
 
Overlays essentially come in two flavours - Home and Root - where the latter covers everything written to / and the former only covering /home, however an Overlay could be made for any combination of directories.
 
 
 
= Extend Utils GUI =
 
On the forums, SomeGuy99 has suggested the creation of a GUI-based app to wrap around the Extend Utils suite. This would greatly simplify what is going on by being able to provide on-screen help, as well as tying the many scripts into one app and not cluttering up the System menu any further.
 
 
 
This also needs to be a cross-platform application, if only for the creation of Extend files. As such, it needs a cross-platform widget toolkit; preferably a light-weight one to run on Pandora as well. Currently, this looks to be [http://www.fltk.org/ FLTK]
 
 
 
For the time being, the Extend Utils has been rewritten ( version 1.0a ) to further use Zenity and create customised mount scripts, in preparation for the work on the FLTK-based GUI.
 
  
 
= Links =
 
= Links =
Line 254: Line 201:
 
* [http://www.gp32x.com/board/index.php?/topic/54064-advanced-filesystem-hackery/ Original Overlay Thread]
 
* [http://www.gp32x.com/board/index.php?/topic/54064-advanced-filesystem-hackery/ Original Overlay Thread]
  
 
+
[[Category:Documentation]]
 
[[Category:Filesystem]]
 
[[Category:Filesystem]]
[[Category:operating system]]
+
[[Category:Operating System]]
 
[[Category:Tutorials]]
 
[[Category:Tutorials]]

Latest revision as of 20:20, 3 July 2011

Overview

Extend Utils provide a variety of tools to manipulate Extend, Overlay and Swap files. These are generally loop-back files stored on SD card for various purposes which include, but is not limited to:

  • OS Extends - running another distribution on top of Angstrom in a chroot environment.
  • Dev Extends - a method of providing dev tool support without installing directly to NAND.
  • File system Overlays - redirect writes from NAND to a file on an SD card.
  • Swap - use a swap file on SD card to provide additional memory for the Pandora when needed.

This page used to only document OS Extends... it will now document all facets of the Extend Utils.

Extend Utils have been written by Stuckie and while very much work-in-progress, the scripts are very useful in day to day tasks.

Bug Reports

In order for a bug report to be useful, I need the following information from you:
Output from "dmesg | tail", "mount" and your preMount script.
A description of the error.

Just PM me them on the GP32X boards, or post in the relevant threads - Links at the bottom of this page.

Available Files

The current ExtendUtils will always be on my website here: Extend Utils

For support, please direct questions to this forum thread: Extend Utils on GP32X
Older threads where all this came from are at the bottom of this page for the interested.

Empty Extends

The following files are pre-formatted ext2 files. These are included in Extend Utils, but are available here as well as they're just loop files and can therefore be used with other things:

Dev Extends

All Angstrom-based Dev Extends are currently deprecated as of 26th June 2011.

Please see the Lenny Debian OS Extend instead.

OS Extends

Debian

Lenny Dev Extend

The Lenny Dev Extend has a Debian bootstrap with dev tools ( enough for the Wakebreaker GL ES example plus gtk 1.2 dev )
About 400mb is taken, so choose the 1GB extend if you think you'll need lots of additional libraries.

Alternatively, check the Debian On SD project for booting Debian directly from SD card.

Step-by-step guide

In this guide, you will mount a Lenny Dev Extend (to give you a protected dev environment), in conjunction with a 1gb overlay home extend (to give you room in which to place your files).

Part 1: the 1GB home extend
  1. Download ExtendUtils.pnd to your Pandora (see here for instructions on where to place PNDs)
  2. Run it. It's called "Extend Utils" and is in "System" category.
  3. Select "New Extend/Overlay/Swap"
  4. Select "New Overlay File"
  5. Select a size (for example, 1gb) .. if you select "Custom", prepare for a long wait.
  6. Place the file somewhere on your SD card (the default folder it suggests is a good choice, because it always looks there first) and give it a name. For example, "1gb"... you don't need to add .overlay or .extend, as it does it for you.
  7. It'll now either extract a premade empty file, or if you selected "Custom" it'll DD a new file, taking forever, and probably raping your SD in the process.. you have been warned ;)
  8. You'll be pushed back to the main menu now.
  9. Select "New Pre-Mount"
  10. Select "New Overlay Mount Script"
  11. Select "Standard Home Overlay"
  12. Choose your file ("1gb.overlay" in our example) that you saved on your SD card in the previous stage
  13. Save the mount script anywhere you like, calling it anything you like... for our purposes, save it in the same folder as before and name it "1gb"
  14. Back to the main menu.
  15. Select "Mount Existing Pre-Mount". Note: If your SD card is formatted to anything other than ext/2/3/4 (for example, if it uses FAT32 or NTFS), you can't run it directly from the Terminal. You'll need to run it through Extend Utils or type /bin/bash /my/mount/script.preMount instead.
  16. Choose the script you saved in the previous part ("1gb.premount")
  17. Type your sudo password (your password to log in to your Pandora at startup)
  18. That should be it.

For checking purposes:

  1. Launch a Terminal (start menu -> Terminal)
  2. Type mount
  3. Look for a line at the bottom reading something like "aufs on /home type aufs (rw, si=xxxxxxxx)"
  4. Type cd /tmp/NAMEOFOVERLAY.overlay/YOURUSERNAME (for our example, if we were stuckie: cd /tmp/1gb.overlay/stuckie)
  5. We'll put a random file here to ensure it works. Type in touch testfile. This will create an empty file named "testfile". Type ls and it should be among the files listed. Now type ls ~, and it should also show up.
Part 2: Debian Dev Extend
  1. Download DebianDevExtend.pnd to your Pandora.
  2. Download the 512mb or the 1GB Lenny Dev Extend, unzip it (for example, using Squeeze), place the file ("debian.lennydev.20110626.extend" for 512mb, "debian.lennydev.20110629.1gb.extend" for 1GB) into the directory /media/yourSDcard/pandora/appdata/DebianDevExtend (it doesn't matter where you put it, but this is easier because it's the default directory it will look in)
  3. Launch Debian Dev Extend. It's in the "System" category.
  4. Click "Ok" past the notifications.
  5. Select the file (named "debian.lennydev.20110626.extend" or "debian.lennydev.20110629.1gb.extend").
  6. Type your sudo password if prompted (you may be quick enough to still be in the sudo cache that it doesn't ask). Once you're done, a Debian Dev Extend terminal will open.
  7. In the Debian Dev Extend terminal, type gcc to ensure that it works. If it says gcc: no input files, all is well.
  8. Type cd to jump directly to your home folder. Type ls to ensure our test file is there. If it is, you're good to go.
  9. Your new password while you're in the Debian Dev Extend terminal is pandora, so remember that if you're planning to do any sudo commands! For example, to add new libraries you would type in sudo apt-get install nameofpackage. You can search which packages are available at this page
How it all links together

You can open a file manager (for example Thunar, or Xfe) and verify that everything is as it should be. Here's how it should look:

  • The contents of /home and /tmp/1gb.overlay should be almost identical
  • The contents of the following three directories should be identical:
    • /home/username
    • /tmp/debextend/home/username
    • /tmp/1gb.overlay/username

("username" represents your username)

Extend Utils Help

Extend Utils ( version 1.0a ) now facilitates the creation of "PreMount" scripts. For users of previous Extend Utils script sets, these separate scripts are essentially a PreMount script - the Ubuntu specific variant being a prime example.

This allows us to create customised mounting procedures - such as being able to mount multiple Extends and Overlays at once; for example, a compressed Base OS Extend with perhaps an Overlay on top to "catch" installed programs.

Unfortunately, as is always the case with more customization, the complexity has shot up. This guide will help you to create a PreMount script given four common scenarios: a Swap mount, a Home or Root Overlay, an OS Extend and a Dev Extend.

Creating a Swap PreMount

Perhaps the simplest PreMount script is a Swap Mount. For this, I shall assume you have not created a .swap file.

  1. Start up Extend Utils.
  2. Select New Extend/Overlay/Swap.
  3. Select New Swap File.
  4. Select 256MB - this uses one of the pre-created zipped Extend files, whereas Custom will create a new one on the card.
  5. Save the file anywhere you like.
  6. It will extract the appropriate zip to where you selected.
  7. Back at the Main Menu, select New Pre-Mount.
  8. Select New Swap Mount Script.
  9. Find your Swap File you just created.
  10. Select a location for your Pre-Mount script to be saved to.
  11. Finally, back at the Main Menu, you can select Mount Existing Pre-Mount.
  12. Find where your Pre-Mount script was saved to, and that's it.. you have swap mounted.

Obviously, now that the script is saved, you only need to repeat the last two steps, or just run the script direct from the Terminal.

Creating a Root or Home Overlay PreMount

Again, I shall assume that you do not have any pre-made .overlay files.

  1. Start up Extend Utils.
  2. Select New Overlay.
  3. Select New Overlay File.
  4. Select 1GB - or whatever you like.. again be aware that Custom may very well chew through your SD card as it DD's direct to it!
  5. Save the file anywhere you like.
  6. It will extract the appropriate zip to where you selected.
  7. Back at the Main Menu, select New Pre-Mount.
  8. Select New Overlay Mount Script.
  9. Select either Standard Home Overlay or Standard Root Overlay.
  10. Find your Overlay that you created.
  11. Select where you want to save the Pre-Mount script.
  12. Finally, back at the Main Menu, you can select Mount Existing Pre-Mount.
  13. Find where your Pre-Mount script was saved to, and that's it.. either the majority of / or your /home is bound to the overlay.

Creating an OS Extend PreMount

Generally, these will have been pre-made downloads, or you'll have created a chroot environment and wrapped it in an extend already. It's also important to note that there are essentially TWO types of OS Extends - compressed and uncompressed. Compressed OS Extends should be affixed with compressed; base-debian-sid-compressed.extend for example, which donates it's a base install of debian sid, in a compressed extend. Compressed extends can only be mounted as read-only, and will likely fail if you try to mount them read-write. This means they require an Overlay in order to be much use. As such, I will describe a "classic" uncompressed OS Extend procedure, and a "modern" compressed OS Extend procedure.

Compressed OS Extends

  1. Start Extend Utils.
  2. Select New Pre-Mount.
  3. Select New Extend Mount.
  4. Select OS Extend.
    This is where you need to start paying attention as to what you're doing.
    For our purposes, we want a basic configuration for say a Base Debian Sid Extend, and one Overlay.
    The Base Debian Sid Extend we would have downloaded already, and the Overlay we should have created as well.
  5. Move the slider to select 1 Read-Only Extend.
  6. Select our base-debian-sid-compressed.extend ( assuming you've downloaded it, and it's been released! )
  7. We don't want any Read-Write Extends, so Cancel or select 0.
  8. We do want an Overlay, so select 1 Read-Write Overlay.
  9. Choose the Overlay you want to use.
  10. Now you must name the mount point. This can be anything you like, but it must be unique and not the name of the Extend or Overlay you've chosen.
  11. You'll then be asked where you want to save the PreMount script.
    Again, we need to pay attention as to what we want here.
    For our purposes, we just want a Terminal, so that we can call apt-get and install whatever we feel like, and run it from the shell to interact with the program in Angstrom.
    You can, of course, re-do this procedure later after having installed a window manager and desktop manager ( IE: something like Fluxbox and GDM ) so that you can jump fully into Debian if you so wish.
  12. Select the Terminal option.
  13. You'll now be sent back to the Main Menu, where you can now select Mount Existing Pre-Mount.
  14. Select your Pre-Mount Script you've just created.
  15. After the gksudo checks, you should have a Terminal open inside your OS Extend, where all writes go to your Overlay.

Uncompressed/Classic OS Extends

These ones are much easier to deal with, and are generally what have been released up till now. These don't require an Overlay ( though you can still use one if you like. ) The procedure for setting up a mount script for one of these runs as follows:

  1. Start Extend Utils.
  2. Select New Pre-Mount.
  3. Select New Extend Mount.
  4. Select OS Extend.
    As said, we don't need an Overlay, and Uncompressed/Classic Extends can be mounted Read-Write without any issues.
    Therefore, we only need to select 1 Read-Write Extend, and 0 everything else.
  5. We don't want any Read-Only Extends, so Cancel or select 0.
  6. We do want a Read-Write Extend...so move the slider to 1 and find your Extend file.
  7. Again, we don't want an Overlay, so select 0 or Cancel it.
  8. Now you must name the mount point. This can be anything you like, but it must be unique and not the name of the Extend.
  9. You'll then be asked where you want to save the PreMount script.
    Now we need to know roughly what's in our Extend.. to be safe, we can just choose a Terminal as that's guaranteed to work. If, however, you know that GDM is installed and setup ( I'll ensure that any of my Extends that are, are documented in saying so ) then you can choose the GDM option.
  10. We'll assume we've got GDM present, so select GDM.
  11. You'll now be sent back to the Main Menu, where you can now select Mount Existing Pre-Mount.
  12. Select your Pre-Mount Script you've just created.
  13. After the gksudo checks, GDM will start up alongside Angstrom and ask you to login. You should have a login/password already - either from you manually setting it up, or the default that whoever set the .Extend up performed.

Creating a Dev Extend Pre-Mount

Dev Extends are essentially Root Extends these days.. however that's not to say there won't eventually be packs so that there's a base Dev Extend with just the bare essentials, then addons for GTK, QT, wxWidgets, etc.. so having them classed as an Extend seems like the best option.

The current Dev Extends are created exactly like the Uncompressed/Classic OS Extends, but for completeness sakes:

  1. Start Extend Utils.
  2. Select New Pre-Mount.
  3. Select New Extend Mount.
  4. Select Dev Extend.
  5. Select 0 Read-Only Extends.
  6. Select 1 Read-Write Extend.
  7. Find your Dev Extend.
  8. Select 0 Read-Write Overlays.
  9. Come up with a mount point.
  10. Save your Pre-Mount somewhere.
  11. You may now load it up through the Mount Existing Pre-Mount script if you so wish.

Links