Difference between revisions of "Extend Utils"

From Pandora Wiki
Jump to: navigation, search
(Debian OS Extend: Added regression info on xhost hack)
(Step-by-step guide)
 
(28 intermediate revisions by 4 users not shown)
Line 1: Line 1:
 
= Overview =
 
= Overview =
OS Extend is a method of running another distribution on top of Angstrom. This is currently very much a hacky WORK IN PROGRESS method of doing so, and this page describes what it does, how to get started, and any major pitfalls in using it.
 
  
Currently, I've been successful in getting Debian, Ubuntu and Arch running to varying degrees. This page will be split up into these distributions so you should be able to jump to the correct section for any oddities that I've uncovered, and any workarounds presented.
+
[[Extend Utils]] provide a variety of tools to manipulate Extend, Overlay and Swap files. These are generally loop-back files stored on SD card for various purposes which include, but is not limited to:
 +
* OS Extends - running another distribution on top of Angstrom in a chroot environment.
 +
* Dev Extends - a method of providing dev tool support without installing directly to NAND.
 +
* File system Overlays - redirect writes from NAND to a file on an SD card.
 +
* Swap - use a swap file on SD card to provide additional memory for the Pandora when needed.
  
Essentially, what I'm doing is using a loop back file, formatted as Ext2, and performing a first stage install within it for Arm. This is called by either debootstrap, rootstock, or whichever system the target distribution uses. These "Extend" files are then copied over to SD card/Memory Stick/etc... and chrooted into.
+
This page used to only document OS Extends... it will now document all facets of the Extend Utils.
  
However, this brings some very important caveats:
+
Extend Utils have been written by Stuckie and while very much work-in-progress, the scripts are very useful in day to day tasks.
* System Services don't always tend to work, and starting them can clobber Angstrom's services ( see the HAL/DBus issues in Ubuntu below. )
 
* Limited resources when running TWO XServers and TWO Desktop Managers does start to show, but the fact that it's still running fairly well is a testament to how powerful the Pandora actually is.
 
* Logging out of XFCE will speed up the chrooted environment, but tends to shut down NetworkManager with it, disabling WiFi for the environment unless it knows how to handle it.
 
  
Hopefully, these can be dealt with in due time.
+
= Bug Reports =
 +
In order for a bug report to be useful, I need the following information from you:<br />
 +
Output from "dmesg | tail", "mount" and your preMount script.<br />
 +
A description of the error.
  
That said, there are benefits of using a system such as this:
+
Just PM me them on the GP32X boards, or post in the relevant threads - Links at the bottom of this page.
* It's (almost) the exact same distribution as you'd be installing on SD Card, so why not test it out first before sacrificing an SD card fully?
 
* Perhaps you only have the one SD card? It may be a gigantic SD card but partitioning scares you, so this allows you to try a chosen distribution first without messing about!
 
* Some distributions have a HUGE library of packages ready and waiting to be played with.. why not see which ones you'd like ported over to Angstrom natively?
 
* We can be sneaky and take advantage of all Pandora specific updates to Angstrom, and have our Extended OS install reap the benefits rather than having to wait for someone to port it over ( kernel drivers, for example. )
 
* Why not show off just how powerful the Pandora actually is - you are running TWO X Servers and TWO Desktop Managers at once!
 
* Development between multiple distributions can be done much more easily, as you could bind your development distribution, and check driver support in Angstrom.
 
* And my absolute favourite - it's a fun hack!
 
  
So, there is good reason to try and pursue a system such as this, if perhaps not generally geared to "End User Consumption."
+
= Available Files =
  
= Extend Utils =
+
The current ExtendUtils will always be on my website here:
The system that is used to work all this is a set of scripts named Extend Utils. These have been written by Stuckie and generally allow the following:
+
[http://www.stuckiegamez.co.uk/apps/pandora/ExtendUtils/ExtendUtils.pnd Extend Utils]
* Creating a new Extend.
 
* Binding a Root Overlay to an Extend.
 
* Binding a Home Overlay to an Extend.
 
* Binding an OS Extend.
 
* Activating Swap on an Extend.
 
  
While very much work-in-progress, the scripts are very useful in day to day tasks.
+
For support, please direct questions to this forum thread: [http://www.gp32x.com/board/index.php?/topic/59740-extend-utils/ Extend Utils on GP32X]<br />
 +
Older threads where all this came from are at the bottom of this page for the interested.
  
The Extend Utils suite is updated regularly, and is available here: [http://www.stuckiegamez.co.uk/apps/pandora/SimpleDev/extendutils.pnd Extend Utils]
+
== Empty Extends ==
 +
The following files are pre-formatted ext2 files. These are included in Extend Utils, but are available here as well as they're just loop files and can therefore be used with other things:
 +
* [http://www.stuckiegamez.co.uk/apps/pandora/ExtendUtils/256MBExtend.zip 256MB Extend]
 +
* [http://www.stuckiegamez.co.uk/apps/pandora/ExtendUtils/512MBExtend.zip 512MB Extend]
 +
* [http://www.stuckiegamez.co.uk/apps/pandora/ExtendUtils/1GBExtend.zip 1GB Extend]
 +
* [http://www.stuckiegamez.co.uk/apps/pandora/ExtendUtils/2GBExtend.zip 2GB Extend]
 +
* [http://www.stuckiegamez.co.uk/apps/pandora/ExtendUtils/4GBExtend.zip 4GB Extend]
  
== Overlays ==
+
== Dev Extends ==
A quick mention about overlays.
+
All Angstrom-based Dev Extends are currently deprecated as of 26th June 2011.
  
Overlays allow the base system to be kept unmodified, and all changes to be propagated to the Extend file. This is immensely useful for development work where you can install a development environment "normally" and compile what is needed, then unbind the overlay and run it on a "vanilla" install, to see if any libraries are missing.
+
Please see the Lenny Debian OS Extend instead.
  
Stuckie's general development environment uses two Overlays - a Root Overlay and a Home Overlay - so that all compiled code sits in the Home Overlay, and the development environment is in the Root Overlay. This allows any programs that have not been configured fully to keep clear of the NAND when running, should they create config files in the user's Home directory, as well as separating the compiled programs from the development environment for easier testing on the "vanilla" state.
+
== OS Extends ==
 +
=== Debian ===
 +
Lenny Dev Extend
 +
* [http://www.stuckiegamez.co.uk/apps/pandora/ExtendUtils/debian.lennydev.20110626.zip 512mb Lenny Extend]
 +
* [http://stuckiegamez.co.uk/apps/pandora/ExtendUtils/debian.lennydev.20110629.1gb.zip 1GB Lenny Extend]
  
= Arch OS Extend =
+
The Lenny Dev Extend has a Debian bootstrap with dev tools ( enough for the Wakebreaker GL ES example plus gtk 1.2 dev )<br />
A proof of concept is available here: [http://www.stuckiegamez.co.uk/apps/pandora/SimpleDev/archmobile.1GbExtend.zip 1Gb Arch Extend]
+
About 400mb is taken, so choose the 1GB extend if you think you'll need lots of additional libraries.
  
Not much work has been taking in furthering the install as of this time.
+
Alternatively, check the [[Debian On SD]] project for booting Debian directly from SD card.
  
= Android OS Extend =
+
====Step-by-step guide====
This might be tricky due to dalvik.. doesn't seem to be any fast way of "borrowing" an existing rootfs, will have to just try compiling manually for the Pandora.
+
In this guide, you will mount a Lenny Dev Extend (to give you a protected dev environment), in conjunction with a 1gb overlay home extend (to give you room in which to place your files).
  
== Links ==
+
=====Part 1: the 1GB home extend=====
* http://elinux.org/Android_on_OMAP
+
#Download [http://stuckiegamez.co.uk/apps/pandora/ExtendUtils/ExtendUtils.pnd ExtendUtils.pnd] to your Pandora (see [[User_manual#Installing_a_PND_file_.28an_application.29|here]] for instructions on where to place PNDs)
* http://omappedia.org/wiki/Android_Getting_Started
+
#Run it. It's called "Extend Utils" and is in "System" category.
 +
#Select "New Extend/Overlay/Swap"
 +
#Select "New Overlay File"
 +
#Select a size (for example, 1gb) .. if you select "Custom", prepare for a long wait.
 +
#Place the file somewhere on your SD card (the default folder it suggests is a good choice, because it always looks there first) and give it a name. For example, "1gb"... you don't need to add .overlay or .extend, as it does it for you.
 +
#It'll now either extract a premade empty file, or if you selected "Custom" it'll DD a new file, taking forever, and probably raping your SD in the process.. you have been warned ;)
 +
#You'll be pushed back to the main menu now.
 +
#Select "New Pre-Mount"
 +
#Select "New Overlay Mount Script"
 +
#Select "Standard Home Overlay"
 +
#Choose your file ("1gb.overlay" in our example) that you saved on your SD card in the previous stage
 +
#Save the mount script anywhere you like, calling it anything you like... for our purposes, save it in the same folder as before and name it "1gb"
 +
#Back to the main menu.
 +
#Select "Mount Existing Pre-Mount". Note: If your SD card is formatted to anything other than ext/2/3/4 (for example, if it uses FAT32 or NTFS), you can't run it directly from the Terminal. You'll need to run it through Extend Utils or type <code>/bin/bash /my/mount/script.preMount</code> instead.
 +
#Choose the script you saved in the previous part ("1gb.premount")
 +
#Type your sudo password (your password to log in to your Pandora at startup)
 +
#That should be it.
  
= Debian OS Extend =
+
'''For checking purposes:'''
Debian was the first distribution to work in this environment.
 
  
Various proof of concepts currently exist:
+
#Launch a Terminal (start menu -> Terminal)
* [http://www.stuckiegamez.co.uk/apps/pandora/SimpleDev/debian-lenny.512MbExtend.zip 512Mb Extend of Lenny]
+
#Type <code>mount</code>
* [http://www.stuckiegamez.co.uk/apps/pandora/SimpleDev/debian-sid.1GbExtend.zip 1Gb Extend of Sid]
+
#Look for a line at the bottom reading something like "aufs on /home type aufs (rw, si=xxxxxxxx)"
* [http://www.stuckiegamez.co.uk/apps/pandora/SimpleDev/debian-sid-synaptic.1GbExtend.zip 1Gb Extend of Sid with Synaptic pre-installed]
+
#Type <code>cd /tmp/NAMEOFOVERLAY.overlay/YOURUSERNAME</code> (for our example, if we were stuckie: <code>cd /tmp/1gb.overlay/stuckie</code>)
* [http://www.stuckiegamez.co.uk/apps/pandora/SimpleDev/debian-sid-synaptic.2GbExtend.zip 2Gb Extend of Sid with Synaptic pre-installed]
+
#We'll put a random file here to ensure it works. Type in <code>touch testfile</code>. This will create an empty file named "testfile". Type <code>ls</code> and it should be among the files listed. Now type <code>ls ~</code>, and it should also show up.
* [http://www.stuckiegamez.co.uk/apps/pandora/SimpleDev/debian-sid-synaptic.4GbExtend.zip 4Gb Extend of Sid with Synaptic pre-installed]
 
  
Work on Debian has currently active...ish.
+
=====Part 2: Debian Dev Extend=====
 +
#Download [http://stuckiegamez.co.uk/apps/pandora/ExtendUtils/DebianDevExtend.pnd DebianDevExtend.pnd] to your Pandora.
 +
#Download the [http://www.stuckiegamez.co.uk/apps/pandora/ExtendUtils/debian.lennydev.20110626.zip 512mb] or the [http://stuckiegamez.co.uk/apps/pandora/ExtendUtils/debian.lennydev.20110629.1gb.zip 1GB] Lenny Dev Extend, unzip it (for example, using Squeeze), place the file ("debian.lennydev.20110626.extend" for 512mb, "debian.lennydev.20110629.1gb.extend" for 1GB) into the directory <code>/media/yourSDcard/pandora/appdata/DebianDevExtend</code> (it doesn't matter where you put it, but this is easier because it's the default directory it will look in)
 +
#Launch Debian Dev Extend. It's in the "System" category.
 +
#Click "Ok" past the notifications.
 +
#Select the file (named "debian.lennydev.20110626.extend" or "debian.lennydev.20110629.1gb.extend").
 +
#Type your sudo password if prompted (you may be quick enough to still be in the sudo cache that it doesn't ask). Once you're done, a Debian Dev Extend terminal will open.
 +
#In the Debian Dev Extend terminal, type <code>gcc</code> to ensure that it works. If it says <code>gcc: no input files</code>, all is well.
 +
#Type <code>cd</code> to jump directly to your home folder. Type <code>ls</code> to ensure our test file is there. If it is, you're good to go.
 +
#Your new password while you're in the Debian Dev Extend terminal is <code>pandora</code>, so remember that if you're planning to do any <code>sudo</code> commands! For example, to add new libraries you would type in <code>sudo apt-get install nameofpackage</code>. You can search which packages are available at [http://www.debian.org/distrib/packages this page]
  
If you get "cannot open display: :0.0" this is a regression in the script for xhost support, please run the following in an Angstrom terminal:
+
=====How it all links together=====
xhost local:localhost
+
You can open a file manager (for example Thunar, or Xfe) and verify that everything is as it should be. Here's how it should look:
 +
*The contents of <code>/home</code> and <code>/tmp/1gb.overlay</code> should be almost identical
 +
*The contents of the following three directories should be identical:
 +
**<code>/home/username</code>
 +
**<code>/tmp/debextend/home/username</code>
 +
**<code>/tmp/1gb.overlay/username</code>
 +
("username" represents your username)
  
Sorry about that!
+
= Extend Utils Help =
 +
Extend Utils ( version 1.0a ) now facilitates the creation of "PreMount" scripts. For users of previous Extend Utils script sets, these separate scripts are essentially a PreMount script - the Ubuntu specific variant being a prime example.
  
= Ubuntu OS Extend =
+
This allows us to create customised mounting procedures - such as being able to mount multiple Extends and Overlays at once; for example, a compressed Base OS Extend with perhaps an Overlay on top to "catch" installed programs.
Ubuntu development is on hold.
 
  
A first release is available here:
+
Unfortunately, as is always the case with more customization, the complexity has shot up. This guide will help you to create a PreMount script given four common scenarios: a Swap mount, a Home or Root Overlay, an OS Extend and a Dev Extend.
* [http://www.stuckiegamez.co.uk/apps/pandora/UbuntuExtend/ubuntu-arm-jaunty-desktop-3.5G.7z 3.5Gb Extend of Ubuntu 9.04 - Jaunty Jackalope] - note this is a 7zip archive due to it's size; there aren't many empty blocks in this for zip to compress.
 
  
Ubuntu OS Extend currently uses a slightly tailored version of Extend Utils, available here: [http://www.stuckiegamez.co.uk/apps/pandora/UbuntuExtend/ubuntuExtend.pnd Ubuntu Extend]
+
== Creating a Swap PreMount ==
 +
Perhaps the simplest PreMount script is a Swap Mount. For this, I shall assume you have not created a .swap file.
 +
# Start up Extend Utils.
 +
# Select New Extend/Overlay/Swap.
 +
# Select New Swap File.
 +
# Select 256MB - this uses one of the pre-created zipped Extend files, whereas Custom will create a new one on the card.
 +
# Save the file anywhere you like.
 +
# It will extract the appropriate zip to where you selected.
 +
# Back at the Main Menu, select New Pre-Mount.
 +
# Select New Swap Mount Script.
 +
# Find your Swap File you just created.
 +
# Select a location for your Pre-Mount script to be saved to.
 +
# Finally, back at the Main Menu, you can select Mount Existing Pre-Mount.
 +
# Find where your Pre-Mount script was saved to, and that's it.. you have swap mounted.
  
Ubuntu can be started via the '' Start Ubuntu '' option in the System Menu. Then just follow the on-screen prompts.
+
Obviously, now that the script is saved, you only need to repeat the last two steps, or just run the script direct from the Terminal.
  
== F.A.Q ==
+
== Creating a Root or Home Overlay PreMount ==
Q: How do I login?<br />
+
Again, I shall assume that you do not have any pre-made .overlay files.
A: The default user is "ubuntu" with password "pandora"
+
# Start up Extend Utils.
 +
# Select New Overlay.
 +
# Select New Overlay File.
 +
# Select 1GB - or whatever you like.. again be aware that Custom may very well chew through your SD card as it DD's direct to it!
 +
# Save the file anywhere you like.
 +
# It will extract the appropriate zip to where you selected.
 +
# Back at the Main Menu, select New Pre-Mount.
 +
# Select New Overlay Mount Script.
 +
# Select either Standard Home Overlay or Standard Root Overlay.
 +
# Find your Overlay that you created.
 +
# Select where you want to save the Pre-Mount script.
 +
# Finally, back at the Main Menu, you can select Mount Existing Pre-Mount.
 +
# Find where your Pre-Mount script was saved to, and that's it.. either the majority of / or your /home is bound to the overlay.
  
Q: Can I create a new user?<br />
+
== Creating an OS Extend PreMount ==
A: Yes, but you'll have to do it manually from the Terminal, or start the DBus service ( see below. )
+
Generally, these will have been pre-made downloads, or you'll have created a chroot environment and wrapped it in an extend already.
 +
It's also important to note that there are essentially TWO types of OS Extends - compressed and uncompressed. Compressed OS Extends should be affixed with compressed; base-debian-sid-compressed.extend for example, which donates it's a base install of debian sid, in a compressed extend. Compressed extends can only be mounted as read-only, and will likely fail if you try to mount them read-write. This means they require an Overlay in order to be much use. As such, I will describe a "classic" uncompressed OS Extend procedure, and a "modern" compressed OS Extend procedure.
  
Q: Can I upgrade to Karmic or Lucid?<br />
+
=== Compressed OS Extends ===
A: I'd highly advise against this due to the usual System V init system being replace by Upstart, which isn't too happy about being wrapped in a chroot.
+
# Start Extend Utils.
 +
# Select New Pre-Mount.
 +
# Select New Extend Mount.
 +
# Select OS Extend.<br/>This is where you need to start paying attention as to what you're doing.<br/>For our purposes, we want a basic configuration for say a Base Debian Sid Extend, and one Overlay.<br/>The Base Debian Sid Extend we would have downloaded already, and the Overlay we should have created as well.
 +
# Move the slider to select 1 Read-Only Extend.
 +
# Select our base-debian-sid-compressed.extend ( assuming you've downloaded it, and it's been released! )
 +
# We don't want any Read-Write Extends, so Cancel or select 0.
 +
# We do want an Overlay, so select 1 Read-Write Overlay.
 +
# Choose the Overlay you want to use.
 +
# Now you must name the mount point. This can be anything you like, but it must be unique and not the name of the Extend or Overlay you've chosen.
 +
# You'll then be asked where you want to save the PreMount script.<br/>Again, we need to pay attention as to what we want here.<br/>For our purposes, we just want a Terminal, so that we can call apt-get and install whatever we feel like, and run it from the shell to interact with the program in Angstrom.<br/>You can, of course, re-do this procedure later after having installed a window manager and desktop manager ( IE: something like Fluxbox and GDM ) so that you can jump fully into Debian if you so wish.
 +
# Select the Terminal option.
 +
# You'll now be sent back to the Main Menu, where you can now select Mount Existing Pre-Mount.
 +
# Select your Pre-Mount Script you've just created.
 +
# After the gksudo checks, you should have a Terminal open inside your OS Extend, where all writes go to your Overlay.
  
== Issues ==
+
=== Uncompressed/Classic OS Extends ===
Ubuntu seems to be very dependent upon it's system services being operational. This unfortunately causes us problems as we're jumping in via a chroot, rather than booting the system normally, so Ubuntu's services haven't started; though Angstrom's have.
+
These ones are much easier to deal with, and are generally what have been released up till now.
 +
These don't require an Overlay ( though you can still use one if you like. )
 +
The procedure for setting up a mount script for one of these runs as follows:
 +
# Start Extend Utils.
 +
# Select New Pre-Mount.
 +
# Select New Extend Mount.
 +
# Select OS Extend. <br/>As said, we don't need an Overlay, and Uncompressed/Classic Extends can be mounted Read-Write without any issues.<br/>Therefore, we only need to select 1 Read-Write Extend, and 0 everything else.
 +
# We don't want any Read-Only Extends, so Cancel or select 0.
 +
# We do want a Read-Write Extend...so move the slider to 1 and find your Extend file.
 +
# Again, we don't want an Overlay, so select 0 or Cancel it.
 +
# Now you must name the mount point. This can be anything you like, but it must be unique and not the name of the Extend.
 +
# You'll then be asked where you want to save the PreMount script. <br/>Now we need to know roughly what's in our Extend.. to be safe, we can just choose a Terminal as that's guaranteed to work. If, however, you know that GDM is installed and setup ( I'll ensure that any of my Extends that are, are documented in saying so ) then you can choose the GDM option.
 +
# We'll assume we've got GDM present, so select GDM.
 +
# You'll now be sent back to the Main Menu, where you can now select Mount Existing Pre-Mount.
 +
# Select your Pre-Mount Script you've just created.
 +
# After the gksudo checks, GDM will start up alongside Angstrom and ask you to login. You should have a login/password already - either from you manually setting it up, or the default that whoever set the .Extend up performed.
  
It is possible to get Ubuntu's services to start by calling ''''' sudo /etc/init.d/dbus start ''''' however, this knocks out Angstrom's services, and in particular, HAL - the Hardware Abstraction Layer - which currently detects the presence of a chroot and refuses to work inside our environment. This is bad as if we log out of Ubuntu, we've lost access to all input devices, unless we have switched back to Angstrom and restarted dbus in it's environment.
+
== Creating a Dev Extend Pre-Mount ==
 +
Dev Extends are essentially Root Extends these days.. however that's not to say there won't eventually be packs so that there's a base Dev Extend with just the bare essentials, then addons for GTK, QT, wxWidgets, etc.. so having them classed as an Extend seems like the best option.
  
Update: Hal can be kept active by following this routine...
+
The current Dev Extends are created exactly like the Uncompressed/Classic OS Extends, but for completeness sakes:
* Start Ubuntu as normal, but don't click the final Ok for GDM to start.
+
# Start Extend Utils.
* Open a Terminal
+
# Select New Pre-Mount.
* ''''' sudo chroot /tmp/osextend '''''
+
# Select New Extend Mount.
* Open another Terminal/Tab ... this'll be Angstrom, whereas your original is Ubuntu.
+
# Select Dev Extend.
* ''''' sudo /etc/init.d/dbus-1 stop ''''' (Angstrom)
+
# Select 0 Read-Only Extends.
* ''''' sudo /etc/init.d/dbus start ''''' (Ubuntu)
+
# Select 1 Read-Write Extend.
* ''''' sudo hald --daemon=no --verbose=yes ''''' (Angstrom)
+
# Find your Dev Extend.
* Wait till HAL has finished, then click OK.
+
# Select 0 Read-Write Overlays.
 
+
# Come up with a mount point.
This gets us slightly further, but still seems to stop Ubuntu's admin services somewhere, as you still cannot use the GUI-based Administration tools. You do however, seem to have access to WiFi and Bluetooth as their icons now appear in the system tray.
+
# Save your Pre-Mount somewhere.
D-Pad control now seems knocked out as well.
+
# You may now load it up through the Mount Existing Pre-Mount script if you so wish.
 
 
Touchscreen support currently does not work. The TSLib has been installed, and the xorg-input-tslib is also available, however it doesn't seem to be picked up.
 
 
 
Due to Ubuntu's services not being active, most GUI-driven Administration utilities do not appear to work. Either perform the manual command line work, or start services as above.
 
 
 
If Angstrom's X is quit ( IE: you log out of XFCE, ) although a speed up is gained, you've lost access to WiFi. I haven't yet checked if starting services inside Ubuntu would allow the setting up of the WiFi from inside the OS Extend.
 
 
 
The CTRL+ALT+F# keys for changing virtual terminals work on a USB keyboard in Ubuntu, but not on the Pandora's keyboard; neither work in Angstrom.. as such, you have to perform ''''' sudo chvt # ''''' replacing # with 2 for Ubuntu, and 7 for Angstrom.
 
 
 
Ubuntu 9.04 - Jaunty Jackalope, is currently the last version of Ubuntu to work "properly" in a chroot due to later versions changing the init deamon from the standard System V to Upstart.
 
 
 
== Further Work ==
 
Ubuntu has been put on hold for the time being.
 
 
 
I also need to have a further look into getting Karmic or Lucid running properly.
 
 
 
== Links ==
 
* [https://wiki.ubuntu.com/ARM/RootfsFromScratch Ubuntu ARM From Scratch/RootStock]
 
* [http://dedors.wordpress.com/2008/12/10/run-kde4-in-chroot-gentoodebianubuntu-howto/ Running KDE4 in chroot] - pay particular attention to the dbus/hal manipulation.
 
* [http://www.gentoo-wiki.info/HOWTO_startx_in_a_chroot Start X in chroot]
 
* [https://answers.launchpad.net/ubuntu/+source/gnome-system-tools/+question/32334 Console-Kit issues] - what we get on Ubuntu
 
 
 
= Extend Utils GUI =
 
On the forums, SomeGuy99 has suggested the creation of a GUI-based app to wrap around the Extend Utils suite. This would greatly simplify what is going on by being able to provide on-screen help, as well as tying the many scripts into one app and not cluttering up the System menu any further.
 
 
 
This also needs to be a cross-platform application, if only for the creation of Extend files. As such, it needs a cross-platform widget toolkit; preferably a light-weight one to run on Pandora as well. Currently, this looks to be [http://www.fltk.org/ FLTK]
 
  
 
= Links =
 
= Links =
* [http://www.gp32x.com/board/index.php?/topic/54651-debian-in-an-extend/ Original Thread]
+
* [http://www.gp32x.com/board/index.php?/topic/54651-debian-in-an-extend/ Original OS Extend Thread]
 +
* [http://www.gp32x.com/board/index.php?/topic/54508-simple-dev-pnd/ Original Dev Extend Thread]
 +
* [http://www.gp32x.com/board/index.php?/topic/54064-advanced-filesystem-hackery/ Original Overlay Thread]
  
 
[[Category:Documentation]]
 
[[Category:Documentation]]
 +
[[Category:Filesystem]]
 +
[[Category:Operating System]]
 +
[[Category:Tutorials]]

Latest revision as of 20:20, 3 July 2011

Overview

Extend Utils provide a variety of tools to manipulate Extend, Overlay and Swap files. These are generally loop-back files stored on SD card for various purposes which include, but is not limited to:

  • OS Extends - running another distribution on top of Angstrom in a chroot environment.
  • Dev Extends - a method of providing dev tool support without installing directly to NAND.
  • File system Overlays - redirect writes from NAND to a file on an SD card.
  • Swap - use a swap file on SD card to provide additional memory for the Pandora when needed.

This page used to only document OS Extends... it will now document all facets of the Extend Utils.

Extend Utils have been written by Stuckie and while very much work-in-progress, the scripts are very useful in day to day tasks.

Bug Reports

In order for a bug report to be useful, I need the following information from you:
Output from "dmesg | tail", "mount" and your preMount script.
A description of the error.

Just PM me them on the GP32X boards, or post in the relevant threads - Links at the bottom of this page.

Available Files

The current ExtendUtils will always be on my website here: Extend Utils

For support, please direct questions to this forum thread: Extend Utils on GP32X
Older threads where all this came from are at the bottom of this page for the interested.

Empty Extends

The following files are pre-formatted ext2 files. These are included in Extend Utils, but are available here as well as they're just loop files and can therefore be used with other things:

Dev Extends

All Angstrom-based Dev Extends are currently deprecated as of 26th June 2011.

Please see the Lenny Debian OS Extend instead.

OS Extends

Debian

Lenny Dev Extend

The Lenny Dev Extend has a Debian bootstrap with dev tools ( enough for the Wakebreaker GL ES example plus gtk 1.2 dev )
About 400mb is taken, so choose the 1GB extend if you think you'll need lots of additional libraries.

Alternatively, check the Debian On SD project for booting Debian directly from SD card.

Step-by-step guide

In this guide, you will mount a Lenny Dev Extend (to give you a protected dev environment), in conjunction with a 1gb overlay home extend (to give you room in which to place your files).

Part 1: the 1GB home extend
  1. Download ExtendUtils.pnd to your Pandora (see here for instructions on where to place PNDs)
  2. Run it. It's called "Extend Utils" and is in "System" category.
  3. Select "New Extend/Overlay/Swap"
  4. Select "New Overlay File"
  5. Select a size (for example, 1gb) .. if you select "Custom", prepare for a long wait.
  6. Place the file somewhere on your SD card (the default folder it suggests is a good choice, because it always looks there first) and give it a name. For example, "1gb"... you don't need to add .overlay or .extend, as it does it for you.
  7. It'll now either extract a premade empty file, or if you selected "Custom" it'll DD a new file, taking forever, and probably raping your SD in the process.. you have been warned ;)
  8. You'll be pushed back to the main menu now.
  9. Select "New Pre-Mount"
  10. Select "New Overlay Mount Script"
  11. Select "Standard Home Overlay"
  12. Choose your file ("1gb.overlay" in our example) that you saved on your SD card in the previous stage
  13. Save the mount script anywhere you like, calling it anything you like... for our purposes, save it in the same folder as before and name it "1gb"
  14. Back to the main menu.
  15. Select "Mount Existing Pre-Mount". Note: If your SD card is formatted to anything other than ext/2/3/4 (for example, if it uses FAT32 or NTFS), you can't run it directly from the Terminal. You'll need to run it through Extend Utils or type /bin/bash /my/mount/script.preMount instead.
  16. Choose the script you saved in the previous part ("1gb.premount")
  17. Type your sudo password (your password to log in to your Pandora at startup)
  18. That should be it.

For checking purposes:

  1. Launch a Terminal (start menu -> Terminal)
  2. Type mount
  3. Look for a line at the bottom reading something like "aufs on /home type aufs (rw, si=xxxxxxxx)"
  4. Type cd /tmp/NAMEOFOVERLAY.overlay/YOURUSERNAME (for our example, if we were stuckie: cd /tmp/1gb.overlay/stuckie)
  5. We'll put a random file here to ensure it works. Type in touch testfile. This will create an empty file named "testfile". Type ls and it should be among the files listed. Now type ls ~, and it should also show up.
Part 2: Debian Dev Extend
  1. Download DebianDevExtend.pnd to your Pandora.
  2. Download the 512mb or the 1GB Lenny Dev Extend, unzip it (for example, using Squeeze), place the file ("debian.lennydev.20110626.extend" for 512mb, "debian.lennydev.20110629.1gb.extend" for 1GB) into the directory /media/yourSDcard/pandora/appdata/DebianDevExtend (it doesn't matter where you put it, but this is easier because it's the default directory it will look in)
  3. Launch Debian Dev Extend. It's in the "System" category.
  4. Click "Ok" past the notifications.
  5. Select the file (named "debian.lennydev.20110626.extend" or "debian.lennydev.20110629.1gb.extend").
  6. Type your sudo password if prompted (you may be quick enough to still be in the sudo cache that it doesn't ask). Once you're done, a Debian Dev Extend terminal will open.
  7. In the Debian Dev Extend terminal, type gcc to ensure that it works. If it says gcc: no input files, all is well.
  8. Type cd to jump directly to your home folder. Type ls to ensure our test file is there. If it is, you're good to go.
  9. Your new password while you're in the Debian Dev Extend terminal is pandora, so remember that if you're planning to do any sudo commands! For example, to add new libraries you would type in sudo apt-get install nameofpackage. You can search which packages are available at this page
How it all links together

You can open a file manager (for example Thunar, or Xfe) and verify that everything is as it should be. Here's how it should look:

  • The contents of /home and /tmp/1gb.overlay should be almost identical
  • The contents of the following three directories should be identical:
    • /home/username
    • /tmp/debextend/home/username
    • /tmp/1gb.overlay/username

("username" represents your username)

Extend Utils Help

Extend Utils ( version 1.0a ) now facilitates the creation of "PreMount" scripts. For users of previous Extend Utils script sets, these separate scripts are essentially a PreMount script - the Ubuntu specific variant being a prime example.

This allows us to create customised mounting procedures - such as being able to mount multiple Extends and Overlays at once; for example, a compressed Base OS Extend with perhaps an Overlay on top to "catch" installed programs.

Unfortunately, as is always the case with more customization, the complexity has shot up. This guide will help you to create a PreMount script given four common scenarios: a Swap mount, a Home or Root Overlay, an OS Extend and a Dev Extend.

Creating a Swap PreMount

Perhaps the simplest PreMount script is a Swap Mount. For this, I shall assume you have not created a .swap file.

  1. Start up Extend Utils.
  2. Select New Extend/Overlay/Swap.
  3. Select New Swap File.
  4. Select 256MB - this uses one of the pre-created zipped Extend files, whereas Custom will create a new one on the card.
  5. Save the file anywhere you like.
  6. It will extract the appropriate zip to where you selected.
  7. Back at the Main Menu, select New Pre-Mount.
  8. Select New Swap Mount Script.
  9. Find your Swap File you just created.
  10. Select a location for your Pre-Mount script to be saved to.
  11. Finally, back at the Main Menu, you can select Mount Existing Pre-Mount.
  12. Find where your Pre-Mount script was saved to, and that's it.. you have swap mounted.

Obviously, now that the script is saved, you only need to repeat the last two steps, or just run the script direct from the Terminal.

Creating a Root or Home Overlay PreMount

Again, I shall assume that you do not have any pre-made .overlay files.

  1. Start up Extend Utils.
  2. Select New Overlay.
  3. Select New Overlay File.
  4. Select 1GB - or whatever you like.. again be aware that Custom may very well chew through your SD card as it DD's direct to it!
  5. Save the file anywhere you like.
  6. It will extract the appropriate zip to where you selected.
  7. Back at the Main Menu, select New Pre-Mount.
  8. Select New Overlay Mount Script.
  9. Select either Standard Home Overlay or Standard Root Overlay.
  10. Find your Overlay that you created.
  11. Select where you want to save the Pre-Mount script.
  12. Finally, back at the Main Menu, you can select Mount Existing Pre-Mount.
  13. Find where your Pre-Mount script was saved to, and that's it.. either the majority of / or your /home is bound to the overlay.

Creating an OS Extend PreMount

Generally, these will have been pre-made downloads, or you'll have created a chroot environment and wrapped it in an extend already. It's also important to note that there are essentially TWO types of OS Extends - compressed and uncompressed. Compressed OS Extends should be affixed with compressed; base-debian-sid-compressed.extend for example, which donates it's a base install of debian sid, in a compressed extend. Compressed extends can only be mounted as read-only, and will likely fail if you try to mount them read-write. This means they require an Overlay in order to be much use. As such, I will describe a "classic" uncompressed OS Extend procedure, and a "modern" compressed OS Extend procedure.

Compressed OS Extends

  1. Start Extend Utils.
  2. Select New Pre-Mount.
  3. Select New Extend Mount.
  4. Select OS Extend.
    This is where you need to start paying attention as to what you're doing.
    For our purposes, we want a basic configuration for say a Base Debian Sid Extend, and one Overlay.
    The Base Debian Sid Extend we would have downloaded already, and the Overlay we should have created as well.
  5. Move the slider to select 1 Read-Only Extend.
  6. Select our base-debian-sid-compressed.extend ( assuming you've downloaded it, and it's been released! )
  7. We don't want any Read-Write Extends, so Cancel or select 0.
  8. We do want an Overlay, so select 1 Read-Write Overlay.
  9. Choose the Overlay you want to use.
  10. Now you must name the mount point. This can be anything you like, but it must be unique and not the name of the Extend or Overlay you've chosen.
  11. You'll then be asked where you want to save the PreMount script.
    Again, we need to pay attention as to what we want here.
    For our purposes, we just want a Terminal, so that we can call apt-get and install whatever we feel like, and run it from the shell to interact with the program in Angstrom.
    You can, of course, re-do this procedure later after having installed a window manager and desktop manager ( IE: something like Fluxbox and GDM ) so that you can jump fully into Debian if you so wish.
  12. Select the Terminal option.
  13. You'll now be sent back to the Main Menu, where you can now select Mount Existing Pre-Mount.
  14. Select your Pre-Mount Script you've just created.
  15. After the gksudo checks, you should have a Terminal open inside your OS Extend, where all writes go to your Overlay.

Uncompressed/Classic OS Extends

These ones are much easier to deal with, and are generally what have been released up till now. These don't require an Overlay ( though you can still use one if you like. ) The procedure for setting up a mount script for one of these runs as follows:

  1. Start Extend Utils.
  2. Select New Pre-Mount.
  3. Select New Extend Mount.
  4. Select OS Extend.
    As said, we don't need an Overlay, and Uncompressed/Classic Extends can be mounted Read-Write without any issues.
    Therefore, we only need to select 1 Read-Write Extend, and 0 everything else.
  5. We don't want any Read-Only Extends, so Cancel or select 0.
  6. We do want a Read-Write Extend...so move the slider to 1 and find your Extend file.
  7. Again, we don't want an Overlay, so select 0 or Cancel it.
  8. Now you must name the mount point. This can be anything you like, but it must be unique and not the name of the Extend.
  9. You'll then be asked where you want to save the PreMount script.
    Now we need to know roughly what's in our Extend.. to be safe, we can just choose a Terminal as that's guaranteed to work. If, however, you know that GDM is installed and setup ( I'll ensure that any of my Extends that are, are documented in saying so ) then you can choose the GDM option.
  10. We'll assume we've got GDM present, so select GDM.
  11. You'll now be sent back to the Main Menu, where you can now select Mount Existing Pre-Mount.
  12. Select your Pre-Mount Script you've just created.
  13. After the gksudo checks, GDM will start up alongside Angstrom and ask you to login. You should have a login/password already - either from you manually setting it up, or the default that whoever set the .Extend up performed.

Creating a Dev Extend Pre-Mount

Dev Extends are essentially Root Extends these days.. however that's not to say there won't eventually be packs so that there's a base Dev Extend with just the bare essentials, then addons for GTK, QT, wxWidgets, etc.. so having them classed as an Extend seems like the best option.

The current Dev Extends are created exactly like the Uncompressed/Classic OS Extends, but for completeness sakes:

  1. Start Extend Utils.
  2. Select New Pre-Mount.
  3. Select New Extend Mount.
  4. Select Dev Extend.
  5. Select 0 Read-Only Extends.
  6. Select 1 Read-Write Extend.
  7. Find your Dev Extend.
  8. Select 0 Read-Write Overlays.
  9. Come up with a mount point.
  10. Save your Pre-Mount somewhere.
  11. You may now load it up through the Mount Existing Pre-Mount script if you so wish.

Links