Libpnd hub

Overview

libpnd is a basic collection of functions and tools to make working Pandora-specific operations easier; to wit, it is hoped multiple applications will make use of this library rather than re-implement similar functionality and lead to problems down the road. (Instead we can run into problems together and thus clobberize them.)

libpnd thus is a collection of:

• handy system code that hopefully is reusable - IO functions, CPU clock setting, etc.
• applications for supporting the Pandora's unique features, such as pndnotifyd for auto-discovery of PXML and .pnd applicatons
• libraries for support of the .pnd and PXML systems; i.e.: for locating, executable, mounting and unmounting PXML-directory and .pnd applications, loading PXML and handling overrides
• hopefully reusable code for support of these things; i.e.: a rudimentary but useful config-file parser, rudimentary singly-linked-list container, etc.

libpnd will/does also include utility functions for setting the CPU clock, reading GPIO inputs without depending on SDL, and other handy bits.

You can, nay, are required to - contribute to libpnd!

Design considerations and philosophy

While much of libpnd is obvious, some decisions had to be made and it may help to know the guiding principle behind it; the Pandora device will be a handheld computer, but libpnd is designed to facilitate certain operations with goals towards..

• simplicity - just work right, configuration not generally needed
• obviousness - work as one would expect for this sort of device; i.e.: the user assumes something will work a certain way, and it does
• like other guys - as an extension of the prior point, we have tried to follow UNIX spirit and provide reusable small tools to do the work those bits can be leveraged, but further tried to follow the spirit of other consoles. So while the device is a powerful little computer, it is probably going to be used by gaming and retro enthusiasts and so we color some of our decisions that way. (i.e.: consider that a PS2/xbox/PSP/etc will have save-game or data management as a separate application, for instance. If thats the best multi-million dollar companies can come up with over decades, well, its not so bad for us to fall back on!)
• not over-defined; libpnd and PXML try to define a bunch of things, but we also do not _over define_; we try to lead by providing examples and styles, and while we have designed ahead for many things, we have also not nailed a lot of that down too early. It is better to be simple and shipped, then complex and a mess.. adaptable rather than scary.

The library is of course broken into parts:

include     - include these files to make use of the lib
lib         - the code that produces the lib
test        - mini tools to test various pieces of the lib in isolation
bin         - test tools - tools for testing the lib as it is developed, not anticipated to be used by many
testdata/   - for testing /etc/pandora; will contain 'conf' dir, for example
apps/       - actual applications people may use, such as pndnotifyd

For now though, here are a couple notes about the design.

• it is straight C code, to be maximumly re-usable (perhaps C++ and other language bindings will tie back to it). I have gone out of my way to limit external references so that bindings are easier to make.
• contrary to the previous comment, we've used a bit of C++ for the tinyxml parser .. hopefully this is not an issue.
• I have tried to keep the API relatively simple and am faking some OO like data hiding to try to keep dev noses out of structures that are subject to change. i.e.: functions for list management and node access are provided, don't dig into the void* cast structs, or you could get burnt.
• the API should be pretty stable; so far very few user-code changes have ever been needed.. lib internal changes should be binary compatible.
• As a result of that decision, 'handles' are used; a handle is a 'black box' type, something you cannot just use 'handle h; h -> foo = 5' type code for. Instead, internally the library will cast the handle to the actual useful type (which may involve pointer arith etc). This way the user can be returned a handle, and pass the handle to various access functions to get useful data/operations.. but the user is not encouraged to look inside the data objects himself
• As another implication, access functions are available aplenty; it is a little unwieldy perhaps, but its all about keeping the API good without the user peeking inside and risking being broken down the road. (By example, consider.. you use ftell() on a FILE object; you don't go looking in FILE for the 'position' member, since it might be platform specific, etc.)
• The lib works pretty well, but there is room for improvement internally; it won't hurt my feelings for you to suggest improvements, and I'm sure we would all be grateful for any patches you provide. We've _started_ this library, but by no means wish to carry it alone!

Opportunities for improvement

Certainly nothing is perfect, especially when chasing moving targets. Some aspects of the library are less efficient, or missing, or buggy, and a few things are noted here should someone have time. The API was designed to be pretty stable so the internal code could be altered without breaking the API interface.

• Better ISO lang handling ... due to shift from original PXML spec to new PXML spec with better lang support I've not updated some of the internal code, API uses etc to handle it. (even .desktop files could include all iso-lang titles, and let the WM pick through as the spec suggests.)
• Validation of files when they're loaded
• An actual parser of PXML metadata

License

libpnd (including all the apps etc) are released under the LGPL (ie: so there should be no encumbrances that prevent linking to commercial applications.):

GNU LESSER GENERAL PUBLIC LICENSE Version 2.1, February 1999

Obtaining the library

libpnd is in the Official Pandora GIT repo: [1] To obtain yourself a copy, use something like:

git clone git://git.openpandora.org/pandora-libraries.git

Thats all you normally need. For copy-paste from the private Wiki sake, heres what I had before: Without going into how to set up a GIT, the summary that I (as a registered dev in the repo) used was:

mkdir libpnd
cd libpnd
git init
git remote add libpnd ssh://foo@git.openpandora.org/srv/git/pandora-libraries.git
git push libpnd master

Building the library

Standard 'make' should be sufficient here; pull down the directory from the GIT, cd into it, and run make. Thats it.

Note that the make default target is 'everything', which will emit:

libpnd.a - the static linked version of the lib
libpnd.so.1.0.1 - the dynlinked version 
libpnd.so.1 - the stub for the shared lib; i.e.: apps should link against libpnd.so.1 and not the 1.0.1 over-specific version
bin/* - the 'test' dir and the 'apps' dir builds will end up in bin

There are a few additional make targets, however.

make clean - wipes out the object files, the bin files, and *~* (emacs backup files), empties some of the deploy stuff, etc. Do this before committing to GIT or we will beat you with sticks.
make deploy - populates the deploy/ directory, which is what is used inthe shipping images; i.e.: testdata/conf includes test configs, while deploy/etc/pandora/conf includes the configs we ship with. This make will copy over some of the libs and binaries and such.
make pnd - create a few sample pnd files for testing the lib on an x86 machine

Included Applications

pndnotifyd

Manages the auto-discovery mechanism and integrating PXML.xml dir-apps and .pnd files into the FreeDesktop standard system (.desktop files). A dedicated libpnd-aware menu could leverage libpnd directly and not need pndnotifyd, though pndnotifyd does offer some handy bits such as HUPping apps to suggest to them that applications may have been added/removed (libpnd includes all the necessary functions for this as well.)

pndvalidator

Is mostly a skeleton right now; in the future, it is hoped it will inspect a PXML.xml and then check to ensure the referenced assets are present; the goal to validate the PXML.xml is spec compliant, and the resulting application would be useful.

discotest

One of the test tools; it essentially runs an auto-discovery to list off the found apps (be they PXML.xml dir apps or .pnd files), and can dump various data or invoke one or other little tasks; used to test lots of bits of libpnd as it was developed.

rawpxmltest

Reads a given PXML.xml and spits out some of the fields found within.

Test tools

See the test/ dir in the libpnd source for assorted goofy little test apps used for testing bits of libpnd over time. They also may serve as source examples for using some of the APIs. They're old and shoddy probably too.

Distributed PXML.xml-directory and pnd applications, how does running them actually work?

This is just a quick summary to save you guessing by reading this whole document.

In essence, libpnd includes lots of real code, but in many places it passes heavy lifting off to shellscripts. This is both handy because the scripts are easily modified without recompilation, and further extends our design goals of making the library bindable to any other language.

Remember, PXML.xml directories are just as executable/mountable as .pnd apps.

So while libpnd is mostly straight-C and so easily bound to python or perl or whatever, libpnd itself invokes many scripts. Those scripts can be directly used by other applications, including other shellscripts.. so libnd functionality can even have bindings in other scripts. This is very flexible and powerful, and handy during debugging or SSHing into the Pandora, etc.

The application lifecycle goes like this:

• device is turned on
• during init, pndnotifyd is brought up and inhales the config giles
• pndnotidyd will attempt to auto-discover what PXML.xml directory apps and .pnd-file apps are in the configured searchpaths
• pndnotifyd will set up filesystem watches on the directories in the notify searchpath; if any of those directories are touched, pndnotifyd will wake up and re-auto-discover .. possibly removing .desktop entries or creating new ones. (i.e.: you drop a new app into the app searchpath, or remove one, or mount an SD card, etc. This is how appications immediately show up or vanish from the window manager desktops.)
• a libpnd aware launcher can execute applications direct (using libpnd)
• a libpnd unaware launcher can execute applications via the .desktop standard system; pndnotidyd will emit .desktop files as needed.
• when libpnd or a .desktop file need to execute an application, they do it via pnd_run.sh (which may be leveraged by other applications, but they hould use libpnd if they can instead, for utmost compatiblity down the road.) pnd_run.sh does the mount of the ISO, runs the actual execable and so on, as directed by libpnd or the command-line args from the .desktop. pnd_run.sh will clean up after everything as well.

So it all works like magic, but it is not that complex a system once you've got the hang of what all is going on.

application → libpnd → shellscripts → system

FAQ: IF you want to make files visible to the apps, do you have to modify the pnd?

No. When apps write back to themselves, the writes go to their /pandora/appdata on the SD that houses them. The next time the app reads a file in its directory, the OS will ensure they get the newer file, which is the one in the appdata.

Likewise, you can just drop a file into the appdata and the app will see it.

The app doesn't know about appdata and its pnd file.. the OS and libpnd take care of all the magic.. the app sees one directory, but the reality is there is magic merging going on behind the scenes.

Assumptions about the filesystem and Configuration

Note that for libpnd to work, it makes certain assumptions about the filesystem and config files. Since libpnd was developed with the images it should work for the Pandora .. but just dropping libpnd onto another Linux distribution and it won't likely 'work out of the box'. Certain config files will need to be set, and certain conventions followed.

Searchpaths

libpnd was designed so that most everything is configured in files, and that these files should be easily hackable and easily overriden, without compromising the machine. It should be feasible to add a config file to an SD card, have it take over if available when required, and then 'undo' those configs simply by removal of the SD card and rebooting.

To facilitate this design goal, all config files are 'sought' using a search-path system.

Developers should be mindful of using the libpnd APIs so this behaviour is inherited to interested applications; developers could also reuse the logic for searching for files, if they find it useful.

Searchpaths are colon-delimited list of paths, searched left to right.

ex: foo:bar:/my/baz means to look for a given file in 'foo', then in 'bar', then in '/my/baz'.

The standard config files have a default searchpath, and those config files will define further searchpaths for scripts and such the system uses. This makes the system highly flexible -- default scripts are included, but can be overridden by dropping the right named script on an SD card, or by altering a config file.

See the example config files below for example searchpaths -- i.e.: how pndnotifyd searches for pnd_run, etc.

Notification paths

The 'apps' config includes a searchpath that specifies the sub-paths to be 'watched'. Various watch mechanisms exist for various filesystem types, but over the course of Pandora development (various kernels, various k-modules, various DM and WM's, I found the included system most stable, though not most current. After a final device image is settled on, it might be time to alter the notification system, but it should not impact the libpnd API.)

Anyway, watches are placed upon the paths defined, and if anything in those paths changes, the auto-discovery code is kicked off to seek our PXML.xml and .pnd type applications in the search paths.

Note that searchpath and notifypaths are different (and usually very similar, but have the option to be very different.)

Config files

libpnd includes a generic simple config file parser and assumes the presence of a few config files. It also includes defaults so if a config is missing or broken things should work.

In general, Pandora specific config files should be located in:

/etc/pandora/conf

The config files are:

conf -- this file is generally _not_ included with the Pandora and left to default; its job is to define the basic default searchpath for config files, so should you wish to override things entirely, you may need to alter this file. i.e.: If you wish to switch to test config files on SD or in /tmp, instead of using any other conf files .. you can create this guy. This provides a mechanism to short circuit the entire conf system.

apps -- this config file defines auto-discovery and other application bundle parameters

desktop -- this config defines "dotdesktop" .desktop and FreeDesktop integration paramters -- example is where to spit out icons or .desktop files from discovered PXML.xml or pnd applications

categories -- this config maps PXML.xml spec based categories to FreeDesktop standard categories; in this way a PXML based applicatoin can show up in the appropriate categories of a WM that does not use libpnd

conf File

No default is included with the distribution.

The default search path is reasonable and assumes we want the official /etc/pandora/conf near the end, with checks to the SD cards for overrides first. (Likely most people will not override, so the canonical location will be used 99.9% of the time.)

#define PND_CONF_SEARCHPATH "/media/mmcblk0p1/pandora/conf:/media/mmcblk1p1/pandora/conf:/etc/pandora/conf:./testdata/conf"

The key for override is:

#define PND_CONF_KEY        "conf.searchpath" /* if not found, use PND_CONF_SEARCHPATH */

Which suggests using something like this.

[conf]
searchpath   /foo/bar:/baz/bing

desktop File

# Open Pandora
# Desktop configuration

[dotfiles]
#(~/Desktop for xfce, /usr/share/applications for WMs that actually follow spec)
dotdesktoppath  ~/Desktop/      # path for pndnotifyd to spit .desktop files into (run as root)
iconpath        /tmp    # path for pndnotifyd to drop icons into (can be same as .desktop if WM permits)

[launcher]
# if hupscript is commented out entirely, pndnotifyd will not try to find/run the hup
# if it is uncommented, pndnotifyd will attempt to invoke the hupscript after emitting .desktop files
# (the hupscript exists to hup the WMs to redisplay .desktop apps)
#hupscript      pnd_hup.sh

apps File

# Open Pandora
# Application configuration

[autodiscovery]
# searchpath is a list of paths (colon separated) in which to look for PXML.xml or .pnd-file applications
searchpath      /media/*/pandora/apps:/usr/pandora/apps
# notifypath is a list of paths to monitor; if anything in those paths changes, the searchpath is rescanned
# note that for each path chunk, all current subdirs of that path are also watched)
notifypath      /media:/media/*/pandora/apps:/usr/pandora/apps:./testdata/app?

# PXMLs may be overridden .. i.e.: overrides are a subset of PXML, where the values are copied over the full PXML
[overrides]
# searchpath to look for PXML overrides (which are named by the unique-id)
searchpath      ~/pxml-overrides

# [pnd] defines where to locate the pnd support scripts, so the user may override pnd_run.sh without clobbering built in
[pnd]
searchpath      /media/*/pandora/scripts:/usr/pandora/scripts
runscript       pnd_run.sh

categories File

# Open Pandora
# dotdesktop configuration

# this config file maps 'PXML' categories to free-desktop standard categories
# i.e.: category 'Foo' could map to more standard 'Utility', thus making .desktop file
# emitting a more useful thing

# the standard listing of categories is:
# http://standards.freedesktop.org/menu-spec/latest/apa.html

# note that 'map' section in the config is _required_ for a match to be found; this
# is done to separate categories from (future) top-level directives

default Application;Utility;Network;

[map]
Development     Development
Education       Education
Games   Game
Graphics        Graphics
Internet        Network
Multimedia      AudioVideo
Office  Office
Settings        Settings
System  System
Utilities       Utility

SD card layout

libpnd concerns itself with things in the 'pandora' root folder on SD. We agonized over this decision, but settled on this to avoid cases where someone with a 16GB SDHC card puts 50-million files with ".pnd" on them, and expects reasonable performance. Instead, applications for instance would have to be in:

/pandora/menu -> apps that will go to /usr/share/applications and should show up in xfce menu by category
/pandora/desktop -> apps that will go to ~/Desktop and show up on user desktop as icons

The following location is used by pmenu (and may get deprecated, but I've not discussed it with cpasjuste yet.)

/pandora/apps

i.e.: the searchpath for autodiscovery includes /media/*/pandora/apps (and desktop and menu now)

Also in the /pandora/ folder is a magic 'appdata' folder:

/pandora/appdata

When a .pndfile or PXML.xml dir get mounted, any writes back to the mounted area are put into appdata, with a subdir by the unique-id. i.e.: With the hypothetical MAME.pnd, it might update its config files after a user changes something, or create new files, or store hiscores or whatever. If it doesn't write them to NAND, and doesn't want ot write them to /mame/some/dir, but instead record it to itself for whatever reason, it would go to /pandora/appdata/MAMEs-ID/ instead. The pnd or PXML.xml app area is never modified during a run, only the appdata. (But when an application reads its own path, it will pick up the appdata version of the file instead of the actual file. Then magic of union-type filesystems!)

Example:

If Myapp with unique-id foo.123 is mounted, then any write-backs are directed to..

/pandora/appdata/foo.123/

On the SD that hosts the application.

NAND layout

In addition to seeking applications in SD cards, /usr/pandora/apps is included in the default searchpath, so that 'built in' apps can be listed, or users can drop them there.

It might be a cool addition to put a user homedir into the searchpath as well, so NAND but user-homedir (for on-device browser downloads, say.)

Command line tools and testing

A number of scripts and tools are included or built with libpnd, to facilitate testing of the lib as its being developed and to assist in making .pnd files and PXML.xml files.

PXML.xml and .pnd files

On most platforms, applications are distributed as an executable and optional datafiles and likely an installer. The installers job is to integrate the application into the filesystem, as the transport mechanism itself is not executable and is considered temporary. In Windows an application might be installed into Program Files, while in Unix an application might be dropped into /usr/local/bin .. data files in various other places.

For the Pandora platform we wanted a very easy to use workflow -- download and run -- and also have a very different operating environment. With multiple SD cards available for expansion (and that space being highly desirable due to a smaller internal store) it is desirable to install applications to SD, and yet SD cards are by definition something users will wish to carry multiple of. With two slots, it could be quite a chore to install an application to SD1, its data to SD2, and then say have only one of them plugged in at a time, or switch which slot they're in, or any number of scenarios. i.e.: Installing onto a medium which may be present or not at any time is very different to how most operating systems work. So we made something new and awesome.

The PXML.xml and .pnd-file approach we came up with attempts to address these issues.

The goals:

• for the user, an application can be a single file download
• furthermore, there is no installation whatsoever, beyond dropping that single file into the right place on the SD card
• using the application should be easy -- stick in the SD card, and tap/launch on the icon.
• for uninstallation, delete the single file. Thats it, no fuss.
• for the developer, creating that single file should be easy to do
• for the system, performance should be high

Remember, an application can be a .pnd 'bundle' (single file containing all the sub-files), or a PXML.xml 'dir application' (an unpacked .pnd esentially.) BOTH PXML.xml-dirs and .pnd's ARE EXECUTABLE.

pndnotifyd will auto-discover PXML.xml in a directory and consider that an application, emitting a .desktop file, just as well as a .pnd file. i.e.: This can be hand for development or users. Remember that a mounted PXML.xml dir still redirect write-backs to the /pandora/appdata directory, just like a full .pnd application. They are treated the same!

What are PXML.xml and .pnd files?

PXML.xml is a file describing an application A directory containing a PXML.xml _is executable_ A .pnd file is the packaged up version of the directory that is executable. i.e.: a PXML.xml in a directory is just as executable as a .pnd representation of that directory.

Example:

mkdir OutcaST
cp outcast.bin OutcaST/
cp PXML.xml OutcaST/
# at this point, the OutcaST directory is now discoverable and executable

pnd_make.sh -this -that OutcaST outcast.pnd

1. at this point, outcast.pnd is executable (as is the OutcaST directory), so upload outcast.pnd to the Open Handhelds Archive and let people enjoy!

A note on including shared libs

A philsophy for packaging that I am advocating is to 'just make applications work' -- avoid dependancy trees.

Some guidelines:

1) you can depend on shared libs included with the firmware to always be there

2 you should include any shared libs you need for your app, that are not in the firmware (yes, this increases app size a little bit, and potentially makes for duplication among apps that use the same shared libs)

3) if a shared lib becomes desirable in the firmware, it can be added there as a firmware patch easily (this has the advantage of gradulaly updating the entire user based, and apps easily saying 'we depend in firmware version X', rather than making for pnd-dependnacies and the equivilent of 'dll hell'

4) You'll have to be careful in linking your application to ensure that the shared libs you are including in the pnd/PXML-dir are to be locatable by your binaries. ie: stick them in a 'lib/' dir in your bundle, and stamp your binary with a library-path './dir/' so they can be found, say. More later.

Future idea: pnd's for containing system-wide shared libs

This is something we've bounced around for quite awhile, but not committed to. As it keeps coming up I thought I'd put some notes here in public.

It is possibly desirable to make a special searchpath that is discovered during machine boot, and any pnd's found there are automatically mounted (and left mounted) into a special path (say /usr/local/pandora/lib or somesuch.) This way, collections of handy utilities or shared libraries could be distributed as a pnd, and become available in consistent locations. If 10 applications need a shared lib, and don't want to duplciate it in each of their pnd files, this could be an approach.

I tend to think it is best avoided, however, since it complicates the user experience -- we end up with pnd depandancies, and apps nolonger about to 'just work', and apps that did work and 'suddenly break' when other files are altered. The philosphy we're going for with pnd files is 'drag and drop' -- user downloads a pnd file, and it works.. a self contained application, nice and easy. We start complicating the dependancies, and it turns into 'dll hell' as seen on other platforms.

That said, it is _not_ difficult to add into the existing system and design; we've been mulling it over, but it is best avoided for now. rather than complicate the system up front, let us try to get a functional and as lightweight-as-possible system on the uptake, gather some usage data points and then worry about upgrades :)

File format - PXML.xml / PXML Specification

See the PXML specification

Alternatively the spec is included in the docs/ directory of the libpnd distribution, though the wiki version linked above should be considered cannon.

A note on history; the PXML spec was originally created by EvilDragon, and went through a number of revisions as suggestions came in; as these things are wont to do, the target was ever moving and the scope and meaning of PXML changed so at some point a more strategic approach was needed. dflemstr took it upon himself to help recode some of the parse and re-evaluate many of the PXML decisions and hence was born the PXML-version-2 spec.

File format - .pnd files

The pnd file format is pretty straight forward, though most will interact with it through the included (or community supplied) tools.

Basically its just an ISO file (or other filesystem type) with a PXML.xml and an optional PNG icon appended. (example: cat appfolder.iso app.pxml icon.png > app.pnd) For ease of discussion, I refer to ISO but in fact you coudl be using squashfs or other types.

The ISO filesystem portion of the file can include anything the developer wishes, as long as it includes a PXML.xml in its root file and an executable somewhere in there. It can also include images, audio files, data files of any kind. It can be small or large.

Yes, the PXML.xml is included in the ISO portion (in the root directory), and also appended to the end of the .pnd file (for _convenience_ .. it is very easy and fast to locate in this position; we can backseek into the ISO to find the PXML, but if its a 500MB ISO it could take awhile. Hence, we append it and can find it in one seek.)

NOTE: The naming of the PXML.xml file is important! The name is case sensitive and must be "PXML.xml", nothing else!

This simple format makes it possible to easily create .pnd files with the default utility's of most operating systems. Further, it is a fairly fail-safe system -- the PXML is appended, but is also included within the ISO image part of the file so if the appended-PXML is ever lost (unpack and repack poorly, say) the system will still work (the PXML inside the ISO can be found easily, even without mounting the ISO _as_ an ISO, since it is a plaintext uncompressed image format.)

Manually creating a .pnd (or even a PXML) is a pain so there will be a few helper scripts or maybe even a gui app out when we launch.

Included with libpnd are some scripts to create PXML and pndfiles, though it is expected people will create handier utilities. See the tool section of this wiki for some details.

Question from the gp32x forum regarding confusion of PXML location -- remember that in general mot developers will keep a directory that represents the .pnd or PXML-dir application. Their makefile/build-process will drop the executable(s) into that directory, which inludes a PXML.xml description of the application, and possibly an icon (and whatever all else they wish.) To build the .pnd file, a script (say) is used which compresses all this into the pnd format (including appending the PXML.) So yes, the PXML is in two places in the .pnd file and one place in the source directory. You'll get used to it ;)

Rationale --

Several iterations of pnd files have been considered, from zipfiles to cramfs to other formats, to putting in a key/value set of information tags in the file, to table-of-contents indexes, to any number of options to help performance, and so on. In the end we went for a good overall and simple solution and dropped getting fancy -- PXML.xml can be extended to define the type of the file if it cannot be magic'd, so we could using other filesystem types in the future (and this is why people should use libpnd rather than rolling their own solution.) For now however, pnd files are ISO files with the PXML.xml appended and an optional icon. This keeps the PXML and icon easy to locate, and yet a easy file format.

Recommended layout of a pnd directory structure, ie: how to make a well behaved pnd file

These are guidelines to make pnd-files 'well behaved', and to help move towards a consistent pnd structure.

• Always include your icon png-file and PXML.xml file in the _Root_ of the pnd-file; this is for reasons that folks may well unpack the pnd-file, and then repack it; in doing so, if the icon and PXML.xml (normally appended to the .iso to become a .pnd) will be lost, unless the user or tool is being careful. Thus, it is wise to include them within the .iso, to make things so the iso is the entire pnd application, in unpacked form.

• Use a 'reasonably unique' unique-id of course (to identify your application.) This is used for many internal uses, so uniqueness is pretty wise.

• Use an 'application relevent' appdata name; make it as unique as you have to, but consider being less unique; a random example could be .. possibly there may be several quake1 ports; they could all share the same appdata (same config files, same data files), but if you intend on having different config or data files than other quake1's, make your appdata named differently.

• Always include a documentation 'info' link, if your app has any setup requirements, has popup help screens that the user needs to know a key to pop up, or other essential trivia; feel free to make the doc link as complete as you like, since its all within the pnd file. (ie: Include the full DosBox documentation if you want to, in your DosBox pnd :)

• Try to set up the appdata directory for the user; if you need a 'roms' or 'image' directory in the appdata (such as in the case of an Atari ST emulator needing a location to house the TOS OS image, say), then do a 'mkdir' in your 'runscript', to create it. Thus the user will find a (example) Hatari appdata directory, and within, any directories he needs (possibly empty.) This just makes it obvious for the user where to put stuff.

• Try using rudimentary zenity scripts to pop up imporant pre-run information, if the application can't do so; ie: Hatari itself has a very fine GUI in it, but if the Atari ST TOS image is not present, it just exits and fails to run, so the GUI doesn't have time to do anything useful. In this case, it could be wise to use a 1 line zenity script that checks before the emu runs for the file in question, then pop up a dialog box saying 'put this file /here', so the user knows what the heck happened

• Always Always Always write your 'state info' to current-working-directory (ie: ./foo, and not ~/foo). The pnd-system will redirect this to the users appdata of course, keeping things clean and predictable. If you're just porting an app, that has ~ all over the darned place, and you don't want to muck up the code.. just have your 'runscript' do something like 'set $HOME=.' and you're good to go!

• Runscripts; the PXML.xml can specify to run your binary, or you can run a .sh script that in turn runs your binary, say. This lets you do various tests, set up LD_LIBRARY_PATH, or other mischief before your binary gets run. Very handy. I often have PXML.xml run 'runfoo.sh', and runfoo.sh in turn runs ./foo after setting things up, making directories, etc.

• For data that is not kept in your appdata directory, try to keep it local to the current SD _or_ better still, search both SD. ie: libpnd uses a basic but really handy system for iterating across a list of directories (including glob type expansion); so you can tell it something like /media/*/foo:/tmp and it'll check every mounted device (including both SD slots) in the 'foo' directory, as well as /tmp; you can have it call your code as it looks through, or you can ask libpnd to find a file for you and report where it was found, across all your searchpaths; you don't have to use libpnd's code to do it, you coudl write oyur own (using libpnd as an example if you need the help), as its pretty easy to build (20 lines of code maybe?) Anyway, the trick is to be multi-SD aware, and bonus marks for handling if theres a USB mounted drive etc as well. (Example; I coded minimenu so that when it caches out a preview picture, it checks for free space first, and tries first on the same SD that the pnd is mounting from, and then across a searchpath of alternate locations to cache to; this way the user can customize the caching, and it can be on various devices.)

What filesystem types can be contained within a pnd-file?

The pnd container can really contain a large variety of filesystem types; day-zero we're aiming for ISO since it is simple, easy build across every major OS, and fast to use. But it is not the only supported filesystem type, and more should be added all the time. (Possibly '-auto' can be added to pnd-run and it'll "just work" for dozens of types!)

A compressed filesystem type has always been desirable for file sizing reasons, though it potentially impacts performance (depending on the application operation really.)

Supported types (not an exhaustive list!):

o ISO

o squashfs

PND generation workflow

- create a directory to represent the application on SD card; i.e.: ./mame-app

- create a PXML.xml fil and drop it into the dir (./mame-app/PXML.xml); genpxml script can be used, or do it by hand

- drop the executable and data files into this dir (say, ./mame-app/mame.bin, and ./mame-app/artwork/foo, etc.) <- note, with a PXML.xml and executabe, it can now be discovered and run without even bundling into .pnd!

- invoke pnd-make.sh to produce mame4all-pandora-1.0-arm.pnd or whatever you want to call it (foobearspanky.pnd, we don't care.)

- you're done, distribute it as you see fit

Making PND files - Windows / DOS commandline

There should be a basic GUI along very soon, and more advanced friendlier GUIs in due time. But for now, we thought it prudent to include some basic instructions on the command line for day-zero.

For this command-line tutorial, I will assume you have already created a PXML.xml file; copy one of the templates and then edit it, using notepad or whatever _TEXT_ editor you see fit.

Prepare in advance:

1) Make a PXML.xml file (through whatever means; copy a template, or get a PXML.xml tool when they come available.) Ensure that you've got a reasonably unique-id in that field (ie: say, app-name, your name, and date/time all catenated together.)

2) Get yourself mkisofs tools; for example, grab from here: http://www.student.tugraz.at/thomas.plank/ -- get cdrtools-2.01-win32-bin.zip and DLL version 1.5.25, and extract to somewhere. That lets you run 'mkisofs' to spit out .iso files

2b) If you want to make a compressed image, use squashfs tools here: ftp://ftp.slax.org/useful-binaries/win32/squashfs-tools/

3) Put your application and all needed files into a single directory .. say "myapp"; it doesn't matter what you call it. Might be cool as a Makefile step

4) In your app-dir, put the PXML.xml file in the root

5) Put the icon in the root of that dir as well (make it a .png file)

6) Make sure the PXML.xml names the icon (just its filename.png), and specifies the executable (foo.bin or whatever.)

Generate the actual PND-file, fully manual method:

1) Make yourself the iso; this step produces the iso, but to turn the iso into a pnd you need a couple last steps. You could also use some fancy iso-building tool. mkisofs -o foo.iso -R /path/to/folder/myapp

2) Append PXML.xml to the .iso (yes, the PXML.xml is included in the iso, but we also want to append it for performance reasons.) copy foo.iso+PXML.xml foo.step2

3) Append the icon PNG file to the .iso (if you have an icon; if not, you can leave this step out.) copy foo.step2+myicon.png foo.pnd

4) Clean up: del foo.step2 foo.iso

5) Profit!

Making PND files - Linux/BSD/Unix (including on Pandora)

Command-line

Tools are included in the libpnd distribution; see below for discussion, but suffice to say they are included in the GIT. Look in testdata/scripts for:

• genpxml - spits out a template PXML.xml for you, and even includes a reasonably unique-id
• pnd_make.sh - given a directory and some arguments, will spit out a .pnd file; great for use in Makefiles

We're working on others as well, so it should be a piece of cake under Unix and Unix-like OSes.

GUI

TBD

PXML overrides

There are a couple of ways to effect an override of the PXML, without modifying a pnd-file.

"In-path" override files: .ovr

The library automatically imports a limited number of overrides from a conf file that can be located in the same location as its matching .pnd file is.

For instance, if /pandora/desktop/awesomegame.pnd exists, then libpnd will automatically also search for /pandora/desktop/awesomegame.ovr

Some menu systems will also be smart enough to watch for an icon override -- /pandora/desktop/awesomegame.png -- and if present, to use it; 'minimenu' (mmenu) is aware of that, as is pndnotifyd daemon when emitting .desktop files. Likely pmenu will also be updated. In effect, the user can override the icon, category and other attributes of the pnd, without needing special tools or to alter the system.

An .ovr file must be named identically to the pnd-file, excepting the 3 character file extension.

NOTE: Given that a pnd may emit multiple applications from its PXML.xml, the ovr will have to be aware of this so it can override the correct application(s) within the pnd. Some menus and information tools will display the application list with 'subapp number' to assist identification. minimenu shows the subapp number in the detail panel, for example. In the case where a pnd-file has only one contained application, that is subapp-number zero. For any additional subapplications, increment by one .. but it can be tricky to know which app is which subapp number. (A careful eye will notice the .desktop filename includes the subapp-number after the pound sign .. foo#3.desktop is subapp_number 3.)

The ovr file may (at this time) override the icon title, the CPU clock speed to set on launch, the main category, and the first subcategory for the main category. Additional fields will become overridable.

Minimenu honors up to 3 lines of 'notes', pulled from the .ovr file. (Make sure they are in the right subapp group). note-1, note-2, note-3, see example below. The notes in minimenu are shown at the bottom of the detail text panel.

[Application-0]
title                   HatariHack0
maincategory            Audio
maincategorysub1        Emulator
[Application-1]
title                   HatariHack1
clockspeed              200
note-1                  My note text line 1

PXML override file -- currently unsupported

PXML.xml files can have any number of their elements over-ridden by user preferences.

The design goal here was to make it so user-preferences could alter the impact of the PXML.xml in any way, but without requiring the actual PXML.xml in the .pnd (and within the ISO therein) to be altered (could be a 500MB ISO after all!), and further, for these changes to persist. Consider than an SD card could be removed and inserted into the other slot, or the app could be removed entirely and come back another day, or other scenarios.

As such, the PXML-overrides are kept in NAND in the user homedirectory (See the config files to find out where.)

libpnd includes functions to pull in PXML, and also functions to then look for and merge in overrides. It is not always done since you may wish to inspect the PXML or the overrides, so the developers job is to do as they see fit -- pull the XML, and likely also pull in the overrides.

At this time I'm not sure if people will ever use overrides, but the system has been built in should it be needed.

Included shellscripts to generate files and other sundry duties

genpxml

This tool can spit out a basic but correct PXML.xml, faking up a halfway-useful unique-id in the process. It might be handy in saving you writing up a PXML.xml from scratch.

pnd_hup.sh

pndnotifyd will attempt to locate this script and invoke it after an auto-discovery run, if the pnd_hup entry is present (and uncommented) in the config files. It is expected to be commented out in release images, as xfce watches directories for .desktop files and does not need a HUP; some desktop managers need to be HUPped to rescan for .desktop files in a timely fashion, so the pnd_hup config entry can be uncommented and this script will attempt to figure out who to HUP.

pnd_make.sh

This is one of the great ones :)

Given a directory with a PXML.xml (see genpxml above) and an executable and any other goo you want in there, this script will spit out a .pnd file you can execute or distribute as an application.

pnd_run.sh

This job does a lot of the heavy lifting for libpnd, including setting up the 'union filesystems', doing .pnd and PXML.xml-app-dir execution and mounting, and so on. Generally you will talk to it via libpnd or .desktop files, but you can invoke it dorectly from your apps if you wish to circumvent libpnd or mount .pnd files within a shellscript/perlscript/etc.

Auto-discovery of applications

TBD.

Integration with FreeDesktop .desktop systems

Icons

There are some rules for Icons in PXML.xml and pnd/directory bundles

- An Icon should be in the root of the directory or .pnd bundle

- An Icon must be specified in the PXML.xml if you expect it to work

- As the PXML.xml is parsed, the icon is sought; if the icon is specified, but not found, it is assumed to be a system default icon and the filename will be placed into the .desktop Icon= line verbatim (ex: Icon=foo.png), and the window manager presumably will know how to locate it. If the icon is specified, and is found, then it will be copied into the configured IconPath, and the full path to it will be used for the Icon= line in the .desktop file ex: Icon=/tmp/myicon.png). If no icon is specified in the PXML file, then no Icon= line will be entered into the .desktop and the WM will presumably apply a default icon based on category or whatever.

APIs

Feel free to just go read the .h files; I have tried to comment liberally :)