Client Repository Communication API

Client <--> Repository Communication API

A Client can do more than just download the list of available applications from a given PND repository, So long as that repository provides the correct API interface.

All data is sent to the server via POST apart from when downloading an application which uses URL Parameters via GET All data is returned to the client in the JSON format.

You must specify the root of your client api in your repository's master json - see "client_api" string.

All the client API requests should be relative to this root e.g.

• <root>/handshake
• <root>/rate
• <root>/comment
• <root>/download
• <root>/history

The current API consists of the following:

API Key

A security md5 hash which would be generated for the user by the PND repository upon request, this would then be entered by the user and stored by the users client of choice.

This has the benefit of not requiring the user to enter his/her password into the client or transfer it via HTTP_POST (this is mainly for convenience but is also more secure).

Handshake

Current Repo Handshake URI is http://repo.openpandora.org/client/handshake

Before any action can be carried out by the server Cyrptographic Authentication using nonce in a similar way to Digest Access Authentication is performed to make sure these requests are authenticated.

//---------> through POST method client sends request for handshake by POST param stage=1

//<--------- the client generates a new session (which lasts only 15 seconds), generates a
random 12 digit nonce sequance and returns this value to the client.

//---------> once the client has this nonce it generates its own client nonce and POSTs
this back to the server along with the username of the user and a special md5 hash
which works like this md5(NONCE+CNONCE+APIKEY). So the following gets posted
back to the server:
cnonce=<CNONCE>&stage=2&user=<username>&hash=md5(NONCE+CNONCE+APIKEY)

//<--------- as long as the session on the server is still active, the server pulls up the
users APIKEY from the db and replicates the client submitted HASH for validation i.e.
md5(NONCE+CNONCE+APIKEY), the server then checks both hashes against each
other (client hash and server hash) and if they match we allow an action, after which
the session is destroyed again.

//if there are any errors in the POST or missing required params the session is
destroyed and the handshake needs to start again.

• The "stage" (stage) parameter should have the value of either 1 or 2.
• The "user" (username) should contain the users repository username.
• The "cnonce" (client nonce) client randomly generated nonce code.
• The "hash" (hash) md5 hash generated in the following method md5(NONCE+CNONCE+APIKEY).

These parameters should be transferred in the URL via the POST method

handling errors or success

Here is an example of how an error would be reported to the client.

{"error":{"number": 1,"text":"Handshake stage not supplied."}}

And here is an example of how success would be reported

{"success":{"number": 1,"text":"Request your action."}}

Server Sessions

when starting a handshake the server will create a session from which it can store data such as the username and access token for the duration of the client/Repository interaction.

If at any point the handshake fails or once the request has been completed, the session is destroyed, any further actions will require a fresh handshake.

A session may only last 15 seconds after a successful handshake meaning if no action is requested within those 15 seconds the session is destroyed and again any further actions will require a fresh handshake.

Rating Retrieval

Current Repo Rate URI is http://repo.openpandora.org/client/rate

After a successful handshake 2 parameters will need to be POSTED to the appropriate server API URL:

• The "a" (action) perameter should be "gr" if you are retrieving a rating.
• The "id" (application id) parameter should be the ID of the application we are assigning the comment to.

These parameters should be transferred via the POST method

Here is an example of what this returned JSON output would look like:

{"success":{"rating": 4}}

Rating Submission

Current Repo Rate URI is http://repo.openpandora.org/client/rate

After a successful handshake 3 parameters will need to be POSTED to the appropriate server API URL:

• The "a" (action) perameter should be "r" if you are submitting a rating.
• The "r" (rating) parameter can only be a number between 1 and 5.
• The "id" (application id) parameter should be the ID of the application we are assigning the comment to.

These parameters should be transferred via the POST method

Upon successful rating submission the server will return the new overall rating for that application.

Here is an example of what this returned JSON output would look like:

{"success":{"number": 3,"text":"Your rating has been updated.","new rating":96}}

Comment Submission

Current Repo Comment URI is http://repo.openpandora.org/client/comment

After a successful handshake 2 parameters will need to be POSTED to the appropriate server API URL:

• The "c" (comment) parameter can be a string of up to 300 characters.
• The "id" (application id) parameter should be the ID of the application we are assigning the rating to.

These parameters should be transferred via the POST method

Comment Pull

Current Repo Comment URI is http://repo.openpandora.org/client/comment

To retrieve a list of comments for a specific app you can do this without the need to perform the handshake as no user details are required. 2 parameters will need to be POSTED to the appropriate server API URL:

• The "pull" (comment) parameter needs to contain the string "true".
• The "id" (application id) parameter should be the ID of the application we are assigning the rating to.

These parameters should be transferred via the POST method

The comments will be returned in the following format:

//Comment list file. (type: json-object)
//
//Will use the JSON common-denominator encoding (can be read as ASCII, ISO-8859-1 or UTF-8)
//Unicode characters are escaped with "\uXXXX" as per the JSON standard.
{
    //The package's queryable id. (type: json-string, see PXML standard)
    "id":"milkyhelper-1.2",
    //Versions array of all versions of app that have at least once comment for. (type: json-array)
    "versions":[
    {
        //Package version at the time of comment. (type: json-object)
        "version":{
            "major":"1",        //(type: json-string, with characters 0-9, a-z, A-Z, +, -)
            "minor":"6",        //(type: json-string, with characters 0-9, a-z, A-Z, +, -)
            "release":"1",      //(type: json-string, with characters 0-9, a-z, A-Z, +, -)
            "build":"1",        //(type: json-string, with characters 0-9, a-z, A-Z, +, -)
            "type":"release"    //(type: json-string, "alpha", "beta", or "release") OPTIONAL
        },
        //All Comments for this (version number above) version of the app. (type: json-array)
        "comments":[
        {
            "date":1333475154,          //The Unix time at which the PND was downloaded. (type: int)
            "username":"Cloudef",       //The username of the user that left the comment. (type: json-string)
            //Comment left by user. Maximum length should be 300 characters. (type: json-string)
            "comment":"New version of milkyhelper relased: http://repo.openpandora.org/?page=detail&app=milkyhelper This PND will be kept here for a while, it might be removed later. It's not supported anymore though."
        }
      ]
    }
  ]
}

Download logging / Purchased App Download

Current Repo Download URI is http://repo.openpandora.org/client/download

This is for if you would like your chosen Repository to keep a list of your downloaded applications (this is not a list of what's installed on your SD card but a list of all the applications you ever downloaded) and or would like to download a purchased application via your chosen client, you can also use this to download archived apps. After a successful handshake 2 parameters will need to be passed to the appropriate server API URL:

• The "id" (application id) parameter should be the ID of the application you wish to download.
• The "a" (anonymous) parameter should be sent with the string value of "false" if you wish to authorise download log or purchased download.
• The "t" (timestamp) parameter should be the time an app was archived as a unix time stamp this will allow you to get an older version of an application if it exists..

These parameters should be transferred in the URL via the GET method

History of Downloads

Current Repo Download History URI is http://repo.openpandora.org/client/history

If you would like to view all your logged downloads, after a successful handshake No parameters need to be sent.

The client will return the list of applications in this format.

//History file. (type: json-object)
//
//Will use the JSON common-denominator encoding (can be read as ASCII, ISO-8859-1 or UTF-8)
//Unicode characters are escaped with "\uXXXX" as per the JSON standard.
{
  //Information about the users download history. (type: json-object)
  "history":{
    //Username of user(type: json-string)
    "user":"milkshake",
    //Information about the PND packages downloaded. (type: json-array)
    "packages":[
    {
        //The package's queryable id. (type: json-string, see PXML standard)
        "id":"sd-install-2011-03-10",
        //OPTIONAL: The Package version at the time of download. (type: json-object)
        "version":{
            "major":"1",        //(type: json-string, with characters 0-9, a-z, A-Z, +, -)
            "minor":"6",        //(type: json-string, with characters 0-9, a-z, A-Z, +, -)
            "release":"1",      //(type: json-string, with characters 0-9, a-z, A-Z, +, -)
            "build":"1",        //(type: json-string, with characters 0-9, a-z, A-Z, +, -)
            "type":"release"    //(type: json-string, "alpha", "beta", or "release") OPTIONAL
        },
        //OPTIONAL: The Unix time at which the PND was downloaded. (type: int)
            "download_date":1335105932
        }
      ]
    }
}

List of archived applications

Current Repo Download History URI is http://repo.openpandora.org/client/archived

To retrieve a list of archived PND's for a specific app you can do this without the need to perform the handshake as no user details are required. 1 parameters will need to be POSTED to the appropriate server API URL:

• The "id" (application id) parameter should be the ID of the application from which you would like to see the archived versions.

These parameters should be transferred via the POST method

The comments will be returned in the following format:

//History file. (type: json-object)
//
//Will use the JSON common-denominator encoding (can be read as ASCII, ISO-8859-1 or UTF-8)
//Unicode characters are escaped with "\uXXXX" as per the JSON standard.
{
  //Information about archved applications. (type: json-object)
  "archived":{
    //The package's queryable id. (type: json-string, see PXML standard)
    "id":"bzar-panorama",
    //Versions of an archived application. (type: json-array)
    "versions":[
    {
        //The Package version of the archived application. (type: json-object)
       "version":{
            "major":"1",        //(type: json-string, with characters 0-9, a-z, A-Z, +, -)
            "minor":"6",        //(type: json-string, with characters 0-9, a-z, A-Z, +, -)
            "release":"1",      //(type: json-string, with characters 0-9, a-z, A-Z, +, -)
            "build":"1",        //(type: json-string, with characters 0-9, a-z, A-Z, +, -)
            "type":"release"    //(type: json-string, "alpha", "beta", or "release") OPTIONAL
        },
        //The Unix time at which the PND was downloaded. (type: int)
        "date":1313502307
     }
   ]
 }
}

Repository with delta transfers

Musing by me below, could use a cleanup. (/Demitar)

Fundamentally it's a binary http protocol for versioned files.

Url scheme is along the lines of /version returning the current version of the entire repo and which from which versions there exist deltas to the current version.

/list/<version> giving the full repo of <version>.

/delta/<version from>_<version to> giving the delta of the repo (a list of add, delete and update operations, rather basic stuff).

This describes a list of appid, versionid, appstring, versionstring tuples. Which are then constructed into urls for download of either full versions or deltas. (exactly what additional metadata is needed is still up there a bit)

The beauty of it is that except for /version every single url can be cached infinitely making on-demand http mirrors perfect.

Actual files are then retrieved through either /full/appid/versionid/appstring-versionstring.pnd (to ensure the download link looks good to users and clients alike).

Patches are when available available found in /patch/appid/versionid-from_versionid-to/appstring-versionstring-from_versionstring_to.pnd.patchalgorithm. (Current testing done with xdelta3.)

This is intentionally geared at single large files with seldom updated data being cheap, and partial updates being cheap.

Cheap:

$ du -sh full/*/*/frogatto*
138M	full/11/1/frogatto-20110206.pnd
138M	full/11/2/frogatto-20110208.pnd
$ du -sh patch/*/*/frogatto*
1.8M	patch/11/1_2/frogatto-20110206_20110208.pnd.xdelta3

What I keep considering is if there should be some knowledge of category/queues to partition the repo.

I have a prototype implementation running at http://frolic.xvx.se/ with an arm binary at http://frolic.xvx.se/static/frolic it needs to be run at the root of the sd card and creates three files in the current dir. Try, ./frolic list and ./frolic install <app>. The prototype is currently simplistic and doesn't use deltas yet even though the repository provides them.

I should move the sources to git soonish.

Binary protocol

The current basic rudimentary protocol is specified as follows:

all messages have the following header
uint32 : protocol version
uint32 : message size in bytes (excluding header)

/version
uint32 : repository version (0 is an empty repository, only ever increases - clients should however check if it equals not a comparative operator, just in case there's an unintentional repository reset)
sequence of (deltas exist from versions)
  uint32 : repository version

/static/<protocol version>/list/<repository version>
uint32 : repository version (as above, also this must match the requested version to guarantee cache integrity, failing if there's a newer version is fine)
sequence of
  uint32 : application id
  string : application string id
  sequence of
    version

/static/<protocol version>/delta/<from repository version>_<to repository version>
uint32 : from repository version
uint32 : to repository version
  sequence of
    uint32 : application id
    sequence of
      delta action

delta action is
  create application of
    string : application string id
  delete application (no payload)
  create version of
    version
  delete version of
    uint32 : application version

version is
  uint32 : application version
  string : application string version
  sequence of (can be patched from versions)
    uint32 : application version

sequence is
  uint32 : number of items
  followed by the specified payload

string is
  sequence of bytes (note, strings are _not_ null-terminated)

/full and /patch are the files, no more, no less.

/static/full/<application id>/<application version>/<application string id>-<application string version>.pnd

/static/patch/<application id>/<from application version>_<to application version>/<application string id>-<from application string version>_<to application string version>.pnd.xdelta3