Creating Debian packages and a repository for Python apps for the N800

I've created some Python scripts for my N800, and the toughest part has turned out to be packaging them so other people can easily install them.

So I created bdist_debian.py which adds Debian Package support to Python Distutils. Note: I've discovered this needs Python 2.5.1. Read the changelog.txt for the latest information.

The Debian New Maintainers' Guide is an absolute nightmare of bureaucratic complexity. They take a really nice packaging system and make it impossible to use. I used to wonder why Debian was always behind the times with package versions.

I'm lazy as hell and I resent having to do bookkeeping FOR the computer like the Debian Guide wants. I already HAVE a changelog! And I don't want to deal with stupid directory names that MUST have the current version tacked on! I already have EMACS automatically updating the version whenever I update the file.

It would be easier if the Debian changelog format was documented anywhere, but it's not, and every time I touched it, the Debian tools refused to ever use it again.

Advantages of a repository

There are two main advantages of using a true repository and click-to-install links over links to plain .deb files. The first is that Application Manager's "Check for updates" button works, which is very important if you're releasing enhancements and bugfixes often. The second is that dependencies can be resolved. If package A needs packages B & C, then Application Manager knows where to get them, because they're in your repository.

Modifying dpkg-scanpackages

If you're not running Debian, you'll need to install the dpkg software, so you can create "Packages.gz" files. Get the dpkg source tarball from the dpkg source page and install it. You only need the "dpkg-scanpackages" PERL script, and fortunately it doesn't depend on anything else, so it can run standalone. You do need to modify it to support the "Maemo-Icon-26" tag by changing the "@fieldpri" definition as follows:

my @fieldpri = (qw(Package Package-Type Source Version Kernel-Version
                   Architecture Subarchitecture Essential Origin Bugs
                   Maintainer Installed-Size Installer-Menu-Item),
                @pkg_dep_fields, qw(Filename Size MD5sum Section Priority
                   Homepage Description Tag Maemo-Icon-26));

Setting up a repository on your personal website

Create a directory named "debs". This is where you'll put all your ".install" and ".deb" files. The .deb files are the actual packages, and the .install files tell Application Manager where to find them.

You'll also need to create the directory path "debs/dists/bora/free/binary-armel" which is where your "Packages.gz" file will go.

The .install files need to be served with a "application/x-install-instructions" MIME type. Unfortunately Opera on the N800 neither recognizes the "type" attribute for links, nor does it recognize the ".install" extension properly. This means you've got to tell your web server to serve the files with the proper type. The procedure to do this is dependent on your ISP, the type of server it's running, and the permissions they've configured. If your ISP is running an Apache server, you can usually do this by creating a file in "debs" named ".htaccess" containing the following line:

AddType application/x-install-instructions .install

Creating "Click-To-Install" links for your apps

The example .install file is named "spend-it.install" and looks like:

The red text is the name of your repository, and can be whatever you wish. The green text is the URL for your "debs" directory. The blue text is the base name of the .deb package. "bora" is the name of the Nokia OS release compatible with this package, and "free" is because your package is free open-source. Everything else should be copied as-is.

The HTML code for the link to the .install file is fairly simple:

This will display the green arrow as a link the user can click on to install your application. My image has alpha transparency so you don't have to have a white background on your page, and you can fetch it right off my page, if you wish.

The postinst, preinst, postrm, prerm scripts

As per chapter 6 of the Debian Policy Manual, you can have scripts execute at various times in the installation, upgrade, or removal of your package. These scripts are named config, preinst, postinst, prerm and/or postrm. If any of these exist in your source directory, then bdist_debian will add them to the package. Take a look at the Making application packages HOWTO for details on utilities that you can use in your scripts.

Icon image

The "icon" keyword specifies the 26x26 PNG file to be used for the Application Manager icon. It's automatically Base64 encoded and inserted into the "control" file.

Creating the package

Use bdist_debian from my main project page. You place your application files in a directory and write a "setup.py" file as per the instructions in the Python Distutils reference. Here's the file for the nokia-sync package:

import bdist_debian
from distutils.core import setup

setup(name='nokia-sync',
      scripts=['nokia_sync', 'nokia_sync.py'],
      version='2007.11.16-2',
      section='user/backups',
      maintainer='Gene Cash',
      maintainer_email='Gene Cash <gene.cash@gmail.com>',
      depends='python2.5, python2.5-hildon, python2.5-gtk2, nokia-utilities',
      description="Wireless backup application",
      long_description="Quickly back up your data to a server over wireless.",
      data_files=[('share/applications/hildon', ['nokia_sync.desktop']),
                  ('share/pixmaps', ['nokia_sync.png']),
                  ('share/dbus-1/services', ['nokia_sync.service'])],
      icon='nokia_sync.png',
      cmdclass={'bdist_debian': bdist_debian.bdist_debian})

Also note that if the "section" doesn't start with "user/" then Application Manager won't show it. After you create this file, you do "python setup.py bdist_debian" and it creates the .deb file in the "dist" subdirectory. That's it! You're done!

A note about permissions: The install process runs as root, so all files/directories are created by root, and directories do not have write permissions! So don't create data directories and then expect your program to be able to write to them later. If your program needs data directories, it should create them, not the setup.py file.

Converting an existing package

This is fairly simple. I needed to do this for the "python-dateutil" and "iCalendar" modules used by my calendar application. Since you're interfacing with the standard Distutils system, the package author will have already written a "setup.py" file for you. In this file, you need to add an import for bdist_debian at the beginning, then add section, maintainer, maintainer_email, depends, and icon keywords to the setup() call. You also have to add "cmdclass={'bdist_debian': bdist_debian.bdist_debian}" to the setup() call as well.

In this case I had to clean up the "long_description" value so it didn't have a large run of embedded spaces, and to keep line length under 65 characters. I created the python.png image from the Python logo, so that Application Manager would have something to display for a plain Python module.

This example "setup.py" file from the iCalendar module:

#!/usr/bin/env python

try:
    from setuptools import setup
except ImportError:
    from distutils.core import setup

version = file('version.txt').read().strip()

setup(name='icalendar',
      package_dir={'': 'src'},
      packages=['icalendar'],
      version=version,

      # metadata for upload to PyPI
      author='MaxM',
      author_email='maxm@mxm.dk',
      description='iCalendar parser/generator',
      license='GPL2.1',
      keywords='calendar icalendar',
      url='http://codespeak.net/icalendar/',
      long_description="""iCalendar is a parser/generator of iCalendar files
          (RFC 2445) for use with Python.""",
      classifiers=['Development Status :: 5 - Production/Stable',
                   'Intended Audience :: Developers',
                   'License :: OSI Approved :: GNU General Public License (GPL)',
                   'Operating System :: OS Independent'],
      platforms='All',
      )

then becomes:

#!/usr/bin/env python

import bdist_debian

try:
    from setuptools import setup
except ImportError:
    from distutils.core import setup

version = file('version.txt').read().strip()

setup(name='icalendar',
      package_dir={'': 'src'},
      packages=['icalendar'],
      version=version,

      # metadata for upload to PyPI
      author='MaxM',
      author_email='maxm@mxm.dk',
      description='iCalendar parser/generator',
      license='GPL2.1',
      keywords='calendar icalendar',
      url='http://codespeak.net/icalendar/',
      long_description="iCalendar is a parser/generator of iCalendar files (RFC 2445) for use\nwith Python.",
      classifiers=['Development Status :: 5 - Production/Stable',
                   'Intended Audience :: Developers',
                   'License :: OSI Approved :: GNU General Public License (GPL)',
                   'Operating System :: OS Independent'],
      platforms='All',
      section='user/libraries',
      maintainer='Gene Cash',
      maintainer_email='gene.cash@gmail.com',
      depends='python2.5',
      icon='python.png',
      cmdclass={'bdist_debian': bdist_debian.bdist_debian})

Releasing your app

In addition to placing the .install file and .deb file in the "debs" directory, you need to update the repository information. Go to the "debs" directory and do:

Releasing updates

Change the version number in your setup.py file, generate a new .deb, copy that to the web server's "debs" directory, and recreate the "Packages.gz" file. That's a whole lot easier than the procedure from the Debian Guide.