Title: Building and Running Software on an Ubuntu Phone

Author: Martin Moene

Date: 03 October 2015 22:20:50 +01:00 or Sat, 03 October 2015 22:20:50 +01:00

Summary: Many of us has a smartphone. Alan Griffiths shows us how to hack an Ubuntu Touch phone.

Body: 

It has often been said that nowadays we carry around more computing power in our pocket than NASA used to put men on the moon. It is likely true but, so far, that computing power has not been readily available for general purpose computing. All that is changing. I’ve been working with Canonical on a project that is included in their ‘Ubuntu Touch’ phone operating system.

We’ve seen Linux on phones before (Google’s Android has been very successful) but one thing different about Ubuntu Touch is that it uses a large part of the same software stack as you’ll find on desktops and servers. Canonical are working on ‘convergence’ where the same software can provide either a phone or desktop experience depending on the connected hardware. Full ‘convergence’ isn’t here yet but you can already accomplish serious computing tasks like building and running software entirely on the phone.

This is quite a cool concept and I thought I should share the experience.

The first thing you need is a suitable phone. I’ve a growing collection owned by Canonical (plus one I own myself – I got sucked into the coolness of carrying a general purpose computer around). You can find a list of the supported hardware at https://wiki.ubuntu.com/Touch/Devices – there you’ll see that there are phones supported by Canonical as well as some community ports. The following works on Nexus 4 (mako), Nexus 10 (manta), BQ Aquaris E4.5 Ubuntu Edition (krillin) and Meizu MX4 Ubuntu Edition (arale) and I expect only small adjustments to be needed for other phones (like specifying an image server if the phone is not supported on the Canonical server).

I work on Ubuntu Desktop which means the desktop tools I mention below are readily available. On Ubuntu you can install these with:

  sudo apt-get install ubuntu-device-flash

On other systems you’ll have to work out the equivalent incantations. (They are in the packages android-tools-adb and ubuntu-device-flash.)

One final warning: if you simply want to develop Apps for the Ubuntu phone, this isn’t the best approach (and there is plenty of guidance at http://www.ubuntu.com/phone/developers). What follows will instead help you with setting up a general purpose development environment on the phone.

Installing an OS image

Depending on what you want to work on you can, of course, skip this section and use the production image that came with a commercial Ubuntu Touch phone. If you’re re-purposing a phone that has Android on it, or you want access to one of the pre-release versions, then you need to write or update the OS image. (I am working on parts of the software that runs the phone and fall into the latter category.)

Note that the tools and commands that follow can completely replace the existing OS and user data. Use backup appropriately!

In this section, and the following one, we assume there’s a USB connection to your phone and that the phone is in ‘developer mode’. Essentially, you need to go into Settings > About this phone > Developer mode. For security reasons connecting to the phone like this requires you to have a lock code set and for the phone to be unlocked with the screen on.

The channels providing OS images for various uses are listed here: https://developer.ubuntu.com/en/start/ubuntu-for-devices/image-channels/.

The tool used to manage installing the OS is ubuntu-device-flash. For example, to change the OS on a phone with an existing Ubuntu Touch installation:

  ubuntu-device-flash touch –channel=ubuntu-touch/devel/ubuntu

If you want a clean installation (or to overwrite Andrioid) you need to get the phone into recovery mode (typically, power on with both volume buttons pressed) and add –bootstrap to the above command. NB you will lose any user data.

The ubuntu-device-flash touch tool does have decent –help options. For example, ubuntu-device-flash query –help.

After installation, start the phone up, go through the setup and tutorial setting a security code and connecting to your wifi.

Getting ssh working

In this section, like the previous one, we assume there’s a USB connection to your phone and that the phone is in ‘developer mode’.

The instructions here are copied from http://askubuntu.com/questions/348714/how-can-i-access-my-ubuntu-phone-over-ssh for convenience. These enable ssh and push your public key to the default ‘phablet’ account.

adb shell android-gadget-service enable ssh
adb shell mkdir /home/phablet/.ssh
adb push ~/.ssh/id_rsa.pub /home/phablet/.ssh/authorized_keys
adb shell chown -R phablet.phablet /home/phablet/.ssh
adb shell chmod 700 /home/phablet/.ssh
adb shell chmod 600 /home/phablet/.ssh/authorized_keys

Next, assuming your phone is connected to your LAN find the phone’s IP address:

  adb shell ip addr show wlan0|grep inet

(You can also find the IP address through System Settings > Wi-Fi > ‘>’next to network name.)

From now on you don’t need the USB connection and can connect over your network:

  ssh phablet@<IP from above command>

Note that the ssh service isn’t started automatically so, if you restart the phone, it may need starting again. You don’t need a USB connection to do this, just type android-gadget-service enable ssh in the terminal app.

Creating a chroot to work in

There are a couple of reasons to do development work in a chroot. Firstly the system partition is almost full, so installing the tools you want would likely overfill it. Secondly, the system image isn’t managed using debian packages, but by downloading deltas. If you make changes to the system file system, things will break. Also, If you later re-flash the OS image while preserving user data you don’t lose the chroot.

The following is pulled together from email discussions and steals ideas from a number of developers. I especially mention Michal Sawicz who offered the original of the script in Listing 1 (he deserves all the credit for the good bits but no blame for the bad bits as I’ve tweaked it since).

I’ve just explained that using apt on the phone makes it incompatible with the OTA updates – but I’m going to take a shortcut and install debootstrap temporarily. I hope this returns the system to its original state – it hasn’t broken on me yet. It is definitely safe if you don’t allow OTA updates.

Now in the ssh session you started at the end of the last section:

 sudo mount -oremount,rw /
 sudo apt-get install debootstrap
 sudo mount -o remount,dev /home
 sudo debootstrap vivid /home/phablet/chroots/vivid
 sudo apt-get purge debootstrap
 sudo mount -o remount,ro /

For convenience, set up a matching user account in the chroot. Listing 1 has a useful “ch” script that makes setting up the and using the chroot simpler.

#!/bin/bash

usage() {
  [ -n "$2" ] && ( echo $2; echo )
  echo "Usage:"
  echo "  sudo $0 [OPTIONS] CHROOT_PATH"
  echo
  echo "  Options:"
  echo "    -h|--help  displays this help message"
  echo "    -r|--root  chroot as root, not current user"
  exit $1
}

ARGS=$(getopt -o r,h --long "root,help" -n "$0" -- "$@");

if [ $? -ne 0 ];
then
  usage 3
fi

eval set -- "$ARGS";

while true; do
  case "$1" in
    -h|--help)
      usage
      ;;
    -r|--root)
      shift;
      USERSPEC=0:0
      TARGET_UID=0
      ;;
    --)
      shift;
      break;
      ;;
  esac
done

setup_chroot() {
  # $1 is chroot path
  # $2 is user id

  # add current user in the chroot if doesn't exist
  chroot $1 id $2 &> /dev/null || useradd -R $1 -u $2 -m phablet

  # make the prompt hint when chrooted
  echo `id -un ${TARGET_UID:-$2}`@`basename $1` > $1/etc/debian_chroot

  mount -t proc none $1/proc
  mount -t sysfs none $1/sys
  mount --bind /dev $1/dev
  mount --bind /usr/lib/locale/ $1/usr/lib/locale/
  mount --bind $HOME $1/$HOME

  cleanup() {
    umount $1/proc $1/sys $1/dev $1/$HOME
      $1/usr/lib/locale/
  }
  trap "cleanup $1" EXIT
}

[ `id -u` == 0 ] || usage 2 "This script must be ran as root."
[ -d  "$1" ] || usage 1 "You need to pass a valid chroot path as argument."

CHROOT=`readlink -f $1` || exit 2
shift
BASE_UID=`id -u $( logname )`
BASE_GID=`id -g $( logname )`

setup_chroot $CHROOT $BASE_UID

chroot --userspec ${USERSPEC:-$BASE_UID:$BASE_GID} $CHROOT $@
			

To add the “phablet” user to sudoers in the chroot. In the ssh session:

  sudo ~/bin/ch --root chroots/vivid/

Now, you’re root in the chroot and type:

  sudo adduser phablet sudo
  passwd phablet

This set up the chroot “phablet” account to be able to use sudo. Now end the chroot root session and return to the ssh shell:

  ^D

Building some software

From the ssh session you get into the “phablet” account inside the chroot:

  sudo ~/bin/ch chroots/vivid/

You’ll use this command every time you want to switch into the chroot for development.

Phone development uses an ‘overlay ppa’ on top of vivid (a.k.a. 15.04) so you will want to add this to the chroot. This will continue into next year when it will switch to a 16.04 base. The following steps add the overlay:

  sudo apt-get install software-properties-common
  sudo add-apt-repository ppa:ci-train-ppa-service/stable-phone-overlay
  sudo apt-get update

Still as phablet within the chroot you should install your favourite build tools. Here are the ones I use for Mir:

  sudo apt-get install cmake g++ make bzr

Now, one ought to be able to set up building a project as normal. I use the Mir project on which I work as an example but you should vary as appropriate for your own needs. The relevant instructions for Mir are at http://unity.ubuntu.com/mir/building_source_for_pc.html. Sill as phablet in the chroot:

  cd
  bzr branch lp:mir mir
  cd mir/

Now the next step tells you to <CodeComment>sudo apt-get install devscripts equivs cmake</CodeComment> but equivs isn’t available in the phone archive so the incantation doesn’t work. So instead of that command (and the one that follows), use what is in Listing 2.

sudo apt-get install cmake-data pkg-config debhelper doxygen xsltproc graphviz libboost-dev libboost-date-time-dev libboost-program-options-dev libboost-system-dev libboost-filesystem-dev protobuf-compiler libdrm-dev libegl1-mesa-dev libgles2-mesa-dev libgbm-dev libglm-dev libprotobuf-dev pkg-config android-headers libhardware-dev libandroid-properties-dev libgoogle-glog-dev liblttng-ust-dev libxkbcommon-dev libumockdev-dev umockdev libudev-dev libgtest-dev google-mock valgrind libglib2.0-dev libfreetype6-dev libevdev-dev uuid-dev python3 dh-python glmark2-es2-mir
			

Then resume the steps to build the project:

  mkdir build
  cd build
  cmake ..
  make -j2 all ptest

Depending on the phone you're using you may be able to start more processes (e.g. -j4), but at this stage you should have built Mir and run the basic test suites. A few of the tests may fail as a side effect of running in the chroot: don’t worry about them. You can also install Mir in the chroot as normal:

  sudo make install

Then exit the chroot for the top-level ssh shell:

  ^D

Running the software

Of course, installing software in a chroot is pretty limited (see the test failures mentioned above) but the following instructions for running the Mir “performance test” is an example of what you can do to work around this.

The performance test uses glmark and that is hard coded to find its data at /usr/share/glmark2. So that can work we once again make the file system writeable. I think adding an empty directory that can used for a “mount --bind” is unlikely to cause problems (but it is a risk you may wish to avoid by removing it after use). Once again from the top-level ssh shell:

  sudo mount -oremount,rw /
  sudo mkdir /usr/share/glmark2
  sudo mount -o remount,ro /
  sudo mount --bind ~/chroots/vivid/usr/share/  glmark2//usr/share/glmark2

Now, stop the graphics stack running the normal phone interface so that Mir can have sole access:

  sudo stop lightdm

Now run the “performance test”:

  export MIR_CLIENT_PLATFORM_PATH=~/chroots/vivid/usr/local/lib/mir/client-platform/
  export MIR_SERVER_PLATFORM_PATH=~/chroots/vivid/usr/local/lib/mir/server-platform/
  export PATH=~/chroots/vivid/usr/bin:~/chroots/vivid/usr/local/bin:$PATH
  export LD_LIBRARY_PATH=~/chroots/vivid/usr/local/lib/
  mirbacklight
  mir_performance_tests

Finally, start the graphics stack running the normal phone interface again.

  sudo start lightdm

Conclusion

The above example shows how to go about working on Mir. Like most open source projects, that’s entirely possible for you to do. If you’re not interested in doing that I hope you will use these notes as a starting point for your own ideas. The possibilities will become ever more interesting as the Ubuntu Touch platform matures.

Acknowledgement

Thanks to Kevin Gunn, Alexandros Frantzis and the Overload team for their feedback, which improved the article.

Notes: 

More fields may be available via dynamicdata ..