Way to AOSC OS Maintainer: Basics

This article has been officially published in the AOSC Wiki. The following information will not receive any update.

So you want to make a package, you got the urge to make a package, you got the nerve to make a package, so go ahead, so go ahead, so go ahead and make a package we can use!

NOTICE: This guide assumes you have moderate knowledge about Linux and its CLI (command line interface). Also, you need to have access to a Linux computer with root access.

Introduction

We will need these tools in order to build packages. Don't worry about them for now, we will investigate them later.

Ciel
- Manages systemd-nspawn(1) containers.
ACBS
- Manages a tree (like our main tree, https://github.com/AOSC-Dev/aosc-os-abbs) of package build specs.
- Calls Autobuild3 to (actually) read and build the package(s) as specified.
Autobuild3
- Reads package specifications and run the build scripts.
pushpkg
- Pushes built packages to the official repository.

Release model

AOSC OS is maintained with a semi-rolling release model which cycles every three months (or so). This means that there's no version number attached to a full AOSC OS release (similar to rolling release distros like Arch Linux). However, within the aosc-os-abbs tree, there is a set of packages that constructs the AOSC OS Core, which consists of core runtime (the GNU C Library, etc.) and toolchains (GCC, etc.). This set of packages are updated in a versioned fashion (Core 7.0.1, 7.0.2, 7.1.1, etc.). Additionally, all updates to the AOSC OS repository undergo a period of testing in what is called a *-proposed repository.

There are two main update branches: stable and testing; and three development branches: stable-proposed, testing-proposed and explosive.

stable-proposed is always open for updates, but only patch releases (x.y.z where z is updated), security updates, bugfixes, and various exceptional updates are allowed into this repository. This branch merges with stable weekly.

testing-proposed is the place where new packages and major updates are introduced. This is where most of the work takes place. The developemnt on this branch follows a three-month iteration schedule (take for example the Winter 2020 Iteration Plan). During the first two months, developers build and test new updates or introduce new packages to the testing-proposed branch.

At the beginning of the last month, testing-proposed gets merged into testing. During this month, users who enables testing repository will receive the updated packages and may help testing them. If everything goes well, at the end of the month, testing will be merged into stable and thus completes the cycle. During this time, the testing-proposed branch is effectively frozen.

explosive is meant to be a "playground," or a place where packages and updates not meant for the current cycle are committed to. During the time when testing-proposed is frozen, developers may push updates ahead of time into this branch, as explosive merges with testing-proposed at the beginning of a new cycle.

Setting up the environment

The first thing is to install ciel on the computer. On AOSC OS, just install ciel from the official repository.

Since Ciel manages standardised AOSC OS build environment (or the BuildKit), the build process does not have to happen on an AOSC OS machine. If you are using Arch Linux, you can install Ciel from AUR.

Next, we will initialise a Ciel workspace. ~/ciel is used as a sample path for demonstration. Notice that Ciel will need to be run as root.

1
2
3


mkdir ~/ciel
cd ~/ciel
ciel init

Now, we can deploy the BuildKit. BuildKit is a minimal AOSC OS variant used specifically for packaging or containerised development. It contains ACBS and Autobuild3, so no additional configuration is required.

1
2
3


ciel load-os
# Or if you have already downloaded BuildKit
ciel load-os PATH_TO_BUILDKIT

It is always a good idea to keep the BuildKit environment up-to-date (and this serves as a requirement for AOSC OS packagers).

1

ciel update-os

The next step is to load an ACBS tree. For this instance, we will work on the official aosc-os-acbs tree.

1
2


ciel load-tree # By default, ciel will load the official tree.
# Or, you can just clone the desired repository to ciel/TREE

Building our very first package!

Now that we have a build environment set-up, we can try to build a package that is already in the tree. Let's start with a relatively trivial one, extra-multimedia/flac.

Before that, we need to create a Ciel instance. It is recommended to use separate instances for different branches. Run:

1

ciel add stable # Since we are going to build on stable

And make sure we are actually on the stable branch.

1
2


cd TREE
git checkout stable

Then, we need to configure Ciel to use the correct repositories. In order to prevent incorrect dependencies, the build environment should use packages that matches the branch (with the exception of stable-proposed, which will only use dependencies from stable). For example, we need stable repository to build stable tree, and testing, stable-proposed, and stable to build testing packages.

1

ciel config -i stable

First enter your info, whether to enable DNSSEC. And when ciel ask if you want to edit source.list, say yes, and modify.

1
2
3
4
5
6
7
8
9


# For building stable packages
deb https://repo.aosc.io/debs stable main

# For building testing packages
deb https://repo.aosc.io/debs testing main
deb https://repo.aosc.io/debs stable-proposed main
deb https://repo.aosc.io/debs stable main

# And you get the idea.

Now we can actually build the package! Simply type:

1
2


ciel build -i stable flac
# -i is used to select the instance used to build

If the build completes without error, and a Build Summary is present, congratulations on your first successful build! You should be able to find the generated deb inside OUTPUT/debs.

Adding a new package

But surely you won't be satisfied by simply building existing packages, right? Here we will discover how to construct a new package from scratch.

Dive into the TREE folder, you will find a lot of categories of folders, including some beginning with base- and core- prefixes, as well as some with extra-. These folders are for organizing purposes, and inside them you will find the various packages (and their build specifications) organised in each of their own directory.

We will use i3 as an example. This package can be found at TREE/extra-wm/i3 for obvious reasons. Upon entering the directory, you should see a file structure as follows:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16


.
├── autobuild
│   ├── beyond
│   ├── conffiles
│   ├── defines
│   ├── overrides
│   │   └── usr
│   │       ├── bin
│   │       │   └── i3exit
│   │       └── share
│   │           └── pixmaps
│   │               └── i3-logo.svg
│   ├── patches
│   │   └── 0001-Use-OVER-operator-for-drawing-text.patch
│   └── prepare
└── spec

We will go through which each file is for.

`spec`

This file is responsible for telling acbs where to download the source file, and the package's version and revision. A basic spec file should look like this:

1
2
3
4


VER=4.17.1  # Version of the software.
# REL=0 The package revision. If not specified, it's 0.
SRCTBL="https://i3wm.org/downloads/i3-$VER.tar.bz2" # Download address for the source code.
CHKSUM="sha256::1e8fe133a195c29a8e2aa3b1c56e5bc77e7f5534f2dd92e09faabe2ca2d85f45" # Checksum of the source tarball.

One thing worth noting is the revision number. You can ignore this line if you are creating a new package, but sometimes (like applying an emergency security patch), the version number is not changed, but we still need to inform the package manager on users computer that there is an update available. In these circumstances, just increase the $REL variable by 1.

`autobuild/`

This is the directory where all the Autobuild3 scripts and definitions live. Autobuild3 is a sophisticated build system that can automatically determine a series of build-time processes, like which build system to use, which build parameter to use, and so on.

`autobuild/defines`

This file contains the core configuration like:

PKGNAME : Package name.
PKGDES : Package description.
PKGSEC : Section (or category) where the package belongs to.
PKGDEP : Package dependencies.
PKGCONFL : Package conflicts.
BUILDDEP : Build dependencies (packages which are required during build-time, but not for run-time).
PKGRECOM : Recommended dependencies, installed automatically, but could be removed by user discretion.

These are only the most common configuration entries. There are much more configurations, but if the software is fairly standard, these configuration should be enough. Other information like which C compiler flags to use, which build system to use, can be filled automatically by Autobuild3.

Here is a basic example taken from TREE/extra-multimedia/i3:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10


PKGNAME=i3
PKGSEC=x11
PKGDEP="dmenu libev libxkbcommon pango perl-anyevent-i3 perl-json-xs \
        startup-notification xcb-util-cursor xcb-util-keysyms \
        xcb-util-wm yajl xcb-util-xrm"
PKGRECOM="i3lock i3status"
BUILDDEP="graphviz doxygen xmlto"
PKGDES="Improved tiling WM (window manager)"

PKGCONFL="i3-gaps"

Notice here that you can actually write bash logic inside defines. This is useful when adding platform-specific flags or dependencies, but this is NO LONGER recommended, and will be prohibited in the future. For adding platform specific info, use $VAR__$ARCH.

For a complete list of available parameters, visit Wiki for Autobuild3.

`autobuild/prepare`

This file is the script that will be executed before the build process begins. Usually it is used to prepare files or set environment variables used in the build process.

`autobuild/patches/`

This is a directory containing all the patches that will be applied to the source codes before the build.

Simple as dropping it in. :)

A complete example: `light`

That's all the basic knowledge you need to build a simple package! Now, we will try to build a really simple program: light.

This program is used to provide a easy command to control the backlight of laptop. Since it only uses file API to interact with the backlight subsystem, this program is very simple and does not require and dependency other than glibc.

Return to the TREE directory (assuming you have Ciel set-up). First, make sure that you are on the right branch. As mentioned above, during the first two months of the cycle, use testing-proposed. For the last month, use explosive.

Since this program is obviously a utility, we create a directory called light under the directory TREE/extra-utils.

1
2
3


cd TREE/extra-utils
mkdir light
cd light

Then, we create the spec file. Look up the project website and find out the download URL for the latest version. After manually checking the sha256 checksum of the latest tarball, we can fill in the file.

1
2
3


VER=1.2.1
SRCTBL="https://github.com/haikarainen/light/archive/v$VER.tar.gz"
CHKSUM="sha256::53d1e74f38813de2068e26a28dc7054aab66d1adfedb8d9200f73a57c73e7293"

Notice here that we replaced the version number inside the tarball URL with an environment variable $VAR. This is considered as a good practice (since it reduces the modification required when updating the package), and should be used when possible.

Then, we create the autobuild folder, and create the defines file.

Since this is an application used in the GUI environment, we give it the section of x11. The complete defines file looks like the following:

1
2
3


PKGNAME=light
PKGSEC=x11
PKGDES="Program to easily change brightness on backlight-controllers."

And we are done! We can now head back to the base directory of the Ciel environment (~/ciel, and run the following command:

1

ciel build -i stable light

Although we didn't write anything about how to build this program, Autobuild3 automatically figured out that this should be built with autotools (i.e., the classic ./configure && make && make install logic), and should build the program successfully. If you want to double check, use dpkg-deb -c DEB_FILE to check the files inside the deb file.

Git conventions

AOSC OS has strict conventions about git logs. We will only mention the most used ones here. For the full list of package styling and development guidelines, please refer to the https://wiki.aosc.io/developers/aosc-os-package-styling-manual.

For example, we are adding a new package to the tree. Then the log should be something like this:

1
2


light: new, 1.2.1
$PKG_NAME: new, $VER

If you are updating the version of an exisiting package, it should be like this:

1
2


bash: update to 5.2
$PKG_NAME: update to $NEW_VER

And please mention all the specific changes made to the package (i.e., dependency changes, feature enablement, etc.) in the long log, for instance:

1
2
3
4
5


bash: update to 5.2

- Make a symbolic link from /bin/bash to /bin/sh for program compatibility.
- Install HTML documentations.
- Build with -O3 optimisation.

Pushing packages to the repository

After a successful build, maintainers will push local Git changes to the tree, and the respective packages to the official repository.

The second task can be done using pushpkg. Grab the script, add the script to PATH, make sure it is executable (0755). Then, invoke pushpkg inside the OUTPUT directory. You will need to provide your LDAP credentials and the destination repository (stable, testing, etc.).

Epilogue

That's it! You have learned the basics about creating new packages for AOSC OS from scratch, as well as how to update, build, and uploading them!

However, as you may see, this article only covers the basics of what you need to know as you continue to prime for further involement in AOSC OS maintenance. When dealing with more complicated build systems, or updating a batch of packages, there's still many skills to learn. Please refer to the Way to AOSC OS Maintainer: Advanced Techniques.