The Linux Binary Distribution Disaster
Linus Torvalds: Provides MacX and Windows binaries for his divelog application
but no Linux binaries, why?
There are no Linux binaries: there are Fedora binaries, Ubuntu
binaries, ArchLinux binaries, etc..
Linux binaries differ..
- ..in package formats (deb, rpm, tgz, ...)
- ..in build rules (compiler flags, fencing, optimization, ...)
- Linux Standard Base (LSB): should actually guarantee ABI and API stability..
- The problem is, there is still too much variation..
Shared libraries
- share common code on a platform, good idea
- there is not a common view of what can be shared and should be shared
- packet names and versions differ
- sometimes compilation flags differ (e.g. no threading support in sqlite3 on RHEL5)
Distribution types
- Frozen releases, e.g. Debian 8, RHEL 7, Ubuntu LTS, ...,
long release cycles (4-8 years), they are stable,
but only if you use very old software
- Rolling releases, e.g. Fedora, ArchLinux, Gentoo, ...
You have the newest and shiniest software, but
stability may suffer, backporting to other distros
gets almost impossible.
Options for software developers
Islands, part 1
- Create your own universe: examples Java, Go, Python, ...
- You deploy your software as a blob of static libraries,
there might be sharing inside your universe but not between
universes.
- In C++ we are "stuck" to the ELF format, shared libarires, binaries
in distribution packages. Build on one Linux (which one?) and hope
it runs everywhere
- Local libraries must be maintained (security patches), imagine
a bundled, unpatched, 5 years old libssl. :-)
Package meta-systems
Islands, part 2
- http://www.openpkg.org/: cross-platform RPMs
- BSD-like port systems (gentoo, netsrc, etc.): but this means going back to distribution of source code only
Package build systems
Islands, part 3
Package build systems
Islands, part 3
- Linux distros try to maintain their bleeding edge product with the help of the community.
- Naturally communities form islands
- Exception: OpenBuild system, they build SuSE, RHEL, Debian, Ubuntu, Mandriva, Arch,..
- This "behaviour" should be honoured and supported as it is the really only way to go right now
OBS
- Basically having one RPM SPEC-file and one Debian/Ubuntu tree covers the majority of Linux distributions
- Choose your dependencies wisely and provide what's not available on old distros on your own.
- Either bundle your stub packages with your software package or package them into a 3rdParty meta package
- Make sure package names DON'T collide!
Automatization
- Combine it with github, build the source trees there (git tag)
- Webhooks draw the tarballs and spec files to the OBS cluster
- Publish releases on a web space
- Keep the current git master as rolling last release of your software
Current process
git commit -a -m 'changed to release 0.2.1'
git tag 0.2.1
Current process
cmake .
dist/obs/deploy_to_obs.sh
Remove source tarballs of old release, add tarballs of new release:
cd home:PatrickFrey/strus
osc up
osc rm strus-0.1.6*
osc add strus-0.2.1*
osc commit -m 'added version 0.2.1'
Issues
- dist/obs/deploy_to_obs.sh should be eliminated
- Release versions need patching, the mirroring system plays mean tricks otherwise..
- How to automatically update the OSC workspace, not having tarballs there, etc.
- Have a central changelogs, create Debian and RHEL style from it
- git commit is always needed, otherwise tarball lacks modified/added artifacts
- Don't change git tags later on! Rather revoke broken releases.
Local OSB builds
sudo chroot /var/tmp/build-root/xUbuntu_12.04-x86_64/ /bin/bash --login
cat /etc/debian_version
wheezy/sid
Local OSB builds
- The artifacts of the build system are somewhat hidden in the chroot:
/usr/src/packages/BUILD/
/home/abuild/rpmbuild/BUILD/strus-0.2.1
/usr/src/packages/BUILD/strus-0.2.1
The chrooted environment lacks things like an editor or a debugger or
even a package manager inside the chroot.
Either add a dependency for them 'BuildRequires:' or use
the package manager from outside the chroot (works nice on Archlinux
and for Ubuntu on Ubuntu, 'apt-get' on RHEL might be a problem).