From 565bef936ded714b81bf7041558eaebcd0a78af8 Mon Sep 17 00:00:00 2001 From: Rob Landley Date: Fri, 15 May 2020 03:09:35 -0500 Subject: How to cross compile with 2 compilers and 3 libc. --- www/faq.html | 99 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 99 insertions(+) (limited to 'www/faq.html') diff --git a/www/faq.html b/www/faq.html index 23f5d818..5e054f02 100755 --- a/www/faq.html +++ b/www/faq.html @@ -16,6 +16,8 @@

Using toybox

@@ -231,6 +233,103 @@ instead of the toybox version (in theory they should work the same), and VERBOSE=1 to see diffs of the expected and actual output when a test fails (VERBOSE=fail to stop at the first such failure)

+ +

Q: How do I cross compile toybox?

+ +

A: toybox is tested against three C libraries (bionic, musl, glibc) +with 2 compilers (llvm, gcc). The easy way to get coverage (if not every +combination) is:

+ +

1) gcc+glibc = host toolchain

+ +

Most Linux distros come with that as a host compiler, just build normally +(make distclean defconfig toybox, or make menuconfig followed +by make).

+ +

You can use LDFLAGS=--static if you want static binaries, but static +glibc is hugely inefficient ("hello world" is 810k on x86-64) and throws a +zillion linker warnings because one of its previous maintainers +was insane +(which meant at the time he refused to fix +obvious bugs), plus it uses dlopen() at runtime to implement basic things like +DNS lookup (which is impossible +to support properly from a static binary because you wind up with two +instances of malloc() managing two heaps which corrupt as soon as a malloc() +from one is free()d into the other), although glibc added +improper support which still requires the shared libraries to be +installed on the system alongside the static binary: +in brief, avoid.) +These days glibc is maintained by a committee instead of a single +maintainer, if you consider that an improvement.

+ +

2) gcc+musl = musl-cross-make + +

The cross compilers I test this with are built from the +musl-libc maintainer's +musl-cross-make +project, built by running toybox's scripts/mcm-buildall.sh in that directory, +and then symlink the resulting "ccc" subdirectory into toybox where +"make root CROSS=" can find them, ala:

+ +
+cd ~
+git clone https://github.com/landley/toybox
+git clone https://github.com/richfelker/musl-cross-make
+cd musl-cross-make
+../toybox/scripts/mcm-buildall.sh # this takes a while
+ln -s $(realpath ccc) ../toybox/ccc
+
+ +

Instead of symlinking ccc, you can specify a CROSS_COMPILE= prefix +in the same format the Linux kernel build uses. You can either provide a +full path in the CROSS_COMPILE string, or add the appropriate bin directory +to your $PATH. I.E:

+ +
+

make LDFLAGS=--static CROSS_COMPILE=~/musl-cross-make/ccc/m68k-linux-musl-cross/bin/m68k-linux-musl- distclean defconfig toybox

+
+ +

Is equivalent to:

+ +

+export "PATH=~/musl-cross-make/ccc/m68k-linux-musl-cross/bin:$PATH"
+LDFLAGS=--static make distclean defconfig toybox CROSS=m68k-linux-musl- +

+ +

Note: a non-static build won't run unless you install musl on your host. +In theory you could "make root" a dynamic root filesystem with musl by copying +the shared libraries out of the toolchain, but I haven't bothered implementing +that yet because a static linked musl hello world is 10k on x86 +(5k if stripped).

+ +

3) llvm+bionic = Android NDK

+ +

The Android +Native Development Kit provides an llvm toolchain with the bionic +libc used by Android. To turn it into something toybox can use, you +just have to add an appropriately prefixed "cc" symlink to the other +prefixed tools, ala:

+ +
+unzip android-ndk-r21b-linux-x86_64.zip
+cd android-ndk-21b/toolchains/llvm/prebuilt/linux-x86_64/bin
+ln -s x86_64-linux-android29-clang x86_64-linux-android-cc
+PATH="$PWD:$PATH"
+cd ~/toybox
+make distclean
+make LDFLAGS=--static CROSS_COMPILE=x86_64-linux-android- defconfig toybox
+
+ +

Again, you need a static link unless you want to install bionic on your +host. Binaries statically linked against bionic are almost as big as with +glibc, but at least it doesn't have the dlopen() issues.

+ +

Unfortunately, although the resulting toybox will run a bionic-based +chroot will not, because even "hello world" statically linked +against bionic will segfault before calling main() if /dev/null isn't +present, and the init script written by mkroot.sh has to run a shell linked +against bionic in order to mount devtmpfs on /dev to provide /dev/null.

+

Q: How do you build a working Linux system with toybox?

-- cgit v1.2.3