diff options
| -rwxr-xr-x | www/about.html | 173 | ||||
| -rw-r--r-- | www/header.html | 12 | ||||
| -rwxr-xr-x | www/roadmap.html | 73 |
3 files changed, 165 insertions, 93 deletions
diff --git a/www/about.html b/www/about.html index 61c57a65..9e55666a 100755 --- a/www/about.html +++ b/www/about.html @@ -3,74 +3,114 @@ <h2><a name="what" />What is toybox?</h2> -<p>Toybox combines the most common Linux command line utilities together into +<p>Toybox combines many common Linux command line utilities together into a single <a href=license.html>BSD-licensed</a> executable. It's simple, small, fast, and reasonably standards-compliant (<a href=http://opengroup.org/onlinepubs/9699919799>POSIX-2008</a> and <a href=http://refspecs.linuxfoundation.org/LSB_4.1.0>LSB 4.1</a>).</p> -<p>Toybox's 1.0 release goal is to turn generic Android into a -development environment capable of compiling <a href=http://www.linuxfromscratch.org>Linux From Scratch</a>. -A tiny system <a href=/aboriginal>built from</a> just toybox, linux, <a href=http://musl-libc.org>a C library</a>, and a C compiler (such as LLVM or -gcc 4.2.1+binutils 2.17) should be -able to rebuild itself from source code without needing any other packages.</p> +<p>Toybox's main goal is to make Android +<a href=http://landley.net/aboriginal/about.html#selfhost>self-hosting</a> +by improving Android's command line utilities so it can +build an installable Android Open Source Project image +entirely from source under a stock Android system. After a talk at the 2013 +Embedded Linux Conference explaining this plan +(<a href=http://landley.net/talks/celf-2013.txt>outline</a>, +<a href=https://www.youtube.com/watch?v=SGmtP5Lg_t0>video</a>), Google +<a href=https://lwn.net/Articles/629362/>merged toybox into AOSP</a> and +began shipping toybox in Android Mashmallow.</p> + +<p>Toybox aims to provide one quarter of a theoretical "minimal native +development environment", which is the simplest Linux system capable of +rebuilding itself from source code and then building +<a href=http://linuxfromscratch.org/lfs>Linux From Scratch</a> +and the <a href=https://source.android.com>Android Open Source Project</a> +under the result. In theory, this should only require four packages: +1) a set of posix-ish command line utilities, +2) a compiler<a name="1_back"></a><sup><font size=-3><a href=#1>[1]</a></font></sup>, +3) a C library, and 4) a kernel. This provides a reproducible and auditable +base system, which with the addition of a few convienciences (vi, top, +shell command line history...) can provide a usable interactive experience +rather than just a headless build server.</p> -<b><h2><a name="status" />What commands are implemented in toybox?</h2></b> +<b><h2><a name="why" />Why is toybox?</h2></b> + +<p>The <a href=http://landley.net/talks/celf-2015.txt>2015 toybox talk</a> +starts with links to three previous talks on the history and motivation of +the project: "Why Toybox", "Why Public Domain", and "Why did I do +Aboriginal Linux (which led me here)?". If you're really bored, +there's even a half-finished +<a href=http://landley.net/aboriginal/history.html>a history page</a>.</p> + +<p>The toybox maintainer's earlier minimal self-hosting system project, +<a href=http://landley.net/aboriginal/about.html>Aboriginal Linux</a>, +got its minimal native development environment down to seven packages in +its 1.0 release (busybox, uClibc, gcc, binutils, make, bash, and linux) +and built Linux From Scratch under the result. That project +<a href=http://landley.net/aboriginal/history.html>was the reason</a> +toybox's maintainer became busybox maintainer, having done so +much work to extend busybox to replace all the gnu tools in a Linux From +Scratch build that the previous maintainer handed over the project (to +spend more time on buildroot).</p> + +<p>Despite the maintainer's history with busybox, toybox is a fresh +from-scratch implementation under an +<a href=https://source.android.com/source/licenses.html>android-compatible</a> +<a href=license.html>license</a>. Busybox predates Android, but has never +shipped with Android due to the license. As long as we're starting over anyway, +we can do a better job.</p> + +<p>These days, toybox is replacing busybox +in Aboriginal Linux one command at a time, and each toybox release is +regression tested by building Aboriginal Linux with it, then building +Linux From Scratch under the result with the new toybox commands. +The list of commands remaining is tracked <a href=roadmap.html#dev_env>in +the roadmap</a>, and the replacing busybox in Aboriginal Linux is +one of the main goals for toybox' 1.0 release.</p> + +<p>Building LFS requres fewer commands than building AOSP, which has a lot more +<a href=http://source.android.com/source/initializing.html>build +prerequisites</a>. In theory some of those can be built from source +as external packages (we're clearly not including our own java implementation), +but some early prerequisites may need to be added to bootstrap AOSP far enough +to build them (such as a read-only version of "git": +how does repo download the AOSP source otherwise?) +<a name="2_back"></a><sup><font size=-3><a href=#2>[2]</a></font></sup></p> + +<b><h2><a name="status" />What commands are planned/implemented in toybox?</h2></b> <p>The current list of commands implemented by toybox is on the <a href=status.html>status page</a>, which is updated each release. -There is also <a href=roadmap.html>roadmap</a> listing all planned commands for the -1.0 release.</p> +There is also a <a href=roadmap.html>roadmap</a> listing all planned commands +for the 1.0 release and the reasons for including them.</p> -<p>In general, configuring toybox for "defconfig" enables all the commands -compete enough to be useful. Configuring "allyesconfig" enables partially +<p>In general, configuring toybox with "make defconfig" enables all the commands +compete enough to be useful. Configuring "allyesconfig" enables partially implemented commands as well, along with debugging features.</p> -<p>Several toybox commands can do things other vesions can't. For example -the toybox "df" isn't confused by initramfs the way other df implementations -are. (If initramfs is visible, df shows it like any other mount point.)</p> - -<b><h3>Command Shell</h3></b> -<p>The toybox shell (toysh) aims to be a reasonable bash replacement. It -implements the "sh" and "toysh" commands, plus the built-in commands "cd" and -"exit". This is the largest single sub-project in toybox.</p> - -<p>The following additional commands may be built into the shell (but not as -separate executables): cd, exit, if, while, for, function, fg, bg, jobs, source, -<a href="http://opengroup.org/onlinepubs/9699919799/utilities/alias.html">alias</a>, -export, set, unset, read, trap, and exec. (Note: not done yet.)</p> - -</ul> - -<h2><a name="commands" />Which commands are planned?</h2> - -<p>The toybox <a href=todo.txt>todo list</a> mentions many potential commands -which may be added to this project. (Whether that file is readable by anybody -but the project's maintainer is open to debate.) The roadmap wiki in the -nav bar has a more human readable version.</p> - -<p>The criteria for a toybox 1.0 release is that a system built from just the -Linux kernel, toybox, C library (such as uClibc), and a compiler (such as -tinycc) can rebuild itself from source code.</p> - <b><h3>Relevant Standards</h3></b> -<p>Most commands are implemented according to +<p>Most commands are implemented according to POSIX-2008 (I.E. <a href=http://opengroup.org/onlinepubs/9699919799/idx/utilities.html>The -Single Unix Specification version 4</a> where applicable. This does not mean +Single Unix Specification version 4</a>) where applicable. This does not mean that toybox is implementing every SUSv4 utility: some such as SCCS and ed are obsolete, while others such as c99 are outside the scope of the project. Toybox also isn't implementing full internationalization support: it should be 8-bit clean and handle UTF-8, but otherwise we leave this to X11 and higher layers. And some things (like $CDPATH support in "cd") await a good -explanation of why to bother with them. (The standard provides an important -frame of reference, but is not infallable set of commandments to be blindly -obeyed.)</p> - -<p>The other major sources of commands are the Linux man pages, and testing -the behavior of existing commands (although not generally looking at their +explanation of why to bother with them. (POSIX provides an important +frame of reference, but is not an infallable set of commandments to be blindly +obeyed. We do try to document our deviations from it in the comment section +at the start of each command under toys/posix.)</p> + +<p>The other major sources of commands are the Linux man pages, the +Linux Standard Base, and testing the behavior of existing command +implementations (although not generally looking at their source code), including the commands in Android's toolbox. SUSv4 does not include many basic commands such as "mount", "init", and "mke2fs", which are kind of nice to have.</p> +<p>For more on this see the <a href=roadmap.html>roadmap</a> and +<a href=design.html>design goals</a>.</p> + <b><h2><a name="downloads" />Download</h2></b> <p>This project is maintained as a <a href=https://github.com/landley/toybox>git @@ -78,29 +118,46 @@ archive</a>, and also offers <a href=http://landley.net/toybox/downloads>source tarballs</a> and <a href=http://landley.net/toybox/bin>static binaries</a> of the release versions.</p> -<p>The maintainer's <a href=/notes.html>development log</a> and the project's +<p>The maintainer's <a href=http://landley.net/notes.html>development log</a> and the project's <a href=http://lists.landley.net/listinfo.cgi/toybox-landley.net>mailing list</a> are also good ways to track what's going on with the project.</p> -<b><h2><a name="why" />Why is toybox?</h2></b> - -<p>The <a href=http://landley.net/talks/celf-2015.txt>2015 toybox talk</a> -starts with links to three previous talks on the history and motivation of -the project: "Why Toybox", "Why Public Domain", and "Why did I do -Aboriginal Linux (which led me here)?".</p> - <b><h2><a name="toycans" />What's the toybox logo image?</h2></b> -<p>It's <a href=toycans-big.jpg>carefully stacked soda cans</a>. Specifically, +<p>It's <a href=toycans-big.jpg>carefully stacked soda cans</a>. Specifically, it's a bunch of the original "Coke Zero" and "Pepsi One" cans, circa 2006, stacked to spell out the binary values of the ascii string "Toybox", with -null terminator at the bottom. (The big picture's on it's side because +null terminator at the bottom. (The big picture's on it's side because the camera was held sideways to get a better shot.)</p> <p>No, it's not photoshopped, I actually had these cans until a coworker who Totally Did Not Get It <sup><font size=-3><a href=http://www.timesys.com>tm</a></font></sup> threw them out one day after I'd gone home, -thinking they were recycling. (I still have two of each kind, but +thinking they were recycling. (I still have two of each kind, but Pepsi One seems discontinued and Coke Zero switched its can color -from black to grey, presumably in celebration. It was fun while it lasted...)</p> +from black to grey, presumably in celebration. It was fun while it lasted...)</p> + +<b><h2>Footnoes</h2></b> + +<p><a name="1" /><a href="#1_back">[1]</a> Ok, most toolchains (gcc, llvm, pcc, libfirm...) +are multiple packages, but the maintainer of toybox used to maintain a +<a href=http://landley.net/tinycc>fork of tinycc</a> (an integrated +compiler/assembler/linker which once upon a +time did <a href=http://bellard.org/tcc/tccboot.html>build a bootable linux +kernel</a> before its original developer abandoned the project), +and has <a href=http://landley.net/hg/qcc/file/tip/todo/todo.txt>vague plans</a> of <a href=http://landley.net/qcc>trying +again someday</a>. The compiler toolchain is _conceptually_ one package, +implementable as a single multicall binary acting like make, cc, as, ld, cpp, +strip, readelf, nm, objdump, and so on as necessary. It's just the existing +packages that do this <strike>kinda suck</strike> don't. (In theory "make" +belongs in qcc, in practice llvm hasn't got its own make so toybox probably +needs to add it after 1.0 to eliminate another gpl build prerequite from +AOSP.)</p> + +<p><a name="2" /><a href="#2_back">[2]</a> +The dividing line is +"Is there an acceptably licensed version Android can ship, or do we have +to write one?" Since android is not "GNU/Linux" in any way, we need to +clean out all traces of gnu software from its build to get a clean +self-hosting system.</p> <!--#include file="footer.html" --> diff --git a/www/header.html b/www/header.html index 56396d5d..8c968008 100644 --- a/www/header.html +++ b/www/header.html @@ -9,10 +9,10 @@ <ul> <li><a href="index.html">News</a></li> <li>What is it?<br> - <a href="about.html">About</a><br> - <a href="roadmap.html">Roadmap</a><br> - <a href="status.html">Status</a><br> - <a href="help.html">Help</a><br> + - <a href="about.html">About</a><br> + - <a href="roadmap.html">Roadmap</a><br> + - <a href="status.html">Status</a><br> + - <a href="help.html">Help</a><br> </li> <li>Why is it?<br> @@ -28,8 +28,8 @@ </ul> <b>Development</b> <ul> - <li><a href="design.html">Design</a></li> - <li><a href="code.html">Source walkthrough</a></li> + <li><a href="design.html">Design goals</a></li> + <li><a href="code.html">Source code walkthrough</a></li> <li><a href="http://lists.landley.net/listinfo.cgi/toybox-landley.net">Mailing List</a> (<a href=http://news.gmane.org/gmane.linux.toybox>backup archive</a>)</li> <li>IRC #toybox on freenode.net</li> <li><a href=https://github.com/landley/toybox/commits/master.atom>Commit RSS feed</a></li> diff --git a/www/roadmap.html b/www/roadmap.html index 9878133e..c31e1f34 100755 --- a/www/roadmap.html +++ b/www/roadmap.html @@ -86,7 +86,10 @@ revisited later, but not for toybox 1.0.)</p> separate executables (alias bg cd command fc fg getopts hash jobs kill read type ulimit umask unalias wait). These may be revisited as part of a built-in toybox shell, but are not exported into $PATH via symlinks. (If you fork a -child process and have it "cd" then exit, you've accomplished nothing.)</p> +child process and have it "cd" then exit, you've accomplished nothing. +This is not a complete list, a shell also needs exit, if, while, for, case, +export, set, unset, trap, exec... And for bash compatability, function and +source.)</p> <p>A few other commands are judgement calls, providing command-line internationalization support (iconv locale localedef), System V inter-process @@ -220,15 +223,23 @@ self-bootstrapping build still uses the following busybox commands, not yet supplied by toybox:</p> <blockquote><p> -awk dd diff expr fdisk ftpd ftpget ftpput gunzip -gzip less pgrep ping pkill ps route sh sha512sum tar test tr unxz vi -wget xzcat zcat +awk bunzip2 bzcat dd diff expr fdisk ftpd ftpget +ftpput gunzip gzip less pgrep ping pkill route sh +sha512sum tar test tr unxz vi wget xzcat zcat </p></blockquote> -<p>Many of those are in "pending". Most of the archive commands are needed -because busybox tar doesn't call external versions. The remaining "difficult" +<p>Many of those are in "pending". The remaining "difficult" commands are vi, awk, and sh.</p> +<p>Building Linux From Scratch is not the same as building the +<a href=https://source.android.com>Android Open Source Project</a>, +but after toybox 1.0 focus may shift to <a href=http://landley.net/aboriginal/about.html#hairball>modifying the AOSP build</a> +to reduce dependencies. (It's fairly likely we'll have to add at least +a read-only git utility so repo can download the build's source code, +but that's actually <a href=https://www.youtube.com/watch?v=P7n6G2IL6eo>not +that hard</a>. We'll probably also need our own "make" at some point after +1.0.)</p> + <hr /> <h2><a name=android /><a href="#android">Use case: Replacing Android Toolbox</a></h2> @@ -311,7 +322,7 @@ from its core system, and is installing toybox as <blockquote><b> <span id=tizen> arch base64 users dir vdir unexpand shred join csplit -hostid nproc runcon sha224 sha256 sha384 sha512 sha3 mkfs.vfat fsck.vfat +hostid nproc runcon sha224sum sha256sum sha384sum sha512sum sha3sum mkfs.vfat fsck.vfat dosfslabel uname stdbuf pinky diff3 sdiff zcmp zdiff zegrep zfgrep zless zmore </span> </b></blockquote> @@ -342,19 +353,19 @@ replacement.</p> musl), klibc builds a random assortment of executables to run init scripts with. There's no multiplexer command, these are individual executables:</p> -<blockquote><p> +<blockquote><p><b> cat chroot cpio dd dmesg false fixdep fstype gunzip gzip halt ipconfig kill kinit ln losetup ls minips mkdir mkfifo mknodes mksyntax mount mv nfsmount nuke pivot_root poweroff readlink reboot resume run-init sh sha1hash sleep sync true umount uname zcat -</p></blockquote> +</b></p></blockquote> <p>To get that list, build klibc according to the instructions (I <a href=http://landley.net/notes-2013.html#23-01-2013>looked at</a> version 2.0.2 and did cd klibc-*; ln -s /output/of/kernel/make/headers_install linux; make) then <b>echo $(for i in $(find . -type f); do file $i | grep -q executable && basename $i; done | grep -v '[.]g$' | sort -u)</b> to find -executables, then eliminated the *.so files and *.shared duplicates.</p> +executables, then eliminate the *.so files and *.shared duplicates.</p> <p>Some of those binaries are build-time tools that don't get installed, which removes mknodes, mksyntax, sha1hash, and fixdep from the list. @@ -368,7 +379,7 @@ for the oddball names.</p> license terms" above), adding nothing to the other projects we've looked at. But we still need sh, gunzip, gzip, and zcat to replace this package.</p> -<p>By the time I did the analysis toybox already had cat, chroot, dmesg, false, +<p>At the time I did the initial analysis toybox already had cat, chroot, dmesg, false, kill, ln, losetup, ls, mkdir, mkfifo, readlink, rm, switch_root, sleep, sync, true, and uname.</p> @@ -514,30 +525,34 @@ it sucks in -lz from the host.)</p> <a name=sbase /> <h2>sbase:</h2> -<p>It's <a href=http://git.suckless.org/sbase>on suckless</a>. So far it's -implemented:</p> +<p>It's <a href=http://git.suckless.org/sbase>on suckless</a> in +<a href=http://git.suckless.org/ubase>two parts</a>. As of November 2015 it's +implemented the following (renaming "cron" to "crond" for +consistency, and yanking "sponge", "mesg", "pagesize", "respawn", and +"vtallow"):</p> <blockquote><p> <span id=sbase_cmd> -basename cat chmod chown cksum cmp cp date dirname echo false fold grep head -kill ln ls mc mkdir mkfifo mv nl nohup pwd rm seq sleep sort tail tee test -touch true tty uname uniq wc yes +basename cal cat chgrp chmod chown chroot cksum cmp cols comm cp crond cut date +dirname du echo env expand expr false find flock fold getconf grep head +hostname join kill link ln logger logname ls md5sum mkdir mkfifo mktemp mv +nice nl nohup od paste printenv printf pwd readlink renice rm rmdir sed seq +setsid sha1sum sha256sum sha512sum sleep sort split strings sync tail +tar tee test tftp time touch tr true tty uname unexpand uniq unlink uudecode +uuencode wc which xargs yes </span> </p></blockquote> -<p>And has a TODO list:</p> +<p>and<p> <blockquote><p> <span id=sbase_cmd> -cal chgrp chvt comm cut df diff du env expand expr id md5sum nice paste -printenv printf readlink rmdir seq sha1sum split sync test tr unexpand unlink +chvt clear dd df dmesg eject fallocate free id login mknod mountpoint +passwd pidof ps stat su truncate unshare uptime watch who </span> </p></blockquote> -<p>At triage time, of the first list I still need to do: fold grep mc mv nl. Of -the second list: diff expr paste printf split test tr unexpand who.</p> - <hr /> <a name=s6 /> <h2>s6</h2> @@ -666,18 +681,18 @@ mount_ffs mv pfctl ping poweroff ps reboot rm route sed sh stty sysctl tar test traceroute umount vi wiconfig </p></blockquote> -<p>Apparently lv is the missing link ed and vi, copyright 1982-1997 (do not -want), ex is another obsolete vi mode, lesskey is "used to +<p>Apparently lv is the missing link between ed and vi, copyright 1982-1997 (do +not want), ex is another obsolete vi mode, lesskey is "used to specify a set of key bindings to be used with less", and csh is a shell they -sucked in, [ is an alias for test. Several more bsd-isms that don't have Linux +sucked in (even though they have mksh?), [ is an alias for test. Several more bsd-isms that don't have Linux equivalents (even in the ubuntu "install this package" search) are -disklabel, fsck_ffs, mount_ffs, and pfctl. And wiconfig is a wavelan interface -network card driver utility. Subtracting all that and the commands toybox -already implements at triage time, we get:</p> +disklabel, fsck_ffs, mount_ffs, and pfctl. And wiconfig is a +wavelan interface network card driver utility. Subtracting all that and the +commands toybox already implements at triage time, we get:</p> <blockquote><p> <span id=beastiebox_cmd> -fdisk fsck getty halt ifconfig init kill less mksh more mount mv ping poweroff +fdisk fsck getty halt ifconfig init kill less more mount mv ping poweroff ps reboot route sed sh stty sysctl tar test traceroute umount vi </span> </p></blockquote> |