Toybox combines common Linux command line utilities together into a single BSD-licensed executable that's simple, small, fast, reasonably standards-compliant, and powerful enough to turn Android into a development environment. See the links on the left for details.
"Space," it says, "is big. Really big. You just won't believe how vastly, hugely, mindbogglingly big it is. I mean, you may think it's a long way down the street to the chemist's, but that's just peanuts to space." - The Hitchhiker's Guide to the Galaxy.
Toybox 0.4.7 is based on commit 1122.
New commands: Brad Conroy submitted blkid. Elie De Brauwer submitted reboot, halt, and poweroff. Strake's nl got cleaned up and promoted from pending to posix. In addition, the existing chvt and vconfig got some cleanup.
That said, I haven't nearly kept up with the flood of new commands going into pending: Ashwini Sharma's team submitted dd, dumpleases, traceroute, top, useradd, groupadd, mkpasswd, tftpd, and an fsck wrapper (with no filesystem drivers yet). Isaac Dunham sent in cpio.
Bugfixes: Jeroen van Rijn added a user count to uptime. Elie De Brauwer added -e to watch, removed a memory leak, and fixed a terminal size problem. William Haddon made xargs call its command line once even with blank input (the standard is vague, but builds expect it), and fixed an off by one bug where grep didn't malloc enough space with -E (leading to a segfault). I fixed a glitch in bunzip2 (same one as went into busybox since they're using the code I wrote), in od to fix -t co, -J, and -c options. Add uname -o as a synonym for -s. Build fix to never use $CC without prefixing it with $CROSS_COMPILE (since $HOSTCC could be different). Anca Emanuel spotted a typo in the web page.
The compile-time command line option parsing got rewritten (ported from bash to C), which should speed up builds a bit and allow code controlled by --longopts to drop out properly when disabled in the configuration. Terminal querying got refactored. Patch's -x option is now more informative (a debug thing if you're trying to figure out why a patch didn't apply). The "toynet.h" file got folded into toys.h since musl supports it and micromanging uClibc options isn't very interesting anymore. The test suite now uses scripts/single.sh when testing a single command.
"Think of a number," said the computer, "any number." Arthur told the computer the telephone number of King's Cross railway station passenger inquiries, on the grounds that it must have some function, and this might turn out to be it. - The Hitchhiker's Guide to the Galaxy
Toybox 0.4.6 is based on commit 1068.
This release adds several new commands: Felix Janda wrote paste and fallocate, Kyungwan Han submitted eject, Strake contributed grep, Ashwini Sharma added pmap (and a testsuite entry for grep), Lukasz Skalski sent pwdx, Isaac Dunham posted acpi, and I did timeout and umount.
The ls command now has a --color=auto option (suggested by Rich Felker). The multiplexer now has a --help option so you can say "./toybox --help blah" instead of using the built-in "help" command. (Which is a shell built-in. Try it on your command line, it's like man for shell builtins. But a certain other project has conditioned people to expect --help, so...) I forget who heehooman at gmail is but they pointed out unshare needed PID and UID namespace support.
A lot of new commands in toys/pending, to the point the next release should probably just focus on cleanup and review of this backlog. We've got klogd, dhcp, dhcpd, watch, route, and ps from Ashwini Sharma (and an fsck wrapper but no fsck.fstype engines yet), syslogd, pgrep, and pkill from Madhur Verma, netstat by Ranjan Kumar, test by Felix Janda, lspci by Isaac Dunham, nl, su, and renice by strake (I.E. M. Farkas-Dyck), and sysvinit by Kyungwan Han.
Some cleanup work on existing pending commands that aren't ready to promote yet: I did a few more rounds on ifconfig and Isaac Dunham's did several cleanups to xzcat, Felix Janda cleaned up logger and syslogd...
Also some cleanup work on commands that predate the pending directory, but weren't quite polished when they went in, most prominently du, expand, and touch.
The new scripts/single.sh builds a standalone command without the multiplexer, although not all commands can be built that way yet (NEWTOY yes, OLDTOY no) and the space savings aren't anything to write home about. (If a command needs the option parsing logic at all, it needs all of it.) If you're curious, you can do:
make defconfig make mkdir singles for i in $(./toybox) do echo $i PREFIX=singles/ scripts/single.sh $i || break done(And then wait a long time and watch almost half the builds fail.)
There is now libbuf analogous to toybuf, another global 4k buffer this time for use by lib/ code instead of command code.
The lib directory got split up a bit, lib/pending.c contains functions not yet used by anything outside of toys/pending/*, and lib/xwrap.c contains functions that wrap other functions and handle failures (via error_exit). This leaves lib/lib.c containing actual new functions.
General improvements and bug fixes to argument parsing. The [-abc] exclude logic should now clear arguments slots when disabling options. Bare --longopts should work now and be able to report errors using their name, the new ; option allows optional arguments to longopts only suppliable with = (I.E. --color and --color=auto but not --color auto).
I'm gradually weaning the code off of itoa()/utoa() because sprintf does this already. In this case "simple" probably means "let libc do it for us".
Rewrote for_each_pid_with_name_in() and renamed it to just names_to_pid(). It shouldn't get confused trying to compare absolute and relative paths quite so much anymore.
lib/llist.c grew a new dlist_pop() function for removing a doubly linked list entry while maintaining a circular list; tail and patch are using it now.
The musl guys suggested a new optimization flag (-fno-asynchronous-unwind-tables) that shaves about 10% off the binary size by removing a C++ism that crept into gcc's idea of C. While I don't normally try to micromanage the compiler, "-fstop-being-stupid" is a thing you have to hit gcc with from time to time.
Felix Janda and I did a largeish rewrite of tail to finally make it work right (we think). Still need to implement tail -f someday (the tricky bit is making -f follow multiple files at once). Felix also reported a bug in xpidfile.
Juhani Haverinen pointed out that python 3 doesn't work with config2help.py, so the detection logic looks for python2 (until I get around to rewriting that in C). Elie De Brauwer then fixed our first attempt at this, and also fixed uname's help string.
Ashwini Sharma pointed out the build was making a FLAG_ macro for " " which broke some configurations. (That's a control character, not a command line option.)
Jacek Bukarewicz pointed out a bug in chdir permission handling, and a way to make env segfault. Both should be fixed now.
The new function xexec_optargs() replaces calls to xexec(toys.optargs) to avoid freeing and reusing optargs during option parsing screwing stuff up (such as netcat's exec mode).
The stat command's %a output was padded with leading zeroes, which didn't match anybody else's behavior and thus made the test suite hiccup between TEST_HOST and testing toybox. (If you go "TEST_HOST=1 scripts/test.sh command" it sanity checks the tests against the host implementation.)
Last release, "mkdir sub/sub && chmod 007 sub/sub && rm -rf sub" didn't delete sub and didn't exit with an error either. Neither was correct, rm should now be fixed.
Georgi Chorbadzhiyski maintains a git mirror of the repository on github, automatically updated from the mercurial every 6 hours. The mirror is read only, but you can generate patches against it and post them to the list.
"Time is an illusion. Lunchtime doubly so." "Very deep. You should send that in to the Reader's Digest. They've got a page for people like you." - The Hitchhiker's Guide to the Galaxy.
Toybox 0.4.5 is based on commit 941. It adds uuencode and uudecode from Erich Plondke, and enables Luis Morales' "who" by default. Felix Janda and I cleaned up last year's "stat" submission and enabled it. Ivo van Poorten added "groups". Andre Renaud added "lsusb". I implemented "split", "pivot_root", and "mv".
The "help" command is implemented differently now (lib/help.c) and each command can now understand --help (including both "toybox --help" and "toybox --help command" in the multiplexer).
The "pending" directory has several commands (find, xzcat, nbd-client, logger, expr) which work but are not enabled by default pending further cleanup. Ifconfig is enabled, but still in pending because it's only 2/3 cleaned up. (It's an awkward halfway state but I'm not holding up the release for it.)
I'm documenting the cleanups to teach more people to do it, but the writeups aren't caught up yet. The roadmap also got updated a bit with further analysis of other projects, and the README and about pages got updated.
Fixed _another_ "ls -C" segfault when terminal size can't be detected, condensed the ls help text to fit on one page, implented --color, and taught -l to print the major, minor numbers when showing block/char devices. Argument parsing now handles "--" properly (to end option checking), and the infrastructure can now handle bare --longopts that have no corresponding short option (both were implemented before but didn't work). Fixed an old bug in "patch", chmod grew -f, who grew -a. Isaac Dunham fixed "-" vs "_" handling in modinfo, added a "firmware" output field, added -b and -k support, and taught it that the ".ko" extension means to look for the file at the specified path instead of under /lib. Felix Janda moved file permission display code to lib so ls and stat could share it. Ashwini Sharma spotted a bug in xabspath when the last path component exists but we haven't got permissions to open it (ala readlink -f /dev/sda as a normal user).
In the build infrastructure, scripts/findglobals.sh finds leaked global variables. (Leaked means they aren't part of the global union: Other than glibc debris, toybox should define "this", "toy_list", "toybuf", and "toys", and that's it; the rest add memory footprint to every command for the benefit of just one command; use GLOBALS() to stick 'em in the union.) Static linking against libraries other than the host's libc now applies to feature probes for unshare and such. Neuter stupid internationalization support that makes various host "sort" commands put things in an order other than alphabetical (breaking the multiplexer's binary search on command names).
You should now be able to build from a source control snapshot on a build system that hasn't got python: if you disable CONFIG_TOYBOX_HELP. (The release tarballs ship generated/help.h, but it's not in source control. Eventually I should rewrite that python script in C.)
LICENSE TWEAK: After discussion on the mailing list the "2 clause BSD" license got slightly simplified so the first paragraph now says:
Permission to use, copy, modify, and/or distribute this software for any purpose with or without fee is hereby granted.
It used to continue "provided that the above copyright notice and this permission notice appear in all copies", but A) what's the point? B) does "all copies" mean binaries, or just source code, or what? C) lots of projects that consider BSD and GPL compatible have files with both license notices on them (sometimes at opposite ends of the file to make the conflict less obvious) because "all copies must include this function" would violate the GPL but "all copies must include this magic text blob" somehow don't?
I don't want to have to care about this anymore. The tweaked version is more or less public domain with a liability disclaimer, but we're still calling it BSD (sometimes "0 clause BSD") to avoid explaining.
Video of my ELC talk "Why is Toybox?" is up on youtube. Related materials include the talk outline and an android self-hosting writeup.
[Updated June 4] The following links jump to specific topics in the video. (Sorry about the ads, it's The Linux Foundation.)
"Ford, you're turning into a penguin. Stop it." - The Hitchhiker's Guide to the Galaxy.
Toybox 0.4.4 is based on commit 813, adding the "time" and "readahead" commands, plus some bugfixes.
The "cp" command now implements the -s symlink option, plus bugfixes getting various corner cases right as used in actual package builds. "id -Gn root" should now print root's groups instead of the current user's. Several build fixes so toybox builds under Ubuntu 8.04 again (which is about as old a build environment as you can expect to find posix-2008 features in).
Unfinished commands have generally been moved to "toys/pending". Everything else should "default y" to participate in make defconfig. Several of those pending commands got some basic cleanup so allyesconfig should at least compile (although defconfig is still what's useful).
Significant roadmap updates, checking several other multicall binaries (klibc, sash, sbase, s6...) to see what commands they include.
This must be Thursday. I never could get the hang of Thursdays. - The Hitchhiker's Guide to the Galaxy.
Toybox 0.4.3 is based on commit 793. There are now exactly 100 commands in defconfig (of a little over 220 on the todo list).
Elie De Brauwer added the rev command, cleaned up tac, implemented the -s and -f flags for seq, added -v and -i to killall (and fixed killall not to kill itself before finishing its pid list), and added to the test suite. Felix Janda added -m to mkdir, pwd -L and -P, and more test suite entries.
Rob Landley added the losetup command, and fixed the existing ls, cp, and readlink commands. The segfault in ls happened when it couldn't determine the screen size (last release changed the default to -C and a screen size of 0 made column view unhappy), and cp got an extensive rewrite bringing it up to date with the dirtree changes and fixing a number of things it never did right in the first place. The xabspath() code in the library now handles a symlink after ".." properly (and the test suite checks for it).
Infrastructure-wise the code is better about automatically setting the error return code properly. Now error_msg() sets the exit code to 1 if it's still defaulting to 0, and the global exit path does a fflush(NULL) with error bit check rather than trying to be quite so granular about flushing. (That means if we use printf() instead of xprintf() it still exits with the right error code, it just doesn't end the program early on an output error.) Minor bugfix so TOYBOX_DEBUG doesn't always warn about the lack of suid bit when toybox is built with at least one STAYROOT command. Bugfix for the option [grouping] logic (and then further fixes to the error reporting pointed out by Ashwini Sharma). dirtree_handle_callback() now has a prefix like the rest of the dirtree functions. A lot of stuff doing manual path handling was switched to using libc basename() (including, embarassingly, the basename command), which means it now correctly detects "/trailing/slash/" which the previous code didn't.
Also, last release included some accidentally checked in debug code that disabled compiler optimization, so the binary size bloated a bit. It's back to -Os by default now.
"The major difference between a thing that might go wrong and a thing that cannot possibly go wrong is that when a thing that cannot possibly go wrong goes wrong it usually turns out to be impossible to get at or repair."
- The Hitchhiker's Guide to the Galaxy.
Toybox 0.4.2 is based on commit 749 and is just a resync. Linux 3.7 came out, meaning it's time to do an Aboriginal Linux release, and that should use a stable version of toybox. So here's a new stable version.
The new commands are cut (from Jason Kyungwan Han), touch (from Choubey Ji), expand (from Jonathan Clairembault, and he fixed a bug in login), and rm (from Rob Landley). Felix Janda added UTF-8 support infrastructure (for non-ascii character sets) with a config option. Elie De Brauwer added tests for cat and sha1sum, and -so options to pidof. The "ls" command defaults to -C (column view) now, and "readlink" now supports -fenq.
Portability work: toybox should now build against the musl C library, and against older glibc versions (circa 2008, much before that and kernel features we depend on start to drop out).
The whole codebase got reindented from "one tab" to "two spaces" per level. The option parsing logic now understands [groups] of commands (when more than one in a group is selected it can switch the others off, or error out, or other things). The error_exit() infrastructure can now longjmp back to an earlier point instead of exiting. Each toys/* directory now has a README, the first line of which is the fancy name menuconfig uses for the directory (so no more hardwired directory list in scripts/genconfig.sh).
Fixed a filehandle leak in getmountlist(). Pass parent pointer to dirtree_add_node() so it can give error messages with full path. The yesno() function now always reads from stdin and writes to stderr (we can retry tty checking complexity once we've got commands needing it).
The open group broke their website so the old links to POSIX 2008 now need to start with pubs. Some of the links in the tree have been updated, others haven't while I wait to see if their webmaster notices and fixes it.
(I note that the current rm implementation is not technically posix compliant because the standard requires infinite recursion depth and the current implementation uses one filehandle per level. I can add a config option to do it Posix's way, which is more brittle and needs extra security checks, but am waiting for somebody to complain first. The default "ulimit -n" is 1024 filehandles, so drilling down over 1000 nested subdirectories).
"Rule Six: The winning team shall be the first team that wins." - The Hitchhiker's Guide to the Galaxy.
Toybox 0.4.1 is based on commit 691.
Elie De Brauwer contributed usleep, Ashwini Kumar contributed du, and Kyungwan Han contributed vconfig. Other new commands include switch_root and md5sum, and the remaining shell wrappers are now proper commands (dos2unix, unix2dos).
The patch command now supports -l, and gethostname is now enabled by default. The df command follows symlinks to get the actual device name. Felix Janda added -m support to wc (for utf8).
On the infrastructure side, the commands have now been grouped into "posix", "lsb", and "other" subdirectories (for things required by Posix-2008, the Linux Standard Base 4.1, and commands in neither). This affects menuconfig and the actual source layout (toys/cp.c is now toys/posix/cp.c, and so on). An android directory is planned (see the updated android roadmap analysis).
The FLAG_ macros for command option parsing and TT alias for the command's
global block are now automatically generated, commands should
#define FOR_commandname before #including
An upgrade to the build infrastructure now allows commands with _ and - in them, such as switch_root.
Bugfixes: Avery Pennarun spotted a case where ls showed uid twice instead of uid and gid, and that nice was using the wrong range of numbers. The ls command also recursed inappropriately last time (not quite properly converted for the dirtree changes last release), and now it's fixed. Roy Tam pointed out a glitch in sh, and fixed df's percentage calculation to match the POSIX spec. The kernel build didn't like our mktemp and it does now. The wc command wasn't quite posix compliant (trailing spaces break stuff). The ls command recursed inappropriately last time (not quite properly converted for the dirtree changes last release), and now it's fixed. The catv command wasn't displaying byte 255 correctly. Some lib fixes (thinko in xpidfile). Fixed uname -m when running a 32 bit x86 binary on an x86-64 host (it lies and says the system is i686, i586, or i486 depending on what the toolchain that built the binary supported. This makes builds in a 32 bit chroot on a 64 bit kernel break less.) The df command was checking partitions in the wrong order (displaying undermounts instead of overmounts: this used to work but some library code changed out from under it and it wasn't updated to match until now). Felix Janda filled out the test suite some more. The patch file creation logic got tweaked several times to successfully apply more patches. Support for older (pre 2.10) glibc versions was added to portability.h.
Miscelaneous cleanups all around (mknod, sha1sum, logname), including a rewrite of taskset to be less dependent on libc getting the headers right. All the command headers should now point to the current relevant standards document, where applicable.
This news page had old news entries from before the relaunch moved into a separate oldnews page.
I forgot to create static binaries last time, but they're back now.
"Ford", Arthur said. "There's an infinite number of monkeys out here who want to talk to us about this script for Hamlet they've worked out." - The Hitchhiker's Guide to the Galaxy.
Toybox 0.4.0 is based on commit 640.
The new status page is calculated from the roadmap info, and should be easier to keep up to date in future.
Andre Renaud contributed od and modinfo. Elie De Brauwer contributed taskset, bugfixes to cmp and tail, and tests for sort and tail. Kyungwan Han contributed passwd. Gaurang Shastri contributed w. Ashwini Sharma spotted a case where dirtree was adding extra slashes to a path.
I rewrote od, cleaned up comm, documented the llist and dirtree infrastructure, added an -r option to date (and fixed a bug where -u wouldn't override /etc/localtime), fixed bugs in chmod +stw, fixed ls to show suid bits properly when the corresponding executable bit wasn't set, and worked around a longstanding glibc bug where static linking prevents stdout from automatically flushing pending output on exit.
"For a moment, nothing happened. Then, after a second or so, nothing continued to happen." - The Hitchhiker's Guide to the Galaxy.
Toybox 0.3.1 is based on commit commit 607. It's mostly a bugfix release for ls -l (which was unhappy on targets other than x86-64), plus a new "date" from Andre Renaud and rewritten chgrp/chown which now support the full set of posix flags, plus a little work on the test suite and some more header tweaks towards eventual compatability with the musl libc.
The todo list runneth over, but "release early, release often", so here it is. The roadmap and documentation are a bit behind, and I've got ~40 pending submissions to review. I need to catch up...
"For instance, on the planet Earth, man had always assumed that he was more intelligent than dolphins because he had achieved so much - the wheel, New York, wars and so on - whilst all the dolphins had ever done was muck about in the water having a good time. But conversely, the dolphins had always believed that they were far more intelligent than man - for precisely the same reasons." - The Hitchhiker's Guide to the Galaxy.
It's well past time for toybox 0.3.0, so here it is, based on commit 595, and the statically linked prebuilt binaries should actually be statically linked this time (thanks Ashwini Sharma for spotting that).
It's hard to figure out where to cut a release, because development doesn't stop. "Long before now" is the obviuos answer, of course. The project's maintainer also moved house during this development cycle, which threw things off for a bit (so many boxes). Releases should hopefully be a bit more frequent from here on.
The big things Rob worked on this time were the new dirtree (directory tree traversal) infrastructure, and a complete rewrite of ls using that which should now implement all 26 posix options.
Georgi Chorbadzhiyski added printenv, whoami, mkdir, mkfifo, chmod, chown, chgrp, and uniq. He also added fraction and extension support to sleep (so if you need a quarter-second sleep, it can do that now), and fixed a build bug on slackware.
Daniel Walter contributed a string to mode_t parser (in use by chmod and mkdir -m). Ilya Kuzmich contributed comm. Elie De Brauwer added mountpoint, vmstat, logname, login, and mktemp. Kevin Chase did some portability cleanups. Pere Orga fixed some documentation.
The "tac" and "clear" commands are now normal commands instead of shell wrappers, and the header #includes have been cleaned up a bit to remove deprecated functions and attempt to increase compatability with the bionic and musl C libraries, "tail" should now use lseek() for large files, and "id" got some cleanups and bugfixes.
The new TOYBOX_FLOAT configuration option selects whether or not to include floating point support (for embedded targets where that's problematic).
Several random bugfixes: unshare() might actually build portably now, yes 'n' | cp -i should no longer bypass stdin and prompt via the tty, the SUID support no longer drops permissions going through the toybox multiplexer command, and a bugfix to xargs -0 means it should no longer segfault. (I have a pending bug report about xargs not doing the full posix whitespace handling that -0 obsoleted, but I'll deal with that next release.)
The build infrastructure is now automatically generating FLAG_ macros for the options, but currently with the wrong names. Some more macro glue is necessary, which I haven't quite figured out how to do yet.
A defconfig toybox at the start of the $PATH has successfully built Linux From Scratch (in my Aboriginal Linux project). The commands that 'default n' in the config are often still broken, cleanup is ongoing. (The new dirtree stuff broke several of them that haven't been converted yet, but if I wait until everything works we won't have a release before 1.0, so here's a checkpoint.)
"They went unnoticed at Goonhilly, passed over Cape Canaveral without a blip, and Woomera and Jodrell Bank looked straight through them. Which was a pity, because it was exactly the sort of thing they'd been looking for all these years."
- The Hitchhiker's Guide to the Galaxy.
Here's toybox 0.2.1 based on commit 512. This time around, there are statically linked prebuilt binaries for various embedded targets.
It's been a busy few weeks, almost entirely due to new contributors. (I have not quite been keeping up.)
Elie De Brauwer contributed free, uptime, swapon, swapoff, lsmod, mknod, insmod, rmmod, and fixed a bug in basename. Andre Renaud contributed ls, ln, realpath, and hostname. Andres Heck contributed pidof and killall. Daniel Walter wrote kill and extended id. Timothy Elliott contributed tail and tests for cmp. Frank Bergmann sent a warning fix. Bryce Fricke added -i to cp. Nathan McSween pointed out an optimization. Georgi Chorbadzhiyski fixed cross compiling to work more reliably.
(My own contribution this time around was just tightening up other people's code, a build fix to unshare, some random bugfixes, and so on. My only new code this time around was writing a bash replacement for the existing python bloat-o-meter.)
Last time (the 0.2.0 release) included the first pass at an id command from Tim Bird, env and basename from Tryn Mirell, cmp and head from Timothy Elliott, more bugfixes from Nathan McSween and Elie De Brauwer, and Luis Felipe Strano Moraes did a first pass at the who command plus other bugfixes and optimizations.
(For that release I did xargs, cal, truncate, unlink, nohup, tty, wc, link, dirname, unshare, and various infrastructure tweaks, but it took me 3 months and those guys did their stuff in a week or so.)
"for though it has many omissions and contains much that is apocryphal, or at least wildly inaccurate, it scores over the older, more pedestrian work in two important respects..."
- The Hitchhiker's Guide to the Galaxy
Here's the first BSD licensed release, toybox-0.2.0, more a synchronization point than anything particularly useful. 47 commands in a reasonably ready-to-use state (what "make defconfig" builds), another ten or so partially finished stubs ("make allyesconfig"), and several patches pending on the mailing list I need to review and merge.
More to come...
November 15, 2011 - Back from the dead, Toybox is now under a 2 clause BSD license, and aiming to become the default command line implementation of Android systems everywhere.
More to come...
Old news from before the relaunch.