From 9a69a929eced859415a46175d891d547ddfbb3bd Mon Sep 17 00:00:00 2001 From: Rob Landley Date: Sat, 23 Feb 2013 18:32:08 -0600 Subject: Accumulated web page changes. (Release announcement for previous release, roadmap updates, etc.) --- www/roadmap.html | 356 ++++++++++++++++++++++++++++++++++++++++++++++++++++--- 1 file changed, 341 insertions(+), 15 deletions(-) (limited to 'www/roadmap.html') diff --git a/www/roadmap.html b/www/roadmap.html index 30d4dd4f..549c9486 100755 --- a/www/roadmap.html +++ b/www/roadmap.html @@ -7,15 +7,36 @@ utilities, and are using those to determine which commands to implement for Toybox's 1.0 release.

-

Our current candidate list combines the commands toybox already implements, -the development environment command list, the toolbox standard commands, -various vendor configurations of busybox, a selected subset of the POSIX/SUSv4 -standard, a couple of the less-insane bits of LSB, a few outright requests, -plus additional to-be-determined shell functionality.

+

The most interesting standards are POSIX-2008 (also known as the Single +Unix Specification version 4) and the Linux Standard Base (version 4.1). +The main test harness including toybox in Aboriginal Linux and if that can +build itself using the result to build Linux From Scratch (version 6.8). +We also aim to replace Android's Toolbox.

+ +

At a secondary level we'd like to meet other use cases. We've analyzed +the commands provided by similar projects (klibc, sash, sbase, s6, embutils, +nash, and beastiebox), along with various vendor configurations of busybox, +and some end user requests.

+ +

Finally, we'd like to provide a good replacement for the Bash shell, +which was the first program Linux ever ran and remains the standard shell +of Linux no matter what Ubuntu says. This doesn't mean including the full +set of Bash 4.x functionality, but does involve {various,features} beyond +posix.

See the status page for the combined list and progress towards implementing it.

+ +

Use case: standards compliance.

@@ -203,6 +224,7 @@ rm mkdir rmdir reboot getevent sendevent date wipe sync umount start stop notify cmp dmesg route hd dd df getprop setprop watchprops log sleep renice printenv smd chmod chown newfs_msdos netstat ioctl mv schedtop top iftop id uptime vmstat nandread ionice touch lsof md5 r +cp du grep watchdogd

If selinux is enabled, you also get:

@@ -259,13 +281,15 @@ bespoke code to install itself.

For reference, combining everything listed above, we get:

-alarm ash cat chcon chmod chown cmp date dd df dmesg exists fs_mgr getenforce -getevent getprop getsebool gpttool hd id ifconfig iftop init insmod ioctl +alarm ash cat chcon chmod chown cmp cp date dd df dmesg du exists fs_mgr +getenforce +getevent getprop getsebool gpttool grep hd id ifconfig iftop init insmod ioctl ionice kill ln load_policy log logcat logwrapper ls lsmod lsof lsusb md5 mkbootimg mkdir mount mv nandread netcfg netstat newfs_msdos notify printenv ps r readtty reboot renice restorecon rm rmdir rmmod rotatefb route run-as runcon schedtop sdcard sendevent setconsole setenforce setkey setprop setsebool -sleep smd start stop sync syren top touch umount uptime vmstat watchprops wipe +sleep smd start stop sync syren top touch umount uptime vmstat watchdogd +watchprops wipe

We may eventually implement all of that, but for toybox 1.0 we need to @@ -277,12 +301,12 @@ functionality, which we can implement a shim interface for later).

This means toybox should implement:

-cat chmod chown cmp date dd df dmesg getevent getprop hd id ifconfig iftop -insmod ioctl ionice kill ln log logcat logwrapper ls lsmod lsof lsusb md5 mkdir -mount mv nandread +cat chmod chown cmp cp date dd df dmesg du getevent getprop grep hd id ifconfig +iftop insmod ioctl ionice kill ln log logcat logwrapper ls lsmod lsof lsusb md5 +mkdir mount mv nandread netstat newfs_msdos notify printenv ps r reboot renice rm rmdir rmmod route schedtop sendevent setconsole setprop sleep smd start stop sync top touch -umount uptime vmstat watchprops wipe +umount uptime vmstat watchprops watchdogd wipe
@@ -290,8 +314,8 @@ umount uptime vmstat watchprops wipe sections of this analysis:

-cat chmod chown cmp date dd df dmesg id ifconfig insmod kill ln ls lsmod -mkdir mount mv ps renice rm rmdir rmmod route sleep sync top touch umount +cat chmod chown cmp cp date dd df dmesg du grep id ifconfig insmod kill ln ls +lsmod mkdir mount mv ps renice rm rmdir rmmod route sleep sync top touch umount

Which leaves the following commands as new from Toolbox:

@@ -299,9 +323,311 @@ mkdir mount mv ps renice rm rmdir rmmod route sleep sync top touch umount
getevent getprop hd iftop ioctl ionice log lsof nandread netstat newfs_msdos notify printenv r reboot schedtop sendevent setconsole -setprop smd start stop top uptime vmstat watchprops wipe +setprop smd start stop top uptime vmstat watchprops watchdogd wipe +
+ +
 +

kblic:

+ +

Long ago some kernel developers came up with a project called +klibc. +After a decade of development it still has no web page or HOWTO, +and nobody's quite sure if the license is BSD or GPL. It inexplicably +requires perl to build, and seems like an ideal candidate for +replacement.

+ +

In addition to a C library even less capable than bionic (obsoleted by +musl), klibc builds a random assortment of executables to run init scripts +with. There's no multiplexer command, these are individual executables:

+ +

+cat chroot cpio dd dmesg false fixdep fstype gunzip gzip halt ipconfig kill +kinit ln losetup ls minips mkdir mkfifo mknodes +mksyntax mount mv nfsmount nuke pivot_root poweroff readlink reboot resume +run-init sh sha1hash sleep sync true umount uname zcat +

+ +

To get that list, build klibc according to the instructions (I +looked at version +2.0.2 and did cd klibc-*; ln -s /output/of/kernel/make/headers_install +linux; make) then echo $(for i in $(find . -type f); do file $i | grep -q +executable && basename $i; done | grep -v '[.]g$' | sort -u) to find +executables, then eliminated the *.so files and *.shared duplicates.

+ +

Some of those binaries are build-time tools that don't get installed, +which removes mknodes, mksyntax, sha1hash, and fixdep from the list. +(And sha1hash is just an unpolished sha1sum anyway.)

+ +

The run-init command is more commonly called switch_root, nuke is just +"rm -rf -- $@", and minips is more commonly called "ps". I'm not doing aliases +for the oddball names.

+ +

Yet more stale forks of dash and gzip sucked in here (see "dubious +license terms" above), adding nothing to the other projects we've looked at. +But we still need sh, gunzip, gzip, and zcat to replace this package.

+ +

By the time I did the analysis toybox already had cat, chroot, dmesg, false, +kill, ln, losetup, ls, mkdir, mkfifo, readlink, rm, switch_root, sleep, sync, +true, and uname.

+ +

The low hanging fruit is cpio, dd, ps, mv, and pivot_root.

+ +

The "kinit" command is another gratuitous rename, it's init running as PID 1. +The halt, poweroff, and reboot commands work with it.

+ +

I've got mount and umount queued up already, fstype and nfsmount go with +those. (And probably smbmount and p9mount, but this hasn't got one. Those +are all about querying for login credentials, probably workable into the +base mount command.)

+ +

The ipconfig command here has a built in dhcp client, so it's ifconfig +and dhcpcd and maybe some other stuff.

+ +

The resume command is... weird. It finds a swap partition and reads data +from it into a /proc file, something the kernel is capable of doing itself. +(Even though the klibc author +attempted +to remove that capability from the kernel, current kernel/power/hibernate.c +still parses "resume=" on the command line). And yet various distros seem to +make use of klibc for this> +Given the history of swsusp/hibernate (and +TuxOnIce +and kexec jump) I've lost track +of the current state of the art here. Ah, Documentation/power/userland-swsusp.txt +has the API docs, and here's a better +tool...

+ +

So the list of things actually in klibc are:

+ +
+ +cat chroot dmesg false kill ln losetup ls mkdir mkfifo readlink rm switch_root +sleep sync true uname + +cpio dd ps mv pivot_root +mount nfsmount fstype umount +sh gunzip gzip zcat +kinit halt poweroff reboot +ipconfig +resume +
+
+ +

Stand-Alone Shell

+ +

Wikipedia has a good +summary of sash, with links. The original Stand-Alone Shell project reached +a stopping point, and then "sash plus +patches" extended it a bit further. The result is a megabyte executable +that provides 40 commands.

+ +

Sash is a shell with built-in commands. It doesn't have a multiplexer +command, meaning "sash ls -l" doesn't work (you have to go "sash -c 'ls -l'"). +

+ +

The list of commands can be obtained via building it and doing +"echo help | ./sash | awk '{print $1}' | sed 's/^-//' | xargs echo", which +gives us:

+ +
+alias aliasall ar cd chattr chgrp chmod chown cmp cp chroot dd echo ed exec +exit file find grep gunzip gzip help kill losetup losetup ln ls lsattr mkdir +mknod more mount mv pivot_root printenv prompt pwd quit rm rmdir setenv source +sum sync tar touch umask umount unalias where +
+ +

Plus sh because it's a shell. A dozen or so commands can only sanely be +implemented as shell builtins (alias aliasall cd exec exit prompt quit setenv +source umask unalias), where is an alias for which, and at triage time toybox +already has chgrp, chmod, chown, cmp, cp, chroot, echo, help, kill, losetup, +ln, ls, mkdir, mknod, printenv, pwd, rm, rmdir, sync, and touch.

+ +

This leaves:

+ +
+ +ar chattr dd ed file find grep gunzip gzip lsattr more mount mv pivot_root +sh sum tar umount + +
+ +

(For once, this project doesn't include a fork of gzip, instead +it sucks in -lz from the host.)

+ +
+ +

sbase:

+ +

It's on suckless. So far it's +implemented:

+ +

+ +basename cat chmod chown cksum cmp cp date dirname echo false fold grep head +kill ln ls mc mkdir mkfifo mv nl nohup pwd rm seq sleep sort tail tee test +touch true tty uname uniq wc yes + +

+ +

And has a TODO list:

+ +

+ +cal chgrp chvt comm cut df diff du env expand expr id md5sum nice paste +printenv printf readlink rmdir seq sha1sum split sync test tr unexpand unlink +who + +

+ +

At triage time, of the first list I still need to do: fold grep mc mv nl. Of +the second list: diff expr paste printf split test tr unexpand who.

+ +
+ +

s6

+ +

The website skarnet has a bunch +of small utilities as part of something called "s6". This includes the +s6-portabile-utils +and the s6-linux-utils. +

+ +

Both packages rely on multiple bespoke external libraries without which +they can't compile. The source is completely uncommented and doesn't wrap at +80 characters. Doing a find for *.c files brings up the following commands:

+ +
+ +basename cat chmod chown chroot clock cut devd dirname echo env expr false +format-filter freeramdisk grep halt head hiercopy hostname linkname ln +logwatch ls maximumtime memoryhog mkdir mkfifo mount nice nuke pause +pivotchroot poweroff printenv quote quote-filter reboot rename rmrf sleep +sort swapoff swapon sync tail test touch true umount uniquename unquote +unquote-filter update-symlinks + +
+ +

Triage: memoryhog isn't even listed on the website nor does it have +a documentation file, clock seems like a subset +of date, devd is some sort of netlink wrapper that spawns its command line +every time it gets a message (maybe this is meant to implement part of +udev/mdev?), format-filter is sort of awk's '{print $2}' function split out +into its own command, hiercopy a subset of "cp -r", maximumtime is something +I implemented as a shell script (more/timeout.sh in Aboriginal Linux), +nuke isn't the same as klibc (this one's "kill SIG -1" only with hardwared +SIG options), pause is a program that literally waits to be killed (I +generally sleep 999999999 which is a little over 30 years), +pivotchroot is a subset of switch_root, rmrf is rm -rf...

+ +

I see "nuke" resurface, and if "rmrf" wasn't also here I might think +klibc had a point. + +

+basename cat chmod chown chroot cut dirname echo env expr false +freeramdisk grep halt head hostname linkname ln +logwatch ls mkdir mkfifo mount nice +pivotchroot poweroff printenv quote quote-filter reboot rename sleep +sort swapoff swapon sync tail test touch true umount uniquename unquote +unquote-filter update-symlinks +
+ + +
+ +

nash:

+ +

Red Hat's nash was part of its "mkinitrd" package, replacement for a shell +and utilities on the boot floppy back in the 1990's (the same general idea +as BusyBox, developed independently). Red Hat discontinued nash development +in 2010, replacing it with dracut (which collects together existing packages, +including busybox).

+ +

I couldn't figure out how to beat source code out of +Fedora's current git +repository. The last release version that used it was Fedora Core 12 +which has a source rpm +that can be unwound with "rpm2cpio mkinitrd.src.rpm | cpio -i -d -H newc +--no-absolute-filenames" and in there is a mkinitrd-6.0.93.tar.bz2 which +has the source.

+ +

In addition to being a bit like a command shell, the nash man page lists the +following commands:

+ +

+access echo find losetup mkdevices mkdir mknod mkdmnod mkrootdev mount +pivot_root readlink raidautorun setquiet showlabels sleep switchroot umount +

+ +

Oddly, the only occurrence of the string pivot_root in the nash source code +is in the man page, the command isn't there. (It seems to have been removed +when the underscoreless switchroot went in.)

+ +

A more complete list seems to be the handlers[] array in nash.c:

+ +

+access buildEnv cat cond cp daemonize dm echo exec exit find kernelopt +loadDrivers loadpolicy mkchardevs mkblktab mkblkdevs mkdir mkdmnod mknod +mkrootdev mount netname network null plymouth hotplug killplug losetup +ln ls raidautorun readlink resume resolveDevice rmparts setDeviceEnv +setquiet setuproot showelfinterp showlabels sleep stabilized status switchroot +umount waitdev +

+ +

This list is nuts: "plymouth" is an alias for "null" which is basically +"true" (which thie above list doesn't have). Things like buildEnv and +loadDrivers are bespoke Red Hat behavior that might as well be hardwired in +to nash's main() without being called.

+ +

Instead of eliminating items +from the list with an explanation for each, I'm just going to cherry pick +a few: the device mapper (dm, raidautorun) is probably interesting, +hotplug (may be obsolete due to kernel changes that now load firmware +directly), and another "resume" ala klibc.

+ +

But mostly: I don't care about this one. And neither does Red Hat anymore.

+ +
+ +

Beastiebox

+ +

Back in 2008, the BSD guys vented some busybox-envy +on sourceforge. Then stopped. +Their repository is still in CVS, hasn't been touched in years, it's a giant +hairball of existing code sucked together. (The web page says the author +is aware of crunchgen, but decided to do this by hand anyway. This is not +a collection of new code, it's a katamari of existing code rolled up in a +ball.)

+ +

Combining the set of commands listed on the web page with the set of +man pages in the source gives us:

+ +

+[ cat chmod cp csh date df disklabel dmesg echo ex fdisk fsck fsck_ffs getty +halt hostname ifconfig init kill less lesskey ln login ls lv mksh more mount +mount_ffs mv pfctl ping poweroff ps reboot rm route sed sh stty sysctl tar test +traceroute umount vi wiconfig +

+ +

Apparently lv is the missing link ed and vi, copyright 1982-1997 (do not +want), ex is another obsolete vi mode, lesskey is "used to +specify a set of key bindings to be used with less", and csh is a shell they +sucked in, [ is an alias for test. Several more bsd-isms that don't have Linux +equivalents (even in the ubuntu "install this package" search) are +disklabel, fsck_ffs, mount_ffs, and pfctl. And wiconfig is a wavelan interface +network card driver utility. Subtracting all that and the commands toybox +already implements at triage time, we get:

+ +

+ +fdisk fsck getty halt ifconfig init kill less mksh more mount mv ping poweroff +ps reboot route sed sh stty sysctl tar test traceroute umount vi + +

+ +

Not a hugely interesting list, but eh.

+

Requests:

-- cgit v1.2.3