From cccb1741140f9b3258294f77cbced98d3f6f16a7 Mon Sep 17 00:00:00 2001 From: Rob Landley Date: Mon, 11 May 2020 14:14:27 -0500 Subject: Release notes for 0.8.3. --- www/news.html | 257 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 257 insertions(+) mode change 100755 => 100644 www/news.html diff --git a/www/news.html b/www/news.html old mode 100755 new mode 100644 index 5497fdf4..c87223b8 --- a/www/news.html +++ b/www/news.html @@ -8,6 +8,263 @@ a development environment. See the links on the left for details.

News

+

May 11, 2020

 +
+

Ford: Ah. A +towel. +Keep this and guard it with your life.

+

Arthur: Huh?

+

Ford: Listen, it's a tough universe. There's all sorts of people and things +trying to do you, kill you, rip you off, everything. If you're going to +survive out there, you've really got to know where your +towel is.

+

- The Hitchhiker's Guide to the Galaxy.

+
+ +

Despite everything Toybox 0.8.3 +(git commit) +is finally out, with new commands rtcwake from Elliott Hughes and +blkdiscard from Patrick Oppenlander. +The big news is "make root" now boots to a shell prompt, +with toysh making it all the way through toyroot's init script. +(Some people were looking forward to patch --fuzz support too.)

+ +

Toyroot: make root now does what it says on the tin, it +builds a bootable toybox-based Linux system using two source +packages (toybox and linux). The trivial version is "make root && sudo chroot +root/host/fs /init". Here's +a post with instructions if you want to know how to build the +cross compilers for testing the various architectures. The self-contained +script +that builds a simple bootable Linux system is 250 lines long, and +should be easy to read if you want to know how it works.

+ +

It also has basic module support, meaning arguments that aren't X=Y variable +assignments are the names of scripts to run to build more +packages at the end of the build. I checked in an example package, +and there's generic "download and extract source tarballs" +plumbing available to such modules. +(The Makefile doesn't know how to pass module names through to the script, +so instead of "make root" you have to +call the script directly, ala "scripts/mkroot.sh CROSS=sh4 LINUX=~/linux dropbear".)

+ +

The resulting root filesystem now uses /root as the home +dir for UID 0, and creates /dev/fd and /dev/shm in devtmpfs. The build works +around a kernel +build bug where "make distclean" doesn't work in a "cp -sfR" symlink +directory. (The bug meant if you pointed LINUX= at unclean source, it was +unhappy, so now it distcleans the source directory before copying it. This +modifies said source directory, which is not ideal, but as usual the kernel guys +ignored +the bug report, so a workaround was required.) +The CROSS=all build announces each target in the title bar, puts +its logs into root/log, and has better trap handling to stop with one +ctrl-c. +Since last release it uses a more concise config format for the various kernel +arch targets (which shrank the script by about 50%), and merged +the old (now removed) scripts/cross.sh into the main script so +"make CROSS=armv7l LINUX=~/linux" is +literally just a call to "scripts/mkroot.sh CROSS=armv7l LINUX=~/linux" now.

+ +

As for scripts/mcm-buildall.sh building cross compilers, +of COURSE gcc 8.3 requires a different configuration to build the same +toolchain as previous versions, it's gcc. It now builds a proper +nommu libc for sh2eb without a broken fork() call in it that can never +be used but prevents compile-time probes from detecting nommu, +and checks that the cross compiler completed before trying to build +the native compiler.

+ +

This was all tested with a recent +version +of musl-cross-make with the top level Makefile +edited to use BINUTILS_VER = 2.32 because the newer one +breaks the kernel +build +and LINUX_VER = 4.19.90 because the default musl-cross-make config uses an +out-of-tree headers package for some reason (those who forget history are +doomed to repeat it) which +breaks m68k and s390x. I won't post binaries of the resulting toolchains +because they're GPLv3, but thalheim might. +(I've also test built +with the Android NDK, but bionic's startup code currently segfaults before +calling main() if it can't open /dev/null, and the kernel guys +ignored my patches to +make CONFIG_DEVTMPFS_MOUNT work in initramfs.)

+ +

At the moment toyroot cheats and uses two commands out of pending: toysh +is about 80% of the way to being useful but still missing some +obvious features like function support, job and terminal control, command +history, $((math)), wildcard expansion... plus a lot of bash features like array +variables, so it isn't out of pending yet. And route needs to be redone to use the +netlink interface that can handle multiple routing tables. For the moment +scripts/mkroot.sh adds both of them to defconfig if you haven't already +got a .config when you run it. (If you build and can't boot, your .config +probably hasn't got CONFIG_SH switched on. Fix it in menuconfig, or +rm .config and try gain.)

+ +

New toybox features: +Elliott taught patch to understand [FILE [PATCH]] arguments, made +cal highlight the current day, added -T to cp/mv, +did a lot of work on lsattr/chattr (including adding -p +and "chattr ="), added tar --absolute-names, taught id to +support numeric lookup and handle unknown groups, made -G show all +groups, and removed context= from -Z. +Rob added patch -F (fuzz factor) support and help -u (usage only), +taught echo -e about bash extensions like \uXXXX unicode escapes, +netcat -L no longer automatically includes stderr (new -E +option does that), +setsid now uses -c (like the man page says) instead of -t, and added +-w (wait) and -d (detach from tty). +Andrew Ilijic added ls -w, removed trailing whitespace on output, +and added tests for -C and -x.

+ +

Library: +another fix to 32 bit option parsing of long long optflag values, +remove getdirname() and just use the libc function (which modifies +its argument, but we don't have to free a malloc), +dlist_terminate() can now be called repeatedly on the same list, +new relative_path() function finds path from one directory to another, +and readfd() works like readfile() but on an already open fd. +Use MPATH macros in mkpathat(). Elliott added macOS versions of +dev_minor()/dev_major()/makedev() to portability.c, +implemented posix_fallocate() for macOS, removed a +leftover uClibc workaround in fallocate that was breaking macOS, +and moved the table of ELF architectures from file to lib/lib.c (so +readelf can share). +Park Ju Hyung pointed out the fast path of fdlength() was commented out, +and we switched it to the 64 bit API while we were there (and then Elliott +added macOS support). +Joeky taught file to recognize 7z archives.

+ +

Pending: +The shell got a bunch of work: standalone "make sh" now includes the +multiplexer for builtin commands like "exit" and "cd". Added MAYFORK annotation +for commands that exist in the $PATH but can also be run within the shell +process (and sometimes have different behavior within the shell): +currently applied to help, echo, false, kill, printf, pwd, test, time, and +true.

+ +

Elliott added new commands getopt and readelf. +Jarno Mäkipää taught wget how to follow http 301 and 302 redirects, +and did lots of work on vi. (Elliott also did work on vi.) +Gavin Howard fixed a comparison bug in bc. +Ethan Sommer fixed numerous small issues (including several build +warnings and FLAG() macro conversions).

+ +

Bugfixes: +tar extract now deletes files and symlinks where it's making a directory, +find -L -type -l now finds dangling symlinks, extra #ifdefs in +portability.h prevent old gcc versions from barfing on __has_include(), +xgetline() now returns NULL at EOF, tee with no arguments was +writing to stdout twice, setsid works on nommu (I.E. vfork) now, +netcat -L no longer accumulates zombie processes, +sntp now uses adjtimex instead of adjtime (to build on bionic). +xcreate_stdio() was checking WARN_ONLY in the wrong field (and tar was +passing it in the wrong field, so it worked there). +Several people wrestled with the xargs "argument too long" problem. +Alessio Balsini fixed memory leaks in loopback_setup() and "wayling" +added a missing continue to losetup. +Elliott fixed xargs -E, various things in modinfo, +added an error check to gzip when using zlib (which copies non-gzip data to +the output verbatim for some reason), +found an case where dirtree could use +uninitialized data and silenced "Invalid argument" warnings (triggered by +Android's selinux policies making stat() and readlink() fail), +fixed locale support in macOS (both in iconv +and in main.c), taught stat to show filesystem time on macOS, +fixed a 32 bit truncation in sntp, fixed a memory access one byte outside +of its array in patch.c, removed the cpio --trailer option, +and widened the pid display fields in ps to +6 digits. Rich Felker fixed find.c assuming +time_t is a long (y2038 issue on 32 bit). +Greg Kaiser fixed a thinko in get_block_device_size(). +Jarno Mäkipää fixed utf8 support in cut -C, and cp --parents. +David Legault complained that dir/.* tells rm to delete dir/.. and we'd do it. +JakeSFR pointed out a bug in file's identification of broken symlinks. +William Haddon fixed cp to treat a directory with a trailing slash +teh same as one without. +Denys Nykula fixed rm -i not to prompt for an empty "" argument. +SebiderSushi reported that chmod g+s wasn't working. +The linux kernel doesn't let an O_PATH fd work with fgetxattr(), so +Elliott switched ls to use the path-based functions now (which is racy, +it means you can stat() one inode and get the xattrs for a different one, +but nobody in kernel land seems to use xattrs much so they're not fully +supported by the API, and who's crazy enough to use xattrs for security +anyway). Derick Pallas pointed out an xclearenv() bug. +Atatsulo (or possibly luolongjuna) did a lot of work on the ping +command: pointed out min/range/max were out of order, +that we shouldn't print a summary unless we received at least one +reply packet, and implemented ttl support.

+ +

Build plumbing: +The "make root CROSS=all" build announces each target in the title bar, puts +its logs into root/log, and has better trap handling to stop with one +ctrl-c.

+ +

Fixed scripts/single.sh to work when PREFIX has no trailing slash, +make silentoldconfig no longer feeds "y" to kconfig (which puts menus in a +loop) and instead just feeds in newlines to accept whatever the default is. +The non-git version number I keep forgetting to update each release moved +from main.c to toys.h. +Fixed a couple different errors in mkflags.c (one of which caused ls --color +to set all the other flags).

+ +

Elliott added more macos support and tests, and added +fallocate, cp, mktemp, and mv to the macOS defconfig.

+ +

Cleanup: +Rob did some cleanup on xargs, ls, md5sum, and sort, tidied up main.c a bit, +made cp, base64, dmesg, and free use FLAG macros, +and switched fallocate to new style global names. +Elliott Hughes switched rfkill from the old byte at a time get_line() +to libc getline(), made du use FLAG() macros, and mad chattr use +standard toybox argument parsing for -v and -p. +Several commands (help, cp) had sub-options removed from menuconfig. +Merged realpath into readlink.c, and use xabspath() instead of libc realpath().

+ +

Documentation: +New roadmap section about other packages +that toybox can (eventually) replace.

+ +

Update roadmap to note that posix-2008 moved to a different URL (content +at the old URL undergoes random changes), and link to the IETF RFCs. +design.html explains more of the history of environment sizes on links +and has a #bits anchor tag.

+ +

Update the LP64 section of design.html to +fish the documents out of archive.org now that unix.org is gone, +and show the actual size table locally.

+ +

Elliott improved the help of date and chattr, and +made the usage: lines in the posix directory more consistent.

+ +

Shrank the sed help from 150 lines to 90-ish.

+ +

Tests: +New "txpect" performs interactive tests, running through a sequence of +writes to a command's stdin and reads from stdout and stderr, when failing it +reports the first non-matching step. (This for example lets sh.test check +the shell line continuation logic prompts with $PS1 and $PS2 appropriately +with various unfinished input lines, and that "echo hello; if" doesn't +output hello before prompting for the next line of input.) Added +VERBOSE=xpect to print out each read and write successfully performed by txpect.

+ +

runtest.sh only creates an "input" file when the input argument isn't empty, +EVAL doesn't supply -- (you have to provide your own if you want that).

+ +

Rob added basic stat, patch, and tee tests. +Jarno added a bunch of vi tests. +Elliott fixed tests for ifconfig, lsattr, chattr, +and date, added tests for find, id, xargs, and made +the id, iconv, env, file, printf and cat tests work on macOS (and skipped +the du tests there). Rob cleaned up chmod tests.

+ +

Sheer pedantry: +true and false now have usage: lines (which you have to "help false" to see +because they (intentionally!) don't support --help. +Renamed get_chunk()/dump_chunk() to read_chunk()/write_chunk() in tail.c +Elliott fixed some typos.

+

October 18, 2019

"In those days spirits were brave, the stakes were high, men were real -- cgit v1.2.3