aboutsummaryrefslogtreecommitdiff
path: root/toys/posix
AgeCommit message (Collapse)Author
2019-08-04Add TOYFLAG_MAYFORK and annotate a couple commands.Rob Landley
A TOYFLAG_NOFORK command must run in the context of toysh, but a MAYFORK can either run standalone or run in the toysh process. MAYFORK means it cleans up after itself: no leaked resources (malloc, mmap, filehandles, etc), even in error_exit() paths that would longjmp() back to the shell. It also doesn't discard anything we need to retain (don't close stdout, change toys.optargs[] so we can't free it, etc)...
2019-07-31Move the empty regex workaround into xregcomp.Elliott Hughes
No current caller except grep needs this, but consistency seems like a good idea. Also change the xregcomp error message to be a bit more human-readable, rather than mention an implementation detail.
2019-07-30Fix signed typecast bug.Rob Landley
We use (char *)1 and (char *)2 to indicate errors (they can never be valid pointers because both malloc() and mmap() return aligned memory and those align down to NULL, plus Linux maps 4k at the bottom to catch null dereferences anyway), and then typecast it to long (trusting in LP64) to do an integer <=2 comparison... except that needs to be UNSIGNED long or else pointers in the top half of the virtual memory space become negative and the <=2 false positives them as errors. (Oops.)
2019-07-29grep: fake GNU behavior for non-POSIX empty regex.Elliott Hughes
POSIX says there's no such thing as an empty regular expression. The grammar excludes the possibility: https://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap09.html BSD agrees with POSIX, and Android and macOS' BSD-based implementations reject the empty regular expression. GNU apparently disagrees. Luckily, BSD does accept the empty *sub* expression `()`, despite their error message for REG_EMPTY being "empty (sub)expression". This is presumably a bug, except there's explicit code to support it that is at least 26 years old: https://github.com/freebsd/freebsd/blame/master/lib/libc/regex/regcomp.c#L383 This workaround also works fine with glibc. If we want GNU behavior, I'm struggling to come up with another way to fake it. If we want POSIX behavior, we could easily just add a check to reject "" on glibc. Also switch to xregcomp().
2019-07-23nl: switch from getline() to loopfiles_lines().Elliott Hughes
This was basically just to help me think out loud while discussing loopfiles_lines().
2019-07-22Start replacing get_line() with getline().Elliott Hughes
I started this last night, but thought I'd aim to send multiple small patches rather than work through all the callers and send one big patch. I've deliberately chosen the ugly name `allocated_length` because we've had historical bugs where folks think this a line length in the sense of the return value. I do wonder whether we should actually have some kind of getline() wrapper that hides the `char *`/`size_t` pair in lib/, which makes the function easier to use in most cases but does add the less common gotcha that you wouldn't be able to getline() through multiple files at once (which does happen in at least one toy). But maybe the real fix is to look harder for places where we can just use loopfiles_lines? Speaking of which, should we actually add two more arguments to that? Specifically: switch it to getdelim() rather than getline() behind the scenes, and also add a way to have the trailing '\n' automatically removed, since that seems to be what most callers want? Anyway, that seemed like enough questions that it was time to send this initial patch out before doing too much more...
2019-07-19xargs: don't pretend to support -I.Elliott Hughes
This was embarrassing... A build script that was using xargs -I silently started to ignore -I when we switched over to toybox xargs. The owner of the script has rewritten it to use the shell read builtin instead, but it's pretty unhelpful to silently ignore an option that radically changes how xargs behaves. (The -I behavior sounds sufficiently different from normal behavior that I'm not sure I've understood, and am just sending this cleanup rather than actually implementing -I correctly.) Bug: http://b/137832162
2019-07-16Fix unaligned access, tweak test suite.Rob Landley
2019-07-16grep: fix two bugs found by hwasan.Elliott Hughes
The first bug appeared as a memory overwrite, but was actually visible without hwasan: basically any `grep -F` that let to multiple matches on the same line was broken. The second bug was another memory overwrite, visible when I ran the existing grep tests. Bug: http://b/137573082
2019-07-12grep: add -R as well as -r.Elliott Hughes
On BSD these are actually the same, and there's a -S that you need in addition. So strictly this is a behavior change for Android (which is going from BSD grep to toybox grep), but it's a behavior preserving change for the AOSP build (which is going from GNU grep to toybox grep), and the latter actually has a checked-in use of -R where the former doesn't.
2019-07-11xargs: bug compatibility with BSD/busybox/findutils.Elliott Hughes
The other xargs implementations in the wild don't seem to count the space taken by each `char *`. Go for bug compatibility unless future experience proves that to be a bad idea.
2019-07-10dd: iflags, oflags, fix ^C, fix the fundamental loop.Elliott Hughes
Investigating why the toybox tar tests fail on Android with toybox dd, I realized I was rewriting a part of dd I'd rewritten before! This is a re-send of my 2019-02-22 patch, rebased against the current ToT... This patch was originally motivated because after suggesting to the author of https://stackoverflow.com/questions/17157820/access-vdsolinux/54797221#54797221 that he could tell dd to work in bytes rather than blocks, I realized that our dd doesn't actually support that. But the rewrite of the main loop is necessary to fix the incorrect output from the dd calls in the tar test. Without this patch, `yes | dd bs=65536 count=1 > fweep` basically gives random output, based on how many bytes the pipe feels like giving you in your first read. (As far as I know, dd *without* bs= was fine, but I can't guarantee that that's true, just that I haven't seen it fail.) Also switch to TAGGED_ARRAY and comma_* for conv rather than add two more copies of an undesired idiom. It turned out -- contrary to the belief of cp(1) -- that comma_scan isn't suitable for this because of its magic handling of "no" prefixes. (It's actually harmless in cp because none of the --preserve options begin with "no", but some dd options do.) To this end, comma_remove is a less-magic comma_scan. I've also changed an `if` to a `while` because other implementations allow things like `--preserve=mode,mode` or `conv=sync,sync`. (If we decide this is a bug rather than a feature, we should at least fix the error message to be clear that we're rejecting the *duplication*, not the option itself.) I've also fixed the ^C behavior by simply adding a direct SIGINT handler rather than trying to be clever inside the read loop (which is why we weren't handling the SIGINT until the read returned). I've also removed `strstarteq` and just added the '=' to each literal when calling regular `strstart`. Plus basic tests.
2019-07-06file: fix xz, add a few other Android types.Elliott Hughes
"xz compressed data" was missing a newline. Also switch a couple of other unnecessary (but correct) xprintf calls over to xputs. Also add .otf fonts, perf/simpleperf data files, and Android boot images, sparse images, and device tree blobs/overlays.
2019-07-06Add comment and test.Rob Landley
2019-07-06Have env exec, not recurse to builtin.Denys Nykula
My app runs npm i -S better-sqlite3, and node-gyp-bin from the npm upstream has #!/usr/bin/env sh, expecting the system shell.
2019-07-03nl: nl -vEric Molitor
Implement nl -v supporting negative and zero starting values.
2019-06-28file, stat: various small improvements.Elliott Hughes
file now shows the target of a symbolic link and calls out broken symbolic links. file now shows the device type for block/character special files. file now shows specific reason when it can't open. stat now includes the device type, plus a little more space between the number of blocks and the human-readable file type. Adjusted tests accordingly, which actually makes more of them pass on the host as a convenient side-effect, but I actually made these changes because I've been finding the desktop file and stat output more convenient in these cases.
2019-06-25tar: implement --mode.Elliott Hughes
Used by build/make/tools/mktarball.sh in AOSP. (Which is why today's switch to toybox tar got reverted.)
2019-06-23Switch cp.c to FLAG() macros.Rob Landley
2019-06-22Use FORCE_FLAGS in kill.c.Elliott Hughes
Otherwise kill flags don't work unless killall5 is also configured. Also switch to the FLAG() macro.
2019-06-22Fix mv with trailing slash in source.Denys Nykula
Press tab, have bash complete dir name with a slash, notice musl rename() dislikes that. Replace trailing slash in the cp loop with a null character, if the command name is mv. Add the slash back if an error occurs.
2019-06-21Fix cp -r dir/. symlink child.makepost
Remove the existing link before trying to re-create, passing the test. Add -p to the -r test as a regression guard, portage calls cp with both.
2019-06-11find: add -true/-false.Elliott Hughes
Used near the end of the AOSP build. Almost there! (This patch also fiddles with the help text to be able to slip the new options in without requiring so much extra space.)
2019-06-10Improve signal name<->number conversions.Elliott Hughes
Include all the signals, not just the POSIX ones. In particular, improve support for real-time signals. My attempt to switch AOSP builds over to toybox timeout got reverted when I broke the ART build bots which use SIGRTMIN+2. Also fix `kill -l 3` to show "QUIT" and `kill -l QUIT` to show "3". Also make the `kill -l` output include numbers and names, and format it to look better on a 80x24 tty, because it's always August 1978 somewhere.
2019-06-08macOS: numerous fixes.Elliott Hughes
This patch adds a BSD version of xgetmountlist (for the path == NULL case only), tested on macOS. It also papers over the differences between macOS' and Linux's xattr APIs. For once I think the macOS one is better. The imitation of mknodat I've had to write swings things back in Linux's favor though. BSD calls f_frsize by the name f_iosize instead. (FWIW, it looks like this is meaningless on Linux and actually meaningful on macOS.) I've added one #if to toys/ --- I'm calling pathconf in stat.c to work around the absence of f_namelen, and have left a TODO with an explanation. I'm not sure what the best fix is here, so punting. No-one can agree what f_fsid is, even if they're all basically the same, so work around the `val` versus `__val` issue between macOS and Linux. With this patch, it's now possible to build cp/mv/install and stat for macOS too. (Which completes the set of "toybox commands currently used on Linux as part of the AOSP build" if you ignore stuff that deals with processes, which I doubt we'll ever be able to support for lack of any API.)
2019-06-07Implement stat %C and find %Z.Elliott Hughes
We'd documented find %Z but not implemented it. We'd neither documented nor implemented stat's corresponding %C (they'd already taken %Z for ctime, which is ironic because %c/%C sounds more obvious than %z/%Z for that to me).
2019-06-03Remove TAIL_SEEK config option, just always include it.Rob Landley
Yeah, it's twice the size and two codepaths, but seekable is the common case, it won't work in pipelines without a non-seek codepath, and the performance penalty not using seek on large files is a enormous.
2019-06-01Teach file to recognize xz archives and old style tarballs.Rob Landley
2019-06-01Teach tar to extract older tarballs.Rob Landley
2019-05-29Teach find -printf about %.Ns patterns, tweak help text, add tests.Rob Landley
2019-05-26Teach sed about +N range ending.Rob Landley
2019-05-25find: add -printf support.Elliott Hughes
This only implements the format specifiers that I've seen used in the wild (which is actually a significant fraction of the total supported by findutils' find). The most obvious gap is in the time support. I'm happy to add more, but didn't want to add stuff "just because". I'd say %A@, %C@, and -- for SELinux users -- %Z are probably the most plausibly useful formats still missing. I don't think the human-readable date formatting is particularly useful unless someone's seen it actually used in the wild. The %T+ "full ISO" format being the most likely exception to that. Anyway, this is enough for me get started building AOSP with toybox find.
2019-05-25grep: add --exclude-dir.Elliott Hughes
Used quite a lot, especially with `--exclude-dir=.git`.
2019-05-25time: fix -v output.Elliott Hughes
Copy-and-paste mistake from the regular output. Bug: http://b/133502489
2019-05-25Move notify.c into portability.c (collate the #ifdefs), move global variablesRob Landley
into a structure passed as arguments, add x prefix to functions that can fail, add overflow test.
2019-05-25tail: support -f on BSD too.Elliott Hughes
Factor out the inotify code and add a kqueue equivalent. Specifically tested on macOS 10.14, but I assume this works for other BSDs too, given that I worked from the FreeBSD man page...
2019-05-25Toybox doesn't provide bzip2 or xz compression side (and the roadmap has it asRob Landley
out of scope), but the AOSP build airlock doesn't provide bzcat and friends. So tar needs to be able to use both: check for *zcat first, and fall back to "compressor -d" if it's not there.
2019-05-23tar: turns out --sparse is also known as -S.Elliott Hughes
Found trying to build the aosp_cf_x86_phone-userdebug target. The good news is that the targets that I knew were using tar with sparse files all pass now.
2019-05-20Less incomplete tar help text.Rob Landley
2019-05-19Fix tar sparse extract with extension blocks.Rob Landley
2019-05-19Fix tar --sparse generation of extension block.Rob Landley
2019-05-19Add pointless trailing --sparse entry for exact binary compartibility.Rob Landley
There's no possible use for this, but debian's tar produces it, so...
2019-05-19Add --sparse file creation support to tar.Rob Landley
And fix tar cv to produce output to stderr when archive going to stdout.
2019-05-17Fix a missing else, and an inverted test hidden by the missing else.Rob Landley
Add test to show failure case.
2019-05-16Have ps.c human_readable fields use all the available space.Rob Landley
2019-05-16Use the space reserved in the header to display more precision.Rob Landley
2019-05-16Minor code shrink.Rob Landley
2019-05-14Checked in a debugging printf. Oops.Rob Landley
2019-05-14Typo caused a segfault. (Oops.)Rob Landley
2019-05-13Use FLAG() macros in lsRob Landley