aboutsummaryrefslogtreecommitdiff
path: root/lib
AgeCommit message (Collapse)Author
2019-08-04Add TOYFLAG_MAYFORK and annotate a couple commands.Rob Landley
A TOYFLAG_NOFORK command must run in the context of toysh, but a MAYFORK can either run standalone or run in the toysh process. MAYFORK means it cleans up after itself: no leaked resources (malloc, mmap, filehandles, etc), even in error_exit() paths that would longjmp() back to the shell. It also doesn't discard anything we need to retain (don't close stdout, change toys.optargs[] so we can't free it, etc)...
2019-07-31Move the empty regex workaround into xregcomp.Elliott Hughes
No current caller except grep needs this, but consistency seems like a good idea. Also change the xregcomp error message to be a bit more human-readable, rather than mention an implementation detail.
2019-07-31Fix error path and tweak comments.Rob Landley
2019-07-24Avoid double-close of fd in loopfiles_lines.Elliott Hughes
Test: no EBADF in `strace -e close ./toybox tac /proc/version`
2019-07-22Start replacing get_line() with getline().Elliott Hughes
I started this last night, but thought I'd aim to send multiple small patches rather than work through all the callers and send one big patch. I've deliberately chosen the ugly name `allocated_length` because we've had historical bugs where folks think this a line length in the sense of the return value. I do wonder whether we should actually have some kind of getline() wrapper that hides the `char *`/`size_t` pair in lib/, which makes the function easier to use in most cases but does add the less common gotcha that you wouldn't be able to getline() through multiple files at once (which does happen in at least one toy). But maybe the real fix is to look harder for places where we can just use loopfiles_lines? Speaking of which, should we actually add two more arguments to that? Specifically: switch it to getdelim() rather than getline() behind the scenes, and also add a way to have the trailing '\n' automatically removed, since that seems to be what most callers want? Anyway, that seemed like enough questions that it was time to send this initial patch out before doing too much more...
2019-07-13net.c: add missing static.Elliott Hughes
Pointed out during code review of the recent refactor.
2019-07-12pidof: fix default behavior, add -x.Elliott Hughes
Before this patch, we're effectively doing `pidof -x` all the time. This patch changes names_to_pid() to allow us to say whether or not we want to include scripts, and adjusts the callers appropriately. Also add tests for `pidof` versus `pidof -x` which pass after this patch, without regressing the existing killall tests.
2019-07-11bind/connect cleanup.Elliott Hughes
Rename the existing xbind/xconnect to xbindany/xconnectany, to make room for new xbind/xconnect that are more like 'x' versions of the regular bind and connect. Move explicit bind/connect callers over to xbind/xconnect. Of the affected commands, only netcat is actually used by Android. It was the most recent patch to netcat that made the lack of a more traditional xbind/xconnect apparent.
2019-07-10dd: iflags, oflags, fix ^C, fix the fundamental loop.Elliott Hughes
Investigating why the toybox tar tests fail on Android with toybox dd, I realized I was rewriting a part of dd I'd rewritten before! This is a re-send of my 2019-02-22 patch, rebased against the current ToT... This patch was originally motivated because after suggesting to the author of https://stackoverflow.com/questions/17157820/access-vdsolinux/54797221#54797221 that he could tell dd to work in bytes rather than blocks, I realized that our dd doesn't actually support that. But the rewrite of the main loop is necessary to fix the incorrect output from the dd calls in the tar test. Without this patch, `yes | dd bs=65536 count=1 > fweep` basically gives random output, based on how many bytes the pipe feels like giving you in your first read. (As far as I know, dd *without* bs= was fine, but I can't guarantee that that's true, just that I haven't seen it fail.) Also switch to TAGGED_ARRAY and comma_* for conv rather than add two more copies of an undesired idiom. It turned out -- contrary to the belief of cp(1) -- that comma_scan isn't suitable for this because of its magic handling of "no" prefixes. (It's actually harmless in cp because none of the --preserve options begin with "no", but some dd options do.) To this end, comma_remove is a less-magic comma_scan. I've also changed an `if` to a `while` because other implementations allow things like `--preserve=mode,mode` or `conv=sync,sync`. (If we decide this is a bug rather than a feature, we should at least fix the error message to be clear that we're rejecting the *duplication*, not the option itself.) I've also fixed the ^C behavior by simply adding a direct SIGINT handler rather than trying to be clever inside the read loop (which is why we weren't handling the SIGINT until the read returned). I've also removed `strstarteq` and just added the '=' to each literal when calling regular `strstart`. Plus basic tests.
2019-07-10Add dlist_lpop() to remove last entry (use dlist as stack).Rob Landley
2019-07-06killall: better handling of long names.Elliott Hughes
Change names_to_pid() so that we can actually match shell scripts with long names (the code to get the shell script's name was correct, but there was an extra test preventing us from actually comparing it to the sought name). In kill.c itself, remove a dead test for -l and switch to the FLAG() macro. Also extend the tests to explicitly cover long and short names.
2019-06-25lib/args.c: add quotes.Elliott Hughes
This failure was a little hard to parse: tar: Unknown option mode (see "tar --help") This would have been clearer: tar: Unknown option 'mode' (see "tar --help")
2019-06-20The O_PATH breaks stuff, have to work around Android's selinux rules for /Rob Landley
some other way.
2019-06-20Teach dlist_pop() to work on a dlist_terminate()d listRob Landley
2019-06-11Add DIRTREE_STATLESS to return entries we couldn't stat().Rob Landley
Sets ->again |= 2 when that happens and ->st is zeroed. While we're there, don't memset(st) and then memcpy(st) over it, and use O_PATH in the open().
2019-06-11Move sig_to_num and num_to_sig into portability.c.Elliott Hughes
macOS doesn't have real-time signals, and it has a slightly different set of non-portable signals from Linux. With this, the toybox `kill -l` output matches the default macOS kill(1).
2019-06-10Improve signal name<->number conversions.Elliott Hughes
Include all the signals, not just the POSIX ones. In particular, improve support for real-time signals. My attempt to switch AOSP builds over to toybox timeout got reverted when I broke the ART build bots which use SIGRTMIN+2. Also fix `kill -l 3` to show "QUIT" and `kill -l QUIT` to show "3". Also make the `kill -l` output include numbers and names, and format it to look better on a 80x24 tty, because it's always August 1978 somewhere.
2019-06-08macOS: numerous fixes.Elliott Hughes
This patch adds a BSD version of xgetmountlist (for the path == NULL case only), tested on macOS. It also papers over the differences between macOS' and Linux's xattr APIs. For once I think the macOS one is better. The imitation of mknodat I've had to write swings things back in Linux's favor though. BSD calls f_frsize by the name f_iosize instead. (FWIW, it looks like this is meaningless on Linux and actually meaningful on macOS.) I've added one #if to toys/ --- I'm calling pathconf in stat.c to work around the absence of f_namelen, and have left a TODO with an explanation. I'm not sure what the best fix is here, so punting. No-one can agree what f_fsid is, even if they're all basically the same, so work around the `val` versus `__val` issue between macOS and Linux. With this patch, it's now possible to build cp/mv/install and stat for macOS too. (Which completes the set of "toybox commands currently used on Linux as part of the AOSP build" if you ignore stuff that deals with processes, which I doubt we'll ever be able to support for lack of any API.)
2019-06-07Remove getprop, setprop, start, and stop from toyboxTom Cherry
These are Android specific, so not really helping the outside community, and are getting more and more Android dependencies to work correctly, so let's drop these from toybox and build them within Android. Change-Id: Ic6022f1f506e10868c61f55d64fa4e7c1b14eba2
2019-06-03tail: fix macOS build.Elliott Hughes
Trivial breakage from the recent refactoring. Also add tail to the default config for macOS. I'm still unconvinced that CONFIG_TAIL_SEEK makes sense in a world where inotify support is always built in, but that's an argument for another day...
2019-06-01Teach file to recognize xz archives and old style tarballs.Rob Landley
2019-05-25find: add -printf support.Elliott Hughes
This only implements the format specifiers that I've seen used in the wild (which is actually a significant fraction of the total supported by findutils' find). The most obvious gap is in the time support. I'm happy to add more, but didn't want to add stuff "just because". I'd say %A@, %C@, and -- for SELinux users -- %Z are probably the most plausibly useful formats still missing. I don't think the human-readable date formatting is particularly useful unless someone's seen it actually used in the wild. The %T+ "full ISO" format being the most likely exception to that. Anyway, this is enough for me get started building AOSP with toybox find.
2019-05-25Move notify.c into portability.c (collate the #ifdefs), move global variablesRob Landley
into a structure passed as arguments, add x prefix to functions that can fail, add overflow test.
2019-05-25tail: support -f on BSD too.Elliott Hughes
Factor out the inotify code and add a kqueue equivalent. Specifically tested on macOS 10.14, but I assume this works for other BSDs too, given that I worked from the FreeBSD man page...
2019-05-19Don't complain about short writes to stdout.Rob Landley
2019-05-16Add human_readable_long() for more than 3 digits of output.Rob Landley
2019-05-13Comment tweak.Rob Landley
2019-05-12Fix readlink0() to blank string on failure.Rob Landley
2019-05-11Teach tar to extract type 'S' sparse file headers.Rob Landley
2019-05-06Greatly simplify and speed up regexec0() using REG_STARTEND.Rob Landley
This is a 15 year old freebsd extension (presumably thus also available on MacOS) that glibc adopted in 2004, uClibc adopted in 2005, and bionic supports. The only thing that DOESN'T support it is musl, once again because its maintainer explicitly decided not to (https://www.openwall.com/lists/musl/2013/01/15/26), so add an #ifdef to let musl stay uniquely broken. (It'll stop at first NUL, everything else can match NULs). Finally fixes "s/x/y/g on a megabyte line of x's takes forever" issue.
2019-05-04Optimize regexec0() for long lines.Rob Landley
Don't strlen() to find NUL to skip to until after we've confirmed first section hasn't got a match (by calling regexec() on it).
2019-05-03Add a couple comments.Rob Landley
2019-05-03env: fix case where a variable is replaced.Elliott Hughes
Found when trying to update the toybox prebuilt used for the Android build. Also add the corresponding test.
2019-05-02Android moved the scheduler policy functions in Q.Elliott Hughes
They're forwarded to libprocessgroup, but we may as well go straight to the source since neither library is in the NDK anyway. This code is unfortunate because it means that even `toybox true` ends up pulling in a JSON parser at runtime, because ps might call get_sched_policy/get_sched_policy_name. I'll experiment with dlopen-on-demand in portability.c and see whether the savings are worthwhile, but for now at least use the current library directly so we can save *one* dlopen!
2019-04-22struct deflate no longer lives in TT so must be explicitly zeroed, and useRob Landley
libbuf instead of toybuf in lib/
2019-04-21Shouldn't need clearenv() for macosx now we've got xclearenv.Rob Landley
2019-04-20Add xunsetenv() for the error checking.Rob Landley
2019-04-20New xsetenv() plumbing (repeatedly set same environment variables withoutRob Landley
leaking memory), and mod env command to test it.
2019-04-16Add argument to xflush() so it can test for stdout err without flushing.Rob Landley
2019-04-12Make help text spacing consistent.Rob Landley
2019-04-11scan_key: support more terminals.Elliott Hughes
Although we can get away with ignoring termcap/terminfo on the output side by restricting ourselves to generally-supported escape sequences, the input side is trickier because we need to support the sequences sent by common terminals. Luckily, this isn't is as bad as it sounds because only Home/End commonly differ. But it does mean we need a slightly different implementation to deal with the many-to-one mapping. Since we can't use TAGGED_ARRAY for this (without inflicting pain on all the callers) I've also switched to OR-ing in the modifier keys, so we have (say) KEY_UP|KEY_SHIFT rather than a separate KEY_SUP. This also generalizes better should we ever need to support multiple modifiers at once. To reduce the number of #defines, I've also switched from KEY_F1, KEY_F2, and so on to KEY_FN+1, KEY_FN+2, and so on. This isn't obviously necessary, and easily undone if we'd rather have move #defines in return for slightly more natural naming. To enable all this, I've inverted scan_key and scan_key_getsize so that scan_key_getsize is now the underlying function, and we don't waste all the top bits encoding width and height between scan_key and scan_key_getsize. Tested by pressing Home and End in hexedit in all of the terminals available to me.
2019-04-02Don't leak stdout pipe filehandle into children.Rob Landley
2019-04-02Have xflush() only flush stdout (that's all it checks errors on),Rob Landley
and tweak a couple comments.
2019-04-02Don't emit random -EPIPE error messages in a pipeline, just exit silently.Rob Landley
2019-03-27xabspath: use O_PATH for dirfd.Elliott Hughes
SELinux on Android is unhappy if you try to read "/": avc: denied { read } for name="/" dev="dm-3" ino=2 scontext=u:r:hal_dumpstate_impl:s0 tcontext=u:object_r:rootfs:s0 tclass=dir permissive=0 That could happen via the open of ".." too, and potentially any other directory might have similar restrictions, so move all of the open calls to using O_PATH. O_PATH seems more intention-revealing given what this function is doing anyway.
2019-03-27Change fileunderdir() to return abspath to file.Rob Landley
2019-03-26xabspath: use O_PATH.Elliott Hughes
SELinux on Android is unhappy if you try to read "/": avc: denied { read } for name="/" dev="dm-3" ino=2 scontext=u:r:hal_dumpstate_impl:s0 tcontext=u:object_r:rootfs:s0 tclass=dir permissive=0 O_PATH seems more intention-revealing anyway.
2019-03-26Make touch use xparsedate() (result: -t and -d the same, autodetects format),Rob Landley
and fix tests to pass on host too.
2019-03-24Factor out xparsedate() and xvali_date() into lib.Rob Landley
2019-03-17Fix xstrndup() bug.Rob Landley
Now there's a second user... the libc function already null terminates at len+1, and it doesn't malloc the full size if strlen() smaller so the redundant termination stomped unallocated memory. Oops. sort.c never noticed because it calculated length to truncate or copy existing string, so never hit this.