aboutsummaryrefslogtreecommitdiff
AgeCommit message (Collapse)Author
2019-08-19echo/printf: expand test cases, fix \x corner cases.Elliott Hughes
The behavior with "\xAV" (where the second hex digit is invalid) is different from the behavior with "\xVA", and echo and printf differ from each other.
2019-08-18Next round of shell plumbing.Rob Landley
You can now run: echo hello; if true; then echo hello; fi; while true; do echo hello; done It's got the start of for loops but needs environment variable resolver to do much with them.
2019-08-15Fix two typos.Elliott Hughes
2019-08-15Switch makedevs to new GLOBALS naming.Rob Landley
2019-08-15find.test: allow any error message about symlink loops.Elliott Hughes
glibc says "Too many levels of symbolic links", but BSD says "Too many symbolic links encountered". Until it becomes a problem, allow any error message.
2019-08-15dd: fix skip= with short reads.Elliott Hughes
Subtract the amount we actually read, not what we asked for.
2019-08-15linestack.c: fix buffer length.Elliott Hughes
GCC 8.2 (which my laptop appears to have been quietly upgraded to) points out that the %04X of an int might actually take 8 characters; between that, the "U+", and the trailing NUL, we're gonna need a bigger boat...
2019-08-13Return md5sum/sha1sum to the slightly slower but more explicit versionRob Landley
(revert afe951b19c0f with the table of magic constants). If you want speed, there's already a config option to use the libssl version instead.
2019-08-12find.test: fix flakiness.Elliott Hughes
Don't depend on directory iteration order in a test.
2019-08-12sort: move off get_rawline.Elliott Hughes
This was the last user of get_rawline, which lets us remove it.
2019-08-12blkid: add -s, SEC_TYPE, f2fs LABEL, and fix vfat/ntfs UUID.Elliott Hughes
This gets the tests passing with both toybox and util-linux 2.32.1 blkid. We use -s to get around the fact that we still don't support ntfs' $VOLUME_NAME attribute.
2019-08-09blkid.test: match TEST_HOST expectations.Elliott Hughes
Missing SEC_TYPE for ext3, msdos, and vfat. Missing LABEL for f2fs and ntfs. Wrong endian of UUID for msdos, ntfs, and vfat.
2019-08-09find: fix dangling symlink behavior.Elliott Hughes
ENOENT is ignored, but other errors are reported.
2019-08-09lsof: simplify octal mode display.Elliott Hughes
Since this masks, it doesn't actually have the same bug as e5942a8c90c2f52550496fdf08efddb564d8e5a3, but we may as well fix the other example of that questionable idiom while we're thinking about it...
2019-08-09losetup: fix the race.Elliott Hughes
There's a race between LOOP_CTL_GET_FREE and LOOP_SET_FD. Work around it by just retrying if we get EBUSY on the LOOP_SET_FD call. This is what similar code in ChromeOS already does. Bug: http://b/135716654
2019-08-09Split out run_function() and free_function(), add start of _TOYSH_LOCALSRob Landley
marshalling for vforked subshells, fix rc returned from run_pipeline for variable assignment. (Well, sort of: X=$(false) still needs rc = 1.)
2019-08-05du: disable the tests if we have SELinux.Elliott Hughes
ext stores extended attributes in a way that makes all the numbers in the test incorrect. For now, just disable the test in that case.
2019-08-04Make "if true; then echo hello; fi" work. More work on redirection.Rob Landley
2019-08-04Add TOYFLAG_MAYFORK and annotate a couple commands.Rob Landley
A TOYFLAG_NOFORK command must run in the context of toysh, but a MAYFORK can either run standalone or run in the toysh process. MAYFORK means it cleans up after itself: no leaked resources (malloc, mmap, filehandles, etc), even in error_exit() paths that would longjmp() back to the shell. It also doesn't discard anything we need to retain (don't close stdout, change toys.optargs[] so we can't free it, etc)...
2019-08-03Fix a bug report from E5ten: stat of a file with suid or sticky bit setRob Landley
output 5 digits of octal stat data instead of 4, due to hardwired leading 0 instead of %04a sprintf prefix.
2019-08-02Next round of infrastructure. Start of redirect logic and runtime flow control.Rob Landley
2019-08-02fix parallel make build failurePatrick Oppenlander
make: *** No rule to make target 'generated/Config.probed', needed by 'Config.in'. Stop. make: *** Waiting for unfinished jobs....
2019-08-02fix typo in MakefilePatrick Oppenlander
2019-07-31Move the empty regex workaround into xregcomp.Elliott Hughes
No current caller except grep needs this, but consistency seems like a good idea. Also change the xregcomp error message to be a bit more human-readable, rather than mention an implementation detail.
2019-07-31Fix error path and tweak comments.Rob Landley
2019-07-31man: switch to xregcomp.Elliott Hughes
All regcomp callers are now going through xregcomp.
2019-07-30Fix signed typecast bug.Rob Landley
We use (char *)1 and (char *)2 to indicate errors (they can never be valid pointers because both malloc() and mmap() return aligned memory and those align down to NULL, plus Linux maps 4k at the bottom to catch null dereferences anyway), and then typecast it to long (trusting in LP64) to do an integer <=2 comparison... except that needs to be UNSIGNED long or else pointers in the top half of the virtual memory space become negative and the <=2 false positives them as errors. (Oops.)
2019-07-29grep: fake GNU behavior for non-POSIX empty regex.Elliott Hughes
POSIX says there's no such thing as an empty regular expression. The grammar excludes the possibility: https://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap09.html BSD agrees with POSIX, and Android and macOS' BSD-based implementations reject the empty regular expression. GNU apparently disagrees. Luckily, BSD does accept the empty *sub* expression `()`, despite their error message for REG_EMPTY being "empty (sub)expression". This is presumably a bug, except there's explicit code to support it that is at least 26 years old: https://github.com/freebsd/freebsd/blame/master/lib/libc/regex/regcomp.c#L383 This workaround also works fine with glibc. If we want GNU behavior, I'm struggling to come up with another way to fake it. If we want POSIX behavior, we could easily just add a check to reject "" on glibc. Also switch to xregcomp().
2019-07-29Add ASAN=1 to the build system.Elliott Hughes
Just use `ASAN=1 make test_grep` or whatever. You'll probably want to set $ASAN_SYMBOLIZER_PATH to point to llvm-symbolizer, but Debian makes that annoying by calling the symbolizer /usr/bin/llvm-symbolizer-4.0 or whatever, and ASan refuses to use it: ==43370==ERROR: External symbolizer path is set to '/usr/bin/llvm-symbolizer-4.0' which isn't a known symbolizer. Please set the path to the llvm-symbolizer binary or other known tool. My usual workaround for this is to drop an llvm-symbolizer symlink in the current directory, and I'm happy to automate that in the script to make it require no knowledge of any of this nonsense, but haven't done so in this initial patch. I tested that this is actually working by reverting the grep fix and running `ASAN=1 make test_grep`.
2019-07-27losetup.test: bail out if loopback devices are already in use.Elliott Hughes
There's probably a way to filter out the existing ones, but it's likely to be pretty painful and isn't immediately necessary. (I hit this case on Android cloud x86 devices.)
2019-07-27modinfo.test: bail out early if there are no modules.Elliott Hughes
We need /proc/modules *and* a directory of modules.
2019-07-27ifconfig.test: actually skip tests if dummy0 up fails.Elliott Hughes
Rather than just claim we're doing so. Android cloud emulators don't have dummy0 like regular devices do.
2019-07-26truncate.test: work around ext extended attributes space usage.Elliott Hughes
Weaken the test slightly so that the extra 4KiB used for SELinux is ignored. Unfortunately all the du tests suffer the same problem, and I'm less sure what to do there. Mounting a read-only file system without the extended attributes might be the best idea? But since this test modifies the file system...
2019-07-26hostname: fix tests for non-root on Android.Elliott Hughes
On Android you can't even get the hostname unless you're root, to avoid device fingerprinting/tracking. Also modernize by using skipnot, which fixes the incorrect test name in the SKIP: output.
2019-07-25killall: implement -w.Elliott Hughes
This isn't currently used in AOSP, but it is used in some of the other codebases I can easily search, and it's a better fix for test flakiness than inserting a sleep.
2019-07-25killall.test: fix flakiness.Elliott Hughes
And when I say "fix", I mean "hack around in the worst possible way". Android+mksh is often slow enough that we can do the pgrep before the killed process has actually exited. (I'm seeing roughly 100% failure rate on cloud x86 emulators, and about 75% on a real Pixel 2.)
2019-07-24tests: fix for empty /etc/passwd or /etc/group.Elliott Hughes
It turns out some Android devices have an empty /etc/passwd and/or /etc/group, which was defeating the previous workaround. Switch to testing the intention more directly: we'll try the file in /etc, and if that didn't work, we'll assume we need a workaround.
2019-07-24Avoid double-close of fd in loopfiles_lines.Elliott Hughes
Test: no EBADF in `strace -e close ./toybox tac /proc/version`
2019-07-24crontab: switch to getline().Elliott Hughes
2019-07-24Rewrite tac to make better use of lib functions.Rob Landley
2019-07-24tac: switch to getline().Elliott Hughes
2019-07-23nl: switch from getline() to loopfiles_lines().Elliott Hughes
This was basically just to help me think out loud while discussing loopfiles_lines().
2019-07-22Start replacing get_line() with getline().Elliott Hughes
I started this last night, but thought I'd aim to send multiple small patches rather than work through all the callers and send one big patch. I've deliberately chosen the ugly name `allocated_length` because we've had historical bugs where folks think this a line length in the sense of the return value. I do wonder whether we should actually have some kind of getline() wrapper that hides the `char *`/`size_t` pair in lib/, which makes the function easier to use in most cases but does add the less common gotcha that you wouldn't be able to getline() through multiple files at once (which does happen in at least one toy). But maybe the real fix is to look harder for places where we can just use loopfiles_lines? Speaking of which, should we actually add two more arguments to that? Specifically: switch it to getdelim() rather than getline() behind the scenes, and also add a way to have the trailing '\n' automatically removed, since that seems to be what most callers want? Anyway, that seemed like enough questions that it was time to send this initial patch out before doing too much more...
2019-07-22test.tar: fix tar tests on Android.Elliott Hughes
There's no /etc/group on Android. Loop devices are in different places, but that test doesn't seem to offer anything beyond the block/char special device tests later, so remove that rather than fix it. Fix the block/char special tests (the expectations were missing, but they're never normally run on the host because the mknod fails). Likewise add the missing expectation to the "ownership" test, which isn't normally run on the host because the chown fails. The "sparse without overflow" test was broken because of the mksh/bash disagreement about whether in `VAR=val shell_function` that's a temporary assignment to VAR or one that outlives the command. Switch SUM to being a function like LST before it. Also rewrite the TARHD stuff because Android (a) doesn't have hd and (b) doesn't have process substitution in its shell. (I wish I'd noticed TARHD earlier --- I'd been manually sending the tar files to xxd for debugging instead!) With this patch, all the tar tests now pass for me on both the device and host.
2019-07-22Next round of plumbing evolution. Left in the todo and debug code this time.Rob Landley
2019-07-20Update design page to answer somebody's question.Rob Landley
2019-07-19xargs: don't pretend to support -I.Elliott Hughes
This was embarrassing... A build script that was using xargs -I silently started to ignore -I when we switched over to toybox xargs. The owner of the script has rewritten it to use the shell read builtin instead, but it's pretty unhelpful to silently ignore an option that radically changes how xargs behaves. (The -I behavior sounds sufficiently different from normal behavior that I'm not sure I've understood, and am just sending this cleanup rather than actually implementing -I correctly.) Bug: http://b/137832162
2019-07-17ifconfig: fix for non-root on Android.Elliott Hughes
The change from ioctl to xioctl in 844f0d1aae28c54bbb672f53ba2cda182c572129 broke ifconfig for non-root on Android. SIOCGIFHWADDR has been protected to prevent device tracking since Nougat. Bug: https://issuetracker.google.com/137671590
2019-07-16Fix unaligned access, tweak test suite.Rob Landley
2019-07-16grep: fix two bugs found by hwasan.Elliott Hughes
The first bug appeared as a memory overwrite, but was actually visible without hwasan: basically any `grep -F` that let to multiple matches on the same line was broken. The second bug was another memory overwrite, visible when I ran the existing grep tests. Bug: http://b/137573082