1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
1001
1002
1003
1004
1005
1006
1007
1008
1009
1010
1011
1012
1013
1014
1015
1016
1017
1018
1019
1020
1021
1022
1023
1024
1025
1026
1027
1028
1029
1030
1031
1032
1033
1034
1035
1036
1037
1038
1039
1040
1041
1042
1043
1044
1045
1046
1047
1048
1049
1050
1051
1052
1053
1054
1055
1056
1057
1058
1059
1060
1061
1062
1063
1064
1065
1066
1067
1068
1069
1070
1071
1072
1073
1074
1075
1076
1077
1078
1079
1080
1081
1082
1083
1084
1085
1086
1087
1088
1089
1090
1091
1092
1093
1094
1095
1096
1097
1098
1099
1100
1101
1102
1103
1104
1105
1106
1107
1108
1109
1110
1111
1112
1113
1114
1115
1116
1117
1118
1119
1120
1121
1122
1123
1124
1125
1126
1127
1128
1129
1130
1131
1132
1133
1134
1135
1136
1137
1138
1139
1140
1141
1142
1143
1144
1145
1146
1147
1148
1149
1150
1151
1152
1153
1154
1155
1156
1157
1158
1159
1160
1161
1162
1163
1164
1165
1166
1167
1168
1169
1170
1171
1172
1173
1174
1175
1176
1177
1178
1179
1180
1181
1182
1183
1184
1185
1186
1187
1188
1189
1190
1191
1192
1193
1194
1195
1196
1197
1198
1199
1200
1201
1202
1203
1204
1205
1206
1207
1208
1209
1210
1211
1212
1213
1214
1215
1216
1217
1218
1219
1220
1221
1222
1223
1224
1225
1226
1227
1228
1229
1230
1231
1232
1233
1234
1235
1236
1237
1238
1239
1240
1241
1242
1243
1244
1245
1246
1247
1248
1249
1250
1251
1252
1253
1254
1255
1256
1257
1258
1259
1260
1261
1262
1263
1264
1265
1266
1267
1268
1269
1270
1271
1272
1273
1274
1275
1276
1277
1278
1279
1280
1281
1282
1283
1284
1285
1286
1287
1288
1289
1290
1291
1292
1293
1294
1295
1296
1297
1298
1299
1300
1301
1302
1303
1304
1305
1306
1307
1308
1309
1310
1311
1312
1313
1314
1315
1316
1317
1318
1319
1320
1321
1322
1323
1324
1325
1326
1327
1328
1329
1330
1331
1332
1333
1334
1335
1336
1337
1338
1339
1340
1341
1342
1343
1344
1345
1346
1347
1348
1349
1350
1351
1352
1353
1354
1355
1356
1357
1358
1359
1360
1361
1362
1363
1364
1365
1366
1367
1368
1369
1370
1371
1372
1373
1374
1375
1376
1377
1378
1379
1380
1381
1382
1383
1384
1385
1386
1387
1388
1389
1390
1391
1392
1393
1394
1395
1396
1397
1398
1399
1400
1401
1402
1403
1404
1405
1406
1407
1408
1409
1410
1411
1412
1413
1414
1415
1416
1417
1418
1419
1420
1421
1422
1423
1424
1425
1426
1427
1428
1429
1430
1431
1432
1433
1434
1435
1436
1437
1438
1439
1440
1441
1442
1443
1444
1445
1446
1447
1448
1449
1450
1451
1452
1453
1454
1455
1456
1457
1458
1459
1460
1461
1462
1463
1464
1465
1466
1467
1468
1469
1470
1471
1472
1473
1474
1475
1476
1477
1478
1479
1480
1481
1482
1483
1484
1485
1486
1487
1488
1489
1490
1491
1492
1493
1494
1495
1496
1497
1498
1499
1500
1501
1502
1503
1504
1505
1506
1507
1508
1509
1510
1511
1512
1513
1514
1515
1516
1517
1518
1519
1520
1521
1522
1523
1524
1525
1526
1527
1528
1529
1530
1531
1532
1533
1534
1535
1536
1537
1538
1539
1540
1541
1542
1543
1544
1545
1546
1547
1548
1549
1550
1551
1552
1553
1554
1555
1556
1557
1558
1559
1560
1561
1562
1563
1564
1565
1566
1567
1568
1569
1570
1571
1572
1573
1574
1575
1576
1577
1578
1579
1580
1581
1582
1583
1584
1585
1586
1587
1588
1589
1590
1591
1592
1593
1594
1595
1596
1597
1598
1599
1600
1601
1602
1603
1604
1605
1606
1607
1608
1609
1610
1611
1612
1613
1614
1615
1616
1617
1618
1619
1620
1621
1622
1623
1624
1625
1626
1627
1628
1629
1630
1631
1632
1633
1634
1635
1636
1637
1638
1639
1640
1641
1642
1643
1644
1645
1646
1647
1648
1649
1650
1651
1652
1653
1654
1655
1656
1657
1658
1659
1660
1661
1662
1663
1664
1665
1666
1667
1668
1669
1670
1671
1672
1673
1674
1675
1676
1677
1678
1679
1680
1681
1682
1683
1684
1685
1686
1687
1688
1689
1690
1691
1692
1693
1694
1695
1696
1697
1698
1699
1700
1701
1702
1703
1704
1705
1706
1707
1708
1709
1710
1711
1712
1713
1714
1715
1716
1717
1718
1719
1720
1721
1722
1723
1724
1725
1726
1727
1728
1729
1730
1731
1732
1733
1734
1735
1736
1737
1738
1739
1740
1741
1742
1743
1744
1745
1746
1747
1748
1749
1750
1751
1752
1753
1754
1755
1756
1757
1758
1759
1760
1761
1762
1763
1764
1765
1766
1767
1768
1769
1770
1771
1772
1773
1774
1775
1776
1777
1778
1779
1780
1781
1782
1783
1784
1785
1786
1787
1788
1789
1790
1791
1792
1793
1794
1795
1796
1797
1798
1799
1800
1801
1802
1803
1804
1805
1806
1807
1808
1809
1810
1811
1812
1813
1814
1815
1816
1817
1818
1819
1820
1821
1822
1823
1824
1825
1826
1827
1828
1829
1830
1831
1832
1833
1834
1835
1836
1837
1838
1839
1840
1841
1842
1843
1844
1845
1846
1847
1848
1849
1850
1851
1852
1853
1854
1855
1856
1857
1858
1859
1860
1861
1862
1863
1864
1865
1866
1867
1868
1869
1870
1871
1872
1873
1874
1875
1876
1877
1878
1879
1880
1881
1882
1883
1884
1885
1886
1887
1888
1889
1890
1891
1892
1893
1894
1895
1896
1897
1898
1899
1900
1901
1902
1903
1904
1905
1906
1907
1908
1909
1910
1911
1912
1913
1914
1915
1916
1917
1918
1919
1920
1921
1922
1923
1924
1925
1926
1927
1928
1929
1930
1931
1932
1933
1934
1935
1936
1937
1938
1939
1940
1941
1942
1943
1944
1945
1946
1947
1948
1949
1950
1951
1952
1953
1954
1955
1956
1957
1958
1959
1960
1961
1962
1963
1964
1965
1966
1967
1968
1969
1970
1971
1972
1973
1974
1975
1976
1977
1978
1979
1980
1981
1982
1983
1984
1985
1986
1987
1988
1989
1990
1991
1992
1993
1994
1995
1996
1997
1998
1999
2000
2001
2002
2003
2004
2005
2006
2007
2008
2009
2010
2011
2012
2013
2014
2015
2016
2017
2018
2019
2020
2021
2022
2023
2024
2025
2026
2027
2028
2029
2030
2031
2032
2033
2034
2035
2036
2037
2038
2039
2040
2041
2042
2043
2044
2045
2046
2047
2048
2049
2050
2051
2052
2053
2054
2055
2056
2057
2058
2059
2060
2061
2062
2063
2064
2065
2066
2067
2068
2069
2070
2071
2072
2073
2074
2075
2076
2077
2078
2079
2080
2081
2082
2083
2084
2085
2086
2087
2088
2089
2090
2091
2092
2093
2094
2095
2096
2097
2098
2099
2100
2101
2102
2103
2104
2105
2106
2107
2108
2109
2110
2111
2112
2113
2114
2115
2116
2117
2118
2119
2120
2121
2122
2123
2124
2125
2126
2127
2128
2129
2130
2131
2132
2133
2134
2135
2136
2137
2138
2139
2140
2141
2142
2143
2144
2145
2146
2147
2148
2149
2150
2151
2152
2153
2154
2155
2156
2157
2158
2159
2160
2161
2162
2163
2164
2165
2166
2167
2168
2169
2170
2171
2172
2173
2174
2175
2176
2177
2178
2179
2180
2181
2182
2183
2184
2185
2186
2187
2188
2189
2190
2191
2192
2193
2194
2195
2196
2197
2198
2199
2200
2201
2202
2203
2204
2205
2206
2207
2208
2209
2210
2211
2212
2213
2214
2215
2216
2217
2218
2219
2220
2221
2222
2223
2224
2225
2226
2227
2228
2229
2230
2231
2232
2233
2234
2235
2236
2237
2238
2239
2240
2241
2242
2243
2244
2245
2246
2247
2248
2249
2250
2251
2252
2253
2254
2255
2256
2257
2258
2259
2260
2261
2262
2263
2264
2265
2266
2267
2268
2269
2270
2271
2272
2273
2274
2275
2276
2277
2278
2279
2280
2281
2282
2283
2284
2285
2286
2287
2288
2289
2290
2291
2292
2293
2294
2295
2296
2297
2298
2299
2300
2301
2302
2303
2304
2305
2306
2307
2308
2309
2310
2311
2312
2313
2314
2315
2316
2317
2318
2319
2320
2321
2322
2323
2324
2325
2326
2327
2328
2329
2330
2331
2332
2333
2334
2335
2336
2337
2338
2339
2340
2341
2342
2343
2344
2345
2346
2347
2348
2349
2350
2351
2352
2353
2354
2355
2356
2357
2358
2359
2360
2361
2362
2363
2364
2365
2366
2367
2368
2369
2370
2371
2372
2373
2374
2375
2376
2377
2378
2379
2380
2381
2382
2383
2384
2385
2386
2387
2388
2389
2390
2391
2392
2393
2394
2395
2396
2397
2398
2399
2400
2401
2402
2403
2404
2405
2406
2407
2408
2409
2410
2411
2412
2413
2414
2415
2416
2417
2418
2419
2420
2421
2422
2423
2424
2425
2426
2427
2428
2429
2430
2431
2432
2433
2434
2435
2436
2437
2438
2439
2440
2441
2442
2443
2444
2445
2446
2447
2448
2449
2450
2451
2452
2453
2454
2455
2456
2457
2458
2459
2460
2461
2462
2463
2464
2465
2466
2467
2468
2469
2470
2471
2472
2473
2474
2475
2476
2477
2478
2479
2480
2481
2482
2483
2484
2485
2486
2487
2488
2489
2490
2491
2492
2493
2494
2495
2496
2497
2498
2499
2500
2501
2502
2503
2504
2505
2506
2507
2508
2509
2510
2511
2512
2513
2514
2515
2516
2517
2518
2519
2520
2521
2522
2523
2524
2525
2526
2527
2528
2529
2530
2531
2532
2533
2534
2535
2536
2537
2538
2539
2540
2541
2542
2543
2544
2545
2546
2547
2548
2549
2550
2551
2552
2553
2554
2555
2556
2557
2558
2559
2560
2561
2562
2563
2564
2565
2566
2567
2568
2569
2570
2571
2572
2573
2574
2575
2576
2577
2578
2579
2580
2581
2582
2583
2584
2585
2586
2587
2588
2589
2590
2591
2592
2593
2594
2595
2596
2597
2598
2599
2600
2601
2602
2603
2604
2605
2606
2607
2608
2609
2610
2611
2612
2613
2614
2615
2616
2617
2618
2619
2620
2621
2622
2623
2624
2625
2626
2627
2628
2629
2630
2631
2632
2633
2634
2635
2636
2637
2638
2639
2640
2641
2642
2643
2644
2645
2646
2647
2648
2649
2650
2651
2652
2653
2654
2655
2656
2657
2658
2659
2660
2661
2662
2663
2664
2665
2666
2667
2668
2669
2670
2671
2672
2673
2674
2675
2676
2677
2678
2679
2680
2681
2682
2683
2684
2685
2686
2687
2688
2689
2690
2691
2692
2693
2694
2695
2696
2697
2698
2699
2700
2701
2702
2703
2704
2705
2706
2707
2708
2709
2710
2711
2712
2713
2714
2715
2716
2717
2718
2719
2720
2721
2722
2723
2724
2725
2726
2727
2728
2729
2730
2731
2732
2733
2734
2735
2736
2737
2738
2739
2740
2741
2742
2743
2744
2745
2746
2747
2748
2749
2750
2751
2752
2753
2754
2755
2756
2757
2758
2759
2760
2761
2762
2763
2764
2765
2766
2767
2768
2769
2770
2771
2772
2773
2774
2775
2776
2777
2778
2779
2780
2781
2782
2783
2784
2785
2786
2787
2788
2789
2790
2791
2792
2793
2794
2795
2796
2797
2798
2799
2800
2801
2802
2803
2804
2805
2806
2807
2808
2809
2810
2811
2812
2813
2814
2815
2816
2817
2818
2819
2820
2821
2822
2823
2824
2825
2826
2827
2828
2829
2830
2831
2832
2833
2834
2835
2836
2837
2838
2839
2840
2841
2842
2843
2844
2845
2846
2847
2848
2849
2850
2851
2852
2853
2854
2855
2856
2857
2858
2859
2860
2861
2862
2863
2864
2865
2866
2867
2868
2869
2870
2871
2872
2873
2874
2875
2876
2877
2878
2879
2880
2881
2882
2883
2884
2885
2886
2887
2888
2889
2890
2891
2892
2893
2894
2895
2896
2897
2898
2899
2900
2901
2902
2903
2904
2905
2906
2907
2908
2909
2910
2911
2912
2913
2914
2915
2916
2917
2918
2919
2920
2921
2922
2923
2924
2925
2926
2927
2928
2929
2930
2931
2932
2933
2934
2935
2936
2937
2938
2939
2940
2941
2942
2943
2944
2945
2946
2947
2948
2949
2950
2951
2952
2953
2954
2955
2956
2957
2958
2959
2960
2961
2962
2963
2964
2965
2966
2967
2968
2969
2970
2971
2972
2973
2974
2975
2976
2977
2978
2979
2980
2981
2982
2983
2984
2985
2986
2987
2988
2989
2990
2991
2992
2993
2994
2995
2996
2997
2998
2999
3000
3001
3002
3003
3004
3005
3006
3007
3008
3009
3010
3011
3012
3013
3014
3015
3016
3017
3018
3019
3020
3021
3022
3023
3024
3025
3026
3027
3028
3029
3030
3031
3032
3033
3034
3035
3036
3037
3038
3039
3040
3041
3042
3043
3044
3045
3046
3047
3048
3049
3050
3051
3052
3053
3054
3055
3056
3057
3058
3059
3060
3061
3062
3063
3064
3065
3066
3067
3068
3069
3070
3071
3072
3073
3074
3075
3076
3077
3078
3079
3080
3081
3082
3083
3084
3085
3086
3087
3088
3089
3090
3091
3092
3093
3094
3095
3096
3097
3098
3099
3100
3101
3102
3103
3104
3105
3106
3107
3108
3109
3110
3111
3112
3113
3114
3115
3116
3117
3118
3119
3120
3121
3122
3123
3124
3125
3126
3127
3128
3129
3130
3131
3132
3133
3134
3135
3136
3137
3138
3139
3140
3141
3142
3143
3144
3145
3146
3147
3148
3149
3150
3151
3152
3153
3154
3155
3156
3157
3158
3159
3160
3161
3162
3163
3164
3165
3166
3167
3168
3169
3170
3171
3172
3173
3174
3175
3176
3177
3178
3179
3180
3181
3182
3183
3184
3185
3186
3187
3188
3189
3190
3191
3192
3193
3194
3195
3196
3197
3198
3199
3200
3201
3202
3203
3204
3205
3206
3207
3208
3209
3210
3211
3212
3213
3214
3215
3216
3217
3218
3219
3220
3221
3222
3223
3224
3225
3226
3227
3228
3229
3230
3231
3232
3233
3234
3235
3236
3237
3238
3239
3240
3241
3242
3243
3244
3245
3246
3247
3248
3249
3250
3251
3252
3253
3254
3255
3256
3257
3258
3259
3260
3261
3262
3263
3264
3265
3266
3267
3268
3269
3270
3271
3272
3273
3274
3275
3276
3277
3278
3279
3280
3281
3282
3283
3284
3285
3286
3287
3288
3289
3290
3291
3292
3293
3294
3295
3296
3297
3298
3299
3300
3301
3302
3303
3304
3305
3306
3307
3308
3309
3310
3311
3312
3313
3314
3315
3316
3317
3318
3319
3320
3321
3322
3323
3324
3325
3326
3327
3328
3329
3330
3331
3332
3333
3334
3335
3336
3337
3338
3339
3340
3341
3342
3343
3344
3345
3346
3347
3348
3349
3350
3351
3352
3353
3354
3355
3356
3357
3358
3359
3360
3361
3362
3363
3364
3365
3366
3367
3368
3369
3370
3371
3372
3373
3374
3375
3376
3377
3378
3379
3380
3381
3382
3383
3384
3385
3386
3387
3388
3389
3390
3391
3392
3393
3394
3395
3396
3397
3398
3399
3400
3401
3402
3403
3404
3405
3406
3407
3408
3409
3410
3411
3412
3413
3414
3415
3416
3417
3418
3419
3420
3421
3422
3423
3424
3425
3426
3427
3428
3429
3430
3431
3432
3433
3434
3435
3436
3437
3438
3439
3440
3441
3442
3443
3444
3445
3446
3447
3448
3449
3450
3451
3452
3453
3454
3455
3456
3457
3458
3459
3460
3461
3462
3463
3464
3465
3466
3467
3468
3469
3470
3471
3472
3473
3474
3475
3476
3477
3478
3479
3480
3481
3482
3483
3484
3485
3486
3487
3488
3489
3490
3491
3492
3493
3494
3495
3496
3497
3498
3499
3500
3501
3502
3503
3504
3505
3506
3507
3508
3509
3510
3511
3512
3513
3514
3515
3516
3517
3518
3519
3520
3521
3522
3523
3524
3525
3526
3527
3528
3529
3530
3531
3532
3533
3534
3535
3536
3537
3538
3539
3540
3541
3542
3543
3544
3545
3546
3547
3548
3549
3550
3551
3552
3553
3554
3555
3556
3557
3558
3559
3560
3561
3562
3563
3564
3565
3566
3567
3568
3569
3570
3571
3572
3573
3574
3575
3576
3577
3578
3579
3580
3581
3582
3583
3584
3585
3586
3587
3588
3589
3590
3591
3592
3593
3594
3595
3596
3597
3598
3599
3600
3601
3602
3603
3604
3605
3606
3607
3608
3609
3610
3611
3612
3613
3614
3615
3616
3617
3618
3619
3620
3621
3622
3623
3624
3625
3626
3627
3628
3629
3630
3631
3632
3633
3634
3635
3636
3637
3638
3639
3640
3641
3642
3643
3644
3645
3646
3647
3648
3649
3650
3651
3652
3653
3654
3655
3656
3657
3658
3659
3660
3661
3662
3663
3664
3665
3666
3667
3668
3669
3670
3671
3672
3673
3674
3675
3676
3677
3678
3679
3680
3681
3682
3683
3684
3685
3686
3687
3688
3689
3690
3691
3692
3693
3694
3695
3696
3697
3698
3699
3700
3701
3702
3703
3704
3705
3706
3707
3708
3709
3710
3711
3712
3713
3714
3715
3716
3717
3718
3719
3720
3721
3722
3723
3724
3725
3726
3727
3728
3729
3730
3731
3732
3733
3734
3735
3736
3737
3738
3739
3740
3741
3742
3743
|
<html><head><title>toybox news</title>
<!--#include file="header.html" -->
<p>Toybox combines common Linux command line utilities together
into a single BSD-licensed executable that's simple, small, fast,
reasonably standards-compliant, and powerful enough to turn Android into
a development environment. See the links on the left for details.</p>
<h2>News</h2>
<a name="19-05-2021" /><a href="#19-15-2021"><hr><h2><b>May 19, 2021</b></h2></a>
<blockquote><p>It was of course a descendant of these eccentric poets
who invented the spurious tales of impending doom which enabled the people of
Golgafrincham to rid themselves of the useless third of their population.
The other two-thirds stayed firmly at home and lived full, rich and happy lives
until they were all suddenly wiped out by a virulent disease contracted from
a dirty telephone.</p><p>- The Hitchhiker's Guide to the Galaxy</p></blockquote>
<p><a href=downloads/toybox-0.8.5.tar.gz>Toybox 0.8.5</a>
(<a href=https://github.com/landley/toybox/releases/tag/0.8.5>git commit</a>)
is out, with prebuilt <a href=downloads/binaries/0.8.5>static binaries</a> and
<a href=downloads/binaries/mkroot/0.8.5>mkroot images</a>
bootable under QEMU (using vanilla linux-5.12, except that s390 needed another
<a href=http://lkml.iu.edu/hypermail/linux/kernel/2105.2/02904.html>perl
removal patch</a>).</p>
<p>This development cycle had thirty commits to sh.c fixing bugs and adding
features to <b>toysh</b>. Shell functions and local variables are now implemented,
along with the "set" builtin and trace support, and job control is
about halfway in. The shell as a whole is <a href=https://en.wikipedia.org/wiki/Ninety%E2%80%93ninety_rule>maybe 80% done</a>, but still
has some obvious gaps like $((math)) and interactive command line
history/editing, plus a bunch of testing.</p>
<p><u>New commands</u>:
Moritz Röhrich added <b>pwgen</b> and <b>base32</b>, and
Elliott Hughes added <b>unicode</b>. The
<b>readelf</b> command got promoted out of pending, and
<b>sha3sum</b> is now switched on in defconfig.</p>
<p>Michael Christensen added <b>chsh</b> to pending, which hasn't been promoted
yet because it's part of a group of commands (useradd, userdel, groupadd,
groupdel) that should all be promoted together at some point.</p>
<p><u>Upgrades</u>:
<b>cpio</b> now supports extracting multiple concatenated archives.
<b>file</b> now recognizes android's new XML binary format.
<b>devmem</b> now works on nommu, can handle 0x prefxes on the address,
and prints
hex or decimal output to match the address type.
<b>df</b> -a now shows overmounted filesystems.
<b>test</b> -k checks the sticky bit.
Added -t to <b>install</b>, <b>cp</b>, and <b>mv</b>, and -u to <b>cp</b>
and <b>cpio</b>.
<b>find</b> -executable checks that _we_ can execute them (u+x could be owned
by a different user or have selinux weirdness).</p>
<p>Elliott added several features to <b>date</b> (--iso, %:z output format, and
--utc as a synonym for -u), added a <b>sed</b> s///x option,
added units to <b>ulimit</b> output, and
made several UI changes to <b>hexedit</b> including a color
mode (which is now the default, press x to toggle between old and new
display), masked out the file type in <b>chmod</b> error messages and
taught it to handle complex modes (like u+s+s),
hardened <b>file</b> and <b>readelf</b> against invalid input,
added a portability.h workaround for macos statvfs for <b>df</b>,
enabled line buffering in <b>echo</b> and <b>yes</b>,
and significatly sped up <b>tr</b>. Rob similarly optimized <b>seq</b>
and <b>count</b>.</p>
<p><b>make tests</b> now defaults to VERBOSE=fail (and now has VERBOSE=all to
continue after failure, and VERBOSE=quiet to show FAIL: lines without
diff -u output).
Cleanups to tee, netstat, file, and df. Added "static" annotations in a
lot of commands, made more things use FLAG() macros. Ethan Sommer pointed out
an unnecessary wrapper function in chgrp.
In pending, Elliott did some cleanups and bugfixes to gettty, telnet, telnetd,
tftpd, init, modprobe, and dd, and added more features to vi.
Rob did some cleanup on modprobe.</p>
<p><u>Bugfixes</u>:
Yi-Yo Chiang fixed a <b>rm</b> bug where -f could get confused by read
only filesystems and produce an error trying to unlink a nonexistent
file, pointed out <b>install</b> -D was broken when the target file
already exists, pointed out a <b>cpio</b> interaction with certain encrypted
filesystems where readlink() wouldn't write as much payload as the header
promised (corrupting the archive) and fixed another cpio bug where -t test
mode would chown() files, and fixed quoting in mount.test.
Elliott fixed a sequencing issue in <b>setsid</b>'s system calls,
fixed some <b>sed</b> tests (sed's lie-to-autoconf --version
output was preventing toyonly tests from triggering) and
fixed <b>tar</b>'s base 256 output for very large files.
Nir Lichtman pointed out that <b>rm</b> -p shouldn't try to rmdir / at the end,
Luke Frankart pointed out a typo that broke <b>test</b> checking file modes,
Firas Khalil Khana fixed a duplicate entry in the roadmap,
Ethan Sommer fixed <b>install</b> -Dt,
Andy Hu requested that <b>mount</b> handle mixed options like
-tnomsdos,smbfs.
Denys Nykula reported a <b>chmod</b> regression,
asm89 pointed out a <b>netcat</b> bug writing transactions larger than
4k,
Daniel Santana fixed <b>df</b> combining -P and -k,
Robin Hsu fixed <b>grep</b> getting confused by libc setting errno when
a library function call succeeded.
Puck Meerburg taught the <b>gunzip</b> code not to be confused by FEXTRA entries
(as described in RFC 1952) and added a test suite entry for that.</p>
<p><b>find</b> handles empty arguments properly now (with test), <b>date</b>
-I with no arguments no longer segfaults, <b>grep</b> -f /dev/null no
longer adds a "" regex matching everything, -z no longer splits patterns
on \n, and -o can print zero length matches,
<b>cp</b> -t works with one argument and with --parents.
<b>readelf</b> displays properly in 80 columns and doesn't reject .bss
off end of file, and <b>sed</b> s///2 now works when there isn't more than one
match.
<b>tar</b> now removes empty directories where it wants to put a non-directory.
<b>microcom</b> sets serial device terminal attributes correctly.</p>
<p>The prebuilt x86-64 binaries shouldn't segfault on older x86-64 processors
anymore (workaround for gcc bug).
Cross compiling was sometimes confusing install.sh because it created
a hyphenated "toybox-$TARGET" and then tried to make symlinks to just "toybox".</p>
<p><u>Build</u>:
CONFIG_TOYBOX_I18N went away: toybox builds always support utf8 now.
Ella-0 added OpenBSD support (but not a separate defconfig, try
"make freebsd_defconfig").</p>
<p><u>Library</u>:
utf8towc() is now unsigned instead of wchar_t, matching wctoutf8().
xclose() tests if fd is -1 so the caller doesn't have to.
txpect (the test suite's built-in "expect", implemented in shell) can
now do regex matches, and sh.test uses this.</p>
<p>Elliott added the failing regex to the xregcomp() error message, and
made multiple portability tweaks (header tweaks to build in more
environments, running tests on Raspberry Pi OS and under mksh,
fix utf8 locale selection on macos).</p>
<p><u>Docs</u>:
<b>chmod</b>'s help text explains why o+s doesn't set +t (and has a
test for it). The <b>echo</b> help text is more compact.
Checked in a better conversion of <a href=https://landley.net/toybox/toycans.png>toycans.png</a>.</p>
<a name="24-10-2020" /><a href="#24-10-2020"><hr><h2><b>October 24, 2020</b></h2></a>
<blockquote><p>
"We are now cruising at a level of two to the power of twenty-five thousand to
one against and falling, and we will be restoring normality just as soon as we
are sure what is normal anyway."</p>
<p>- The Hitchhiker's Guide to the Galaxy</p>
</blockquote>
<p>After a longer and slightly more
<a href=https://github.com/landley/toybox/commit/07a896862ddf>turbulent</a>
development cycle than some,
<a href=downloads/toybox-0.8.4.tar.gz>Toybox 0.8.4</a>
(<a href=https://github.com/landley/toybox/releases/tag/0.8.4>git commit</a>)
is out with new commands <b>sha3sum</b> and <b><a href=https://github.com/landley/toybox/commit/6b4c32ae3986>watchdog</a></b>.</p>
<p>There are <a href=downloads/binaries/mkroot/0.8.4>prebuilt mkroot binaries</a>
now, tiny toybox linux systems for a dozen architectures, all bootable under
qemu. (The vmlinux in each tarball is a vanilla linux-5.9 kernel built
from the included config file.)</p>
<p>The FAQ got <a href=faq.html>noticeably larger</a>, and the README has
more links. New command features include the <b>sed -s</b> flag,
<b>cpio --no-preserve-owner</b> now affects archive creation,
Elliott added <b>tar -I</b> and multi-type
<b>find -type a,b,c</b> support, Mark Salyzyn added <b>xargs -P</b> to
run parallel jobs, the <b>ps</b> and <b>top</b> commands
now autodetect pid length, and <b>top</b> adjusts units for memory display based on
system size.</p>
<p><u>Toysh and toyroot</u>:
lots of new work on <b>toysh</b>: 29 commits to sh.c since last release, adding 1500
lines and deleting 700, plus a bunch of sh.tests entries). Since last release
we implemented wildcards, case/esac and select, brace expansion sequences
(ala {1..10..2} and {a..z}),
the remaining variable slice types ${a#y} ${a%y} ${a^y} ${a,y}
${a/search/replace}, the "<b>source</b>" shell builtin,
the start of job control, and several bugfixes.
Plus the standalone "<b>make sh</b>" build understands MAYFORK now. It's
still missing features like functions() and $((math)) but it's getting
close to usable now.</p>
<p><u>Documentation</u>:
Some of the README contents moved to the FAQ, and the README's "presentations"
section got some new links. Several new <a href=faq.html>FAQ</a> entries explaining things like
mkroot ("how do I build a working Linux system
with toybox"), cross compiling (how to get/setup the 2 compilers and
3 libc we regression test against), and "where does toybox fit into the
linux/android ecosystem".
The "toybox --help" output now lists the project's web page (<a href=https://github.com/landley/toybox/issues/50>by request</a>).
Elliott removed
getevent (an android board bringup/hardware debugging tool built by running
a python script against kernel headers, not really in scope for toybox),
fixed xargs help formatting, and taught the toybox multiplexer's
command list output (and "kill -l") to measure the current the terminal width
when wordwrapping (previously hardwired to 80 columns).
If you're curious, I checked in my <a href=www/release.txt>release procedure
checklist</a>, and
fixed a stale link in the nav bar on the left ("statistics" changed domains).
Firas Khaliki Khana fixed some issues in the roadmap, and Rob
tweaked the roadmap so status.html is slightly more current.</p>
<p><u>Tests</u>:
The "make tests" target now fails if any of the tests it ran failed.
Lots of TEST_HOST tests got fixed (checking for specific error messages exposes
TEST_HOST to version skew, and scripts/test.sh will now skip TEST_HOST
commands that aren't installed) and added toyonly annotations as necessary
(replacing most uses of SKIP_HOST).
Eric Molitor added automated github tests on MacOS and Ubuntu using their
"workflows" thing.
The clang asan plumbing slows some code down more than 10 times, so some testing timeouts were expanded.
<p><u>Pending</u>:
Eric Molitor did a bunch of work on route: moved it to sbin, added xsend and
xrecv, taught display_routes() and setroute to use the rtnetlink API and
do hostname lookups, added support for mss, win, and irtt, merged ipv6 and ipv4
codepaths with autodetection of address type, implemented RTA_CACHEINFO
support, switched exit paths to perror_exit(), and removed unused code.
Ethan Sommer fixed warnings in dhcpd, removed a bunch of unnecessary
; after GLOBALS() blocks and an unnecessary return in df.
Erik Moqvist fixed dns setting in the dhcp client.
Rob did some cleanup on bootchartd, traceroute, getty...
Elliott fixed getty to reliably update utmp.
Chris Sarra added ipv6 support to wget.
Ariadne Conill submitted several small fixes from testing Alpine Linux with
toybox, and Antoni Villalonga i Noceras fixed typos in error messages.</p>
<p><u>Bugfixes</u>:
Elliott avoided sign extension in devmem, and Ethan Sommer switched it from
getpagesize() to posix sysconf(_SC_PAGESIZE).
Elliott fixed chmod -R
ignoring dangling symlinks, fixed stty <a href=https://github.com/landley/toybox/issues/251>misparsing c_iflags</a>, fixed the chattr f2fs test with
compression enabled, taught blkid not to show empty tags, taught xparsedate()
to read date's default output format, fixed a recent echo -e \0 regression,
changed cpio -p parsing to match a <a href=http://lists.landley.net/pipermail/toybox-landley.net/2020-August/011955.html>bug</a> in the gnu version
which an existing script <a href=https://github.com/landley/toybox/commit/fa1af3b085cc>depended on</a>,
and switched hwclock back to looking
only at /dev/rtc0 for <a href=https://github.com/landley/toybox/commit/70e2232ce61c>reasons</a> involving kernel version skew and vendor bug reports.</p>
<p>David Legault pointed out that unescape2() (and thus echo -e) wasn't handling
\0 right.
Khem Raj reported that mips glibc doesn't have SIGSTKFLT, leading to
a build break if we assume glibc behaves consistently, which led to a bunch of
macos signal portability cleanups from Elliott.
William Djupström fixed tar extracting long file paths, adding hardlinks
to an archive, and reported that --exclude wasn't working.
Peter McConalogue pointed out that cp/mv -i prompt should
default N, and mv should only prompt when stdin is a tty.
Patrick Oppenlander made rtcwake and hwclock default to UTC if /dev/adjtime
isn't available, suggested watchdog catch SIGINT, and caught an
uninitialized offset in blkdiscard which gcc apparently didn't.
Ryan Pritchard reported that <b>file</b> wasn't getting gif heights right,
and Elliott added gif version output so TEST_HOST provided similar output.
Martin Stjernholm fixed <b>cp -P</b> to not follow symlinks and updated
the help text to say it's not the default.
Antoni Villalonga added more --long asiases to <b>chgrp/chown/rmdir</b>.
<p>The symlink indirection code subtly broke xexec() so it would still try to call
a builtin when given a path to a command (fixed now).
The code to trip \n off xgetline() was using the length of the allocation
instead of the length of the read.
Netcat no longer leaks sockfd into child processes.
Patch "fuzz" support was outputting context lines from the hunk, not the
file. Chrisrfq fixed i2cdetect parameter reading.
Chris Sarra taught dd not to throw an error trying to truncate device files,
added a misshing fflush() to lib/password.c, fixed a parse error in logger
preventing local facility logging from working right, taught init
to reload inittab when it receives SIGHUP, and fixed a memory leak in tar.</p>
<p><u>Library</u>:
New xvdaemon() function daemonizing on nommu systems, to wean commands
off daemon() which requires TOYBOX_FORK.
Teach sendfile_len() to use copy_file_range() when available, with compile
time probe. Fix xsignal_all_killers() to install the correct handler.
Rob switched dirtree_path() to a non-recursive implementation, and
taught dirtree that a top level entry named "" is equivalent to
"." but should not show up in dirtree_path(),
read_password() now reads from tty device rather than stdin, and handles ctrl-c and ctrl-d, and
human_readable() now has HR_NODOT so it can print single digit numbers without going "0.0".
Petri Gynther increased the vmstat column sizes so it didn't fit
in 80 columns anymore.
Elliott added UTC offset support to xparsedate().</p>
<p><u>Infrastructure</u>:
mkroot now only passes --no-preserve-owner to cpio (so initramfs files
belong to root rather than whichever user ran the build) when using toybox's
version, because the other one is brain-damaged and errors out not
understanding it. (It understands it for extract, but not create. Yes really.)
New PENDING variable you can set to enable more than just sh and route out
of pending.
The "make distclean" no longer deletes root_download (where mkroot's "extra plumbing"
file saves source code downloads it tarballs, such as dropbear).</p>
<p><u>Cleanup</u>:
Cleanup openvt (35 lines added, 78 lines removed)
Cleanup blkdiscard so lib/args.c parses the -o and -l numbers, which means
32 bit systems are limited to 2 gigabyte sizes which is a TODO item systemwide.
oneit now uses flag macros and mentions -rn in the help. demo-utf8towc
now has main.c call setlocale for it.
Some cleanup on stty.</p>
<a name="11-05-2020" /><a href="#11-05-2020"><hr><h2><b>May 11, 2020</b></h2></a>
<blockquote>
<p>Ford: Ah. A
<a href=https://www.youtube.com/watch?v=WSsR419HBpQ>towel</a>.
Keep this and guard it with your life.</p>
<p>Arthur: Huh?</p>
<p>Ford: Listen, it's a tough universe. There's all sorts of people and things
trying to do you, kill you, rip you off, everything. If you're going to
survive out there, you've really got to know where your
<a href=towel.jpg>towel</a> is.</p>
<p>- The Hitchhiker's Guide to the Galaxy</p>
</blockquote>
<p>Despite everything <a href=downloads/toybox-0.8.3.tar.gz>Toybox 0.8.3</a>
(<a href=https://github.com/landley/toybox/releases/tag/0.8.3>git commit</a>)
is finally out, with new commands <b>rtcwake</b> from Elliott Hughes and
<b>blkdiscard</b> from Patrick Oppenlander.
The big news is "<b>make root</b>" now boots to a shell prompt,
with toysh making it all the way through toyroot's init script.
(Some people were looking forward to <b>patch --fuzz</b> support too.)</p>
<p><u>Toyroot</u>: <b>make root</b> now does what it says on the tin, it
builds a bootable toybox-based Linux system using two source
packages (toybox and linux). The trivial version is "make root && sudo chroot
root/host/fs /init". Here's
a <a href=http://lists.landley.net/pipermail/toybox-landley.net/2020-April/011667.html>post with instructions</a> if you want to know how to build the
cross compilers for testing the various architectures. The self-contained
<a href=https://github.com/landley/toybox/blob/0.8.3/scripts/mkroot.sh>script
that builds a simple bootable Linux system</a> is 250 lines long, and
should be easy to read if you want to know how it works.</p>
<p>It also has basic module support, meaning arguments that aren't X=Y variable
assignments are the names of scripts to run to build more
packages at the end of the build. I checked in an <a href=https://github.com/landley/toybox/blob/master/scripts/root/dropbear>example package</a>,
and there's generic "download and extract source tarballs"
<a href=https://github.com/landley/toybox/blob/master/scripts/root/plumbing>plumbing</a> available to such modules.
(The Makefile doesn't know how to pass module names through to the script,
so instead of "make root" you have to
call the script directly, ala "scripts/mkroot.sh CROSS=sh4 LINUX=~/linux dropbear".)</p>
<p>The resulting root filesystem now uses /root as the home
dir for UID 0, and creates /dev/fd and /dev/shm in devtmpfs. The build works
around a kernel
build bug where "make distclean" doesn't work in a "cp -sfR" symlink
directory. (The bug meant if you pointed LINUX= at unclean source, it was
unhappy, so now it distcleans the source directory before copying it. This
modifies said source directory, which is not ideal, but as usual the kernel guys
<a href=http://lkml.iu.edu/hypermail/linux/kernel/2002.2/00083.html>ignored
the bug report</a>, so a workaround was required.)
The CROSS=all build announces each target in the title bar, puts
its logs into root/log, and has better trap handling to stop with one
ctrl-c.
Since last release it uses a more concise config format for the various kernel
arch targets (which shrank the script by about 50%), and merged
the old (now removed) scripts/cross.sh into the main script so
"make CROSS=armv7l LINUX=~/linux" is
literally just a call to "scripts/mkroot.sh CROSS=armv7l LINUX=~/linux" now.</p>
<p>As for <b>scripts/mcm-buildall.sh</b> building cross compilers,
of COURSE gcc 8.3 requires a different configuration to build the same
toolchain as previous versions, it's gcc. It now builds a proper
nommu libc for sh2eb without a broken fork() call in it that can never
be used but prevents compile-time probes from detecting nommu,
and checks that the cross compiler completed before trying to build
the native compiler.</p>
<p>This was all tested with a recent
<a href=https://github.com/richfelker/musl-cross-make/commit/5086175f2902>version</a>
of musl-cross-make with the top level Makefile
edited to use BINUTILS_VER = 2.32 because the newer one
<a href=https://www.spinics.net/lists/linux-sh/msg56844.html>breaks the kernel
build</a>
and LINUX_VER = 4.19.90 because the default musl-cross-make config uses an
out-of-tree headers package for some reason (those who forget history are
<a href=https://lkml.org/lkml/2006/4/28/194>doomed to repeat it</a>) which
breaks m68k and s390x. I won't post binaries of the resulting toolchains
because they're GPLv3, but <a href=https://musl.cc/>Thalheim might</a>.
(I've also <a href=http://lists.landley.net/pipermail/toybox-landley.net/2020-May/011673.html>test built</a>
with the Android NDK, but bionic's startup code currently segfaults before
calling main() if it can't open /dev/null, and the kernel guys
<a href=https://lkml.org/lkml/2017/9/13/651>ignored my patches</a> to
make CONFIG_DEVTMPFS_MOUNT work in initramfs.)</p>
<p>At the moment toyroot cheats and uses two commands out of pending: toysh
is about 80% of the way to being useful but still missing some
obvious features like function support, job and terminal control, command
history, $((math)), wildcard expansion... plus a lot of bash features like array
variables, so it isn't out of pending yet. And route needs to be redone to use the
netlink interface that can handle multiple routing tables. For the moment
scripts/mkroot.sh adds both of them to defconfig if you haven't already
got a .config when you run it. (If you build and can't boot, your .config
probably hasn't got CONFIG_SH switched on. Fix it in menuconfig, or
rm .config and try gain.)</p>
<p><u>New toybox features</u>:
Elliott taught <b>patch</b> to understand [FILE [PATCH]] arguments, made
<b>cal</b> highlight the current day, added -T to <b>cp</b>/<b>mv</b>,
did a lot of work on <b>lsattr</b>/<b>chattr</b> (including adding -p
and "chattr ="), added tar --absolute-names, taught <b>id</b> to
support numeric lookup and handle unknown groups, made -G show all
groups, and removed context= from -Z.
Rob added <b>patch -F</b> (fuzz factor) support and <b>help -u</b> (usage only),
taught <b>echo -e</b> about bash extensions like \uXXXX unicode escapes,
<b>netcat -L</b> no longer automatically includes stderr (new -E
option does that),
<b>setsid</b> now uses -c (like the man page says) instead of -t, and added
-w (wait) and -d (detach from tty).
Andrew Ilijic added <b>ls -w</b>, removed trailing whitespace on output,
and added tests for -C and -x.</p>
<p><u>Library</u>:
another fix to 32 bit option parsing of long long optflag values,
remove getdirname() and just use the libc function (which modifies
its argument, but we don't have to free a malloc),
dlist_terminate() can now be called repeatedly on the same list,
new relative_path() function finds path from one directory to another,
and readfd() works like readfile() but on an already open fd.
Use MPATH macros in mkpathat(). Elliott added macOS versions of
dev_minor()/dev_major()/makedev() to portability.c,
implemented posix_fallocate() for macOS, removed a
leftover uClibc workaround in fallocate that was breaking macOS,
and moved the table of ELF architectures from file to lib/lib.c (so
readelf can share).
Park Ju Hyung pointed out the fast path of fdlength() was commented out,
and we switched it to the 64 bit API while we were there (and then Elliott
added macOS support).
Joeky taught file to recognize 7z archives.</p>
<p><u>Pending</u>:
The shell got a bunch of work: standalone "make sh" now includes the
multiplexer for builtin commands like "exit" and "cd". Added MAYFORK annotation
for commands that exist in the $PATH but can also be run within the shell
process (and sometimes have different behavior within the shell):
currently applied to help, echo, false, kill, printf, pwd, test, time, and
true.</p>
<p>Elliott added new commands <b>getopt</b> and <b>readelf</b>.
Jarno Mäkipää taught wget how to follow http 301 and 302 redirects,
and did lots of work on vi. (Elliott also did work on vi.)
Gavin Howard fixed a comparison bug in bc.
Ethan Sommer fixed numerous small issues (including several build
warnings and FLAG() macro conversions).</p>
<p><u>Bugfixes</u>:
<b>tar</b> extract now deletes files and symlinks where it's making a directory,
<b>find -L -type -l</b> now finds dangling symlinks, extra #ifdefs in
portability.h prevent old gcc versions from barfing on __has_include(),
xgetline() now returns NULL at EOF, tee with no arguments was
writing to stdout twice, setsid works on nommu (I.E. vfork) now,
<b>netcat -L</b> no longer accumulates zombie processes,
<b>sntp</b> now uses adjtimex instead of adjtime (to build on bionic).
xcreate_stdio() was checking WARN_ONLY in the wrong field (and tar was
passing it in the wrong field, so it worked there).
Several people wrestled with the <b>xargs</b> "argument too long" problem.
Alessio Balsini fixed memory leaks in loopback_setup() and "wayling"
added a missing continue to losetup.
Elliott fixed xargs -E, various things in modinfo,
added an error check to gzip when using zlib (which copies non-gzip data to
the output verbatim for some reason),
found an case where dirtree could use
uninitialized data and silenced "Invalid argument" warnings (triggered by
Android's selinux policies making stat() and readlink() fail),
fixed locale support in macOS (both in iconv
and in main.c), taught stat to show filesystem time on macOS,
fixed a 32 bit truncation in sntp, fixed a memory access one byte outside
of its array in patch.c, removed the cpio --trailer option,
and widened the pid display fields in ps to
6 digits. Rich Felker fixed find.c assuming
time_t is a long (y2038 issue on 32 bit).
Greg Kaiser fixed a thinko in get_block_device_size().
Jarno Mäkipää fixed utf8 support in cut -C, and cp --parents.
David Legault complained that dir/.* tells rm to delete dir/.. and we'd do it.
JakeSFR pointed out a bug in file's identification of broken symlinks.
William Haddon fixed cp to treat a directory with a trailing slash
teh same as one without.
Denys Nykula fixed rm -i not to prompt for an empty "" argument.
SebiderSushi reported that chmod g+s wasn't working.
The linux kernel doesn't let an O_PATH fd work with fgetxattr(), so
Elliott switched <b>ls</b> to use the path-based functions now (which is racy,
it means you can stat() one inode and get the xattrs for a different one,
but nobody in kernel land seems to use xattrs much so they're not fully
supported by the API, and who's crazy enough to use xattrs for security
anyway). Derick Pallas pointed out an xclearenv() bug.
Atatsulo (or possibly luolongjuna) did a lot of work on the <b>ping</b>
command: pointed out min/range/max were out of order,
that we shouldn't print a summary unless we received at least one
reply packet, and implemented ttl support.</p>
<p><u>Build plumbing</u>:
The "make root CROSS=all" build announces each target in the title bar, puts
its logs into root/log, and has better trap handling to stop with one
ctrl-c.</p>
<p>Fixed scripts/single.sh to work when PREFIX has no trailing slash,
make silentoldconfig no longer feeds "y" to kconfig (which puts menus in a
loop) and instead just feeds in newlines to accept whatever the default is.
The non-git version number I keep forgetting to update each release moved
from main.c to toys.h.
Fixed a couple different errors in mkflags.c (one of which caused ls --color
to set all the other flags).</p>
<p>Elliott added more macos support and tests, and added
fallocate, cp, mktemp, and mv to the macOS defconfig.</p>
<p><u>Cleanup</u>:
Rob did some cleanup on xargs, ls, md5sum, and sort, tidied up main.c a bit,
made cp, base64, dmesg, and free use FLAG macros,
and switched fallocate to new style global names.
Elliott Hughes switched rfkill from the old byte at a time get_line()
to libc getline(), made du use FLAG() macros, and mad chattr use
standard toybox argument parsing for -v and -p.
Several commands (help, cp) had sub-options removed from menuconfig.
Merged realpath into readlink.c, and use xabspath() instead of libc realpath().</p>
<p><u>Documentation</u>:
New roadmap section about <a href=roadmap.html#packages>other packages</a>
that toybox can (eventually) replace.</p>
<p>Update roadmap to note that posix-2008 moved to a different URL (content
at the old URL undergoes random changes), and link to the IETF RFCs.
design.html explains more of the history of environment sizes on links
and has a #bits anchor tag.</p>
<p>Update the <a href=design.html#bits>LP64 section</a> of design.html to
fish the documents out of archive.org now that unix.org is gone,
and show the actual size table locally.</p>
<p>Elliott improved the help of <b>date</b> and <b>chattr</b>, and
made the usage: lines in the posix directory more consistent.</p>
<p>Shrank the sed help from 150 lines to 90-ish.</p>
<p><u>Tests</u>:
New "txpect" performs interactive tests, running through a sequence of
writes to a command's stdin and reads from stdout and stderr, when failing it
reports the first non-matching step. (This for example lets sh.test check
the shell line continuation logic prompts with $PS1 and $PS2 appropriately
with various unfinished input lines, and that "echo hello; if" doesn't
output hello before prompting for the next line of input.) Added
VERBOSE=xpect to print out each read and write successfully performed by txpect.</p>
<p>runtest.sh only creates an "input" file when the input argument isn't empty,
EVAL doesn't supply -- (you have to provide your own if you want that).</p>
<p>Rob added basic <b>stat</b>, <b>patch</b>, and <b>tee</b> tests.
Jarno added a bunch of <b>vi</b> tests.
Elliott fixed tests for <b>ifconfig</b>, <b>lsattr</b>, <b>chattr</b>,
and <b>date</b>, added tests for find, id, xargs, and made
the id, iconv, env, file, printf and cat tests work on macOS (and skipped
the du tests there). Rob cleaned up chmod tests.</p>
<p><u>Sheer pedantry</u>:
true and false now have usage: lines (which you have to "help false" to see
because they (intentionally!) don't support --help.
Renamed get_chunk()/dump_chunk() to read_chunk()/write_chunk() in tail.c
Elliott fixed some typos.</p>
<a name="18-10-2019" /><a href="#18-10-2019"><hr><h2><b>October 18, 2019</b></h2></a>
<blockquote>
<p>"In those days spirits were brave, the stakes were high, men were real
men, women were real women and small furry creatures from Alpha Centauri
were real small furry creatures from Alpha Centauri."
</p> <p>- The Hitchhiker's Guide to the Galaxy</p>
</blockquote>
<p><a href=downloads/toybox-0.8.2.tar.gz>Toybox 0.8.2</a>
(<a href=https://github.com/landley/toybox/releases/tag/0.8.2>git commit</a>)
is finally out (only two months late).</p>
<p>The new commands this time are minor variants of existing ones
(<b>dnsdomainname</b> is hostname -d, <b>arch</b> is uname -m), but about
1/3 of toysh got implemented in pending, <a href=https://github.com/landley/mkroot>mkroot</a> got merged as a "make root" target (see Build),
and Android's <a href=http://lists.landley.net/pipermail/toybox-landley.net/2019-August/010845.html>hermetic build</a>
work continued resulting in a lot of fixes.</p>
<p>The android-specific <b>getprop</b>, <b>setprop</b>, <b>start</b>, and <b>stop</b>
commands were removed from toybox because they grew dependencies on android
libraries, and since toybox tries to avoid mandatory external dependencies
they moved to another package.</p>
<p><u>Build</u>:
The new "make root" target builds a simple toybox root filesystem,
using scripts/mkroot.sh which is a stripped
down version of the external <a href=https://github.com/landley/mkroot>mkroot</a>
project merged into toybox. (Alas, at the moment it wants sh and route out
of pending, which aren't quite load bearing yet.)</p>
<p>The new "scripts/mcm-buildall.sh" convenience script builds cross
compilers based on gcc+musl multiple targets (arm, x86, mips, m68k, s390,
microblaze...)
using <a href=https://github.com/richfelker/musl-cross-make>musl-cross-make</a>,
and a new scripts/cross.sh to easily cross compile for one or more targets
using the resulting compilers. (This works with "make root", "make
toybox", and other builds understanding the CROSS_COMPILE environment variable.)</p>
<p>Setting the environment variable ASAN=1 enables the
llvm address sanitizer. (You may need to
export ASAN_SYMBOLIZER_PATH on debian to use it, they add the version number to
the filename in /usr/bin which confuses llvm.)</p>
<p>Patrick Oppenlander fixed make -j (our make is a wrapper around
scripts/make.sh which was already doing a parallel build, but if you called
make with -j it wouldn't wait for make.sh to finish and the build broke).</b>
<p>Lots of fixes for the MacOS build.</p>
<p><u>New options</u>:
We upgraded <b>tar</b> to extract some older tarballs, <b>file</b> added -b and -s
options and can also recognize older tarballs,
some xz archives, .otf fonts, perf/simpleperf data files,
android boot images, and dtb files. Upgraded <b>file</b> to show the device type
for block and char devices, the target for symlinks, and it says _why_ it can't
open a file.</p>
<p>Several commands ignore some options rather than erroring on them:
<b>patch</b> ignores the -f -g# and --no-backup-if-mismatch options
(all of which we were already doing by default anyway), and <b>xargs</b>
ignores -P# (which requests parallel execution, maybe add it later but
single threading works), and Denys Nykula made <b>wget</b> ignore
--no-check-ceritificate and <b>gzip</b> ignore -n (again, we weren't doing
either anyway) to satisfy pkgsrc.</p>
<p>William Haddon taught <b>diff</b> and <b>patch</b> to support special characters in path
names (via quoting) and more timestamp formats.
Eric Molitor implemented <b>nl</b> -v (with negative and zero starting values),
and fixed a MacOS compatibility issue in the build scripts.
Tom Cherry added <b>netcat</b> -U (UNIX domain socket support).
Denys Nykula taught <b>wget</b> to call ftpget for ftp:// URLs.
Ethan Sommer added ln -t and rmdir --ignore-fail-on-non-empty.
Elliott Hughes added <b>grep</b> -R, <b>killall</b> -w, tar --mode,
fixed <b>ls</b> -Z and spacing in -lZ and --full-time,
added <b>xargs</b> --max-args and removed xargs -I (which we never actually
supported) and tweaked the size calculation to more closely match other
versions,
added xargs -o and made -p read from /dev/tty, added <b>blkid</b> -s, SEC_TYPE for ext3,
msdos, and ntfs, f2fs and ntfs LABEL, fixed vfat/ntfs UUID, and switched
the endianness for UUID of msdos/ntfs/vfat, simplified the octal mode
display in <b>lsof</b>, added <b>pidof</b> -x (thus making the default behavior _not_
be -x).</p>
<p>When readdir() succeeds but stat() fails, <b>ls</b> now prints "??? ? ? filename"
entries instead of "cannot access" errors. (Still trying to make ls / on
android work more gracefully with the SELinux weirdness they added in 9.)</p>
<p>Added %C to <b>stat</b> and made it show device type in its default output,
<b>cmp</b> now works
with 1 argument (implicit - as second argument reading from stdin).
<b>sed</b> added exit codes to the q command (q123), added Q, and fixed a
bad error message with "!".
Removed TAIL_SEEK config option (it just always does that now),
<b>find</b> grew several new options (%Z, -true, -false, and -newerXY), <b>ln</b> added -T.
The argument to <b>mktmp</b> --tmpdir is now optional.</p>
<p><u>Bugfixes</u>:
The unshare probe bit-rotted so it was impossible to build <b>unshare</b>
and <b>nsenter</b> on some distros.
Commit 771e94e2a08 broke toybox's ability to say "unknown command" when
you pointed a symlink at it that it that didn't correspond to a command,
pointing a symlink at another symlink
to make toybox run a command under a name it doesn't recognize
(ln -s toybox sh; ln -s sh ash) had an off by one error,
<b>ifconfig</b> no longer error_exits if the "read mac address" ioctl fails (Android N
and later block that for non-root users to prevent device tracking).
When the libcrypto library was disabled, md5sum was building commands it
hasn't got built-in support for yet (like sha3sum) which acted like redundant
copies of md5sum,
<b>tar</b> wasn't correctly reading or writing sparse files with a hole at
the end, and tar tzf blah.tar.xz was calling xz when it meant xzcat,
<b>xargs</b> would spuriously fail on large memory systems (due to a missing
"unsigned" in a typecast),
<b>basename</b> can now remove suffixes starting with a dash,
<b>cp</b> -r unlinks() an existing symlink before trying to
recreate the symlink (otherwise cp -r into an existing directory will
try to follow the existing symlink and create another symlink at its
destination, failing if a file already exists there, which
was confusing gentoo's package manager), <b>timeout</b> now produces the
right exit code when the timed command intercepts the exit signal,
and <b>hostname</b> -d no longer segfaults on machines with no domain name.</p>
<p>Denys Nykula fixed the <b>ftpget</b> freeze (it was sending REST instead
of RETR), made <b>mv</b> work with a trailing slash in the source,
fixed a <b>find</b> segfault,
and made <b>env</b> always exec rather than recursing to a builtin
(on the theory #!/usr/bin/env is expected to search the $PATH).</p>
<p>Elliott improved <b>killall</b>'s handling of long filenames, made <b>kill</b> work
when killall5 is disabled (missing FORCE_FLAGS), fixed error reporting in
<b>hostname</b>, fixed several things in <b>xargs</b>,
made <b>printf</b> \c and \0 octal escapes work, fixed a <b>find</b> -name corner
case (find src/*.c -name file.c) and find dangling symlink behavior,
made some <b>rm</b> error messages consistent, fixed \x behavior in echo and
printf, fixed linestack.c's buffer length for utf8 U+XXXX escape printing,
fixed a race in <b>losetup</b>, fixed <b>grep</b> -F with multiple matches on the same
line and an -I failure in the existing grep tests, made <b>readlink</b> notice
when it has more than one argument, and made various
fixes to <b>modinfo</b>.</p>
<p>Ethan Sommer reported that <b>stat</b> of an suid/sticky file output 5 digits of
octal stat date instead of 4.
Rob fixed an unaligned access in <b>grep</b>. Alessio Balsini removed an incorrect
null termination in <b>losetup</b>.
Jarno Mäkipää fixed <b>hexedit</b>
scrolling up and down on older terminals, and fixed <b>ls</b> segfaulting with a broken
(non-UTF8) locale (didn't expect wcrtomb to return -1).
Andrew Ilijic made ls put 2 spaces between columns like other implementations
(which helps when terminals get confused and apply UTF8 combining characters
to the space after a filename).
</p>
<p><u>Library</u>:
Added dlist_lpop() to use a dlist as a stack (it removes the last entry
instead of the first), and dlist_pop() now works on a dlist_terminate()d list.
DIRTREE_STATLESS returns entries we couldn't stat() (with a zeroed ->st
field, and ->again |= 2), but filling in the file type from readdir().
Coversions between signal names and numbers now include all the Linux signals,
not just posix ones. The functions sig_to_num() and num_to_sig() moved into
lib/portability.h, because MacOS has different signals.
TOYFLAG_MAYFORK allows commands to run in toysh's process without forking,
but also makes them accessable from the toybox multiplexer or standalone.
A MAYFORK command has to clean up after itself (even in error paths) and
can't discard anything we need to keep (such as closing stdout).</p>
<p>Elliott did a big xbind/xconnect cleanup (adding xbindany and xconnectany),
taught xregcomp that an empty regex matches the whole line
(because FreeBSD, and thus MacOS, doesn't already do that), and
replaced several uses of get_line() (which does single byte reads)
with getline() (which doesn't).</p>
<p><u>Plumbing</u>:
"make clean" doesn't produce a screenful of unnecessary output now,
rm, ln, cp, kill, and netcat use the FLAG() macros now, makedevs uses the new
GLOBALS naming format, timeout now uses xwaitpid(), and we fixed an
off by one error in xwaitpid().</p>
<p>Elliott removed a workaround for old NDK versions from getconf and
moved sort off of get_rawline() (which let us remove it).
Rob undid some loop unrolling in md5sum/sha1sum (we have libcrypto if you want
an assembly optimized version instead of an understandable version).</p>
<p>Android ndk-r20 doesn't define the __ANDROID_NDK__ symbol, so we switched
to using clang's __has_include() (which is an undefined macro and thus a NOP
resolving to false on gcc).</p>
<p><u>Pending</u>:
Rob did a lot of work on toysh: continuation support works (it knows when to ask
for another line of text and when to run what it's got), as do
nested if/else/elif statements, and while and for loops (supporting "for i"
and "for i in", but not for ((;;)) yet).
It parses pipes and redirections but doesn't perform them yet.
The next big todo chunk is environment variable resolution.
</p>
<p>Gavin Howard fixed some bugs in bc, such as not returning error when raising
zero to a negative power.
Jarno Mäkipää continues to extend vi, adding yank and push, support for tabs,
rewriting delete, changing the drawing code, etc. Elliott also added several
fixes and features to vi,
fixed host to cope with large DNS responses, switched crontab to
use getline(), did work on man, fixed fixed several things in dd,
implemented diff --strip-trailing-cr and made diff not need /tmp.
Mike Bennett added extended attribute support to ip.c's route change.</p>
<p><u>Test suite</u>: The killall and pgrep tests were having occasional
spurious failures due to a race condition (the test script would fork()
to call sleep and for just a moment there were two instances of the test
script, reporting an extra process if killall or pgrep ran then).
<p>Elliott Hughes did a lot of work on the test suite: add kill -l tests,
make losetup.test notice (and skip) if loopback
devices area already in use, blkid.test can TEST_HOST e2fsprogs blkid now,
modinfo.test notices (and skips) if there's no /proc/modules or modules
directory, ifconfig.test skips if "dummy0 up" fails and disabled the
pointtopoint tests entirely (the kernel never supported it?),
truncate.test is less
confused by SELinux, hostname.test can run some tests as non-root,
addressed a race condition in killall.test on slow systems,
made chgrp.test, chown.test, and tar.test handle an empty /etc/passwd or /etc/group
(we were checking whether they were _there_ but some android devics have
empty ones),
fixed a couple things in env.test, extended killall.test to explicitly cover
long and short names (and not assume the shell is /bin/sh), taught split.test not to rely on bash process substitution,
added missing TZ=utc to touch.test, fixed cp.test to pass for root
or with a restrictive umask, and fixed the FAILCOUNT arithmetic to use more portable
$(()). Elliott also made some tests more flexible (find.test can handle any
error message about symlink loops, etc), and disabled others (such as du)
when SELinux is enabled.</p>
<p>Rob taught testcmd not to use the shell builtin for TEST_HOST unless there
isn't one in the $PATH, work if $PWD has a space in it,
and make kill.test stop testing the shell builtin
insead of the command, quieted tests that were producing output
other than their PASS/FAIL line, and made NOSPACE tests ignore the
presence/absence of leading/trailing whitespace as well as differences
in the amount of space.</p>
<p><u>Documentation</u>:
The <a href=roadmap.html>roadmap</a> got freshened up a bit, with updates
for android from Elliott.
In <a href=design.html>the design page</a> the Features section starts with
4 new paragraphs about the scope of toybox, and a new License section talks
about 0BSD.
The "./toybox --help" output now explains how to do install_flat by hand.</p>
<p><u>Sheer pedantry</u>:
Elliott fixed a double close in loopfiles_lines() so strace doesn't
have an EBADF in it, and added a missing "static" in lib/net.c.</p>
<a name="21-05-2019" /><a href="#21-05-2019"><hr><h2><b>May 21, 2019</b></h2></a>
<blockquote>
<p>"I like the cover," he said. "'Don't Panic.' It's the first helpful or intelligible thing anybody's said to me all day."</p> <p>- The Hitchhiker's Guide to the Galaxy</p>
</blockquote>
<p><a href=downloads/toybox-0.8.1.tar.gz>Toybox 0.8.1</a>
(<a href=https://github.com/landley/toybox/releases/tag/0.8.1>git commit</a>)
is out.</p>
<p>Android is now using toybox as part of a <a href=http://lists.landley.net/pipermail/toybox-landley.net/2018-November/009824.html>"hermetic" build</a>
(meaning Android is building under toybox, which is halfway to Android
building under Android). See Android's <a href=https://android.googlesource.com/platform/prebuilts/build-tools/+log>build-tools</a> and <a href=https://android.googlesource.com/platform/build/soong/+log>soong</a> repositories to follow along with development, or check the list for the <a href=http://lists.landley.net/pipermail/toybox-landley.net/2019-May/010485.html>current status</a>.</p>
<p><u>New commands</u>: Isaac Dunham added <b>mcookie</b>, which is used by
X11. Elliott Hughes added <b>devmem</b>, which replaces toolbox's "r".</p>
<p><u>Promoted</u>: <b>tar</b>, <b>gunzip</b>, and <b>zcat</b> were cleaned
up and moved out of pending.
Tar is a near-rewrite with support for --owner --group and --sparse,
capable of autodetecting "wget blah.tar.bz2 -O - | tar xv", a --restrict option
that only allows files to extract into a single subdirectory, and so on.</p>
<p><u>Added to pending</u>:
Jarno Mäkipää contributed a new <b>vi</b> implementation. Makepost added
<b>man</b>.</p>
<p><u>Updates</u>: Elliott added <b>find</b> -wholename/-iwholename, -printf,
and -context, <b>grep</b> --exclude-dir, and <b>echo</b> -E.
Rob added <b>blkid</b> -U and -L, <b>mount</b> now has UUID= support via blkid -U,
you can now specify TZ="blah" in the <b>date</b> -d input string (although
why the double quotes are required is an open question), and <b>sed</b> now
understands ,+N relative ranges. Makepost added <b>rm</b> -v.
The human_readable fields in <b>ps</b>/<b>top</b>/<b>iotop</b> now use all
available space to show more precision, <b>iotop</b> displays swap bandwidth
properly (it was showing a signed field as unsigned), and
<b>touch</b> -t and -d autodetect the time format.</p>
<p><u>Cleanups</u>: In pending Rob did some bc cleanup,
Colin Davidson did some arp cleanup,
Uros Prestor fixed route argument parsing, and
in bc Daniel Rosenberg fixed bc_vec_concat()'s allocation and
Gavin Howard fixed an overflow bug in bc_num_ulong().</p>
<p>Elliott made scan_key() detect cursor keys and such with several more
terminal types, made more better about directories and non-tty output,
cleaned up inconsistent uses of ARRAY_LEN(), taught file
to recognize a c-sky ELF machine, and fixed timezone and daylight
savings time issues in date and its tests.
Rob simplified netcat and ping using new lib/net.c infrastructure.
When run as non-PID 1 (for a chroot) oneit's exit code now indicates
which signal it's exiting due to.
Hadrian Węgrzynowski #defined constants for mkpathat.
Date now uses the current time for unspecified fields (instead of zeroing
them), and should set the weekday properly.
Sort's -k2.3,4.5 syntax is now documented and behaves more closely to
other implementations (silly as that behavior is).
Significant speedup to sed s/a/b/g on a megabyte long line (it was
O(n^2) and now it's quite snappy).</p>
<p>Several cases of warning about stdout failing to write were silenced
because things like piping the output to head means stdout is a special
case: its output is often discarded and that's fine. And xprintf()
and friends only checks stdout for errors so it can exit early ("yes | head"
should not spin endlessly) without actually flushing anything.
There's an ongoing design debate about line buffering stdin and stdout
that'll have to be resolved next dev cycle.</p>
<p><u>Bugs</u>:
Elliott sent in a lot of fixes for building AOSP with toybox:
support for more date formats, sed substitution of empty capture groups
and an endless loop with -EPIPE, fixed find -path/-ipath, made loggit()
only feed LOG_AUTH to syslog when we have a tty, gave grep a bunch
of --longname synonyms for short options, added getfattr --only-values,
and timeout --foreground --preserve-status and --signal.</p>
<p>Top got a lot of tweaks: it no longer collates thread's CPU usage into the
parent thread (in -H mode), hides the cursor in interactive mode, updates the
display properly in batch mode, and accepts ENTER
as a synonym for SPACE when refreshing the display. The header lines
now abbreviate long numbers with units (via human_readable()),
more reliably spans the terminal width, and says "Thread" instead
of "Task" when it's showing threads. The task/thread count now adds up
(understanding more kernel states and treating unknown ones as "sleeping").
Top now treats -o "" as an error (previously it displayed an empty screen).</p>
<p>William Djupström pointed out login needs to chown() the tty, and made it
clear the environment when -p is absent.
Vidar Holen fixed inverted find -exec exit status.
Rob made chgrp() stop relying on libc's printf("%s", NULL) printing "(null)"
because a <a href=https://github.com/varnishcache/varnish-cache/issues/2879>newly added gcc bug</a> turns that into a build break, and
fixed an outright embarassing xstrndup() bug.
Liwugang fixed a segfault when stat's format string had a trailing %.
A missing else in grep meant -x was sometimes ignored.</p>
<p><u>Infrastructure</u>:
in the build scripts/portability.sh now tests that the compiler it's trying
to use exists up front. We no longer link against librt because on glibc that's
pulling in libpthread for some unknown reason.</p>
<p>TOYFLAG_ARGFAIL() lets commands return custom exit values when option parsing
fails (because AOSP build scripts cared), so timeout --unknown returns 125
and so on: chroot, diff, cmp, env, grep, nohup, and sort also use it now with
various values.</p>
<p>New FLAG() macros: FLAG(x) expands to (toys.optflags&FLAG_x) and returns
zero if the command line argument's bit isn't set and nonzero if it is
(meaning -x was or wasn't encountered on the command line this time by
lib/args.c). Lots of commands got converted to use them.</p>
<p>New lib/env.c with reset_env(), xsetenv(), and xunsetenv() which frees
the old environment variable when it wasn't inherited from exec(). This
lets long-running loops reset environment variables without memory leaks.</p>
<p>New xnotify() plumbing in lib/portability.c with Linux and Mac versions
(which should also work on BSD).</p>
<p>New xparsedate() and xvali_date() functions allow date, tar --mtime, and
touch to understand the same date formats,
xpopen_both() now knows how to use existing filehandles for the child's
stdin/stdout, and no longer redirects stderr (which hides errors),
new nanomove(), nanodiff(), union socksaddr, xrecvwait(),
and human_readable_long() to show more than 3 digits of output.</p>
<p>Taught xgetaddrinfo() to return a wildcard address for a NULL host,
xconnbind() sets SO_REUSEADDR by default,
fileunderdir() returns the abspath to the file now (since we had to work
it out anyway), and
xabspath() uses O_PATH now (because Android's SELinux rules got more aggressive).</p>
<p>Several new xsendfile() variants, including sendfile_len() to copy
a specific amount of data and xsendfile_pad() to append zeroes to the
output when there isn't enough input.</p>
<p>regexec0() is now using libc's REG_STARTEND supported by every libc out
there _except_ musl. (It originated in BSD and was adopted by glibc, bionic,
and uClibc 15 years ago, and is also supported in macosx. Musl's maintainer
<a href=https://www.openwall.com/lists/musl/2013/01/15/26>turned down</a>
the feature request as an invalid use case he didn't think they should
be trying to do.)</p>
<p><u>Docs</u>:
the <a href=license.html>license page</a> now has an SPDX identifier, and links to more
history/context. The actual LICENSE file had its title line removed
so <a href=https://github.com/github/choosealicense.com/pull/643#issuecomment-463746027>github could recognize it</a> as 0BSD. (We're still waiting for
"<a href=https://github.com/github/choosealicense.com/pull/643#issuecomment-464380789>a while</a>" to pass before github actually does, though.)</p>
<p>A new www/doc/mount.txt file describes how mount works under the covers.</p>
<p>Rob consistently misspelled "canonical" and "millisecond", mountpoint
has a synopsis now,
Kevin van der Kamp fixed a typo in netstat's help text, and Elliott
Hughes fixed typos everywhere and made the tense, capitalization, and
punctuation of help text more consistent.</p>
<p>New paragraph in the code style part of design.html explaining that
"char* a, b;" is not how C works, and since you wind up with "char *a, b;"
anyway you might as well say that.</p>
<p>Given dreamhost's tendency to repeatedly delete the mailing list
archive, the nav bar on the left links to a backup web archive now too.</p>
<p>The FAQ now says why we do time based releases.</p>
<p><u>Testing</u>: Added VERBOSE=nopass to not show passing tests.
The test suite now does an rm -rf on testdir between each command so
debris files don't accumulate.
New skipnot function skips the next test if a command line fails, and
toyonly function only runs command when we're testing the toybox version
of command. (This isn't the same as TEST_HOST, the host version could be
toybox.)
The find -newer test calls "sleep .1" to avoid intermittently failing with an
identical timestamp.
Elliott added the start of a file.test rogues gallery.
No longer call lsattr in "make tests" because the behavior differs by
filesystem and it needs fixing somehow.</p>
<a name="08-02-2019" /><a href="#08-02-2019"><hr><h2><b>February 8, 2019</b></h2></a>
<blockquote>
<p>"Most readers get as far as the Future Semiconditionally Modified
Subinverted Plagal Past Subjunctive Intentional before giving up; and
in fact in later editions of the book all pages beyond this point have
been left blank to save on printing costs."</p>
<p>- The Hitchhiker's Guide to the Galaxy</p>
</blockquote>
<p><a href=downloads/toybox-0.8.0.tar.gz>Toybox 0.8.0</a>
(<a href=https://github.com/landley/toybox/releases/tag/0.8.0>git commit</a>)
is out.</p>
<p>Toybox now builds on MacOS and FreeBSD, thanks to the efforts of Elliott
Hughes and Ed Maste respectively. Use the "make macos_defconfig" and "make
freebsd_defconfig" targets to enable the set of commands that compile on
each so far.</p>
<p><u>New commands</u>: Added an <b>sntp</b> client/server (RFC 4330 Simple Network Time
Protocol, a compatible subset of ntp). The <b>test</b> command was rewritten
and promoted out of pending.</p>
<p><u>New options</u>: <b>grep</b> now has --color, supports embedded NUL bytes in its input,
recognizes binary files, and passes all of grep.test (in case you
needed the middle numerical field of -bB, etc).
Josh Gao added ipv6 and UDP support to <b>netcat</b>.
Volodymyr Medvid reported that install -d doesn't honor -o or -g.
Elliott Hughes did a lot of work to support hermetic Android/AOSP builds:
<b>mkdir</b> accepts both --parent and --parents as synonyms for -p, <b>touch</b> ignores -f,
<b>basename</b> added -s to remove a trailing suffix, <b>dirname</b> now supports multiple
arguments, <b>cmp</b> accepts --quiet and --silent as synonyms for -s, <b>hostname</b>
added -sfd, <b>head</b> added --bytes as a synonym for -c and --lines as a synonym
for -n, <b>mktemp</b> added -t and fixed -u, <b>sed</b> added -z and -iEXT to keep backup files,
<b>md5sum</b> and sha1sum added --status and --check as synonyms -s and -c,
<b>readlink</b> added --canonicalize as a synonym for -f, <b>sort</b> grew -V,
<b>patch</b> added -s its synonym --quiet, <b>stat</b> added --format as
a synonym for -c, <b>xargs</b> added -p -t -r,
Eduardas Meile asked
that <b>umount</b> ignore -c. Reverend Homer added a small optimization to file.c,
and Elliott taught <b>file</b> to recognize riscv ELF binaries. Peter Collingbourne
taught <b>ls</b> -t to use the nanoseconds field.
<b>patch</b> has better support for patching a file with a tab in the name.</p>
<p><u>Bugfixes</u>:
<b>cp</b> --preserve was segfaulting when you didn't specify _what_ to
preserve (it now correctly defaults to "mot") and didn't get the permissions
right when copying a symlink's contents as a regular file,
<b>sort</b> -x didn't work when
attached to a key, <b>host</b> didn't allocate a big enough buffer for worst
case ipv6 address size, <b>sed</b> needed a ; between b and } when other
implementations don't (we're already well past what posix says but a script
out in the field broke...),
and several fixes to <b>hostname</b> in a container, <b>ps</b>/<b>top</b> were
misreading the VIRT and SHM fields.</p>
<p><u>Pending</u>: Gavin Howard updated <b>bc</b> and added more tests,
Marius Adaskevicius pointed out <b>mdev</b> -s should follow symlinks,
Yangchun Fu reported a <b>dhcp</b> checksum bug, <b>modprobe</b> needed errno reset
to avoid reporting spurious errors in verbose mode and no longer
exits with status 1 if it can't find /etc/modprobe.conf, the <b>more</b> command
wasn't always flushing stdout when it exited.</p>
<p><u>Build</u>:
<b>chrt</b> no longer #warns about the musl sched_get_priority_min() bug, but
instead works around it.
We were mixing setjmp/siglongjmp (harmless in some contexts but it kind
of annoyed FreeBSD), and
make install_airlock now adds sha256sum (because the Linux 4.20 build now
needs that for the s390x target).</p>
<p><u>Coding style</u>:
Rob converted the rest of the option GLOBALS() to the new single letter
coding style, and the new FLAG(x) macro is a slightly tidier way to say
"toys.optflags&FLAG_x".
Removed CFG_SORT_BIG (the sort command always
has the full functionality now. The general future direction or toybox
is to either have a command or not have it; multiple versions of the
same command aren't worth the complexity in testing, documentation,
or system adminstration).</p>
<p><u>Library code</u>:
The mkflags plumbing can now support arbitrary punctuation as option
names via an escape syntax (because mkfs.vfat specifies an offset with -@),
and lib/args.c now only sets FLAGS_NODASH when the first argument
didn't have a dash (allowing "ps ax" and "ps -ax" to behave differently).</p>
<p>Added an xrename() function,
xchdir() has better error reporting, xconnect()/xbind() had their
implementations merged, xsendto() moved from ping to lib, xpoll() now
measures elapsed time and only waits for the remaining period when restarted,
and Eduardas Meile converted several error_exit() to error_exit_raw() (because
yocto 2.6 buils with -Werror=format-security by default).</p>
<p>There's a scripts/portability.sh now which lets the build use gsed and gmake
(when available) in more places to avoid broken host versions on MacOS and FreeBSD.
The syntax of the "noreturn" attribute
changed slightly, some calls to strnstr() were replaced with strcasestr(),
we work around MacOS' lack of features.h, portability.h can now do Apple's
endianness macros, and so on.</p>
<p>Following the <a href=faq.html#support_horizon>seven year rule</a>
Elliott removed support or glibc 2.10, and also
reformatted a lot of --help text for consistency (removing trailing
periods and such).</p>
<p><u>Test suite</u>:
The test suite now has its own version of "pending", specifically "make tests"
skips test files without the executable bit set (unless you export $TEST_ALL).
This eliminates "expected failures" (I.E. files with unresolved todo items)
from the global regression test. Additional tests can be added to the global
regression test with chmod +x as the todo items they document get resolved.</p>
<p>Several places the test suite got fluffed out, including guards to skip
root-only tests when run as a normal user. The testcmd function prints the
short name instead of the full path to the command.
The tests for the test command
("test.test") no longer test the shell builtin but correctly test the
toybox command. (Now say that ten times fast.)</p>
<p>Bash version skew required replacing "continue" with "return" to stop
tests early in bash 4.4 (for VERBOSE=fail).</p>
<a name="31-10-2018" /><a href="#31-10-2018"><hr><h2><b>October 31, 2018</b></h2></a>
<blockquote><p>
"In the beginning the Universe was created.
This has made a lot of people very angry and been widely regarded as a bad move."</p><p>- The Hitchhiker's Guide to the Galaxy
</p>
</blockquote>
<p><a href=downloads/toybox-0.7.8.tar.gz>Toybox 0.7.8</a>
(<a href=https://github.com/landley/toybox/releases/tag/0.7.8>git commit</a>)
is out (a month late).</p>
<p>Toybox <a href=http://lists.landley.net/pipermail/toybox-landley.net/2018-October/009769.html>now builds</a> with the Android NDK (r18 release), almost out of the box.
You have to add an "llvm-cc" symlink to "clang", then use CROSS_COMPILE=llvm-
with the appropriate $PATH (or absolute path in CROSS_COMPILE) for the
target you want to build for. (To run it on a non-android host, you probably
want "LDFLAGS=--static" too.)</p>
<p><u>New commands</u>: <b>i2cdetect</b>, <b>i2cdump</b>, <b>i2cget</b>, and <b>i2cset</b>
courtesy of Elliott/Android. The <b>watch</b> command got a complete (tty-aware)
rewrite, and was promoted out of pending. The <b>prlimit</b> command is back
(a configuration bug was always disabling it), and <b>ascii</b> now defaults y
in defconfig (an oversight).</p>
<p><u>New options</u>:
The <b>ifconfig</b> command added -S (short view) giving one line of
information per interface, in "name ipv4/mask macaddr [ipv6/mask@type...]"
format. <b>grep</b> grew binary file detection, with -I and -a options to
ignore them or force treating them as ascii (really utf8).
Zach Van Rijn asked for <b>find -empty</b>.
In the <b>file</b> command, Elliott improved java
.class support and added .wav, .bmp, and android .dex.
Elliott also added "getconf -a" and a lot of missing getconf symbols the
AOSP build uses, including support for pathconf(3) symbols requiring two
arguments. This means getconf's arguments work more like ubuntu's
now (one or two arguments, not iterating through the supplied list and
processing each like last release), and -l grew section identifiers (so
the "getconf -l" output is no longer a directly consumable list of symbols
it takes).
Command line options specifying durations (like top -d or ping -i) can
now use fractions and units, like "1.5" and ".1m". See "toybox --help" for
details.
Toybox now dereferences one layer of symlinks if it doesn't recognize the
name it's called under (so if you "ln -s $(which sleep) blah" and then
"./blah 30", it should figure out you mean sleep 30).</p>
<p><u>Pending</u>:
Gavin Howard contributed a large new <b>bc</b> implementation to pending,
which covers everything the kernel build needs (and more) but is going to
take a while to review.
In <b>diff</b> the android guys added --color and timestamps on the
+++/--- lines. In <b>mdev</b> Faustas Azuolas Bagdonas added support
for mdev.conf's fourth field. Reverend Homer suggested an error message
improvement in <b>wget</b>, Kevin Spiteri did a cleanup pass on test.c,
and there was a cleanup pass on the passwd command.</p>
<p><u>Bugfixes</u>:
Elliott responded to a bug in <b>getconf</b>'s sed-based header generation by
replacing it with a more conventional array of #defined symbols, with #ifdefs
for missing symbols on the three libc implementations (glibc, bionic, musl)
we've tested so far. (This requires portability.h work to add new supported
build environments, but at least it works on bionic now.)
The <b>file</b> command now fails gracefully when reading pathologically
broken ELF files (the tests for integer overflow missed one), and
<b>date</b> no longer shows a meaningless errno when it can't parse the
date format.
Several fixes in <b>ping</b> (-w and -W didn't work, -c kept sending packets
while waiting for replies, and when no reply packets were received the summary
data was printed twice).
Elliott fixed <b>wc</b> column widths (traditional reality doesn't match
posix), fixed <b>modprobe</b>'s error handling (reported by Wen Xie), and
also fixed the units on blockdev --blkraget (reported by Martijn Coenen).
Several small fixes to the <b>stat</b> command's output (help text, quoting
style, leading zeroes, switch -f from %t to %T), <b>oneit</b> no longer
reboots the system when it's not called as PID 1.
Last release's commit letting $STRIP be redefined had a typo.
Nick Kralevich added an istty() check before calling TIOCGWINSZ (which
triggered Android's strict ioctl filtering on non-filesystem fifos).
An unstripped build (where the "strip" command fails for a given
CROSS_COMPILE) will now overwrite the (read-only) output file instead of
failing a rebuild. The dirtree plumbing no longer prints a "No" on front
of the filename in error messages (such as "permission denied").</p>
<p><u>Docmentation</u>
Elliott updated the roadmap with Android AOSP status, towards turning that
into a hermetic build (hopefully with an eventual self-hosting option, work
is <a href=b33d37d6f735>already underway</a> on the airlock step).
Rob added a little more context for LSB and described the move from Aboriginal
Linux to mkroot.
The <b>ps</b> help text changed: now "ps -o help" shows the fields and the normal
--help is much more manageable. Ifconfig's --help text also got tidied up.
Our LICENSE file has the SPDX identifier (0BSD) now, and the note about
the kconfig/ subdirectory still having (build only, non-shipping) GPL code
in it moved into the kconfig subdirectory.</p>
<p><u>Coding style change:</u>
GLOBALS() now uses the argument letter as the variable
name for variables automatically set by command line arguments via lib/args.c,
and no longer requires they be one per line.
Removed use of the gcc
\e extension in string constants, replacing it with \033.</p>
<p><u>Library code:</u>
lib/interestingtimes.c was renamed to lib/tty.c.
The deflate code from toys/pending/compress.c moved to lib/deflate.c,
which gunzip.c now uses when it's not configured to use zlib. (This reduces
the bootstrap dependencies for the "airlock" step of a hermetic build.)
The compression side remains a todo item.
Added "%" to lib/args.c reading seconds (including fractions/units) into a
long milliseconds argument. Redid xparsetime() to not need floating point,
added xparsemillitime() for milliseconds common case, and
parsetime() now treats leading garbage as an error instead of returning zero.
The bufgetpwuid()/pufgetgrgid() functons can now handle more than 4k of group
data. The set_terminal() function can now set serial speed, and microcom.c
uses it. Added xsignal_flags(), and more consistently use xsignal() (which wraps
sigaction()) instead of signal().
A new xgetrandom() function calls the new getrandom() system call, falling
back to /dev/urandom on older kernel/libc (made to actually _work_ by
Patrick Oppenlander).</p>
<p><u>Build infrastructure:</u>
Updated dependency detection so "make distclean defconfig toybox" now works
all on the same command line. (This is nice when $CROSS_COMPILE changes).
scripts/config2help.c no longer includes parts of lib/ and just copies what
it needs into itself, improving build portability.
The build now checks the specified cross compiler exists, hopefully providing
a more informative error message when it doesn't. Similarly, when .config is
missing the error message suggests running make defconfig.</p>
<a name="23-06-2018" /><a href="#23-06-2018"><hr><h2><b>June 23, 2018</b></h2></a>
<blockquote>
<p>There is a theory which states that if ever anyone discovers exactly what
the Universe is for and why it is here, it will instantly disappear and be
replaced by something even more bizarre and inexplicable. There is another
theory which states that this has already happened.</p>
<p> - The Hitchhiker's Guide to the Galaxy</p>
</blockquote>
<p><a href=downloads/toybox-0.7.7.tar.gz>Toybox 0.7.7</a>
(<a href=https://github.com/landley/toybox/releases/tag/0.7.7>git commit</a>)
is out.</p>
<p><u>New commands</u>: Rob Landley added <b>getconf</b>, Elliott Hughes added
<b>uuidgen</b>, and <b>ping</b> and <b>fmt</b> were promoted out of pending.</p>
<p><u>New options</u>: The <b>cp</b> command added the --parents option,
<b>readlink</b> now has -m to show where a missing path would be,
<b>netstat</b> got some updates (better nommu support, it can use /etc/services
names, more command line options are bounds checked),
Ng Zhi An added --getra and --setra (get/set readahead) to <b>blockdev</b>,
Elliott Hughes added <b>xxd</b> -o and made <b>top</b> -d understand fractional
sections (because the Linux Test Project uses that), and
Minghui Liu added <b>watch</b> -b.</p>
<p><u>Bugfixes</u>: Several fixes to <b>vmstat</b>: Elliott Hughes made it work on a
48-core machine
and fixed a glitch with -n, Haroon Maqsood pointed out "vmstat 1 1" didn't
stop and "vmstat -n" didn't print the first theader line.
Minghui Liu fixed a bug where <b>cp</b> -p didn't preserve
timestamps, and pointed out the b, c, and d suffixes on numbers weren't
working. Ryan Pritchard pointed out that <b>du</b> -d0 should act like du -s.
Filip Perich made an RFC2347 OACK compliance fix to <b>tftp</b> in pending.
Zach van Rijn fixed a bug in <b>xxd</b> causing incorrect translation of
uppercase characters.
Elliott fixed several things in <b>top</b> (removed spurious '\r' characters from -b
output, removed interactive flicker, made running processes bold), and
pushed Rob to make <b>file</b> work better recognizing things on stdin
("cat /bin/ls | file -" still won't work but "file - < /bin/ls" should).
Rob fixed a bug in <b>netstat</b> on 64 bit big endian systems,
and fixed <b>cut</b> -DF
(a posix compliance fix broke its ability to act as a decent awk replacment
because outputting all delimiterless lines isn't what you want there, -D
now disables that too).</p>
</u></p>
<p><u>Documentation</u>: Rob added a buildroot section to the roadmap with the
prequisites that needs to run, and reformatted more of the ps --help text
to two columns.</p>
<p><u>Library</u>:
FLAGS_NODASH is now set in toys.optargs when an optstring starting
with & has no dash in its first argument. (This lets "ps -ax" and "ps ax"
behave differently.) Factored out xtestfile() into lib/.
The comma-separated-list parsing infrastructure moved to lib/commas.c.
Added mkpath() for the common case of mkpathat() and #defined MKPATHAT_*
constants for the uncommon cases. Elliott added better error checking
to xparsetime() and fixed a bug in names_to_pid() (used in <b>pidof</b>
and <b>killall</b> and such).
Rob inlined the old toys/e2fs.h into pending's mke2fs,
and removed a rogue semicolon from the WOULD_EXIT() macro that screwed
up if() statement levels. The do_lines() semantics changed adding a
callback(0,0) at the end of each file.</p>
<p><u>Tests</u>:
The test_* commands under toys/example were all renamed demo_* so
they don't share a namespace with running a standalone command
through the test suite (ala "make test_sed"), then
demo_human_readable became demo_number so it can also test atolx()
suffixes. The readlink.test was getting confused by a
behavior difference between toybox and ubuntu (Ubuntu's ln -sf wouldn't
replace a symlink that pointed to itself, toybox's would), now there's
an explicit rm so it works on ubuntu too. The test suite plumbing's
error message for a test with the wrong number of arguments now
correctly identifies which test it's complaining about, and it can now
run a test on a command name beginning with "-" (none yet but it
comes up in toysh).</p>
<p><u>Building</u>: You can now override strip with $STRIP. The set of
"make install_airlock" commands got tweaked for the 4.17 kernel (which
needs bison and flex because the new kconfig plumbing doesn't have _shipped
versions of those generated files like the old kconfig did), and for
the new release (remove ping, toybox provides it now).
$LDFLAGS now applies to library probing (the android NDK provides
dynamic libraries without corresponding --static versions).
Elliott took a CFG_TOYBOX_DEBUG check out of an error_exit() to make
clang stop complaining about an error that should never happen.</p>
<a name="24-02-2018" /><a href="#24-02-2018"><hr><h2><b>February 24, 2018</b></h2></a>
<blockquote><p>
"Arthur," said Ford.<br />
"Hello? Yes?" said Arthur.<br />
"Just believe everything I tell you, and it will all be very, very simple."<br />
"Ah, well, I'm not sure I believe that."<br />
</p>
<p> - The Hitchhiker's Guide to the Galaxy</p>
</blockquote>
<p><a href=downloads/toybox-0.7.6.tar.gz>Toybox 0.7.6</a>
(<a href=https://github.com/landley/toybox/releases/tag/0.7.6>git commit</a>)
is out, a month and change late.</p>
<p>This release adds a <b>crc32</b> command and
cleans up and promotes <b>iconv</b>, <b>logger</b>, and <b>setfattr</b>.
Pending added <b>stty</b> and <b>fmt</b>, the other half of <b>ping</b>,
and got a lot of cleanup on <b>gzip</b>/<b>gunzip</b>/<b>zcat</b>.</p>
<p>The <b>cut</b> command got a complete rewrite adding support for multiple
ranges, utf8/unicode characters, and new -CDFO options. Use -C to measure
columns rather than characters (for double width chars), and "cut -DF 3-5"
selects words (like "awk '{print $3 $4 $5}'").</p>
<p>This release adds <b>top -m</b> and <b>iotop -H</b> (and fixed top width truncating for narrow terminal windows),
added <b>iconv -c</b>, fixed <b>ps -T 1234</b> to show threads belonging to
that PID, improved <b>file</b>'s executable identification (properly recognizing
both endiannesses and adding sh/frv fdpic, bpf, and new microblaze
identifiers), and <b>blkid</b> now only shows LABEL= when it isn't blank.</p>
<p>Running ./configure now does "make defconfig", help_exit() now shows the
"See %s --help" message on the same line, and several web pages
(about, code, roadmap, cleanup) got updates.</p>
<p>Elliott and the Android devs added <b>df -i</b>, <b>uptime -p</b>,
<b>time -v</b>, and <b>xxd -i -ri</b>,
fixed <b>id -G</b> to only output the group IDs, fixed the <b>xargs -0 -n</b>
combination, made <b>find -exec +</b> obey ARG_MAX (just like xargs), made
<b>killall</b> kill scripts too, fixed an off by one error in <b>pidof</b>
that could cause false positive matches in 15 character names, added
gzip/gunzip/zcat tests, and taught <b>file</b> to identify more stuff
(ogg, TrueType font, LLVM bitcode, PEM certs, and PE executables), and
for some reason needed TOYBOX_VENDOR in the version stuff.
Android <b>getprop</b>/<b>setprop</b> now allows ro. properties to have
arbitrary lengths, <b>losetup</b> can now find loop devices under /dev/block (where android puts
them for historical reasons), and Andreas Gampe pointed out a thinko in
<b>top</b> that triggered llvm's address sanitizer checks.</p>
<p>Gael PORTAY asked for mdev to work when Linux has the
block layer configured out, Patrick Oppenlander pointed out a problem
with make bloatcheck's build dependencies, and
xuphung on github fixed config2help building on MacOS X.</p>
<p>Commands with locale support now setlocale(LC_TYPE, "C.UTF-8") for more
targeted locale support (enable character parsing with
utf8 support, use "C" semantics otherwise). The toys/example/test_*.c
commands got renamed demo_*.c.</p>
<p>In lib.c the millitime() function returns current unix time in milliseconds
(which as a 64 bit number is good for + or - 292 million years from
midnight Jan 1 1970), and the old xgetty() function became tty_fd() returning
-1 instead of erroring out.</p>
<p>The "make help" text moved from the Makefile to scripts/help.txt,
ls.c now lists some of its deviations from posix, and work is underway
to make ps.c more intelligible (and figure out how to break it up into
multiple files).</p>
<a name="12-10-2017" /><a href="#12-10-2017"><hr><h2><b>October 12, 2017</b></h2></a>
<blockquote><p>Would it save you a lot of time if I just gave up and went
mad now?</p>
<p> - The Hitchhiker's Guide to the Galaxy</p>
</blockquote>
<p><a href=downloads/toybox-0.7.5.tar.gz>Toybox 0.7.5</a>
(<a href=https://github.com/landley/toybox/releases/tag/0.7.5>git commit</a>)
is out.</p>
<p>This is another <b>mostly bugfix release</b>. I delayed it a bit trying to get some
of the many half-finished projects (dd, ping, lsof, iconv, cut, sysconf,
the cp --parents option...) finished/promoted/tested, but decided adding big
things at the end of the dev cycle would be too destabilizing.</p>
<p>A couple <b>new features</b> slipped in anyway. Ilya Kuzmich added head -c
and strings -t (and corrected strings' output in a couple places). Elliott
Hughes taught file to recognize Macintosh (Mach-O) binaries.</p>
<p><b>NOTE</b>: the chrt command is broken when built against musl-libc
because that project's maintainer decided he didn't like the system calls
it depends on, so he
<a href=https://git.musl-libc.org/cgit/musl/commit/?id=1e21e78bf7a5>removed
them from his libc</a>. The prebuilt binaries are built against musl,
so the chrt command there will always say -ENOSYS. (It works fine built against
glibc, and presumably bionic or uClibc.)</p>
<p><b>Bugfixes</b>: Remounting (mount -o remount,rw) should work properly now,
and we fixed another place ps was segfaulting when /proc
entries vanished out from under us (a hard to hit race condition
mostly noticeable in long-running "top" exiting; error value was treated
as a pointer). Elliott Hughes fixed a -Wformat warning in expr.c,
and made xargs split the command
line at ARG_MAX (which is 1/4 the stack size ulimit, which normal users
can arbitrarily increase but execve() will complain if they don't).
Josh Gao made netcat -l exit after handling a request, and reported a
seq bug (seq "1000000 10000001" output 1e+06, fixed now). Zach Riggle fixed a
broken URL in the README. Rob fixed env -0 checking the flag wrong so
it didn't work in combination with other flags (typo).</p>
<p><b>Plumbing</b>:
Replaced mbrtowc() with a new utf8towc() that doesn't have a context
struct or care about locale. A bugfix in comma_scan() (wasn't removing an entry at the end
of the list) is why remount _sometimes_ worked (depending on argument
order), xgetaddrinfo() is now separate from xconnect(),
atolx_range() learned the w (word=2) suffix, and b is now (block=512) instead
of (byte=1).</p>
<p><b>Build</b>: Patrick Oppenlander added a workaround for a bug in config2help.c
that resulted in segfaults on newer toolchains. (It was actually a use
after free error; the build infrastructure isn't nearly as heavily audited as
code that gets installed on the target; oops.) A new GITHASH
environment variable can force the build version from
the command line, and the build checks for .git in the top directory
before asking git what our version is (so it isn't confused by ../../.git
in an enclosing directory). Added a build #warning about musl intentionally
breaking chrt (as with nommu fork() musl now provides a broken
stub function so compile-time probes for its existence think it's there, and
you can't run the result to test behavior when cross compiling).</p>
<a name="19-06-2017" /><a href="#19-06-2017"><hr><h2><b>June 19, 2017</b></h2></a>
<blockquote><p>It is a well-known fact that those people who most want to rule people are, ipso facto, those least suited to do it.
To summarize the summary: anyone who is capable of getting themselves made President should on no account be allowed to do the job.</p>
<p>- The Hitchhiker's Guide to the Galaxy</p>
</blockquote>
<p><a href=downloads/toybox-0.7.4.tar.gz>Toybox 0.7.4</a>
(<a href=https://github.com/landley/toybox/releases/tag/0.7.4>git commit</a>)
is out. No new commands this time, but
<b>chrt</b> and <b>dmesg</b> got promoted out of pending.</p>
<p><u>New features</u>:
Rob rewrote paste, which should work much better now, and added grep
-M and -S to match and skip wildcards respectively (useful with -r).
Elliott's updated dmesg has -T and --color. The file
command can recognize gzip now, uptime grew -s, date grew %N, env knows - as a first argument
means -i (posix!) and grew -0, ls defaults to -b
instead of -q now when there's a tty, and ls has a new -ll option (with
--full-time as a compatibility synonym) showing nanoseconds and (for some
reason) timezone. (Why do individual files have timezones?) Elliott added
"uudecode -o -" support. Illya Kuzmich taught head -v and -q. The cpio
code no longer adds the "TRAILER!!!" entry by default (initramfs extractor
doesn't care) without which you can concatenate cpio archives with "cat".
(Use the new --trailer option if you want the legacy behavior.)
In pending, fdisk compiles now and tar understands bzip2.</p>
<p><u>Build</u>:
The "make install_airlock" target now symlinks bc from the host because
the kernel <a href=https://landley.net/notes-2013.html#28-03-2013>inexplicably</a>
needs that to build. This was motivated by
<a href=https://github.com/landley/mkroot>mkroot</a>, which builds under
a toybox airlock directory.</p>
<p>Lots of work on the test suite, mostly from the Android guys who are now
running it under Android. This fixed several existing tests that didn't
pass, made more tests run on a toybox-only system, and so on. The test suite
infrastructure now has a second testing function,
"testcmd", which supplies the command name being tested (bypassing
shell builtins).</p>
<p>Various android build and config fixes, getting closer to being able
to let android someday use scripts/make.sh instead of generated/* snapshots.
Also more work into building under android's NDK; not quite there yet
but much closer.
Use nproc in scripts/make.sh detect available processors (so you can control the SMP level with taskset).
Removed the old uClibc compatibility glue, it's been 5 years since their
<a href=http://lists.busybox.net/pipermail/buildroot/2016-December/180102.html>last release</a>.</p>
<p>The new config option TOYBOX_PEDANTIC_ARGS checks arguments when there
are no arguments, so things like "uptime" no longer silently ignore arguments
you pass but instead refuse to run.</p>
<p><u>Docs</u>:
The FAQ now has more than one entry. Commands no longer output the full
help text for argument errors but instead just say "See %s --help" with the
command name (in addition to the actual error message).
Elliott did a big period-ectomy on all the --help text, and
we cleaned up some tab/space inconsistency. The
non-html help -a output now has separators with the command name.
The top/iotop and pkill/pgrep help text now describe a lot more of what
the commands can do. Twitter's code of conduct page went down so we
mirrored the text locally.</p>
<p><u>Bugfixes</u>:
Fixed a race condition in ps/top where a process that exited right as we
read its data returned a different error value than we were expecting (which
was causing long-running top instances to occasionally exit),
mount now gives an error if it can't autodetect the filesystem
type, ps no longer queries the terminal size when output isn't to a tty
(so "ps -A | cat" doesn't vary), date's chkmktime() was replaced with
simple range checks for fields (to avoid false positives from things like
timezones and daylight savings time), removed %s from date's help (we
didn't implement it, we have @seconds[.nanoseconds] instead), fixed
zcat's buffer flush logic (which was always failing on files larger
than 32k), and factor now detects requests for numbers >64 bits and fails
loudly instead of producing incorrect answers.
Elliott fixed touch -a/-m (they were backwards), and allowed ':' in
setprop's property names. Grep now exits with 2 for errors (so -q can
distinguish "didn't find" from "didn't work"), doesn't stop on symlinks
that point nowhere (there was an error_exit() that should just be a warning),
and provides error messages for files we could open but not read.</p>
<p><u>Library</u>:
New library functions: strend() complements strstart(), minof()/maxof()
are min/max macros that evalute arguments once and autodetect type (why
isn't this in libc?), xmmap() checks MAP_FAILED (which is not NULL).</p>
<a name="21-02-2017" /><a href="#21-02-2017"><hr><h2><b>February 21, 2017</b></h2></a>
<blockquote><p>Only six people in the Galaxy knew that the job of the
Galactic President was not to wield power but to attract attention
away from it. Zaphod Beeblebrox was amazingly good at his job.</p>
<p>- The Hitchhiker's Guide to the Galaxy</p></blockquote>
<p>Despite everything, <a href=downloads/toybox-0.7.3.tar.gz>Toybox 0.7.3</a>
(<a href=https://github.com/landley/toybox/releases/tag/0.7.3>git commit</a>)
is out. The <u>new commands</u> this time are <b>ftpget</b>, <b>ftpput</b>, <b>microcom</b>, and <b>ascii</b>.<p>
<p>We also had two command _demotions_ out of defconfig:
<b>hostid</b> got moved to toys/example and
switched to "default n" because despite <a href=http://pubs.opengroup.org/onlinepubs/9699919799/functions/gethostid.html>still being in posix</a>
the concept of a unique 32 bit number identifying a system is something
Linux outgrew about the time Pauline Middelink wrote the first IP
Masquerading code. And Elliott did a complete rewrite of <b>dmesg</b> introducing
two codepaths that I didn't get a chance to unify and didn't want to
hold up the release for, so that's back in pending.</p>
<p><u>New features</u>: Rob added units to <b>find</b> -atime and friends
(with the legacy -amin alias). Elliott added color and -w to dmesg, fallocate
-o, and improved file's ELF parsing. Steve Muckle added -d and finit_module
support to modprobe. Rob and Elliott tweaked the
ps/top display format a bit more (extending the USER field from 8 to 18 chars
and putting + at the end of string fields that got truncated).
df -a isn't entirely new, but wasn't documented and needed a bugfix.</p>
<p><u>Bugfixes</u>:
Last release broke oneit because -c didn't get moved to xopen_stdio() (oops).
Rob and Elliott simultaneously spotted ps padding each line to 99999
chars when there's no tty (serial console or adb); now it pads to 80 in
that case but also switches on -w to avoid field truncation. The "tty"
field also sometimes had trailing debris (that's fixed now). And "top" was
endlessly redrawing with out tty because receipt of the ANSI size probe
results would set SIGWINCH, and handling that sent another ansi probe. (Sigh.)
And while we're there, replace "ADDR" with "BIT" in ps -l so there are
more than 4 chars left for the "CMD" field on 64 bit systems.</p>
<p>Izabera pointed out that split -b and -l can't mix, and suggested seq should
multiply to avoid accumulating rounding errors from repeated fractional
increments. Wang Xiao Jian fixed a bug in sort -k.
Elliott let getprop use the @ character in property names, and
Dimitry Ivanov removed the name length limit for system properties.
Elliott also improved some error reporting and improved top -H's display
of thread names.
<p>Josh Gao pointed out that recursive operations on . and .. could be ignored
in chmod -R (and the resulting generic fix to dirtree_notdotdot() fixed
it in several other places).</p>
<p>Justin Cormack caught tar producing a warning to stdout that screwed up
"tar c" to stdout.
Rob fixed an option parsing bug (where switching off a --longopt in menuconfig
confused the parser), and another one where an option excluding itself
(ala "abc[-ab][!abc]" with "command -a -b") would segfault.</p>
<p>There's some sort of gcc stack over-optimization bug where musl-libc's
version of vfork() doesn't get marked with attribute(returns_twice) so
stack varabiles in the same function after that get semi-randomly overwritten
when the optimizer decides to reclaim the space. So add the attribute
to the function the XVFORK() wrapper macro calls. (It's a nommu thing.)</p>
<p>Fixed a couple variable size mismatch bugs that were only tested on 64 bit
(printf %x 64) or only tested on 32 bit (modprobe), removed some
unnecessary casts in stat.</p>
<p>Continuing attempts to build under Android NDK brought up that posix
defines the global 'stdout' as a macro, which older versions of bionic
turned into an array member, but a function was using it as an argument
name. (This worked in the AOSP build because it only builds against current
bionic, where there's a global 'stdout'.)</p>
<p>Several commits argued with clang's warning generation, eventually
settling on a variant of __attribute__((__shut_up__)).</p>
<p>Android should no longer give spurious error messages
when you "ps -A | head" about EPIPE on output. (Older versions of bionic
set an error handler on SIGPIPE, but it shouldn't do that now. More recent
versions of adb set the SIGPIPE handler to SIGIGN instead of SIGDFL,
leading to write returning an error message instead of silently killing
the program. So we set it back to the default.)</p>
<p><u>Docs</u>:
Removed website link to the gmane archive (which didn't survive gmane's
change of ownership). The FAQ now answers a _second_ question. (Woo!)
Some roadmap updates.</p>
<p><u>Build tweaks</u>:
Upgraded "make install_airlock" target to only warn about missing
commands (unless $PEDANTIC is set) when it sets up the hermetic build
path. (The plan is still to implement everything but the toolchain
binaries in toybox, but in the meantime we're symlinking other stuff from
the $HOST that isn't ready yet. See
<a href=https://github.com/landley/mkroot>mkroot</a> for an example using
this.)</p>
<p>Elliott and Rob continue to poke at building toybox with Android's NDK,
but it's a work in progress (<a href=http://lists.landley.net/pipermail/toybox-landley.net/2016-December/008767.html>thread</a>). Various changes
removing libcutils dependencies and adding an selinux dependency to getprop
are fallout from this.</p>
<p>Cross-compiling from Macs needs to use "gsed" instead of apple's
version, so teach the build to use that name if it exists in the $PATH.
If you try to build without running config first, you should get better
error reporting now. Added a workaround for Centos' broken "which" command
producing output when it _can't_ find a name in the $PATH.</p>
<p><u>Library</u>:
The new dirtree flag DIRTREE_PROC skips non-numeric entries so things
like ps and top can scan /proc more efficiently.</p>
<a name="21-10-2016" /><a href="#21-10-2016"><hr><h2><b>October 21, 2016</b></h2></a>
<blockquote><p>Probability factor of one to one. We have normality. I repeat,
we have normality. Anything you still can't cope with is therefore your
own problem.</p><p> - The Hitchhiker's Guide to the Galaxy</p></blockquote>
<p><a href=downloads/toybox-0.7.2.tar.gz>Toybox 0.7.2</a>
(<a href=https://github.com/landley/toybox/releases/tag/0.7.2>git commit</a>)
is out.</p>
<p>During this development cycle, Elliott Hughes <a href=http://androidbackstage.blogspot.com/2016/07/episode-53-adb-on-adb.html>got interviewed
on the ADB podcast</a>
and Rob Landley <a href=http://linuxluddites.com/shows/episode-88/>got interviewed on Linux Luddites</a> (<a href=http://linuxluddites.com/shows/episode-11/>again</a>).
Both talk about toybox and many other things.
The web page also grew a new <a href=faq.html>FAQ page</a>, currently with
just the one.</p>
<p>New comands added to defconfig are <b>tunctl</b>,
<b>log</b>, <b>start</b>, <b>stop</b>, and <b>sendevent</b>.
The commands <b>file</b> and <b>netstat</b> got promoted out of pending.
Pending added <b>chrt</b>, <b>setfattr</b>, and <b>getfattr</b>, and saw
a lot of cleanups to diffstat and dd but not enough to promote them to
defconfig yet. A new toys/net directory was added, moving ifconfig, netcat,
netstat, rfkill, and tunctl there so far.</p>
<p><b>Upgrades</b>: All commands now parse --version when they understand
--help, but "true" and "false" should now ignore their arguments entirely.
We taught stat to handle "%12x" and "%.12x" printf-style escapes, which
apparently other versions do. The ifconfig output now shows the interface's
device driver. Added patch -d and --dry-run, wc can now do -cm together,
find has a NOP -noleaf so scripts that use that don't break, add -c to md5sum
and sha1sum. Elliott taught ps to treat extra aguments as additional -p
pids, implemented xxd -s, did a number of upgrades to file (added -HL,
support for ar files, improved ELF support to report android API level
and stripped/not stripped and it no longer prints a guessed build ID type).
Elliott also added optional build-time support for using openssl's
assembly-optimized md5sum/sha1sum implementations (leading to a new
<a href=design.html>design</a> policy on shared libraries).</p>
<p><b>Bugfixes</b>: Too many fixes to "ps" and "touch" to list, from both
Elliott and Rob. Rob taught sed to handle s/[[:space:]/]// type sequences
properly, switched grep to a better
workaround for <a href=https://sourceware.org/bugzilla/show_bug.cgi?id=17829>glibc bug 17829</a>, made sed -i preserve ownership when run as root,
made du max out at 2 terabytes instead of 2 gigabytes on 32-bit systems
(it was always designed to, but was missing a typecast),
fixed the option parsing infrastructure (config options that remove command
line options got the placeholders wrong), fix to printf for printing
octal digits and handling the (posix-mandated) difference between %b and
non-%b octal output, reading from "-" no longer closes stdin when done,
netcat -L works with nommu (although it may need more portability work),
and you can now "make test_scankey" if you want to. Several commands
(stat, makedeves, chgrp, cp, find) handled user name lookup failure badly
(stat was segfaulting if you interrogated a file belonging to a nonexistent
user, "chown 12345 file" errored out if you didn't have that user
in /etc/passwd... now they should all print/accept the number when
appropriate). "LC_ALL=C ls -Cs --color" produces the same output
as other versions (two spaces padding, -k hardwired on).<p>
<p>Kyungsik Lee fixed a bug
in cp (readlink() doesn't actually null terminate the string it reads in),
Elliott Hughes made pgrep/pkill return success/failure, fixed trailing
whitespace in netstat, fixed a SMACK symbol conflict due to linux/xattr.h
changing, fixed ls -sh, and added a lot of
stuff to the <a href=roadmap.html>roadmap page</a>.
Izabera pointed out cmp -l and -s can't be selected at the same time,
that timeout was never actually checking -v, that ls should default to -q
when output is to a tty, and that "file -" would sometimes try to open "-"
instead of stdin.
Usischev Yury pointed out a use after free error, and that id shouldn't
call exit() directly. Matthias Urhahn pointed out that stat(2) returns
hardwired 512-byte units, so stat.c was wrong. David Hedges pointed out
that route could only handle 10 character interface names when the kernel
can do 15 (it's still in pending for a reason, but fixed). Evgenii
Stepanov found and helped diagnose one of the more subtle ps bugs fixed
this time around.</p>
<p>Calling "make test_blah" no longer causes make to error out if the last
test fails (and thus returns a nonzero error code).
Building single commands and the multiplexer used to require a "make clean"
between them (because they had different config files both of which were older
than generated/config.h so it didn't get rebuilt; now it just always
rebuilds it).</p>
<p>The defconfig build is now slightly less broken on older centos versions
(although <a href=http://lists.landley.net/pipermail/toybox-landley.net/2016-September/008664.html>the consensus</a> is that Centos is just generally broken).<p>
<p>Several commands were over-using xprintf(), which flushes its output
to check for error (something you only need to do maybe once per line,
and even then maybe only in loops because xexit() flushes and checks
ferror() for you and adjusts the exit code if we wrote stuff to stdout
that couldn't be printed). Lots of little flushes are inefficient,
so most things can use normal printf(). (Retransmission
of short writes is presumably libc's problem since it's buffering the
output and all.)</p>
<p><b>Library:</b>
New library functions readlink0() and readlinkat0() which properly null
terminates the symlink value (which the stock libc function inexplicably
doesn't).
The new do_lines() function interates reading lines from a filehandle
and calling a function on each line.
New function pollinate() factoring out netcat's poll() loop so things
like telnet can use it.
New functions getusername() and getgroupname() return a
char * given a uid/gid (and return a string representation of the number
if the lookup fails), and xgetpwnamid/xgetgrnamid were renamed to
xgetuid/xgetgid and now return an integer instead of a struct (also helping
handle lookup failures, you can still return the uid/get for "12345").</p>
<p>Switched atolx() to use long long internally.
Renamed xopen() to xopen_stdio() and made a new xopen() that never returns
stdin, stdout, or stderr (duping /dev/null into the filehandles as necessary).
New function xopenro() opens a file read only with one less argument, and
understands that "-" means stdin. New flag WARN_ONLY tells these functions
to just print a warning on failure, and return -1 instead of exiting.
Misc new functions like openro() which defaults to the WARN_ONLY behavior
and notstdio() which dup()s a filehandle up beyond stdin/out/err backfilling
with /dev/null as necessary. The WARN_ONLY flag let us remove the failok
argument from loopfiles().</p>
<p>New TOYFLAG_NOHELP disables --help processing (which "true" and "false"
should not do).</b>
<p>The test suite now has NOSPACE=1 to ignore whitespace (using diff -b to
check results), which helps TEST_HOST pass the same tests as toybox.
Fixes to chattr and date tests. It also has a new variable $C with the
absolute path to the command being tested (bypassing shell builtins),
and a function testcmd() which is just like testing() except it prepends
the command name ($C) to the test command line as well as the test
description.</p>
<a name="02-06-2016" /><a href="#02-06-2016"><hr><h2><b>June 2, 2016</b></h2></a>
<blockquote><p>When the 'Drink' button is pressed it makes an instant but
highly detailed examination of the subject's taste buds, a spectroscopic
analysis of the subject's metabolism, and then sends tiny experimental
signals down the neural pathways to the taste centres of the subject's
brain to see what is likely to be well received. However, no-one knows
quite why it does this because it then invariably delivers a cupful of
liquid that is almost, but not quite, entirely unlike tea.</p>
<p> - The Hitchhiker's Guide to the Galaxy</p></blockquote>
<p><a href=downloads/toybox-0.7.1.tar.gz>Toybox 0.7.1</a>
(<a href=https://github.com/landley/toybox/releases/tag/0.7.1>git commit</a>)
is out. (Yes, I forgot to update the --version string, but I already
uploaded the <a href=downloads/binaries/0.7.1>binaries</a>.)</p>
<p>The website has https support now, you can "make cat ps ls"
to get standalone commands (and "make list list_pending" to see what's
available), and a whole lot of bugfixes and new options to existing
commands.</p>
<h2>New Commands</h2>
<p>Rob implemented <b>ulimit</b>. In pending, Elliott Hughes implemented
file. and Lipi Lee implemented a simple wget. (Pending also had minor
cleanups to more and lsof, but no promotions this time around.)</p>
<h2>New Options</h2>
<p>Izabera implemented env -u, suggested adding seq -w, made factor
use full unsigned 64 bit math even on 32 bit platforms, pointed out base64
-w0 should disable wrapping, and sped up wc -c.
Elliott Hughes added mount -o relatime, xxd -p -r, and od -w.
Sameer Pradhan (or possibly Bilal Qureshi) suggested adding stat -tL -c %m%t%T.
Tom Cherry added getprop -Z. Paul Barker added hostname -b and -F.
Rob added ls -b, made ls -q work with utf8,
made sed -f - read from stdin, and added top -O (like ps -O).</p>
<p>Elliott and Rob <b>added Thread support to ps and top</b>,
with -o TID, TNAME. We also added -o PCY (android scheduling policy),
-o BIT (process is 32 or 64 bit), and -o TNAME now shows the parent
command name for threads.</p>
<h2>Documentation</h2>
<p>Rob added the sed invocations to convert tabs/spaces and back to
design.html. Isaac Dunham updated hexedit's help text.
Jakob Flierl pointed out a broken URL in the README.</p>
<p>Rob also redid the naming scheme of sed's pattern manipulation code to
remove the gratuitous references to Roger Zelazny's "Amber" series, since it
was confusing people.</p>
<h2>Bugfixes</h2>
<p>Grep -H and -n should now work properly with -ABC. Andy Chu pointed out an
out of bounds access for zero length lines in rev, fixed a buffer overflow
in diff -r, and fixed operator precedence in expr (although Rob is rewriting
chunks of expr so toysh can use its plumbing for $(( )) ).
Patrick Ohly fixed the too-aggressive suid permission dropping logic.
Josh Gao fixed a segfault when find -iname got no argument, and
made tail -f work right with just one file argument.
Tom Marshall cleaned up tar's long filename support and improved
the tar tests, and reported another find bug (with -iname -o -iname not
tracking copy lifetimes properly) that got fixed.</p>
<p>Elliott Hughes fixed wc -c to not trust zero length files to actually be
zero length (/proc does that), fixed "mount -o rw,remount /system" on
Android, removed trailing spaces on ps -o cmdline, fixed pkill -9 and
the corresponding tests, made "insmod -" work, fixed top -b and tail -NUM,
pointed out that ps shouldn't trim numeric fields for display size limits,
and added some more
explicit "sort" calls to make pipelines so build tempfiles are easier to cache.
Rob <a href=https://github.com/landley/toybox/commit/32b3587af261>fixed an insane sed thing</a> the perl 5.22 build was doing.
Fixed mount -o to properly pass in leftover string data, and
<a href=http://lists.landley.net/pipermail/toybox-landley.net/2016-March/004790.html>documented how to use toybox to mount nfs</a> (warning: kernel patch
to fix some bitrot in the kernel NFS driver's string parsing attached
to that message).</p>
<p>George Burgess IV corrected some variable types in traceroute.
Base64 now wraps == properly. Fixed two bzcat segfaults reported by
John Regehr. Andy Chu found a segfault in "sed -e 'c\'" with no trailing
line, and implemented mv -n and cp -n. The cyanogenmod guys pointed out that
cp -a shouldn't complain if a non-root user can't chown, and we added
the output path to cp -r error messages while we were there (before was just
the filename).</p>
<p>Samuel Holland fixed blkid's handling of vfat labels, and
fixed a segfault when basename was passed an empty string and an empty
suffix. Davis Mosenkovs fixed touch -t seconds parsing.
Rob fixed a bunzip bug reported by John Regehr (the bad CRC
error message was printing a NUL argument).</p>
<p>Not all of the commands build standalone, but more of them do now;
scripts/single.sh can now build a "mv" that isn't actually "cp".
The dependencies are more granualr, so "make top; make ps" no longer
produces a broken ps that ignores -A (because ps.o wasn't getting rebuilt
even though top had the FLAG macros for -A zeroed).</p>
<h2>Build</h2>
<p>Rob added a <b>warning when building commands out of
pending</b>. (The pending directory is full of code that hasn't been
properly vetted. Use at your own risk.)</p>
<p><b>New build targets let you build individual commands by name</b>, ala
"make ls cat ps", and you can run the test suite for each standalone
command with "make test_ls" and such.
"make list" shows all such standalone commands in defconfig, and
"make list_pending" shows unfinished commands from toys/pending
("make list list_pending" shows both together). "make clean" now deletes
these filenames at the top level, and the corresponding unstripped files
live in the directory generated/unstripped.</p>
<p>Nicholas Boichat suggested switching make.sh to use $! for process
enumeration during parallel builds (which is both more efficient and more
portable), and suggested shell builtin replacements for wc/awk/sed so the
build loop has fewer forks now.</p>
<p>Lots of work on the test suite, much of it due to Andy Chu. It now
consistently prints the name of the command being tested at the start of each
test (and the common infrastructure does that, not each individual test), and
"make tests" actually runs all the available tests now.
Seperated pgrep and pkill tests, split lsattr/chattr, added fstype and base64
tests. The "tests/files" directory now collects files for tests to
use, with blkid, bzcat, and utf8 subdirectories: the $FILES variable
gives a path to it, so "$FILES/blkid/ext2.bz2" and so on.
The testsuite now has test files with 3 different types of "not utf8 output"
sequences that require escaping, plus some combining character torture
tests, direction reversals, and so on.</p>
<p>Added dependencies on TOYBOX_FORK to various pending commands that need
nommu conversion (which should fix the allyesconfig build).</p>
<p>Static builds with selinux should work again.</p>
<h2>Library</h2>
<p>New bufgetgrgid() and bufgetpwuid() functions cache previous lookup info
rather than repeatedly traversing /etc/passwd and /etc/group (which is slow).
Added xpipe() to lib to catch pipe creation failure.
The HELP_ macros generated by config2help.c now use a capital prefix
to avoid collicing with help_exit() and such.</p>
<p>The dirtree infrastructure got a cleanup pass in preparation for adding
infinite recursion depth support (needed by rm -r), updated the
<a href=code.html#ib_dirtree>documentation</a> to describe the new
semantics (removing dirtree_start() and adding dirtree_flagread()).
Now dirtree_recurse() takes the new dirfd as an argument.</p>
<p>Split out _xexit() from xexit() and let sigatexit() set multiple
callbacks.</p>
<p>For years the man pages have said to #include <sys/types.h> to get
major/minor/makedev but now that glibc
<a href=https://sourceware.org/ml/libc-alpha/2015-11/msg00253.html>has vowed
to break existing programs</a> and replace it with another nonstandard header
not in posix or lsb, we added our own functions to lib/ to do the transform
ourselves (based on what the kernel actually expects).</p>
<h2>Portability</h2>
<p>Debian unstable started needing an extra header #include for some reason, and
although printf("%.*s", INT_MAX, s) worked fine on Ubuntu 12.04 it
didn't on 14.04, so added a workaround for that. Typecast a printf because
wchar_t isn't a rigidly defined size. RLIMIT_RTTIME was
added to the kernel in 2008 but you can't expect uClibc to have noticed yet,
nor did it #define MS_RELATIME (added in 2006), or prlimit (2010)...
(Given the improvements in musl and bionic, uClibc support may be dropped
in a future release.) Given that the xattr functions were added during
linux 2.5, we can #include its header unconditionally.</p>
<p>Renamed basename_r() to something else to avoid conflicting with freebsd's
libc, and both scripts/install.h and scripts/config2help.c no longer include
toys.h (to make cross-compiling from systems we don't run on easier).</p>
<p>Debian bug 635570 did something unspeakably nonportable, depending on
"sed -e 'a\'" (with no next line of the pattern, so an unterminated
continuation) to add a newline to the last line of input if and only if
that last line of the input didn't have a newline, and to take no other
action. This is well into "depending on a bug" territory, but we implemented
it because otherwise Debian's install broke. (Of course this behavior
is undocumented, non-obvious, and doesn't really make logical sense.)</p>
<p>CONFIG_TOYBOX_NORECURSE now disables the stack measuring logic (which
was giving some "security" code fits). Also we typecast pointers to (long)
before comparing them to avoid spurious compiler "optimizations" that
break the code.</p>
<a name="02-02-2016" /><a href="#02-02-2016"><hr><h2><b>February 2, 2016</b></h2></a>
<blockquote><p>"I checked it very thoroughly," said the computer, "and that
quite definitely is the answer. I think the problem, to be quite honest with
you, is that you've never actually known what the question is."
</p><p>- The Hitchhiker's Guide to the Galaxy</p></blockquote>
<p><a href=downloads/toybox-0.7.0.tar.gz>Toybox 0.7.0</a>
(<a href=https://github.com/landley/toybox/releases/tag/0.7.0>git commit</a>)
is out.</p>
<p>The new commands in defconfig are <b>iotop</b>, <b>top</b>, <b>pgrep</b>,
and <b>pkill</b>
(most replacing corresponding versions from pending). Added grep -ABC,
swapon -d (discard), mkswap -L (label) and UUID support, and find -delete.
Izabera added free -h and unshare -f. Josh Gao implemented tail -f.
Jose Bollo submitted cp --preserve=context,attr. Kylie McClain added
mktemp -u.</p>
<p>In pending there's the start of a vi command, and Sameer Pradhan contributed
a new dhcp6. This cycle saw several rounds of route cleanup and a little dhcp
cleanup, but neither are complete yet. Lipi Lee did some cleanup to netstat.c
and Elliott Hughes removed warnings from traceroute.</p>
<p>Lots of updates to ps: several new -o options, -k (--sort) -O and -M,
improved compatibility with Android's historical behavior, and
extensive internal code cleanup (including the removal of all
the magic constants).</p>
<h3><b>Website</b></h3>
<p>Dreamhost restored the <a href="#12-21-2015">missing 11 months</a>
to the mailing list archive, in the process deleting the month after
that. Now they've asked if I have mbox files archiving the new
gap (between December 20, 2015 to January 21, 2016, and presumably they
could also fill in the gap from December 14, 2014 to January 3, 2015 that's
been there since the last time they did this),
but due to some gmail filtering I've
<a href=http://landley.net/notes-2012.html#15-10-2012>never
been able to disable</a>, my copy of those files is spread among 3 different
mbox files I'd have to sort/filter/collate. (It's on the todo list.)</p>
<p>Added a code of conduct to the README (we're
<a href=https://engineering.twitter.com/opensource/code-of-conduct>borrowing twitter's</a>) because somebody
made it necessary.</p>
<h3><b>Bugfixes</b></h3>
<p>Fixed another sed bug where any ] right after [ was skipped (not just the
first one in the range, so [[] didn't terminate). Fixed sort -f and added test cases.
Assume 80 columns in "ls -m | cat", ls -L is no longer backwards,
and ls of files with no paths no longer uses an uninitialized (zero) dirfd.
Several bugfixes
to find (Gilad Arnold fixed -perm, Daniel K. Levy fixed "find . -exec echo {}",
and while we're there I fixed find --prune, made "find . -execdir
echo {} + -execdir ls {} +" work, and ripped out the environment size
measuring code that checked for a 128k limit removed back in linux 2.6.22).
Elliott Hughes fixed the date command's parsing of 4 digit
years and documented the %s escape, fixed hwclock -u, and pointed out
that runcon needs to exec to do its job (not recursively call another
command_main() in the same process). Tom Marshall reported that blkid was
handling ext2 wrong. Mike Moreton corrected cpio extraction's uid and gid
values, and added a --no-preserve-owner option. Fixed the SUID permission
dropping logic (which was a bit over-zealous, preventing some commands from
running at all).</p>
<p>I'm told that debian-testing broke its libc so the nsenter build breaks,
but my attempts to install the debian-testing network cd image under
qemu keep breaking. Maybe someday they'll fix it enough I can actually
reproduce the problem. (Debootstrap under unbuntu builds an ubuntu-flavored
chroot in which toybox builds fine.)</p>
<h3><b>Documentation</b></h3>
<p>Rewrite of the about.html page, tweaks to design.html, and a re-triage of
sbase in roadmap.html. Update to mkstatus.py to collate multiple span
tags with the same id, resulting in a larger status.html page (which
was previously ignoring some commands in the roadmap).</p>
<p>Expanded the defconfig/allyesconfig/allnoconfighelp text in "make help"
to explain what they're for.</p>
<h3><b>infrastructure</b></h3>
<ul>
<li><p>Expanded toys.optargs to 64 bits so a command can have more than 32 options.</p></li>
<li><p>Added NOEXIT() wrapper to turn xwrap() functions into warning versions
using the existing longjump(toys.rebound) infrastructure.</p></li>
<li><p>Renamed dirtree->data to dirfd and stopped storing symlink length
into it (this fixed a bug where following symlinks to directories
didn't give a valid directory filehandle, noticeable with ls -Z).</p></li>
<li><p>New TAGGED_ARRAY() infrastructure generates index and bitmask macros
for arrays of structures starting with a name string.</p></li>
<li><p>New lib/linestack.c for utf8 fontmetrics (draw_str() and utf8len()
and so on), and for tracking multiple lines of text
(vi, less, shell history) that need wordwrapping and scrolling up/down.</p></li>
<li><p>Upgrades to lib/interestingtimes.c: scan_key() now has a timeout
in milliseconds and recognizes more sequences including ANSI
window size probes. New utf8 test files in tests/files/utf8 including
sequence reversing, stacked combining chars, and all three types of
unprintable sequences (low ascii <32 ala ^X, invalid utf8 sequences ala
<AB><CD>, and invalid unicode code points ala U+1234).</p></li>
<li><p>More comma handling code in lib.c: comma_args()</p></li>
<li><p>Added error_msg_raw() to shut up fortify's endless static checking false
positives.</p></li>
<li><p>readfileat() can now realloc() in a loop to read long files
("zcat | insmod" needed it).</p></li>
</ul>
<h3><b>Roadmap</b></h3>
<p>We're getting close to having a self-hosting development environment
using toybox for the command line. The remaining busybox commands in
<a href=http://landley.net/aboriginal/about.html>Aboriginal Linux</a> are:</p>
<blockquote><p><b>
awk bunzip2 bzcat bzip2 dd diff expr fdisk ftpd ftpget ftpput gunzip gzip
less ping route sh sha512sum tar test tr unxz vi wget xzcat zcat
</b></p></blockquote>
<p>And the remaining non-busybox commands in Aboriginal Linux's build/host
directory (from the distcc, genext2fs, e2fsprogs, zlib, and squashfs packagesi)
are:</p>
<blockquote><p><b>
mke2fs fsck.ext2 resize2fs distcc genext2fs unsquashfs distccd mksquashfs tune2fs
</b></p></blockquote>
<p>Squashfs and distcc are probably out of scope for toybox, but mke2fs,
fsck.ext2, resize2fs, genext2fs, and tune2fs should all be added to the
above "busybox" replacement list.</p>
<p>Remind me to include this countdown in future releases. Once they've all
been replaced, the next goal is <a href=http://landley.net/aboriginal/about.html#selfhost>building AOSP under itself</a>.</p>
<p>See the full <a href=roadmap.html>roadmap</a> and <a href=status.html>status</a>
pages for more details.</p>
<a name="12-21-2015" /><a href="#12-21-2015"><hr><h2><b>December 21, 2015</b></h2></a>
<p>Yes, 11 months have gone missing from the mailing list web archive.</p>
<p>Yesterday evening Dreamhost's mailman server went down (timing out trying
to connect). I poked them about it, they
<a href=https://twitter.com/landley/status/678781271670149121>blamed
DNS</a>, I explained that the hang was _after_ the DNS lookup and
entered the dig info into the trouble ticket showing the IPs the DNS
queries were returning, they reinstalled the server at that IP from what I
assume was their most recent backup, and that's how 11 months of messages
vanished out of the archive.</p>
<p>I've <a href=https://twitter.com/landley/status/679114451975467008>continued to poke them about it</a> but I honestly believe that's the best they
can do. <a href=http://landley.net/dreamhost.txt>Last time</a>
this sort of thing <a href=http://landley.net/dreamhost2.txt>happened</a>
we went back and forth for months, so I added a link to a backup web
archive (in the nav bar on the left) that isn't controlled by dreamhost,
and thus doesn't gratuitously lose data on a regular basis. (I note
the earlier hole in Dreamhost's archive was never fixed either. That
was data never getting archived, this is a year's worth of data that
was in the archive until yesterday vanishing after the fact.)</p>
<p>If you're wondering why the <a href=http://lists.landley.net>top level</a>
list page has been "temporarily disabled" for multiple years now... you'd have
to ask Dreamhost. I know I have. More than a dozen times.</p>
<a name="03-11-2015" /><a href="#03-11-2015"><hr><h2><b>November 3, 2015</b></h2></a>
<blockquote><p>"Alright," said Ford. "How would you react if I said that I'm
not from Guildford after all, but from a small planet somewhere in the vicinity
of Betelgeuse?" Arthur shrugged in a so-so sort of way. "I don't know," he
said, taking a pull of beer. "Why - do you think it's the sort of thing you're
likely to say?"</p><p>- The Hitchhiker's Guide to the Galaxy</p></blockquote>
<p><a href=downloads/toybox-0.6.1.tar.gz>Toybox 0.6.1</a>
(<a href=https://github.com/landley/toybox/releases/tag/0.6.1>git commit</a>)
is out.</p>
<p>We have a new <b>ps</b> command with all the -o fields posix wants (although
it doesn't accept BSD non-dash option syntax yet), and <b>bunzip2</b> (not just
bzcat but the proper extract-in-place command).
Sameer Pradhan added <b>hostid</b> and <b>fsync</b>.
Elliott Hughes added <b>flock</b>.
<p>The people waiting for <b>human readable number support</b> (du -hH, ls -h,
and so on) can thank Elliott Hughes for implementing it. (Our output doesn't
exactly match others' because we our "binary" mode will say 1.0G instead of
1024M, which is a bug in the other one we didn't emulate.)</p>
<p>The other big news is <b>nommu support</b>, tested on the new
<a href=http://nommu.org/jcore>jcore</a> processor but presumaby working
on any nommu system. A few commands don't support nommu yet, but those
are disabled by dependencies on TOYBOX_FORK in menuconfig when building
for nommu. The roadmap now has a large section analyzing the uClinux
project (note that <a href=http://nommu.org>nommu.org</a> is slowly replacing
<a href=http://uclinux.org>uclinux.org</a> as the standard repository of
all knowledge and wisdom about nommu. The old site <a href=#12-02-2012>contains
much that is apocryphal</a>, or at least wildly inaccurate, and the new one
is trying to improve on that).</p>
<p>Both "make change" and scripts/single.sh (for building standalone commands
without the multiplexer logic) now use the top level .config
for toybox global settings such as Linux Security Blanket Module selection,
(so make defconfig before change now).</p>
<p>Documentation updates to the <a href=code.html>code</a> and
<a href=roadmap.html>roadmap</a> pages.</p>
<h3>pending</h3>
<p>In the pending directory Sameer Pradhan added tftp,
and Elliott Hughes sent lsof. Isaac Dunham upgraded mdev,
reboot, init, login, and modprobe, and fixed a distro-specific build break in
scripts/mkflags.h. Elliott Hughes and Lipi Lee made netstat -p handle
command lines longer than 21 characters, and Elliott fixed netstat -e and
some build warnings. Yeongdeok Suh fixed a warning in dhcpd.
I started cleanup on pgrep/pkill.</p>
<h3>Command updates, bugfixes, and infrastructure</h3>
<p>The multiplexer's "command not found" error exit is now 127, so now you can't
distinguish between a command not being found in the multiplexer and
the multiplexer itself not being found by the shell, because people wanted
that for some reason.</p>
<p>Elliott Hughes made date reject invalid dates rather
than set the clock to something weird (setting the clock 100 years into the
future makes most Linux desktops surprisingly unhappy, and ntpdate won't fix it
either), fixed several ls -l display issues (user/group field ordering,
make user/group/lsmcontext left aligned), did the aforementioned
extensive work on human readable number output, fixed ionice's default
class, fixed a mv overwrite bug, made df's columns auto-size, added
--ppid and -Z to ps, and teamed up with Daniel K. Levy to fix
a segfault in find's handling of -newer -group or -user.</p>
<p>Hyejin Kim added stat -c %T support. Colin Cross worked
on vmstat fixing
a header printing bug and calculating the bi and bo columns in the right
units. Isabella Parakiss reported that sed -e "/x/c\" -e "y" added an extra
newline and that grep -w '\(x\)\1' didn't work, both now fixed.
Alistair Strachan fixed several problems with switch_root. Kylie McClain
pointed out env should be able to clear variables via NAME= syntax.
Dima Krasner added support for running blkid without a partition (so it shows
all partitions). Hyejin Kim sent in a bunch of static analysis bug reports.</p>
<p>Isabella Parakiss reported that sed -e "/x/c\" -e "y" added an extra
newline and that grep -w '\(x\)\1' didn't work, both now fixed.
Alistair Strachan fixed several problems with switch_root. Kylie McClain
pointed out env should be able to clear variables via NAME= syntax.
Dima Krasner added support for running blkid without a partition (so it shows
all partitions). Hyejin Kim sent in a bunch of static analysis bug reports.</p>
<p>Two large thinko fixes in oneit: -3 was always enabled (which would
eventually block if the child never read the exiting PID numbers from its file
descriptor #3 until the pipe filled up), and the signal handlers weren't
set up right (for requesting semi-graceful halt/poweroff/reboot).
Calling install without a mode is now 0755, and install -g 0 no longer clashes
with cp --preserve. Better error message for ls -r on unreadable
directories, and ls -Z now uses O_PATH (with the /proc/self/fd/%d
workaround for kernel stupidity as necessary).</p>
<p>Date now understands @unixtime[.fraction] and uses -D for
the set-side format (matching busybox's extension for this). The seq -f
string now checks that it's got exactly one %f escape with the correct
attributes (and a whole bunch of test cases for it). Fixed a bug
in od that screwed up the position indicator on arm and mips.
In stat the d/h units moved from %d %D to the default string.
And patch can now correctly apply hunks with trailing context to the start of
the file.</p>
<p>The prompt argument moved out of yesno() (the caller can print the prompt
themselves). Replaced toys.exithelp with help_exit(). Added new
XVFORK() macro, and xpopen_both() calls /proc/self/exe when passed
a NULL argv (see cpio -p for example usage). Replaced toys.recurse
with toys.stacktop so the recurse or re-exec decision is now based
on bytes of stack space used. Marked a bunch of command-local functions
static.</p>
<p>New additions to lib/ include strlower(), xconnect(), and the
aforementioned help_exit().
The testsuite now has some infrastructure tests based on "example"
commands such as toys/examples/test_human_readable.c.
The login command finally got a long-overdue cleanup (it's one of the
commands that predate the "pending" directory but were part of the reason
for it). Hexedit had an
uninitialized variable (of course gcc didn't spot it, it was too busy
warning about "may be used uninitialized but never actually is" variables).</p>
<p>Tweaked makefile so
"make CROSS_COMPILE=prefix-" (as well as "CROSS_COMPILE=prefix- make",
which still works). Toybox is now installed chmod -w so broken installers
(like the bunzip2 package's) that try to overwrite existing binaries won't
knock out the whole of toybox.
GCC 5.2.0 stopped being able to compile Linux 2.6.12's kconfig, but
we added a workaround. You can now build uptime without utmpx.h.
Alejandro Joya pointed out that enabling smack required smack on the host
as well as target when cross compiling, which is now fixed.</p>
<p>Note: toybox can autodetect nommu support when building with a uClibc
toolchain such as <a href=http://landley.net/aboriginal/downloads/binaries/old/1.4.3/cross-compiler-sh2eb.tar.gz>the one from Aboriginal Linux</a>,
but <a href=http://github.com/richfelker/musl-cross-make>with musl-libc</a>
you'll have to enable CONFIG_TOYBOX_MUSL_NOMMU_IS_BROKEN to work around the
fact they provide a non-functional fork() implementation that always returns
-ENOSYS, to prevent you from compile-time probing for nommu support when
cross-compiling. Unfortunately "preventing you from probing" seems to be
an explicit policy with musl, they also don't provide an "#ifdef __MUSL__"
because their library is perfect and you're only ever allowed to work around
other people's bugs, not theirs. So we have to use menuconfig to manually
enable musl-specific bug workarounds.</p>
<a name="23-07-2015" /><a href="#23-07-2015"><hr><h2><b>July 23, 2015</b></h2></a>
<p>I recreated the <a href=downloads/toybox-0.6.0.tar.gz>0.6.0 source tarball</a>
(new sha1sum 08fb1c23f520c25a15f262a8a95ea5b676a98d54)
because I forgot to add --prefix to the git archive command when I updated
my release script from mercurial, so the files weren't in an enclosing
directory. (Ooops.)</p>
<a name="19-07-2015" /><a href="#19-07-2015"><hr><h2><b>July 19, 2015</b></h2></a>
<blockquote><p>
The reason why it was published in the form of a micro sub meson electronic
component is that if it were printed in normal book form, an interstellar
hitchhiker would require several inconveniently large buildings to carry it
around in.</p><p>- The Hitchhiker's Guide to the Galaxy</p></blockquote>
<p><a href=downloads/toybox-0.6.0.tar.gz>Toybox 0.6.0</a>
(<a href=https://github.com/landley/toybox/releases/tag/0.6.0>git commit</a>)
is out. (Yes, git. See the <a href=#05-04-2015>previous news entry</a>.)</p>
<p>Sorry for the unusually long gap between releases. Since last release Ye
Olde Project Maintainer traveled to japan twice and had two more "once
a century" floods at home. (Probably a coincidence.) Still catching up.</p>
<h3><b>CELF/ELC talk and Wikipedia[citation needed] article</b></h3>
<p>I gave another State Of The Toybox talk
(<a href=https://www.youtube.com/watch?v=04XwAbtPmAg>video</a>
<a href=http://landley.net/talks/celf-2015.txt>outline</a>), in which I
repeat my <a href=http://landley.net/notes-2013.html#07-11-2013>perennial</a>
<a href=https://twitter.com/landley/status/557309224535851009>complaint</a>
that Wikipedia[citation needed]
<a href=http://en.wikipedia.org/wiki/Toybox>still</a>
<a href=https://en.wikipedia.org/wiki/BusyBox#Controversy_over_Toybox>says</a>
toybox was relicensed before its hiatus, when relicensing was why
the hiatus ended.</p>
<p>Since Wikipedia[citation needed] seems unable to do the
<a href=#15-11-2011>most</a>
<a href=http://landley.net/hg/toybox/log/tip/LICENSE>basic</a>
<a href=http://landley.net/notes-2011.html#13-11-2011>research</a> on
this point, and has stuck to an incorrect sequence of events for years,
I've been gradually escalating my attempts to correct them. Toybox
came out of mothballs in November 2011 <b>because</b> it could be
relicensed. That's what opened up a new niche busybox wasn't already
filling with a 10 year headstart.</p>
<a name="asterisk_back" />
<p>The article has plenty of smaller issues<a href=#asterisk>*</a>, but
given that I gave an entire talk at Ohio LinuxFest in 2013
(<a href=http://landley.net/talks/ohio-2013.txt>outline</a>,
<a href=https://archive.org/download/OhioLinuxfest2013/24-Rob_Landley-The_Rise_and_Fall_of_Copyleft.mp3>audio</a>) on why I switched away from GPL for
my projects, that one bugs me.</p>
<h3><b>New stuff this release</b></h3>
<p>There's a new android menu in menuconfig, and rather a lot of Linux
Security Module support (Smack for Tizen from Xavier Roche and José Bollo,
and SELinux for Android from Elliott Hughes; see
the Security Blanket menu under global settings in menuconfig) has
trickled in, although there's still more to come.</p>
<p><b>New commands:</b> Added reset, nproc, ionice, and iorenice.
Elliott Hughes contributed xxd, runcon,
restorecon, load_policy, getenforce, setenforce, getprop, and setprop.
Promoted shred, nsenter, and hwclock.</p>
<p>You can once again build catv now the flag infrastructure's been updated to
let it coexist with cat -v.
And on a long plane flight I wrote
hexedit, an interactive hex editor that implements the start of
cursor control infrastructure (for eventual use by less and vi and shell
command history and so on).</p>
<p><b>New options:</b> Added sed -E as a BSD-compatible synonym for -r.
Upgraded oneit with -r (restart), -3 (send exiting PID values to child),
and signal handling. Added -v option to timeout, -m to mknod, -u to shred,
-t to dmesg, and -123 to head and tail. Added implicit "." to grep -r without
any files to work on. Hyejin Kim requested prefix support for truncate -s.
Greg Hackman added -inum to find.
Jan Cybulski added the smack side of ls -Z support. Various patches also
added -Z to mkdir, mknod, and mkfifo.
Basic cp --preserve support went in, but not yet the xattr/LSM parts.</p>
<p>The toybox command now has a --version option,
which uses "git describe" if available.</p>
<p><b>Build infrastructure:</b>
The "make change" target now saves the output of each failed standalone
command build in a .bad file, and "make defconfig" is quieter now.</p>
<p>Paul Barker submitted a large patch changing command install paths so
"toybox can be installed alongside busybox without confusing
update-alternatives". (There's some argument over
what the right paths should be, and I'm waiting for
people to tell me what else needs fixing because I have no idea. I've
been symlinking /bin to /usr/bin since 2002
<a href=http://landley.net/writing/hackermonthly-issue022-pg33.pdf>for
historical reasons</a>.)</p>
<p><b>Docs:</b> The repository link now goes to github, with another link
to the commit rss feed.</p>
<p>Elliott Hughes updated the Android section of the roadmap
(and he would know). Redid bits of scripts/mkstatus.py to make updating
status.html easier, and the README is larger.</p>
<p>More description of option parsing in code.html, which now describes the
FLAG_x macros, switching flag macro sets with FOR_newcommand, how
configuration zeroes flag macros and using FORCE_FLAGS to suppress the
zeroing of options shared between commands. Also added description of ";"
to make --longopts take an optional =value part, and more about TOYBOX_DEBUG
to check NEWTOY() option strings (otherwise a bad option string makes
lib/args.c obviously segfault, but doesn't explain why).</p>
<p>Added a "Why 0BSD?" section to license.html when submitting zero clause bsd
to SPDX (according to the pending license spreadsheet, it's been approved for
SPDX 2.2).</p>
<p>The old list of commands needing cleanup but not in pending was
removed from toys/pending/README and instead the issues were added
as TODO comments in the individual commands.</p>
<p><b>Bugfixes:</b>
Fixed mount -a segfaulting without -O (reported by Janus Troelsen),
and made it try a "become rw" ioctl() on the block device before falling
back to mounting read only (because Android expects that).
Fixed printf -- and printf ---. Lots of tweaks to ls -l spacing with
different options. Make touch -d and -t actually set time when you don't
specify nanoseconds.
Fixed a subtle bug where recursive calls (toybox commands that run other
toybox commands) weren't resetting all their state. (This manifested as
a "no }" error from "find | xargs sed", but could cause other problems.)
And David Halls reported another sed bug trying to compile libiconv (which
left extra \ at the start of lines in a generated shell script, breaking
the build). Output an error message for "cat /mnt".</p>
<p>Kylie McClain reported that mktemp broke when $TMPDIR was set to an empty
string (which is not the same as unset), that install/find didn't support
numeric uid/gids, and that sort -z affects both input and output.
Isabella Parakiss fixed a printf.c bug.
David Halls fixed bugs in install -D and find -exec. Samuel Holland
fixed unshare -r. Hyejin Kim fixed makedevs with a count of 1, fold -w
range checking, an error path in scripts/mkflags.c, added -i to dhcpd,
and stopped su from prompting the root user for the new user's password.
Jan Cybulski spotted wrong indentation when combining ls -s and -i with -C and
-x. José Bollo fixed stat %G. Sameer Pradhan fixed a bug in mkfifo -Z.</p>
<p>Elliott Hughes asked for a default SIGPIPE handler to disable
the signal handler bionic's dynamic loader installs (yes really). Still not
100% sure what the correct behavior is there. (Posix is
(<a href=http://permalink.gmane.org/gmane.comp.standards.posix.austin.general/10915>actively unhelpful</a>, but at least they're taking
<a href=http://austingroupbugs.net/view.php?id=789#c1976>years to
make up their mind</a>. Elliott also sent patches to fix a typo in
useradd.test, add missing arguments to error_exit() calls and clean up
printf() format strings, fix an off by one error in human_readable(),
fix dmesg -c error reporting, fix a segfault in comma_scan where the option
was the last item in optlist (triggered by mount -o ro,remount), fix
hwclock -w, made ifconfig print lowercase MAC addresses (it was bothering
him), and make terminal_size() read the right environment variable
(LINES, not ROWS). And he suggested the test suite notice high command exit
values (corresponding to segfault or other signals).</p>
<p>People are apparently using toys/pending commands, despite the police tape
and flashing lights, so added louder warnings to toys/pending/README.
Elliott Hughes fixed various problems with tar, dd, more, and top.
Hyejin Kim cleaned up syslogd and dumpleases. Isaac Dunham added hotplug
support to mdev. Yeongdeok Suh added RFC-3315 ipv6 support to dhcpd.</p>
<p>I rewrote ps.c from scratch (in pending), but it's not ready for real use
yet.</p>
<p><b>Portability:</b>
On the portability front Bernhard Rosenkranzer fixed a problem where the
menuconfig code wouldn't compile in C99 mode. (This led to me documenting
the craptacular nature of kconfig in a README, and the plan to replace it
sometime before 1.0.) Some extra flags to shut up overzealous llvm warnings
were added (and have to be probed for because gcc complains about
arguments it doesn't recognize even when they switch stuff _off_ using
a standard syntax). Don't depend on malloc(0) to return non-null in ls.
David Halls fixed some mac/ios portability issues,
implying somebody's built at least part of toybox on a mac.</p>
<p>Added basename_r() to lib/lib.c because the posix semantics for basename()
are stupid but what the gnu guys did to it was appalling.
Turns out bionic already had a basename_r(), but posix still doesn't.
Fixed it up in portability.h, but this
could break more stuff in future. (Correct fix is to lobby posix to add it,
which would probably take about 15 years...)</p>
<p><b>Infrastructure:</b>
The build now checks $LDFLAGS for linker-only flags, and allows the strip
command to fail (binflt toolchains provide a strip that doesn't work).
Since time.c uses floating point, added TOYBOX_FLOAT dependency in config.</p>
<p>There's a lib/lsm.h defining varous inline functions for linux
security modules stuff, if (lsm_enabled()) should turn into a compile-time
constant 0 and let code drop out when TOYBOX_LSM_NONE selected, but
testing against CFG_TOYBOX_LSM_NONE or derived symbols is still useful
becuase when it _is_ enabled the probe turns into a system call you
don't want to repeat too much.</p>
<p>Switched a bunch of commands from signal() to xsignal(). Factored out
xgetgrnamid() and xgetpwnamid() into xwrap.c. Make time.c depend on
TOYBOX_FLOAT (since it always uses float so shouldn't be available on
build targets without even software float). Added readfileat() to lib/lib.c.</p>
<p>The dirtree infrastructure now passes in full flags for the old symlink
field, and the new DIRTREE_SHUTUP flag disables warnings if a file vanishes
out from under you during traverse. New dirtree_start() wrapper to
create dirtree root with only two arguments.</p>
<p>The not-curses infrastructure introduced by hexedit mostly moved to
lib/interestingtimes.c.</p>
<a name="asterisk" />
<a href="#asterisk_back" />Asterisk:</a> such when
Tim contacted me (my blog says a couple days before nov 13, 2011, I.E.
11/11/11 not some specific day 2 months later) to ask if I wanted to work
on a new project he was proposing called
<a href=http://www.elinux.org/Busybox_replacement_project>BentoBox</a>
(because I used to do busybox, he'd forgotten toybox existed
until I brought it up). And don't ask me what "focuses not on compatibility
with its GNU counterparts" means when CP_MORE adds 7 non-posix options
and toys/other has 84 commands in neither posix nor LSB. I think they're
struggling to explain the difference having dismissed "licensing" as being
the reason it started up again after a long hiatus? The reason I don't think
GNU is special is there are a half-dozen other independent
implementations of the same unix command tools out there (AT&T,
BSD, Coherent, Minix, plan 9, busybox, toybox, and several more analyzed in
the <a href=roadmap.html>roadmap</a>, and that's ignoring the implementations
written for DOS or in assembly over the years). But I do care what
Linux From Scratch expects, and if it's
<a href=http://archive.linuxfromscratch.org/lfs-museum/7.6/LFS-BOOK-7.6-NOCHUNKS.html#ch-tools-gcc-pass1>calling mv -v</a>
then I impelement mv -v
even if <a href=http://landley.net/toybox/roadmap.html>posix hasn't got
it</a>. And I don't know why "gnu counterparts" would describe this when
util-linux isn't a gnu package, nor are info-zip, e2fsprogs, kmod, less,
procps, shadow, sysklogd, vim, zlib, sudo, dhcpcd...</p>
<a name="05-04-2015" /><a href="#05-04-2015"><hr><h2><b>April 5, 2015</b></h2></a>
<p>Since <a href=https://android.googlesource.com/platform/external/toybox/>android</a> and
<a href=https://git.tizen.org/cgit/platform/upstream/toybox.git>tizen</a>
and <a href=https://github.com/kraj/meta-musl/tree/master/recipes-core/toybox>openembedded</a>
and <a href=https://packages.gentoo.org/package/sys-apps/toybox>gentoo</a>
and so on have all been using Georgi Chorbadzhiyski's git mirror rather
than the mercurial repository, I bit the bullet and switched the project's repo
<a href=https://github.com/landley/toybox>to git</a>. Georgi's
<a href=https://github.com/gfto/toybox>mirror</a> is now pulling from that.</p>
<a name="25-02-2015" /><a href="#25-02-2015"><hr><h2><b>February 25, 2015</b></h2></a>
<blockquote><p>"A common mistake that people make when trying to design
something completely foolproof is to underestimate the ingenuity of
complete fools."</p><p>- The Hitchhiker's Guide to the Galaxy</p></blockquote>
<p><a href=downloads/toybox-0.5.2.tar.gz>Toybox 0.5.2</a>
(<a href=/hg/toybox/shortlog/1702>commit 1702</a>) is out.</p>
<p>New promoted commands: sed (finally fixed enough it builds Linux From
Scratch), printf (cleaned up and promoted), shred and
base64 (the Tizen guys wanted them), getenforce, setenforce, and chcon (android),
mix (promoted with fixes from Isaac Dunham), nsenter (from
Andy Lutomirski, merged into unshare).</p>
<p>Elliott Hughes submited a bunch of patches to support Android (to
both toybox and Bionic libc, which he maintains). On toybox's end this
involved a lot of fixups to portability.[ch] and fixes to over a dozen
commands, plus several new ones. Other portability fixes included working
with buildroot's uclibc fork and building for nommu targets.</p>
<p>The new "make change" target builds each toybox command as a standalone
binary. Rather a lot of commands that didn't build by themselves (mv depending
on cp and so on) were hit with a large rock until they built standalone.
This involved rewriting bits of option parsing, more elaborate dependency
generation, making each command have its own config
symbol and main() function (even when it's just a wrapper calling another
command's main()), and so on. Also, some commands can't be built standalone
at a conceptual level: "help" describes other enabled commands and "sh"
has a number of bulitin commands (cd, exit, set) that require the
multiplexer infrastructure, so "make change" filters them out.</p>
<p>The mailing list's web archive is still screwed up. Dreamhost has
been trying to fix it since approximately September. There are
<a href=http://www.mail-archive.com/toybox@lists.landley.net/>two</a>
<a href=http://news.gmane.org/gmane.linux.toybox>other</a> less broken
archives, but neither has quite the same UI as mailman.</p>
<h3>Bugfixes and tweaks</h3>
<p>Cynt Rynt sent in tests for ifconfig,
Robert Thompson taught factor to accept whitespace separated arguments,
Hyejin Kim pointed out that some of mktemp's longopts were attached to
the wrong short options,
Luis Felipe Strano Moraes fixed a wrong free() call in bootchartd in pending.
Patches from Ashwini Sharma to make "df /dev/node" work, prevent du from
looping endlessly following symlinks, and to make expr.c
(in pending) understand == and regex matches. (Speaking of expr, it gets
priority groupings wrong but the bug was actually in the posix spec's
HTML conversion. They fixed the posix spec upstream for us. Still need
to fix the expr code, but it's in pending for a reason...)</p>
<p>Some commands grew new option flags, such as cp --remove-destination
and touch -h.</p>
<p>The parallel build has better error reporting now. When toybox needs to
re-exec itself to regain suid root permissions and hasn't got the suid bit,
it now gives the right error message ("not root" instead of "no such command").
<p>Added a test to "mount" to not mount the same device/directory combination
over itself (the OS catches this for block devices, but not for tmpfs).
Make blkid distinguish ext3 from ext4. Added catv back into cat (because
the Android guys wanted it, and they have historical usage on their side,
so...). Handle nanoseconds in touch.</p>
<p>Fixed a segfault when CP_MORE was disabled (the resulting option flag list
no longer defined -d but still had it in option groups at the end).
Workaround for glibc redefining dirname() and basename() to random non-posix
semantics because gnu. (They could have created dirname_r() but didn't want
to.)</p>
<p>Fix an ifconfig test that was preventing assigning an ipv4 address to
interface aliases. Several cleanup passes on hwclock but not quite
promoted out of pending yet.<p>
<p>Fixed a wrong error message in rm (if you had a chmod 000 directory and
did rm -r on it without -f, after the prompt it would complain it was a
directory, which was not the problem).</p>
<p>The gzip compression code now does "store only" output to stdout, for
what that's worth.</p>
<p>Cleanup mountpoint and expand, and remove them from toys/pending/README
(a list of commands that predate the toys/pending directory but needed
another pass).</p>
<h3>Library and infrastructure:</h3>
<p>Reworked the option parsing infrastructure so more commands build
standalone (via scripts/single.sh or "make change"). The option flag bit
values are no longer packed, it leaves spaces where currently disabled
flags go, and you can #define FORCE_FLAGS so disabled flags aren't zeroed.
This allows multiple commands to more easily share infrastructure, even if
your current flag context is for a disabled command (switched off in config),
you can force them to stay on and as long as the flags read the same right
to left they'll have the same values.</p>
<p>We've started removing use of strncpy() because it's a hugely broken
standard C function: the length is the maximum length to _append_, not
the size of the destination buffer. It memsets the remaining space it didn't
copy ala "memset(dest+strlen(dest), 0, len);" so
if you think len is the size of dest you're guaranteed to stomp memory off the
end). And if it runs out of space it won't null terminate because reasons.
(Meanwhile sprintf("%*s", len, str) is counting wide characters in your current
locale, so if you set a locale other than "C" it will also go past your
allocated buffer size. Whoever is maintining the C library standards is really
bad at strings.)
Instead we have xstrncat() which will error_exit() if src+dest+1 doesn't
fit in the buffer. (Because randomly truncating input data isn't necessarily
an improvement.) And there's always xmprintf().</p>
<p>Similarly, strtol() doesn't return an error indicator on overflow,
you have to clear and then check errno. So new xstrtol() that cares
about overflow.</p>
<p>The bionic and musl guys agree faccessat(AT_SYMLINK_NOFOLLOW) is not
supported, so stop using it.</p>
<p>Fixed toy_exec() to detect when argc is in optargs, so we don't
need a separate xexec_optargs().</p>
<a name="18-02-2015" /><a href="#18-02-2015"><hr><h2><b>February 18, 2015</b></h2></a>
<p>Dreamhost continues to be unable to make mailing list archives work, so
here's <a href=http://www.mail-archive.com/toybox@lists.landley.net/>another
list archive</a> with a less awkward interface than gmane.</p>
<p>(Neither gives you the convenient historical monthly views of mailman,
but I still have hopes dreamhost will someday figure out what they're doing
wrong. They've only been trying since October. Last month they did a
<a href=http://www.dreamhoststatus.com/2015/01/14/discussion-list-hardware-maintenance/>hardware upgrade to fix a software problem</a>, and the stale
data loads much faster now, so that's something.)</p>
<p>Update (Feb 19): the archive started updating again, by discarding
all the pending data. So there are now _two_ giant holes in Dreamhost's
web archive, from Dec 15-Jan 3, and then another hole from Jan 16-Feb 18.
The relevant messages are in both of the other archives. Here's hoping
the chronic archive constipation problem won't happen a sixth time.</p>
<a name="30-12-2014" /><a href="#30-12-2014"><hr><h2><b>December 30, 2014</b></h2></a>
<p>Due to Dreamhost's <a href=http://landley.net/dreamhost.txt>ongoing</a>
<a href=http://landley.net/dreamhost2.txt>inability</a> to make mailman
work reliably, I've added a link to a backup web archive at
<a href=http://news.gmane.org/gmane.linux.toybox>gmane</a> to the nav bar
on the left.</p>
<p>You still subscribe to the list through
<a href=http://lists.landley.net/listinfo.cgi/toybox-landley.net>the first link</a>.</p>
<p>Update (January 27, 2015): they're <a href=https://twitter.com/landley/status/558428839462703104>still working on it</a>.</p>
<a name="19-11-2014" /><a href="#19-11-2014"><hr><h2><b>November 19, 2014</b></h2></a>
<blockquote><p>"This time it was right, it would work, and no one would have to get nailed to anything."</p><p>- The Hitchhiker's Guide to the Galaxy</p></blockquote>
<p><a href=downloads/toybox-0.5.1.tar.bz2>Toybox 0.5.1</a>
(<a href=/hg/toybox/shortlog/1566>commit 1566</a>) is out.</p>
<p>It's an interim release, mostly bugfixes. There are several new commands,
but they're all in pending.</p>
<h3>Development</h3>
<p>Finally implemented sed, which is still in pending because although
it's feature complete according to posix, and even passes the parts of
Busybox's sed test suite that aren't explicitly testing for gnu bugs we
don't want to copy, it's not yet good enough to build Linux From Scratch.
(The ./configure stages use very long sed scripts. 20 commits worth of
implementation and debugging, just under 1000 lines of code, and there's
still more to do. We're definitely up to some of the "fiddly" commands now.
Did you know "echo hello | sed p - -" segfaults gnu sed in Ubuntu 12.04?
Yeah...)</p>
<p>Talked with the Tizen developers to follow up on their desire to
make toybox a part of the base Tizen system, and got a list of commands
to add to the roadmap. The tizen todo list is:</p>
<blockquote><p>
wget, sha256*, gzip, gunzip, bunzip2, rsync, zdiff*,
less, ar, arch, base64, csplit, dir, fmt, join,
nproc, shred, shuf, stdbuf, stty, test, tr, unexpand,
users, vdir, diff3, sdiff, dosfsck (fsck.vfat), awk, fdisk
</p></blockquote>
<p>(Most of which was already on the todo list, but it helps prioritize.)</p>
<p>Fixed md5sum and sha1sum on big endian systems (reported by James McMechan).
Andy Lutomirski fixed unshare's help text and option parsing,
and submitted nsenter (a tool to use setns(2)) to pending.
Isaac Dunham implemented acpi -ctV options, and spotted the bug that ls -d
was inappropraitely following command line symlinks without -H or -L (it
should act like ls -l does), and ls -F handles symlinks wrong too.
Lukasz Szpakowski sent in two bugfixes to tail.c. Cynt Rynt spotted an
unnecessary assignment in lib/password.c.</p>
<p>Ashwini Sharma's team was as busy as usual, submitting tr, crontab, and
ipcrm, and hwclock to pending, more features to the pending ip.c, and a
pile of bugfixes (to chgrp, killall, ifconfig, insmod,
losetup, comm, cp, id, xwrap, netcat, modprobe, nohup...) mostly found by
static analysis. (These fixes are mostly to seldom-used codepaths like the
TOYBOX_FREE config option, but test coverage is always appreciated.) Ashwini
also suggested upgrading ln -f to leave the original target alone if link
creation fails, and reported that mv -f and -i weren't implemented (now fixed).</p>
<p>New config option: TOYBOX_NORECURSE prevents xexec() from making internal
function calls (for nommu systems with a finite stack).</p>
<p>The "toybox" multiplexer command no longer adds a trailing space to each
line of command names, so things like "./toybox | tr ' \n' '|'" to create
a grep pattern snippet are easier to do. (Why you'd want to is your business,
but the output is tidier now.)</p>
<h3>Infrastructure</h3>
<p>Isaac Dunham added Android support to portability.h, including compile
probes for functions missing from bionic-libc, and annotated the commands that
use those functions. We haven't really tested building against bionic,
but in theory it's possible now.</p>
<p>Running the test suite now color codes the PASS/SKIP/FAIL notifications
if output is to a tty. (And in case you missed it last time, VERBOSE=fail
to stop at the first failure is really useful.)</p>
<p>In loopfiles_rw() use O_CLOEXEC instead of O_RDONLY to request the loop
function close filehandles for us. (Otherwise the callback function must
close each supplied filehandle itself.)</p>
<p>The printf-style escape parsing ("\n" and friends) got factored out into
a new unescape() function.</p>
<a name="02-10-2014" /><a href="#02-10-2014"><hr><h2><b>October 2, 2014</b></h2></a>
<blockquote><p>"There is an art, it says, or rather, a knack to flying.
The knack lies in learning how to throw yourself at the ground and miss...
Clearly, it is this second part, the missing, which presents the
difficulties."</p><p>- The Hitchhiker's Guide to the Galaxy<p></blockquote>
<p><a href=downloads/toybox-0.5.0.tar.bz2>Toybox 0.5.0</a>
(<a href=/hg/toybox/shortlog/1512>commit 1512</a>) is out.</p>
<h3>New commands</h3>
<p>The new commands are find, install, factor, and mount. Promoted commands
(cleaned up and moved out of "pending") are lspci, inotifyd, and blockdev.</p>
<p>cp now implements -HL and -F to force delete of pending files, cpio now
ignores -m and implements -p, ls -C now has utf8 support (using wcwidth
instead of strlen), and umount got a number of upgrades involving
looking things up in /proc/mounts. Other minor cleanups happend to
cut, touch, free, and id.</p>
<p>In pending: Bradley Controy submitted mix (adjusts OSS sound volume). Ashwini
Sharma submitted diff, userdel, blockdev, ipcs, and crond, upgraded
fdisk, fsck, and ftpget, and ran a static analyzer on a lot of other code.
Partial cleanup was done to useradd, userdel, groupadd, and groupdel.</p>
<h3>Build infrastructure</h3>
<p><b>Parallel builds</b></p>
<p>The build now takes advantage of SMP, autodetecting the number of
processors. (Export the environment variable CPUS to pick a specific number.)
Other build changes: split out $LDOPTIMIZE because old compilers complain
about linker options passed with -c, and the entire "generated" directory now
gets deleted by clean (the README that was in there got merged into code.html).</p>
<p><b>Standalone builds</b></p>
<p>The standalone build infrastructure (scripts/single.sh) got upgraded to
build more commands as standalone executables. In make.sh the source file
selection uses a regex to find the source files with the NEWTOY/OLDTOY macro
for the command. It enables each command's
sub-options (so CP has CP_MORE), enables I18N and FLOAT support to build
full-featured commands, and includes --help text (at least when
the command doesn't use another command's help). The OLDTOY() macro
now produces (redundant) function prototypes so you can build an OLDTOY
without the NEWTOY</p>
<p>It doesn't quite have complete coverage yet, the defconfig entries that
aren't building standalone yet are:</p>
<blockquote><p>chown, egrep, fgrep, fstype, halt, mv, nc, poweroff, unix2dos,
whoami</p></blockquote>
<p>The main reason for standalone build failures is NEWTOY() or OLDTOY()
entries that don't have their own config symbol. Another problem is entries
that depend on another entry in kconfig, usually because common infrastructure
is using one command's flags (which the other commands copy): if that command
is disabled, the FLAG macros become 0 so dead code elimination can remove the
code. It's <a href=http://landley.net/hg/toybox/rev/1503>possible
to untangle</a> this, but a bit awkward. (It boils down to conflicting
design goals in the two contexts.)</p>
<p>Standalone builds are used by the test suite when testing individual
commands.
<p><b>Snapshot builds</b></p>
<p>A new addition to the "generated" directory is generated/build.sh
containing a single compiler command line to build toybox in its current
configuration. Combined with the generated/*.{h,sh} files from an
exisiting build, this may let you build on a new system that hasn't quite
got enough OS bits working to run a full configureand make.</p>
<h3>Internals</h3>
<p>Library code: xcreate/xopen now O_CLOEXEC by default to avoid leaking
filehandles to child processes. DIRTREE_COMEAGAIN's second callback is now
done with the directory filehandle still open (new dir->again variable added
to distinguish first from second callback, and requesting DIRTREE_RECURSE now
requires passing in the specific macro value, not just a true/false).
Use daemon() out of libc instead of hand-rolled daemonize() in various
pending commands. string_to_mode() now passes through type bits so you can
use it to more easily modify a file's existing mode.
Split xpopen() into xpopen_both(), xopen(), and xrun() depending on whether
we want to redirect both, one, or neither of stdin/stdout.</p>
<p>Bugfixes: Better error message when TOYBOX_SUID option can't drop priviliges
(which happens when you suid something _other_ than root).
The old pending version of nbd_client.c wasn't deleted when the
command was promoted (and the build would break if both were enabled),
toy_exec() sometimes needs to re-exec from $PATH rather than recurse
internally (to gain dropped root permissions or limit stack depth),
always call setlocale() when I18N is enabled to switch it back _off_ when
we run commands that expect sscanf("%n") to return bytes,
dirtree() had a memory leak in an error path, patch.c had some bugs in
error paths (didn't report problem clearly). Ashwini Sharma spotted an
option parsing bug where [-abc] would forget _all_ command line arguments
saved in the GLOBALS() block (not just the ones for options being switched
off), plus various minor fixes to nbd_client and cpio.
Lukasz Szpakowski fixed rm -f on a broken symlink (failed), and killall
with no arguments (segfaulted).</p>
<p><b>Portability</b></p>
<p>A somewhat fiddly fix to rm -rf (which needs to chmod directories to u+rwx
to descend into them) which hit a musl bug in faccessat() which the musl
maintainer refuses to fix. (He literally wants the man page changed
instead, despite other libcs working.) Added an #ifdef __MUSL__ section
to portability.h with a workaround, you may need CFLAGS=-D__MUSL__ in your
build if your musl build's features.h doesn't #define that. (I may do
a different workaround in future, but sometimes you've just got to make
it work so you can ship. Also, toybox grep with multiple patterns
requires <a href=http://landley.net/hg/aboriginal/rev/1692>a patch
to musl's regex engine</a>, which applies to 1.1.4 but not to the current
musl source control.)</p>
<p>More portability.h fixes for uClibc too. (I don't expect that to ever have
another release, so locally patching around posix-2008 violations is silly).</p>
<p><b>Change to username filtering</b></p>
<p>Posix recommends the username creation logic filter usernames to a small
allowed set of characters (which even Red Hat breaks by explicitly allowing
"$" at the end), but this prevents UTF-8 usernames. Posix' stated logic
is to allow filesystems to create the user's home directory, but Linux
filesystems can accept any character but NUL and "/". The only characters
we actually _need_ to filter out are ":" (field separator in passwd),
newline (line separator in passwd), and "/" (directory separator in
filesystem).</p>
<h3>Documentation</h3>
<p>Web pages updated: cleanup.html documents more cleanup, code.html
documents more code, and about.html now capitalizes "toybox" consistently
(it's just a word, capitalize at start of sentence).</p>
<p>The pending/README file now lists commands that needed review/cleanup
before the pending directory was added.</p>
<h3>Test Suite</h3>
<p>Moved out of scripts/test into top level "tests" directory, and the
testing.sh script is now in scripts rather than mixed into the *.test files.</p>
<p>Johan Bergström requested VERBOSE=fail to make tests (telling it to
stop at the first failure), and spotted a build bug where using gnu
sort on the host broke in non-C locales.</p>
<p>Divya Kothari submitted tests for chmod, link, tar, bzcat, xzcat, zcat,
and hostname. (And more, but that's all that's merged so far.)</p>
<a name="07-07-2014" /><a href="#07-07-2014"><hr><h2><b>July 7, 2014</b></h2></a>
<blockquote><p>"This planet has - or rather had - a problem, which was this:
most of the people living on it were unhappy for pretty much of the time. Many
solutions were suggested for this problem, but most of these were largely
concerned with the movement of small green pieces of paper, which was odd
because on the whole it wasn't the small green pieces of paper that were
unhappy."</p><p>- The Hitchhiker's Guide to the Galaxy</p></blockquote>
<p><a href=downloads/toybox-0.4.9.tar.bz2>Toybox 0.4.9</a> (<a href=/hg/toybox/shortlog/1385>commit 1385</a>) is out.</p>
<p><b>New commands</b> added to pending include:
lsattr, chattr, inotifyd, rfkill, sulogin, strings, makedevs,
killall5, and tar from Ashwini Sharma, arp from Kyungwan Han,
sysctl by Bilal Qureshi, partprobe from Bertold Van den Bergh,
host from Rich felker, and I did nbd-client and the first 2/3 of mount.</p>
<p>Finished cleanups (commands promoted out of pending):
sysctl, rfkill, strings, mkpasswd, makedevs, partprobe, killall5,
fallocate, and nbd-client.</p>
<p>(Along the way partial cleanups got made to: last, fold, lspci, ps,
bootchartd, init, fsck, telnetd, telnet, vconfig, toysh, iconv, useradd,
login, host, openvt, deallocvt, getty, tftpd, and modprobe. But there's
still more to do on all of those.)</p>
<p>This time around the <a href=bin>static binaries</a> are linked against
musl instead of uClibc. (That's why there's no sparc version, musl doesn't
support that target yet.)</p>
<p><b>Documentation:</b></p>
<p>The help text parser expects lower case "usage:" lines with
a blank line after them, so go through and regularize those. Expand the
"coding style" section in the docs and move it to design.html. (Not a show
stopper for incoming
contributions, just an explanation of some of the things I'll do to them
during cleanup.) The help text for the "toybox" command now includes
the shell script snippet to install symlinks to the toybox binary.</p>
<p>The <a href=cleanup.html>cleanup page</a> now has descriptions for the
full ifconfig cleanup series, among others.</p>
<p>The new toys/examples directory contains hello.c and skeleton.c. The first is
a simple hello world program in toybox style, the second is a much more
elaborate example program using showing how to use the command line option
parsing and how to provide multiple commands in the same C file.</p>
<p><b>Fixes</b>:</p>
<p>Fix od bug reported by Samuel Holland ("od -v -b" was appending the default
output type even though an output type was specified). Ashwini Sharma reported
bugs where readfile() was incorrectly freeing its buffer, and where toy_init()
was zeroing the wrong data because the field it was using to measure (rebound)
had moved (when I moved it back I added a comment why the field needs to be
there), fixed a segfault in the dhcp client, and made a 0 length read at
the start of password entry count as EOF. Make the "we are not root" test
in the init code show the help text. Posix implies that fflush() can return
success even when the stream's error bit is set, so call both fflush() and
ferror() from xprintf().</p>
<p>Isaac Dunham pointed out that bloatcheck couldn't deal with diff
implementations that only implement "unified diff" format, and that some
diff implementations can't handle nonseekable input (I.E. reading from
a pipe). Bugfix so "help -a" works again. Option parsing on nohup now stops
at first nonoption argument. Fix segfault in "which" if PATH wasn't set,
which was actually a bug in lib function find_in_path(). Made rm -rf of
chmod 000 directories actually remove them.</p>
<p>The build now passes the same $CFLAGS to the library probe as the final
build, because arch linux is so broken it provides different sets of
libraries for static and dynamic linking.</p>
<p>It turns out sprintf("%.123s", str) is counting characters, not bytes,
so globally enabling locale support opens stack smashing vulnerabilities.
So there's a new TOYFLAGS_LOCALE you set in toyflags when you want the
setup code to setlocale().</p>
<p><b>Upgrades:</b></p>
<p>Isaac Dunham extended cpio to archive unreadable empty files, and I taught it
to set uid/gid and timestamp when extracting archives. Isaac also
added tests for cpio, link, and du, added lspci -i, made the pci database
parsing skip # comment lines, merged logname and whoami into id.</p>
<p>Daniel Verkamp sped up md5sum about 30% with some loop unrolling, making
it actually smaller in the process. I added -b flags to md5sum and sha1sum
for "brief" output that's just the hash with no filename. (I'm aware other
implementations use that for MSDOS "binary" mode, and don't care.)</p>
<p>When building standalone commands (scripts/singleconfig.sh commandname),
the build now switches on all the sub-options of the command so we get
a standalone version with all the bells and whistles enabled.</p>
<p>Add -ds flags to date and document +FORMAT escapes. Add the shell NOP
command ":" as an alias for true (for toysh).</p>
<p>Add uClibc probe for iconv() and fallocate. (The fact it didn't always
build against uClibc is why fallocate wasn't enabled in defconfig before.)</p>
<p>The umount command now does an losetup -d on the device by default, so
we don't leak loopback devices. Bugfix to losetup so "losetup /dev/loop0
filename" actually works again.</p>
<p>Divya Kothari sent in test suite entries for ls, ln, rm, mv, printf, dd,
and renice. Then a second round for lsattr/chattr, mount, chmod, pgrep/pkill,
groupadd, groupdel, and useradd. Several of these uncovered bugs, still
working to fix them.</p>
<p>There are now free() functions for the predefined llist types and a
dlist_terminate() function to break doubly linked lists. The new
generic_signal() handler either sets "toys.signal" or writes a byte
to toys.signalfd with the signal number if signalfd isn't -1 (which it's
initialized to in toy_init).</p>
<p>The option parsing logic can now detect when a double fits in a long and
use the more precise type for floating point arguments (the FLOAT macro
contains the type used). The human_readable() function now just outputs
decimal kilo/mega/gigabytes (so when du -u says 5.0G it means 5.0 billion
bytes). The build infrastructure now notices duplicate commands (so if you
cp toys/pending/command.c toys/other/command.c and forget to delete the
first one, the build break is now more informative).</p>
<a name="20-04-2014" /><a href="#20-04-2014"><hr><h2><b>April 20, 2014</b></h2></a>
<blockquote><p>And to this end they built themselves a stupendous supercomputer
which was so amazingly intelligent that even before the data banks
had been connected up it had started from "I think therefore I am" and got as
far as the existence of rice pudding and income tax before anyone managed to
turn it off.</p><p>- The Hitchhiker's Guide to the Galaxy</p></blockquote>
<p><a href=downloads/toybox-0.4.8.tar.bz2>Toybox 0.4.8</a> is based on
<a href=http://landley.net/hg/toybox/shortlog/1262>commit 1262</a>. And
about time too.</p>
<p>The big news is that the build no longer needs python to generate help.h,
that's now done in C. The help text generation is also collating help text
from multiple options, merging command line option blocks and usage: lines.
There's even a new <a href=help.html>help web page</a>.</p>
<p><b>New commands:</b> Ifconfig, cpio, and su were cleaned up the rest of the
way and promoted out of pending. That saga is mostly explained on the
<a href=cleanup.html>cleanup page</a>. Vivek Bhagat's freeramdisk,
Isaac Dunham's fsfreeze, and Felix Janda's iconv are also new.</p>
<p><b>In pending:</b>
Ashwini Sharma's team submitted tcpsvd, udpsvd, telnet, telnetd, last, more,
groupdel/delgroup, arping, brctl, ftpget, ftpput, printf, reset, and added
ipv6 support to traceroute. Kyungwan Han's team submitted modprobe and getty.
Vivek Bhagat submitted openvt and deacllocvt. Samuel Holland submitted fold.
I wrote a new inflate (zip/zlib/gzip decompression) implementation in
compress.c, and still need to do a corresponding deflate (compression-side)
and plug them into gzip and zip and so on. (Right now it does zcat.)</p>
<p>Several commands (vmstat, login, du, vconfig, mountpoint, free, chroot,
cut, touch, modinfo, expand) predate the "pending" directory, and are thus
in other directories but still need cleanup. Of these, vmstat got some
work this time (which would be much easier other vmstat implementations
documented what their output actually meant).</p>
<p><b>Upgrades:</b> Ifconfig grew /prefix netmask support (ala 1.2.3.4/24). Grep now has -zZ to
handle null terminated data, cksum grew -H for hex output. Upgraded od so the
fields align better when producing multiple output types. Help has -a and -h
options (all commands, html output).
Bugfix to blkid building for a 32 bit target. The date command can actually
set dates now. The O_NOFOLLOW compile time probe didn't work with cross
compiling, so it's back to an #ifdef test in portability.h. Nathan McSween
sent in a bugfix to od and a portability fix in the common library code.
Ashwini Sharma spotted a bug in pidof -o, and added verbose (-v) options
to mkdir and ln, and suggested killall should have an -s option and
allow -l to take zero arguments. Ashwini Sharma and Felix Janda upgraded
tftpd. Fixed dumpleases still using toynet.h after
that was removed. Corrected killall return code and error reporting.
Isacc Dunham fixed bugs all over the tree, did cleanup on a bunch of
pending commands (getty, ftpget, init, openvt, modprobe...), and clarified
find's help text. Tom Sparrow ran three different static analyzers on
the code, which resulted in a few cleanups. The peek()/poke() functions
now use "volatile" to prevent broken compiler "optimizations" to do with
aliasing.</p>
<p><b>Build stuff:</b> Each FOR_xxxx macro now has a complementary CLEANUP_xxxx macro, so you
can put multiple commands with different command line options in the same
.c file, so they can share infrastructure outside of lib. (This let the
bunzip logic move out of lib into bzcat.c.) See XXX for example.
i
<p>The headers #included in toys.h are now grouped by standard, and headers
not listed in Posix or LSB were moved to portability.h. The old xregcomp.h
was folded into lib.h because it's posix (and supporting oddball uClibc
configurations isn't as important as it once was).</p>
<p>Regression tested against Ubuntu 8.04 to fix up bit-rot in defconfig
build on older systems. (We depend on Posix-2008, but not necessarily
the absolute latest build environment.)</p>
<p><b>In lib</b>: lib/xwrap.c added xgetpwnam(), xchroot(), and lib/lib.c now has names_to_pid().
xsetuid() was replaced with xsetuser() which takes a struct passwd
and sets both gid and uid, mkpathat() got factored out into a library command,
get_int_value() became atolx_range(), and
xmsprintf() is now just xmprintf(). The bunzip2 logic moved from lib into
bzcat.c.</p>
<p><b>Documentation</b>: new <a href=help.html>help page</a> with the
help text for all the defconfig commands, using the new help -ah output.
The <a href=code.html>source code walkthrough</a> now says more about
#including header files, and how the generated/* directory works. The
<a href=design.html>design page</a> has some new paragraphs about trading
of different kinds of simplicity, and why comments aren't a substitute for
good code. The README no longer trails off into obvious unfinished confusion
at the end. Each page on the website should now have its own title.</p>
<a name="18-11-2013" /><a href="#18-11-2013"><hr><h2><b>November 18, 2013</b></h2></a>
<blockquote><p>"Space," it says, "is big. Really big. You just won't believe how vastly, hugely, mindbogglingly big it is. I mean, you may think it's a long way down the street to the chemist's, but that's just peanuts to space."</p><p>-
The Hitchhiker's Guide to the Galaxy</p></blockquote>
<p><a href=downloads/toybox-0.4.7.tar.bz2>Toybox 0.4.7</a> is based on
<a href=http://landley.net/hg/toybox/shortlog/1122>commit 1122</a>.</p>
<p>New commands: Brad Conroy submitted blkid. Elie De Brauwer submitted
reboot, halt, and poweroff. Strake's nl got cleaned up and promoted from
pending to posix. In addition, the existing chvt and vconfig got some
cleanup.</p>
<p>That said, I haven't nearly kept up with the flood of new commands going
into pending: Ashwini Sharma's team submitted
dd, dumpleases, traceroute, top, useradd, groupadd, mkpasswd, tftpd, and
an fsck wrapper (with no filesystem drivers yet). Isaac Dunham sent in cpio.</p>
<p>Bugfixes: Jeroen van Rijn added a user count to uptime. Elie De Brauwer
added -e to watch, removed a memory leak, and fixed a terminal size problem.
William Haddon made xargs call its command line once even with blank input
(the standard is vague, but builds expect it), and fixed an off by one bug
where grep didn't malloc enough space with -E (leading to a segfault).
I fixed a glitch in bunzip2 (same one as went into busybox since they're using
the code I wrote), in od to fix -t co, -J, and -c options. Add uname -o as a
synonym for -s. Build fix to never use $CC without prefixing it with
$CROSS_COMPILE (since $HOSTCC could be different). Anca Emanuel spotted
a typo in the web page.</p>
<p>The compile-time command line option parsing got rewritten (ported from
bash to C), which should speed up builds a bit and allow code controlled by
--longopts to drop out properly when disabled in the configuration. Terminal
querying got refactored. Patch's -x option is now more informative (a
debug thing if you're trying to figure out why a patch didn't apply).
The "toynet.h" file got folded into toys.h since musl supports it and
micromanging uClibc options isn't very interesting anymore. The test suite
now uses scripts/single.sh when testing a single command.</p>
<a name="17-09-2013" /><a href="#17-09-2013"><hr><h2><b>September 17, 2013</b></h2></a>
<blockquote><p>"Think of a number," said the computer, "any number."
Arthur told the computer the telephone number of King's Cross railway
station passenger inquiries, on the grounds that it must have some function,
and this might turn out to be it.</p><p>- The Hitchhiker's Guide to the Galaxy</p>
</blockquote>
<p><a href=downloads/toybox-0.4.6.tar.bz2>Toybox 0.4.6</a> is based on
<a href=http://landley.net/hg/toybox/shortlog/1068>commit 1068</a>.</p>
<p>This release adds
several new commands: Felix Janda wrote paste and fallocate, Kyungwan Han
submitted eject, Strake contributed grep, Ashwini Sharma added pmap (and
a testsuite entry for grep), Lukasz Skalski sent pwdx, Isaac Dunham posted
acpi, and I did timeout and umount.</p>
<p>The ls command now has a --color=auto option (suggested by Rich Felker).
The multiplexer now has a --help option so you can say "./toybox --help blah"
instead of using the built-in "help" command. (Which is a shell built-in.
Try it on your command line, it's like man for shell builtins. But a certain
other project has conditioned people to expect --help, so...) I forget who
heehooman at gmail is but they pointed out unshare needed PID and UID
namespace support.</p>
<h3>Pending</h3>
<p>A lot of new commands in toys/pending, to the point the next release should
probably just focus on cleanup and review of this backlog. We've got klogd,
dhcp, dhcpd, watch, route, and ps from
Ashwini Sharma (and an fsck wrapper but no fsck.fstype engines yet),
syslogd, pgrep, and pkill from Madhur Verma, netstat by Ranjan Kumar,
test by Felix Janda, lspci by Isaac Dunham, nl, su, and renice by strake (I.E.
M. Farkas-Dyck), and sysvinit by Kyungwan Han.</p>
<p>Some cleanup work on existing pending commands that aren't
ready to promote yet: I did a few more rounds on ifconfig
and Isaac Dunham's did several cleanups to xzcat, Felix Janda cleaned up
logger and syslogd...</p>
<p>Also some cleanup work on commands that predate the pending directory,
but weren't quite polished when they went in, most prominently du,
expand, and touch.</p>
<h3>Infrastructure</h3>
<p>The new scripts/single.sh builds a standalone command without the
multiplexer, although not all commands can be built that way yet (NEWTOY yes,
OLDTOY no) and the space savings aren't anything to write home about. (If a
command needs the option parsing logic at all, it needs all of it.) If
you're curious, you can do:</p>
<blockquote><pre>
make defconfig
make
mkdir singles
for i in $(./toybox)
do
echo $i
PREFIX=singles/ scripts/single.sh $i || break
done
</pre>
<p>(And then wait a long time and watch almost half the builds fail.)</p>
</blockquote>
<p>There is now libbuf analogous to toybuf, another global 4k buffer this
time for use by lib/ code instead of command code.</p>
<p>The lib directory got split up a bit, lib/pending.c contains functions
not yet used by anything outside of toys/pending/*, and lib/xwrap.c contains
functions that wrap other functions and handle failures (via error_exit).
This leaves lib/lib.c containing actual new functions.</p>
<p>General improvements and bug fixes to argument parsing. The [-abc] exclude
logic should now clear arguments slots when disabling options. Bare --longopts
should work now and be able to report errors using their name, the new ;
option allows optional arguments to longopts only suppliable with = (I.E.
--color and --color=auto but not --color auto).</p>
<p>I'm gradually weaning the code off of itoa()/utoa() because sprintf
does this already. In this case "simple" probably means "let libc do it
for us".</p>
<p>Rewrote for_each_pid_with_name_in() and renamed it to just names_to_pid().
It shouldn't get confused trying to compare absolute and relative paths quite
so much anymore.</p>
<p>lib/llist.c grew a new dlist_pop() function for removing a doubly
linked list entry while maintaining a circular list; tail and patch are
using it now.</p>
<p>The musl guys suggested a new optimization flag
(-fno-asynchronous-unwind-tables)
that shaves about 10% off the binary size by removing a C++ism that crept
into gcc's idea of C. While I don't normally try to micromanage the compiler,
"-fstop-being-stupid" is a thing you have to hit gcc with from time to time.</p>
<h3>Bugfixes</h3>
<p>Felix Janda and I did a largeish rewrite of tail to
finally make it work right (we think). Still need to implement tail -f
someday (the tricky bit is making -f follow multiple files at once).
Felix also reported a bug in xpidfile.</p>
<p>Juhani Haverinen pointed out that
python 3 doesn't work with config2help.py, so the detection logic looks
for python2 (until I get around to rewriting that in C). Elie De Brauwer
then fixed our first attempt at this, and also fixed uname's help string.</p>
<p>Ashwini Sharma
pointed out the build was making a FLAG_ macro for " " which broke some
configurations. (That's a control character, not a command line option.)</p>
<p>Jacek Bukarewicz pointed out a bug in chdir permission handling, and
a way to make env segfault. Both should be fixed now.</p>
<p>The new function xexec_optargs()
replaces calls to xexec(toys.optargs) to avoid freeing and reusing optargs
during option parsing screwing stuff up (such as netcat's exec mode).</p>
<p>The stat command's %a output was padded with leading zeroes, which
didn't match anybody else's behavior and thus made the test suite hiccup
between TEST_HOST and testing toybox. (If you go "TEST_HOST=1 scripts/test.sh
command" it sanity checks the tests against the host implementation.)</p>
<p>Last release, "mkdir sub/sub && chmod 007 sub/sub && rm -rf sub" didn't
delete sub and didn't exit with an error either. Neither was correct, rm
should now be fixed.</p>
<p>
<a name="26-07-2013" /><a href="#26-07-2013"><hr><h2><b>July 26, 2013</b></h2></a>
<p>Georgi Chorbadzhiyski maintains a <a href=https://github.com/gfto/toybox>git
mirror</a> of the repository on github, automatically updated from the
mercurial every 6 hours. The mirror is read only, but you can generate patches
against it and post them to the list.</p>
<a name="02-07-2013" /><a href="#02-07-2013"><hr><h2><b>July 2, 2013</b></h2></a>
<blockquote><p>"Time is an illusion. Lunchtime doubly so." "Very deep. You
should send that in to the Reader's Digest. They've got a page for people
like you."</p><p>- The Hitchhiker's Guide to the Galaxy</p></blockquote>
<p><a href=downloads/toybox-0.4.5.tar.bz2>Toybox 0.4.5</a> is based on
<a href=http://landley.net/hg/toybox/shortlog/941>commit 941</a>. It adds
uuencode and uudecode from Erich Plondke, and enables Luis Morales' "who" by
default. Felix Janda and I cleaned up last year's "stat" submission and
enabled it. Ivo van Poorten added "groups".
Andre Renaud added "lsusb". I implemented "split", "pivot_root", and "mv".
</p>
<p>The "help" command is implemented differently now (lib/help.c) and
each command can now understand --help (including both "toybox --help"
and "toybox --help command" in the multiplexer).</p>
<p>The "pending" directory has several commands (find, xzcat, nbd-client,
logger, expr) which work but are not enabled by default pending further cleanup.
Ifconfig is enabled, but still in pending because it's only 2/3 cleaned up.
(It's an awkward halfway state but I'm not holding up the release for it.)</p>
<p>I'm <a href=cleanup.html>documenting the cleanups</a> to teach
more people to do it, but the writeups aren't caught up yet. The
<a href=roadmap.html>roadmap</a> also got updated a bit with further analysis
of other projects, and the README and about pages got updated.</p>
<p>Fixed _another_ "ls -C" segfault when terminal size can't be detected,
condensed the ls help text to fit on one page, implented --color, and taught
-l to print the major, minor numbers when showing block/char devices.
Argument parsing now handles "--" properly (to end option checking),
and the infrastructure can now handle bare --longopts that have no
corresponding short option (both were implemented before but didn't work).
Fixed an old bug in "patch", chmod grew -f, who grew -a. Isaac Dunham
fixed "-" vs "_" handling in modinfo, added a "firmware" output
field, added -b and -k support, and taught it that the ".ko" extension means
to look for the file at the specified path instead of under /lib. Felix Janda
moved file permission display code to lib so ls and
stat could share it. Ashwini Sharma spotted a bug in xabspath when the
last path component exists but we haven't got permissions to open it
(ala readlink -f /dev/sda as a normal user).
</p>
<p>In the build infrastructure, scripts/findglobals.sh finds leaked global
variables. (Leaked means they aren't part of the global union: Other than glibc
debris, toybox should define "this", "toy_list", "toybuf", and "toys", and
that's it; the rest add memory footprint to every command for the benefit of
just one command; use GLOBALS() to stick 'em in the union.) Static linking
against libraries other than the host's libc now applies to feature probes
for unshare and such. Neuter stupid internationalization support that makes
various host "sort" commands put things in an order other than alphabetical
(breaking the multiplexer's binary search on command names).
<p>You should now be able to build from a source control snapshot on a build
system that hasn't got python: if you disable CONFIG_TOYBOX_HELP. (The
release tarballs ship generated/help.h, but it's not in source control.
Eventually I should rewrite that python script in C.)</p>
</p>
<p><b>LICENSE TWEAK</b>: After <a href=http://lists.landley.net/pipermail/toybox-landley.net/2013-March/000794.html>discussion</a> on the mailing list the "2 clause
BSD" <a href=license.html>license</a> got slightly simplified so the first
paragraph now says:</p>
<blockquote><p>Permission to use, copy, modify, and/or distribute this
software for any purpose with or without fee is hereby granted.</p></blockquote>
<p>It used to continue "provided that the above copyright notice and this
permission notice appear in all copies", but A) what's the point? B) does "all
copies" mean binaries, or just source code, or what? C) lots of projects
that consider BSD and GPL compatible have <a href=https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/tree/crypto/aes_generic.c>files with
both license notices</a> on them (sometimes at <a href=http://git.busybox.net/busybox/tree/shell/ash.c>opposite ends of the file</a> to make the conflict
less obvious) because "all copies must include this function" would violate
the GPL but "all copies must include this magic text blob" somehow don't?</p>
<p>I don't want to have to care about this anymore. The tweaked version is more
or less public domain with a liability disclaimer, but we're still calling it
BSD (sometimes "0 clause BSD") to avoid explaining.</p>
<a name="21-03-2013" /><a href="#21-03-2013"><hr><h2><b>March 21, 2013</b></h2></a>
<p>Video of my ELC talk
"<a href=http://youtu.be/SGmtP5Lg_t0>Why is Toybox?</a>"
is up on youtube. Related materials include the
<a href=http://landley.net/talks/celf-2013.txt>talk outline</a> and an
<a href=/aboriginal/about.html#selfhost>android self-hosting writeup</a>.</p>
<p>[Updated June 4] The following links jump to specific topics in the video. (Sorry about
the ads, it's The Linux Foundation.)</p>
<ul>
<li>0m29s <a href=http://www.youtube.com/watch?v=SGmtP5Lg_t0#t=0m29s>The smartphone is replacing the PC</a></li>
<ul>
<li>4m22s <a href=http://www.youtube.com/watch?v=SGmtP5Lg_t0#t=4m22s>Software needed to become self-hosting</a></li>
<li>6m20s <a href=http://www.youtube.com/watch?v=SGmtP5Lg_t0#t=6m20s>Do we care if android or iphone wins?</a></li>
</ul>
<li>9m45s <a href=http://www.youtube.com/watch?v=SGmtP5Lg_t0#t=9m45s>Android not vanilla: oppose or accept?</a></li>
<ul>
<li>11m30s <a href=http://www.youtube.com/watch?v=SGmtP5Lg_t0#t=11m30s>Open source can't do User Interfaces</a></li>
</ul>
<li>15m09s <a href=http://www.youtube.com/watch?v=SGmtP5Lg_t0#t=15m09s>Android is not copyleft: oppose or accept?</a></li>
<li>18m23s <a href=http://www.youtube.com/watch?v=SGmtP5Lg_t0#t=18m23s>Security issues</a></li>
<li>21m15s <a href=http://www.youtube.com/watch?v=SGmtP5Lg_t0#t=21m15s>Solutions to the software problems</a></li>
<ul>
<li>22m55s <a href=http://www.youtube.com/watch?v=SGmtP5Lg_t0#t=22m55s>What toybox needs to be/do</a></li>
<li>28m17s <a href=http://www.youtube.com/watch?v=SGmtP5Lg_t0#t=28m17s>What is toybox?</a></li>
<ul>
<li>28m58s <a href=http://www.youtube.com/watch?v=SGmtP5Lg_t0#t=28m58s>Why toybox started...</a></li>
<li>37m50s <a href=http://www.youtube.com/watch?v=SGmtP5Lg_t0#t=37m50s>What does toybox actually implement?</a></li>
</ul>
</ul>
</ul>
</span>
<a name="14-03-2013" /><a href="#14-03-2013"><hr><h2><b>March 14, 2013</b></h2></a>
<blockquote><p>"Ford, you're turning into a penguin. Stop it." -
</p><p>The Hitchhiker's Guide to the Galaxy</p></blockquote>
<p><a href=downloads/toybox-0.4.4.tar.bz2>Toybox 0.4.4</a> is based on
<a href=http://landley.net/hg/toybox/shortlog/813>commit 813</a>, adding
the "time" and "readahead" commands, plus some bugfixes.</p>
<p>The "cp" command now implements the -s symlink option, plus bugfixes
getting various corner cases right as used in actual package builds.
"id -Gn root" should now print root's groups
instead of the current user's. Several build fixes so toybox builds under
Ubuntu 8.04 again (which is about as old a build environment as you
can expect to find posix-2008 features in).</p>
<p>Unfinished commands have generally been moved to "toys/pending".
Everything else should "default y" to participate in make defconfig.
Several of those pending commands got some basic cleanup so allyesconfig
should at least compile (although defconfig is still what's useful).</p>
<p>Significant roadmap updates, checking several other multicall binaries
(klibc, sash, sbase, s6...) to see what commands they include.</p>
<a name="18-01-2013" /><a href="#18-01-2013"><hr><h2><b>January 18, 2013</b></h2></a>
<blockquote><p>This must be Thursday. I never could get the hang of Thursdays.</p><p>- The Hitchhiker's Guide to the Galaxy</p></blockquote>
<p><a href=downloads/toybox-0.4.3.tar.bz2>Toybox 0.4.3</a> is based on
<a href=http://landley.net/hg/toybox/shortlog/793>commit 793</a>. There
are now exactly 100 commands in defconfig (of a little over 220 on the
<a href=roadmap.html>todo list</a>).</p>
<p>Elie De Brauwer added the rev command, cleaned up tac, implemented the -s
and -f flags for seq, added -v and -i to killall (and fixed killall not to
kill itself before finishing its pid list), and added to the test suite.
Felix Janda added -m to mkdir, pwd -L and -P, and more test suite entries.</p>
<p>Rob Landley added the losetup command, and fixed the existing ls, cp, and
readlink commands. The segfault in ls
happened when it couldn't determine the screen size (last release changed the
default to -C and a screen size of 0 made column view unhappy), and cp got an
extensive rewrite bringing it up to date with the dirtree changes and fixing
a number of things it never did right in the first place. The xabspath()
code in the library now handles a symlink after ".." properly (and the test
suite checks for it).</p>
<p>Infrastructure-wise the code is better about automatically setting the
error return code properly. Now error_msg() sets the exit code to 1 if it's
still defaulting to 0, and the global exit path does a fflush(NULL) with error
bit check rather than trying to be quite so granular about flushing. (That
means if we use printf() instead of xprintf() it still exits with the right
error code, it just doesn't end the program early on an output error.)
Minor bugfix so TOYBOX_DEBUG
doesn't always warn about the lack of suid bit when toybox is built with
at least one STAYROOT command. Bugfix for the option [grouping] logic
(and then further fixes to the error reporting pointed out by Ashwini Sharma).
dirtree_handle_callback() now has a prefix like the rest of the dirtree
functions. A lot of stuff doing manual path handling was switched to using
libc basename() (including, embarassingly, the basename command), which means
it now correctly detects "/trailing/slash/" which the previous code didn't.</p>
<p>Also, last release included some accidentally checked in debug code that
disabled compiler optimization, so the binary size bloated a bit. It's back
to -Os by default now.</p>
<a name="15-12-2012" /><a href="#15-12-2012"><hr><h2><b>December 15, 2012</b></h2></a>
<blockquote><p>"The major difference between a thing that might go wrong and a
thing that cannot possibly go wrong is that when a thing that cannot possibly
go wrong goes wrong it usually turns out to be impossible to get at or repair."
</p><p>- The Hitchhiker's Guide to the Galaxy</p></blockquote>
<p><a href=downloads/toybox-0.4.2.tar.bz2>Toybox 0.4.2</a> is based on
<a href=http://landley.net/hg/toybox/shortlog/749>commit 749</a> and is
just a resync. Linux 3.7 came out, meaning it's time to do an Aboriginal
Linux release, and that should use a stable version of toybox. So here's
a new stable version.</p>
<p>The new commands are cut (from Jason Kyungwan Han), touch
(from Choubey Ji), expand (from Jonathan Clairembault, and he fixed a
bug in login), and rm (from Rob Landley). Felix Janda added UTF-8
support infrastructure (for non-ascii character sets) with a config option.
Elie De Brauwer added tests for cat and sha1sum, and -so options to pidof.
The "ls" command defaults to -C (column view) now, and "readlink" now supports
-fenq.</p>
<p>Portability work: toybox should now build against the musl C library,
and against older glibc versions (circa 2008, much before that and kernel
features we depend on start to drop out).</p>
<p>The whole codebase got reindented from "one tab" to "two spaces" per
level. The option parsing logic now understands [groups] of commands (when more
than one in a group is selected it can switch the others off, or error out,
or other things). The error_exit() infrastructure can now longjmp back to an
earlier point instead of exiting. Each toys/* directory now has a README,
the first line of which is the fancy name menuconfig uses for the directory
(so no more hardwired directory list in scripts/genconfig.sh).</p>
<p>Fixed a filehandle leak in getmountlist().
Pass parent pointer to dirtree_add_node() so it can give error messages with
full path. The yesno() function now always reads from stdin and writes to
stderr (we can retry tty checking complexity once we've got commands needing
it).</p>
<p>The open group broke their website so the
<a href=http://opengroup.org/onlinepubs/9699919799>old links</a> to POSIX 2008
now <a href=http://pubs.opengroup.org/onlinepubs/9699919799>need to start with
pubs</a>. Some of the links in the tree have been updated, others haven't while
I wait to see if their webmaster notices and fixes it.</p>
<p>(I note that the current rm implementation is not technically posix compliant
because the standard requires infinite recursion depth and the current
implementation uses one filehandle per level. I can add a config option
to do it Posix's way, which is more brittle and needs extra security checks,
but am waiting for somebody to complain first. The default "ulimit -n" is 1024
filehandles, so drilling down over 1000 nested subdirectories).</p>
<a name="13-11-2012" /><a href="#13-11-2012"><hr><h2><b>November 13, 2012</b></h2></a>
<blockquote><p>"Rule Six: The winning team shall be the first team that wins."
</p><p>- The Hitchhiker's Guide to the Galaxy</p></blockquote>
<p><a href=downloads/toybox-0.4.1.tar.bz2>Toybox 0.4.1</a> is based on
<a href=http://landley.net/hg/toybox/shortlog/691>commit 691</a>.</p>
<p>Elie De Brauwer contributed usleep, Ashwini Kumar contributed du, and
Kyungwan Han contributed vconfig. Other new commands include switch_root and
md5sum, and the remaining shell wrappers are now proper commands (dos2unix,
unix2dos).</p>
<p>The patch command now supports -l, and gethostname is now enabled by
default. The df command follows symlinks to get the actual device name.
Felix Janda added -m support to wc (for utf8).</p>
<p>On the infrastructure side, the commands have now been grouped into
"posix", "lsb", and "other" subdirectories (for things required by Posix-2008,
the Linux Standard Base 4.1, and commands in neither). This affects menuconfig
and the actual source layout (toys/cp.c is now toys/posix/cp.c, and so on).
An android directory is planned (see the updated
<a href=roadmap.html#android>android roadmap analysis</a>).</p>
<p>The FLAG_ macros for command option parsing and TT alias for the command's
global block are now automatically generated, commands should
#define FOR_commandname before #including <toys.h> to get the macros for that
command.</p>
<p>An upgrade to the build infrastructure now allows commands with _ and -
in them, such as switch_root.</p>
<p>Bugfixes: Avery Pennarun spotted a case where ls showed uid twice instead of
uid and gid, and that nice was using the wrong range of numbers.
The ls command also recursed inappropriately last time (not quite
properly converted for the dirtree changes last release), and now it's
fixed. Roy Tam pointed out a glitch in sh, and fixed df's percentage
calculation to match the POSIX spec. The kernel build didn't like our mktemp
and it does now. The wc command wasn't quite posix compliant (trailing spaces
break stuff). The ls command recursed inappropriately last time (not quite
properly converted for the dirtree changes last release), and now it's
fixed. The catv command wasn't displaying byte 255 correctly. Some lib
fixes (thinko in xpidfile). Fixed uname -m when running a 32 bit x86 binary
on an x86-64 host (it lies and says the system is i686, i586, or i486 depending
on what the toolchain that built the binary supported. This makes builds in
a 32 bit chroot on a 64 bit kernel break less.) The df command was checking
partitions in the wrong order (displaying undermounts instead of overmounts:
this used to work but some library code changed out from under it and it
wasn't updated to match until now). Felix Janda filled out the test suite
some more. The patch file creation logic got tweaked several times to
successfully apply more patches. Support for older (pre 2.10) glibc
versions was added to portability.h.</p>
<p>Miscelaneous cleanups all around (mknod, sha1sum, logname), including a
rewrite of taskset to be less dependent on libc getting the headers right. All
the command headers should now point to the current relevant standards
document, where applicable.</p>
<p>This news page had old news entries from before the relaunch moved into
a separate <a href=oldnews.html>oldnews</a> page.</p>
<p>I forgot to create <a href=bin>static binaries</a> last time, but they're
back now.</p>
</span>
<a name="23-07-2012" /><a href="#23-07-2012"><hr><h2><b>July 23, 2012</b></h2></a>
<blockquote><p>"Ford", Arthur said. "There's an infinite number of monkeys
out here who want to talk to us about this script for Hamlet they've worked
out."</p><p>- The Hitchhiker's Guide to the Galaxy</p></blockquote>
<p><a href=downloads/toybox-0.4.0.tar.bz2>Toybox 0.4.0</a> is based on
<a href=http://landley.net/hg/toybox/shortlog/640>commit 640</a>.</p>
<p>The new <a href=status.html>status page</a> is calculated from
the roadmap info, and should be easier to keep up to date in future.</p>
<p>Andre Renaud contributed od and modinfo. Elie De Brauwer contributed
taskset, bugfixes to cmp and tail, and tests for sort and tail. Kyungwan Han
contributed passwd. Gaurang Shastri contributed w. Ashwini Sharma spotted a
case where dirtree was adding extra slashes to a path.</p>
<p>I rewrote od, cleaned up comm, documented the
<a href=code.html#lib_llist>llist</a> and
<a href=code.html#lib_dirtree>dirtree</a> infrastructure, added an -r option
to date (and fixed a bug where -u wouldn't override /etc/localtime),
fixed bugs in chmod +stw, fixed ls to show suid bits properly when the
corresponding executable bit wasn't set, and worked around a longstanding
glibc bug where static linking prevents stdout from automatically flushing
pending output on exit.</p>
<a name="25-06-2012" /><a href="#25-06-2012"><hr><h2><b>June 25, 2012</b></h2></a>
<blockquote><p>"For a moment, nothing happened. Then, after a second or so, nothing continued to happen."</p><p>- The Hitchhiker's Guide to the Galaxy</p></blockquote>
<p><a href=downloads/toybox-0.3.1.tar.bz2>Toybox 0.3.1</a> is based on commit
<a href=http://landley.net/hg/toybox/shortlog/607>commit 607</a>. It's
mostly a bugfix release for ls -l (which was unhappy on targets other than
x86-64), plus a new "date" from Andre Renaud and rewritten chgrp/chown which
now support the full set of posix flags, plus a little work on the test
suite and some more header tweaks towards eventual compatability with the
musl libc.</p>
<p>The todo list runneth over, but "release early, release often", so here
it is. The roadmap and documentation are a bit behind, and I've got ~40
pending submissions to review. I need to catch up...</p>
</span>
<a name="12-06-2012" /><a href="#12-06-2012"><hr><h2><b>June 12, 2012</b></h2></a>
<blockquote><p>"For instance, on the planet Earth, man had always assumed that
he was more intelligent than dolphins because he had achieved so much - the
wheel, New York, wars and so on - whilst all the dolphins had ever done was
muck about in the water having a good time. But conversely, the dolphins had
always believed that they were far more intelligent than man - for precisely
the same reasons."</p><p>- The Hitchhiker's Guide to the Galaxy</p></blockquote>
<p>It's well past time for <a href=downloads/toybox-0.3.0.tar.bz2>toybox 0.3.0</a>,
so here it is, based
on <a href=http://landley.net/hg/toybox/shortlog/595>commit 595</a>, and the
statically linked <a href=downloads/binaries>prebuilt binaries</a> should
actually be statically linked this time (thanks Ashwini Sharma for spotting
that).</p>
<p>It's hard to figure out where to cut a release, because development
doesn't stop. "Long before now" is the obviuos answer, of course.
The project's maintainer also moved house during this development cycle, which
threw things off for a bit (so many boxes). Releases should hopefully be a bit
more frequent from here on.</p>
<p>The big things Rob worked on this time were the new dirtree (directory
tree traversal) infrastructure, and a complete rewrite of ls using that
which should now implement all 26 posix options.</p>
<p>Georgi Chorbadzhiyski added printenv, whoami, mkdir, mkfifo, chmod, chown,
chgrp, and uniq. He also added fraction and extension support to sleep (so if
you need a quarter-second sleep, it can do that now), and fixed a build bug
on slackware.</p>
<p>Daniel Walter contributed a string to mode_t parser (in use by chmod and
mkdir -m). Ilya Kuzmich contributed comm. Elie De Brauwer added mountpoint,
vmstat, logname, login, and mktemp. Kevin Chase did some portability cleanups.
Pere Orga fixed some documentation.</p>
<p>The "tac" and "clear" commands are now normal commands instead of shell
wrappers, and the header #includes have been cleaned up a bit to remove
deprecated functions and attempt to increase compatability with the bionic and
musl C libraries, "tail" should now use lseek() for large files, and "id" got
some cleanups and bugfixes.</p>
<p>The new TOYBOX_FLOAT configuration option selects whether or not
to include floating point support (for embedded targets where that's
problematic).</p>
<p>Several random bugfixes: unshare() might actually build portably now,
yes 'n' | cp -i should no longer bypass stdin and prompt via the tty, the
SUID support no longer drops permissions going through the toybox
multiplexer command, and a bugfix to xargs -0 means it should no longer
segfault. (I have a pending bug report about xargs not doing the full
posix whitespace handling that -0 obsoleted, but I'll deal with that next
release.)</p>
<p>The build infrastructure is now automatically generating FLAG_ macros
for the options, but currently with the wrong names. Some more macro glue
is necessary, which I haven't quite figured out how to do yet.</p>
<p>A defconfig toybox at the start of the $PATH has successfully built
Linux From Scratch (in my Aboriginal Linux project). The commands that
'default n' in the config are often still broken, cleanup is ongoing.
(The new dirtree stuff broke several of them that haven't been converted
yet, but if I wait until everything works we won't have a release before
1.0, so here's a checkpoint.)</p>
<a name="03-03-2012" /><a href="#03-03-2012"><hr><h2><b>March 3, 2012</b></h2></a>
<blockquote><p>"They went unnoticed at Goonhilly, passed over Cape Canaveral
without a blip, and Woomera and Jodrell Bank looked straight through them.
Which was a pity, because it was exactly the sort of thing they'd been looking
for all these years."</p><p>- The Hitchhiker's Guide to the Galaxy</p>
</p></blockquote>
<p>Here's <a href=downloads/toybox-0.2.1.tar.bz2>toybox 0.2.1</a> based
on <a href=http://landley.net/hg/toybox/shortlog/512>commit 512</a>. This
time around, there are statically linked <a href=downloads/binaries>prebuilt
binaries</a> for various embedded targets.</p>
<p>It's been a busy few weeks, almost entirely due to new contributors. (I
have not quite been keeping up.)</p>
<p>Elie De Brauwer contributed free, uptime, swapon, swapoff, lsmod, mknod,
insmod, rmmod, and fixed a bug in basename. Andre Renaud contributed ls, ln,
realpath, and hostname. Andres Heck contributed pidof and killall. Daniel
Walter wrote kill and extended id. Timothy Elliott contributed tail and tests
for cmp. Frank Bergmann sent a warning fix. Bryce Fricke added -i to cp.
Nathan McSween pointed out an optimization. Georgi Chorbadzhiyski fixed
cross compiling to work more reliably.</p>
<p>(My own contribution this time around was just tightening up other people's
code, a build fix to unshare, some random bugfixes, and so on. My only new
code this time around was writing a bash replacement for the existing python
bloat-o-meter.)</p>
<p>Last time (the 0.2.0 release) included the first pass at an id command from
Tim Bird, env and basename from Tryn Mirell, cmp and head from Timothy Elliott,
more bugfixes from Nathan McSween and Elie De Brauwer, and Luis Felipe Strano
Moraes did a first pass at the who command plus other bugfixes and
optimizations.</p>
<p>(For that release I did xargs, cal, truncate, unlink, nohup, tty, wc, link,
dirname, unshare, and various infrastructure tweaks, but it took me 3 months
and those guys did their stuff in a week or so.)</p>
<a name="12-02-2012" /><a href="#12-02-2012"><hr><h2><b>February 12, 2012</b></h2></a>
<blockquote><p>
"for though it has many omissions and contains much that is apocryphal, or at
least wildly inaccurate, it scores over the older, more pedestrian work in two
important respects..."</p>
<p> - The Hitchhiker's Guide to the Galaxy</p></blockquote>
<p>Here's the first BSD licensed release,
<a href=downloads/toybox-0.2.0.tar.bz2>toybox-0.2.0</a>, more a synchronization
point than anything particularly useful. 47 commands in a reasonably
ready-to-use state (what "make defconfig" builds), another ten or so partially
finished stubs ("make allyesconfig"), and several
patches pending on the mailing list I need to review and merge.</p>
<p>More to come...</p>
<hr>
<a name="15-11-2011" /><a href="#15-11-2011"><hr><h2><b>November 15, 2011</b></h2></a>
- Back from the dead, Toybox is now under a 2
clause BSD license, and aiming to become the default command line
implementation of Android systems everywhere.</p>
<p>More to come...</p>
<hr>
<p><a href=oldnews.html>Old news</a> from before the relaunch.</p>
<!--#include file="footer.html" -->
|
