Age | Commit message (Collapse) | Author |
|
Even though we had a portable extraction method for tarballs, this slowed
down the extraction immensely. I have added 2 new methods for tarball
extraction so that we have faster extraction speeds based on the system
tar implementation. Here are the options with the order of preference:
- pax: Although an uncommon program, pax is defined by POSIX, and
supports regular expressions, meaning that we can 'strip' components.
This is slightly faster than bsdtar and GNU tar.
- bsdtar/gnutar: We simply use '--strip-components 1' for these tar
implementations.
- tar: This is the portable method we had been using for a while, it's
good, gets the job done, but it is slower.
Here is also a little benchmarking I did by extracting the Linux tarball
for 10 times per method.
tarball=$CPT_CACHE/sources/linux-headers/linux-5.10.12.tar.xz
for extract in pax bsdtar tar; do
time sh -c ". cpt-lib
extract=$extract
for i in \$(seq 10); do
mkdir -p hdr; cd hdr;
tar_extract \"$tarball\"; cd ..;
rm -rf hdr
done"
done
This yields the following output, methods are as ordered above:
sh -c 75.21s user 11.11s system 115% cpu 1:14.90 total
sh -c 78.68s user 14.64s system 124% cpu 1:15.15 total
sh -c 99.88s user 69.99s system 119% cpu 2:21.70 total
FossilOrigin-Name: 45e3a9a03e7eff9b30b0e911d68d44068e8681ebf4501f271c8a686118f29243
|
|
FossilOrigin-Name: ff26aa326167c4751040d86e06e298ac11b4ed7923b729ed3940e881bdc24124
|
|
FossilOrigin-Name: fe4da8579b2d3046d754f207680c00f1d73e38ada24a60f2e05fb1e8c55532fe
|
|
FossilOrigin-Name: 536a4979e768c7b605a393251f0468a878e46a23b554045049b7404072b5258b
|
|
FossilOrigin-Name: a7f5ddfe5f7fdc9ea329ed8175c3ad237e7a7460bcbb3de4855d2af3f233176b
|
|
FossilOrigin-Name: a3feee4df8569e69b9b3581078ac9771e18f10af48476f16ae1cf798f07440f5
|
|
FossilOrigin-Name: acca0276be8cc7993174072783c5d7573ce6da9062a762ff1404ffcf27b4c99a
|
|
This ensures that we create the needed directories regardless of pkg_extract()
FossilOrigin-Name: c7c7c11246aeb74343f06e1d5d53ea9ad4cab2f4153d745cb00ae0c62f51b048
|
|
FossilOrigin-Name: fbdd0898cb6bcbfeda903ed9eb3c5641c2e0f504e503fcc6eba5ed8c500e236f
|
|
FossilOrigin-Name: a77a19063999297dfeb45dd725cb78dea0482030571dba0a34ae79b02e1c99fd
|
|
FossilOrigin-Name: 4bd8dc7830c8308c31510e5624550b1cf39df1401b4091a5bb80a2e24e109a30
|
|
- cleanup usage
- don't return with failure if a package doesn't have dependencies
FossilOrigin-Name: ea75189322f12eda82c9dfdff60f5997d79c97e8ecf4a433df1be024776ba651
|
|
- Respect given filenames even when the first argument is empty
- Clean up usage output
FossilOrigin-Name: a755fa3c9719f79ebf3855a2a54ba73cd63481d211bf0430fbfe5572d43e3ccd
|
|
FossilOrigin-Name: aae19d6384369835236023c30bfb54d24e9bf4021ff8d6fa10dbed8611419229
|
|
FossilOrigin-Name: 81b5d4e8c446aa4e4df789882bb5900d9636f4e5f6e9206e01ed42dfce946ad0
|
|
FossilOrigin-Name: ec9c5047e0bd1ef84104e66206db9a44e1b1e519f60519011c92b5c4e8e03657
|
|
FossilOrigin-Name: b254f7b748a13302feaf8d437dbce4d7ee743988d250a8f41143e96147dc30d5
|
|
FossilOrigin-Name: d97192df979c94b54e6beb46c60ddde4aa71705f9b78123e4496239ac5ea755b
|
|
FossilOrigin-Name: f47dc78435caed9f695a4f5cf61761945e741dd9fe5ee26d10532ab1d5f90f6d
|
|
FossilOrigin-Name: 6e3814b7119c2d167c02bdbfd747029fe3a8a9af78c38d430de5574706c22b04
|
|
FossilOrigin-Name: 4423fa4ed6d8b958a2cab88b834ef267b979861431542b79abf02a00fefeff0d
|
|
FossilOrigin-Name: 035deb0250c22e7feafad486f8f2b1a40893ad87073c8e41ba4bdc0db0d23d62
|
|
FossilOrigin-Name: ded347af5f33221e20649e7531847ab28a2a905c72b0c7a2f1bca9270cb20960
|
|
This bug caused swapping prefix files to malfunction on manifest files.
Example:
swapping redo caused redo-* files to be swapped on the manifest with the
name 'redo' without actually moving the files to the alternatives
directory either.
FossilOrigin-Name: 5966d079397154246173b94f53de83212c20edeb53f75089092cec352e6316f4
|
|
FossilOrigin-Name: d48d21b88e83f4bf4730dcaf3a528381821c415731f24620f2f09bf86fbc33a6
|
|
FossilOrigin-Name: 2586054342500fb820de12b297f7e93ff242c6f98ebcd19898e13a10aa78e675
|
|
FossilOrigin-Name: 6a651165f4dd8eb0357965743314a3aecb265b1ba8f4796d375a9b4521e50d3d
|
|
FossilOrigin-Name: 930c5a9d43e4b804cbbf6dc103b99cbc1b2e488d1935f7a32153c8ea46bfac84
|
|
FossilOrigin-Name: 86f635837ee62d81b8742fe845239233dd0f31f8cc763e35e7e699efc0631dbd
|
|
FossilOrigin-Name: f6b84f77b10b0ddc6f9560fe44b6ec4504dc3dac81c9c30df5b2017a333dbb26
|
|
Just so you know, the error is due to a bug on the older versions, but
we still don't want checks to fail because of it. Installing newer
versions of shellcheck takes too long on test containers that I am okay
with fixing a faulty error.
FossilOrigin-Name: 179ced7baf35e7eace835e1f7903ce72d2249e7ec6c809cfdd92ba7aa14b2517
|
|
FossilOrigin-Name: cc958373d206ec3dd65008461a715120b25f05772b9a2d70539b3a53839569c2
|
|
FossilOrigin-Name: e56971ac89f4a4e1c3efd62dad13b3a1c43bc6266c2803600995954cb61ee6cd
|
|
FossilOrigin-Name: f282158e981153846bc1aca83263de583403dd78444435cb139a7f25e6658c4e
|
|
FossilOrigin-Name: 3c44de3bfd6fbf69658371b2d8cd4d30043dd6f9b568c43ab30291925ebc582e
|
|
FossilOrigin-Name: e512f7d12e12c16fe109cfdf642361bb46ba594273ecacd93531c0ddfb144a8e
|
|
FossilOrigin-Name: ca05dfca6f8529b23647ddac4e9d7d3317c4508ee15045e60a4eac63787a04b7
|
|
FossilOrigin-Name: 47fe062d11c48e1cc51d6a2a4d6e3e4d54524e27281dc18ef69ec71f3bd9bcb4
|
|
FossilOrigin-Name: 779971096227b4fe8f2333a4104dfe6bd75dcebea08505aa21799a68fc66587a
|
|
FossilOrigin-Name: e34f2e3bf42cdfa6bdc94ffafbf1c7d58bc2a377608549e70c7b8bc426592272
|
|
FossilOrigin-Name: 17a020af2f5ea1a22f4f361781b4090f33021af001559e6040d49b71805f272f
|
|
FossilOrigin-Name: ba9f9fd3cd76c916c02529ef75189536fe0e3f35e0af65b2c22a268b19878d42
|
|
Using awk, we can achieve the following things:
- Ignore the rest of the file (2 times faster than sed)
- If the line doesn't match the docstring pattern, we can just print a
newline instead.
I am not exactly really familiar with awk scripting so there might be a
better way of doing this. In that case please send a patch and I will
apply it.
FossilOrigin-Name: bf65db74ad6552bb374f3535ef2a5333bbc0dc3d20563ec37e821036e07f621f
|
|
FossilOrigin-Name: 601842a9a0147dcf4ad04386ebd690ccc252cb6b27b98b7cd7cd06ea8b7ce6ad
|
|
FossilOrigin-Name: 9483e73d74a4ca38735f8128d695a008b1b93768e2b62a4b7184e7e90fcb0e62
|
|
FossilOrigin-Name: 63e4717d7cc2790f055d562b165fb7ada2a6a34760078c2d178027fae28034b2
|
|
FossilOrigin-Name: 76e5ec56a8f045f2b4a31feb4cacfdf02cc6cd86cf7d7884ff48addc4862e5aa
|
|
FossilOrigin-Name: 9e9a372496a106e5300bcd2b157e2eaae204795430b68be33226834bc550f6c1
|
|
FossilOrigin-Name: 2d2891423ef16e2e42572b38faa4f94061662b27131a4729053098e10851716a
|
|
FossilOrigin-Name: 3f1ed6e4cfa21578935f3f4cf88840be57bae853d5553f726fc5890e9c2ea5ee
|