diff options
51 files changed, 3655 insertions, 903 deletions
@@ -1,17 +1,37 @@ image: alpine/edge packages: + - musl-dev + - gcc + - bison - curl + - fossil - rsync - - shellcheck + - emacs-nox - texinfo - gzip + - xz tasks: + - install-pax: | + git clone --quiet https://github.com/carbslinux/otools + cd otools + ./configure + make pax + sudo install -Dm755 pax /usr/bin/pax + - install-blake3: | + git clone --quiet https://git.sr.ht/~mcf/b3sum + cd b3sum + sudo make PREFIX=/usr install + - install-shellspec: | + curl -fsSL https://git.io/shellspec | sudo sh -s -- -y -p /usr + - install-shellcheck: | + curl -fsLo- https://github.com/koalaman/shellcheck/releases/download/v0.8.0/shellcheck-v0.8.0.linux.x86_64.tar.xz | pax -Jr + sudo install -Dm755 shellcheck-v0.8.0/shellcheck /usr/bin/shellcheck - build: | cd cpt + ./configure make - test: | cd cpt - curl -fsSL https://git.io/shellspec | sudo sh -s -- -y -p /usr make test triggers: - action: email diff --git a/.fossil-settings/ignore-glob b/.fossil-settings/ignore-glob index d4cc917..84a1fc5 100644 --- a/.fossil-settings/ignore-glob +++ b/.fossil-settings/ignore-glob @@ -17,3 +17,5 @@ tests/etc/cpt-hook .shellspec-quick.log report coverage +config.mk +docs/config.org diff --git a/CHANGELOG.md b/CHANGELOG.md new file mode 100644 index 0000000..e7ee439 --- /dev/null +++ b/CHANGELOG.md @@ -0,0 +1,1011 @@ +CHANGELOG +================================================================================ + +This is the CHANGELOG for the Carbs Packaging Tools, initially a customised fork +of the `kiss` package manager. The format is based on [Keep a Changelog], and +this project _somewhat_ adheres to [Semantic Versioning]. + +[Keep a Changelog]: https://keepachangelog.com/en/1.0.0/ +[Semantic Versioning]: https://semver.org/spec/v2.0.0.html + + +[7.0.2] - 2023-02-05 +-------------------------------------------------------------------------------- + +### Fixed +- Fixed a bug that caused extra dependencies being added to the later packages + during multi-package build operations. +- Fixed file attribute issue with the `_tmp_cp()` function causing modified + dependency files to receive `600` permission bits. + + +[7.0.1] - 2023-02-05 +-------------------------------------------------------------------------------- + +### Fixed +- Made the `_tsort()` function compatible with POSIX +- Fixed dependency calculation issue in `pkg_depends()` where some packages + would be removed. +- Fixed `pkg_gentree()` not generating the dependency tree due to the dependency + calculation changes. + + +[7.0.0] - 2023-01-31 +-------------------------------------------------------------------------------- + +### Configuration Directory +- In order to simplify file locations and messing up the `/etc` directory, CPT + now uses the `/etc/cpt` directory for reading related files. The location of + your system configuration directory is defined by the `--sysconfdir` flag in + the `./configure` script, it uses `/etc` if the prefix is `/usr`. +- Since the location of the configuration can differ between installations, + `$cpt_confdir` variable can be used in programs using `cpt-lib` to get the + user's configuration directory. +- This change currently doesn't break `cpt-base`, but you are advised to + rename your configuration files. +- `/etc/cpt-base` is renamed to `/etc/cpt/base` (considering `$cpt_confdir` is + `/etc/cpt`) + +### Changes on hook behaviour +- `/etc/cpt-hook` will no longer be used. +- User hooks (as defined by `$CPT_HOOK` will be run regardless of the hook type. + I have realised that overriding user hooks on some operations was a mistake. + If the users already have the privilege to install packages, they should also + be able to run hooks without an interruption of the package manager. +- Even though `/etc/cpt-hook` file is removed, a collection of systemwide hooks + can be added to the `/etc/cpt/hooks`directory. Any file in this directory will + be sourced by the package manager when running hooks. User hooks are run + _after_ systemwide hooks are run. +- Added new hooks: `end-install` and `end-remove` that are run when + installation/removal is complete (not per-package). + +### Added +- `cpt-size` can now sort files based on size. +- `$CPT_NOSTRIP` variable can now be set to 1 in order to disable package + stripping. Make sure to add `-g` to your CFLAGS in order to keep debugging + symbols. +- `cpt-build` now accepts `-d` and `-S` options to enable `$CPT_DEBUG` and + `$CPT_NOSTRIP` respectively. + +### Changed +- `cpt-update` is now re-entrant, meaning that it is no longer needed to run the + update twice, `cpt-update` will continue the updates with the new version of + itself. +- The package manager now can handle circular dependencies and exit gracefully. + +### Fixed +- Fixed the behaviour of `cpt bi` and `cpt cbi` by merging the flag usage. +- Fixed the `aria2c` usage on `pkg_download()` function. + +### Library +- In order to get the `$deps` variable, one now has to use the new + `pkg_depends_commit()` function. + + +[6.2.4] - 2022-02-07 +-------------------------------------------------------------------------------- + +### Fixed +- Fixed a bug in missing dependency where if the user had chosen 'ldd', it would + fail to fix dependencies due to a typo. + + +[6.2.3] - 2022-02-02 +-------------------------------------------------------------------------------- + +### Fixed +- Fixed a checksum verification bug where adding an extra source did not require + checksum verification. +- `cpt-manifest-tree` now modifies the output of `tree(1)` according to the new + version. +- `cpt-reset` is now much more verbose. +- Fixed the displayed messages on `cpt-install` when it is given a tarball as an + argument. +- Fixed a faulty implementation in `pkg_tar()` where it used `pkg_find()` + instead of using the built package's database directory for gathering + information. + + +[6.2.2] - 2021-11-09 +-------------------------------------------------------------------------------- + +### Fixed +- `cpt-alternatives` now properly logs file swaps even when the original file + no longer exists. +- Minor fixes + + +[6.2.1] - 2021-09-20 +-------------------------------------------------------------------------------- + +### Fixed +- `cpt-fork` follows symbolic links when forking packages. +- Fixed "crux-like" usage in `cpt-size` +- Fixed documentation path in the manual page + + +[6.2.0] - 2021-08-14 +-------------------------------------------------------------------------------- + +### BLAKE3 checksums + +The package manager now uses `b3sum` for creating digests. The change is +backwards compatible, which means that BLAKE3 will slowly replace the SHA256 +algorithm in packages. The `cpt` package in the repository will continue to use +the sha256 until the end of 2021. + +The `checksums` files generated with BLAKE3 has the header `%BLAKE3` which is +used to distinguish digest algorithms. If the file does not include such a +header, it is assumed to be a file created using the SHA256 algorithm. This is +especially handy for keeping the /etc checksums intact. If the package being +built is already installed on the system `cpt` makes sure that the generated +etcsums are also backwards compatible. + + +### Description searching + +`cpt-search` utility has a new mode for searching through the package names and +descriptions, which is enabled by the `-q` flag. The output is really similar to +how the `apt search` command works, but the output is not meant for scripting. +Descriptions are defined by the `description` keys in the meta file. + +Instead of wildcards, the passed argument is expected to be a POSIX Basic +Regular Expression, and is interpreted by `grep`. `cpt-search` also accepts the +`-F` flag for passing literal expressions. + + +### Added +- `cpt-checksum` now has the `-s` flag to generate checksums using the SHA256 + digest algorithm. +- Added `CPT_DOWNLOADER` variable to change the download program. Available + options are: `curl`, `wget`, `wget2`, `aria2c`, and `axel`. +- `cpt-chroot` now has the flag `-m` to disable mounting/unmounting pseudo + filesystems. +- This changelog is now installed by the `Makefile`. +- `cpt-chbuild` now has `-r` flag to redownload the chroot. + +### Changed +- `cpt-size` has been rewritten to support POSIX `du`, and to support packages + with a file count of over 50,000. +- Installation now requires to run `./configure`. + + +[6.1.1] - 2021-08-04 +-------------------------------------------------------------------------------- + +### Fixed +- Fixed a rare bug during manifest generation that caused an empty line inside + the package manifest. + + +[6.1.0] - 2021-07-22 +-------------------------------------------------------------------------------- + +### IMPORTANT +- The package manager now enforces the usage of `pax` instead of `tar`. + +### Repository Backend + +`cpt` now has a faster and modular repository backend. `fossil` is now supported +by the package manager. During the repository fetch process, the repository +backend is stored in the cache directory so that the update takes less time on +the next pull. The usage of the repository cache can be disabled by setting +`$CPT_REPO_CACHE` to 0. + +### Added +- Added `-q, --quiet` flags to `cpt-list`. +- Re-added `cpt-maintainer`. It now supports `meta` and repository backends + other than `git`. +- The package manager now prints out `MOTD` files found on the repository root. +- Added the `$CPT_VERBOSE` variable and the `--verbose` flags to some utilities. + With the addition of these, some parts of the package manager will be more + quiet unless verbosity is explicitly requested. + +### Changed +- Reworked the package repository backend. + + +[6.0.4] - 2021-05-12 +-------------------------------------------------------------------------------- + +### Fixed +- Fixed the declaration place of the `$pid` variable + + +[6.0.3] - 2021-05-10 +-------------------------------------------------------------------------------- + +### Fixed +- Fixed interrupt behaviour when downloading package sources. +- Fixed `cpt --help` output when inside a directory prefixed with `cpt-` + + +[6.0.2] - 2021-04-05 +-------------------------------------------------------------------------------- + +### Fixed +- Fixed `make dist` target. + + +[6.0.1] - 2021-04-05 +-------------------------------------------------------------------------------- + +### Fixed +- Fixed inconsistencies of the `Ctrl+C` interrupt behaviour + + +[6.0.0] - 2021-04-03 +-------------------------------------------------------------------------------- + +### Added +- Added unit tests using `shellspec`. +- Added support for `pax` for tarball extraction. +- [ssu] support has been added for privilege escalation. +- Added `-p` flag for specifying package when using `cpt-link`. +- Added manual pages for all `cpt-contrib` scripts. +- Added `lz` compression/decompression support to `$CPT_COMPRESS`. + +### Changed +- Moved `cpt-lib` to `cpt-lib.in`. +- All `src` scripts now exit with success after being called with `--help`. +- Minor optimisations on `contrib` scripts. +- Huge speed improvement on `cpt-export`. +- Updated the `getoptions` library to version `2.5.0`. + +### Fixed +- Fixed `cpt-cat` not using the `CPT_ROOT` value. +- Fixed an error with the usage of `sbase grep` that resulted in exit when + removing packages. + +### Removed +- Removed C binaries `cpt-readlink` and `cpt-stat` and instead replaced them + with `_readlink()` and `_stat()` library functions. + + +[ssu]: https://github.com/illiliti/ssu + +[5.1.2] - 2021-01-04 +-------------------------------------------------------------------------------- + +### Fixed +- Fixed the `Makefile` installing files other than `cpt-*` files. +- Fixed `pkg_swap()` bug where swapping a prefix file would change the following + file locations on the manifest as well. + + +[5.1.1] - 2020-12-20 +-------------------------------------------------------------------------------- + +### Fixed +- Fixed an issue where the package build is continued even when it failed when + `$CPT_DEBUG` was set + + +[5.1.0] - 2020-11-25 +-------------------------------------------------------------------------------- + +### IMPORTANT +- The `pkg_fixdeps()` function has been renamed to `pkg_fix_deps()`. +- `cpt-lib` now parses options for you if `parser_definition()` is defined + before being called. + +### Changed +- `cpt-fork` can now take full path for packages. +- `cpt-size` can now display the total size of multiple packages. +- Updated `getoptions()` parser to version `2.0.1` +- Added `git` to the default `cpt-base`. +- Temporary cache creation scheme is changed from `build-$pid/` to + `proc/$pid/build/` + +### Fixed +- Moved option parsing to cpt-lib if `parser_definition` exists. This shouldn't + affect existing scripts where `cpt-lib` is called before the parser is + defined. + + +[5.0.0] - 2020-10-06 +-------------------------------------------------------------------------------- + +### IMPORTANT +- `cpt-fetch` has been removed. `cpt-update -o` can be used instead. + +### Added +- Added an `/etc/cpt-base` file to define a base. It can be used in order to + ship a default base, but to make it redefinable by the user. This file isn't + installed by default, it serves as a template. +- Added `cpt-orphans` to view orphaned packages. This tool uses `/etc/cpt-base` + and doesn't output any packages in the defined base. +- Added a `global_options()` function in order to add into the option parser. +- Added `cpt-update -o` flag to replace the functionality of `cpt-fetch`. +- Added `cpt-list -c` to use the current directory as the argument string. + +### Changed +- `pkg_build()` now notifies the user if the build file was modified inside a + hook (the `pre-build` hook to be precise). +- In git repository sources, `@` can now be used to specify tags. + E.g. `git+git://git.carbslinux.org/cpt@4.2.0` +- `cpt-fork` now removes `manifest` and `etcsums` files. +- `cpt-fork` can now be used to fork multiple packages. +- `cpt-reset` now uses `/etc/cpt-base` when removing packages. +- `cpt-build` now exports the `CPT_TEST` variable, so some tests that can't be + done in a `test` script can be done from the build itself. + + +[4.1.1] - 2020-09-25 +-------------------------------------------------------------------------------- + +### Changed +- Git clones now fetch tags if commits are specified. This makes the operation + longer, but not as long as cloning the whole repository while building a + package. +- `pkg_fixdeps()` now outputs to `stderr` instead of `/dev/tty`. You can now + have fully silent builds. + +### Fixed +- Fixed the `as_root()` function when using `su`. + + +[4.1.0] - 2020-09-11 +-------------------------------------------------------------------------------- + +### Added +- Added `bi` action to cpt for building and installing packages at the same time + +### Fixed +- Fixed `as_root()` call on `cpt-chbuild`. + + +[4.0.1] - 2020-09-10 +-------------------------------------------------------------------------------- + +### Fixed +- Fixed flags starting with `--no-` + + +[4.0.0] - 2020-09-09 +-------------------------------------------------------------------------------- + +With this update, all the documentation was moved to the `docs` repository, +which can be accessed from the following sources: + +- [Docs Repository](https://github.com/carbslinux/docs) +- [Online User Manual](https://carbslinux.org/docs) +- `carbs-docs` package + +### Added +- Added the ability to test packages using a new executable file `test`. +- Added `$CPT_TEST` variable for testing packages. +- Added `--test|-t` option to build. +- Added support for `mercurial` repositories. +- Added options to install the tools "static" so they don't depend on cpt-lib. +- Added basic unit tests. See `make test`. + +### Changed +- Most contrib scripts now use the current directory as the package name. + +### Removed +- Removed the `docs/` folder. + +### Fixed +- Fixed `getoptions` parsers while declaring initial variables. +- Fixed build `cpt-stat` on the Makefile. + +[3.3.1] - 2020-08-31 +-------------------------------------------------------------------------------- + +### Changed +- Reverted `sh256()` to the previous way. + + +[3.3.0] - 2020-08-31 +-------------------------------------------------------------------------------- + +### Added +- Added `trap_set()` in order to manage traps. + +### Changed +- Moved from `getopt` to a shell implementation of option parsing. This ensures + portability, and doesn't depend on a C program with GNU extensions. That was + a mistake. The new implementation is taken from the public domain library, + `getoptions`. +- `warn()` function was modified to use `log "$1" "$2" "${3:-WARNING}"` instead. +- Made `cpt` checksum method compatible with the KISS Community repository. + + +[3.2.0] - 2020-08-22 +-------------------------------------------------------------------------------- + +### Added +- A `.build.cpt` file can be edited during the pre-build hook, so that a build + script can be modified. If the build is modified, a diff file will be + generated to the package database. +- Some of the tools now use getopt. Since getopt isn't a POSIX utility, + util-linux implementation has been added to the `getopt-ul` directory. It + consists only of files required for the tool to be built. +- Added `pre-chroot` hook for the `cpt-chbuild` utility. + +### Changed +- `cpt-chbuild` now uses library functions such `sh256()`, and `as_root()`. +- `cpt` programs no longer exit if `$CPT_PATH` is unset. + + +[3.1.1] - 2020-08-11 +-------------------------------------------------------------------------------- + +### Changed +- `as_root()` now sets package manager variables with env. + + +[3.1.0] - 2020-08-07 +-------------------------------------------------------------------------------- + +### Changed +- `cpt-lib` no longer creates temporary directories. This will need manual + adjustments for scripts that make use of the cache directories. Those + directories can now be created by calling the `create_cache()` function. +- Dropping libtool's `*.la` library files from packages. + + +[3.0.0] - 2020-07-24 +-------------------------------------------------------------------------------- + +This is the 3.0.0 release. This will make `kiss` (now renamed as `cpt`) a +toolchain for package management rather than a single script program. The main +functionality is moved into a `lib.sh` file which the tools will source. This +comes with nice benefits such as: + +- Easier option parsing for each tool. +- Easier to extend the package manager as it is only a library. It no longer + requires dirty hacks to source the package manager functions and variables. +- Clearer usage information is outputted, so the user doesn't have to delve into + documents to see the syntax/options of a tool. + +### Added +- `$CPT_CACHE` to change the cache directory. +- Added a bunch of flags, here is a table: + +| Flag | Function | Added tool | +|------------|-------------------------------------------|------------------------| +| --force | Force removal/installation | cpt-remove/cpt-install | +| --root | Specify root directory | lots of tools | +| --download | Only download packages | cpt-update | +| --no-fetch | Do not fetch repositories before update | cpt-update | +| --single | Only show the first instance of a package | cpt-search | + + +### Changed + +- Renamed all variables from `KISS-*` to `CPT-*` +- Moved database to `/var/db/cpt` +- Changed the code style and did some minor nitpicks for the C programs. + +### Removed +- Removed the ability to control color output (for now). +- Removed `kiss-outdated` and `kiss-which`. + + +[2.3.0] - 2020-07-16 +-------------------------------------------------------------------------------- + +### Added +- Added `KISS_FETCH` to optionally disable repository fetches while performing + a system update. You can now run `KISS_FETCH=0 kiss u` in order to update + your system without syncing repositories. + +### Changed +- Changed usage outputs for kiss and contrib utilities. +- `rsync` repositories are now synced based on checksums rather than timestamps. +- `kiss-chroot` now uses system flags if available. +- `kiss-chbuild` now installs extra packages if specified. +- hooks now default to `null` if no arguments are given +- `*-pull` hooks have been renamed to `*-fetch` and is run only once instead of + for every single git repository. + +### Fixed +- Fixed an issue where using `su` to install packages resulted in a wrong + package ownership. + + +[2.2.1] - 2020-06-11 +-------------------------------------------------------------------------------- + +### Fixed +- Fixed directory checking on package removal + + +[2.2.0] - 2020-06-10 +-------------------------------------------------------------------------------- + +### Added +- Makefile configurations were moved to config.mk. + +### Changed +- `kiss` no longer ignores musl and gcc on `fixdeps()`. This will result in an + influx of musl dependencies. But you will be needing the C library to be + installed anyway if you want your programs to work. If your program links + to `libgcc`, you will need the gcc package for that given program to function. +- Makefile now properly accepts `LIBS`, `LDFLAGS`, and `CFLAGS`. +- Updated documentation. + +### Fixed +- `C89` compatibility on C programs. +- Fixed an alternatives issue where a file containing special regular expression + characters (such as `/usr/bin/[`) would result in a manifest deletion. + + +[2.1.2] - 2020-06-03 +-------------------------------------------------------------------------------- + +### Fixed +- Fixed a segmentation fault on `kiss-stat` when a file didn't have on owner on + the `passwd` database. + + +[2.1.1] - 2020-06-03 +-------------------------------------------------------------------------------- + +### Fixed +- Fixed 'No message in queue' message being outputted for every single package. +- Fixed CFLAGS for x86_64 on `kiss-chroot`. +- Fixed setting binary packages as dependencies. + + +[2.1.0] - 2020-05-29 +-------------------------------------------------------------------------------- + +### Added +- Added '$2' '$3' for build scripts which specifies version and architecture + information. + +### Changed +- `kiss-chroot` now sets architecture based on the system +- Updated documentation + +### Removed +- Removed strip messages + + +[2.0.0] - 2020-05-28 +-------------------------------------------------------------------------------- + +### Added +- Rsync repository support. +- pre/post hooks for package removal (pre-remove, post-remove). +- pre/post hooks for git pulls (pre-pull, post-pull). + +### Changed +- `kiss` no longer removes empty directories if they are defined on a different + package. +- `$KISS_NOPROMPT` has been renamed to `$KISS_PROMPT` and must be set to 0 in + order to disable prompts. +- `kiss-chbuild` now checks tarball digest. +- `kiss-chbuild` now downloads tarballs according to arch (x86_64 or i686 + currently). +- Submodule repository fetching has been modified to match compatibility. + +### Removed +- Removed `kiss-maintainer` and moved it to [kiss-extra] + +[kiss-extra]: https://github.com/carbslinux/kiss-extra + + +[1.22.4] - 2020-05-26 +-------------------------------------------------------------------------------- + +**NOTE:** `1.22.x` is the last minor version before `2.0.0`, meaning I will not +be doing any releases except for patches and fixes. My attention is now on +implementing binary repositories. I will be doing some 'release candidates' +before release, as binary repositories will need user feedback. + +### Added +- Added new documents. +- Added `post-package` hook. + +### Changed +- Renamed the `hashcheck` function to `sh256` for compatibility. +- Enabled the usage of glob characters for `kiss-bin`. + + +[1.22.3] - 2020-05-18 +-------------------------------------------------------------------------------- + +### SECURITY +- Fixed a bug regarding privilege escalation using `$KISS_HOOK`. `kiss` will now + use `$KISS_ROOT/etc/kiss-hook` on installation operations (which are run by + root) so that the hooks are defined by the system administrator rather than the + user. See [related proof-of-concept] + +[related proof-of-concept]: https://github.com/kisslinux/kiss/pull/157#issuecomment-629880775 + + +[1.22.2] - 2020-05-16 +-------------------------------------------------------------------------------- + +### Fixed +- Fixed an issue where `pkg_conflicts` would abort if `kiss-readlink` failed due + to missing components. It now fallbacks to the original directory name. + + + +[1.22.1] - 2020-05-15 +-------------------------------------------------------------------------------- + +### REMOVED +- Removed some contrib scripts and moved them to [kiss-extra] +- `kiss-cargo-urlgen` +- `kiss-cargolock-urlgen` +- `kiss-changelog` +- `kiss-depends-finder` +- `kiss-exec` +- `kiss-message` +- `kiss-orphans` +- `kiss-reporevdepends` + + +### Fixed +- Fixed a `kiss-owns` typo that caused it to fail. +- Fixed a `kiss-readlink` bug where it would fail if the last component wouldn't + exist. +- Fixed an error on tarball extraction where a file name containing spaces would + be parsed as two files. + + +[kiss-extra]: https://github.com/carbslinux/kiss-extra + + +[1.22.0] - 2020-05-14 +-------------------------------------------------------------------------------- + +### Added +- Added `kiss-exec`, a tool to execute commands inside the alternatives system. + +### Changed +- Replaced `KISS_COLOUR` with `KISS_COLOR` to match upstream. +- Renamed `colour` variable to `color` for consistency. +- The package manager no longer needs root privileges if the `KISS_ROOT` is + writable by the user. +- `kiss` now uses the host cache regardless of `KISS_ROOT`. + +### Fixed +- Fixed an issue where `kiss-owns` would output the wrong package because of + symbolic links. The script now reads the link of the directory instead of the + full file. + + +[1.21.1] - 2020-05-14 +-------------------------------------------------------------------------------- + +### Changed +- All contrib messages now output usage information when called with `--help` + and `-h`. +- `hashcheck` function now uses `$1` instead of `${file#\*}`. + +### Fixed +- Fixed a non-POSIX `find` call. Thanks to @illiliti. + + +[1.21.0] - 2020-05-12 +-------------------------------------------------------------------------------- + +### Added +- Added a `d|download` option to acquire the sources of given packages. If no + packages are given, it acquires the sources of outdated packages. This can be + used to acquire a package's sources to build it later, or periodically + downloading outdated package sources, so the user doesn't wait for the download + when updating the system. +- kiss now understands `.txz` tarballs. (BSD `src.txz` wink wink) +- `KISS_TMPDIR` can now be used to specify a temporary build directory. This + will be useful for those of you who would want to build on ram or a different + file system. + +### Changed +- Simplified tarball extraction method. +- Removed the 'esc' function inside kiss. +- Added a 'warn' function to standardise warnings inside kiss + +### Fixed +- Removed the `sys_db` usage on `pkg_find()` where directories could clash with + external utilities. + + +[1.20.3] - 2020-05-09 +-------------------------------------------------------------------------------- + +### Fixed +- Fixed an alternatives bug caused by the previous patch, where the package + moving to `/var/db/kiss/choices` would take the name of the preferred package. + + +[1.20.2] - 2020-05-09 +-------------------------------------------------------------------------------- + +### Fixed +- Fixed an issue regarding manifest format when using pkg_swap (alternatives). + + +[1.20.1] - 2020-05-08 +-------------------------------------------------------------------------------- + +### Changed +- Faster conflict resolution by using a conflict cache file. +- Standardised `kiss-readlink` usage output. + + +[1.20.0] - 2020-05-07 +-------------------------------------------------------------------------------- + +### Added +- `KISS_NOPROMPT` can be specified in order to skip prompts. + + +[1.19.1] - 2020-05-07 +-------------------------------------------------------------------------------- + +### Added +- Added `e|extension` to `kiss` which can be used to output kiss-extensions. + +### Changed +- `kiss` no longer outputs the extensions when called with `kiss help`. The + output was too large for an average terminal, and a user had to scroll up + for actual package manager options. These can now be retrieved with `kiss e`. +- When called from a subshell, `kiss` disables colour escape sequences. This + behaviour can be overridden by setting `KISS_COLOUR` environment variable. + If set to 1, it will be enabled globally, if set to 0 it will be disabled + globally. + + +[1.19.0] - 2020-05-06 +-------------------------------------------------------------------------------- + +### Added +- Added `kiss-reporevdepends` for finding all the packages on the repository + that depends on the specified package. + +### Changed +- Removed the `-p` flag from tar while installing packages. busybox ignores it + and we don't need it. +- Replaced tar flags with keys for historical compatibility. +- `kiss` now decompresses a tarball once and uses the decompressed tarball for + listing and extraction. + +### Fixed +- Fixed the output of doc-strings in contrib scripts. +- `kiss` now ignores the binary programs in the repository for + `kiss extensions`. + + +[1.18.0] - 2020-05-04 +-------------------------------------------------------------------------------- + +### Added +- Added editorconfig file since we now have 4 languages (roff, Makefile, sh, C) + in the repository. +- Added `kiss-readlink` as a `readlink -f` replacement. +- Added `kiss-message` for checking package messages. +- Added this CHANGELOG + +### Changed +- Made tar calls portable. `kiss` is no longer limited to busybox, bsdtar, or + gnu tar! + +### Removed +- Dropped the usage of `readlink` in `kiss`. + + +[1.17.0] - 2020-05-03 +-------------------------------------------------------------------------------- + +### Added +- Added `kiss-stat`, a simple C program for getting the owner name of a + file/directory. + +### Removed +- Removed the usage of `stat` calls, as they are not standardised. + +### Changed +- `kiss` now doesn't report `Generating etcsums` if there isn't an `/etc` + directory in the package + +### Fixed +- `kiss` now uses `sys_db` instead of `pkg_db` when removing packages from the + system. +- `kiss` now explicitly exits if prompt returns with a status of 1. This is for + `ksh` compatibility. + + +[1.16.3] - 2020-05-03 +-------------------------------------------------------------------------------- + +### Fixed +- Fixed etcsum location. + + +[1.16.2] - 2020-05-03 +-------------------------------------------------------------------------------- + +### Added +- Added fallbacks for sha256sum. `kiss` now fallbacks to `sha256`, and `openssl` + for hash checking, respectively. +- Added `kiss-changelog` and `kiss-which` entries to the `kiss-contrib.1` manual + page. + +### Fixed +- Fixed `kiss` not using the preferred `tar` program on decompression. +- Fixed `pkg_conflicts()` getting stuck when you reinstall the same single + package on a system. +- Fixed `pkg_conflicts()` giving an error if no packages are installed on a + system. + + +[1.16.1] - 2020-05-01 +-------------------------------------------------------------------------------- + +### Fixed +- Fixed `ksh` Ctrl+C interrupt signals. + + +[1.16.0] - 2020-05-01 +-------------------------------------------------------------------------------- + +### Added +- New message queue system implementation. If a package includes a file named + `message` it will output its contents in a pretty way. +- Added `kiss-which`, a tool like `which` but for `kiss` packages. +- Added `kiss-changelog`, a tool for outputting the git log of a given package. + +### Fixed +- Fixed colour outputting on `OpenBSD ksh`. +- Made compatibility fixes on the Makefile +- Fixed an installation issue where `kiss` would look for the manifest with the + `$KISS_ROOT` variable + + +[1.15.0] - 2020-04-30 +-------------------------------------------------------------------------------- + +I have decided to revert to rsync installation method as it is faster and safer. +Now, rsync is not a huge dependency. Saving 500Kb is not worth it when you look +at the trade-off. + +### REMOVED +- Removed the new installation method. + +### Changed +- Reverted to `rsync` for installation. +- We are now forcing decompression to standard output while using `decompress()` +- `kiss` now accepts decompressed tar archives for package installation as well. + + +[1.14.2/1.14.3] - 2020-04-27 +-------------------------------------------------------------------------------- + +### Fixed +- The new installation method now complies to the `$KISS_ROOT` value. + + +[1.14.1] - 2020-04-27 +-------------------------------------------------------------------------------- + +### Removed +- Removed the unnecessary `[ -d ]` from the path query. + +### Fixed +- Fixed directory permissions in the new installation method. +- Added support for `$KISS_ROOT` to the new installation method. +- Added a check for symlinks that could overwrite a directory during + installation. +- Whitespace cleanup. + + +[1.14.0] - 2020-04-25 +-------------------------------------------------------------------------------- + +This release removes `rsync` from `kiss` and implements its own installation +method. + +### Added +- `kiss` now implements user scripts available in the `$PATH`. This can be used + to call `kiss manifest` from `kiss` itself. +- `kiss` now displays a warning if the user has a `$KISS_ROOT` that ends with + a `/`. +- `kiss` now uses its own new package installation method. + +### Removed +- usage of rsync as an installation method. +- usage of `old_ifs` + + +[1.13.1] - 2020-04-19 +-------------------------------------------------------------------------------- + +### Removed +- Reverted and removed `kiss ss` changes. + + +[1.13.0] - 2020-04-19 +-------------------------------------------------------------------------------- + +### Added +- `kiss ss` for outputting a single location for scripting. + +### Changed +- `kiss` now elevates permissions during checksum if the file isn't owned by us. +- Hide read error messages when `version` file cannot be found. +- Made the `pkg_fixdeps()` usage of `diff` portable. + +### Fixed +- Fixed a shellcheck error. + + +[1.12.3] - 2020-04-17 +-------------------------------------------------------------------------------- + +### Changed +- Changed indentation style on decompression. + +### Fixed +- `kiss-export` fallbacks to gzip if the compression method is unknown. +- `kiss-repodepends` now outputs the proper depends file. + + +[1.12.2] - 2020-04-15 +-------------------------------------------------------------------------------- + +### Added +- `kiss` can now decompress zip files. + +### Changed +- `checksum`, `build`, `install`, and `remove` operations can now be done on the + current package directory without arguments. So you can `cd` into a package an + type `kiss b` to build that package. + +### Fixed +- `kiss-export` now honours your `KISS_COMPRESS` value + + +[1.12.1] - 2020-04-12 +-------------------------------------------------------------------------------- + +### Fixed +- Fixed printing empty arguments in `kiss-outdated`. + + +[1.12.0] - 2020-04-05 +-------------------------------------------------------------------------------- + +### Added +- `kiss-cargolock-urlgen`: a tool that can read a Cargo.lock file to generate a + list of urls. +- `kiss-cat`: a tool to concatenate package build files. +- Manual page for `kiss-contrib`. + +### Changed +- `kiss-owns` now gives an error on directories. +- `kiss-link` can now take multiple file names and will create symbolic links + of them all. +- Simplified `kiss-link` + +### Fixed +- `kiss-cargo-urlgen`: Fixed an issue where if a package version contained a '-' + it could lead to wrong url generation. + + +[1.9.0 - 1.11.2] - 2020-04-04 +-------------------------------------------------------------------------------- + +### Added +- `kiss f` to fetch repositories without an update prompt. +- Support for submodules in the repository. +- Added a Makefile to standardise the installation. +- Zstd compression method. + +### Changed +- Modified `kiss-chbuild` to fit Carbs Linux. +- Changed README to notify about forking status. +- `pkg_find()` now also checks for symlinks instead of just directories. +- `pkg_find()` now uses a `for` loop instead of `find`. +- Force C locale for faster grepping. +- Easily readable manual page. + +### Fixed +- Compression method now fallbacks to gzip if unknown. @@ -1,6 +1,6 @@ The MIT License (MIT) -Copyright (c) 2020-2021 Cem Keylan +Copyright (c) 2020-2023 Cem Keylan Copyright (c) 2019-2020 Dylan Araps Permission is hereby granted, free of charge, to any person obtaining a copy @@ -5,20 +5,29 @@ INSTALL_SH = ./tools/install.sh CONTRIB = `find contrib -name 'cpt*' ! -name '*.*'` SRC = `find src -name 'cpt*' ! -name '*.*'` BIN = ${SRC} ${CONTRIB} -LIB = src/cpt-lib -LIB_IN = ${LIB:=.in} all: src/cpt-lib - test "${DOCS}" != yes || ${MAKE} -C docs all + @test "${DOCS}" != yes || ${MAKE} -C docs all src/cpt-lib: src/cpt-lib.in - sed -e "s|@VERSION@|${VERSION}|g" \ - -e "s|@DOCSTRING@|Call functions from the library|g" < src/cpt-lib.in > $@ + sed -n '/^Copyright/{s,^, ",;s,$$," \\,;p}' LICENSE | \ + sed -e '/@LICENSE@/r /dev/stdin' \ + -e '/@LICENSE@/d' \ + -e 's|@SYSCONFDIR@|${SYSCONFDIR}|g' \ + -e "s|@VERSION@|${VERSION}|g" \ + -e "s|@DOCSTRING@|Call functions from the library|g" src/cpt-lib.in > $@ chmod 755 $@ -test: all tests/etc/cpt-hook +shellspec: all tests/etc/cpt-hook shellspec - cd src; find . ../contrib -name 'cpt*' ! -name '*.*' -exec shellcheck -e 2119 -x -f gcc {} + + +shellcheck: all + @cd src; find . ../contrib -name 'cpt*' ! -name '*.*' | while read -r file; do \ + echo SHELLCHECK "$$file"; \ + shellcheck -e 2119 -x -f gcc "$$file"; \ + done + +test: shellspec shellcheck tests/etc/cpt-hook: ln -s ../hook-file $@ @@ -28,6 +37,11 @@ dist: docs/cpt.info install: all test "${DOCS}" != yes || ${MAKE} -C docs install + [ -f docs/cpt.info ] && \ + ${INSTALL_SH} -Dm644 docs/cpt.info ${DESTDIR}${INFODIR}/cpt.info ||: + [ -f docs/cpt.txt ] && \ + ${INSTALL_SH} -Dm644 docs/cpt.txt ${DESTDIR}${DOCDIR}/cpt.txt ||: + ${INSTALL_SH} -Dm644 CHANGELOG.md ${DESTDIR}${DOCDIR}/CHANGELOG ${INSTALL_SH} -Dm755 -t ${DESTDIR}${BINDIR} ${BIN} ${INSTALL_SH} -Dm644 -t ${DESTDIR}${MAN1} man/*.1 for man in ${CONTRIB}; do \ @@ -36,15 +50,19 @@ install: all done uninstall: - test "${DOCS}" != yes || ${MAKE} -C docs uninstall for bin in ${BIN}; do \ rm -f ${DESTDIR}${BINDIR}/$${bin##*/}; done for man in man/*.1; do rm -f ${DESTDIR}${MAN1}/$${man##*/}; done for man in ${CONTRIB}; do rm -f ${DESTDIR}${MAN1}/$${man##*/}.1; done + rm -rf ${DESTDIR}${DOCDIR} + rm -f ${DESTDIR}${INFODIR}/cpt.info clean: - test "${DOCS}" != yes || ${MAKE} -C docs clean + ${MAKE} -C docs clean rm -rf src/cpt-lib "cpt-${VERSION}.tar.xz" coverage report rm -f tests/etc/cpt-hook -.PHONY: all dist clean install uninstall +allclean: clean + rm -f config.mk + +.PHONY: all dist allclean clean install uninstall shellspec shellcheck test @@ -1,8 +1,15 @@ Carbs Packaging Tools ===================== -Package management toolset for Carbs Linux. Forked from KISS[1]. All usage -information can be obtained from the manual page. Refer to the ChangeLog[2] to + ##### #### ##### + # # # # # + # #### # + # # # # + ##### # # + + +Package management toolset for Carbs Linux. Forked from [KISS]. All usage +information can be obtained from the manual page. Refer to the [ChangeLog] to learn what's new. @@ -14,7 +21,7 @@ To build and use cpt, you need the following software: - rsync - curl - POSIX base utilities [coreutils, busybox, sbase, etc.] -- pax or tar [GNU tar, busybox, toybox, libarchive, etc.] +- pax - Common compression utilities such as `gzip`, `bzip2`, `xz`, etc. @@ -36,10 +43,11 @@ either passing `DOCS=no` to `make` or editing `config.mk` to disable it. Installation ------------ -In order to install CPT, you can run the following with `make`: +In order to install CPT, you can run the following: + ./configure make - make PREFIX=/usr install + make install Documentation @@ -47,12 +55,12 @@ Documentation The documentation for each tool along with some examples can be found on manpages installed with the package manager. User manual of CPT can be found -online[3], or installed as both plain-text and as info pages. Without any +[online], or installed as both plain-text and as info pages. Without any changes to the Makefile configuration those files can be found at: - /usr/local/share/docs/cpt/cpt.txt - /usr/local/share/info/cpt.info -[1]: https://github.com/kisslinux/kiss -[2]: https://fossil.carbslinux.org/cpt/log -[3]: https://carbslinux.org/docs/cpt +[KISS]: https://github.com/kisslinux/kiss +[ChangeLog]: https://fossil.carbslinux.org/cpt/doc/trunk/CHANGELOG.md +[online]: https://carbslinux.org/docs/cpt diff --git a/config.mk b/config.mk deleted file mode 100644 index 1bc27fe..0000000 --- a/config.mk +++ /dev/null @@ -1,18 +0,0 @@ -# Carbs Packaging Tools -VERSION = dev - -# Installation paths -PREFIX = /usr/local -BINDIR = ${PREFIX}/bin -SHAREDIR = ${PREFIX}/share -INFODIR = ${SHAREDIR}/info -DOCDIR = ${SHAREDIR}/doc -CPTDOC = ${DOCDIR}/cpt -MANPREFIX = ${SHAREDIR}/man -MAN1 = ${MANPREFIX}/man1 - -EMACS = emacs -MAKEINFO = makeinfo - -# Comment or change if you don't want to build/install the documentation -DOCS = yes diff --git a/configure b/configure new file mode 100755 index 0000000..6268da7 --- /dev/null +++ b/configure @@ -0,0 +1,121 @@ +#!/bin/sh -e + +version=Fossil + +die() { + printf '%s: %s\n' "${0##*/}" "$*" >&2 + exit 1 +} + +out() { printf '%s\n' "$@" ;} + +_check() { + for arg; do + printf 'checking for %s... ' "$arg" + command -v "$arg" || { out no; die "'$arg' not found" ;} + done +} + +_check_multi() { + c=$1; shift + printf 'checking for %s... ' "$c" + for arg; do command -v "$arg" && return 0; done + out no; die "no $c was found" +} + +usage() { + out "usage: $0 [options]" \ + "Options:" \ + " --prefix=dir Set prefix directory" \ + " --bindir=dir User executables [PREFIX/bin]" \ + " --sysconfdir=dir System configuration directory [PREFIX/etc]" \ + " --datarootdir=dir Data root directory [PREFIX/share]" \ + " --mandir=dir Manual pages [DATAROOTDIR/man]" \ + " --infodir=dir info documentation [DATAROOTDIR/info]" \ + " --docdir=dir Documentation directory [DATAROOTDIR/doc/cpt]" \ + " --with-docs=opt Whether to build the texinfo documentation [auto]" "" \ + " MAKEINFO Name of the 'makeinfo' executable" \ + " EMACS Name of the 'emacs' executable" "" \ + "Use these variables to override the behaviour of '$0'." + exit 1 +} + +prefix=/usr/local +# We don't want expansion +# shellcheck disable=2016 +{ +sysconfdir='$(PREFIX)/etc' +bindir='$(PREFIX)/bin' +datarootdir='$(PREFIX)/share' +mandir='$(DATAROOTDIR)/man' +infodir='$(DATAROOTDIR)/info' +docdir='$(DATAROOTDIR)/doc/cpt' +} +docs=auto + +for arg; do + case $arg in + -h|--help) usage ;; + --prefix=*) prefix=${arg#*=} ;; + --bindir=*) bindir=${arg#*=} ;; + --sysconfdir=*) sysconfdir=${arg#*=} ;; + --mandir=*) mandir=${arg#*=} ;; + --infodir=*) infodir=${arg#*=} ;; + --docdir=*) docdir=${arg#*=} ;; + --without-docs) docs=no ;; + --with-docs) docs=yes ;; + --with-docs=*) docs=${arg#*=} ;; + *-*) die "Unknown option: '$arg'" ;; + *=*) export "${arg:?}" ;; + *) die "Unknown argument: '$arg'" + esac +done + +# If the prefix is /usr and sysconfdir is not modified, make it /etc +# shellcheck disable=2016 +[ "$prefix" = /usr ] && [ "$sysconfdir" = '$(PREFIX)/etc' ] && sysconfdir=/etc + +trap 'rm -f config.mk' EXIT +trap 'rm -f config.mk; exit 1' INT + +: "${EMACS:=emacs}" "${MAKEINFO:=makeinfo}" + +out "starting configuration..." + +cat <<EOF > config.mk +PREFIX = $prefix +BINDIR = $bindir +SYSCONFDIR = $sysconfdir +DATAROOTDIR = $datarootdir +MANDIR = $mandir +INFODIR = $infodir +DOCDIR = $docdir +MAN1 = \$(MANDIR)/man1 + +VERSION = $version +EMACS = $EMACS +EOF + +case $docs in + auto|yes) + printf 'checking for makeinfo... ' + if makeinfo=$(command -v "$MAKEINFO"); then + out "$makeinfo" + docs=yes + out "MAKEINFO = $makeinfo" >>config.mk + else + out no + [ "$docs" = yes ] && die "'$MAKEINFO' not found" + docs=no + fi +esac +[ "$docs" = no ] && out "not building documentation" +out "DOCS = $docs" >>config.mk + +out "checking runtime dependencies" +_check pax rsync sed awk grep b3sum +_check_multi "sha256 provider" sha256sum sha256 openssl +_check_multi "download utility" curl wget wget2 aria2c axel + +trap - EXIT INT +out "written config.mk" "Run 'make' to build cpt" diff --git a/contrib/cpt-cat b/contrib/cpt-cat index bfc9d66..45e0dd9 100755 --- a/contrib/cpt-cat +++ b/contrib/cpt-cat @@ -28,23 +28,29 @@ ## .Em version ## files. -case "$1" in - --help|-h) - printf 'usage: %s [pkg] [file...]\n' "${0##*/}" - exit 0 - ;; - '') - [ "$#" -gt 1 ] && shift - set -- "${PWD##*/}" "$@" -esac +parser_definition() { + setup REST help:usage -- "usage: ${0##*/} [pkg] [file...]" + global_options silent +} + +# shellcheck disable=1091 +. cpt-lib + +[ "$1" ] || { + # Usage such as `cpt-cat '' build` is also valid. + [ "$#" -gt 1 ] && shift + set -- "${PWD##*/}" "$@" +} pkg=$1; shift -cpt-list "$pkg" >/dev/null +pkg_list "$pkg" >/dev/null [ "$1" ] || set -- build depends sources version +# $sys_db and color variables are defined by cpt-lib +# shellcheck disable=2154 for file; do - [ -f "$CPT_ROOT/var/db/cpt/installed/$pkg/$file" ] || continue - printf '\033[1m%s:\033[m\n' "$file" >&2 - cat "$CPT_ROOT/var/db/cpt/installed/$pkg/$file" + [ -f "$sys_db/$pkg/$file" ] || continue + printf '%b%s:%b\n' "$colbold" "$file" "$colre" >&2 + cat "$sys_db/$pkg/$file" done diff --git a/contrib/cpt-chbuild b/contrib/cpt-chbuild index e53d903..7a71c33 100755 --- a/contrib/cpt-chbuild +++ b/contrib/cpt-chbuild @@ -3,6 +3,7 @@ ## SYNOPSIS: ## .Nm +## .Op Fl r ## .Op Ar pkg... ## DESCRIPTION: @@ -13,13 +14,19 @@ ## exist in the user's cache directory, it will download it from the Carbs Linux ## website. If any packages are given as arguments, ## .Nm -## will install those packages to this temporary chroot. +## will install those packages to this temporary chroot. If the +## .Fl r +## flag is given, +## .Nm +## will remove the rootfs tarball and directory to download it again. -case "$1" in - --help|-h) printf '\033[1;33m-> \033[m%s\n' "usage: ${0##*/} [pkg...]"; exit 0 -esac +parser_definition() { + setup REST help:usage -- "usage: ${0##*/} [-r] [pkg...]" + flag redownload -r hidden:1 + global_options silent +} -# shellcheck disable=1091 +# shellcheck source=../src/cpt-lib . cpt-lib die() { @@ -34,34 +41,46 @@ die() { case "$(uname -m)" in i*86) arch=i686; esac url="https://dl.carbslinux.org/releases/${arch:-$(uname -m)}/carbs-rootfs.tar.xz" -cd "$cac_dir" +cd "${cac_dir:?}" + +# Remove the existing tarball and the chroot directory, so that they can be +# downloaded again. +[ "$redownload" ] && as_root rm -rf carbs-rootfs.tar.xz \ + carbs-rootfs.tar.xz.sum \ + carbs-chroot [ -f carbs-rootfs.tar.xz ] || { log "Downloading chroot tarball" - curl -fLO "$url" + pkg_download "$url" } [ -f carbs-rootfs.tar.xz.sum ] || { log "Downloading checksums" - curl -fLo- "${url%/*}/sha256sums.txt" | - grep ' carbs-rootfs.tar.xz$' > carbs-rootfs.tar.xz.sum + pkg_download "$url.sha256" carbs-rootfs.tar.xz.sum } -log "Verifying checksums" -sh256 carbs-rootfs.tar.xz | diff - carbs-rootfs.tar.xz.sum || - die "Checksum verification failed" - +# We don't want to create the rootfs as a non-priviliged user, because there may +# arise certain problems if the files inside the chroot don't belong to root. +[ "$uid" = 0 ] || { + as_root "$0" "$@" + exit $? +} [ -d carbs-chroot ] || { + log "Verifying checksums" + sh256 carbs-rootfs.tar.xz | diff - carbs-rootfs.tar.xz.sum || + die "Checksum verification failed" + log "Extracting chroot" mkdir -p carbs-chroot - ( cd carbs-chroot; tar xf ../carbs-rootfs.tar.xz ) + (cd carbs-chroot; xz -cd ../carbs-rootfs.tar.xz | pax -r) } -create_cache empty +mkdir -p "${tmp_dir:?}" log "Creating temporary chroot" -cp -a carbs-chroot "${chr_dir:=$tmp_dir/chroot}" +mkdir -p "${chr_dir:=$tmp_dir/chroot}" +rsync -a carbs-chroot/ "$chr_dir" [ "$1" ] && { log "Installing extra packages" @@ -71,9 +90,5 @@ cp -a carbs-chroot "${chr_dir:=$tmp_dir/chroot}" run_hook pre-chroot "" "$chr_dir" log "Entering chroot" -if [ "$(id -u)" -eq 0 ]; then - cpt-chroot "$chr_dir" - rm -rf "$chr_dir" -else - as_root sh -c "cpt-chroot $chr_dir; rm -rf $chr_dir" -fi +cpt-chroot "$chr_dir" +rm -rf "$chr_dir" diff --git a/contrib/cpt-chroot b/contrib/cpt-chroot index 2f7eee7..a2ea151 100755 --- a/contrib/cpt-chroot +++ b/contrib/cpt-chroot @@ -1,22 +1,88 @@ #!/bin/sh -e # Enter a chroot +# shellcheck disable=2004 ## SYNOPSIS: ## .Nm cpt-chroot +## .Op Fl m ## .Op Ar dir ## DESCRIPTION: ## .Nm ## is a wrapper script to chroot inside other root filesystems. It automatically ## mounts important filesystems in to the chroot directory, and unmounts them -## when the user exits the chroot and cleans up leftover host files. +## when the user exits the chroot and cleans up leftover host files. If the flag +## .Fl m +## is given, +## .Nm +## does not try to mount or unmount any filesystems. + +# We generate the parser into the script, because we don't want this script to +# depend on cpt-lib. +nomount='' +REST='' +parse() { + OPTIND=$(($#+1)) + while OPTARG= && [ $# -gt 0 ]; do + case $1 in + --?*=*) OPTARG=$1; shift + eval 'set -- "${OPTARG%%\=*}" "${OPTARG#*\=}"' ${1+'"$@"'} + ;; + --no-*) unset OPTARG ;; + -[mh]?*) OPTARG=$1; shift + eval 'set -- "${OPTARG%"${OPTARG#??}"}" -"${OPTARG#??}"' ${1+'"$@"'} + OPTARG= ;; + esac + case $1 in + '-m') + [ "${OPTARG:-}" ] && OPTARG=${OPTARG#*\=} && set "noarg" "$1" && break + eval '[ ${OPTARG+x} ] &&:' && OPTARG='1' || OPTARG='' + nomount="$OPTARG" + ;; + '-h'|'--help') + usage + exit 0 ;; + --) + shift + while [ $# -gt 0 ]; do + REST="${REST} \"\${$(($OPTIND-$#))}\"" + shift + done + break ;; + [-]?*) + set "unknown" "$1"; break ;; + *) + REST="${REST} \"\${$(($OPTIND-$#))}\"" + esac + shift + done + [ $# -eq 0 ] && { OPTIND=1; unset OPTARG; return 0; } + case $1 in + unknown) set "Unrecognized option: $2" "$@" ;; + noarg) set "Does not allow an argument: $2" "$@" ;; + required) set "Requires an argument: $2" "$@" ;; + pattern:*) set "Does not match the pattern (${1#*:}): $2" "$@" ;; + notcmd) set "Not a command: $2" "$@" ;; + *) set "Validation error ($1): $2" "$@" + esac + echo "$1" >&2 + exit 1 +} +usage() { printf '%s\n' "usage: ${0##*/} [-m] [dir]" "" "Options:" \ + " -m Don't mount or unmount directories" +} + +parser_definition() { + setup REST help:usage -- "usage: ${0##*/} [-m] [dir]" + msg -- '' 'Options:' + flag nomount -m -- "Don't mount or unmount directories" + disp :usage -h --help hidden:1 +} log() { printf '\033[32m->\033[m %s.\n' "$*" } -usage() { printf '%s [dir]\n' "${0##*/}"; exit 0;} - die() { log "$*" >&2 exit 1 @@ -35,7 +101,7 @@ clean() { } main() { - case "$1" in ''|--help|-h) usage; esac + parse "$@" && eval set -- "$REST" [ -d "$1" ] || die Given path does not exist [ "$(id -u)" = 0 ] || die Script needs to be run as root @@ -48,17 +114,19 @@ main() { esac } - trap 'clean "$1"' EXIT INT + [ -z "$nomount" ] && { + trap 'clean "$1"' EXIT INT - log Mounting /dev, /proc and /sys from host; { - mountpoint -q "$1/dev" || mount -o bind /dev "$1/dev" - mountpoint -q "$1/proc" || mount -t proc proc "$1/proc" - mountpoint -q "$1/sys" || mount -t sysfs sys "$1/sys" + log Mounting /dev, /proc and /sys from host; { + mountpoint -q "$1/dev" || mount -o bind /dev "$1/dev" + mountpoint -q "$1/proc" || mount -t proc proc "$1/proc" + mountpoint -q "$1/sys" || mount -t sysfs sys "$1/sys" - } + } - log Copying /etc/resolv.conf from host; { - [ -f "$1/etc/resolv.conf" ] || cp /etc/resolv.conf "$1/etc" + log Copying /etc/resolv.conf from host; { + [ -f "$1/etc/resolv.conf" ] || cp /etc/resolv.conf "$1/etc" + } } log Entering chroot; { diff --git a/contrib/cpt-export b/contrib/cpt-export index 300c7c4..478ae27 100755 --- a/contrib/cpt-export +++ b/contrib/cpt-export @@ -12,47 +12,29 @@ ## .Nm ## will use the name of the current directory as the package. -case "$1" in - --help|-h) - printf 'usage: %s [pkg]\n' "${0##*/}" - exit 0 - ;; - '') set -- "${PWD##*/}" -esac - -cpt-list "${1:-null}" >/dev/null - -# Grab the package's version.. -read -r ver rel 2>/dev/null < \ - "$CPT_ROOT/var/db/cpt/installed/$1/version" - ### Environment variables: ### The compression method can be changed while creating a tarball, using the ### .Ev CPT_COMPRESS ### environment variable. -# Fallback to gzip if there is a typo -case "$CPT_COMPRESS" in bz2|gz|xz|zst|lz) ;; *) CPT_COMPRESS=gz; esac +parser_definition() { + setup REST help:usage -- "usage: ${0##*/} [pkg]" + global_options silent +} -# Reset the argument list. -pkg=$1 -tarball="$PWD/$1#$ver-$rel.tar.$CPT_COMPRESS" -set -- +# shellcheck disable=1091 +. cpt-lib -# Construct the argument list using each file. -eval set -- "$(sed '/\/$/d;s|^|".|;s|$|"|' \ - "$CPT_ROOT/var/db/cpt/installed/$pkg/manifest" | tr '\n' ' ')" +[ "$1" ] || set -- "${PWD##*/}" -# Turn the list of files back into a package. -cd "$CPT_ROOT/" -tar cf - -- "$@" | +(pkg_list "$1" >/dev/null) + +# Grab the package's version.. +read -r ver rel 2>/dev/null < "$sys_db/$1/version" -case "$CPT_COMPRESS" in - bz2) bzip2 -z ;; - gz) gzip -6 ;; - xz) xz -zT 0 ;; - zst) zstd -3 ;; - lz) lzip -6 ;; -esac > "$tarball" +tarball="$PWD/$1#$ver-$rel.tar.$CPT_COMPRESS" -printf 'tarball created in %s\n' "$tarball" +# Turn the list of files back into a package. +cd "$CPT_ROOT/" +sed 's/^/./' "$sys_db/$1/manifest" | pax -wd | compress > "$tarball" +out "tarball created in $tarball" diff --git a/contrib/cpt-fork b/contrib/cpt-fork index 21e1618..733c942 100755 --- a/contrib/cpt-fork +++ b/contrib/cpt-fork @@ -31,12 +31,12 @@ for pkg; do case "$pkg" in */*) [ -d "$pkg" ] || die "$pkg is not a directory" - cp -r "$pkg" . + cp -Hr "$pkg" . pkg=${pkg##*/} ;; *) cpt-search "$pkg" >/dev/null - cp -r "$(cpt-search --single "$pkg")" . + cp -Hr "$(cpt-search --single "$pkg")" . esac # Sometimes forked packages are from the database and not from a repository. diff --git a/contrib/cpt-maintainer b/contrib/cpt-maintainer new file mode 100755 index 0000000..34f212f --- /dev/null +++ b/contrib/cpt-maintainer @@ -0,0 +1,49 @@ +#!/bin/sh -e +# Find the maintainer of a package + +## SYNOPSIS: +## .Nm +## .Op Ar pkg... + +## DESCRIPTION: +## .Nm +## finds the maintainer of the given pacage. If no package name is given, +## .Nm +## will use the name of the current directory as the package. + +# shellcheck disable=1091 +. cpt-lib + +usage() { + out "usage: ${0##*/} [pkg...]" + exit +} + +case $1 in + --help|-h) usage ;; + '') set -- "${PWD##*/}" +esac + +for pkgname; do + cpt-search -d "$pkgname" | while read -r pkg; do + # Default to the 'meta' file of the package instead of jumping through + # VCS hoops to find out. + log "$pkg" " " + pkg_query_meta "$pkg" maintainer && continue + + cd "$pkg" + # Use pkg_vcs_info to find out the repository type, but don't save + # repository information to the repository cache file. + repo_type=$(CPT_REPO_CACHE=0 pkg_vcs_info) + repo_type=${repo_type##*:} + + # We use the latest author who made a change to the version file to + # identify the maintainer of a package. + case $repo_type in + git) git log -1 --format='%an <%ae>' version ;; + fossil) fossil time par cur -n 1 -p version -F "%a" | sed \$d ;; + hg) hg log -l1 -T '{user}\n' -- version ;; + *) out "Maintainer information not available" + esac + done +done diff --git a/contrib/cpt-manifest-tree b/contrib/cpt-manifest-tree index 599c0bd..bc38bc2 100755 --- a/contrib/cpt-manifest-tree +++ b/contrib/cpt-manifest-tree @@ -21,4 +21,4 @@ case "$1" in esac cpt-list "$1" >/dev/null printf '%s\n' "[$1]:" -tree -C --fromfile "$CPT_ROOT/var/db/cpt/installed/$1/manifest" | sed 1,2d +tree -C --fromfile "$CPT_ROOT/var/db/cpt/installed/$1/manifest" | sed 1d diff --git a/contrib/cpt-owns b/contrib/cpt-owns index 9bc7d77..1f362d7 100755 --- a/contrib/cpt-owns +++ b/contrib/cpt-owns @@ -40,7 +40,7 @@ case "$1" in esac # Strip 'CPT_ROOT' from the file path if passed and follow symlinks. -file="${1#$CPT_ROOT}" +file="${1#"$CPT_ROOT"}" dirname=$(_readlinkf "$CPT_ROOT/${file%/*}") file="$dirname/${file##*/}" diff --git a/contrib/cpt-reset b/contrib/cpt-reset index 2a1b66e..a2336e3 100755 --- a/contrib/cpt-reset +++ b/contrib/cpt-reset @@ -3,6 +3,7 @@ # # Disable word-splittng warnings as they're safe here. # shellcheck disable=SC2046 +# shellcheck source=../src/cpt-lib ## SYNOPSIS: ## .Nm @@ -11,10 +12,10 @@ ## removes all packages from the system that is not defined as a base package in ## .Pa /etc/cpt-base . -[ "$1" ] && { - printf 'usage: %s\n\nRemove all packages not defined in the base.\n' \ - "${0##*/}" - exit 0 +parser_definition() { + setup REST help:usage -- "usage: ${0##*/}" + global_options compact + msg -- '' "Remove all packages outside of base definition" } . cpt-lib @@ -27,9 +28,14 @@ set +f; for pkg in *; do contains "$base" "$pkg" || set -- "$pkg" "$@" done -[ "$1" ] && { - printf 'WARNING: This will remove \033[1m%s\033[m package(s).\n' "$#" - printf 'Base packages can be redefined in %s\n' "$CPT_ROOT/etc/cpt-base" - printf 'Continue? [Enter/Ctrl+C]\n' - read -r _ && CPT_FORCE=1 cpt-remove "$@" +[ -z "$1" ] && { + log "No package outside of the base definition could be found, not continuing." + exit 0 } + +warn "" "This is a potentially harmful operation, do NOT continue unless" +warn "" "you know exactly what you are doing. Continuing will remove $#" +warn "" "packages that are not listed in the base definition or that the" +warn "" "base packages don't depend on. See the CPT BASE section on the" +warn "" "user manual to learn more." +prompt && cpt-remove "$@" diff --git a/contrib/cpt-revdepends b/contrib/cpt-revdepends index 833dea2..3529a53 100755 --- a/contrib/cpt-revdepends +++ b/contrib/cpt-revdepends @@ -22,7 +22,7 @@ ## .El parser_definition() { - setup REST help:usage -- "usage: ${0##*/} [options...] [pkg...]" + setup REST help:usage -- "usage: ${0##*/} [options...] [pkg]" flag tree -t --tree -- "Also print indirect reverse dependencies" flag make -m --make -- "Include make dependencies" global_options @@ -41,11 +41,11 @@ parser_definition() { cd "$sys_db" get_revdep() { - query="^$1\$" + pkg=$1; set -- "^$pkg\$" # Defined by parser. # shellcheck disable=2154 - [ "$make" ] && query="$query\\|^$1 *make\$" - grep "$query" -- */depends | while read -r pkg _; do + [ "$make" ] && set -- -e "$1" -e "^$pkg " + grep "$@" -- */depends | while read -r pkg _; do printf '%s\n' "${pkg%%/*}" done } @@ -53,7 +53,7 @@ get_revdep() { # Defined by parser. # shellcheck disable=2154 if [ "$tree" ]; then - create_cache nobuild + mkdir -p "$tmp_dir" :> "$tmp_dir/processed" get_revdep "$1" > "$tmp_dir/dep" while ! diff -q "$tmp_dir/dep" "$tmp_dir/processed" >/dev/null 2>&1; do diff --git a/contrib/cpt-size b/contrib/cpt-size index 2812d56..22a77b7 100755 --- a/contrib/cpt-size +++ b/contrib/cpt-size @@ -3,6 +3,7 @@ ## SYNOPSIS: ## .Nm +## .Op Fl st ## .Op Ar pkg... ## DESCRIPTION: @@ -10,41 +11,68 @@ ## calculates the sizes of given ## .Ar packages ## using the files from the package manifest and outputs a total size of the -## packages along with all the files associated with them. - -## CAVEATS: +## packages along with all the files associated with them. If no arguments have +## been given, ## .Nm -## uses the non-POSIX -## .Fl h -## and -## .Fl c -## flags for -## .Xr du 1 , -## which will not work with -## .Em sbase , -## but it is a major performance improvement compared to calculating -## total and human-readable sizes by hand. - -case "$1" in - --help|-h) - printf '%s\n' "usage: ${0##*/} [pkg...]" - exit 0 - ;; - '') set -- "${PWD##*/}" -esac - -for pkg; do cpt-list "$pkg" >/dev/null; done - -files= -for pkg; do - while read -r file; do - # Filter directories from manifest and leave only files. - # Directories in the manifest end in a trailing '/'. - case $file in */) continue; esac - files="$files '$file'" - done < "$CPT_ROOT/var/db/cpt/installed/$pkg/manifest" -done -eval "set -- $files" - -# Send the file list to 'du'. -du -shc -- "$@" 2>/dev/null +## will use the name of the current directory as an argument. +## .Pp +## The options are as follows: +## .Bl -tag -width 13n +## .It Fl s +## Sort the output by size. +## .It Fl t +## Output only the size of given packages and not individual files. +parser_definition() { + setup REST help:usage -- "usage: ${0##*/} [-st] [pkg...]" + flag sort -s hidden:1 + flag total -t hidden:1 + disp :usage -h --help hidden:1 +} + +# shellcheck source=../src/cpt-lib +# shellcheck disable=1091 +. cpt-lib + +# Use the current directory if no arguments have been given. +[ "$1" ] || set -- "${PWD##*/}" + +# Ensure that all the packages given as arguments are installed. +pkg_list "$@" >/dev/null + +mkdir -p "$tmp_dir" + +# We don't immediately pipe into awk as we want to exit in an error. +if [ "$total" ]; then + for pkg; do + sed '/\/$/d;s/./\\&/g' "$sys_db/$pkg/manifest" | + xargs du -k | + awk -v name="$pkg" '{size+=$1}END{printf("%s %s\n", size, name)}' >> "$tmp_dir/size" + done +else + for pkg; do sed '/\/$/d;s/./\\&/g' "$sys_db/$pkg/manifest"; done | + xargs du -k > "$tmp_dir/size" +fi + +# Do a numerical sort on the file if requested. +[ "$sort" ] && sort -no "$tmp_dir/size" "$tmp_dir/size" + +# This awk function formats the `du` output similar to the '-hc' flags. We +# could have used a shell `while read` loop to do the exact same thing, but that +# would be much much slower. +awk 'function fmtsize(s) { + if (s==0) f="" + else if (s<1024) f="K" + else if (s<(1048576)){f="M";s=s/1024;} + else if (s<(1073741824)){f="G";s=s/1048576;} + else f="" + return int(s) f + } + { + sc = $1 + size += $1 + sub(sprintf("^%s\s*", $1), "") + printf("%-6s %s\n", fmtsize(sc), $0) + } + END { + printf("%-6s total\n", fmtsize(size)) + }' "$tmp_dir/size" diff --git a/docs/Makefile b/docs/Makefile index 344bd70..7c5a431 100644 --- a/docs/Makefile +++ b/docs/Makefile @@ -1,9 +1,16 @@ # Carbs Packaging Tools Documentation -include ../config.mk +-include ../config.mk INSTALL_SH = ../tools/install.sh all: cpt.txt cpt.texi cpt.info +config.org: ../config.mk + printf '%s\n' '#+MACRO: version $(VERSION)' \ + '#+MACRO: conf_dir (eval (concat "=$(SYSCONFDIR)/cpt/" $$1 "="))' \ + '#+MACRO: data_dir (eval (concat "=$(DATAROOTDIR)/" $$1 "="))' > config.org + +cpt.txt cpt.texi cpt.info: cpt.org config.org + .SUFFIXES: .info .texi .org .txt .org.texi: rm -f $@ @@ -20,14 +27,6 @@ clean: rm -f cpt.info all-clean: clean - rm -f cpt.texi cpt.txt - -install: all - ${INSTALL_SH} -Dm644 cpt.txt ${DESTDIR}${CPTDOC}/cpt.txt - ${INSTALL_SH} -Dm644 cpt.info ${DESTDIR}${INFODIR}/cpt.info - -uninstall: - rm -rf ${DESTDIR}${CPTDOC} - rm -f ${DESTDIR}${INFODIR}/cpt.info + rm -f cpt.texi cpt.txt config.org .PHONY: all clean all-clean install uninstall diff --git a/docs/cpt.org b/docs/cpt.org index 3949cea..b3fc97b 100644 --- a/docs/cpt.org +++ b/docs/cpt.org @@ -6,10 +6,14 @@ #+TEXINFO_DIR_CATEGORY: Development #+TEXINFO_DIR_TITLE: Carbs Packaging Tools: (cpt) #+TEXINFO_DIR_DESC: Carbs Package Management Library +#+INCLUDE: ./config.org #+OPTIONS: html-scripts:nil todo:nil +#+MACRO: index (eval (format (if (org-export-derived-backend-p org-export-current-backend 'texinfo) "%s Index\n:PROPERTIES:\n:INDEX: %s\n:DESCRIPTION: %ss mentioned in this manual\n:END:\n" "%s%s%s :noexport:\n") $1 $2 $1)) + This is a reference document containing both the user-guide and the development -manual for *Carbs Packaging Tools*. For development logs see [[https://git.carbslinux.org/cpt][the git repository]]. +manual for *Carbs Packaging Tools* version {{{version}}}. For development logs +see [[https://fossil.carbslinux.org/cpt][the fossil repository]]. * Copying :PROPERTIES: @@ -38,6 +42,10 @@ it revolves around the shell library =cpt-lib=, and many tools that wrap around it. This document aims to document both the usage of the distributed tools and document the library functions. +If you happen to find something that is not properly covered by the +documentation, or an area that can be improved, please feel free to submit a +patch, or [[https://fossil.carbslinux.org/cpt/tktnew][open a ticket]]. + * Usage :PROPERTIES: :DESCRIPTION: Basic usage of Carbs Packaging Tools @@ -107,19 +115,32 @@ man cpt-build The package manager does *NOT* have a configuration file, but there are a variety of ways in order to interact with and configure the package manager. -** CPT Base +** Configuration directory + +Some features of the package manager can be configured from the files found +under {{{conf_dir}}}. Even though this doesn't sound like the premise of "no +configuration" files, these files are completely optional to the package +manager, and still the majority of configuration is done through environment +variables. The files on this directory are for configuration that don't have a +big impact on how the package manager behaves, and are not feasible to be used +inside simple environment variables (such as the base package list and +package manager hooks). + +*** CPT Base :PROPERTIES: :DESCRIPTION: Defining base packages :END: -An =/etc/cpt-base= file can be used in order to define the base to the package -manager. Base packages are the packages that receive special treatment by -utilities such as =cpt-reset=, and =cpt-orphans=. +#+CINDEX: Base packages + +The file {{{conf_dir(base)}}} can be used in order to define the base to the +package manager. Base packages are the packages that receive special treatment +by utilities such as =cpt-reset=, and =cpt-orphans=. #+begin_example # This file defines the base packages of the system. You can add or remove # package names in order to redefine the base. This file will be used by -# cpt-orphans and cpt-reset. If this file doesn't exist on /etc/cpt-base, both +# cpt-orphans and cpt-reset. If this file doesn't exist on /etc/cpt/base, both # of the tools will assume that there is no defined base, so use with caution. baselayout binutils @@ -144,6 +165,13 @@ xz zlib #+end_example +*** Systemwide hooks + +A collection of hooks can be installed under {{{conf_dir(hooks/)}}}. All of the +files installed under this directory will then be sourced by the package manager +whenever a hook is called. Some examples for system hooks can be found under the +{{{data_dir(cpt/examples/hooks/)}}} directory. + ** Environment Variables :PROPERTIES: :DESCRIPTION: Change the behaviour of cpt through environment configuration @@ -156,41 +184,105 @@ variables that alter the behaviour of =cpt=, some of them have separate sections to provide detailed information. - ~CPT_PATH~ :: + + #+VINDEX: CPT_PATH Set the locations of your repositories. It is similar to the ~PATH~ variable. + - ~CPT_CACHE~ :: + + #+VINDEX: CPT_CACHE The cache directory for =cpt=. Default: ~$XDG_CACHE_HOME/cpt~. + - ~CPT_CHOICE~ :: + + #+VINDEX: CPT_CHOICE If this is set to 0, a package installation will be aborted on conflicts. + - ~CPT_COLOR~ :: + + #+VINDEX: CPT_COLOR If this is set to 1, =cpt= tools will be forced to display coloured output. If set to 0, they will be forced to display them without colours. Otherwise, =cpt= will output colour as long as it is outputting to a terminal. + - ~CPT_DEBUG~ :: + + #+VINDEX: CPT_DEBUG If set to 1, temporary directories will not be removed after the operation. + +- ~CPT_DOWNLOADER~ :: + + The tool to be used to download package sources. One of =curl=, =wget=, + =wget2=, =axel=, =aria2c=. Defaults to the first one found in that order. + - ~CPT_FETCH~ :: + + #+VINDEX: CPT_FETCH If set to 0, ~cpt-update~ will not fetch repositories. + - ~CPT_FORCE~ :: + + #+VINDEX: CPT_FORCE If set to 1, =cpt= tools will force operation. + - ~CPT_HOOK~ :: + + #+VINDEX: CPT_HOOK Absolute path to the package manager hook file. + - ~CPT_KEEPLOG~ :: + + #+VINDEX: CPT_KEEPLOG If set to 1, =cpt= will keep logs regardless of operation success. + +- ~CPT_NOSTRIP~ :: + + #+VINDEX: CPT_NOSTRIP + If set to 1, =cpt= will not strip debug information from the binaries. Keep in + mind that your compiler already strips most debug information during + compilation, so you also need to add ~-g~ flag to your ~$C{XX}FLAGS~ + - ~CPT_PID~ :: + + #+VINDEX: CPT_PID Set the temporary build directory name. + - ~CPT_PROMPT~ :: + + #+VINDEX: CPT_PROMPT If set to 0, =cpt= will not prompt you for anything. + +- ~CPT_REPO_CACHE~ :: + + #+VINDEX: CPT_REPO_CACHE + If set to 0, =cpt= will not use or write repository information cache. + - ~CPT_ROOT~ :: + + #+VINDEX: CPT_ROOT If this variable is set, =cpt= will assume the given path as the system root. + - ~CPT_TEST~ :: + + #+VINDEX: CPT_TEST If set to 1, ~cpt-build~ will run tests whenever available. + - ~CPT_TMPDIR~ :: + + #+VINDEX: CPT_TMPDIR The directory to create the temporary directories. +- ~CPT_VERBOSE~ :: + + #+VINDEX: CPT_VERBOSE + If this variable is set to 1, the package manager will print more information. + *** =CPT_PATH= :PROPERTIES: :DESCRIPTION: Set the locations of your repositories :END: +#+CINDEX: Setting up repositories Similar to the =PATH= variable, =cpt= find repositories from the =CPT_PATH= variable. Here is an example: @@ -213,6 +305,7 @@ This example brings us to the next section of this document. :DESCRIPTION: Prioritise package repositories :END: +#+CINDEX: package conflicts When you are using multiple repositories from multiple vendors, you will find out that some repositories have the same packages. =cpt= doesn't care about conflicting packages. If you want to build a package that exists on multiple @@ -310,8 +403,10 @@ There are a variety of package hooks, mostly self explanatory: - test-fail :: Run if the ~test~ script fails - pre-install :: Run before a package is installed for each package - post-install :: Run after a package is installed for each package +- end-install :: Run after all given packages are installed - pre-remove :: Run before a package is removed for each package - post-remove :: Run after a package is removed for each package +- end-remove :: Run after all given packages are removed - pre-fetch :: Run before all repositories are fetched - post-fetch :: Run after all repositories are fetched - post-package :: Run after a tarball for a package is created @@ -369,8 +464,8 @@ EOF :DESCRIPTION: More detail on creating packages :END: -A package is formed of several files, from these files, only ~build~, -~checksums~, and ~version~ files are mandatory. +A package is a directory formed of several files, from these files, only +~build~, ~checksums~, and ~version~ files are mandatory. This section talks about files that are interpreted specially by the package manager. Any other file can be added to the package directory at the discretion @@ -378,6 +473,21 @@ of the package maintainer. Everything in the package directory will also be added to the package database that is located on =/var/db/cpt/installed=. These can be patches, configuration files, etc. +Below is a table that provides a small summary for each file, see the relevant +section to learn detailed information on each of them. + +| File | Language | Executable | Mandatory | +|--------------+------------------------------------+------------+-----------| +| build | any | yes | yes | +| checksums | generated by =cpt-checksum= | no | no | +| meta | key-value pairs as in RFC822[fn:1] | no | no[fn:2] | +| depends | custom format | no | no | +| sources | custom format | no | no | +| version | custom format | no | yes | +| message | plaintext | no | no | +| post-install | any | yes | no | +| test | any | yes | no | + ** build :PROPERTIES: :DESCRIPTION: The build script @@ -451,7 +561,7 @@ the sha256 algorithm. ** version :PROPERTIES: -:DESCRIPTION: The file containing the version and the release numbers of a package +:DESCRIPTION: The file containing version information for a package :END: The version file includes the version of the software and the release number of @@ -498,6 +608,9 @@ license: MIT maintainer: Linux User <linux-user@example.com> #+end_example +Even though =meta= is not mandatory by the packaging system, it is a mandatory +file for submitting packages to Carbs Linux repositories. + ** post-install :PROPERTIES: :DESCRIPTION: The post-installation script @@ -520,12 +633,30 @@ installed. :DESCRIPTION: The test script for a package :END: +#+VINDEX: CPT_TEST Test files are mainly for the repository maintainer to test the packages, and will only run if the user has the =CPT_TEST= variable set, or the build is run with the =-t= or =--test= options. This script is run on the build directory. It is run right after the build script is finished. -* Rsync Repositories +* Package Repositories +:PROPERTIES: +:DESCRIPTION: Ways of distributing packages +:END: + +*cpt* has backends to support the use of a variety of distribution methods. You +can currently use Git, Mercurial, Fossil, and Rsync to distribute a package +repository. That, however, does not mean that you need to setup either of those, +if you are simply going for a local repository on your system. + +#+CINDEX: Setting up repositories +In the broad sense, a package repository is any directory that contains packages +that were described in [[Packaging System]]. This means that as long as you can +serve them, there is not much needed to do in order to distribute a repository. +The following subsections aim to detail the notes and the caveats of certain +distribution methods. + +** Rsync Repositories :PROPERTIES: :DESCRIPTION: Information on using or creating rsync repositories :END: @@ -572,7 +703,7 @@ repository and will sync the entire repository instead of each individual reposi If the upper directory doesn't have this =.rsync= file, this means that this is an individual repository, and the package manager will fetch accordingly. -** Setting up an Rsync repository +*** Setting up an Rsync repository :PROPERTIES: :DESCRIPTION: Set up a repository for distribution :END: @@ -626,6 +757,37 @@ Create a service file at =/etc/sv/rsync/run= (runit): exec rsync --daemon --no-detach #+END_SRC +** Fossil repositories +:PROPERTIES: +:DESCRIPTION: Advantages and disadvantages of Fossil +:END: + +Setting up a Fossil repository is no different than setting up any other +repository. There are certainly many advantages of using Fossil as a means of +distributing packages. You can create a Linux distribution and have your +website, forum, documentation, and your package repository entirely contained +inside a single Fossil repository. Fossil's built-in wiki and forum features +make it the ultimate single-tool distribution software. + +However, the biggest caveat of Fossil is that it doesn't allow symlinks by +default unless it's manually set by the user, and this feature cannot even be +set globally. Symbolic links aren't quite common within distribution +repositories, but they come in handy where there are two packages that use the +same source files (=emacs= and =emacs-nox=, or =libelf= and =libdw= from +elfutils). If symbolic links are too big of a deal for your repository, this can +be a huge issue for you. + +** Message of the Day +:PROPERTIES: +:DESCRIPTION: Communicate with the users using your repository +:END: + +If a file named =MOTD= (all uppercase) is found on the root directory of the +package repository, its contents will be printed to the standard output when the +users are updating their repositories. This method can be used to communicate +messages to the users, such as package removals or otherwise important +information. + * Comparison Between CPT and KISS Lots of things have changed since ~cpt~ was forked from ~kiss~ in terms of @@ -649,12 +811,13 @@ this is the ~cpt~ documentation, so it may be biased regardless. - Package Repositories :: - In addition to git repositories, ~cpt~ also makes use of [[Rsync Repositories][rsync repositories]]. + In addition to git repositories, ~cpt~ also supports Rsync, Fossil, and + Mercurial repositories. - Package Sources :: - In addition to git repositories for sources, ~cpt~ also supports mercurial - repositories. + In addition to git repositories for sources, ~cpt~ also supports Mercurial and + Fossil repositories. - Post-Installation Messages :: @@ -697,6 +860,52 @@ files: This will load the library inside your script, and will set some environment variables that are used inside the package manager. +** Variables + +This section lists some of the variables defined by the package manager that can +be used in scripts. These variables usually cannot be defined by the user, so +they are not part of the [[* Environment Variables][variables]] section above. + +#+VINDEX: cpt_version +- =$cpt_version= :: + Package manager version. +#+VINDEX: cpt_confdir +- =$cpt_confdir= :: + Location of the CPT system configuration directory. This is usually either + =/etc/cpt= or =PREFIX/etc/cpt=. +#+VINDEX: pkg_db +- =$pkg_db= :: + Location of the package database without the root (=var/db/cpt/installed=). +#+VINDEX: sys_db +#+VINDEX: CPT_ROOT +- =$sys_db= :: + Location of the package manager database, making use of the current + =$CPT_ROOT= (=$CPT_ROOT/$pkg_db=). This is the database you probably want to + use. +#+CINDEX: Base packages +#+VINDEX: cpt_base +- =$cpt_base= :: + Location of the file that defines the base packages. + +#+CINDEX: Scripts that use CPT cache directories +If for some reason, your script interacts with the directories created and +managed by the package manager you should use the following variables instead of +the user assigned variables such as =$CPT_CACHE= or =$CPT_TMPDIR=. The variables +below are the ones used for package operations (which are assigned by using a +combination of user-assigned values and their fallbacks). + +#+VINDEX: cac_dir +- =$cac_dir= :: + Cache directory used by the package manager. +- =$src_dir= :: + Directory containing downloaded sources for packages. +- =$log_dir= :: + Directory where logs are saved. +- =$bin_dir= :: + Directory where built package tarballs are saved. +- =$tmp_dir= :: + Temporary directory for the package manager operations. + ** Option parsing :PROPERTIES: :DESCRIPTION: Easy way of parsing options with cpt-lib @@ -743,13 +952,21 @@ inside. Here is the proper way of doing it. The =global_options()= function is a simple convenience call to include flags that can be used inside most =cpt= tools. It defines the following flags: -| Flag | Long Option | Calls | -|------+---------------+--------------| -| ~-f~ | ~--force~ | =CPT_FORCE= | -| ~-y~ | ~--no-prompt~ | =CPT_PROMPT= | -| | ~--root~ | =CPT_ROOT= | -| ~-h~ | ~--help~ | =usage()= | -| ~-v~ | ~--version~ | =version()= | +| Flag | Long Option | Calls | +|------+---------------+---------------| +| ~-f~ | ~--force~ | =CPT_FORCE= | +| ~-y~ | ~--no-prompt~ | =CPT_PROMPT= | +| | ~--root~ | =CPT_ROOT= | +| ~-h~ | ~--help~ | =usage()= | +| ~-v~ | ~--version~ | =version()= | +| | ~--verbose~ | =CPT_VERBOSE= | + +This function can take two arguments: + +- =silent= :: If this argument is specified, the function does not print the usage + information defined by its flags. +- =compact= :: If this argument is specified, the function only prints the help + output of the ~--help~ and ~--version~ flags. ** Message functions :PROPERTIES: @@ -1058,7 +1275,7 @@ SEARCH_PATH=$PATH pkg_find 'cpt-*' all -x :DESCRIPTION: List system base packages :END: -This function returns the base packages as defined in =/etc/cpt-base=. If an +This function returns the base packages as defined in the base file. If an optional argument is present, it will print all package names in a single line. If it is not given any arguments, it will return one package per line. See [[CPT Base]] for more information on base packages. @@ -1110,12 +1327,24 @@ rsync curl zlib ca-certificates bearssl This function is used to query the [[meta][meta file]] inside package directories. It can be used to retrieve information on a package that is otherwise irrelevant to the -package manager itself. It takes two arguments, first being the package and the -second being the key to be retrieved. If the package does not have a =meta= -file or the file does not contain the requested key, the function will return -with 1. +package manager itself. It takes two arguments, first being the package (or the +full path to a package directory) and the second being the key to be retrieved. +If the package does not have a =meta= file or the file does not contain the +requested key, the function will return with 1. #+begin_src sh $ pkg_query_meta cpt description Carbs Packaging Tools + +$ pkg_query_meta /path/to/cpt license +MIT #+end_src + +* {{{index(Concept, cp)}}} +* {{{index(Variable,vr)}}} + +* Footnotes + +[fn:1] https://datatracker.ietf.org/doc/html/rfc822#section-3.2 +[fn:2] Not mandatory for the packaging system, but mandatory for inclusion in +the repositories diff --git a/docs/cpt.texi b/docs/cpt.texi index c39b393..e6b331c 100644 --- a/docs/cpt.texi +++ b/docs/cpt.texi @@ -41,7 +41,8 @@ the section entitled "GNU Free Documentation License." @top Carbs Packaging Tools This is a reference document containing both the user-guide and the development -manual for @strong{Carbs Packaging Tools}. For development logs see @uref{https://git.carbslinux.org/cpt, the git repository}. +manual for @strong{Carbs Packaging Tools} version Fossil. For development logs +see @uref{https://fossil.carbslinux.org/cpt, the fossil repository}. @end ifnottex @menu @@ -49,20 +50,27 @@ manual for @strong{Carbs Packaging Tools}. For development logs see @uref{https: * Usage:: Basic usage of Carbs Packaging Tools * Configuration:: Configuring the package manager * Packaging System:: More detail on creating packages -* Rsync Repositories:: Information on using or creating rsync repositories +* Package Repositories:: Ways of distributing packages * Comparison Between CPT and KISS:: * CPT Library:: Documentation of the Library +* Concept Index:: Concepts mentioned in this manual +* Variable Index:: Variables mentioned in this manual @detailmenu --- The Detailed Node Listing --- Configuration -* CPT Base:: Defining base packages +* Configuration directory:: * Environment Variables:: Change the behaviour of cpt through environment configuration * Hooks:: Use hooks to customize the package manager operations * Editing the build file during pre-build:: Modify a package build with your hooks +Configuration directory + +* CPT Base:: Defining base packages +* Systemwide hooks:: + Environment Variables * @samp{CPT_PATH}:: Set the locations of your repositories @@ -75,13 +83,19 @@ Packaging System * build:: The build script * sources:: The file containing package sources * checksums:: The file containing sha256sum of the sources -* version:: The file containing the version and the release numbers of a package +* version:: The file containing version information for a package * depends:: The file containing the dependencies of a package * meta:: File containing more information on packages * post-install:: The post-installation script * message:: The post-installation message to be displayed * test:: The test script for a package +Package Repositories + +* Rsync Repositories:: Information on using or creating rsync repositories +* Fossil repositories:: Advantages and disadvantages of Fossil +* Message of the Day:: Communicate with the users using your repository + Rsync Repositories * Setting up an Rsync repository:: Set up a repository for distribution @@ -89,6 +103,7 @@ Rsync Repositories CPT Library * Calling the library:: Including the library on your code +* Variables:: * Option parsing:: Easy way of parsing options with cpt-lib * Message functions:: Communicate to users * Text functions:: Manipulate or check text @@ -152,6 +167,10 @@ it revolves around the shell library @samp{cpt-lib}, and many tools that wrap ar it. This document aims to document both the usage of the distributed tools and document the library functions. +If you happen to find something that is not properly covered by the +documentation, or an area that can be improved, please feel free to submit a +patch, or @uref{https://fossil.carbslinux.org/cpt/tktnew, open a ticket}. + @node Usage @chapter Usage @@ -218,23 +237,42 @@ The package manager does @strong{NOT} have a configuration file, but there are a variety of ways in order to interact with and configure the package manager. @menu -* CPT Base:: Defining base packages +* Configuration directory:: * Environment Variables:: Change the behaviour of cpt through environment configuration * Hooks:: Use hooks to customize the package manager operations * Editing the build file during pre-build:: Modify a package build with your hooks @end menu +@node Configuration directory +@section Configuration directory + +Some features of the package manager can be configured from the files found +under @samp{/etc/cpt/}. Even though this doesn't sound like the premise of "no +configuration" files, these files are completely optional to the package +manager, and still the majority of configuration is done through environment +variables. The files on this directory are for configuration that don't have a +big impact on how the package manager behaves, and are not feasible to be used +inside simple environment variables (such as the base package list and +package manager hooks). + +@menu +* CPT Base:: Defining base packages +* Systemwide hooks:: +@end menu + @node CPT Base -@section CPT Base +@subsection CPT Base + +@cindex Base packages -An @samp{/etc/cpt-base} file can be used in order to define the base to the package -manager. Base packages are the packages that receive special treatment by -utilities such as @samp{cpt-reset}, and @samp{cpt-orphans}. +The file @samp{/etc/cpt/base} can be used in order to define the base to the +package manager. Base packages are the packages that receive special treatment +by utilities such as @samp{cpt-reset}, and @samp{cpt-orphans}. @example # This file defines the base packages of the system. You can add or remove # package names in order to redefine the base. This file will be used by -# cpt-orphans and cpt-reset. If this file doesn't exist on /etc/cpt-base, both +# cpt-orphans and cpt-reset. If this file doesn't exist on /etc/cpt/base, both # of the tools will assume that there is no defined base, so use with caution. baselayout binutils @@ -259,6 +297,14 @@ xz zlib @end example +@node Systemwide hooks +@subsection Systemwide hooks + +A collection of hooks can be installed under @samp{/etc/cpt/hooks/}. All of the +files installed under this directory will then be sourced by the package manager +whenever a hook is called. Some examples for system hooks can be found under the +@samp{/usr/share/cpt/examples/hooks/} directory. + @node Environment Variables @section Environment Variables @@ -270,35 +316,80 @@ to provide detailed information. @table @asis @item @code{CPT_PATH} +@vindex CPT_PATH Set the locations of your repositories. It is similar to the @code{PATH} variable. + @item @code{CPT_CACHE} +@vindex CPT_CACHE The cache directory for @samp{cpt}. Default: @code{$XDG_CACHE_HOME/cpt}. + @item @code{CPT_CHOICE} +@vindex CPT_CHOICE If this is set to 0, a package installation will be aborted on conflicts. + @item @code{CPT_COLOR} +@vindex CPT_COLOR If this is set to 1, @samp{cpt} tools will be forced to display coloured output. If set to 0, they will be forced to display them without colours. Otherwise, @samp{cpt} will output colour as long as it is outputting to a terminal. + @item @code{CPT_DEBUG} +@vindex CPT_DEBUG If set to 1, temporary directories will not be removed after the operation. + +@item @code{CPT_DOWNLOADER} +The tool to be used to download package sources. One of @samp{curl}, @samp{wget}, +@samp{wget2}, @samp{axel}, @samp{aria2c}. Defaults to the first one found in that order. + @item @code{CPT_FETCH} +@vindex CPT_FETCH If set to 0, @code{cpt-update} will not fetch repositories. + @item @code{CPT_FORCE} +@vindex CPT_FORCE If set to 1, @samp{cpt} tools will force operation. + @item @code{CPT_HOOK} +@vindex CPT_HOOK Absolute path to the package manager hook file. + @item @code{CPT_KEEPLOG} +@vindex CPT_KEEPLOG If set to 1, @samp{cpt} will keep logs regardless of operation success. + +@item @code{CPT_NOSTRIP} +@vindex CPT_NOSTRIP +If set to 1, @samp{cpt} will not strip debug information from the binaries. Keep in +mind that your compiler already strips most debug information during +compilation, so you also need to add @code{-g} flag to your @code{$C@{XX@}FLAGS} + @item @code{CPT_PID} +@vindex CPT_PID Set the temporary build directory name. + @item @code{CPT_PROMPT} +@vindex CPT_PROMPT If set to 0, @samp{cpt} will not prompt you for anything. + +@item @code{CPT_REPO_CACHE} +@vindex CPT_REPO_CACHE +If set to 0, @samp{cpt} will not use or write repository information cache. + @item @code{CPT_ROOT} +@vindex CPT_ROOT If this variable is set, @samp{cpt} will assume the given path as the system root. + @item @code{CPT_TEST} +@vindex CPT_TEST If set to 1, @code{cpt-build} will run tests whenever available. + @item @code{CPT_TMPDIR} +@vindex CPT_TMPDIR The directory to create the temporary directories. + +@item @code{CPT_VERBOSE} +@vindex CPT_VERBOSE +If this variable is set to 1, the package manager will print more information. @end table @menu @@ -311,6 +402,7 @@ The directory to create the temporary directories. @node @samp{CPT_PATH} @subsection @samp{CPT_PATH} +@cindex Setting up repositories Similar to the @samp{PATH} variable, @samp{cpt} find repositories from the @samp{CPT_PATH} variable. Here is an example: @@ -330,9 +422,10 @@ This example brings us to the next section of this document. @enumerate @item -Repository preferences +@anchor{Repository preferences}Repository preferences +@cindex package conflicts When you are using multiple repositories from multiple vendors, you will find out that some repositories have the same packages. @samp{cpt} doesn't care about conflicting packages. If you want to build a package that exists on multiple @@ -347,7 +440,7 @@ CPT_PATH=$HOME/repos/personal:$HOME/repos/carbs/extra @end example @item -Setting the @samp{CPT_PATH} +@anchor{Setting the @samp{CPT_PATH}}Setting the @samp{CPT_PATH} You can set the @samp{CPT_PATH} variable on your shell configuration or your @@ -442,10 +535,14 @@ Run if the @code{test} script fails Run before a package is installed for each package @item post-install Run after a package is installed for each package +@item end-install +Run after all given packages are installed @item pre-remove Run before a package is removed for each package @item post-remove Run after a package is removed for each package +@item end-remove +Run after all given packages are removed @item pre-fetch Run before all repositories are fetched @item post-fetch @@ -505,8 +602,8 @@ EOF @node Packaging System @chapter Packaging System -A package is formed of several files, from these files, only @code{build}, -@code{checksums}, and @code{version} files are mandatory. +A package is a directory formed of several files, from these files, only +@code{build}, @code{checksums}, and @code{version} files are mandatory. This section talks about files that are interpreted specially by the package manager. Any other file can be added to the package directory at the discretion @@ -514,11 +611,58 @@ of the package maintainer. Everything in the package directory will also be added to the package database that is located on @samp{/var/db/cpt/installed}. These can be patches, configuration files, etc. +Below is a table that provides a small summary for each file, see the relevant +section to learn detailed information on each of them. + +@multitable {aaaaaaaaaaaa} {aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa} {aaaaaaaaaa} {aaaaaaaaa} +@headitem File +@tab Language +@tab Executable +@tab Mandatory +@item build +@tab any +@tab yes +@tab yes +@item checksums +@tab generated by @samp{cpt-checksum} +@tab no +@tab no +@item meta +@tab key-value pairs as in RFC822@footnote{@uref{https://datatracker.ietf.org/doc/html/rfc822#section-3.2}} +@tab no +@tab no@footnote{Not mandatory for the packaging system, but mandatory for inclusion in +the repositories} +@item depends +@tab custom format +@tab no +@tab no +@item sources +@tab custom format +@tab no +@tab no +@item version +@tab custom format +@tab no +@tab yes +@item message +@tab plaintext +@tab no +@tab no +@item post-install +@tab any +@tab yes +@tab no +@item test +@tab any +@tab yes +@tab no +@end multitable + @menu * build:: The build script * sources:: The file containing package sources * checksums:: The file containing sha256sum of the sources -* version:: The file containing the version and the release numbers of a package +* version:: The file containing version information for a package * depends:: The file containing the dependencies of a package * meta:: File containing more information on packages * post-install:: The post-installation script @@ -644,6 +788,9 @@ license: MIT maintainer: Linux User <linux-user@@example.com> @end example +Even though @samp{meta} is not mandatory by the packaging system, it is a mandatory +file for submitting packages to Carbs Linux repositories. + @node post-install @section post-install @@ -660,13 +807,35 @@ installed. @node test @section test +@vindex CPT_TEST Test files are mainly for the repository maintainer to test the packages, and will only run if the user has the @samp{CPT_TEST} variable set, or the build is run with the @samp{-t} or @samp{--test} options. This script is run on the build directory. It is run right after the build script is finished. +@node Package Repositories +@chapter Package Repositories + +@strong{cpt} has backends to support the use of a variety of distribution methods. You +can currently use Git, Mercurial, Fossil, and Rsync to distribute a package +repository. That, however, does not mean that you need to setup either of those, +if you are simply going for a local repository on your system. + +@cindex Setting up repositories +In the broad sense, a package repository is any directory that contains packages +that were described in @ref{Packaging System}. This means that as long as you can +serve them, there is not much needed to do in order to distribute a repository. +The following subsections aim to detail the notes and the caveats of certain +distribution methods. + +@menu +* Rsync Repositories:: Information on using or creating rsync repositories +* Fossil repositories:: Advantages and disadvantages of Fossil +* Message of the Day:: Communicate with the users using your repository +@end menu + @node Rsync Repositories -@chapter Rsync Repositories +@section Rsync Repositories Rsync repositories are simple to serve and simple to use. In the repository directory, there needs to be a @samp{.rsync} file that points to the remote of the @@ -715,7 +884,7 @@ is an individual repository, and the package manager will fetch accordingly. @end menu @node Setting up an Rsync repository -@section Setting up an Rsync repository +@subsection Setting up an Rsync repository Carbs Linux repositories automatically sync from the git repostitories and serve it through the rsync daemon. Here is a sample shell script that I use in order to @@ -766,6 +935,33 @@ Create a service file at @samp{/etc/sv/rsync/run} (runit): exec rsync --daemon --no-detach @end example +@node Fossil repositories +@section Fossil repositories + +Setting up a Fossil repository is no different than setting up any other +repository. There are certainly many advantages of using Fossil as a means of +distributing packages. You can create a Linux distribution and have your +website, forum, documentation, and your package repository entirely contained +inside a single Fossil repository. Fossil's built-in wiki and forum features +make it the ultimate single-tool distribution software. + +However, the biggest caveat of Fossil is that it doesn't allow symlinks by +default unless it's manually set by the user, and this feature cannot even be +set globally. Symbolic links aren't quite common within distribution +repositories, but they come in handy where there are two packages that use the +same source files (@samp{emacs} and @samp{emacs-nox}, or @samp{libelf} and @samp{libdw} from +elfutils). If symbolic links are too big of a deal for your repository, this can +be a huge issue for you. + +@node Message of the Day +@section Message of the Day + +If a file named @samp{MOTD} (all uppercase) is found on the root directory of the +package repository, its contents will be printed to the standard output when the +users are updating their repositories. This method can be used to communicate +messages to the users, such as package removals or otherwise important +information. + @node Comparison Between CPT and KISS @chapter Comparison Between CPT and KISS @@ -788,11 +984,12 @@ through environment variables. Additionally, all @code{cpt} tools can receive fl that alter their functionality. @code{kiss} does not accept flags. @item Package Repositories -In addition to git repositories, @code{cpt} also makes use of @ref{Rsync Repositories, , rsync repositories}. +In addition to git repositories, @code{cpt} also supports Rsync, Fossil, and +Mercurial repositories. @item Package Sources -In addition to git repositories for sources, @code{cpt} also supports mercurial -repositories. +In addition to git repositories for sources, @code{cpt} also supports Mercurial and +Fossil repositories. @item Post-Installation Messages @code{kiss} and @code{cpt} interact with @samp{post-install} messages differently. @code{kiss} @@ -818,6 +1015,7 @@ package manager library. @menu * Calling the library:: Including the library on your code +* Variables:: * Option parsing:: Easy way of parsing options with cpt-lib * Message functions:: Communicate to users * Text functions:: Manipulate or check text @@ -840,6 +1038,65 @@ files: This will load the library inside your script, and will set some environment variables that are used inside the package manager. +@node Variables +@section Variables + +This section lists some of the variables defined by the package manager that can +be used in scripts. These variables usually cannot be defined by the user, so +they are not part of the @ref{Environment Variables, , variables} section above. + +@vindex cpt_version +@table @asis +@item @samp{$cpt_version} +Package manager version. +@end table +@vindex cpt_confdir +@table @asis +@item @samp{$cpt_confdir} +Location of the CPT system configuration directory. This is usually either +@samp{/etc/cpt} or @samp{PREFIX/etc/cpt}. +@end table +@vindex pkg_db +@table @asis +@item @samp{$pkg_db} +Location of the package database without the root (@samp{var/db/cpt/installed}). +@end table +@vindex sys_db +@vindex CPT_ROOT +@table @asis +@item @samp{$sys_db} +Location of the package manager database, making use of the current +@samp{$CPT_ROOT} (@samp{$CPT_ROOT/$pkg_db}). This is the database you probably want to +use. +@end table +@cindex Base packages +@vindex cpt_base +@table @asis +@item @samp{$cpt_base} +Location of the file that defines the base packages. +@end table + +@cindex Scripts that use CPT cache directories +If for some reason, your script interacts with the directories created and +managed by the package manager you should use the following variables instead of +the user assigned variables such as @samp{$CPT_CACHE} or @samp{$CPT_TMPDIR}. The variables +below are the ones used for package operations (which are assigned by using a +combination of user-assigned values and their fallbacks). + +@vindex cac_dir +@table @asis +@item @samp{$cac_dir} +Cache directory used by the package manager. +@item @samp{$src_dir} +Directory containing downloaded sources for packages. +@item @samp{$log_dir} +Directory where logs are saved. +@item @samp{$bin_dir} +Directory where built package tarballs are saved. +@item @samp{$tmp_dir} +Temporary directory for the package manager operations. +@end table + @node Option parsing @section Option parsing @@ -885,7 +1142,7 @@ parser_definition() @{ The @samp{global_options()} function is a simple convenience call to include flags that can be used inside most @samp{cpt} tools. It defines the following flags: -@multitable {aaaa} {aaaaaaaaaaaaa} {aaaaaaaaaaaa} +@multitable {aaaa} {aaaaaaaaaaaaa} {aaaaaaaaaaaaa} @headitem Flag @tab Long Option @tab Calls @@ -904,8 +1161,22 @@ that can be used inside most @samp{cpt} tools. It defines the following flags: @item @code{-v} @tab @code{--version} @tab @samp{version()} +@item +@tab @code{--verbose} +@tab @samp{CPT_VERBOSE} @end multitable +This function can take two arguments: + +@table @asis +@item @samp{silent} +If this argument is specified, the function does not print the usage +information defined by its flags. +@item @samp{compact} +If this argument is specified, the function only prints the help +output of the @code{--help} and @code{--version} flags. +@end table + @node Message functions @section Message functions @@ -1224,7 +1495,7 @@ SEARCH_PATH=$PATH pkg_find 'cpt-*' all -x @node @samp{pkg_get_base()} @subsection @samp{pkg_get_base()} -This function returns the base packages as defined in @samp{/etc/cpt-base}. If an +This function returns the base packages as defined in the base file. If an optional argument is present, it will print all package names in a single line. If it is not given any arguments, it will return one package per line. See @ref{CPT Base} for more information on base packages. @@ -1250,7 +1521,7 @@ Print all packages in a single line instead of a package per line. @enumerate @item -Examples +@anchor{Examples}Examples This example uses the @samp{cpt} package for Carbs Linux. The package itself is @@ -1282,14 +1553,27 @@ rsync curl zlib ca-certificates bearssl This function is used to query the @ref{meta, , meta file} inside package directories. It can be used to retrieve information on a package that is otherwise irrelevant to the -package manager itself. It takes two arguments, first being the package and the -second being the key to be retrieved. If the package does not have a @samp{meta} -file or the file does not contain the requested key, the function will return -with 1. +package manager itself. It takes two arguments, first being the package (or the +full path to a package directory) and the second being the key to be retrieved. +If the package does not have a @samp{meta} file or the file does not contain the +requested key, the function will return with 1. @example $ pkg_query_meta cpt description Carbs Packaging Tools + +$ pkg_query_meta /path/to/cpt license +MIT @end example +@node Concept Index +@chapter Concept Index + +@printindex cp + +@node Variable Index +@chapter Variable Index + +@printindex vr + @bye
\ No newline at end of file diff --git a/docs/cpt.txt b/docs/cpt.txt index c04a52f..08433e9 100644 --- a/docs/cpt.txt +++ b/docs/cpt.txt @@ -1,10 +1,10 @@ - _______________________ + _______________________ - CARBS PACKAGING TOOLS - User Manual + CARBS PACKAGING TOOLS + User Manual - Cem Keylan - _______________________ + Cem Keylan + _______________________ Table of Contents @@ -14,7 +14,9 @@ _________________ 2. Preface 3. Usage 4. Configuration -.. 1. CPT Base +.. 1. Configuration directory +..... 1. CPT Base +..... 2. Systemwide hooks .. 2. Environment Variables ..... 1. `CPT_PATH' ..... 2. `CPT_COMPRESS' @@ -32,32 +34,36 @@ _________________ .. 7. post-install .. 8. message .. 9. test -6. Rsync Repositories -.. 1. Setting up an Rsync repository +6. Package Repositories +.. 1. Rsync Repositories +..... 1. Setting up an Rsync repository +.. 2. Fossil repositories +.. 3. Message of the Day 7. Comparison Between CPT and KISS 8. CPT Library .. 1. Calling the library -.. 2. Option parsing +.. 2. Variables +.. 3. Option parsing ..... 1. Defining a parser ..... 2. `global_options()' -.. 3. Message functions +.. 4. Message functions ..... 1. `out()' ..... 2. `log()' ..... 3. `die()' ..... 4. `warn()' ..... 5. `prompt()' -.. 4. Text functions +.. 5. Text functions ..... 1. `contains()' ..... 2. `regesc()' ..... 3. `pop()' ..... 4. `sepchar()' -.. 5. Portability functions +.. 6. Portability functions ..... 1. `_seq()' ..... 2. `_stat()' ..... 3. `_readlinkf()' -.. 6. System Functions +.. 7. System Functions ..... 1. `as_root()' -.. 7. Package Functions +.. 8. Package Functions ..... 1. `pkg_build()' ..... 2. `pkg_depends()' ..... 3. `pkg_order()' @@ -71,11 +77,11 @@ _________________ This is a reference document containing both the user-guide and the -development manual for *Carbs Packaging Tools*. For development logs see -[the git repository]. +development manual for *Carbs Packaging Tools* version Fossil. For +development logs see [the fossil repository]. -[the git repository] <https://git.carbslinux.org/cpt> +[the fossil repository] <https://fossil.carbslinux.org/cpt> 1 Copying @@ -103,9 +109,15 @@ development manual for *Carbs Packaging Tools*. For development logs see aims to document both the usage of the distributed tools and document the library functions. + If you happen to find something that is not properly covered by the + documentation, or an area that can be improved, please feel free to + submit a patch, or [open a ticket]. + [kiss] <https://github.com/kisslinux/kiss> +[open a ticket] <https://fossil.carbslinux.org/cpt/tktnew> + 3 Usage ======= @@ -175,17 +187,31 @@ development manual for *Carbs Packaging Tools*. For development logs see package manager. -4.1 CPT Base -~~~~~~~~~~~~ +4.1 Configuration directory +~~~~~~~~~~~~~~~~~~~~~~~~~~~ - An `/etc/cpt-base' file can be used in order to define the base to the - package manager. Base packages are the packages that receive special - treatment by utilities such as `cpt-reset', and `cpt-orphans'. + Some features of the package manager can be configured from the files + found under `/etc/cpt/'. Even though this doesn't sound like the + premise of "no configuration" files, these files are completely + optional to the package manager, and still the majority of + configuration is done through environment variables. The files on this + directory are for configuration that don't have a big impact on how + the package manager behaves, and are not feasible to be used inside + simple environment variables (such as the base package list and + package manager hooks). + + +4.1.1 CPT Base +-------------- + + The file `/etc/cpt/base' can be used in order to define the base to + the package manager. Base packages are the packages that receive + special treatment by utilities such as `cpt-reset', and `cpt-orphans'. ,---- | # This file defines the base packages of the system. You can add or remove | # package names in order to redefine the base. This file will be used by - | # cpt-orphans and cpt-reset. If this file doesn't exist on /etc/cpt-base, both + | # cpt-orphans and cpt-reset. If this file doesn't exist on /etc/cpt/base, both | # of the tools will assume that there is no defined base, so use with caution. | baselayout | binutils @@ -211,6 +237,16 @@ development manual for *Carbs Packaging Tools*. For development logs see `---- +4.1.2 Systemwide hooks +---------------------- + + A collection of hooks can be installed under `/etc/cpt/hooks/'. All of + the files installed under this directory will then be sourced by the + package manager whenever a hook is called. Some examples for system + hooks can be found under the `/usr/share/cpt/examples/hooks/' + directory. + + 4.2 Environment Variables ~~~~~~~~~~~~~~~~~~~~~~~~~ @@ -223,40 +259,72 @@ development manual for *Carbs Packaging Tools*. For development logs see `CPT_PATH' Set the locations of your repositories. It is similar to the `PATH' variable. + `CPT_CACHE' The cache directory for `cpt'. Default: `$XDG_CACHE_HOME/cpt'. + `CPT_CHOICE' If this is set to 0, a package installation will be aborted on conflicts. + `CPT_COLOR' If this is set to 1, `cpt' tools will be forced to display coloured output. If set to 0, they will be forced to display them without colours. Otherwise, `cpt' will output colour as long as it is outputting to a terminal. + `CPT_DEBUG' If set to 1, temporary directories will not be removed after the operation. + + `CPT_DOWNLOADER' + The tool to be used to download package sources. One of `curl', + `wget', `wget2', `axel', `aria2c'. Defaults to the first one + found in that order. + `CPT_FETCH' If set to 0, `cpt-update' will not fetch repositories. + `CPT_FORCE' If set to 1, `cpt' tools will force operation. + `CPT_HOOK' Absolute path to the package manager hook file. + `CPT_KEEPLOG' If set to 1, `cpt' will keep logs regardless of operation success. + + `CPT_NOSTRIP' + If set to 1, `cpt' will not strip debug information from the + binaries. Keep in mind that your compiler already strips most + debug information during compilation, so you also need to add + `-g' flag to your `$C{XX}FLAGS' + `CPT_PID' Set the temporary build directory name. + `CPT_PROMPT' If set to 0, `cpt' will not prompt you for anything. + + `CPT_REPO_CACHE' + If set to 0, `cpt' will not use or write repository information + cache. + `CPT_ROOT' If this variable is set, `cpt' will assume the given path as the system root. + `CPT_TEST' If set to 1, `cpt-build' will run tests whenever available. + `CPT_TMPDIR' The directory to create the temporary directories. + `CPT_VERBOSE' + If this variable is set to 1, the package manager will print + more information. + 4.2.1 `CPT_PATH' ---------------- @@ -383,10 +451,14 @@ development manual for *Carbs Packaging Tools*. For development logs see Run before a package is installed for each package post-install Run after a package is installed for each package + end-install + Run after all given packages are installed pre-remove Run before a package is removed for each package post-remove Run after a package is removed for each package + end-remove + Run after all given packages are removed pre-fetch Run before all repositories are fetched post-fetch @@ -446,8 +518,8 @@ development manual for *Carbs Packaging Tools*. For development logs see 5 Packaging System ================== - A package is formed of several files, from these files, only `build', - `checksums', and `version' files are mandatory. + A package is a directory formed of several files, from these files, + only `build', `checksums', and `version' files are mandatory. This section talks about files that are interpreted specially by the package manager. Any other file can be added to the package directory @@ -456,6 +528,21 @@ development manual for *Carbs Packaging Tools*. For development logs see on `/var/db/cpt/installed'. These can be patches, configuration files, etc. + Below is a table that provides a small summary for each file, see the + relevant section to learn detailed information on each of them. + + File Language Executable Mandatory + ---------------------------------------------------------------------- + build any yes yes + checksums generated by `cpt-checksum' no no + meta key-value pairs as in RFC822[1] no no[2] + depends custom format no no + sources custom format no no + version custom format no yes + message plaintext no no + post-install any yes no + test any yes no + 5.1 build ~~~~~~~~~ @@ -579,8 +666,11 @@ development manual for *Carbs Packaging Tools*. For development logs see | maintainer: Linux User <linux-user@example.com> `---- + Even though `meta' is not mandatory by the packaging system, it is a + mandatory file for submitting packages to Carbs Linux repositories. -[pkg_query_meta()] See section 8.7.10 + +[pkg_query_meta()] See section 8.8.10 5.7 post-install @@ -608,8 +698,24 @@ development manual for *Carbs Packaging Tools*. For development logs see script is finished. -6 Rsync Repositories -==================== +6 Package Repositories +====================== + + *cpt* has backends to support the use of a variety of distribution + methods. You can currently use Git, Mercurial, Fossil, and Rsync to + distribute a package repository. That, however, does not mean that you + need to setup either of those, if you are simply going for a local + repository on your system. + + In the broad sense, a package repository is any directory that + contains packages that were described in 5. This means that as long as + you can serve them, there is not much needed to do in order to + distribute a repository. The following subsections aim to detail the + notes and the caveats of certain distribution methods. + + +6.1 Rsync Repositories +~~~~~~~~~~~~~~~~~~~~~~ Rsync repositories are simple to serve and simple to use. In the repository directory, there needs to be a `.rsync' file that points to @@ -658,8 +764,8 @@ development manual for *Carbs Packaging Tools*. For development logs see fetch accordingly. -6.1 Setting up an Rsync repository -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +6.1.1 Setting up an Rsync repository +------------------------------------ Carbs Linux repositories automatically sync from the git repostitories and serve it through the rsync daemon. Here is a sample shell script @@ -712,6 +818,37 @@ development manual for *Carbs Packaging Tools*. For development logs see `---- +6.2 Fossil repositories +~~~~~~~~~~~~~~~~~~~~~~~ + + Setting up a Fossil repository is no different than setting up any + other repository. There are certainly many advantages of using Fossil + as a means of distributing packages. You can create a Linux + distribution and have your website, forum, documentation, and your + package repository entirely contained inside a single Fossil + repository. Fossil's built-in wiki and forum features make it the + ultimate single-tool distribution software. + + However, the biggest caveat of Fossil is that it doesn't allow + symlinks by default unless it's manually set by the user, and this + feature cannot even be set globally. Symbolic links aren't quite + common within distribution repositories, but they come in handy where + there are two packages that use the same source files (`emacs' and + `emacs-nox', or `libelf' and `libdw' from elfutils). If symbolic links + are too big of a deal for your repository, this can be a huge issue + for you. + + +6.3 Message of the Day +~~~~~~~~~~~~~~~~~~~~~~ + + If a file named `MOTD' (all uppercase) is found on the root directory + of the package repository, its contents will be printed to the + standard output when the users are updating their repositories. This + method can be used to communicate messages to the users, such as + package removals or otherwise important information. + + 7 Comparison Between CPT and KISS ================================= @@ -736,12 +873,12 @@ development manual for *Carbs Packaging Tools*. For development logs see functionality. `kiss' does not accept flags. Package Repositories - In addition to git repositories, `cpt' also makes use of [rsync - repositories]. + In addition to git repositories, `cpt' also supports Rsync, + Fossil, and Mercurial repositories. Package Sources In addition to git repositories for sources, `cpt' also supports - mercurial repositories. + Mercurial and Fossil repositories. Post-Installation Messages `kiss' and `cpt' interact with `post-install' messages @@ -761,9 +898,6 @@ development manual for *Carbs Packaging Tools*. For development logs see favour of portability. -[rsync repositories] See section 6 - - 8 CPT Library ============= @@ -787,7 +921,52 @@ development manual for *Carbs Packaging Tools*. For development logs see environment variables that are used inside the package manager. -8.2 Option parsing +8.2 Variables +~~~~~~~~~~~~~ + + This section lists some of the variables defined by the package + manager that can be used in scripts. These variables usually cannot be + defined by the user, so they are not part of the [variables] section + above. + + `$cpt_version' + Package manager version. + `$cpt_confdir' + Location of the CPT system configuration directory. This is + usually either `/etc/cpt' or `PREFIX/etc/cpt'. + `$pkg_db' + Location of the package database without the root + (`var/db/cpt/installed'). + `$sys_db' + Location of the package manager database, making use of the + current `$CPT_ROOT' (`$CPT_ROOT/$pkg_db'). This is the database + you probably want to use. + `$cpt_base' + Location of the file that defines the base packages. + + If for some reason, your script interacts with the directories created + and managed by the package manager you should use the following + variables instead of the user assigned variables such as `$CPT_CACHE' + or `$CPT_TMPDIR'. The variables below are the ones used for package + operations (which are assigned by using a combination of user-assigned + values and their fallbacks). + + `$cac_dir' + Cache directory used by the package manager. + `$src_dir' + Directory containing downloaded sources for packages. + `$log_dir' + Directory where logs are saved. + `$bin_dir' + Directory where built package tarballs are saved. + `$tmp_dir' + Temporary directory for the package manager operations. + + +[variables] See section 4.2 + + +8.3 Option parsing ~~~~~~~~~~~~~~~~~~ `cpt-lib' includes a POSIX-shell option parser inside named @@ -799,7 +978,7 @@ development manual for *Carbs Packaging Tools*. For development logs see [documentation] <https://github.com/ko1nksm/getoptions/blob/v2.5.0/README.md> -8.2.1 Defining a parser +8.3.1 Defining a parser ----------------------- Some functions are called and set automatically when you call @@ -826,29 +1005,39 @@ development manual for *Carbs Packaging Tools*. For development logs see `---- -8.2.2 `global_options()' +8.3.2 `global_options()' ------------------------ The `global_options()' function is a simple convenience call to include flags that can be used inside most `cpt' tools. It defines the following flags: - Flag Long Option Calls - ----------------------------------- - `-f' `--force' `CPT_FORCE' - `-y' `--no-prompt' `CPT_PROMPT' - `--root' `CPT_ROOT' - `-h' `--help' `usage()' - `-v' `--version' `version()' + Flag Long Option Calls + ------------------------------------ + `-f' `--force' `CPT_FORCE' + `-y' `--no-prompt' `CPT_PROMPT' + `--root' `CPT_ROOT' + `-h' `--help' `usage()' + `-v' `--version' `version()' + `--verbose' `CPT_VERBOSE' + + This function can take two arguments: + + `silent' + If this argument is specified, the function does not print the + usage information defined by its flags. + `compact' + If this argument is specified, the function only prints the help + output of the `--help' and `--version' flags. -8.3 Message functions +8.4 Message functions ~~~~~~~~~~~~~~~~~~~~~ `cpt' has various functions to print information to users. -8.3.1 `out()' +8.4.1 `out()' ------------- `out()' is a really simple function that prints messages to the @@ -863,7 +1052,7 @@ development manual for *Carbs Packaging Tools*. For development logs see `---- -8.3.2 `log()' +8.4.2 `log()' ------------- `log()' is the most commonly used message function in the package @@ -881,7 +1070,7 @@ development manual for *Carbs Packaging Tools*. For development logs see above. -8.3.3 `die()' +8.4.3 `die()' ------------- `die()' wraps the `log()' function and exits with an error (1). It @@ -889,14 +1078,14 @@ development manual for *Carbs Packaging Tools*. For development logs see function. The third argument for `log()' is set as `!>'. -8.3.4 `warn()' +8.4.4 `warn()' -------------- `warn()' is another function that wraps `log()'. In place of the third argument, it uses the word `WARNING'. -8.3.5 `prompt()' +8.4.5 `prompt()' ---------------- `prompt()' is an interactive function that waits for user input to @@ -906,14 +1095,14 @@ development manual for *Carbs Packaging Tools*. For development logs see `CPT_PROMPT' to 0. -8.4 Text functions +8.5 Text functions ~~~~~~~~~~~~~~~~~~ Following functions are used to manipulate, check, or interact with text. -8.4.1 `contains()' +8.5.1 `contains()' ------------------ `contains' function can be used to check whether a list variable @@ -929,7 +1118,7 @@ development manual for *Carbs Packaging Tools*. For development logs see `---- -8.4.2 `regesc()' +8.5.2 `regesc()' ---------------- `regesc()' can be used to escape regular expression characters that @@ -941,7 +1130,7 @@ development manual for *Carbs Packaging Tools*. For development logs see `---- -8.4.3 `pop()' +8.5.3 `pop()' ------------- `pop()' can be used to remove a word from a "string list" without a @@ -955,7 +1144,7 @@ development manual for *Carbs Packaging Tools*. For development logs see `---- -8.4.4 `sepchar()' +8.5.4 `sepchar()' ----------------- This function can be used to separate characters from the given string @@ -975,7 +1164,7 @@ development manual for *Carbs Packaging Tools*. For development logs see `---- -8.5 Portability functions +8.6 Portability functions ~~~~~~~~~~~~~~~~~~~~~~~~~ These helper functions are used so that we don't depend on non-POSIX @@ -983,7 +1172,7 @@ development manual for *Carbs Packaging Tools*. For development logs see character. -8.5.1 `_seq()' +8.6.1 `_seq()' -------------- This function is similar to `seq(1)' except that it only takes a @@ -997,7 +1186,7 @@ development manual for *Carbs Packaging Tools*. For development logs see `---- -8.5.2 `_stat()' +8.6.2 `_stat()' --------------- This function imitates `stat %U'. `stat' isn't defined by POSIX, and @@ -1005,7 +1194,7 @@ development manual for *Carbs Packaging Tools*. For development logs see file. If the owner cannot be found, it will return `root'. -8.5.3 `_readlinkf()' +8.6.3 `_readlinkf()' -------------------- This function was taken from [POSIX sh readlinkf library by Koichi @@ -1017,10 +1206,10 @@ development manual for *Carbs Packaging Tools*. For development logs see <https://github.com/ko1nksm/readlinkf> -8.6 System Functions +8.7 System Functions ~~~~~~~~~~~~~~~~~~~~ -8.6.1 `as_root()' +8.7.1 `as_root()' ----------------- `as_root()' calls the rest of the arguments as a different @@ -1037,7 +1226,7 @@ development manual for *Carbs Packaging Tools*. For development logs see `$CPT_SU' variable. -8.7 Package Functions +8.8 Package Functions ~~~~~~~~~~~~~~~~~~~~~ Obviously, package functions are the most important ones for @@ -1045,7 +1234,7 @@ development manual for *Carbs Packaging Tools*. For development logs see manipulate, or to otherwise interact with packages. -8.7.1 `pkg_build()' +8.8.1 `pkg_build()' ------------------- This function builds all given packages. It resolves dependencies for @@ -1063,7 +1252,7 @@ development manual for *Carbs Packaging Tools*. For development logs see `---- -8.7.2 `pkg_depends()' +8.8.2 `pkg_depends()' --------------------- This function calculates the dependencies for the requested package, @@ -1072,17 +1261,17 @@ development manual for *Carbs Packaging Tools*. For development logs see packages. -[pkg_order()] See section 8.7.3 +[pkg_order()] See section 8.8.3 -8.7.3 `pkg_order()' +8.8.3 `pkg_order()' ------------------- This function receives package names and returns `$order' and `$redro' variables that can be used for building and removing packages. -8.7.4 `pkg_owner()' +8.8.4 `pkg_owner()' ------------------- This function can be used to determine the owner of a package. The @@ -1103,7 +1292,7 @@ development manual for *Carbs Packaging Tools*. For development logs see `---- -8.7.5 `pkg_isbuilt()' +8.8.5 `pkg_isbuilt()' --------------------- This function returns with success when the given package has a built @@ -1111,7 +1300,7 @@ development manual for *Carbs Packaging Tools*. For development logs see repository. -8.7.6 `pkg_lint()' +8.8.6 `pkg_lint()' ------------------ This function checks whether a given package fits the proper package @@ -1119,7 +1308,7 @@ development manual for *Carbs Packaging Tools*. For development logs see outright* if it fails. -8.7.7 `pkg_find()' +8.8.7 `pkg_find()' ------------------ `pkg_find()' is the tool for searching packages. It accepts up to 3 @@ -1152,17 +1341,17 @@ development manual for *Carbs Packaging Tools*. For development logs see `---- -8.7.8 `pkg_get_base()' +8.8.8 `pkg_get_base()' ---------------------- - This function returns the base packages as defined in - `/etc/cpt-base'. If an optional argument is present, it will print all - package names in a single line. If it is not given any arguments, it - will return one package per line. See 4.1 for more information on base + This function returns the base packages as defined in the base + file. If an optional argument is present, it will print all package + names in a single line. If it is not given any arguments, it will + return one package per line. See 4.1.1 for more information on base packages. -8.7.9 `pkg_gentree()' +8.8.9 `pkg_gentree()' --------------------- This function generates a dependency tree for the given package. The @@ -1182,7 +1371,7 @@ development manual for *Carbs Packaging Tools*. For development logs see line. -* 8.7.9.1 Examples +* 8.8.9.1 Examples This example uses the `cpt' package for Carbs Linux. The package itself is listed to depend on `curl' and `rsync'. Here is the output @@ -1208,20 +1397,34 @@ development manual for *Carbs Packaging Tools*. For development logs see `---- -8.7.10 `pkg_query_meta()' +8.8.10 `pkg_query_meta()' ------------------------- This function is used to query the [meta file] inside package directories. It can be used to retrieve information on a package that is otherwise irrelevant to the package manager itself. It takes two - arguments, first being the package and the second being the key to be - retrieved. If the package does not have a `meta' file or the file does - not contain the requested key, the function will return with 1. + arguments, first being the package (or the full path to a package + directory) and the second being the key to be retrieved. If the + package does not have a `meta' file or the file does not contain the + requested key, the function will return with 1. ,---- | $ pkg_query_meta cpt description | Carbs Packaging Tools + | + | $ pkg_query_meta /path/to/cpt license + | MIT `---- [meta file] See section 5.6 + + + +Footnotes +_________ + +[1] <https://datatracker.ietf.org/doc/html/rfc822#section-3.2> + +[2] Not mandatory for the packaging system, but mandatory for +inclusion in the repositories diff --git a/examples/hooks/clean-packages b/examples/hooks/clean-packages new file mode 100644 index 0000000..972eb31 --- /dev/null +++ b/examples/hooks/clean-packages @@ -0,0 +1,7 @@ +# -*- mode: sh; -*- +# clean-packages -- Remove documentation, and locales from packages. +case $TYPE in + post-build) + rm -rf "$DEST/usr/share/locale" \ + "$DEST/usr/share/doc" +esac diff --git a/examples/hooks/makewhatis b/examples/hooks/makewhatis new file mode 100644 index 0000000..5c38001 --- /dev/null +++ b/examples/hooks/makewhatis @@ -0,0 +1,10 @@ +# -*- mode: sh; -*- +# Run makewhatis if a manual page is installed, or removed +case $TYPE in + post-install|pre-remove) + grep -q "^/usr/share/man/" "$DEST/manifest" && + run_makewhatis=1 + ;; + end-install|end-remove) + [ "$run_makewhatis" ] && makewhatis "$DEST/usr/share/man" +esac diff --git a/man/cpt-build.1 b/man/cpt-build.1 index 79b24c6..ec65eeb 100644 --- a/man/cpt-build.1 +++ b/man/cpt-build.1 @@ -5,7 +5,7 @@ .Nd build given packages .Sh SYNOPSIS .Nm cpt-build -.Op Fl tfy +.Op Fl dfSty .Op Fl -root Ar ROOT .Op Fl hv .Op Ar package... @@ -21,18 +21,36 @@ package. .Pp The options are as follows: .Bl -tag -width 13n +.It Fl d , -debug +Keep the build directories after operation .It Fl t , -test Run tests (if they exist) +.It Fl S , -nostrip +Don't strip debug information from the binaries +.Po +might want to add +.Fl g +to your +.Ev $CFLAGS +.Pc .It Fl f , -force Force operation .It Fl y , -no-prompt Do not prompt for confirmation +.It Fl -color Ar CPT_COLOR +Enable/disable output color +.Bo +.Sy auto , +always, never, 1, 0 +.Bc .It Fl -root Ar CPT_ROOT Use an alternate root directory .It Fl h , -help Show help message .It Fl v , -version Print version information +.It Fl -verbose +Be more verbose .El .Sh AUTHOR .An Cem Keylan Aq Mt cem@ckyln.com diff --git a/man/cpt-checksum.1 b/man/cpt-checksum.1 index 4c637eb..bfced46 100644 --- a/man/cpt-checksum.1 +++ b/man/cpt-checksum.1 @@ -5,15 +5,27 @@ .Nd generate checksums .Sh SYNOPSIS .Nm +.Op Fl s .Op Ar package... .Sh DESCRIPTION .Nm -lints a package, downloads the sources, and creates a checksum file -including the SHA256 digests of the sources. If no +lints a package, downloads the sources, and creates a file named +.Sq checksums +which includes the +.Em BLAKE3 +digests of the sources. If no .Ar package is specified, .Nm will generate checksums assuming the working directory as the package. +.Pp +If the +.Fl s +flag is given, +.Nm +will generate checksums using the +.Em SHA256 +algorithm. .Sh AUTHOR .An Cem Keylan Aq Mt cem@ckyln.com .Sh LICENSE diff --git a/man/cpt-install.1 b/man/cpt-install.1 index 65069f3..dfd483c 100644 --- a/man/cpt-install.1 +++ b/man/cpt-install.1 @@ -23,12 +23,20 @@ The options are as follows: Force installation .It Fl y , -no-prompt Do not prompt for confirmation +.It Fl -color Ar CPT_COLOR +Enable/disable output color +.Bo +.Sy auto , +always, never, 1, 0 +.Bc .It Fl -root Ar CPT_ROOT Use an alternate root directory .It Fl h , -help Show help message .It Fl v , -version Print version information +.It Fl -verbose +Be more verbose .El .Sh AUTHOR .An Cem Keylan Aq Mt cem@ckyln.com diff --git a/man/cpt-list.1 b/man/cpt-list.1 index 292ba13..778a57d 100644 --- a/man/cpt-list.1 +++ b/man/cpt-list.1 @@ -5,7 +5,7 @@ .Nd list packages .Sh SYNOPSIS .Nm -.Op Fl c +.Op Fl cq .Ar pkg... .Nm .Fl C @@ -20,7 +20,7 @@ list the given packages. If any of the given packages are not installed on the system, .Nm will exit with an error. However, -.Fl c +.Fl C and .Fl -check flags can be used in order to change the behaviour of the utility: @@ -28,6 +28,8 @@ flags can be used in order to change the behaviour of the utility: .It Fl c , -current Assumes that the current directory is a package and will use it instead of arguments. +.It Fl q , -quiet +Make the operation quiet, only reporting exit status. .It Fl C , -check Ar package true_statement false_statement Checks whether the .Em package diff --git a/man/cpt-remove.1 b/man/cpt-remove.1 index f4bb6fb..6ebc415 100644 --- a/man/cpt-remove.1 +++ b/man/cpt-remove.1 @@ -21,12 +21,20 @@ The options are as follows: Force removal .It Fl y , -no-prompt Do not prompt for confirmation +.It Fl -color Ar CPT_COLOR +Enable/disable output color +.Bo +.Sy auto , +always, never, 1, 0 +.Bc .It Fl -root Ar CPT_ROOT Use an alternate root directory .It Fl h , -help Show help message .It Fl v , -version Print version information +.It Fl -verbose +Be more verbose .El .Sh AUTHOR .An Cem Keylan Aq Mt cem@ckyln.com diff --git a/man/cpt-search.1 b/man/cpt-search.1 index 05452bd..6e96954 100644 --- a/man/cpt-search.1 +++ b/man/cpt-search.1 @@ -5,8 +5,15 @@ .Nd search for cpt packages .Sh SYNOPSIS .Nm -.Op Fl dso -.Op Ar query +.Op Fl ds +.Ar package... +.Nm +.Fl o +.Op Fl ds +.Nm +.Fl q +.Op Fl Fds +.Ar query .Sh DESCRIPTION .Nm can be used to search packages. Glob characters can also be used in the search. @@ -17,14 +24,50 @@ The options are as follows: Do not search the installed package database. .It Fl s , -single Only show the first instance of a package. +.It Fl q , -query +Search packages making use of package descriptions. +.It Fl F , -fixed +Run query mode interpreting the given pattern as a fixed string .It Fl o , -others Use the current directory as the package and show other instances of that package. +.It Fl -color Ar CPT_COLOR +Enable/disable output color +.Bo +.Sy auto , +always, never, 1, 0 +.Bc .It Fl h , -help Show help message .It Fl v , -version Print version information +.It Fl -verbose +Be more verbose .El +.Pp +The program has three modes of operations. The default operation is to search +for the packages given as positional arguments. +.Pp +If the +.Fl o +flag is specified, +.Nm +will use the name of the current directory to search for instances of other +packages with the same name. +.Pp +If the +.Fl q +flag is specified, +.Nm +will search through the name and description of packages using the given +.Ar query , +and run a case-insensitive search through +.Xr grep 1 . +If additionally the +.Fl F +flag is given, the given +.Ar query +will be considered a fixed string. .Sh EXAMPLES Below are usage examples for .Nm , diff --git a/man/cpt-update.1 b/man/cpt-update.1 index 3af1205..bc63ea7 100644 --- a/man/cpt-update.1 +++ b/man/cpt-update.1 @@ -30,6 +30,8 @@ Use an alternate root directory Show help message .It Fl v , -version Print version information +.It Fl -verbose +Be more verbose .El .Sh AUTHOR .An Cem Keylan Aq Mt cem@ckyln.com @@ -21,7 +21,7 @@ program or through the web from .Lk https://carbslinux.org/docs.html . It can also be read plaintext by running .Pp -.Dl less Pa /usr/share/doc/cpt.txt +.Dl less Pa /usr/share/doc/cpt/cpt.txt .Sh AUTHOR .An Cem Keylan Aq Mt cem@ckyln.com .Sh LICENSE diff --git a/spec/01_lib_spec.sh b/spec/01_lib_spec.sh index 581ba70..b7d83f3 100644 --- a/spec/01_lib_spec.sh +++ b/spec/01_lib_spec.sh @@ -1,4 +1,5 @@ # shellcheck disable=2091,2034 +CPT_VERBOSE=1 Describe 'CPT Library' export CPT_COLOR=0 @@ -13,7 +14,7 @@ Describe 'CPT Library' VERSION=$(sed -n '/VERSION/s/.* //gp' config.mk) It 'prints version information' When run script src/cpt-lib version - The stderr should eq "-> Carbs Packaging Tools $VERSION" + The line 1 of stdout should eq "Carbs Packaging Tools, version $VERSION" End End Describe 'text functions' @@ -111,16 +112,16 @@ Describe 'CPT Library' Describe '_readlinkf()' mklink() { :> tests/testfile; ln -s testfile tests/testfile2 ;} - rmlink() { rm -f tests/testfile tests/testfile2 ;} + rmlink() { rm tests/testfile tests/testfile2 ;} RPWD=$(cd -P .||:; printf %s "$PWD") - Before mklink - After rmlink + BeforeEach mklink + AfterEach rmlink Parameters "#1" . "$RPWD" - "#2" "$PWD/tests/testfile2" "$RPWD/tests/testfile" + "#2" "./tests/testfile2" "$RPWD/tests/testfile" End - It "outputs the real location of the given file ($1)" - When call _readlinkf "$2" + It "outputs the real location of the given file [$1] ($2 -> $3)" + When run _readlinkf "$2" The output should eq "$3" End End @@ -150,6 +151,53 @@ Describe 'CPT Library' End End + Describe 'version control functions' + check_internet_connection() { ! curl -L git.carbslinux.org >/dev/null 2>&1 ;} + Skip if "no internet connection" check_internet_connection + Describe 'pkg_vcs_clone_git()' + tmpfos=$$ + setup() { mkdir "/tmp/test_repository.$tmpfos" && cd "/tmp/test_repository.$tmpfos" || return ;} + cleanup() { cd /tmp && rm -rf "test_repository.$tmpfos" ;} + check_version() { [ "$1" = "$(sed -n '/^version=/s/.*=//p' configure)" ] ;} + BeforeEach setup + AfterEach cleanup + It "clones the given git repository to the current directory" + When call pkg_vcs_clone_git https://git.carbslinux.org/cpt + The output should not eq "" + The stderr should not eq "" + The status should be success + Assert [ ! -d test_repository ] + Assert [ -f README.md ] + Assert check_version Fossil + End + It "clones the given tag when asked for it" + When call pkg_vcs_clone_git https://git.carbslinux.org/cpt @6.2.4 + The output should not eq "" + The stderr should not eq "" + The status should be success + Assert [ ! -d test_repository ] + Assert [ -f README.md ] + Assert check_version 6.2.4 + End + End + Describe 'pkg_vcs_clone_fossil()' + tmpfos=$$ + setup() { mkdir "/tmp/test_repository.$tmpfos" && cd "/tmp/test_repository.$tmpfos" || return ;} + cleanup() { cd /tmp && rm -rf "test_repository.$tmpfos" ;} + check_version() { [ "$1" = "$(sed -n '/^version=/s/.*=//p' configure)" ] ;} + BeforeEach setup + AfterEach cleanup + It "clones the given fossil repository to the current directory" + When call pkg_vcs_clone_fossil https://fossil.carbslinux.org/cpt + The output should not eq "" + The stderr should eq "" + The status should be success + Assert [ ! -d test_repository ] + Assert [ -f README.md ] + Assert check_version Fossil + End + End + End Describe 'package functions' Describe 'run_hook()' CPT_HOOK=$PWD/tests/hook-file @@ -161,15 +209,9 @@ Describe 'CPT Library' End It "doesn't log 'running hook' if no package is given" When call run_hook 2 '' destination - The stderr should eq "" + The stderr should eq "-> Running 2 hook" The output should eq "$CPT_HOOK 2 null destination" End - It "uses the /etc/cpt-hook file of the root when called with a fourth arg" - When call run_hook 3 cpt destdir root - The stderr should eq "-> cpt Running 3 hook" - The output should eq "$CPT_ROOT/etc/cpt-hook 3 cpt destdir" - The variable CPT_HOOK should eq "$PWD/tests/hook-file" - End It "returns with success even when the file doesn't exist" CPT_HOOK=$PWD/some-non-existent-file When call run_hook 4 thiswillnotrun @@ -177,33 +219,42 @@ Describe 'CPT Library' The stderr should eq "" The status should be success End - It "restores the \$CPT_HOOK variable when called with root" - CPT_ROOT=$PWD/nonexistentdir - When call run_hook 5 cpt dest root - The variable CPT_ROOT should not be exist - The stderr should eq "" - The status should be success - The variable CPT_HOOK should eq "$PWD/tests/hook-file" - End End - Describe 'create_cache()' + Describe 'create_tmp()' After pkg_clean It 'creates cache directories' - When call create_cache + When call create_tmp The variable mak_dir should be a directory End - It "doesn't create build directories if an argument is passed" - When call create_cache nobuild - The variable mak_dir should be undefined - End End Describe 'pkg_get_base()' CPT_ROOT=$PWD/tests CPT_PATH=$PWD/tests/repository + cpt_base=$PWD/tests/etc/cpt-base It 'returns packages defined in base' When call pkg_get_base nonl The output should eq "dummy-pkg contrib-dummy-pkg " End End + Describe 'pkg_query_meta()' + CPT_PATH=$PWD/tests/repository + It 'queries package meta information' + When call pkg_query_meta contrib-dummy-pkg description + The output should eq "This is a dummy package" + End + It 'returns an error if there is no meta file' + When call pkg_query_meta dummy-pkg description + The status should be failure + End + It 'returns an error if the queried key is unavailable' + When call pkg_query_meta contrib-dummy-pkg license + The status should be failure + End + It "accepts full paths to the package location" + When call pkg_query_meta "$PWD/tests/repository/contrib-dummy-pkg" description + The output should eq "This is a dummy package" + The status should be success + End + End End End diff --git a/spec/02_src_spec.sh b/spec/02_src_spec.sh index 4685ec1..2c20849 100644 --- a/spec/02_src_spec.sh +++ b/spec/02_src_spec.sh @@ -1,7 +1,8 @@ Describe 'Main toolchain' - export PATH=$PWD/src:$PATH - export CPT_ROOT=$PWD/tests/02 - export CPT_PATH=$PWD/tests/repository + PATH=$PWD/src:$PATH + CPT_ROOT=$PWD/tests/02 + CPT_PATH=$PWD/tests/repository + export PATH CPT_ROOT CPT_PATH install_dummy() { CPT_HOOK='' ./src/cpt bi dummy-pkg >/dev/null 2>&1 ;} remove_dummy() { rm -rf "${CPT_ROOT:?}/var" ;} BeforeAll install_dummy @@ -13,14 +14,14 @@ Describe 'Main toolchain' VERSION=$(sed -n '/VERSION/s/.* //gp' config.mk) It 'outputs cpt version' When run script src/cpt --version - The stderr should eq "-> Carbs Packaging Tools $VERSION" + The line 1 of stdout should eq "Carbs Packaging Tools, version $VERSION" End End Describe '--help' It 'outputs usage information' When run script src/cpt --help - The line 1 of stderr should eq "-> Carbs Packaging Tool " + The line 1 of stderr should eq "-> Carbs Packaging Tool" End End @@ -52,7 +53,7 @@ Describe 'Main toolchain' It "expands the '$1' shortcut to '$2'" When run script src/cpt "$1" --help The status should be success - The word 2 of line 1 should eq "cpt-${2%% *}" + The word 3 of line 1 should eq "$1" End End End diff --git a/spec/03_contrib_spec.sh b/spec/03_contrib_spec.sh index 9c7589b..b3d6df4 100644 --- a/spec/03_contrib_spec.sh +++ b/spec/03_contrib_spec.sh @@ -1,8 +1,9 @@ Describe 'contrib scripts' - export PATH=$PWD/src:$PWD/contrib:$PATH - export CPT_ROOT=$PWD/tests/03 - export CPT_PATH=$PWD/tests/repository - export CPT_COMPRESS='' + PATH=$PWD/src:$PWD/contrib:$PATH + CPT_ROOT=$PWD/tests/03 + CPT_PATH=$PWD/tests/repository + CPT_COMPRESS='' + export PATH CPT_ROOT CPT_PATH CPT_COMPRESS install_tmp() { CPT_HOOK='' ./src/cpt b -y contrib-dummy-pkg >/dev/null 2>&1 CPT_HOOK='' ./src/cpt-install -y contrib-dummy-pkg >/dev/null 2>&1 @@ -17,13 +18,13 @@ Describe 'contrib scripts' It 'outputs the file contents in the given package directory' When run script ./contrib/cpt-cat "$firstpkg" The stdout should not eq "" - The line 1 of stderr should eq "$(printf '\033[1mbuild:\033[m\n')" + The line 1 of stderr should eq "build:" End It "uses the current directory for the package name if none is supplied (contrib-dummy-pkg)" cd "$CPT_PATH/contrib-dummy-pkg" || return 1 When run script "$(command -v cpt-cat)" The stdout should not eq "" - The line 1 of stderr should eq "$(printf '\033[1mbuild:\033[m\n')" + The line 1 of stderr should eq "build:" End It "exits with error if the package isn't installed" When run script ./contrib/cpt-cat somerandompackage @@ -44,13 +45,13 @@ Describe 'contrib scripts' It "outputs the given file contents in the given package directory ($1)" When run script ./contrib/cpt-cat "$firstpkg" "$1" The stdout should eq "$(cat "$CPT_ROOT/var/db/cpt/installed/$firstpkg/$1")" - The stderr should eq "$(printf '\033[1m%s:\033[m\n' "$1")" + The stderr should eq "$1:" End It "outputs the given file contents for the name of the current directory (contrib-dummy-pkg - $1)" cd "$CPT_PATH/contrib-dummy-pkg" || return 1 When run script "$(command -v cpt-cat)" "" "$1" The stdout should eq "$(cat "$CPT_ROOT/var/db/cpt/installed/${PWD##*/}/$1")" - The stderr should eq "$(printf '\033[1m%s:\033[m\n' "$1")" + The stderr should eq "$1:" End End Describe 'cpt-depends' @@ -94,6 +95,7 @@ Describe 'contrib scripts' export CPT_COMPRESS=typo When run script "$(command -v cpt-export)" contrib-dummy-pkg The stdout should eq "tarball created in $CPT_ROOT/tmp/contrib-dummy-pkg#1-1.tar.gz" + The stderr should eq "WARNING 'typo' is not a valid CPT_COMPRESS value, falling back to 'gz' " End Parameters bz2 bzip2 @@ -1,5 +1,52 @@ #!/bin/sh -ef +bi() { + # Build and install function for cpt. + # + # shellcheck disable=2317 + parser_definition() { + setup REST help:usage -- "usage: ${0##*/} bi [-dfSty] [--root ROOT] [pkg...]" + msg -- '' 'Options:' + flag CPT_TEST -t --test export:1 init:@export -- "Run tests (if they exist)" + flag CPT_DEBUG -d --debug export:1 init:@export -- "Keep the build directories after operation" + flag CPT_NOSTRIP -S --nostrip export:1 init:@export -- "Don't strip debug information from the binaries" \ + "(might want to add '-g' to your '\$CFLAGS')" + global_options + } + + eval "$(getoptions parser_definition parse "$0")" + parse "$@" + eval set -- "$REST" + cpt-build "$@" + + # When building multiple packages, cpt will already ask to install + # the packages, so no need for this here. + [ "$2" ] || cpt-install "$@" +} + +cbi() { + # Checksum, build and install. + # + # shellcheck disable=2317 + parser_definition() { + setup REST help:usage -- "usage: ${0##*/} cbi [-dfSsty] [--root ROOT] [pkg...]" + msg -- '' 'Options:' + flag CPT_TEST -t --test export:1 init:@export -- "Run tests (if they exist)" + flag CPT_DEBUG -d --debug export:1 init:@export -- "Keep the build directories after operation" + flag CPT_NOSTRIP -S --nostrip export:1 init:@export -- "Don't strip debug information from the binaries" \ + "(might want to add '-g' to your '\$CFLAGS')" + flag sha -s -- "Generate checksums using the depracated sha256 algorithm" + global_options + } + + eval "$(getoptions parser_definition parse "$0")" + parse "$@" + eval set -- "$REST" + + cpt-checksum "$@"; cpt-build "$@" + [ "$2" ] || cpt-install "$@" +} + if [ -f ./cpt-lib ]; then . ./cpt-lib; else . cpt-lib; fi # If none of the tools below are specified, we will reenable glob @@ -11,7 +58,7 @@ case "$arg" in log "Carbs Packaging Tool" set -- for path in $(SEARCH_PATH=$PATH pkg_find cpt-* all -x); do - set -- "${path#*/cpt-}" "$@" + set -- "${path##*/cpt-}" "$@" max=$((${#1} > max ? ${#1} : max)) done @@ -35,21 +82,7 @@ case "$arg" in r|remove) arg=remove ;; s|search) arg=search ;; u|update) arg=update ;; - bi) - # Build and install function for cpt. - cpt-build "$@" - - # When building multiple packages, cpt will already ask to install - # the packages, so no need for this here. - [ "$2" ] || cpt-install "$@" - exit - ;; - cbi) - # Checksum, build and install. - cpt-checksum "$@"; cpt-build "$@" - [ "$2" ] || cpt-install "$@" - exit - ;; + bi|cbi) "$arg" "$@"; exit "$?" ;; *) glob=1 ;; esac diff --git a/src/cpt-alternatives b/src/cpt-alternatives index c4c7726..87946f0 100755 --- a/src/cpt-alternatives +++ b/src/cpt-alternatives @@ -2,7 +2,11 @@ # List and swap to alternatives parser_definition() { - setup REST help:usage -- "usage: ${0##*/} [-] [package file]" + setup REST help:usage -- "usage: ${0##*/} package file" + msg -- "or: ${0##*/} [-p]" + msg -- "or: ${0##*/} -" + msg -- '' 'Options:' + flag preferred -p -- "List current owners of alternative files" global_options } @@ -10,23 +14,71 @@ if [ -f ./cpt-lib ]; then . ./cpt-lib; else . cpt-lib; fi # We don't need to be root in order to list alternatives, so skip privilege # elevation if no arguments are passed to the script. -[ -z "$1" ] || [ -w "$CPT_ROOT/" ] || [ "$uid" = 0 ] || { +[ -z "$1" ] && [ -t 0 ] || [ -w "$CPT_ROOT/" ] || [ "$uid" = 0 ] || { as_root "$0" "$@" exit $? } +list_alternatives() { + # Go over each alternative and format the file name for listing. + # (pkg_name>usr>bin>ls) + set +f + for pkg in "$sys_db/../choices/"*; do + printf '%s\n' "${pkg##*/}" + done | sed 's|>|\t/|;s|>|/|g;/\*/d' +} + +stdin_swap() { + # Swap packages by reading the standard input. + while IFS=$(printf '\t') read -r pkg path _; do + pkg_swap "$pkg" "$path" + done + exit +} + case "$1" in - -) - while read -r pkg path; do - pkg_swap "$pkg" "$path" - done - ;; + -) stdin_swap ;; '') - # Go over each alternative and format the file name for listing. - # (pkg_name>usr>bin>ls) - set +f; for pkg in "$sys_db/../choices/"*; do - printf '%s\n' "${pkg##*/}" - done | sed 's|>| /|; s|>|/|g; /\*/d' + # We still want to read the standard input when there are no arguments + # if the standard input is being used. + [ -t 0 ] || stdin_swap + + if [ "$preferred" ]; then + # We are not using the pkg_owner() function here. It's much slower + # when searching items in bulk. + altlist=$(_tmp_create altlist) + pathlist=$(_tmp_create pathlist) + owners=$(_tmp_create owners) + list_alternatives | tee "$altlist" | sed 's,^[^\t]*\t,,' > "$pathlist" + + # Save all matching items in a single file, so we don't ever read + # manifests again. + set +f + grep -Fxf "$pathlist" "$sys_db/"*/manifest > "$owners" + sys_db_esc=$(regesc "$sys_db") + + while read -r pkg path; do + case $path in + # Running regesc() for each file slows us down, don't use + # it unless we detect a regular expression to escape. + *\[*|*\$*|*\\*|*.*|*^*) path_str=$(regesc "$path") ;; + *) path_str=$path + esac + grep_str="$sys_db_esc/[^/]*/manifest:$path_str" + owns=$(grep -x -- "$grep_str" "$owners") || owns=null + owns=${owns%:*} owns=${owns%/*} owns=${owns##*/} + printf '%s\t%s\t(owned by: %s)\n' "$pkg" "$path" "$owns" + done < "$altlist" + + # We read the output of list_alternatives(), because that seems to + # be the faster option. + # list_alternatives | while read -r pkg path; do + # printf '%s %s (owned by: %s)\n' \ + # "$pkg" "$path" "$(pkg_owner -lFx "$path" || out "null")" + # done + else + list_alternatives + fi ;; *) pkg_swap "$@" ;; esac diff --git a/src/cpt-build b/src/cpt-build index cb93949..eaf8849 100755 --- a/src/cpt-build +++ b/src/cpt-build @@ -2,15 +2,18 @@ # Build a package parser_definition() { - setup REST help:usage -- "usage: ${0##*/} [pkg...]" + setup REST help:usage -- "usage: ${0##*/} [-dfSty] [--root ROOT] [pkg...]" msg -- '' 'Options:' - flag CPT_TEST -t --test export:1 init:@export -- "Run tests (if they exist)" + flag CPT_DEBUG -d --debug export:1 init:@export -- "Keep the build directories after operation" + flag CPT_TEST -t --test export:1 init:@export -- "Run tests (if they exist)" + flag CPT_NOSTRIP -S --nostrip export:1 init:@export -- "Don't strip debug information from the binaries" \ + "(might want to add '-g' to your '\$CFLAGS')" global_options } if [ -f ./cpt-lib ]; then . ./cpt-lib; else . cpt-lib; fi -[ "$1" ] || { set -- "${PWD##*/}"; export CPT_PATH=${PWD%/*}:$CPT_PATH ;} +[ "$1" ] || { set -- "${PWD##*/}"; export CPT_PATH="${PWD%/*}:$CPT_PATH" ;} create_cache diff --git a/src/cpt-checksum b/src/cpt-checksum index 5c6de52..488ab87 100755 --- a/src/cpt-checksum +++ b/src/cpt-checksum @@ -1,22 +1,32 @@ #!/bin/sh -ef # Generate checksums -if [ -f ./cpt-lib ]; then . ./cpt-lib; else . cpt-lib; fi +parser_definition() { + setup REST help:usage -- "usage: ${0##*/} [-s] [pkg...]" + msg -- '' 'Options:' + flag sha -s -- "Generate checksums using the depracated sha256 algorithm" + global_options +} -case "$1" in - --help|-h) out "usage: ${0##*/} [pkg...]"; exit 0 ;; - --version|-v) version ;; - '') set -- "${PWD##*/}"; export CPT_PATH=${PWD%/*}:$CPT_PATH ;; -esac +if [ -f ./cpt-lib ]; then . ./cpt-lib; else . cpt-lib; fi +[ "$1" ] || { set -- "${PWD##*/}"; CPT_PATH=${PWD%/*}:$CPT_PATH ;} create_cache for pkg; do pkg_lint "$pkg" c; done for pkg; do pkg_sources "$pkg" c; done - for pkg; do - pkg_checksums "$pkg" | { - repo_dir=$(pkg_find "$pkg") + # Do not generate checksums if the 'sources' file is empty or it doesn't + # exist. + repo_dir=$(pkg_find "$pkg") + [ -s "$repo_dir/sources" ] || { + log "$pkg" "No 'sources' file, skipping checksums" + continue + } + + # $sha is defined by the parser. + # shellcheck disable=2154 + pkg_checksums "$pkg" "${sha:+sh256}" | { if [ -w "$repo_dir" ]; then tee "$repo_dir/checksums" diff --git a/src/cpt-download b/src/cpt-download index d2c9aeb..231e4bb 100755 --- a/src/cpt-download +++ b/src/cpt-download @@ -9,7 +9,7 @@ case "$1" in exit 0 ;; --version|-v) version ;; - '') set -- "${PWD##*/}"; export CPT_PATH=${PWD%/*}:$CPT_PATH + '') set -- "${PWD##*/}"; export CPT_PATH="${PWD%/*}:$CPT_PATH" esac create_cache diff --git a/src/cpt-install b/src/cpt-install index aab70ea..8bafba6 100755 --- a/src/cpt-install +++ b/src/cpt-install @@ -10,7 +10,7 @@ parser_definition() { if [ -f ./cpt-lib ]; then . ./cpt-lib; else . cpt-lib; fi -[ "$1" ] || { set -- "${PWD##*/}"; export CPT_PATH=${PWD%/*}:$CPT_PATH ;} +[ "$1" ] || { set -- "${PWD##*/}"; export CPT_PATH="${PWD%/*}:$CPT_PATH" ;} [ -w "$CPT_ROOT/" ] || [ "$uid" = 0 ] || { as_root "$0" "$@" @@ -18,7 +18,6 @@ if [ -f ./cpt-lib ]; then . ./cpt-lib; else . cpt-lib; fi } pkg_order "$@" - create_cache # shellcheck disable=2154 @@ -28,14 +27,21 @@ for pkg in $order; do pkg_install "$pkg"; done log "Retrieving post-installation message queue" unset msg +# After all the installations are finished, run an end-install hook. There may +# be some things that we may want to run, but not per package. +run_hook end-install "" "$CPT_ROOT" + for pkg in $order; do + # Ensure that we use package names itself, and not the tarball name if given. + pkg=${pkg##*/} pkg=${pkg%#*} + [ -f "$sys_db/$pkg/message" ] && { - printf '%s\n%s\n%s\n\n' \ - "=======================================" \ + printf '\033[1m%s\n%s\n%s\033[m\n\n' \ + "$(_multiply_char '=' 60)" \ "$pkg" \ - "=======================================" - cat "$sys_db/$pkg/message" >&2 + "$(_multiply_char '=' 60)" + cat "$sys_db/$pkg/message" msg=1 } -done +done >&2 [ "$msg" ] || log "No message in queue" diff --git a/src/cpt-lib.in b/src/cpt-lib.in index bf58fbe..1896920 100644 --- a/src/cpt-lib.in +++ b/src/cpt-lib.in @@ -9,7 +9,9 @@ # Currently maintained by Cem Keylan. version() { - log "Carbs Packaging Tools" @VERSION@ + out "Carbs Packaging Tools, version $cpt_version" \ + @LICENSE@ + exit 0 } @@ -23,11 +25,40 @@ log() { # # All messages are printed to stderr to allow the user to hide build # output which is the only thing printed to stdout. - # - # '${3:-->}': If the 3rd argument is missing, set prefix to '->'. - # '${2:+colorb}': If the 2nd argument exists, set text style of '$1'. - printf '%b%s %b%b%s%b %s\n' \ - "$colory" "${3:-->}" "$colre" "${2:+$colorb}" "$1" "$colre" "$2" >&2 + case $# in + 1) printf '%b->%b %s\n' "$colory" "$colre" "$1" ;; + 2) printf '%b->%b %b%s%b %s\n' "$colory" "$colre" "$colorb" "$1" "$colre" "$2" ;; + 3) printf '%b%s%b %b%s%b %s\n' "$colory" "${3:-->}" "$colre" "$colorb" "$1" "$colre" "$2" ;; + *) return 1 + esac >&2 +} + +warn() { + # Print a warning message + log "$1" "$2" "${3:-WARNING}" +} + +outv() { + # Call `out()` when CPT_VERBOSE is set. + [ "$CPT_VERBOSE" = 1 ] || return 0 + out "$@" +} + +logv() { + # Call `log()` when CPT_VERBOSE is set. + [ "$CPT_VERBOSE" = 1 ] || return 0 + log "$@" +} + +warnv() { + # Call `warn()` when CPT_VERBOSE is set. + [ "$CPT_VERBOSE" = 1 ] || return 0 + warn "$@" +} + +execv() { + # Redirect the output to /dev/null unless CPT_VERBOSE is set. + if [ "$CPT_VERBOSE" = 1 ]; then "$@"; else "$@" >/dev/null 2>&1; fi } die() { @@ -36,6 +67,124 @@ die() { exit 1 } +colors_enabled() { + case ${CPT_COLOR:=auto} in + auto) [ -t 1 ] ;; + 1|always) return 0 ;; + 0|never) return 1 ;; + *) die "Unknown color value: '$CPT_COLOR'" + esac +} + +_dep_append() { + dep_graph=$(printf '%s\n%s %s\n' "$dep_graph" "$@" ;) +} + +_tsort() { + # Return a linear reverse topological sort of the piped input, so we + # generate a proper build order. Returns 1 if a dependency cycle occurs. + # + # I was really excited when I saw POSIX specified a tsort(1) implementation, + # but the specification is quite vague, it doesn't specify cycles as a + # reason of error, and implementations differ on how it's handled. coreutils + # tsort(1) exits with an error, while openbsd tsort(1) doesn't. Both + # implementations are correct according to the specification. + # + # The script below was taken from <https://gist.github.com/apainintheneck/1803fb91dde3ba048ec51d44fa6065a4> + # + # The MIT License (MIT) + # Copyright (c) 2023 Kevin Robell + # + # Permission is hereby granted, free of charge, to any person obtaining a + # copy of this software and associated documentation files (the “Software”), + # to deal in the Software without restriction, including without limitation + # the rights to use, copy, modify, merge, publish, distribute, sublicense, + # and/or sell copies of the Software, and to permit persons to whom the + # Software is furnished to do so, subject to the following conditions: + # + # The above copyright notice and this permission notice shall be included in + # all copies or substantial portions of the Software. + # + # THE SOFTWARE IS PROVIDED “AS IS”, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR + # IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, + # FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL + # THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER + # LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING + # FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER + # DEALINGS IN THE SOFTWARE. + awk '{ + for (i = 1; i <= NF; ++i) { + # Store each node. + nodes[$i] = 1 + if (is_child) { + child = $i + # Skip nodes that point to themselves. + # This traditionally means that the node + # is disconnected from the rest of the graph. + if (parent != child) { + # Store from parent to child. + idx = ++child_count[parent] + child_graph[parent, idx] = child + # Store count from child to parent. + ++parent_count[child] + } + } else { + parent = $i + } + # Flip switch + is_child = !is_child + } + } + END { + # Print errors to the stderr + stderr = "/dev/stderr" + + # Sanity Check + if (is_child) { + print("Error: odd number of input values: expected pairs of values") > stderr + exit(1) + } + + ##### + # Topological Sort + ##### + + # Remove unconnected nodes first. + for (node in nodes) { + if (parent_count[node] == 0 && child_count[node] == 0) { + delete nodes[node] + print(node) + } + } + + # Remove the rest of the nodes starting with those without parents. + while (length(nodes) > 0) { + removed_node = 0 + for (node in nodes) { + # Delete and print nodes without any remaining parents. + if (parent_count[node] == 0) { + delete nodes[node] + removed_node = 1 + # Decrease child_count for each parent node. + for (i = child_count[node]; i > 0; --i) { + child = child_graph[node, i] + --parent_count[child] + } + print(node) + } + } + + # If we havent removed any nodes, it means that there + # are no nodes without any remaining parents so we have + # a cycle. + if (!removed_node) { + print("Error: Cycle found") > stderr + exit(1) + } + } + }' +} + trap_set() { # Function to set the trap value. case ${1:-cleanup} in @@ -43,25 +192,30 @@ trap_set() { trap pkg_clean EXIT trap 'pkg_clean; exit 1' INT ;; - block) trap '' INT ;; - unset) trap - EXIT INT ;; + handle-int) + trap pkg_clean INT + ;; + block) trap '' INT ;; + unset) trap - EXIT INT ;; esac } -sepchar() ( +sepchar() { # Seperate every character on the given string without resorting to external # processes. [ "$1" ] || return 0; str=$1; set -- while [ "$str" ]; do - str_tmp=$str - for i in $(_seq $(( ${#str} - 1 ))); do - str_tmp=${str_tmp%?} - done - set -- "$@" "$str_tmp" - str=${str#$str_tmp} + set -- "$@" "${str%"${str#?}"}" + str=${str#?} done printf '%s\n' "$@" -) +} + +_re() { + # Check that the string supplied in $2 conforms to the regular expression + # of $1. + printf %s "${2:?}" | grep -Eq "$1" +} _seq() ( # Pure shell counter meant to be used in 'for' loops. @@ -72,13 +226,21 @@ _seq() ( printf '%s' "$buf" ) +_multiply_char() ( + buf= + for i in $(_seq "$2"); do + buf="$buf$1" + done + out "$buf" +) + _stat() ( _user=; eval set -- "$(ls -ld "$1")" id -u "${_user:=$3}" >/dev/null 2>&1 || _user=root printf '%s' "$_user" ) -_readlinkf() ( +_readlinkf() { # Public domain POSIX sh readlink function by Koichi Nakashima [ "${1:-}" ] || return 1 max_symlinks=40 @@ -112,13 +274,29 @@ _readlinkf() ( target=${link#*" $target -> "} done return 1 -) +} + +_get_digest() { + # Get digest algorithm from the given file. It looks for a header on the + # file declaring the digest algorithm. Currently only BLAKE3 is supported. + # If the file does not include a header, the function will assume that it is + # using sha256 as a digest algorithm. If the given file doesn't exist it will + # return 1. + [ -r "$1" ] || return 1 + read -r chk < "$1" + case $chk in + %BLAKE3) chk=b3sum ;; + %*) die "Unknown digest algorithm: '${chk#\%}'" ;; + *) chk=sh256 + esac + out "$chk" +} # This is the public domain getoptions shell library. It also forms a usage # function. # URL: https://github.com/ko1nksm/getoptions (v2.5.0) # License: Creative Commons Zero v1.0 Universal -# shellcheck disable=2016,2086 +# shellcheck disable=2016,2086,2317 getoptions() { _error='' _on=1 _off='' _export='' _plus='' _mode='' _alt='' _rest='' _flags='' _nflags='' _opts='' _help='' _abbr='' _cmds='' _init=@empty IFS=' ' @@ -313,6 +491,7 @@ getoptions() { } # URL: https://github.com/ko1nksm/getoptions (v2.5.0) # License: Creative Commons Zero v1.0 Universal +# shellcheck disable=2317 getoptions_help() { _width='30,12' _plus='' _leading=' ' @@ -357,18 +536,34 @@ getoptions_help() { echo "}" } +# 2086: +# The lack of quotes are intentional. We do this so `getoptions()` do not try +# to parse the empty string. +# 2120: +# The library does not call this function with any positional arguments, but +# that does not mean that other programs will not do it, so this can also be +# safely ignored. +# shellcheck disable=2086,2120 global_options() { - msg -- '' 'Global Options:' - flag CPT_FORCE -f --force init:@export -- "Force operation" - flag CPT_PROMPT -y --no-prompt on:0 off:0 init:@export -- "Do not prompt for confirmation" - param CPT_ROOT --root init:@export -- "Use an alternate root directory" - disp :usage -h --help -- "Show this help message" - disp :version -v --version -- "Print version information" -} - -warn() { - # Print a warning message - log "$1" "$2" "${3:-WARNING}" + # These are options that are supported by most utilities. If the optional + # argument 'silent' is given, the usage will not print these options, but + # the arguments will still be accepted. Alternatively, if the 'compact' + # argument is given, the function only prints the '--help' and '--version' + # flags. Sometimes it doesn't make sense to pollute the screen with options + # that will be rarely ever used. + _h=hidden:1 + case $1 in + silent) _c=$_h ;; + compact) _c='' ;; + *) msg -- '' 'Global Options:'; _c='' _h='' + esac + flag CPT_FORCE -f --force $_h init:@export -- "Force operation" + flag CPT_PROMPT -y --no-prompt on:0 off:0 $_h init:@export -- "Do not prompt for confirmation" + param CPT_ROOT --root $_h init:@export -- "Use an alternate root directory" + param CPT_COLOR --color $_h init:@export -- "Colorize the output [default:auto]" + disp :usage -h --help $_c -- "Show this help message" + disp :version -v --version $_c -- "Print version information" + flag CPT_VERBOSE --verbose $_h init:@export -- "Be more verbose" } contains() { @@ -383,6 +578,22 @@ regesc() { sed 's|\\|\\\\|g;s|\[|\\[|g;s|\$|\\$|g;s|\.|\\.|g;s|\*|\\*|g;s|\^|\\^|g' } +pkg_download() { + # $1: URL + # $2: Output (Optional) + set -- "$1" "$(_readlinkf "${2:-${1##*/}}")" + case ${dl_prog##*/} in + axel) set -- -o "$2" "$1" ;; + aria2c) set -- -d "${2%/*}" -o "${2##*/}" "$1" ;; + curl) set -- -fLo "$2" "$1" ;; + wget|wget2) set -- -O "$2" "$1" ;; + esac + + "$dl_prog" "$@" || { + rm -f "$2" + return 1 + } +} prompt() { # If a CPT_NOPROMPT variable is set, continue. @@ -407,6 +618,9 @@ as_root() { # We are exporting package manager variables, so that we still have the # same repository paths / access to the same cache directories etc. + # + # It doesn't matter whether CPT_HOOK is defined or not. + # shellcheck disable=2153 set -- HOME="$HOME" \ USER="$user" \ XDG_CACHE_HOME="$XDG_CACHE_HOME" \ @@ -421,8 +635,10 @@ as_root() { CPT_PATH="$CPT_PATH" \ CPT_PID="$CPT_PID" \ CPT_PROMPT="$CPT_PROMPT" \ + CPT_REPO_CACHE="$CPT_REPO_CACHE" \ CPT_ROOT="$CPT_ROOT" \ CPT_TMPDIR="$CPT_TMPDIR" \ + CPT_VERBOSE="$CPT_VERBOSE" \ "$@" case ${su##*/} in @@ -446,29 +662,48 @@ pop() { } run_hook() { - # Store the CPT_HOOK variable so that we can revert it if it is changed. - oldCPT_HOOK=$CPT_HOOK - - # If a fourth parameter 'root' is specified, source the hook from a - # predefined location to avoid privilige escalation through user scripts. - [ "$4" ] && CPT_HOOK=$CPT_ROOT/etc/cpt-hook - - [ -f "$CPT_HOOK" ] || { CPT_HOOK=$oldCPT_HOOK; return 0 ;} + # Check that hooks exist before announcing that we are running a hook. + set +f + for hook in "$cpt_confdir/hooks/"* "$CPT_HOOK"; do + [ -f "$hook" ] && { + if [ "$2" ]; then + logv "$2" "Running $1 hook" + else + logv "Running $1 hook" + fi + break + } + done - [ "$2" ] && log "$2" "Running $1 hook" + # Run all the hooks found in the configuration directory, and the user + # defined hook. + for hook in "$cpt_confdir/hooks/"* "$CPT_HOOK"; do + set -f + [ -f "$hook" ] || continue + TYPE=${1:-null} PKG=${2:-null} DEST=${3:-null} . "$hook" + done +} - TYPE=${1:-null} PKG=${2:-null} DEST=${3:-null} . "$CPT_HOOK" - CPT_HOOK=$oldCPT_HOOK +# An optional argument could be provided to enforce a compression algorithm. +# shellcheck disable=2120 +compress() { + case ${1:-$CPT_COMPRESS} in + bz2) bzip2 -z ;; + gz) gzip -6 ;; + xz) xz -zT 0 ;; + zst) zstd -3 ;; + lz) lzip -6 ;; + esac } decompress() { case $1 in - *.tar) cat ;; - *.bz2) bzip2 -cd ;; - *.lz) lzip -cd ;; - *.xz|*.txz) xz -dcT 0 ;; - *.tgz|*.gz) gzip -cd ;; - *.zst) zstd -cd ;; + *.tar|*.cpio) cat ;; + *.bz2) bzip2 -cd ;; + *.lz) lzip -cd ;; + *.xz|*.txz) xz -dcT 0 ;; + *.tgz|*.gz) gzip -cd ;; + *.zst) zstd -cd ;; esac < "$1" } @@ -486,69 +721,6 @@ sh256() { while read -r hash _; do printf '%s %s\n' "$hash" "$1"; done } -tar_extract() { - # Tarball extraction function that prefers pax(1) over tar(1). The reason we - # are preferring pax is that we can strip components without relying on - # ugly hacks such as the ones we are doing for 'tar'. Using 'tar' means that - # we either have to sacrifice speed or portability, and we are choosing to - # sacrifice speed. Fortunately, we don't have to make such a choice when - # using pax. - case "${extract##*/}" in - pax) decompress "$1" | pax -r -s '/[^\/]*/./' ;; - gtar|bsdtar) decompress "$1" | "$tar" xf - --strip-components 1 ;; - tar) decompress "$1" > .ktar - - "$tar" xf .ktar || return - - # We now list the contents of the tarball so we can do our - # version of 'strip-components'. - "$tar" tf .ktar | - while read -r file; do printf '%s\n' "${file%%/*}"; done | - - # Do not repeat files. - uniq | - - # For every directory in the base we move each file - # inside it to the upper directory. - while read -r dir ; do - - # Skip if we are not dealing with a directory here. - # This way we don't remove files on the upper directory - # if a tar archive doesn't need directory stripping. - [ -d "${dir#.}" ] || continue - - # Change into the directory in a subshell so we don't - # need to cd back to the upper directory. - ( - cd "$dir" - - # We use find because we want to move hidden files - # as well. - # - # Skip the file if it has the same name as the directory. - # We will deal with it later. - # - # Word splitting is intentional here. - # shellcheck disable=2046 - find . \( ! -name . -prune \) ! -name "$dir" \ - -exec mv -f {} .. \; - - # If a file/directory with the same name as the directory - # exists, append a '.cptbak' to it and move it to the - # upper directory. - ! [ -e "$dir" ] || mv "$dir" "../${dir}.cptbak" - ) - rmdir "$dir" - - # If a backup file exists, move it into the original location. - ! [ -e "${dir}.cptbak" ] || mv "${dir}.cptbak" "$dir" - done - - # Clean up the temporary tarball. - rm -f .ktar - esac -} - pkg_owner() { set +f @@ -562,6 +734,18 @@ pkg_owner() { [ "$1" ] && printf '%s\n' "$1" } +pkg_owner_multi() { + set +f + + [ "$3" ] || set -- "$1" "$2" "$sys_db"/*/manifest + + grep "$@" | while read -r pkg_owner; do + pkg_owner=${pkg_owner%/*} + pkg_owner=${pkg_owner##*/} + printf '%s\n' "${pkg_owner##*/}" + done +} + pkg_isbuilt() ( # Check if a package is built or not. read -r ver rel < "$(pkg_find "$1")/version" @@ -580,14 +764,20 @@ pkg_lint() { repo_dir=$(pkg_find "$1") cd "$repo_dir" || die "'$repo_dir' not accessible" - [ -f sources ] || warn "$1" "Sources file not found" + [ -f sources ] || warnv "$1" "Sources file not found" [ -x build ] || die "$1" "Build file not found or not executable" [ -s version ] || die "$1" "Version file not found or empty" read -r _ release 2>/dev/null < version || die "Version file not found" [ "$release" ] || die "Release field not found in version file" - [ "$2" ] || [ -f checksums ] || die "$pkg" "Checksums are missing" + # If we have a second argument, we are generating the checksums file, + # so we don't need to check whether there is one. + [ -z "$2" ] || return 0 + + # Check for a checksums file only if there is a sources file. + [ -f sources ] || return 0 + [ -f checksums ] || die "$pkg" "Checksums are missing" } pkg_find() { @@ -688,12 +878,8 @@ pkg_sources() { repo_dir=$(pkg_find "$1") while read -r src dest || [ "$src" ]; do - # Remote git/hg repository or comment. - if [ -z "${src##\#*}" ] || - [ -z "${src##git+*}" ] || - [ -z "${src##hg+*}" ] - - then : + # Remote repository or comment. + if _re "$re_vcs_or_com" "$src"; then : # Remote source (cached). elif [ -f "${src##*/}" ]; then @@ -703,10 +889,15 @@ pkg_sources() { elif [ -z "${src##*://*}" ]; then log "$1" "Downloading $src" - curl "$src" -fLo "${src##*/}" || { - rm -f "${src##*/}" - die "$1" "Failed to download $src" - } + # We don't want our trap to exit immediately here if we receive an + # interrupt, we handle this ourselves. + trap_set handle-int + + # Download the source + pkg_download "$src" || die "$1" "Failed to download $src" + + # Restore original trap value. + trap_set cleanup # Local source. elif [ -f "$repo_dir/$src" ]; then @@ -732,32 +923,17 @@ pkg_extract() { mkdir -p "$mak_dir/$1/$dest" && cd "$mak_dir/$1/$dest" case $src in - # Git repository. - git+*) - # Split the source into URL + OBJECT (branch or commit). - url=${src##git+} com=${url##*[@#]} com=${com#${url%[@#]*}} - - log "$1" "Cloning ${url%[@#]*}"; { - git init - git remote add origin "${url%[@#]*}" - case "$url" in - # Tags are specified via '@' - *@*) git fetch -t --depth=1 origin "$com" || git fetch ;; - *) git fetch --depth=1 origin "$com" || git fetch - esac - git checkout "${com:-FETCH_HEAD}" - } || die "$1" "Failed to clone $src" - ;; - # Mercurial repository. - hg+*) - # Split the source into URL + OBJECT (branch or commit). - url=${src##hg+} com=${url##*[@#]} com=${com#${url%[@#]*}} + # VCS Repository + git+*|hg+*|fossil+*) + backend=${src%%+*} + url=${src##"${backend}"+} com=${url##*[@#]} com=${com#"${url%[@#]*}"} - # Unfortunately, there is no shallow cloning with Mercurial. - log "$1" "Cloning ${url%[@#]*}" - hg clone -u "${com:-tip}" + # Add back @ to com + case $url in *@*) com=@$com; esac + log "$1" "Cloning ${url%[#@]*}" + "pkg_vcs_clone_$backend" "${url%[#@]*}" "$com" ;; # Comment or blank line. @@ -766,11 +942,9 @@ pkg_extract() { # Only 'tar', 'cpio', and 'zip' archives are currently supported for # extraction. Other filetypes are simply copied to '$mak_dir' # which allows for manual extraction. - *://*.tar|*://*.tar.??|*://*.tar.???|*://*.tar.????|*://*.tgz|*://*.txz) - tar_extract "$src_dir/$1/${src##*/}" ;; - - *://*.cpio|*://*.cpio.??|*://*.cpio.???|*://*.cpio.????) - decompress "$src_dir/$1/${src##*/}" | pax -r ;; + *://*.tar|*://*.tar.??|*://*.tar.???|*://*.tar.????|*://*.tgz|\ + *://*.txz|*://*.cpio|*://*.cpio.??|*://*.cpio.???|*://*.cpio.????) + decompress "$src_dir/$1/${src##*/}" | pax -rs '|[^/]*|.|' ;; *://*.zip) unzip "$src_dir/$1/${src##*/}" || @@ -799,12 +973,10 @@ pkg_depends() { # Resolve all dependencies and generate an ordered list. # This does a depth-first search. The deepest dependencies are # listed first and then the parents in reverse order. - contains "$deps" "$1" || { - # Filter out non-explicit, aleady installed dependencies. - # Only filter installed if called from 'pkg_build()'. - [ "$pkg_build" ] && [ -z "$2" ] && - (pkg_list "$1" >/dev/null) && return - + # + # shellcheck disable=2015 + contains "$pkgs" "$1" && [ -z "$2" ] || { + [ "$2" = raw ] && _dep_append "$1" "$1" while read -r dep type || [ "$dep" ]; do # Skip comments and empty lines. [ "${dep##\#*}" ] || continue @@ -817,6 +989,16 @@ pkg_depends() { make) [ "$2" = tree ] && [ -z "${3#first-nomake}" ] && continue esac + # Filter out non-explicit, already installed dependencies if called + # from 'pkg_build()'. + [ "$pkg_build" ] && (pkg_list "$dep" >/dev/null) && continue + + if [ "$2" = explicit ] || [ "$3" ]; then + _dep_append "$dep" "$dep" + else + _dep_append "$dep" "$1" + fi + # Recurse through the dependencies of the child packages. Forward # the 'tree' operation. if [ "$2" = tree ]; then @@ -826,12 +1008,15 @@ pkg_depends() { fi done 2>/dev/null < "$(pkg_find "$1")/depends" ||: - # After child dependencies are added to the list, - # add the package which depends on them. - [ "$2" = explicit ] || [ "$3" ] || deps="$deps $1 " + pkgs="$pkgs $1 " } } +pkg_depends_commit() { + # Set deps, and cleanup dep_graph, pkgs + deps=$(printf '%s\n' "$dep_graph" | _tsort) dep_graph='' pkgs='' || warn "Dependency cycle detected" +} + pkg_order() { # Order a list of packages based on dependence and # take into account pre-built tarballs if this is @@ -839,9 +1024,10 @@ pkg_order() { order=; redro=; deps= for pkg do case $pkg in - *.tar.*) deps="$deps $pkg " ;; + *.tar.*) _dep_append "$pkg" "$pkg" ;; *) pkg_depends "$pkg" raw esac done + pkg_depends_commit # Filter the list, only keeping explicit packages. # The purpose of these two loops is to order the @@ -859,7 +1045,7 @@ pkg_strip() { # system as well as on the tarballs we ship for installation. # Package has stripping disabled, stop here. - [ -f "$mak_dir/$pkg/nostrip" ] && return + [ "$CPT_NOSTRIP" ] || [ -f "$mak_dir/$pkg/nostrip" ] && return log "$1" "Stripping binaries and libraries" @@ -882,21 +1068,31 @@ pkg_strip() { done 2>/dev/null ||: } +pkg_fix_deps_fullpath() { + # Return the canonical path of libraries extracted by readelf. + while read -r line _ rslv _; do + [ "$line" = "$1" ] || continue + case $rslv in + ldd) out "$line" ;; + *) out "$rslv" ;; + esac + done +} + pkg_fix_deps() { # Dynamically look for missing runtime dependencies by checking each binary # and library with either 'ldd' or 'readelf'. This catches any extra # libraries and or dependencies pulled in by the package's build suite. - log "$1" "Checking for missing dependencies" + log "$1" "Checking for missing dependencies (using ${elf_prog##*/})" # Go to the directory containing the built package to # simplify path building. cd "$pkg_dir/$1/$pkg_db/$1" - # Make a copy of the depends file if it exists to have a - # reference to 'diff' against. + # Make a copy of the depends file if it exists to have a reference to 'diff' + # against. if [ -f depends ]; then - cp -f depends "$mak_dir/d" - dep_file=$mak_dir/d + dep_file=$(_tmp_cp depends) else dep_file=/dev/null fi @@ -905,40 +1101,72 @@ pkg_fix_deps() { pkg_name=$1 set +f; set -f -- "$sys_db/"*/manifest - # Get a list of binaries and libraries, false files - # will be found, however it's faster to get 'ldd' to check - # them anyway than to filter them out. - find "$pkg_dir/$pkg_name/" -type f 2>/dev/null | + # We create two separate files for storing dependency information. + # + # 'lddfile' is where we will be storing the output of ldd, so that we can + # reference it later. + # + # 'dep_file_list' is where we will be listing the needed files which we will + # be passing to grep. + # + lddfile=$(_tmp_create lddfile) dep_file_list=$(_tmp_create dfl) + + pkg_fix_deps_find() { + # We run the similar command twice, might as well be a function. + # Basically runs find on the package directory and executes the + # given command. + end=+; [ "$1" = ldd ] && end=';' - while read -r file; do - case ${elf_prog:-ldd} in - *readelf) "$elf_prog" -d "$file" 2>/dev/null ;; - *) ldd "$file" 2>/dev/null ;; - esac | - while read -r dep; do - # Skip lines containing 'ldd'. - [ "${dep##*ldd*}" ] || continue - case $dep in *NEEDED*\[*\] | *'=>'*) ;; *) continue; esac - - # readelf output: - # 0x0000 (NEEDED) Shared library: [libc.so] - dep=${dep##*\[} - dep=${dep%%\]*} - - # ldd output: - # libc.so => /lib/ld-musl-x86_64.so.1 - dep=${dep#* => } - dep=${dep% *} - - # Figure out which package owns the file. Skip file if it is owned - # by the current package. This also handles cases where a '*-bin' - # package exists on the system, so the package manager doesn't think - # that the package we are building depends on the *-bin version of - # itself, or any other renamed versions of the same software. - pkg_owner -l "/${dep#/}\$" "$PWD/manifest" >/dev/null && continue - pkg_owner -l "/${dep#/}\$" "$@" ||: - done ||: - done >> depends + # Get a list of binaries and libraries, false files will be found, + # however it's faster to get 'ldd' to check them anyway than to filter + # them out. + # + # We are terminating exec, so no worries. + # shellcheck disable=2067 + find "$pkg_dir/$pkg_name/" -type f -exec "$@" {} "$end" 2>/dev/null | + sed 's/([^)]*) *$//' | sort -u + } + + # Record all the dependencies in the 'lddfile'. This will include all + # dependencies, including non-direct ones. Unless the user prefers ldd, + # readelf will be used to filter the non-direct dependencies out. + pkg_fix_deps_find ldd -- > "$lddfile" + + case "$elf_prog" in + *readelf) pkg_fix_deps_find "$elf_prog" -d ;; + *) cat "$lddfile" + esac | while read -r dep; do + # Skip lines containing 'ldd'. + [ "${dep##*ldd*}" ] || continue + case $dep in *NEEDED*\[*\] | *'=>'*) ;; *) continue; esac + + # readelf output: + # 0x0000 (NEEDED) Shared library: [libc.so] + dep=${dep##*\[} + dep=${dep%%\]*} + + # Retrieve the fullpath of the library from our ldd buffer. + case $elf_prog in + *readelf) dep=$(pkg_fix_deps_fullpath "$dep" < "$lddfile") + esac + + # ldd output: + # libc.so => /lib/ld-musl-x86_64.so.1 + dep=${dep#* => } + dep=${dep% *} + + # Figure out which package owns the file. Skip file if it is owned + # by the current package. This also handles cases where a '*-bin' + # package exists on the system, so the package manager doesn't think + # that the package we are building depends on the *-bin version of + # itself, or any other renamed versions of the same software. + pkg_owner -l "/${dep#/}\$" "$PWD/manifest" >/dev/null && continue + out "/${dep#/}\$" + done >> "$dep_file_list" + + # We write all the files into 'dep_file_list' so that we don't need to call + # grep on our entire database manifest hundreds of times. + pkg_owner_multi -lf "$dep_file_list" "$@" >> depends # Remove duplicate entries from the new depends file. # This removes duplicate lines looking *only* at the @@ -946,7 +1174,7 @@ pkg_fix_deps() { sort -uk1,1 -o depends depends 2>/dev/null ||: # Display a diff of the new dependencies against the old ones. - diff -U 3 "$dep_file" depends 2>/dev/null ||: + execv diff -U 3 "$dep_file" depends 2>/dev/null ||: # Remove the depends file if it is empty. [ -s depends ] || rm -f depends @@ -968,7 +1196,7 @@ pkg_manifest() ( # sed: Remove the first character in each line (./dir -> /dir) and # remove all lines which only contain '.'. find . -type d -exec printf '%s/\n' {} + -o -print | - sort -r | sed '/^\.\/$/d;ss.ss' > "${2:-$pkg_dir}/$1/$pkg_db/$1/manifest" + sort -r | sed '/^\.\/*$/d;ss.ss' > "${2:-$pkg_dir}/$1/$pkg_db/$1/manifest" ) pkg_etcsums() ( @@ -980,10 +1208,16 @@ pkg_etcsums() ( # /etc/ directory for use in "smart" handling of these files. log "$1" "Generating etcsums" + # Try to get the digest algorithm from the installed etcsums file. This + # makes sure that old packages continue to have the same digest algorithm + # and not a bunch of '.new' files are installed. It's not foolproof at all, + # but at least it keeps the /etc directory as clean as possible. + digest=$(_get_digest "$sys_db/$1/etcsums") || digest=b3sum + case $digest in b3sum) out "%BLAKE3"; esac > "$pkg_dir/$1/$pkg_db/$1/etcsums" find etc -type f | while read -r file; do - sh256 "$file" - done > "$pkg_dir/$1/$pkg_db/$1/etcsums" + "$digest" "$file" + done >> "$pkg_dir/$1/$pkg_db/$1/etcsums" ) pkg_tar() { @@ -992,22 +1226,12 @@ pkg_tar() { log "$1" "Creating tarball" # Read the version information to name the package. - read -r version release < "$(pkg_find "$1")/version" + read -r version release < "$pkg_dir/$1/$pkg_db/$1/version" # Create a tarball from the contents of the built package. - "$tar" cf - -C "$pkg_dir/$1" . | - case $CPT_COMPRESS in - bz2) bzip2 -z ;; - xz) xz -zT 0 ;; - gz) gzip -6 ;; - zst) zstd -3 ;; - lz) lzip -6 ;; - *) gzip -6 ;; # Fallback to gzip - esac \ - > "$bin_dir/$1#$version-$release.tar.$CPT_COMPRESS" - + cd "$pkg_dir/$1" + pax -w . | compress > "$bin_dir/$1#$version-$release.tar.$CPT_COMPRESS" log "$1" "Successfully created tarball" - run_hook post-package "$1" "$bin_dir/$1#$version-$release.tar.$CPT_COMPRESS" } @@ -1026,6 +1250,7 @@ pkg_build() { # separately from those detected as dependencies. explicit="$explicit $pkg " } done + pkg_depends_commit [ "$pkg_update" ] || explicit_build=$explicit @@ -1203,10 +1428,12 @@ pkg_checksums() { [ -f "$repo_dir/sources" ] || return 0 + case ${2:-b3sum} in b3sum) out "%BLAKE3"; esac + while read -r src _ || [ "$src" ]; do - # Comment. - if [ -z "${src##\#*}" ]; then - continue + + # Skip checksums if it's a comment, or a VCS repository. + if _re "$re_vcs_or_com" "$src"; then continue # File is local to the package. elif [ -f "$repo_dir/$src" ]; then @@ -1216,17 +1443,14 @@ pkg_checksums() { elif [ -f "$src_dir/$1/${src##*/}" ]; then src_path=$src_dir/$1 - # File is a git repository. - elif [ -z "${src##git+*}" ]; then continue - # Die here if source for some reason, doesn't exist. else die "$1" "Couldn't find source '$src'" fi - # An easy way to get 'sha256sum' to print with the 'basename' + # An easy way to get 'b3sum' to print with the 'basename' # of files is to 'cd' to the file's directory beforehand. - (cd "$src_path" && sh256 "${src##*/}") || + (cd "$src_path" && "${2:-b3sum}" "${src##*/}") || die "$1" "Failed to generate checksums" done < "$repo_dir/sources" } @@ -1234,13 +1458,18 @@ pkg_checksums() { pkg_verify() { # Verify all package checksums. This is achieved by generating a new set of # checksums and then comparing those with the old set. - verify_cmd="NR==FNR{a[\$1];next}/^git .*/{next}!((\$1)in a){exit 1}" + vcmd="NR==FNR{a[\$1];next}/^git .*/{next}!((\$1)in a){exit 1}END{if(NR/2!=FNR)exit 1}" for pkg; do repo_dir=$(pkg_find "$pkg") + [ -f "$repo_dir/sources" ] || continue - pkg_checksums "$pkg" | awk "$verify_cmd" - "$repo_dir/checksums" || { + # Determine the type of digest algorithm from the checksums file to do + # verification with. + digest="$(_get_digest "$repo_dir/checksums")" + + pkg_checksums "$pkg" "$digest" | awk "$vcmd" - "$repo_dir/checksums" || { log "$pkg" "Checksum mismatch" # Instead of dying above, log it to the terminal. Also define a @@ -1256,6 +1485,9 @@ pkg_conflicts() { # Check to see if a package conflicts with another. log "$1" "Checking for package conflicts" + c_manifest=$(_tmp_create conflict-manifest) + c_conflicts=$(_tmp_create conflicts) + # Filter the tarball's manifest and select only files # and any files they resolve to on the filesystem # (/bin/ls -> /usr/bin/ls). @@ -1275,9 +1507,9 @@ pkg_conflicts() { # Combine the dirname and file values, and print them into the # temporary manifest to be parsed. - printf '%s/%s\n' "${dirname#$CPT_ROOT}" "${file##*/}" + printf '%s/%s\n' "${dirname#"$CPT_ROOT"}" "${file##*/}" - done < "$tar_dir/$1/$pkg_db/$1/manifest" > "$CPT_TMPDIR/$pid/manifest" + done < "$tar_dir/$1/$pkg_db/$1/manifest" > "$c_manifest" p_name=$1 @@ -1286,7 +1518,7 @@ pkg_conflicts() { # shellcheck disable=2046,2086 set -- $(set +f; pop "$sys_db/$p_name/manifest" from "$sys_db"/*/manifest) - [ -s "$CPT_TMPDIR/$pid/manifest" ] || return 0 + [ -s "$c_manifest" ] || return 0 # In rare cases where the system only has one package installed # and you are reinstalling that package, grep will try to read from @@ -1302,13 +1534,12 @@ pkg_conflicts() { # Store the list of found conflicts in a file as we will be using the # information multiple times. Storing it in the cache dir allows us # to be lazy as they'll be automatically removed on script end. - "$grep" -Fxf "$CPT_TMPDIR/$pid/manifest" -- "$@" > "$CPT_TMPDIR/$pid/conflict" ||: - + sed '/\/$/d' "$@" | sort "$c_manifest" - | uniq -d > "$c_conflicts" ||: # Enable alternatives automatically if it is safe to do so. # This checks to see that the package that is about to be installed # doesn't overwrite anything it shouldn't in '/var/db/cpt/installed'. - "$grep" -q ":/var/db/cpt/installed/" "$CPT_TMPDIR/$pid/conflict" || + "$grep" -q "/var/db/cpt/installed/" "$c_conflicts" || choice_auto=1 # Use 'grep' to list matching lines between the to @@ -1337,7 +1568,7 @@ pkg_conflicts() { # this work. # # Pretty nifty huh? - while IFS=: read -r _ con; do + while read -r con; do printf '%s\n' "Found conflict $con" # Create the "choices" directory inside of the tarball. @@ -1359,13 +1590,13 @@ pkg_conflicts() { log "this must be fixed in $p_name. Contact the maintainer" die "by checking 'git log' or by running 'cpt-maintainer'" } - done < "$CPT_TMPDIR/$pid/conflict" + done < "$c_conflicts" # Rewrite the package's manifest to update its location # to its new spot (and name) in the choices directory. pkg_manifest "$p_name" "$tar_dir" 2>/dev/null - elif [ -s "$CPT_TMPDIR/$pid/conflict" ]; then + elif [ -s "$c_conflicts" ]; then log "Package '$p_name' conflicts with another package" "" "!>" log "Run 'CPT_CHOICE=1 cpt i $p_name' to add conflicts" "" "!>" die "as alternatives." @@ -1398,15 +1629,22 @@ pkg_swap() { # its manifest file to reflect this. We then resort this file # so no issues arise when removing packages. cp -Pf "$CPT_ROOT/$2" "$pkg_owns>${alt#*>}" - sed "s#^$(regesc "$2")\$#${PWD#$CPT_ROOT}/$pkg_owns>${alt#*>}#" \ + sed "s#^$(regesc "$2")\$#${PWD#"$CPT_ROOT"}/$pkg_owns>${alt#*>}#" \ "../installed/$pkg_owns/manifest" | sort -r -o "../installed/$pkg_owns/manifest" + else + # If the file doesn't exist, we assume that there was a previous owner, + # but the package was then removed. We want the message to be short + # and clear, I thought of writing "Swapping [...] from 'null' to '$1'", + # but that would probably sound more like a package manager bug. Instead + # we are printing the message below which should be informative enough. + log "Installing '$2' from '$1'" fi # Convert the desired alternative to a real file and rewrite # the manifest file to reflect this. The reverse of above. mv -f "$alt" "$CPT_ROOT/$2" - sed "s#^${PWD#$CPT_ROOT}/$(regesc "$alt")\$#$2#" "../installed/$1/manifest" | + sed "s#^${PWD#"$CPT_ROOT"}/$(regesc "$alt")\$#$2#" "../installed/$1/manifest" | sort -r -o "../installed/$1/manifest" } @@ -1420,23 +1658,25 @@ pkg_etc() { mkdir -p "$CPT_ROOT/$dir" done + digest=$(_get_digest "$_etcsums") || digest=b3sum + # Handle files in /etc/ based on a 3-way checksum check. find etc ! -type d | while read -r file; do - { sum_new=$(sh256 "$file") - sum_sys=$(cd "$CPT_ROOT/"; sh256 "$file") - sum_old=$("$grep" "$file$" "$mak_dir/c"); } 2>/dev/null ||: + { sum_new=$("$digest" "$file") + sum_sys=$(cd "$CPT_ROOT/"; "$digest" "$file") + sum_old=$("$grep" "$file$" "$_etcsums"); } 2>/dev/null ||: - log "$pkg_name" "Doing 3-way handshake for $file" - printf '%s\n' "Previous: ${sum_old:-null}" - printf '%s\n' "System: ${sum_sys:-null}" - printf '%s\n' "New: ${sum_new:-null}" + logv "$pkg_name" "Doing 3-way handshake for $file" + outv "Previous: ${sum_old:-null}" + outv "System: ${sum_sys:-null}" + outv "New: ${sum_new:-null}" # Use a case statement to easily compare three strings at # the same time. Pretty nifty. case ${sum_old:-null}${sum_sys:-null}${sum_new} in # old = Y, sys = X, new = Y "${sum_new}${sum_sys}${sum_old}") - log "Skipping $file" + logv "Skipping $file" continue ;; @@ -1446,7 +1686,7 @@ pkg_etc() { "${sum_old}${sum_old}${sum_old}"|\ "${sum_old:-null}${sum_sys}${sum_sys}"|\ "${sum_sys}${sum_old}"*) - log "Installing $file" + logv "Installing $file" new= ;; @@ -1489,10 +1729,11 @@ pkg_remove() { # remove anything from packages that create empty directories for a # purpose (such as baselayout). manifest_list="$(set +f; pop "$sys_db/$1/manifest" from "$sys_db/"*/manifest)" + dirs="$(_tmp_name "directories")" # shellcheck disable=2086 - [ "$manifest_list" ] && grep -h '/$' $manifest_list | sort -ur > "$mak_dir/dirs" + [ "$manifest_list" ] && grep -h '/$' $manifest_list | sort -ur > "$dirs" - run_hook pre-remove "$1" "$sys_db/$1" root + run_hook pre-remove "$1" "$sys_db/$1" while read -r file; do # The file is in '/etc' skip it. This prevents the package @@ -1500,7 +1741,7 @@ pkg_remove() { [ "${file##/etc/*}" ] || continue if [ -d "$CPT_ROOT/$file" ]; then - "$grep" -Fxq "$file" "$mak_dir/dirs" 2>/dev/null && continue + "$grep" -Fxq "$file" "$dirs" 2>/dev/null && continue rmdir "$CPT_ROOT/$file" 2>/dev/null || continue else rm -f "$CPT_ROOT/$file" @@ -1511,7 +1752,7 @@ pkg_remove() { # we no longer need to block 'Ctrl+C'. trap_set cleanup - run_hook post-remove "$1" "$CPT_ROOT/" root + run_hook post-remove "$1" "$CPT_ROOT/" log "$1" "Removed successfully" } @@ -1534,12 +1775,14 @@ pkg_install() { fi mkdir -p "$tar_dir/$pkg_name" + cd "$tar_dir/$pkg_name" + log "$pkg_name" "Extracting $tar_file" # Extract the tarball to catch any errors before installation begins. - decompress "$tar_file" | "$tar" xf - -C "$tar_dir/$pkg_name" + decompress "$tar_file" | pax -rpp - [ -f "$tar_dir/$pkg_name/$pkg_db/$pkg_name/manifest" ] || + [ -f "./$pkg_db/$pkg_name/manifest" ] || die "'${tar_file##*/}' is not a valid CPT package" # Ensure that the tarball's manifest is correct by checking that @@ -1547,13 +1790,13 @@ pkg_install() { [ "$CPT_FORCE" != 1 ] && log "$pkg_name" "Checking package manifest" && while read -r line; do # Skip symbolic links - [ -h "$tar_dir/$pkg_name/$line" ] || - [ -e "$tar_dir/$pkg_name/$line" ] || { - log "File $line missing from tarball but mentioned in manifest" "" "!>" - TARBALL_FAIL=1 - } - done < "$tar_dir/$pkg_name/$pkg_db/$pkg_name/manifest" - [ "$TARBALL_FAIL" ] && { + [ -h "./$line" ] || + [ -e "./$line" ] || { + log "File $line missing from tarball but mentioned in manifest" "" "!>" + tarball_fail=1 + } + done < "$pkg_db/$pkg_name/manifest" + [ "$tarball_fail" ] && { log "You can still install this package by setting CPT_FORCE variable" die "$pkg_name" "Missing files in manifest" } @@ -1562,18 +1805,17 @@ pkg_install() { # Make sure that all run-time dependencies are installed prior to # installing the package. - [ -f "$tar_dir/$pkg_name/$pkg_db/$pkg_name/depends" ] && + [ -f "$pkg_db/$pkg_name/depends" ] && [ "$CPT_FORCE" != 1 ] && while read -r dep dep_type || [ "$dep" ]; do [ "${dep##\#*}" ] || continue [ "$dep_type" ] || pkg_list "$dep" >/dev/null || install_dep="$install_dep'$dep', " - done < "$tar_dir/$pkg_name/$pkg_db/$pkg_name/depends" + done < "$pkg_db/$pkg_name/depends" [ "$install_dep" ] && die "$1" "Package requires ${install_dep%, }" - run_hook pre-install "$pkg_name" "$tar_dir/$pkg_name" root - + run_hook pre-install "$pkg_name" "$tar_dir/$pkg_name" pkg_conflicts "$pkg_name" log "$pkg_name" "Installing package incrementally" @@ -1585,8 +1827,8 @@ pkg_install() { # If the package is already installed (and this is an upgrade) make a # backup of the manifest and etcsums files. - cp -f "$sys_db/$pkg_name/manifest" "$mak_dir/m" 2>/dev/null ||: - cp -f "$sys_db/$pkg_name/etcsums" "$mak_dir/c" 2>/dev/null ||: + _manifest=$(_tmp_cp "$sys_db/$pkg_name/manifest" 2>/dev/null) ||: + _etcsums=$(_tmp_cp "$sys_db/$pkg_name/etcsums" 2>/dev/null) ||: # This is repeated multiple times. Better to make it a function. pkg_rsync() { @@ -1601,7 +1843,7 @@ pkg_install() { pkg_etc # Remove any leftover files if this is an upgrade. - "$grep" -vFxf "$sys_db/$pkg_name/manifest" "$mak_dir/m" 2>/dev/null | + "$grep" -vFxf "$sys_db/$pkg_name/manifest" "$_manifest" 2>/dev/null | while read -r file; do file=$CPT_ROOT/$file @@ -1638,144 +1880,231 @@ pkg_install() { "$sys_db/$pkg_name/post-install" ||: fi - run_hook post-install "$pkg_name" "$sys_db/$pkg_name" root + run_hook post-install "$pkg_name" "$sys_db/$pkg_name" log "$pkg_name" "Installed successfully" } -pkg_fetch() { - log "Updating repositories" +pkg_repository_update() { + # Function to update the given package repository. + cd "$1" + repo_type=$(pkg_vcs_info) + repo_root=${repo_type#"$PWD":} + repo_type=${repo_type##*:} repo_root=${repo_root%:*} + contains "$repos" "$repo_root" || { + repos="$repos $repo_root " + cd "$repo_root" + + "pkg_vcs_pull_$repo_type" + + # Repositories can contain a "Message of the Day" file in order to + # relay important information to their users. + ! [ -r "$repo_root/MOTD" ] || { + printf '%s\n%s\n%s\n\n' \ + "$(_multiply_char '=' 60)" \ + "Message of the Day [$PWD]" \ + "$(_multiply_char '=' 60)" + cat "$repo_root/MOTD" + printf '\n%s\n' "$(_multiply_char '=' 60)" + } + } +} - run_hook pre-fetch +pkg_vcs_clone_git() { + # $1: Clone URL + # $2: Branch or Commit Object + git init + git remote add origin "${1%[#@]*}" + case $2 in + @*) git fetch -t --depth=1 origin "${2#@}" || git fetch; set -- "$1" "${2#@}" ;; + *) git fetch --depth=1 origin "$2" || git fetch + esac + git checkout "${2:-FETCH_HEAD}" +} - # Create a list of all repositories. - # See [1] at top of script. - # shellcheck disable=2046,2086 - { IFS=:; set -- $CPT_PATH; IFS=$old_ifs ;} +pkg_vcs_clone_hg() { + # $1: Clone URL + # $2: Branch or Commit Object + hg clone -u "${2:-tip}" "${1%[#@]*}" . +} - # Update each repository in '$CPT_PATH'. It is assumed that - # each repository is 'git' tracked. - for repo; do - # Go to the root of the repository (if it exists). - cd "$repo" - cd "$(git rev-parse --show-toplevel 2>/dev/null)" 2>/dev/null || - cd "$(hg root 2>/dev/null)" 2>/dev/null ||: +pkg_vcs_clone_fossil() { + # $1: Clone URL + # $2: Branch or Commit Object + fossil open -f "${1%[#@]*}" "${2:-trunk}" +} - if [ -d .git ]; then +pkg_vcs_pull_fossil() { + # Pull function for Fossil. + log "$PWD" " " + [ "$(fossil remote 2>/dev/null)" != off ] || { + out "No remote, skipping." + return 0 + } - [ "$(git remote 2>/dev/null)" ] || { - log "$repo" " " - printf '%s\n' "No remote, skipping." - continue - } + # Ensure we have proper permissions to do the pull operation. + if [ -w "$PWD" ] && [ "$uid" != 0 ]; then + fossil pull + fossil update + else + pkg_vcs_as_root "fossil pull && fossil update" + fi +} - contains "$repos" "$PWD" || { - repos="$repos $PWD " +pkg_vcs_pull_git() { + # Pull function for Git. + if [ "$(git remote 2>/dev/null)" ]; then + # Display a message if signing is enabled for this repository. + case $(git config merge.verifySignatures) in + true) log "$PWD" "[signed] " ;; + *) log "$PWD" " " ;; + esac - # Display a tick if signing is enabled for this - # repository. - case $(git config merge.verifySignatures) in - true) log "$PWD" "[signed] " ;; - *) log "$PWD" " " ;; - esac + # Ensure we have proper permissions to do the pull operation. + if [ -w "$PWD" ] && [ "$uid" != 0 ]; then + git fetch + git merge + git submodule update --remote --init -f + else + pkg_vcs_as_root \ + "git fetch && git merge && git submodule update --remote --init -f" + fi + else + log "$PWD" " " + # Skip if there are no submodules + [ -f .gitmodules ] || { + out "No remote, skipping." + return 0 + } + if [ -w "$PWD" ] && [ "$uid" != 0 ]; then + git submodule update --remote --init -f + else + pkg_vcs_as_root "git submodule update --remote --init -f" + fi + fi +} - if [ -w "$PWD" ] && [ "$uid" != 0 ]; then - git fetch - git merge - git submodule update --remote --init -f +pkg_vcs_pull_hg() { + # Pull function for Mercurial. + log "$PWD" " " + [ "$(hg showconfig paths 2>/dev/null)" ] || { + out "No remote, skipping." + return 0 + } - else - [ "$uid" = 0 ] || log "$PWD" "Need root to update" - - # Find out the owner of the repository and spawn - # git as this user below. - # - # This prevents 'git' from changing the original - # ownership of files and directories in the rare - # case that the repository is owned by a 3rd user. - ( - user=$(_stat "$PWD") - - [ "$user" = root ] || - log "Dropping permissions to $user for pull" - - git_cmd="git fetch && git merge && git submodule update --remote --init -f" - case $su in *su) git_cmd="'$git_cmd'"; esac - - # Spawn a subshell to run multiple commands as - # root at once. This makes things easier on users - # who aren't using persist/timestamps for auth - # caching. - user=$user as_root sh -c "$git_cmd" - ) - fi - } - elif [ -d .hg ]; then + if [ -w "$PWD" ] && [ "$uid" != 0 ]; then + hg pull + hg update + else + pkg_vcs_as_root "hg pull && hg update" + fi +} - [ "$(hg showconfig paths 2>/dev/null)" ] || { - log "$repo" " " - printf '%s\n' "No remote, skipping." - continue - } +pkg_vcs_pull_rsync() { + # Pull function for rsync repositories. The details of our rsync + # repositories are explained in the user manual. + log "$PWD" " " - contains "$repos $PWD" || { - repos="$repos $PWD" + # Read remote repository address from the '.rsync' file. + read -r remote < .rsync + if [ -w "$PWD" ] && [ "$uid" != 0 ]; then + rsync -acvzzC --include=core --delete "$remote/" "$PWD" + else + pkg_vcs_as_root "rsync -acvzzC --include=core --delete \"$remote/\" \"$PWD\"" + fi +} - if [ -w "$PWD" ] && [ "$uid" != 0 ]; then - hg pull - hg update - else - [ "$uid" ] || log "$PWD" "Need root to update" +pkg_vcs_pull_local() { + # Local repository. We don't do a "pull" here, we just notify the user that + # this is the case. + log "$PWD" " " + out "Not a remote repository, skipping." +} - # We are going to do the same operation as above, to - # find the owner of the repository. - ( - user=$(_stat "$PWD") +pkg_vcs_as_root() ( + # Helper function for pkg_vcs_pull* functions used for proper + # privilege escalation. + [ "$uid" = 0 ] || log "$PWD" "Need root to update" - [ "$user" = root ] || - log "Dropping permissions to $user for pull" + # Find out the owner of the repository and spawn the operation as the user + # below. + # + # This prevents the VCS from changing the original ownership of files and + # directories in the rare case that the repository is owned by a third user. + user=$(_stat "$PWD") - hg_cmd="hg pull && hg update" + [ "$user" = root ] || log "Dropping permissions to $user for pull" + case ${su##*/} in su) set -- "'$1'"; esac - case $su in *su) hg_cmd="'$hg_cmd'"; esac - user=$user as_root sh -c "$hg_cmd" - ) - fi - } - elif [ -f .rsync ]; then - # If an .rsync_root file exists, we check that the repository root - # exists. If it does, we change to that directory to do the fetch. - # This way, we allow for partial repositories while making sure that - # we can fetch the repository in a single operation. - [ -f .rsync_root ] && { - read -r rsync_root < .rsync_root - [ -f "$rsync_root/.rsync" ] && cd "$rsync_root" - } - contains "$repos" "$PWD" || { - repos="$repos $PWD" - read -r remote < .rsync - if [ -w "$PWD" ] && [ "$uid" != 0 ]; then - rsync -acvzzC --include=core --delete "$remote/" "$PWD" - else - [ "$uid" = 0 ] || log "$PWD" "Need root to update" + # Spawn a subhsell to run multiple commands as root at once. This makes + # things easier on users who aren't using persist/timestamps for auth + # caching. + as_root sh -c "$@" +) - # Similar to the git update, we find the owner of - # the repository and spawn rsync as that user. - ( - user=$(_stat "$PWD") +pkg_vcs_info() { + # Finds and returns repository information for the current directory. It + # will return current directory, repository root, and the type of repository + # in a colon separated format. - [ "$user" = root ] || - log "Dropping permissions to $user for pull" + : "${repo_file:=$cac_dir/repository-cache}" + set -- - user=$user as_root rsync -acvzzC --include=core --delete "$remote/" "$PWD" - ) - fi - } - else - log "$repo" " " - printf '%s\n' "Not a remote repository, skipping." - fi - done + if [ "$CPT_REPO_CACHE" != 0 ] && information=$(grep "^$PWD:" "$repo_file" 2>/dev/null); then + # Repository information is already cached. + printf '%s\n' "$information" | sed 1q + return + elif rootdir=$(git rev-parse --show-toplevel 2>/dev/null); then + # Git repository + backend=git + elif rootdir=$(hg root 2>/dev/null); then + # Mercurial repository + backend=hg + elif rootdir=$(fossil info 2>/dev/null | grep ^local-root:); then + # Fossil repository + backend=fossil + + # We want to remove the initial spacing before the root directory, and + # the leading dash on the root directory. + rootdir=${rootdir#local-root: *} rootdir=${rootdir%/} + elif [ -f .rsync ]; then + backend=rsync + rootdir=$PWD + + # If an .rsync_root file exists, we check that the repository root + # exists. If it does, we change to that directory to do the fetch. + # This way, we allow for partial repositories while making sure that + # we can fetch the repository in a single operation. + [ -f .rsync_root ] && { + read -r rsync_root < .rsync_root + [ -f "$rsync_root/.rsync" ] && rootdir=$(_readlinkf "$rsync_root") + } + else + # Local repository + backend=local + rootdir=$PWD + fi + + # We cache all these information, so that we don't have to spend much time + # looking these up the next time we are doing it. If CPT_REPO_CACHE is set + # to 0, we will not write this cache. + [ "$CPT_REPO_CACHE" = 0 ] || set -- "$repo_file" + printf '%s:%s:%s\n' "$PWD" "$rootdir" "$backend" | tee -a "$@" +} + +pkg_fetch() { + log "Updating repositories" + + run_hook pre-fetch + + # Create a list of all repositories. + # See [1] at top of script. + # shellcheck disable=2046,2086 + { IFS=:; set -- $CPT_PATH; IFS=$old_ifs ;} + + # Update each repository in '$CPT_PATH'. It is assumed that + # each repository is 'git' tracked. + for repo; do pkg_repository_update "$repo"; done run_hook post-fetch } @@ -1786,7 +2115,12 @@ pkg_updates(){ # an update. [ "$CPT_FETCH" = 0 ] || pkg_fetch - log "Checking for new package versions" + # Be quiet if we are doing self update, no need to print the same + # information twice. We add this basic function, because we will be using it + # more than once. + _not_update () { [ "$cpt_self_update" ] || "$@" ;} + + _not_update log "Checking for new package versions" set +f @@ -1800,7 +2134,7 @@ pkg_updates(){ # Compare installed packages to repository packages. [ "$db_ver-$db_rel" != "$re_ver-$re_rel" ] && { - printf '%s\n' "$pkg_name $db_ver-$db_rel ==> $re_ver-$re_rel" + _not_update printf '%s\n' "$pkg_name $db_ver-$db_rel ==> $re_ver-$re_rel" outdated="$outdated$pkg_name " } done @@ -1821,6 +2155,13 @@ pkg_updates(){ exit 0 } + [ "$outdated" ] || { + log "Everything is up to date" + return + } + + _not_update log "Packages to update: ${outdated% }" + contains "$outdated" cpt && { log "Detected package manager update" log "The package manager will be updated first" @@ -1831,18 +2172,17 @@ pkg_updates(){ cpt-install cpt log "Updated the package manager" - log "Re-run 'cpt update' to update your system" - - exit 0 - } - - [ "$outdated" ] || { - log "Everything is up to date" - return + log "Re-executing the package manager to continue the update" + + # We export this variable so that cpt knows it's running for the second + # time. We make the new process promptless, and we avoid fetching + # repositories. We are assuming that the user was already prompted once, + # and that their repositories are up to date, or they have also passed + # the '-y' or '-n' flags themselves which leads to the same outcome. + export cpt_self_update=1 + exec cpt-update -yn } - log "Packages to update: ${outdated% }" - # Tell 'pkg_build' to always prompt before build. pkg_update=1 @@ -1858,12 +2198,12 @@ pkg_updates(){ } pkg_get_base() ( - # Print the packages defined in the /etc/cpt-base file. + # Print the packages defined in the CPT base file. # If an argument is given, it prints a space seperated list instead # of a list seperated by newlines. - # cpt-base is an optional file, return with success if it doesn't exist. - [ -f "$CPT_ROOT/etc/cpt-base" ] || return 0 + # CPT base is an optional file, return with success if it doesn't exist. + [ -f "$cpt_base" ] || return 0 # If there is an argument, change the format to use spaces instead of # newlines. @@ -1874,13 +2214,20 @@ pkg_get_base() ( # subshell. That is our purpose here, thank you very much. # shellcheck disable=SC2030 while read -r pkgname _; do + # Ignore comments [ "${pkgname##\#*}" ] || continue + + # Store the package list in arguments set -- "$@" "$pkgname" + + # Retrieve the dependency tree of the package, so they are listed as + # base packages too. This ensures that no packages are broken in a + # "base reset", and the user has a working base. deps=$(pkg_gentree "$pkgname" xn) for dep in $deps; do contains "$*" "$dep" || set -- "$@" "$dep" done - done < "$CPT_ROOT/etc/cpt-base" + done < "$cpt_base" # Format variable is intentional. # shellcheck disable=2059 @@ -1905,6 +2252,7 @@ pkg_gentree() ( esac done pkg_depends "$1" tree "$make_deps" + pkg_depends_commit # Unless 'f' is given, pop the package from the list so that we don't list # the package (for example if it's part of the base package list). Normally @@ -1916,7 +2264,9 @@ pkg_gentree() ( # shellcheck disable=2086 [ -z "${2##*f*}" ] || deps=$(pop "$1" from $deps) - eval set -- "$deps" + # Word splitting is intentional. + # shellcheck disable=2086 + set -- $deps pkg_order "$@" if [ "$reverse" ]; then eval set -- "$redro"; else eval set -- "$order"; fi [ "$1" ] || return 0 @@ -1929,8 +2279,11 @@ pkg_gentree() ( pkg_query_meta() { # Query the 'meta' file of the given meta package. If there is no meta file, # or the key being queried is unavailable, the function will return with - # error. - repo_dir=$(pkg_find "$1") + # error. Full path can be specified instead of package names. + case $1 in + */*) repo_dir=$1 ;; + *) repo_dir=$(pkg_find "$1") + esac [ -f "$repo_dir/meta" ] || return while IFS=': ' read -r key val; do case $key in @@ -1952,29 +2305,51 @@ pkg_clean() { rm -rf -- "${CPT_TMPDIR:=$cac_dir/proc}/$pid" } +_tmp_name() { + # Name a temporary file/directory + out "$tmp_dir/$1" +} + +_tmp_cp() { + # Copy given file to the temporary directory and return its name. If a + # second argument is not given, use the basename of the copied file. + _ret=${2:-${1##*/}} + _ret=$(_tmp_name "$_ret") + cp -p "$1" "$_ret" + out "$_ret" +} + +_tmp_create() { + # Create given file to the temporary directory and return its name + create_tmp + _ret=$(_tmp_name "$1") + :> "$_ret" || return 1 + out "$_ret" +} + +create_tmp() { + # Create the required temporary directories and set the variables which + # point to them. + mak_dir=$tmp_dir/build + pkg_dir=$tmp_dir/pkg + tar_dir=$tmp_dir/export + mkdir -p "$mak_dir" "$pkg_dir" "$tar_dir" +} + create_cache() { - # A temporary directory can be specified apart from the cache - # directory in order to build in a user specified directory. - # /tmp could be used in order to build on ram, useful on SSDs. - # The user can specify CPT_TMPDIR for this. + # DEPRECATED, use create_tmp() instead. # - # Create the required temporary directories and set the variables - # which point to them. - mkdir -p "${tmp_dir:=${CPT_TMPDIR:=$cac_dir/proc}/$pid}" - # If an argument is given, skip the creation of other cache directories. - # This here makes shellcheck extremely angry, so I am globally disabling - # SC2119. - [ "$1" ] || mkdir -p "${mak_dir:=$tmp_dir/build}" \ - "${pkg_dir:=$tmp_dir/pkg}" \ - "${tar_dir:=$tmp_dir/export}" - + [ "$1" ] || create_tmp } # main() { set -ef + # Package manager version. + cpt_version=@VERSION@ + # If a parser definition exists, let's run it ourselves. This makes sure we # get the variables as soon as possible. command -v parser_definition >/dev/null && { @@ -1983,24 +2358,44 @@ create_cache() { eval set -- "$REST" } - # Create the cache directories for CPT and set the variables which point - # to them. This is seperate from temporary directories created in - # create_cache(). That's because we need these variables set on most - # occasions. - mkdir -p "${cac_dir:=${CPT_CACHE:=${XDG_CACHE_HOME:-$HOME/.cache}/cpt}}" \ - "${src_dir:=$cac_dir/sources}" \ - "${log_dir:=$cac_dir/logs}" \ - "${bin_dir:=$cac_dir/bin}" - - # Set the location to the repository and package database. - pkg_db=var/db/cpt/installed - # The PID of the current shell process is used to isolate directories # to each specific CPT instance. This allows multiple package manager # instances to be run at once. Store the value in another variable so # that it doesn't change beneath us. pid=${CPT_PID:-$$} + # A temporary directory can be specified apart from the cache directory in + # order to build in a user specified directory. /tmp could be used in order + # to build on ram, useful on SSDs. The user can specify $CPT_TMPDIR for + # this. We now also support the usage of $XDG_RUNTIME_DIR, so the directory + # naming can be confusing to some. Here are possible $tdir names (by order + # of preference): + # + # 1. $CPT_TMPDIR + # 2. $XDG_RUNTIME_DIR/cpt + # 3. $XDG_CACHE_DIR/cpt/proc + # 4. $HOME/.cache/cpt/proc + # + # We create the main temporary directory here to avoid permission issues + # that can arise from functions that call as_root(). However, the + # $pid directories are special for each process and aren't created unless + # `create_tmp()` is used. + # + # We used to assign and create the directories at the same time using a + # shell hack, but it made the variables editable outside of the package + # manager, but we don't actually want that. Variables that are lower case + # aren't meant to be interacted or set by the user. + cac_dir=${CPT_CACHE:=${XDG_CACHE_HOME:-${HOME:?}/.cache}}/cpt + src_dir=$cac_dir/sources + log_dir=$cac_dir/logs + bin_dir=$cac_dir/bin + tdir=${CPT_TMPDIR:=${XDG_RUNTIME_DIR:-$cac_dir/proc}${XDG_RUNTIME_DIR:+/cpt}} + tmp_dir=$tdir/$pid + mkdir -p "$cac_dir" "$src_dir" "$log_dir" "$bin_dir" "$tdir" + + # Set the location to the repository and package database. + pkg_db=var/db/cpt/installed + # Force the C locale to speed up things like 'grep' which disable unicode # etc when this is set. We don't need unicode and a speed up is always # welcome. @@ -2015,17 +2410,6 @@ create_cache() { # POSIX correctness (grep quoted to avoid shellcheck false-positive). grep=$(command -v ggrep) || grep='grep' - # Prefer libarchive tar or GNU tar if installed as they are much - # much faster than busybox's implementation. Very much worth it if - # you value performance. - tar=$(command -v bsdtar || command -v gtar) || tar=tar - - # Prefer libarchive tar, GNU tar, or the POSIX defined pax for tarball - # extraction, as they can strip components, which is much much faster than - # our portability function. Our first preference is pax, because it is - # actually slightly faster than bsdtar and GNU tar. - extract=$(command -v pax || command -v "$tar") - # Figure out which 'sudo' command to use based on the user's choice or # what is available on the system. su=${CPT_SU:-$(command -v ssu || @@ -2045,6 +2429,15 @@ create_cache() { command -v llvm-readelf || command -v eu-readelf)"} || elf_prog=ldd + # Use one of the following programs to download package sources. Downloads + # are made using the `pkg_download()` function. + dl_prog=${CPT_DOWNLOADER:="$( + command -v curl || + command -v wget || + command -v wget2 || + command -v axel || + command -v aria2c)"} || dl_prog=curl + # Make note of the user's current ID to do root checks later on. # This is used enough to warrant a place here. uid=$(id -u) @@ -2054,10 +2447,7 @@ create_cache() { # Make sure that the CPT_ROOT doesn't end with a '/'. This might # break some operations. - [ -z "$CPT_ROOT" ] || [ "${CPT_ROOT##*/}" ] || { - warn "" "Your CPT_ROOT variable shouldn't end with '/'" - CPT_ROOT=${CPT_ROOT%/} - } + CPT_ROOT=${CPT_ROOT%"${CPT_ROOT##*[!/]}"} # Define an optional sys_arch variable in order to provide # information to build files with architectural information. @@ -2067,20 +2457,32 @@ create_cache() { # the get go. It will be created as needed by package installation. sys_db=$CPT_ROOT/$pkg_db + # CPT system configuration directory + cpt_confdir=$CPT_ROOT@SYSCONFDIR@/cpt + + # Backwards compatibility for the old cpt-base location + cpt_base=$CPT_ROOT/etc/cpt-base + [ -f "$cpt_confdir/base" ] && cpt_base=$cpt_confdir/base + + # Regular expression used in pkg_checksums() and pkg_sources() in order to + # identify VCS and comments + re_vcs_or_com='^(#|(fossil|git|hg)\+)' + # This allows for automatic setup of a CPT chroot and will # do nothing on a normal system. mkdir -p "$CPT_ROOT/" 2>/dev/null ||: - # Set a value for CPT_COMPRESS if it isn't set. - : "${CPT_COMPRESS:=gz}" - - # Unless being piped or the user specifically doesn't want colors, set - # colors. This can of course be overriden if the user specifically want - # colors during piping. - if { [ "$CPT_COLOR" != 0 ] && [ -t 1 ] ;} || [ "$CPT_COLOR" = 1 ]; then - colory="\033[1;33m" colorb="\033[1;36m" colre="\033[m" - fi + # Set the default compression to gzip, and warn the user if the value is + # invalid. + case ${CPT_COMPRESS:=gz} in + bz2|gz|xz|zst|lz) ;; + *) warn "'$CPT_COMPRESS' is not a valid CPT_COMPRESS value, falling back to 'gz'" + CPT_COMPRESS=gz + esac + # Set colors if they are to be enabled. + # shellcheck disable=2034 + colors_enabled && colory="\033[1;33m" colorb="\033[1;34m" colre="\033[m" colbold="\033[1m" } # If the library is being called with its own name, run arguments. diff --git a/src/cpt-list b/src/cpt-list index 4eb7cf6..fcb1f49 100755 --- a/src/cpt-list +++ b/src/cpt-list @@ -3,10 +3,11 @@ parser_definition() { setup REST help:usage -- \ - "usage: ${0##*/} [-c] pkg..." \ + "usage: ${0##*/} [-cq] pkg..." \ "or: ${0##*/} -C pkg true-statement false-statement" msg -- '' 'Options:' flag CURRENT -c --current -- "Use the current directory as a package" + flag quiet -q --quiet -- "Make the operation quiet" param PKG -C --check label:" -C,--check PKG TRUE FALSE" -- \ "Check if PKG exists and return the string of TRUE if"\ "it exists, and the string of FALSE if it doesn't." \ @@ -23,7 +24,7 @@ if [ "$PKG" ]; then printf %s "$2" fi else -[ "$CURRENT" ] && set -- "${PWD##*/}" - -pkg_list "$@" + [ "$CURRENT" ] && set -- "${PWD##*/}" + [ "$quiet" ] && exec >/dev/null 2>&1 + pkg_list "$@" fi diff --git a/src/cpt-remove b/src/cpt-remove index cce3739..f5a5abf 100755 --- a/src/cpt-remove +++ b/src/cpt-remove @@ -10,7 +10,7 @@ parser_definition() { if [ -f ./cpt-lib ]; then . ./cpt-lib; else . cpt-lib; fi -[ "$1" ] || { set -- "${PWD##*/}"; export CPT_PATH=${PWD%/*}:$CPT_PATH ;} +[ "$1" ] || { set -- "${PWD##*/}"; export CPT_PATH="${PWD%/*}:$CPT_PATH" ;} [ -w "$CPT_ROOT/" ] || [ "$uid" = 0 ] || { as_root "$0" "$@" @@ -20,3 +20,7 @@ if [ -f ./cpt-lib ]; then . ./cpt-lib; else . cpt-lib; fi create_cache pkg_order "$@" for pkg in $redro; do pkg_remove "$pkg" "${CPT_FORCE:-check}"; done + +# After all the removals are finished, run an end-remove hook. There may +# be some things that we may want to run, but not per package. +run_hook end-remove "" "$CPT_ROOT" diff --git a/src/cpt-search b/src/cpt-search index fbe0b04..9eeac75 100755 --- a/src/cpt-search +++ b/src/cpt-search @@ -2,26 +2,64 @@ # Search for a package parser_definition() { - setup REST help:usage -- "usage: ${0##*/} [pkg...]" + setup REST help:usage -- "usage: ${0##*/} [-dFoqs] [pkg...]" msg -- '' 'Options:' flag SEARCH_PATH -d "on:$CPT_PATH" -- "Do not search the installed package database" flag all -s --single init:=1 on:'' -- "Only show the first instance of a package" - flag others -o --others -- "Use the current directory as the package" \ + flag mode -q --query on:2 -- "Search packages making use of package descriptions" + flag fflag -F --fixed -- "Run query mode interpreting the given pattern as a" \ + "fixed string" + flag mode -o --others on:1 -- "Use the current directory as the package" \ "and show other instances" - global_options + global_options compact } if [ -f ./cpt-lib ]; then . ./cpt-lib; else . cpt-lib; fi -# The 'all' variable is set by the option parser. -# shellcheck disable=2154 -case $others in - '') for pkg; do pkg_find "$pkg" "${all:+all}"; done ;; - *) pkg_find "${PWD##*/}" all | - while read -r pkg_dir; do case $pkg_dir in - "$PWD") ;; - *) printf '%s\n' "$pkg_dir" - [ "$all" ] || exit 0 - esac - done + +# The 'all' and 'mode' variables are set by the option parser, and are never +# modified in the subshell. +# shellcheck disable=2154,2030,2031 +case $mode in + '') + # Default mode of operation. + for pkg; do pkg_find "$pkg" "${all:+all}"; done + ;; + 1) + # Use the current directory as the package and show other instances. + pkg_find "${PWD##*/}" all | + while read -r pkg_dir; do + case $pkg_dir in + "$PWD") ;; + *) printf '%s\n' "$pkg_dir" + [ "$all" ] || exit 0 + esac + done + ;; + 2) + # Make a partial string search using the name and the description of all + # packages. This is a "pretty information" mode, and its output is not + # meant to be used in scripting. There is a whole library meant for + # scripting. + pkg_find \* all | + while read -r pkg_dir; do + name=${pkg_dir##*/} + desc=$(pkg_query_meta "$pkg_dir" description ||:) + + # We pipe the name and description to the given query and + # continue if it's not a match + printf '%s %s\n' "$name" "$desc" | + "$grep" "-iq${fflag:+F}" -- "$1" || continue + + read -r ver rel < "$pkg_dir/version" + printf '%b%s%b@%s %s-%s\n %s\n\n' \ + "$colorb" "$name" "$colre" \ + "$pkg_dir" \ + "$ver" "$rel" \ + "$desc" + + # I don't know why someone use the '-s' flag on this operation + # mode, but go ahead. + [ "$all" ] || exit 0 + done esac diff --git a/src/cpt-update b/src/cpt-update index a8f9616..7e7ee16 100755 --- a/src/cpt-update +++ b/src/cpt-update @@ -6,11 +6,15 @@ parser_definition() { msg -- '' 'Options:' flag download_only -d --download -- "Only download updatable packages" flag CPT_FETCH -n --no-fetch on:0 off:0 init:@export -- "Do not refresh the repositories" - disp :pkg_fetch -o --only-fetch -- "Only fetch repositories" + flag pkg_fetch -o --only-fetch -- "Only fetch repositories" global_options } if [ -f ./cpt-lib ]; then . ./cpt-lib; else . cpt-lib; fi -create_cache -pkg_updates +if [ "$pkg_fetch" ]; then + pkg_fetch +else + create_cache + pkg_updates +fi diff --git a/tests/repository/contrib-dummy-pkg/meta b/tests/repository/contrib-dummy-pkg/meta new file mode 100644 index 0000000..305bd33 --- /dev/null +++ b/tests/repository/contrib-dummy-pkg/meta @@ -0,0 +1 @@ +description: This is a dummy package diff --git a/tools/mkdist.sh b/tools/mkdist.sh index c21e964..98f985c 100755 --- a/tools/mkdist.sh +++ b/tools/mkdist.sh @@ -25,6 +25,7 @@ fossil ls | while read -r file; do cp "$file" "$basedir/$file" done -tar cf "$basedir.tar" "$basedir" -xz -z "$basedir.tar" +cp docs/cpt.info "$basedir/docs/cpt.info" + +pax -w "$basedir" | xz -zT 0 > "$basedir.tar.xz" rm -rf -- "$basedir" diff --git a/tools/tool2man.sh b/tools/tool2man.sh index 38cdbf4..c4bbfb8 100755 --- a/tools/tool2man.sh +++ b/tools/tool2man.sh @@ -69,8 +69,8 @@ while read -r line; do esac done < "$file" -out ".Sh AUTHOR" ".An Cem Keylan Aq Mt cem@ckyln.com" -out ".Sh LICENSE" "See LICENSE for copyright information." +out ".Sh AUTHOR" ".An Cem Keylan Aq Mt cem@carbslinux.org" +out ".Sh COPYING" "See LICENSE for copyright information." out ".Sh SEE ALSO" ".Xr cpt 1" [ "$see" ] && for man in $see; do @@ -79,6 +79,6 @@ out ".Sh SEE ALSO" ".Xr cpt 1" out ".Pp" out "The full documentation of cpt is available as an info page." out "If either" ".Ic info" or ".Ic texinfo" -out "package is installed on your system, you can run" -out ".Bd -literal -offset indent" "info cpt" ".Ed" -out .Pp "to learn more about the package manager." +out "package is installed on your system, you can learn more about the package" +out "manager by running" +out ".Sq Sy info cpt ." diff --git a/www/index.md b/www/index.md index 99a1f13..e518540 100644 --- a/www/index.md +++ b/www/index.md @@ -1,5 +1,7 @@ # Home +[](https://builds.sr.ht/~carbslinux/cpt?) + CPT is the package management toolset written for Carbs Linux. Its aim is to provide a stable, powerful, and easily used library for package management that complements the tools that come with it. It has the following features: @@ -27,13 +29,13 @@ complements the tools that come with it. It has the following features: users with the `$PATH` variable. - **Serve repositories with your method** - Package repositories can be served - in a variety of formats, they can be either local, served with Git, or through - the `rsync` method, with Fossil integration to be added soon. + in a variety of formats, they can be either local, served with `git`, + `mercurial`, `fossil`, or through the `rsync` method. <hr> -### Latest Release: 6.0.2 ([2021-04-05](/timeline?c=6.0.2)) +### Latest Release: 7.0.2 ([2023-02-05](/timeline?c=7.0.2)) - [Download](/uvlist?byage=1) -- [Changelog](/wiki?name=Changelog) +- [Changelog](/doc/trunk/CHANGELOG.md) - [User Manual](https://carbslinux.org/docs/cpt) |