diff options
| -rw-r--r-- | CHANGELOG.md | 7 | ||||
| -rw-r--r-- | Makefile | 6 | ||||
| -rw-r--r-- | README | 58 | ||||
| -rw-r--r-- | README.md | 60 | ||||
| -rw-r--r-- | config.mk | 24 | ||||
| -rw-r--r-- | docs/cpt.org | 44 | ||||
| -rw-r--r-- | docs/cpt.texi | 270 | ||||
| -rw-r--r-- | docs/cpt.txt | 3 | ||||
| -rw-r--r-- | spec/01_lib_spec.sh | 2 | ||||
| -rw-r--r-- | src/cpt-lib.in | 287 | ||||
| -rwxr-xr-x | tests/repository/faulty-pkg/build | 3 | ||||
| -rw-r--r-- | tests/repository/faulty-pkg/checksums | 0 | ||||
| -rw-r--r-- | tests/repository/faulty-pkg/sources | 1 | ||||
| -rw-r--r-- | tests/repository/faulty-pkg/version | 1 | ||||
| -rwxr-xr-x | tools/mkdist.sh | 30 | ||||
| -rw-r--r-- | www/index.md | 39 |
16 files changed, 514 insertions, 321 deletions
diff --git a/CHANGELOG.md b/CHANGELOG.md index 90e5b16..9bf9e89 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -9,6 +9,13 @@ this project _somewhat_ adheres to [Semantic Versioning]. [Semantic Versioning]: https://semver.org/spec/v2.0.0.html +UNRELEASED +-------------------------------------------------------------------------------- + +### Added +- Added fossil support + + 6.0.4 - 2021-05-12 -------------------------------------------------------------------------------- @@ -24,11 +24,7 @@ tests/etc/cpt-hook: ln -s ../hook-file $@ dist: docs/cpt.info - mkdir "cpt-${VERSION}" - cp -r ${DISTFILES} "cpt-${VERSION}" - tar cf "cpt-${VERSION}.tar" "cpt-${VERSION}" - xz -z "cpt-${VERSION}.tar" - rm -rf -- "cpt-${VERSION}" + ./tools/mkdist.sh "${VERSION}" install: all test "${DOCS}" != yes || ${MAKE} -C docs install @@ -0,0 +1,58 @@ +Carbs Packaging Tools +===================== + +Package management toolset for Carbs Linux. Forked from KISS[1]. All usage +information can be obtained from the manual page. Refer to the ChangeLog[2] to +learn what's new. + + +Dependencies +------------ + +To build and use cpt, you need the following software: + +- rsync +- curl +- POSIX base utilities [coreutils, busybox, sbase, etc.] +- pax or tar [GNU tar, busybox, toybox, libarchive, etc.] +- Common compression utilities such as `gzip`, `bzip2`, `xz`, etc. + + +In order to build the documentation from source, you will need the following +software: + +- GNU Texinfo (for generating `.info` pages) +- GNU Emacs (for generating `.texi` and `.txt` pages) + +However, distributed tarballs come with info pages, and `.texi` and `.txt` files +are committed directly into the repository, meaning that you don't need Texinfo +if you are using a tarball, and you don't need Emacs as long as you don't edit +the documentation. + +You can also completely disable the build/installation of the documentation by +either passing `DOCS=no` to `make` or editing `config.mk` to disable it. + + +Installation +------------ + +In order to install CPT, you can run the following with `make`: + + make + make PREFIX=/usr install + + +Documentation +------------- + +The documentation for each tool along with some examples can be found on +manpages installed with the package manager. User manual of CPT can be found +online[3], or installed as both plain-text and as info pages. Without any +changes to the Makefile configuration those files can be found at: + +- /usr/local/share/docs/cpt/cpt.txt +- /usr/local/share/info/cpt.info + +[1]: https://github.com/kisslinux/kiss +[2]: https://fossil.carbslinux.org/cpt/log +[3]: https://carbslinux.org/docs/cpt diff --git a/README.md b/README.md deleted file mode 100644 index b7db05a..0000000 --- a/README.md +++ /dev/null @@ -1,60 +0,0 @@ -Carbs Packaging Tools --------------------------------------------------------------------------------- - -Package management toolset for Carbs Linux. Forked from [KISS]. All usage -information can be obtained from the manual page. For changes please refer to -the `CHANGELOG.md` file. - -[KISS]: https://github.com/kisslinux/kiss - - -Dependencies --------------------------------------------------------------------------------- - -To build and use cpt, you need the following software: - -- `rsync` -- `curl` -- POSIX base utilities [`coreutils`, `busybox`, `sbase`, etc.] -- `pax` or `tar` [GNU tar, busybox, toybox, libarchive, etc.] -- Common compression utilities such as `gzip`, `bzip2`, `xz`, etc. - - -In order to build the documentation, you will need the following software: - -- `texinfo` (for generating `.info` pages) -- `Emacs` (for generating `.texi` and `.txt` pages) - -However, distributed tarballs come with info pages, and `.texi` and `.txt` files -are committed directly into the repository, meaning that you don't need -`texinfo` if you are using a tarball, and you don't need Emacs as long as you -don't edit the documentation. - -You can also completely disable the build/installation of the documentation by -either passing `DOCS=no` to `make` or editing `config.mk` to disable it. - - -Directory Structure --------------------------------------------------------------------------------- - - / -- README, LICENSE, CHANGELOG - contrib/ -- for Shell scripts that wrap around cpt. - docs/ -- for documentation. - man/ -- for manual pages. - src/ -- for the tools that make up the package manager. - - -Defining Base --------------------------------------------------------------------------------- - -Tools such as cpt-orphans and cpt-reset define the base from the file -`/etc/cpt-base`. An example cpt-base file can be found from the root directory -of the repository, which is the default base for Carbs Linux. A user can modify -this file to fit their needs and redefine their base for the system. For -example, a user can decide that they want `sbase` instead of `busybox` for their -base, so if they reset their system, busybox will be removed instead of sbase. - -This file is used to ship a predefined base, while leaving the base to a user's -choice. However, it isn't installed by the Makefile so that the packagers may -define their own base, or so that a user can install cpt without using it as -their main package manager. @@ -1,5 +1,5 @@ # Carbs Packaging Tools -VERSION = 6.0.3 +VERSION = dev # Installation paths PREFIX = /usr/local @@ -16,25 +16,3 @@ MAKEINFO = makeinfo # Comment or change if you don't want to build/install the documentation DOCS = yes - -# Files to be added into the distribution tarball -DISTFILES = \ - .build.yml \ - .dir-locals.el \ - .editorconfig \ - .fossil-settings \ - .gitignore \ - .shellspec \ - CHANGELOG.md \ - LICENSE \ - Makefile \ - README.md \ - config.mk \ - contrib \ - cpt-base \ - docs \ - man \ - spec \ - src \ - tests \ - tools diff --git a/docs/cpt.org b/docs/cpt.org index 3949cea..3289942 100644 --- a/docs/cpt.org +++ b/docs/cpt.org @@ -8,6 +8,8 @@ #+TEXINFO_DIR_DESC: Carbs Package Management Library #+OPTIONS: html-scripts:nil todo:nil +#+MACRO: index (eval (format (if (org-export-derived-backend-p org-export-current-backend 'texinfo) "%s Index\n:PROPERTIES:\n:INDEX: %s\n:DESCRIPTION: %ss mentioned in this manual\n:END:\n" "%s%s%s :noexport:\n") $1 $2 $1)) + This is a reference document containing both the user-guide and the development manual for *Carbs Packaging Tools*. For development logs see [[https://git.carbslinux.org/cpt][the git repository]]. @@ -156,34 +158,66 @@ variables that alter the behaviour of =cpt=, some of them have separate sections to provide detailed information. - ~CPT_PATH~ :: + + #+VINDEX: CPT_PATH Set the locations of your repositories. It is similar to the ~PATH~ variable. - ~CPT_CACHE~ :: + + #+VINDEX: CPT_CACHE The cache directory for =cpt=. Default: ~$XDG_CACHE_HOME/cpt~. - ~CPT_CHOICE~ :: + + #+VINDEX: CPT_CHOICE If this is set to 0, a package installation will be aborted on conflicts. - ~CPT_COLOR~ :: + + #+VINDEX: CPT_COLOR If this is set to 1, =cpt= tools will be forced to display coloured output. If set to 0, they will be forced to display them without colours. Otherwise, =cpt= will output colour as long as it is outputting to a terminal. - ~CPT_DEBUG~ :: + + #+VINDEX: CPT_DEBUG If set to 1, temporary directories will not be removed after the operation. - ~CPT_FETCH~ :: + + #+VINDEX: CPT_FETCH If set to 0, ~cpt-update~ will not fetch repositories. - ~CPT_FORCE~ :: + + #+VINDEX: CPT_FORCE If set to 1, =cpt= tools will force operation. - ~CPT_HOOK~ :: + + #+VINDEX: CPT_HOOK Absolute path to the package manager hook file. - ~CPT_KEEPLOG~ :: + + #+VINDEX: CPT_KEEPLOG If set to 1, =cpt= will keep logs regardless of operation success. - ~CPT_PID~ :: + + #+VINDEX: CPT_PID Set the temporary build directory name. - ~CPT_PROMPT~ :: + + #+VINDEX: CPT_PROMPT If set to 0, =cpt= will not prompt you for anything. +- ~CPT_REPO_CACHE~ :: + + #+VINDEX: CPT_REPO_CACHE + If set to 0, =cpt= will not use or write repository information cache. - ~CPT_ROOT~ :: + + #+VINDEX: CPT_ROOT If this variable is set, =cpt= will assume the given path as the system root. - ~CPT_TEST~ :: + + #+VINDEX: CPT_TEST If set to 1, ~cpt-build~ will run tests whenever available. - ~CPT_TMPDIR~ :: + + #+VINDEX: CPT_TMPDIR The directory to create the temporary directories. *** =CPT_PATH= @@ -191,6 +225,7 @@ to provide detailed information. :DESCRIPTION: Set the locations of your repositories :END: +#+CINDEX: Setting up repositories Similar to the =PATH= variable, =cpt= find repositories from the =CPT_PATH= variable. Here is an example: @@ -213,6 +248,7 @@ This example brings us to the next section of this document. :DESCRIPTION: Prioritise package repositories :END: +#+CINDEX: package conflicts When you are using multiple repositories from multiple vendors, you will find out that some repositories have the same packages. =cpt= doesn't care about conflicting packages. If you want to build a package that exists on multiple @@ -520,11 +556,16 @@ installed. :DESCRIPTION: The test script for a package :END: +#+VINDEX: CPT_TEST Test files are mainly for the repository maintainer to test the packages, and will only run if the user has the =CPT_TEST= variable set, or the build is run with the =-t= or =--test= options. This script is run on the build directory. It is run right after the build script is finished. +* TODO Package Repositories :noexport: + +Repositories are + * Rsync Repositories :PROPERTIES: :DESCRIPTION: Information on using or creating rsync repositories @@ -1119,3 +1160,6 @@ with 1. $ pkg_query_meta cpt description Carbs Packaging Tools #+end_src + +* {{{index(Concept, cp)}}} +* {{{index(Variable,vr)}}} diff --git a/docs/cpt.texi b/docs/cpt.texi index c39b393..c479c11 100644 --- a/docs/cpt.texi +++ b/docs/cpt.texi @@ -14,7 +14,7 @@ Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.3 or any later version published by the Free Software Foundation; with no Invariant Sections, with no Front-Cover Texts and no Back-Cover Texts. A copy of the license is included in -the section entitled "GNU Free Documentation License." +the section entitled ``GNU Free Documentation License.'' @end quotation @end copying @@ -52,6 +52,8 @@ manual for @strong{Carbs Packaging Tools}. For development logs see @uref{https: * Rsync Repositories:: Information on using or creating rsync repositories * Comparison Between CPT and KISS:: * CPT Library:: Documentation of the Library +* Concept Index:: Concepts mentioned in this manual +* Variable Index:: Variables mentioned in this manual @detailmenu --- The Detailed Node Listing --- @@ -161,45 +163,45 @@ tools and their explanations. Here is an example call with extra scripts on my system: @example --> Carbs Packaging Tool --> add Commit the current directory as a new package --> alternatives List and swap to alternatives --> build Build a package --> bump Commit the current directory as a version bump --> cargo-urlgen Create static cargo sources for Rust packages --> cargolock-urlgen Convert the given Cargo.lock file to sources --> cat Concatanate package files in the installed package database --> changelog Print the git log of the specific package --> chbuild Create/destroy temporary chroots --> checkmissing Verify package manifests --> checksum Generate checksums --> chroot Enter a chroot --> commit Commit a package without the prefix of 'package:' --> depends Display a package's dependencies --> download Download sources for the given package --> exec Execute a command inside the alternatives system --> export Turn an installed package into a CPT tarball --> fork Fork a package to the current directory --> getchoice Prints the full path to a file in the alternatives system. --> install Install a package --> link Link a forked package's files to the other repository --> list List installed packages --> maintainer Find the maintainer of a package --> manifest Display all files owned by a package --> manifest-tree Display all files owned by a package with a tree view --> new Create a boilerplate CPT package --> orphans List orphaned packages --> owns Check which package owns a file --> rel Bump the release number of a package --> remove Remove a package --> repodepends Display a package's dependencies in the repository --> reporevdepends Display packages on the repository which depend on package --> reset Remove all packages except for the base --> revdepends Display packages which depend on package --> search Search for a package --> size Show the size on disk for a package --> source Extract sources of a given package to the current directory --> update Check for updates + -> Carbs Packaging Tool + -> add Commit the current directory as a new package + -> alternatives List and swap to alternatives + -> build Build a package + -> bump Commit the current directory as a version bump + -> cargo-urlgen Create static cargo sources for Rust packages + -> cargolock-urlgen Convert the given Cargo.lock file to sources + -> cat Concatanate package files in the installed package database + -> changelog Print the git log of the specific package + -> chbuild Create/destroy temporary chroots + -> checkmissing Verify package manifests + -> checksum Generate checksums + -> chroot Enter a chroot + -> commit Commit a package without the prefix of 'package:' + -> depends Display a package's dependencies + -> download Download sources for the given package + -> exec Execute a command inside the alternatives system + -> export Turn an installed package into a CPT tarball + -> fork Fork a package to the current directory + -> getchoice Prints the full path to a file in the alternatives system. + -> install Install a package + -> link Link a forked package's files to the other repository + -> list List installed packages + -> maintainer Find the maintainer of a package + -> manifest Display all files owned by a package + -> manifest-tree Display all files owned by a package with a tree view + -> new Create a boilerplate CPT package + -> orphans List orphaned packages + -> owns Check which package owns a file + -> rel Bump the release number of a package + -> remove Remove a package + -> repodepends Display a package's dependencies in the repository + -> reporevdepends Display packages on the repository which depend on package + -> reset Remove all packages except for the base + -> revdepends Display packages which depend on package + -> search Search for a package + -> size Show the size on disk for a package + -> source Extract sources of a given package to the current directory + -> update Check for updates @end example The documentation of @samp{cpt} aims to keep tool flags and related usage information @@ -270,34 +272,51 @@ to provide detailed information. @table @asis @item @code{CPT_PATH} +@vindex CPT_PATH Set the locations of your repositories. It is similar to the @code{PATH} variable. @item @code{CPT_CACHE} +@vindex CPT_CACHE The cache directory for @samp{cpt}. Default: @code{$XDG_CACHE_HOME/cpt}. @item @code{CPT_CHOICE} +@vindex CPT_CHOICE If this is set to 0, a package installation will be aborted on conflicts. @item @code{CPT_COLOR} +@vindex CPT_COLOR If this is set to 1, @samp{cpt} tools will be forced to display coloured output. If set to 0, they will be forced to display them without colours. Otherwise, @samp{cpt} will output colour as long as it is outputting to a terminal. @item @code{CPT_DEBUG} +@vindex CPT_DEBUG If set to 1, temporary directories will not be removed after the operation. @item @code{CPT_FETCH} +@vindex CPT_FETCH If set to 0, @code{cpt-update} will not fetch repositories. @item @code{CPT_FORCE} +@vindex CPT_FORCE If set to 1, @samp{cpt} tools will force operation. @item @code{CPT_HOOK} +@vindex CPT_HOOK Absolute path to the package manager hook file. @item @code{CPT_KEEPLOG} +@vindex CPT_KEEPLOG If set to 1, @samp{cpt} will keep logs regardless of operation success. @item @code{CPT_PID} +@vindex CPT_PID Set the temporary build directory name. @item @code{CPT_PROMPT} +@vindex CPT_PROMPT If set to 0, @samp{cpt} will not prompt you for anything. +@item @code{CPT_REPO_CACHE} +@vindex CPT_REPO_CACHE +If set to 0, @samp{cpt} will not use or write repository information cache. @item @code{CPT_ROOT} +@vindex CPT_ROOT If this variable is set, @samp{cpt} will assume the given path as the system root. @item @code{CPT_TEST} +@vindex CPT_TEST If set to 1, @code{cpt-build} will run tests whenever available. @item @code{CPT_TMPDIR} +@vindex CPT_TMPDIR The directory to create the temporary directories. @end table @@ -311,11 +330,12 @@ The directory to create the temporary directories. @node @samp{CPT_PATH} @subsection @samp{CPT_PATH} +@cindex Setting up repositories Similar to the @samp{PATH} variable, @samp{cpt} find repositories from the @samp{CPT_PATH} variable. Here is an example: @example -CPT_PATH=$HOME/repos/repo1:$HOME/repos/repo2:$HOME/repos/repo3 + CPT_PATH=$HOME/repos/repo1:$HOME/repos/repo2:$HOME/repos/repo3 @end example This is a simplistic and a structured example for repository locations, but it @@ -323,7 +343,7 @@ doesn't necessarily need to be as tidy as the example above. Here is an example for something a little more complex. @example -CPT_PATH=$HOME/repos/overrides:/var/db/cpt/repo/core:/var/db/cpt/repo/extra:$HOME/repos/personal + CPT_PATH=$HOME/repos/overrides:/var/db/cpt/repo/core:/var/db/cpt/repo/extra:$HOME/repos/personal @end example This example brings us to the next section of this document. @@ -333,6 +353,7 @@ This example brings us to the next section of this document. Repository preferences +@cindex package conflicts When you are using multiple repositories from multiple vendors, you will find out that some repositories have the same packages. @samp{cpt} doesn't care about conflicting packages. If you want to build a package that exists on multiple @@ -343,7 +364,7 @@ to install from your personal repository, you must set @samp{CPT_PATH} so that y personal repository is listed before the @samp{extra} repository. @example -CPT_PATH=$HOME/repos/personal:$HOME/repos/carbs/extra + CPT_PATH=$HOME/repos/personal:$HOME/repos/carbs/extra @end example @item @@ -357,12 +378,12 @@ The below example sets @samp{CPT_PATH} in a way that is easy to understand which repository comes first: @example -CPT_PATH=$HOME/repos/overrides -CPT_PATH=$CPT_PATH:$HOME/repos/carbs/core -CPT_PATH=$CPT_PATH:$HOME/repos/carbs/extra -CPT_PATH=$CPT_PATH:$HOME/repos/carbs/xorg -CPT_PATH=$CPT_PATH:$HOME/repos/personal -export CPT_PATH + CPT_PATH=$HOME/repos/overrides + CPT_PATH=$CPT_PATH:$HOME/repos/carbs/core + CPT_PATH=$CPT_PATH:$HOME/repos/carbs/extra + CPT_PATH=$CPT_PATH:$HOME/repos/carbs/xorg + CPT_PATH=$CPT_PATH:$HOME/repos/personal + export CPT_PATH @end example @end enumerate @@ -415,7 +436,7 @@ disadvantage is that there will be issues with multiple operations at the same time. So the best way to use this variable is during one-time @samp{cpt} calls. @example -CPT_PID=mesa cpt b mesa + CPT_PID=mesa cpt b mesa @end example By running the above, you will know that the created build directories will end @@ -564,10 +585,10 @@ parameter is optional. It is the directory that the source will be placed in. Here is the @samp{sources} file for the @samp{gst-plugins} package: @example -https://gstreamer.freedesktop.org/src/gst-plugins-good/gst-plugins-good-1.16.2.tar.xz good -https://gstreamer.freedesktop.org/src/gst-plugins-bad/gst-plugins-bad-1.16.2.tar.xz bad -https://gstreamer.freedesktop.org/src/gst-plugins-ugly/gst-plugins-ugly-1.16.2.tar.xz ugly -https://gstreamer.freedesktop.org/src/gst-libav/gst-libav-1.16.2.tar.xz libav + https://gstreamer.freedesktop.org/src/gst-plugins-good/gst-plugins-good-1.16.2.tar.xz good + https://gstreamer.freedesktop.org/src/gst-plugins-bad/gst-plugins-bad-1.16.2.tar.xz bad + https://gstreamer.freedesktop.org/src/gst-plugins-ugly/gst-plugins-ugly-1.16.2.tar.xz ugly + https://gstreamer.freedesktop.org/src/gst-libav/gst-libav-1.16.2.tar.xz libav @end example This file is read from the package manager as space seperated. Files that begin @@ -583,11 +604,11 @@ history pointer, git will checkout the relevant revision. So, @table @asis @item @samp{git+git://example.com/pub/repo@@v1.2.3} -will checkout the tag named "v1.2.3" +will checkout the tag named ``v1.2.3'' @item @samp{git+git://example.com/pub/repo#development} -will checkout the branch named "development" +will checkout the branch named ``development'' @item @samp{git+git://example.com/pub/repo#1a314s87} -will checkout the commit named "1a314s87" +will checkout the commit named ``1a314s87'' @end table Other files are assumed to be residing in the package directory. They should be @@ -609,24 +630,24 @@ of the package on a space seperated format. The contents of the file should look like below. @example -1.3.2 1 + 1.3.2 1 @end example @node depends @section depends This is a list of dependencies that must be installed before a package build. You -can append "make" after a dependency to mark a package is only required during +can append ``make'' after a dependency to mark a package is only required during the build process of a package. Packages marked as a make dependency can be -removed after the build. There are also "test" dependencies. These dependencies +removed after the build. There are also ``test'' dependencies. These dependencies are only installed if either the @samp{CPT_TEST} is set to 1, or the build is run with the @samp{-t} or @samp{--test} options. So, a package package could have the following @samp{depends} file: @example -linux-headers make -python test -zlib + linux-headers make + python test + zlib @end example @node meta @@ -660,6 +681,7 @@ installed. @node test @section test +@vindex CPT_TEST Test files are mainly for the repository maintainer to test the packages, and will only run if the user has the @samp{CPT_TEST} variable set, or the build is run with the @samp{-t} or @samp{--test} options. This script is run on the @@ -674,33 +696,33 @@ repository. This is used in order to fetch changes from the upstream. @samp{.rsy file looks like this for the core repository: @example -rsync://carbslinux.org/repo/core + rsync://carbslinux.org/repo/core @end example Rsync repositories have some few distinctions when it comes to fetching them. -They can be either synced individually or as a "root". There are 2 important +They can be either synced individually or as a ``root''. There are 2 important files, those are @samp{.rsync} and @samp{.rsync_root}. Here is the Carbs Linux rsync repository structure. @example - / - ----------------- - | | -.rsync core/ - ---------------- - | | - .rsync .rsync_root + / + ----------------- + | | + .rsync core/ + ---------------- + | | + .rsync .rsync_root @end example -Unlike git repositories, they don't have a defined "root" directory. This is +Unlike git repositories, they don't have a defined ``root'' directory. This is both an advantage and a disadvantage. This way, we can sync individual repositories, but that also means we need extra files to define root directories and repository locations. Here is the content for each of these files: @example -/.rsync: rsync://carbslinux.org/repo -/core/.rsync: rsync://carbslinux.org/repo/core -/core/.rsync_root: .. + /.rsync: rsync://carbslinux.org/repo + /core/.rsync: rsync://carbslinux.org/repo/core + /core/.rsync_root: .. @end example The @samp{.rsync_root} file on the core repository points to the upper directory. @@ -722,48 +744,48 @@ it through the rsync daemon. Here is a sample shell script that I use in order t sync repositories. Feel free to customize for your own use. @example -#!/bin/sh -HOSTNAME="rsync://carbslinux.org/repo" -GITDIR="/pub/git/repo" -SHAREDIR="/pub/share/repo" -git -C "$GITDIR" pull - -rsync -avcC --delete --include=core --exclude=.rsync,.rsync_root "$GITDIR/." "$SHAREDIR" - -printf '%s\n' "$HOSTNAME" > "$GITDIR/.rsync" -for dir in "$GITDIR/"*; do - [ -d "$dir" ] || continue - [ -f "$dir/.rsync" ] || - printf '%s/%s\n' "$HOSTNAME" "$@{dir##*/@}" > "$dir/.rsync" - printf '..\n' > "$dir/.rsync_root" -done + #!/bin/sh + HOSTNAME="rsync://carbslinux.org/repo" + GITDIR="/pub/git/repo" + SHAREDIR="/pub/share/repo" + git -C "$GITDIR" pull + + rsync -avcC --delete --include=core --exclude=.rsync,.rsync_root "$GITDIR/." "$SHAREDIR" + + printf '%s\n' "$HOSTNAME" > "$GITDIR/.rsync" + for dir in "$GITDIR/"*; do + [ -d "$dir" ] || continue + [ -f "$dir/.rsync" ] || + printf '%s/%s\n' "$HOSTNAME" "$@{dir##*/@}" > "$dir/.rsync" + printf '..\n' > "$dir/.rsync_root" + done @end example You can then create an @strong{rsync} user for serving the repositories. @example -$ adduser -SD rsync + $ adduser -SD rsync @end example Create @samp{/etc/rsyncd.conf} and a service configuration as well. @example -uid = rsync -gid = rsync -address = example.com -max connections = 10 -use chroot = yes - -[repo] - path = /pub/share/repo - comment = My repository + uid = rsync + gid = rsync + address = example.com + max connections = 10 + use chroot = yes + + [repo] + path = /pub/share/repo + comment = My repository @end example Create a service file at @samp{/etc/sv/rsync/run} (runit): @example -#!/bin/sh -e -exec rsync --daemon --no-detach + #!/bin/sh -e + exec rsync --daemon --no-detach @end example @node Comparison Between CPT and KISS @@ -833,8 +855,8 @@ You can call the library on your scripts by adding the following line to your files: @example -#!/bin/sh -e -. cpt-lib + #!/bin/sh -e + . cpt-lib @end example This will load the library inside your script, and will set some environment @@ -865,18 +887,18 @@ cpt-lib will handle the option parsing itself by calling @samp{getoptions} inside. Here is the proper way of doing it. @example -#!/bin/sh -e + #!/bin/sh -e -parser_definition() @{ - # The rest arguments MUST be defined as 'REST' - setup REST help:usage -- "usage: $@{0##*/@} [options] [pkg...]" - msg -- '' 'Options:' - flag CPT_TEST -t export:1 init:@@export -- "Enable tests" + parser_definition() @{ + # The rest arguments MUST be defined as 'REST' + setup REST help:usage -- "usage: $@{0##*/@} [options] [pkg...]" + msg -- '' 'Options:' + flag CPT_TEST -t export:1 init:@@export -- "Enable tests" - global_options -@} + global_options + @} -. cpt-lib + . cpt-lib @end example @node @samp{global_options()} @@ -928,9 +950,9 @@ with the user, it just exists to have a simple function to interact with other functions. @example -$ out "This is an example call" "How are you?" -This is an example call -How are you? + $ out "This is an example call" "How are you?" + This is an example call + How are you? @end example @node @samp{log()} @@ -1013,7 +1035,7 @@ regesc '^[$\' # Returns \^\[\$\\ @node @samp{pop()} @subsection @samp{pop()} -@samp{pop()} can be used to remove a word from a "string list" without a @samp{sed} +@samp{pop()} can be used to remove a word from a ``string list'' without a @samp{sed} call. Word splitting is intentional when using this function. @example @@ -1292,4 +1314,14 @@ $ pkg_query_meta cpt description Carbs Packaging Tools @end example -@bye
\ No newline at end of file +@node Concept Index +@chapter Concept Index + +@printindex cp + +@node Variable Index +@chapter Variable Index + +@printindex vr + +@bye diff --git a/docs/cpt.txt b/docs/cpt.txt index c04a52f..8620539 100644 --- a/docs/cpt.txt +++ b/docs/cpt.txt @@ -249,6 +249,9 @@ development manual for *Carbs Packaging Tools*. For development logs see Set the temporary build directory name. `CPT_PROMPT' If set to 0, `cpt' will not prompt you for anything. + `CPT_REPO_CACHE' + If set to 0, `cpt' will not use or write repository information + cache. `CPT_ROOT' If this variable is set, `cpt' will assume the given path as the system root. diff --git a/spec/01_lib_spec.sh b/spec/01_lib_spec.sh index c90eac1..0ff2752 100644 --- a/spec/01_lib_spec.sh +++ b/spec/01_lib_spec.sh @@ -104,7 +104,7 @@ Describe 'CPT Library' Describe '_stat()' It 'outputs the owner of the given file' - When call _stat README.md + When call _stat LICENSE The output should eq "$(id -un)" End End diff --git a/src/cpt-lib.in b/src/cpt-lib.in index d6fa0a4..328149e 100644 --- a/src/cpt-lib.in +++ b/src/cpt-lib.in @@ -892,11 +892,19 @@ pkg_strip() { done 2>/dev/null ||: } +pkg_fix_deps_fullpath() { + # Return the canonical path of libraries extracted by readelf. + while read -r dep _ rslv _; do + [ "$dep" = "$1" ] || continue + printf '%s\n' "$rslv" + done +} + pkg_fix_deps() { # Dynamically look for missing runtime dependencies by checking each binary # and library with either 'ldd' or 'readelf'. This catches any extra # libraries and or dependencies pulled in by the package's build suite. - log "$1" "Checking for missing dependencies" + log "$1" "Checking for missing dependencies (using ${elf_prog##*/})" # Go to the directory containing the built package to # simplify path building. @@ -921,9 +929,15 @@ pkg_fix_deps() { find "$pkg_dir/$pkg_name/" -type f 2>/dev/null | while read -r file; do + # We call ldd regardless here, because we also use it to retrieve the + # fullpath of a library when using readelf. Best use we have here is + # saving it in a buffer, so we don't use the dynamic loader everytime we + # need to reference it. + lddbuf=$(ldd -- "$file" 2>/dev/null) ||: + case ${elf_prog:-ldd} in *readelf) "$elf_prog" -d "$file" 2>/dev/null ;; - *) ldd "$file" 2>/dev/null ;; + *) pirntf '%s\n' "$lddbuf" ;; esac | while read -r dep; do # Skip lines containing 'ldd'. @@ -935,6 +949,12 @@ pkg_fix_deps() { dep=${dep##*\[} dep=${dep%%\]*} + # Retrieve the fullpath of the library from our ldd buffer. + case $elf_prog in + *readelf) line=$(printf '%s\n' "$lddbuf" | + pkg_fix_deps_fullpath "$line") + esac + # ldd output: # libc.so => /lib/ld-musl-x86_64.so.1 dep=${dep#* => } @@ -1653,138 +1673,179 @@ pkg_install() { log "$pkg_name" "Installed successfully" } -pkg_fetch() { - log "Updating repositories" - - run_hook pre-fetch +pkg_vcs_pull_fossil() { + # Pull function for Fossil. + [ "$(fossil remote 2>/dev/null)" != off ] || { + log "$repo" " " + printf '%s\n' "No remote, skipping." + return 0 + } - # Create a list of all repositories. - # See [1] at top of script. - # shellcheck disable=2046,2086 - { IFS=:; set -- $CPT_PATH; IFS=$old_ifs ;} + # Ensure we have proper permissions to do the pull operation. + if [ -w "$PWD" ] && [ "$uid" != 0 ]; then + fossil update + else + pkg_vcs_as_root "fossil update" + fi +} - # Update each repository in '$CPT_PATH'. It is assumed that - # each repository is 'git' tracked. - for repo; do - # Go to the root of the repository (if it exists). - cd "$repo" - cd "$(git rev-parse --show-toplevel 2>/dev/null)" 2>/dev/null || - cd "$(hg root 2>/dev/null)" 2>/dev/null ||: +pkg_vcs_pull_git() { + # Pull function for Git. + [ "$(git remote 2>/dev/null)" ] || { + log "$repo" " " + printf '%s\n' "No remote, skipping." + return 0 + } - if [ -d .git ]; then + # Display a message if signing is enabled for this repository. + case $(git config merge.verifySignatures) in + true) log "$PWD" "[signed] " ;; + *) log "$PWD" " " ;; + esac - [ "$(git remote 2>/dev/null)" ] || { - log "$repo" " " - printf '%s\n' "No remote, skipping." - continue - } + # Ensure we have proper permissions to do the pull operation. + if [ -w "$PWD" ] && [ "$uid" != 0 ]; then + git fetch + git merge + git submodule update --remote --init -f + else + pkg_vcs_as_root \ + "git fetch && git merge && git submodule update --remote --init -f" + fi +} - contains "$repos" "$PWD" || { - repos="$repos $PWD " +pkg_vcs_pull_hg() { + # Pull function for Mercurial. + [ "$(hg showconfig paths 2>/dev/null)" ] || { + log "$repo" " " + printf '%s\n' "No remote, skipping." + return 0 + } - # Display a tick if signing is enabled for this - # repository. - case $(git config merge.verifySignatures) in - true) log "$PWD" "[signed] " ;; - *) log "$PWD" " " ;; - esac + if [ -w "$PWD" ] && [ "$uid" != 0 ]; then + hg pull + hg update + else + pkg_vcs_as_root "hg pull && hg update" + fi +} - if [ -w "$PWD" ] && [ "$uid" != 0 ]; then - git fetch - git merge - git submodule update --remote --init -f +pkg_vcs_pull_rsync() { + # Pull function for rsync repositories. The details of our rsync + # repositories are explained in the user manual. - else - [ "$uid" = 0 ] || log "$PWD" "Need root to update" - - # Find out the owner of the repository and spawn - # git as this user below. - # - # This prevents 'git' from changing the original - # ownership of files and directories in the rare - # case that the repository is owned by a 3rd user. - ( - user=$(_stat "$PWD") + # Read remote repository address from the '.rsync' file. + read -r remote < .rsync + if [ -w "$PWD" ] && [ "$uid" != 0 ]; then + rsync -acvzzC --include=core --delete "$remote/" "$PWD" + else + pkg_vcs_as_root "rsync -acvzzC --include=core --delete \"$remote/\" \"$PWD\"" + fi +} - [ "$user" = root ] || - log "Dropping permissions to $user for pull" +pkg_vcs_pull_local() { + # Local repository. We don't do a "pull" here, we just notify the user that + # this is the case. + log "$repo" " " + printf '%s\n' "Not a remote repository, skipping." +} - git_cmd="git fetch && git merge && git submodule update --remote --init -f" - case $su in *su) git_cmd="'$git_cmd'"; esac +pkg_vcs_as_root() ( + # Helper function for pkg_vcs_pull* functions used for proper + # privilege escalation. + [ "$uid" = 0 ] || log "$PWD" "Need root to update" - # Spawn a subshell to run multiple commands as - # root at once. This makes things easier on users - # who aren't using persist/timestamps for auth - # caching. - user=$user as_root sh -c "$git_cmd" - ) - fi - } - elif [ -d .hg ]; then + # Find out the owner of the repository and spawn the operation as the user + # below. + # + # This prevents the VCS from changing the original ownership of files and + # directories in the rare case that the repository is owned by a third user. + user=$(_stat "$PWD") - [ "$(hg showconfig paths 2>/dev/null)" ] || { - log "$repo" " " - printf '%s\n' "No remote, skipping." - continue - } + [ "$user" = root ] || log "Dropping permissions to $user for pull" + case ${su##*/} in su) set -- "'$1'"; esac - contains "$repos $PWD" || { - repos="$repos $PWD" + # Spawn a subhsell to run multiple commands as root at once. This makes + # things easier on users who aren't using persist/timestamps for auth + # caching. + as_root sh -c "$@" +) - if [ -w "$PWD" ] && [ "$uid" != 0 ]; then - hg pull - hg update - else - [ "$uid" ] || log "$PWD" "Need root to update" +pkg_vcs_info() { + # Finds and returns repository information for the current directory. It + # will return current directory, repository root, and the type of repository + # in a colon separated format. - # We are going to do the same operation as above, to - # find the owner of the repository. - ( - user=$(_stat "$PWD") + : "${repo_file:=$cac_dir/repository-cache}" + set -- - [ "$user" = root ] || - log "Dropping permissions to $user for pull" + if [ "$CPT_REPO_CACHE" != 0 ] && information=$(grep "^$PWD:" "$repo_file" 2>/dev/null); then + # Repository information is already cached. + printf '%s\n' "$information" | sed 1q + return + elif rootdir=$(git rev-parse --show-toplevel 2>/dev/null); then + # Git repository + backend=git + elif rootdir=$(hg root 2>/dev/null); then + # Mercurial repository + backend=hg + elif rootdir=$(fossil info 2>/dev/null | grep ^local-root:); then + # Fossil repository + backend=fossil + + # We want to remove the initial spacing before the root directory, and + # the leading dash on the root directory. + rootdir=$(printf '%s\n' "$rootdir" | cut -d ' ' -f2-) + rootdir=${rootdir%/} + elif [ -f .rsync ]; then + backend=rsync + rootdir=$PWD + + # If an .rsync_root file exists, we check that the repository root + # exists. If it does, we change to that directory to do the fetch. + # This way, we allow for partial repositories while making sure that + # we can fetch the repository in a single operation. + [ -f .rsync_root ] && { + read -r rsync_root < .rsync_root + [ -f "$rsync_root/.rsync" ] && rootdir=$(_readlinkf "$rsync_root") + } + else + # Local repository + backend=local + rootdir=$PWD + fi - hg_cmd="hg pull && hg update" + # We cache all these information, so that we don't have to spend much time + # looking these up the next time we are doing it. If CPT_REPO_CACHE is set + # to 0, we will not write this cache. + [ "$CPT_REPO_CACHE" = 0 ] || set -- "$repo_file" + printf '%s:%s:%s\n' "$PWD" "$rootdir" "$backend" | tee -a "$@" +} - case $su in *su) hg_cmd="'$hg_cmd'"; esac - user=$user as_root sh -c "$hg_cmd" - ) - fi - } - elif [ -f .rsync ]; then - # If an .rsync_root file exists, we check that the repository root - # exists. If it does, we change to that directory to do the fetch. - # This way, we allow for partial repositories while making sure that - # we can fetch the repository in a single operation. - [ -f .rsync_root ] && { - read -r rsync_root < .rsync_root - [ -f "$rsync_root/.rsync" ] && cd "$rsync_root" - } - contains "$repos" "$PWD" || { - repos="$repos $PWD" - read -r remote < .rsync - if [ -w "$PWD" ] && [ "$uid" != 0 ]; then - rsync -acvzzC --include=core --delete "$remote/" "$PWD" - else - [ "$uid" = 0 ] || log "$PWD" "Need root to update" +pkg_fetch() { + log "Updating repositories" - # Similar to the git update, we find the owner of - # the repository and spawn rsync as that user. - ( - user=$(_stat "$PWD") + run_hook pre-fetch - [ "$user" = root ] || - log "Dropping permissions to $user for pull" + # Create a list of all repositories. + # See [1] at top of script. + # shellcheck disable=2046,2086 + { IFS=:; set -- $CPT_PATH; IFS=$old_ifs ;} - user=$user as_root rsync -acvzzC --include=core --delete "$remote/" "$PWD" - ) - fi - } - else - log "$repo" " " - printf '%s\n' "Not a remote repository, skipping." - fi + # Update each repository in '$CPT_PATH'. It is assumed that + # each repository is 'git' tracked. + for repo; do + # Go to the root of the repository. + cd "$repo" + repo_type=$(pkg_vcs_info) + repo_root=${repo_type#$PWD:} + repo_type=${repo_type##*:} repo_root=${repo_root%:*} + contains "$repos" "$repo_root" || { + repos="$repos $repo_root " + cd "$repo_root" + + "pkg_vcs_pull_$repo_type" + } done run_hook post-fetch diff --git a/tests/repository/faulty-pkg/build b/tests/repository/faulty-pkg/build new file mode 100755 index 0000000..4cf1c4f --- /dev/null +++ b/tests/repository/faulty-pkg/build @@ -0,0 +1,3 @@ +#!/bin/sh -e + +exit 1 diff --git a/tests/repository/faulty-pkg/checksums b/tests/repository/faulty-pkg/checksums new file mode 100644 index 0000000..e69de29 --- /dev/null +++ b/tests/repository/faulty-pkg/checksums diff --git a/tests/repository/faulty-pkg/sources b/tests/repository/faulty-pkg/sources new file mode 100644 index 0000000..8b13789 --- /dev/null +++ b/tests/repository/faulty-pkg/sources @@ -0,0 +1 @@ + diff --git a/tests/repository/faulty-pkg/version b/tests/repository/faulty-pkg/version new file mode 100644 index 0000000..2fb73a0 --- /dev/null +++ b/tests/repository/faulty-pkg/version @@ -0,0 +1 @@ +1 1 diff --git a/tools/mkdist.sh b/tools/mkdist.sh new file mode 100755 index 0000000..c21e964 --- /dev/null +++ b/tools/mkdist.sh @@ -0,0 +1,30 @@ +#!/bin/sh -e + +out() { printf '%s\n' "$@" >&2 ;} +die() { printf '\033[1;31mERR: \033[m%s\n' "$@" >&2; exit 1;} + +case "$1" in ''|-*) + die "Run this script by calling 'make dist' from the" \ + "root repository directory" +esac + +fossil stat >/dev/null 2>&1 || { + printf '\033[1;31mERR: \033[m%s\n' \ + "Distribution tarballs can only be generated using the Fossil repository." \ + "Exiting..." >&2 + exit 1 +} + +basedir=cpt-$1 +mkdir -p "$basedir" + +fossil ls | while read -r file; do + case "$file" in + */*) mkdir -p "$basedir/${file%/*}" + esac + cp "$file" "$basedir/$file" +done + +tar cf "$basedir.tar" "$basedir" +xz -z "$basedir.tar" +rm -rf -- "$basedir" diff --git a/www/index.md b/www/index.md new file mode 100644 index 0000000..e96aea6 --- /dev/null +++ b/www/index.md @@ -0,0 +1,39 @@ +# Home + +CPT is the package management toolset written for Carbs Linux. Its aim is to +provide a stable, powerful, and easily used library for package management that +complements the tools that come with it. It has the following features: + +- **POSIX shell library** - with its shell library, it is really easy to create + advanced scripts that wrap around the package manager and extend its + functionality. + +- **Powerful, but simple** - even with the multitudes of functionalities that + CPT provides, the tools provided aim to be as "low-interaction" as possible + and get out of your way. No user should spend countless hours on wiki pages + just to understand the proper way of installing a "masked" package. If the + user seeks to modify the build of a package, easy tools should be provided, + but those tools should not be required for basic functionality. + +- **Simple packaging system** - CPT has a easy to understand, simple, and static + packaging system, that makes it really easy to write and maintain packages. + Instead of complex `PKGBUILD` scripts or weird templates, packages are formed + of multiple little files that are easily written and easily parsed, even + without needing the help of the package manager itself. + +- **Easy Repository Management** - CPT makes it easy to create or use multiple + repositories at the same time. Repositories are added and prioritised by the + `$CPT_PATH` variable, an environment value that is already familiar to many + users with the `$PATH` variable. + +- **Serve repositories with your method** - Package repositories can be served + in a variety of formats, they can be either local, served with Git, Mercurial, + or through the `rsync` method, with Fossil integration to be added soon. + +<hr> + +### Latest Release: 6.0.3 ([2021-05-10](/timeline?c=6.0.3)) + +- [Download](/uvlist?byage=1) +- [Changelog](/doc/trunk/CHANGELOG.md) +- [User Manual](https://carbslinux.org/docs/cpt) |