diff options
Diffstat (limited to 'docs')
| -rw-r--r-- | docs/cpt.org | 940 | ||||
| -rw-r--r-- | docs/cpt.texi | 1114 |
2 files changed, 2054 insertions, 0 deletions
diff --git a/docs/cpt.org b/docs/cpt.org new file mode 100644 index 0000000..bff8ac9 --- /dev/null +++ b/docs/cpt.org @@ -0,0 +1,940 @@ +#+TITLE: Carbs Packaging Tools +#+SUBTITLE: User Manual +#+AUTHOR: Cem Keylan +#+EMAIL: cem@ckyln.com +#+TEXINFO_FILENAME: cpt.info +#+TEXINFO_DIR_CATEGORY: Development +#+TEXINFO_DIR_TITLE: Carbs Packaging Tools: (cpt) +#+TEXINFO_DIR_DESC: Carbs Package Management Library +#+DATE: {{{modification-time(%B %d\, %Y, 1)}}} + +This is a reference document containing both the user-guide and the development +manual for *Carbs Packaging Tools*. For development logs see [[https://git.carbslinux.org/cpt][the git repository]]. + +* Table of Contents :TOC:noexport: +- [[#copying][Copying]] +- [[#preface][Preface]] +- [[#usage][Usage]] + - [[#cpt-alternatives][cpt-alternatives]] + - [[#cpt-build][cpt-build]] + - [[#cpt-checksum][cpt-checksum]] + - [[#cpt-download][cpt-download]] + - [[#cpt-install][cpt-install]] + - [[#cpt-list][cpt-list]] + - [[#cpt-remove][cpt-remove]] + - [[#cpt-search][cpt-search]] + - [[#cpt-update][cpt-update]] + - [[#global-flags][Global Flags]] +- [[#environment-variables][Environment Variables]] + - [[#cpt_path][=CPT_PATH=]] + - [[#cpt_compress][=CPT_COMPRESS=]] + - [[#cpt_force][=CPT_FORCE=]] + - [[#cpt_pid][=CPT_PID=]] +- [[#hooks][Hooks]] + - [[#editing-the-build-file-during-pre-build][Editing the =build= file during pre-build]] +- [[#packaging-system][Packaging System]] + - [[#build][=build=]] + - [[#sources][=sources=]] + - [[#checksums][=checksums=]] + - [[#version][=version=]] + - [[#depends][=depends=]] + - [[#post-install][=post-install=]] + - [[#message][=message=]] + - [[#test][=test=]] +- [[#rsync-repositories][Rsync Repositories]] + - [[#setting-up-an-rsync-repository][Setting up an Rsync repository]] +- [[#cpt-library][CPT Library]] + - [[#calling-the-library][Calling the library]] + - [[#option-parsing][Option parsing]] + - [[#message-functions][Message functions]] + - [[#text-functions][Text functions]] + - [[#system-functions][System Functions]] + +* Copying +:PROPERTIES: +:COPYING: t +:END: + +Copyright \copy 2020 Cem Keylan + +#+begin_quote +Permission is granted to copy, distribute and/or modify this document under the +terms of the GNU Free Documentation License, Version 1.3 or any later version +published by the Free Software Foundation; with no Invariant Sections, with no +Front-Cover Texts and no Back-Cover Texts. A copy of the license is included in +the section entitled "GNU Free Documentation License." +#+end_quote + +* Preface +:PROPERTIES: +:DESCRIPTION: Introduction to Carbs Packaging Tools +:END: + +Carbs Linux uses its own package management toolchain named =cpt= which was +initially forked from the [[https://github.com/kisslinux/kiss][kiss]] package manager. Unlike =kiss=, however, its main +goal is being easily extendable. Instead of being a single file package manager, +it revolves around the shell library =cpt-lib=, and many tools that wrap around +it. This document aims to document both the usage of the distributed tools and +document the library functions. + +* Usage +:PROPERTIES: +:DESCRIPTION: Basic usage of Carbs Packaging Tools +:END: + +=cpt= is formed of many tools combined in a single environment, similar to +=git=. When you run =cpt= without any arguments, it will show all available +tools and their explanations. Here is an example call with extra scripts on my +system: + +#+BEGIN_EXAMPLE + -> Carbs Packaging Tool + -> add Commit the current directory as a new package + -> alternatives List and swap to alternatives + -> build Build a package + -> bump Commit the current directory as a version bump + -> cargo-urlgen Create static cargo sources for Rust packages + -> cargolock-urlgen Convert the given Cargo.lock file to sources + -> cat Concatanate package files in the installed package database + -> changelog Print the git log of the specific package + -> chbuild Create/destroy temporary chroots + -> checkmissing Verify package manifests + -> checksum Generate checksums + -> chroot Enter a chroot + -> commit Commit a package without the prefix of 'package:' + -> depends Display a package's dependencies + -> download Download sources for the given package + -> exec Execute a command inside the alternatives system + -> export Turn an installed package into a CPT tarball + -> fork Fork a package to the current directory + -> getchoice Prints the full path to a file in the alternatives system. + -> install Install a package + -> link Link a forked package's files to the other repository + -> list List installed packages + -> maintainer Find the maintainer of a package + -> manifest Display all files owned by a package + -> manifest-tree Display all files owned by a package with a tree view + -> new Create a boilerplate CPT package + -> orphans List orphaned packages + -> owns Check which package owns a file + -> rel Bump the release number of a package + -> remove Remove a package + -> repodepends Display a package's dependencies in the repository + -> reporevdepends Display packages on the repository which depend on package + -> reset Remove all packages except for the base + -> revdepends Display packages which depend on package + -> search Search for a package + -> size Show the size on disk for a package + -> source Extract sources of a given package to the current directory + -> update Check for updates +#+END_EXAMPLE + +** cpt-alternatives +:PROPERTIES: +:DESCRIPTION: List and swap to alternatives +:END: + +You can list and swap to alternatives using =cpt-alternatives=, or +=cpt a= for short. When run without alternatives, it will list +alternatives. It can read from standard input if =-= is given as an +argument. + +*Examples* + +1. List alternatives. + +#+BEGIN_EXAMPLE + $ cpt-alternatives + ncurses /usr/bin/clear + ncurses /usr/bin/reset +#+END_EXAMPLE + +2. Swap to =clear= from =ncurses=. + +#+BEGIN_EXAMPLE + $ cpt-alternatives ncurses /usr/bin/clear + -> Swapping '/usr/bin/clear' from 'busybox' to 'ncurses' +#+END_EXAMPLE + +3. Swap in bulk (all of =sbase=). + +#+BEGIN_EXAMPLE + $ cpt a | grep ^sbase | cpt a - +#+END_EXAMPLE + +** cpt-build +:PROPERTIES: +:DESCRIPTION: Build a package +:END: + +=cpt-build= will build given packages and their dependencies. If multiple +packages are specified, it will ask to install the packages as well. + +| Flags | Explanation | +|----------------+--------------| +| =-t=, =--test= | Run tests | + +** cpt-checksum +:PROPERTIES: +:DESCRIPTION: Generate checksums +:END: + +=cpt-checksum= will generate a =checksums= file from the package's sources. + +** cpt-download +:PROPERTIES: +:DESCRIPTION: Download sources for the given package +:END: + +=cpt-download= will download the sources of a package. + +** cpt-install +:PROPERTIES: +:DESCRIPTION: Install a package +:END: + +=cpt-install= will install given packages. + +** cpt-list +:PROPERTIES: +:DESCRIPTION: List installed packages +:END: + +When called without arguments, =cpt-list= will print all installed +packages. You can add package names as arguments to check whether they are +installed or not. In success, =cpt-list= will exit with status 0 if all +given packages are installed, it will return 1 if any of the given packages +aren't installed. + +| Flags | Explanation | +|-------------------+----------------------------------------| +| =-c=, =--current= | Use the current directory as a package | + +** cpt-remove +:PROPERTIES: +:DESCRIPTION: Remove a package +:END: + +=cpt-remove= will remove given packages. + +** cpt-search +:PROPERTIES: +:DESCRIPTION: Search for a package +:END: + +=cpt-search= will search for packages, it accepts regular expressions as well. + +| Flags | Explanation | +|------------------+-------------------------------------------| +| =-s=, =--single= | Only show the first instance of a package | + + +#+BEGIN_EXAMPLE + $ cpt-search 'alsa-*' + /var/db/cpt/repo/extra/alsa-lib + /var/db/cpt/repo/extra/alsa-utils + /var/db/cpt/installed/alsa-lib + /var/db/cpt/installed/alsa-utils + + $ cpt-search emacs + /home/cem/repos/main/community/emacs + /home/cem/repos/kiss-community/community/emacs + /var/db/cpt/installed/emacs + + $ cpt-search --single emacs + /home/cem/repos/main/community/emacs +#+END_EXAMPLE + +** cpt-update +:PROPERTIES: +:DESCRIPTION: Check for updates +:END: + +=cpt-update= will update the packages on your system. It fetches remote +repositories, and builds, and installs packages that have versions different +from the ones installed on the system. It doesn't check if the version string +is actually higher, it only checks whether they differ. + +| Flags | Explanation | +|----------------------+--------------------------------------| +| =-d=, =--download= | Only download updatable packages | +| =-n=, =--no-fetch= | Do not update remote repositories | +| =-o=, =--only-fetch= | Only fetch the repositories and exit | + +** Global Flags +:PROPERTIES: +:DESCRIPTION: Flags that work globally on some cpt utilities +:END: + +| Flags | Explanation | +|---------------------+----------------------------------| +| =-f=, =--force= | Force operation, [[=CPT_FORCE=][See =CPT_FORCE=]] | +| =-y=, =--no-prompt= | Do not prompt for confirmation | +| =-root CPT_ROOT= | Use an alternate root directory | +| =-h=, =--help= | Show this help message | +| =-v=, =--version= | Print version information | + +* Environment Variables +:PROPERTIES: +:DESCRIPTION: Change the behaviour of cpt through environment configuration +:END: + +Since there is no configuration file for cpt, the package manager is configured +through environment variables. These can be set per operation, or be set to your +shell configuration or =~/.profile=. Here are the environment variables that +alter the behaviour of =cpt=: + +| ENVIRONMENT VARIABLE | Effects | +|----------------------+-------------------------------------------------------------------------------| +| =CPT_PATH= | Set the locations of your repositories. It is similar to the =PATH= variable. | +| =XDG_CACHE_HOME= | Unless this is set, the =~/.cache= directory will be used instead. | +| =CPT_CACHE= | The cache directory for =cpt=. Default: =$XDG_CACHE_HOME/cpt= | +| =CPT_CHOICE= | If this is set to 0, a package installation will be aborted on conflicts. | +| =CPT_COMPRESS= | Program used to compress package tarballs. | +| =CPT_DEBUG= | If set to 1, temporary directories will not be removed after the operation. | +| =CPT_FETCH= | If set to 0, =cpt-update= will not fetch repositories. | +| =CPT_FORCE= | Force operation. | +| =CPT_HOOK= | Location for the hook file. | +| =CPT_KEEPLOG= | If set to 1, cpt will keep logs regardless of operation success. | +| =CPT_PID= | Set the temporary build directory name. | +| =CPT_PROMPT= | If set to 0, =cpt= will not prompt you for anything. | +| =CPT_ROOT= | If this variable is set, =cpt= will assume this as the system root. | +| =CPT_TEST= | If set to 1, =cpt-build= will run tests whenever available. | +| =CPT_TMPDIR= | The directory to create the temporary directories. | + +** =CPT_PATH= +:PROPERTIES: +:DESCRIPTION: Set the locations of your repositories +:END: + +Similar to the =PATH= variable, =cpt= find repositories from the =CPT_PATH= +variable. Here is an example: + +#+begin_src sh + CPT_PATH=$HOME/repos/repo1:$HOME/repos/repo2:$HOME/repos/repo3 +#+end_src + +This is a simplistic and a structured example for repository locations, but it +doesn't necessarily need to be as tidy as the example above. Here is an example +for something a little more complex. + +#+begin_src sh + CPT_PATH=$HOME/repos/overrides:/var/db/cpt/repo/core:/var/db/cpt/repo/extra:$HOME/repos/personal +#+end_src + +This example brings us to the next section of this document. + +*** Repository preferences +:PROPERTIES: +:DESCRIPTION: Prioritise package repositories +:END: + +When you are using multiple repositories from multiple vendors, you will find +out that some repositories have the same packages. =cpt= doesn't care about +conflicting packages. If you want to build a package that exists on multiple +repositories, =cpt= will build the first matching package. This means that if +=grep= package (for the sake of an example) exists on both +=$HOME/repos/personal= and =$HOME/repos/carbs/extra=, and you want +to install from your personal repository, you must set =CPT_PATH= so that your +personal repository is listed before the =extra= repository. + +#+begin_src sh + CPT_PATH=$HOME/repos/personal:$HOME/repos/carbs/extra +#+end_src + +*** Setting the =CPT_PATH= +:PROPERTIES: +:DESCRIPTION: Set the value of CPT_PATH on your shell configuration +:END: + +You can set the =CPT_PATH= variable on your shell configuration or your +=.profile= file in a way that is easy to read. + +The below example sets =CPT_PATH= in a way that is easy to understand which +repository comes first: + +#+begin_src sh + CPT_PATH=$HOME/repos/overrides + CPT_PATH=$CPT_PATH:$HOME/repos/carbs/core + CPT_PATH=$CPT_PATH:$HOME/repos/carbs/extra + CPT_PATH=$CPT_PATH:$HOME/repos/carbs/xorg + CPT_PATH=$CPT_PATH:$HOME/repos/personal + export CPT_PATH +#+end_src + +** =CPT_COMPRESS= +:PROPERTIES: +:DESCRIPTION: Compression tool to use in cpt +:END: + +When setting the =CPT_COMPRESS= value, you should set the name of the default +suffixes for the program. Available values are: + +- =gz= +- =zst= +- =bz2= +- =xz= + +Defaults to =gz=. + +** =CPT_FORCE= +:PROPERTIES: +:DESCRIPTION: Force operations on cpt +:END: + +If this is set to 1, some of the =cpt= tools will continue regardless of +errors or skip certain checks. Here are some examples: + +- =cpt-install= will install a package without verifying its manifest. +- =cpt-install= will install a package even when there are missing dependencies. +- =cpt-remove= will remove packages even when there are other packages that + depend on the current package. + +Defaults to 0. + +** =CPT_PID= +:PROPERTIES: +:DESCRIPTION: Set reproducible temporary directories +:END: + +If this variable is set, the temporary files will be created with this variable +as the suffix, instead of the PID of the =cpt= process. The advantage is that +you can know exactly where the build directory is located, while the +disadvantage is that there will be issues with multiple operations at the same +time. So the best way to use this variable is during one-time =cpt= calls. + +#+BEGIN_EXAMPLE + CPT_PID=mesa cpt b mesa +#+END_EXAMPLE + +By running the above, you will know that the created build directories will end +with the =*-mesa= suffix. + +* Hooks +:PROPERTIES: +:DESCRIPTION: Use hooks to customize the package manager operations +:END: + +Hooks can be used in order to change the runtime behaviour of the package manager. +There are a variety of package hooks, mostly self explanatory: + +- pre-build +- post-build +- build-fail +- pre-test +- test-fail +- pre-install +- post-install +- pre-remove +- post-remove +- pre-fetch +- post-fetch +- post-package + +In order to use hooks, you will need to set the =CPT_HOOK= variable pointing to +your hook file. Your hook file *MUST* be a POSIX shell script as its contents +are sourced by the package manager. + +The hook is given 3 variables when it is executed. Those are: + +| Variable | Explanation | +|----------+--------------------------------------------------------------| +| =$TYPE= | The type of the hook, (=pre-build=, =post-build=, etc.) | +| =$PKG= | The package that =cpt= is currently working on. Can be null. | +| =$DEST= | The destination of the operation. Can be null. | + +** Editing the =build= file during pre-build +:PROPERTIES: +:DESCRIPTION: Modify a package build with your hooks +:END: + +You can edit the =build= file during pre-build. The file is copied from the +repository to the build directory named as =.build.cpt=. You can use =sed= or +any other tool to edit the build file. After the build is complete, a =diff= +file will be placed to the package database named as =build.diff=. Here is an +example =build= file manipulation during the pre-build hook. + +#+BEGIN_SRC sh + cat <<EOF> .build.cpt + #!/bin/sh -e + + for patch in bash50-0??; do + patch -p0 < "\$patch" + done + + export LDFLAGS=-static + + ./configure \ + --prefix=/usr \ + --without-bash-malloc \ + --disable-nls + + export MAKEFLAGS="TERMCAP_LIB=/usr/lib/libncursesw.a $MAKEFLAGS" + + make + make DESTDIR="\$1" install + + ln -s bash "\$1/usr/bin/sh" + EOF +#+END_SRC + +* Packaging System +:PROPERTIES: +:DESCRIPTION: More detail on creating packages +:END: + +A package is formed of several files, these are: + +- [[=build=]] +- [[=sources=]] +- [[=checksums=]] +- [[=version=]] +- [[=depends=]] +- [[=post-install=]] +- [[=message=]] +- [[=test=]] + +Any other file can be added to the package directory at the discretion of the +package maintainer. Everything in the package directory will also be added to the +package database that is located on =/var/db/cpt/installed=. These can be +patches, configuration files, etc. + +** =build= +:PROPERTIES: +:DESCRIPTION: The build script +:END: + +Typically =build= files are shell scripts that run commands to prepare the source +code to be installed on the target system. Even though we will be assuming that +the =build= file is a POSIX shell script (for portability's sake), =build= +files can be any executable program from binary programs to =perl= scripts. + +The contents of a build script do not need to follow a certain rule for the +package manager, except for the fact that the user needs the permission to +execute the file. + +An important advice is to append an '-e' to the shebang (#!/bin/sh -e) so that +the build script exits on compilation error. + +Build is run with three arguments (=$#=) + +- Location of the package directory (DESTDIR) +- Package version +- System architecture + +** =sources= +:PROPERTIES: +:DESCRIPTION: The file containing package sources +:END: + +=sources= file is a list of files and sources that will be put to the build +directory during the build process. Those can be remote sources (such as tarballs), +git repositories, and files that reside on the package directory. + +The syntax is pretty simple for the =soures= file; =src dest=. The =dest= +parameter is optional. It is the directory that the source will be placed in. +Here is the =sources= file for the =gst-plugins= package: + +#+BEGIN_EXAMPLE + https://gstreamer.freedesktop.org/src/gst-plugins-good/gst-plugins-good-1.16.2.tar.xz good + https://gstreamer.freedesktop.org/src/gst-plugins-bad/gst-plugins-bad-1.16.2.tar.xz bad + https://gstreamer.freedesktop.org/src/gst-plugins-ugly/gst-plugins-ugly-1.16.2.tar.xz ugly + https://gstreamer.freedesktop.org/src/gst-libav/gst-libav-1.16.2.tar.xz libav +#+END_EXAMPLE + +This file is read from the package manager as space seperated. Files that begin +with a =#= comment are ignored. The first value points to the location of the +source. + +If it starts with a protcol url, (such as ftp:// http:// https://) it will be +downloaded with =curl=. + +If the source is a git repository, it shall be prefixed with a =git+= git(1) will +be used to do a shallow clone of the repository. If the commit is suffixed by a +history pointer, git will checkout the relevant revision. So, + +- =git+git://example.com/pub/repo@v1.2.3= :: will checkout the tag named "v1.2.3" +- =git+git://example.com/pub/repo#development= :: will checkout the branch named "development" +- =git+git://example.com/pub/repo#1a314s87= :: will checkout the commit named "1a314s87" + +Other files are assumed to be residing in the package directory. They should be +added with their paths relative to the package directory. + +** =checksums= +:PROPERTIES: +:DESCRIPTION: The file containing sha256sum of the sources +:END: + +checksums file is generated by the =cpt c pkg= command. It is generated +according to the order of the sources file. That's why you shouldn't be editing +it manually. The checksums file is created with the digests of the files using +the sha256 algorithm. + +** =version= +:PROPERTIES: +:DESCRIPTION: The file containing the version and the release numbers of a package +:END: + +The version file includes the version of the software and the release number of +of the package on a space seperated format. The contents of the file should look +like below. + +#+BEGIN_EXAMPLE + 1.3.2 1 +#+END_EXAMPLE + +** =depends= +:PROPERTIES: +:DESCRIPTION: The file containing the dependencies of a package +:END: + +This is a list of dependencies that must be installed before a package build. You +can append "make" after a dependency to mark a package is only required during +the build process of a package. Packages marked as a make dependency can be +removed after the build. There are also "test" dependencies. These dependencies +are only installed if either the =CPT_TEST= is set to 1, or the build is run +with the =-t= or =--test= options. So, a package package could have +the following =depends= file: + +#+BEGIN_EXAMPLE + linux-headers make + python test + zlib +#+END_EXAMPLE + +** =post-install= +:PROPERTIES: +:DESCRIPTION: The post-installation script +:END: + +=post-install= files have the same requirements as the build script. They +will be run after the package is installed as root (or as the user if the user +has write permissions on =CPT_ROOT=). + +** =message= +:PROPERTIES: +:DESCRIPTION: The post-installation message to be displayed +:END: + +This plaintext file will be outputted with =cat= after every package is +installed. + +** =test= +:PROPERTIES: +:DESCRIPTION: The test script for a package +:END: + +Test files are mainly for the repository maintainer to test the packages, and +will only run if the user has the =CPT_TEST= variable set, or the build is +run with the =-t= or =--test= options. This script is run on the +build directory. It is run right after the build script is finished. + +* Rsync Repositories +:PROPERTIES: +:DESCRIPTION: Information on using or creating rsync repositories +:END: + +Rsync repositories are simple to serve and simple to use. In the repository +directory, there needs to be a =.rsync= file that points to the remote of the +repository. This is used in order to fetch changes from the upstream. =.rsync= +file looks like this for the core repository: + +#+BEGIN_EXAMPLE + rsync://carbslinux.org/repo/core +#+END_EXAMPLE + +Rsync repositories have some few distinctions when it comes to fetching them. +They can be either synced individually or as a "root". There are 2 important +files, those are =.rsync= and =.rsync_root=. Here is the Carbs Linux +rsync repository structure. + +#+BEGIN_EXAMPLE + / + ----------------- + | | + .rsync core/ + ---------------- + | | + .rsync .rsync_root +#+END_EXAMPLE + +Unlike git repositories, they don't have a defined "root" directory. This is +both an advantage and a disadvantage. This way, we can sync individual +repositories, but that also means we need extra files to define root directories +and repository locations. Here is the content for each of these files: + +#+BEGIN_EXAMPLE + /.rsync: rsync://carbslinux.org/repo + /core/.rsync: rsync://carbslinux.org/repo/core + /core/.rsync_root: .. +#+END_EXAMPLE + +The =.rsync_root= file on the core repository points to the upper directory. +If a =.rsync= file exists on the upper directory, this means that is the whole +repository and will sync the entire repository instead of each individual repository. + +If the upper directory doesn't have this =.rsync= file, this means that this +is an individual repository, and the package manager will fetch accordingly. + +** Setting up an Rsync repository +:PROPERTIES: +:DESCRIPTION: Set up a repository for distribution +:END: + +Carbs Linux repositories automatically sync from the git repostitories and serve +it through the rsync daemon. Here is a sample shell script that I use in order to +sync repositories. Feel free to customize for your own use. + +#+BEGIN_SRC sh + #!/bin/sh + HOSTNAME="rsync://carbslinux.org/repo" + GITDIR="/pub/git/repo" + SHAREDIR="/pub/share/repo" + git -C "$GITDIR" pull + + rsync -avcC --delete --include=core --exclude=.rsync,.rsync_root "$GITDIR/." "$SHAREDIR" + + printf '%s\n' "$HOSTNAME" > "$GITDIR/.rsync" + for dir in "$GITDIR/"*; do + [ -d "$dir" ] || continue + [ -f "$dir/.rsync" ] || + printf '%s/%s\n' "$HOSTNAME" "${dir##*/}" > "$dir/.rsync" + printf '..\n' > "$dir/.rsync_root" + done +#+END_SRC + +You can then create an *rsync* user for serving the repositories. + +#+BEGIN_EXAMPLE + $ adduser -SD rsync +#+END_EXAMPLE + +Create =/etc/rsyncd.conf= and a service configuration as well. + +#+BEGIN_EXAMPLE + uid = rsync + gid = rsync + address = example.com + max connections = 10 + use chroot = yes + + [repo] + path = /pub/share/repo + comment = My repository +#+END_EXAMPLE + +Create a service file at =/etc/sv/rsync/run= (runit): + +#+BEGIN_SRC sh + #!/bin/sh -e + exec rsync --daemon --no-detach +#+END_SRC + +* CPT Library +:PROPERTIES: +:DESCRIPTION: Documentation of the Library +:END: + +=cpt-lib= is the library of Carbs Packaging Tools which can be used to extend +the functionality of the package manager. This is the API documentation of the +package manager library. + +** Calling the library +:PROPERTIES: +:DESCRIPTION: Including the library on your code +:END: + +You can call the library on your scripts by adding the following line to your +files: + +#+begin_src sh + #!/bin/sh -e + . cpt-lib +#+end_src + +This will load the library inside your script, and will set some environment +variables that are used inside the package manager. + +** Option parsing +:PROPERTIES: +:DESCRIPTION: Easy way of parsing options with cpt-lib +:END: + +=cpt-lib= includes a POSIX-shell option parser inside named =getoptions=. You +can see its own [[https://github.com/ko1nksm/getoptions/blob/v2.0.1/README.md][documentation]] for writing an option parser. The built-in version +of the =getoptions= library is 2.0.1 and there are no plans for updating it +apart from bug fixes. + +*** Defining a parser +:PROPERTIES: +:DESCRIPTION: Correct way of using getoptions +:END: + +Some functions are called and set automatically when you call =cpt-lib=, so you +shouldn't define the option parser after calling the library, as some of the +variables will already be set. + +If the function =parser_definition()= as defined when =cpt-lib= is called, +cpt-lib will handle the option parsing itself by calling =getoptions= +inside. Here is the proper way of doing it. + +#+begin_src sh + #!/bin/sh -e + + parser_definition() { + # The rest arguments MUST be defined as 'REST' + setup REST help:usage -- "usage: ${0##*/} [options] [pkg...]" + msg -- '' 'Options:' + flag CPT_TEST -t export:1 init:@export -- "Enable tests" + + global_options + } + + . cpt-lib +#+end_src + +*** =global_options()= +:PROPERTIES: +:DESCRIPTION: Convenience function for defining common flags +:END: + +The =global_options()= function is a simple convenience call to include flags +that can be used inside most =cpt= tools. It defines the following flags: + +| Flag | Long Option | Calls | +|------+-------------+--------------| +| -f | --force | =CPT_FORCE= | +| -y | --no-prompt | =CPT_PROMPT= | +| | --root | =CPT_ROOT= | +| -h | --help | =usage()= | +| -v | --version | =version()= | + +** Message functions +:PROPERTIES: +:DESCRIPTION: Communicate to users +:END: +=cpt= has various functions to print information to users. +*** =out()= +:PROPERTIES: +:DESCRIPTION: Print a message as-is +:END: + +=out()= is a really simple function that prints messages to the standard +output. It prints every argument with a newline. It is not meant to communicate +with the user, it just exists to have a simple function to interact with other +functions. + +#+begin_src sh + $ out "This is an example call" "How are you?" + This is an example call + How are you? +#+end_src + +*** =log()= +:PROPERTIES: +:DESCRIPTION: Print a message prettily +:END: + +=log()= is the most commonly used message function in the package manager. It is +used to pretty print messages with visual cues, so it is easier to read and +understand for the users. It changes message output for each argument it +receives (takes up to three arguments). + +- If it takes a single argument, it prints a yellow leading arrow followed by + colorless text. +- If it takes two arguments, it prints a yellow leading arrow followed by the + first argument (colored blue), and then followed by colorless second argument. +- If it takes three arguments, instead of a yellow arrow, it prints the third + argument in yellow, followed by the same two arguments as above. + +*** =die()= +:PROPERTIES: +:DESCRIPTION: Print a message and exit with error +:END: + +=die()= wraps the =log()= function and exits with an error (1). It takes one or +two arguments, which are sent to the =log()= function. The third argument for +=log()= is set as =!>=. + +*** =warn()= +:PROPERTIES: +:DESCRIPTION: Print a warning message +:END: + +=warn()= is another function that wraps =log()=. In place of the third argument, +it uses the word =WARNING=. + +*** =prompt()= +:PROPERTIES: +:DESCRIPTION: Ask the user whether they want to continue +:END: + +=prompt()= is an interactive function that waits for user input to continue. +It takes a single argument string to print a message, and then asks the user +whether they want to continue or not. Prompts can be disabled by the user if +they use a flag to disable them or set =CPT_PROMPT= to 0. + +** Text functions +:PROPERTIES: +:DESCRIPTION: Manipulate or check text +:END: + +Following functions are used to manipulate, check, or interact with text. + +*** =contains()= +:PROPERTIES: +:DESCRIPTION: Check if a "string list" contains a word +:END: + +=contains= function can be used to check whether a list variable contains a +given string. If the string is inside the list, it will return 0, otherwise 1. + +#+begin_src sh + # Usage + contains "$LIST" foo + + contains "foo bar" foo # Returns 0 + contains "foo bar" baz # Returns 1 +#+end_src + +*** =regesc()= +:PROPERTIES: +:DESCRIPTION: Escape regular expression characters +:END: + +=regesc()= can be used to escape regular expression characters that are defined +in POSIX BRE. Those characters are, =$=, =.=, =*=, =[=, =\\=, and =^=. + +#+begin_src sh + regesc '^[$\' # Returns \^\[\$\\ +#+end_src + +*** =pop()= +:PROPERTIES: +:DESCRIPTION: Remove an item from a string list +:END: + +=pop()= can be used to remove a word from a "string list" without a =sed= +call. Word splitting is intentional when using this function. + +#+begin_src sh + # Usage + pop foo from $LIST + + pop foo from foo baz bar # Returns baz bar +#+end_src + +** System Functions +*** =as_root()= +:PROPERTIES: +:DESCRIPTION: Run a command as the root user +:END: + +=as_root()= calls the rest of the arguments as a different user. Unless a =user= +environment variable is set, it will call the following arguments as the root +user. It supports the following programs for privilege escalation with the +following order: + +1. =sudo= +2. =doas= +3. =su= diff --git a/docs/cpt.texi b/docs/cpt.texi new file mode 100644 index 0000000..9a9974f --- /dev/null +++ b/docs/cpt.texi @@ -0,0 +1,1114 @@ +\input texinfo @c -*- texinfo -*- +@c %**start of header +@setfilename cpt.info +@settitle Carbs Packaging Tools +@documentencoding UTF-8 +@documentlanguage en +@c %**end of header + +@copying +Copyright @copyright{} 2020 Cem Keylan + +@quotation +Permission is granted to copy, distribute and/or modify this document under the +terms of the GNU Free Documentation License, Version 1.3 or any later version +published by the Free Software Foundation; with no Invariant Sections, with no +Front-Cover Texts and no Back-Cover Texts. A copy of the license is included in +the section entitled "GNU Free Documentation License." + +@end quotation +@end copying + +@dircategory Development +@direntry +* Carbs Packaging Tools: (cpt). Carbs Package Management Library. +@end direntry + +@finalout +@titlepage +@title Carbs Packaging Tools +@subtitle User Manual +@author Cem Keylan +@page +@vskip 0pt plus 1filll +@insertcopying +@end titlepage + +@contents + +@ifnottex +@node Top +@top Carbs Packaging Tools + +This is a reference document containing both the user-guide and the development +manual for @strong{Carbs Packaging Tools}. For development logs see @uref{https://git.carbslinux.org/cpt, the git repository}. +@end ifnottex + +@menu +* Preface:: Introduction to Carbs Packaging Tools +* Usage:: Basic usage of Carbs Packaging Tools +* Environment Variables:: Change the behaviour of cpt through environment configuration +* Hooks:: Use hooks to customize the package manager operations +* Packaging System:: More detail on creating packages +* Rsync Repositories:: Information on using or creating rsync repositories +* CPT Library:: Documentation of the Library + +@detailmenu +--- The Detailed Node Listing --- + +Usage + +* cpt-alternatives:: List and swap to alternatives +* cpt-build:: Build a package +* cpt-checksum:: Generate checksums +* cpt-download:: Download sources for the given package +* cpt-install:: Install a package +* cpt-list:: List installed packages +* cpt-remove:: Remove a package +* cpt-search:: Search for a package +* cpt-update:: Check for updates +* Global Flags:: Flags that work globally on some cpt utilities + +Environment Variables + +* @samp{CPT_PATH}:: Set the locations of your repositories +* @samp{CPT_COMPRESS}:: Compression tool to use in cpt +* @samp{CPT_FORCE}:: Force operations on cpt +* @samp{CPT_PID}:: Set reproducible temporary directories + +@samp{CPT_PATH} + +* Repository preferences:: Prioritise package repositories +* Setting the @samp{CPT_PATH}:: Set the value of CPT_PATH on your shell configuration + +Hooks + +* Editing the @samp{build} file during pre-build:: Modify a package build with your hooks + +Packaging System + +* @samp{build}:: The build script +* @samp{sources}:: The file containing package sources +* @samp{checksums}:: The file containing sha256sum of the sources +* @samp{version}:: The file containing the version and the release numbers of a package +* @samp{depends}:: The file containing the dependencies of a package +* @samp{post-install}:: The post-installation script +* @samp{message}:: The post-installation message to be displayed +* @samp{test}:: The test script for a package + +Rsync Repositories + +* Setting up an Rsync repository:: Set up a repository for distribution + +CPT Library + +* Calling the library:: Including the library on your code +* Option parsing:: Easy way of parsing options with cpt-lib +* Message functions:: Communicate to users +* Text functions:: Manipulate or check text +* System Functions:: + +Option parsing + +* Defining a parser:: Correct way of using getoptions +* @samp{global_options()}:: Convenience function for defining common flags + +Message functions + +* @samp{out()}:: Print a message as-is +* @samp{log()}:: Print a message prettily +* @samp{die()}:: Print a message and exit with error +* @samp{warn()}:: Print a warning message +* @samp{prompt()}:: Ask the user whether they want to continue + +Text functions + +* @samp{contains()}:: Check if a "string list" contains a word +* @samp{regesc()}:: Escape regular expression characters +* @samp{pop()}:: Remove an item from a string list + +System Functions + +* @samp{as_root()}:: Run a command as the root user + +@end detailmenu +@end menu + +@node Preface +@chapter Preface + +Carbs Linux uses its own package management toolchain named @samp{cpt} which was +initially forked from the @uref{https://github.com/kisslinux/kiss, kiss} package manager. Unlike @samp{kiss}, however, its main +goal is being easily extendable. Instead of being a single file package manager, +it revolves around the shell library @samp{cpt-lib}, and many tools that wrap around +it. This document aims to document both the usage of the distributed tools and +document the library functions. + +@node Usage +@chapter Usage + +@samp{cpt} is formed of many tools combined in a single environment, similar to +@samp{git}. When you run @samp{cpt} without any arguments, it will show all available +tools and their explanations. Here is an example call with extra scripts on my +system: + +@example +-> Carbs Packaging Tool +-> add Commit the current directory as a new package +-> alternatives List and swap to alternatives +-> build Build a package +-> bump Commit the current directory as a version bump +-> cargo-urlgen Create static cargo sources for Rust packages +-> cargolock-urlgen Convert the given Cargo.lock file to sources +-> cat Concatanate package files in the installed package database +-> changelog Print the git log of the specific package +-> chbuild Create/destroy temporary chroots +-> checkmissing Verify package manifests +-> checksum Generate checksums +-> chroot Enter a chroot +-> commit Commit a package without the prefix of 'package:' +-> depends Display a package's dependencies +-> download Download sources for the given package +-> exec Execute a command inside the alternatives system +-> export Turn an installed package into a CPT tarball +-> fork Fork a package to the current directory +-> getchoice Prints the full path to a file in the alternatives system. +-> install Install a package +-> link Link a forked package's files to the other repository +-> list List installed packages +-> maintainer Find the maintainer of a package +-> manifest Display all files owned by a package +-> manifest-tree Display all files owned by a package with a tree view +-> new Create a boilerplate CPT package +-> orphans List orphaned packages +-> owns Check which package owns a file +-> rel Bump the release number of a package +-> remove Remove a package +-> repodepends Display a package's dependencies in the repository +-> reporevdepends Display packages on the repository which depend on package +-> reset Remove all packages except for the base +-> revdepends Display packages which depend on package +-> search Search for a package +-> size Show the size on disk for a package +-> source Extract sources of a given package to the current directory +-> update Check for updates +@end example + +@menu +* cpt-alternatives:: List and swap to alternatives +* cpt-build:: Build a package +* cpt-checksum:: Generate checksums +* cpt-download:: Download sources for the given package +* cpt-install:: Install a package +* cpt-list:: List installed packages +* cpt-remove:: Remove a package +* cpt-search:: Search for a package +* cpt-update:: Check for updates +* Global Flags:: Flags that work globally on some cpt utilities +@end menu + +@node cpt-alternatives +@section cpt-alternatives + +You can list and swap to alternatives using @samp{cpt-alternatives}, or +@samp{cpt a} for short. When run without alternatives, it will list +alternatives. It can read from standard input if @samp{-} is given as an +argument. + +@strong{Examples} + +@enumerate +@item +List alternatives. +@end enumerate + +@example +$ cpt-alternatives +ncurses /usr/bin/clear +ncurses /usr/bin/reset +@end example + +@enumerate +@item +Swap to @samp{clear} from @samp{ncurses}. +@end enumerate + +@example +$ cpt-alternatives ncurses /usr/bin/clear +-> Swapping '/usr/bin/clear' from 'busybox' to 'ncurses' +@end example + +@enumerate +@item +Swap in bulk (all of @samp{sbase}). +@end enumerate + +@example +$ cpt a | grep ^sbase | cpt a - +@end example + +@node cpt-build +@section cpt-build + +@samp{cpt-build} will build given packages and their dependencies. If multiple +packages are specified, it will ask to install the packages as well. + +@multitable {aaaaaaaaaaaaaa} {aaaaaaaaaaa} +@headitem Flags +@tab Explanation +@item @samp{-t}, @samp{--test} +@tab Run tests +@end multitable + +@node cpt-checksum +@section cpt-checksum + +@samp{cpt-checksum} will generate a @samp{checksums} file from the package's sources. + +@node cpt-download +@section cpt-download + +@samp{cpt-download} will download the sources of a package. + +@node cpt-install +@section cpt-install + +@samp{cpt-install} will install given packages. + +@node cpt-list +@section cpt-list + +When called without arguments, @samp{cpt-list} will print all installed +packages. You can add package names as arguments to check whether they are +installed or not. In success, @samp{cpt-list} will exit with status 0 if all +given packages are installed, it will return 1 if any of the given packages +aren't installed. + +@multitable {aaaaaaaaaaaaaaaaa} {aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa} +@headitem Flags +@tab Explanation +@item @samp{-c}, @samp{--current} +@tab Use the current directory as a package +@end multitable + +@node cpt-remove +@section cpt-remove + +@samp{cpt-remove} will remove given packages. + +@node cpt-search +@section cpt-search + +@samp{cpt-search} will search for packages, it accepts regular expressions as well. + +@multitable {aaaaaaaaaaaaaaaa} {aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa} +@headitem Flags +@tab Explanation +@item @samp{-s}, @samp{--single} +@tab Only show the first instance of a package +@end multitable + + +@example +$ cpt-search 'alsa-*' +/var/db/cpt/repo/extra/alsa-lib +/var/db/cpt/repo/extra/alsa-utils +/var/db/cpt/installed/alsa-lib +/var/db/cpt/installed/alsa-utils + +$ cpt-search emacs +/home/cem/repos/main/community/emacs +/home/cem/repos/kiss-community/community/emacs +/var/db/cpt/installed/emacs + +$ cpt-search --single emacs +/home/cem/repos/main/community/emacs +@end example + +@node cpt-update +@section cpt-update + +@samp{cpt-update} will update the packages on your system. It fetches remote +repositories, and builds, and installs packages that have versions different +from the ones installed on the system. It doesn't check if the version string +is actually higher, it only checks whether they differ. + +@multitable {aaaaaaaaaaaaaaaaaaaa} {aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa} +@headitem Flags +@tab Explanation +@item @samp{-d}, @samp{--download} +@tab Only download updatable packages +@item @samp{-n}, @samp{--no-fetch} +@tab Do not update remote repositories +@item @samp{-o}, @samp{--only-fetch} +@tab Only fetch the repositories and exit +@end multitable + +@node Global Flags +@section Global Flags + +@multitable {aaaaaaaaaaaaaaaaaaa} {aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa} +@headitem Flags +@tab Explanation +@item @samp{-f}, @samp{--force} +@tab Force operation, @ref{@samp{CPT_FORCE}, , See @samp{CPT_FORCE}} +@item @samp{-y}, @samp{--no-prompt} +@tab Do not prompt for confirmation +@item @samp{-root CPT_ROOT} +@tab Use an alternate root directory +@item @samp{-h}, @samp{--help} +@tab Show this help message +@item @samp{-v}, @samp{--version} +@tab Print version information +@end multitable + +@node Environment Variables +@chapter Environment Variables + +Since there is no configuration file for cpt, the package manager is configured +through environment variables. These can be set per operation, or be set to your +shell configuration or @samp{~/.profile}. Here are the environment variables that +alter the behaviour of @samp{cpt}: + +@multitable {aaaaaaaaaaaaaaaaaaaa} {aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa} +@headitem ENVIRONMENT VARIABLE +@tab Effects +@item @samp{CPT_PATH} +@tab Set the locations of your repositories. It is similar to the @samp{PATH} variable. +@item @samp{XDG_CACHE_HOME} +@tab Unless this is set, the @samp{~/.cache} directory will be used instead. +@item @samp{CPT_CACHE} +@tab The cache directory for @samp{cpt}. Default: @samp{$XDG_CACHE_HOME/cpt} +@item @samp{CPT_CHOICE} +@tab If this is set to 0, a package installation will be aborted on conflicts. +@item @samp{CPT_COMPRESS} +@tab Program used to compress package tarballs. +@item @samp{CPT_DEBUG} +@tab If set to 1, temporary directories will not be removed after the operation. +@item @samp{CPT_FETCH} +@tab If set to 0, @samp{cpt-update} will not fetch repositories. +@item @samp{CPT_FORCE} +@tab Force operation. +@item @samp{CPT_HOOK} +@tab Location for the hook file. +@item @samp{CPT_KEEPLOG} +@tab If set to 1, cpt will keep logs regardless of operation success. +@item @samp{CPT_PID} +@tab Set the temporary build directory name. +@item @samp{CPT_PROMPT} +@tab If set to 0, @samp{cpt} will not prompt you for anything. +@item @samp{CPT_ROOT} +@tab If this variable is set, @samp{cpt} will assume this as the system root. +@item @samp{CPT_TEST} +@tab If set to 1, @samp{cpt-build} will run tests whenever available. +@item @samp{CPT_TMPDIR} +@tab The directory to create the temporary directories. +@end multitable + +@menu +* @samp{CPT_PATH}:: Set the locations of your repositories +* @samp{CPT_COMPRESS}:: Compression tool to use in cpt +* @samp{CPT_FORCE}:: Force operations on cpt +* @samp{CPT_PID}:: Set reproducible temporary directories +@end menu + +@node @samp{CPT_PATH} +@section @samp{CPT_PATH} + +Similar to the @samp{PATH} variable, @samp{cpt} find repositories from the @samp{CPT_PATH} +variable. Here is an example: + +@example +CPT_PATH=$HOME/repos/repo1:$HOME/repos/repo2:$HOME/repos/repo3 +@end example + +This is a simplistic and a structured example for repository locations, but it +doesn't necessarily need to be as tidy as the example above. Here is an example +for something a little more complex. + +@example +CPT_PATH=$HOME/repos/overrides:/var/db/cpt/repo/core:/var/db/cpt/repo/extra:$HOME/repos/personal +@end example + +This example brings us to the next section of this document. + +@menu +* Repository preferences:: Prioritise package repositories +* Setting the @samp{CPT_PATH}:: Set the value of CPT_PATH on your shell configuration +@end menu + +@node Repository preferences +@subsection Repository preferences + +When you are using multiple repositories from multiple vendors, you will find +out that some repositories have the same packages. @samp{cpt} doesn't care about +conflicting packages. If you want to build a package that exists on multiple +repositories, @samp{cpt} will build the first matching package. This means that if +@samp{grep} package (for the sake of an example) exists on both +@samp{$HOME/repos/personal} and @samp{$HOME/repos/carbs/extra}, and you want +to install from your personal repository, you must set @samp{CPT_PATH} so that your +personal repository is listed before the @samp{extra} repository. + +@example +CPT_PATH=$HOME/repos/personal:$HOME/repos/carbs/extra +@end example + +@node Setting the @samp{CPT_PATH} +@subsection Setting the @samp{CPT_PATH} + +You can set the @samp{CPT_PATH} variable on your shell configuration or your +@samp{.profile} file in a way that is easy to read. + +The below example sets @samp{CPT_PATH} in a way that is easy to understand which +repository comes first: + +@example +CPT_PATH=$HOME/repos/overrides +CPT_PATH=$CPT_PATH:$HOME/repos/carbs/core +CPT_PATH=$CPT_PATH:$HOME/repos/carbs/extra +CPT_PATH=$CPT_PATH:$HOME/repos/carbs/xorg +CPT_PATH=$CPT_PATH:$HOME/repos/personal +export CPT_PATH +@end example + +@node @samp{CPT_COMPRESS} +@section @samp{CPT_COMPRESS} + +When setting the @samp{CPT_COMPRESS} value, you should set the name of the default +suffixes for the program. Available values are: + +@itemize +@item +@samp{gz} +@item +@samp{zst} +@item +@samp{bz2} +@item +@samp{xz} +@end itemize + +Defaults to @samp{gz}. + +@node @samp{CPT_FORCE} +@section @samp{CPT_FORCE} + +If this is set to 1, some of the @samp{cpt} tools will continue regardless of +errors or skip certain checks. Here are some examples: + +@itemize +@item +@samp{cpt-install} will install a package without verifying its manifest. +@item +@samp{cpt-install} will install a package even when there are missing dependencies. +@item +@samp{cpt-remove} will remove packages even when there are other packages that +depend on the current package. +@end itemize + +Defaults to 0. + +@node @samp{CPT_PID} +@section @samp{CPT_PID} + +If this variable is set, the temporary files will be created with this variable +as the suffix, instead of the PID of the @samp{cpt} process. The advantage is that +you can know exactly where the build directory is located, while the +disadvantage is that there will be issues with multiple operations at the same +time. So the best way to use this variable is during one-time @samp{cpt} calls. + +@example +CPT_PID=mesa cpt b mesa +@end example + +By running the above, you will know that the created build directories will end +with the @samp{*-mesa} suffix. + +@node Hooks +@chapter Hooks + +Hooks can be used in order to change the runtime behaviour of the package manager. +There are a variety of package hooks, mostly self explanatory: + +@itemize +@item +pre-build +@item +post-build +@item +build-fail +@item +pre-test +@item +test-fail +@item +pre-install +@item +post-install +@item +pre-remove +@item +post-remove +@item +pre-fetch +@item +post-fetch +@item +post-package +@end itemize + +In order to use hooks, you will need to set the @samp{CPT_HOOK} variable pointing to +your hook file. Your hook file @strong{MUST} be a POSIX shell script as its contents +are sourced by the package manager. + +The hook is given 3 variables when it is executed. Those are: + +@multitable {aaaaaaaa} {aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa} +@headitem Variable +@tab Explanation +@item @samp{$TYPE} +@tab The type of the hook, (@samp{pre-build}, @samp{post-build}, etc.) +@item @samp{$PKG} +@tab The package that @samp{cpt} is currently working on. Can be null. +@item @samp{$DEST} +@tab The destination of the operation. Can be null. +@end multitable + +@menu +* Editing the @samp{build} file during pre-build:: Modify a package build with your hooks +@end menu + +@node Editing the @samp{build} file during pre-build +@section Editing the @samp{build} file during pre-build + +You can edit the @samp{build} file during pre-build. The file is copied from the +repository to the build directory named as @samp{.build.cpt}. You can use @samp{sed} or +any other tool to edit the build file. After the build is complete, a @samp{diff} +file will be placed to the package database named as @samp{build.diff}. Here is an +example @samp{build} file manipulation during the pre-build hook. + +@example +cat <<EOF> .build.cpt +#!/bin/sh -e + +for patch in bash50-0??; do + patch -p0 < "\$patch" +done + +export LDFLAGS=-static + +./configure \ + --prefix=/usr \ + --without-bash-malloc \ + --disable-nls + +export MAKEFLAGS="TERMCAP_LIB=/usr/lib/libncursesw.a $MAKEFLAGS" + +make +make DESTDIR="\$1" install + +ln -s bash "\$1/usr/bin/sh" +EOF +@end example + +@node Packaging System +@chapter Packaging System + +A package is formed of several files, these are: + +@itemize +@item +@ref{@samp{build}} +@item +@ref{@samp{sources}} +@item +@ref{@samp{checksums}} +@item +@ref{@samp{version}} +@item +@ref{@samp{depends}} +@item +@ref{@samp{post-install}} +@item +@ref{@samp{message}} +@item +@ref{@samp{test}} +@end itemize + +Any other file can be added to the package directory at the discretion of the +package maintainer. Everything in the package directory will also be added to the +package database that is located on @samp{/var/db/cpt/installed}. These can be +patches, configuration files, etc. + +@menu +* @samp{build}:: The build script +* @samp{sources}:: The file containing package sources +* @samp{checksums}:: The file containing sha256sum of the sources +* @samp{version}:: The file containing the version and the release numbers of a package +* @samp{depends}:: The file containing the dependencies of a package +* @samp{post-install}:: The post-installation script +* @samp{message}:: The post-installation message to be displayed +* @samp{test}:: The test script for a package +@end menu + +@node @samp{build} +@section @samp{build} + +Typically @samp{build} files are shell scripts that run commands to prepare the source +code to be installed on the target system. Even though we will be assuming that +the @samp{build} file is a POSIX shell script (for portability's sake), @samp{build} +files can be any executable program from binary programs to @samp{perl} scripts. + +The contents of a build script do not need to follow a certain rule for the +package manager, except for the fact that the user needs the permission to +execute the file. + +An important advice is to append an '-e' to the shebang (#!/bin/sh -e) so that +the build script exits on compilation error. + +Build is run with three arguments (@samp{$#}) + +@itemize +@item +Location of the package directory (DESTDIR) +@item +Package version +@item +System architecture +@end itemize + +@node @samp{sources} +@section @samp{sources} + +@samp{sources} file is a list of files and sources that will be put to the build +directory during the build process. Those can be remote sources (such as tarballs), +git repositories, and files that reside on the package directory. + +The syntax is pretty simple for the @samp{soures} file; @samp{src dest}. The @samp{dest} +parameter is optional. It is the directory that the source will be placed in. +Here is the @samp{sources} file for the @samp{gst-plugins} package: + +@example +https://gstreamer.freedesktop.org/src/gst-plugins-good/gst-plugins-good-1.16.2.tar.xz good +https://gstreamer.freedesktop.org/src/gst-plugins-bad/gst-plugins-bad-1.16.2.tar.xz bad +https://gstreamer.freedesktop.org/src/gst-plugins-ugly/gst-plugins-ugly-1.16.2.tar.xz ugly +https://gstreamer.freedesktop.org/src/gst-libav/gst-libav-1.16.2.tar.xz libav +@end example + +This file is read from the package manager as space seperated. Files that begin +with a @samp{#} comment are ignored. The first value points to the location of the +source. + +If it starts with a protcol url, (such as @uref{ftp://} @uref{http://} @uref{https://}) it will be +downloaded with @samp{curl}. + +If the source is a git repository, it shall be prefixed with a @samp{git+} git(1) will +be used to do a shallow clone of the repository. If the commit is suffixed by a +history pointer, git will checkout the relevant revision. So, + +@table @asis +@item @samp{git+git://example.com/pub/repo@@v1.2.3} +will checkout the tag named "v1.2.3" +@item @samp{git+git://example.com/pub/repo#development} +will checkout the branch named "development" +@item @samp{git+git://example.com/pub/repo#1a314s87} +will checkout the commit named "1a314s87" +@end table + +Other files are assumed to be residing in the package directory. They should be +added with their paths relative to the package directory. + +@node @samp{checksums} +@section @samp{checksums} + +checksums file is generated by the @samp{cpt c pkg} command. It is generated +according to the order of the sources file. That's why you shouldn't be editing +it manually. The checksums file is created with the digests of the files using +the sha256 algorithm. + +@node @samp{version} +@section @samp{version} + +The version file includes the version of the software and the release number of +of the package on a space seperated format. The contents of the file should look +like below. + +@example +1.3.2 1 +@end example + +@node @samp{depends} +@section @samp{depends} + +This is a list of dependencies that must be installed before a package build. You +can append "make" after a dependency to mark a package is only required during +the build process of a package. Packages marked as a make dependency can be +removed after the build. There are also "test" dependencies. These dependencies +are only installed if either the @samp{CPT_TEST} is set to 1, or the build is run +with the @samp{-t} or @samp{--test} options. So, a package package could have +the following @samp{depends} file: + +@example +linux-headers make +python test +zlib +@end example + +@node @samp{post-install} +@section @samp{post-install} + +@samp{post-install} files have the same requirements as the build script. They +will be run after the package is installed as root (or as the user if the user +has write permissions on @samp{CPT_ROOT}). + +@node @samp{message} +@section @samp{message} + +This plaintext file will be outputted with @samp{cat} after every package is +installed. + +@node @samp{test} +@section @samp{test} + +Test files are mainly for the repository maintainer to test the packages, and +will only run if the user has the @samp{CPT_TEST} variable set, or the build is +run with the @samp{-t} or @samp{--test} options. This script is run on the +build directory. It is run right after the build script is finished. + +@node Rsync Repositories +@chapter Rsync Repositories + +Rsync repositories are simple to serve and simple to use. In the repository +directory, there needs to be a @samp{.rsync} file that points to the remote of the +repository. This is used in order to fetch changes from the upstream. @samp{.rsync} +file looks like this for the core repository: + +@example +rsync://carbslinux.org/repo/core +@end example + +Rsync repositories have some few distinctions when it comes to fetching them. +They can be either synced individually or as a "root". There are 2 important +files, those are @samp{.rsync} and @samp{.rsync_root}. Here is the Carbs Linux +rsync repository structure. + +@example + / + ----------------- + | | +.rsync core/ + ---------------- + | | + .rsync .rsync_root +@end example + +Unlike git repositories, they don't have a defined "root" directory. This is +both an advantage and a disadvantage. This way, we can sync individual +repositories, but that also means we need extra files to define root directories +and repository locations. Here is the content for each of these files: + +@example +/.rsync: rsync://carbslinux.org/repo +/core/.rsync: rsync://carbslinux.org/repo/core +/core/.rsync_root: .. +@end example + +The @samp{.rsync_root} file on the core repository points to the upper directory. +If a @samp{.rsync} file exists on the upper directory, this means that is the whole +repository and will sync the entire repository instead of each individual repository. + +If the upper directory doesn't have this @samp{.rsync} file, this means that this +is an individual repository, and the package manager will fetch accordingly. + +@menu +* Setting up an Rsync repository:: Set up a repository for distribution +@end menu + +@node Setting up an Rsync repository +@section Setting up an Rsync repository + +Carbs Linux repositories automatically sync from the git repostitories and serve +it through the rsync daemon. Here is a sample shell script that I use in order to +sync repositories. Feel free to customize for your own use. + +@example +#!/bin/sh +HOSTNAME="rsync://carbslinux.org/repo" +GITDIR="/pub/git/repo" +SHAREDIR="/pub/share/repo" +git -C "$GITDIR" pull + +rsync -avcC --delete --include=core --exclude=.rsync,.rsync_root "$GITDIR/." "$SHAREDIR" + +printf '%s\n' "$HOSTNAME" > "$GITDIR/.rsync" +for dir in "$GITDIR/"*; do + [ -d "$dir" ] || continue + [ -f "$dir/.rsync" ] || + printf '%s/%s\n' "$HOSTNAME" "$@{dir##*/@}" > "$dir/.rsync" + printf '..\n' > "$dir/.rsync_root" +done +@end example + +You can then create an @strong{rsync} user for serving the repositories. + +@example +$ adduser -SD rsync +@end example + +Create @samp{/etc/rsyncd.conf} and a service configuration as well. + +@example +uid = rsync +gid = rsync +address = example.com +max connections = 10 +use chroot = yes + +[repo] + path = /pub/share/repo + comment = My repository +@end example + +Create a service file at @samp{/etc/sv/rsync/run} (runit): + +@example +#!/bin/sh -e +exec rsync --daemon --no-detach +@end example + +@node CPT Library +@chapter CPT Library + +@samp{cpt-lib} is the library of Carbs Packaging Tools which can be used to extend +the functionality of the package manager. This is the API documentation of the +package manager library. + +@menu +* Calling the library:: Including the library on your code +* Option parsing:: Easy way of parsing options with cpt-lib +* Message functions:: Communicate to users +* Text functions:: Manipulate or check text +* System Functions:: +@end menu + +@node Calling the library +@section Calling the library + +You can call the library on your scripts by adding the following line to your +files: + +@example +#!/bin/sh -e +. cpt-lib +@end example + +This will load the library inside your script, and will set some environment +variables that are used inside the package manager. + +@node Option parsing +@section Option parsing + +@samp{cpt-lib} includes a POSIX-shell option parser inside named @samp{getoptions}. You +can see its own @uref{https://github.com/ko1nksm/getoptions/blob/v2.0.1/README.md, documentation} for writing an option parser. The built-in version +of the @samp{getoptions} library is 2.0.1 and there are no plans for updating it +apart from bug fixes. + +@menu +* Defining a parser:: Correct way of using getoptions +* @samp{global_options()}:: Convenience function for defining common flags +@end menu + +@node Defining a parser +@subsection Defining a parser + +Some functions are called and set automatically when you call @samp{cpt-lib}, so you +shouldn't define the option parser after calling the library, as some of the +variables will already be set. + +If the function @samp{parser_definition()} as defined when @samp{cpt-lib} is called, +cpt-lib will handle the option parsing itself by calling @samp{getoptions} +inside. Here is the proper way of doing it. + +@example +#!/bin/sh -e + +parser_definition() @{ + # The rest arguments MUST be defined as 'REST' + setup REST help:usage -- "usage: $@{0##*/@} [options] [pkg...]" + msg -- '' 'Options:' + flag CPT_TEST -t export:1 init:@@export -- "Enable tests" + + global_options +@} + +. cpt-lib +@end example + +@node @samp{global_options()} +@subsection @samp{global_options()} + +The @samp{global_options()} function is a simple convenience call to include flags +that can be used inside most @samp{cpt} tools. It defines the following flags: + +@multitable {aaaa} {aaaaaaaaaaa} {aaaaaaaaaaaa} +@headitem Flag +@tab Long Option +@tab Calls +@item -f +@tab --force +@tab @samp{CPT_FORCE} +@item -y +@tab --no-prompt +@tab @samp{CPT_PROMPT} +@item +@tab --root +@tab @samp{CPT_ROOT} +@item -h +@tab --help +@tab @samp{usage()} +@item -v +@tab --version +@tab @samp{version()} +@end multitable + +@node Message functions +@section Message functions + +@samp{cpt} has various functions to print information to users. + +@menu +* @samp{out()}:: Print a message as-is +* @samp{log()}:: Print a message prettily +* @samp{die()}:: Print a message and exit with error +* @samp{warn()}:: Print a warning message +* @samp{prompt()}:: Ask the user whether they want to continue +@end menu + +@node @samp{out()} +@subsection @samp{out()} + +@samp{out()} is a really simple function that prints messages to the standard +output. It prints every argument with a newline. It is not meant to communicate +with the user, it just exists to have a simple function to interact with other +functions. + +@example +$ out "This is an example call" "How are you?" +This is an example call +How are you? +@end example + +@node @samp{log()} +@subsection @samp{log()} + +@samp{log()} is the most commonly used message function in the package manager. It is +used to pretty print messages with visual cues, so it is easier to read and +understand for the users. It changes message output for each argument it +receives (takes up to three arguments). + +@itemize +@item +If it takes a single argument, it prints a yellow leading arrow followed by +colorless text. +@item +If it takes two arguments, it prints a yellow leading arrow followed by the +first argument (colored blue), and then followed by colorless second argument. +@item +If it takes three arguments, instead of a yellow arrow, it prints the third +argument in yellow, followed by the same two arguments as above. +@end itemize + +@node @samp{die()} +@subsection @samp{die()} + +@samp{die()} wraps the @samp{log()} function and exits with an error (1). It takes one or +two arguments, which are sent to the @samp{log()} function. The third argument for +@samp{log()} is set as @samp{!>}. + +@node @samp{warn()} +@subsection @samp{warn()} + +@samp{warn()} is another function that wraps @samp{log()}. In place of the third argument, +it uses the word @samp{WARNING}. + +@node @samp{prompt()} +@subsection @samp{prompt()} + +@samp{prompt()} is an interactive function that waits for user input to continue. +It takes a single argument string to print a message, and then asks the user +whether they want to continue or not. Prompts can be disabled by the user if +they use a flag to disable them or set @samp{CPT_PROMPT} to 0. + +@node Text functions +@section Text functions + +Following functions are used to manipulate, check, or interact with text. + +@menu +* @samp{contains()}:: Check if a "string list" contains a word +* @samp{regesc()}:: Escape regular expression characters +* @samp{pop()}:: Remove an item from a string list +@end menu + +@node @samp{contains()} +@subsection @samp{contains()} + +@samp{contains} function can be used to check whether a list variable contains a +given string. If the string is inside the list, it will return 0, otherwise 1. + +@example +# Usage +contains "$LIST" foo + +contains "foo bar" foo # Returns 0 +contains "foo bar" baz # Returns 1 +@end example + +@node @samp{regesc()} +@subsection @samp{regesc()} + +@samp{regesc()} can be used to escape regular expression characters that are defined +in POSIX BRE@. Those characters are, @samp{$}, @samp{.}, @samp{*}, @samp{[}, @samp{\\}, and @samp{^}. + +@example +regesc '^[$\' # Returns \^\[\$\\ +@end example + +@node @samp{pop()} +@subsection @samp{pop()} + +@samp{pop()} can be used to remove a word from a "string list" without a @samp{sed} +call. Word splitting is intentional when using this function. + +@example +# Usage +pop foo from $LIST + +pop foo from foo baz bar # Returns baz bar +@end example + +@node System Functions +@section System Functions + +@menu +* @samp{as_root()}:: Run a command as the root user +@end menu + +@node @samp{as_root()} +@subsection @samp{as_root()} + +@samp{as_root()} calls the rest of the arguments as a different user. Unless a @samp{user} +environment variable is set, it will call the following arguments as the root +user. It supports the following programs for privilege escalation with the +following order: + +@enumerate +@item +@samp{sudo} +@item +@samp{doas} +@item +@samp{su} +@end enumerate + +@bye
\ No newline at end of file |