Guile-JSON 3.x is incompatible with Guile-JSON 1.x, which we relied on
until now: it maps JSON dictionaries to alists (instead of hash tables),
and JSON arrays to vectors (instead of lists). This commit is about
adjusting all the existing code to this new mapping.
* m4/guix.m4 (GUIX_CHECK_GUILE_JSON): New macro.
* configure.ac: Use it.
* doc/guix.texi (Requirements): Mention the Guile-JSON version.
* guix/git-download.scm (git-fetch)[guile-json]: Use GUILE-JSON-3.
* guix/import/cpan.scm (string->license): Expect vectors instead of
lists.
(module->dist-name): Use 'json-fetch' instead of 'json-fetch-alist'.
(cpan-fetch): Likewise.
* guix/import/crate.scm (crate-fetch): Likewise, and call 'vector->list'
for DEPS.
* guix/import/gem.scm (rubygems-fetch): Likewise.
* guix/import/json.scm (json-fetch-alist): Remove.
* guix/import/pypi.scm (pypi-fetch): Use 'json-fetch' instead of
'json-fetch-alist'.
(latest-source-release, latest-wheel-release): Call 'vector->list' on
RELEASES.
* guix/import/stackage.scm (stackage-lts-info-fetch): Use 'json-fetch'
instead of 'json-fetch-alist'.
(lts-package-version): Use 'vector->list'.
* guix/import/utils.scm (hash-table->alist): Remove.
(alist->package): Pass 'vector->list' on the inputs fields, and default
to the empty vector.
* guix/scripts/import/json.scm (guix-import-json): Remove call to
'hash-table->alist'.
* guix/swh.scm (define-json-reader): Expect pair? or null? instead of
hash-table?.
[extract-field]: Use 'assoc-ref' instead of 'hash-ref'.
(json->branches): Use 'map' instead of 'hash-map->list'.
(json->checksums): Likewise.
(json->directory-entries, origin-visits): Call 'vector->list' on the
result of 'json->scm'.
* tests/import-utils.scm ("alist->package with dependencies"): New test.
* gnu/installer.scm (build-compiled-file)[builder]: Use GUILE-JSON-3.
* gnu/installer.scm (installer-program)[installer-builder]: Likewise.
* gnu/installer/locale.scm (iso639->iso639-languages): Use 'assoc-ref'
instead of 'hash-ref', and pass vectors through 'vector->list'.
(iso3166->iso3166-territories): Likewise.
* gnu/system/vm.scm (system-docker-image)[build]: Use GUILE-JSON-3.
* guix/docker.scm (manifest, config): Adjust for Guile-JSON 3.
* guix/scripts/pack.scm (docker-image)[build]: Use GUILE-JSON-3.
* guix/import/github.scm (fetch-releases-or-tags): Update docstring.
(latest-released-version): Use 'assoc-ref' instead of 'hash-ref'. Pass
the result of 'fetch-releases-or-tags' to 'vector->list'.
* guix/import/launchpad.scm (latest-released-version): Likewise.
* guix/scripts/pack.scm (singularity-environment-file): New procedure.
(squashfs-image): Use it, and create /.singularity/env/90-environment.sh.
* gnu/tests/singularity.scm (run-singularity-test)["singularity run,
with environment"]: New test, currently skipped.
* gnu/tests/singularity.scm (build-tarball&run-singularity-test): Add
GUILE-JSON to the profile.
* guix/docker.scm (config): Add #:environment parameter and honor it.
(build-docker-image): Likewise, and pass it to 'config'.
* guix/scripts/pack.scm (docker-image): Import (guix profiles) and (guix
search-paths). Call 'profile-search-paths' and pass #:environment to
'build-docker-image'.
* gnu/tests/docker.scm (run-docker-test)["Load docker image and run it"]:
Add example that expects (json) to be available.
* gnu/tests/docker.scm (build-tarball&run-docker-test): Replace
%BOOTSTRAP-GUILE by GUILE-2.2 and GUILE-JSON in the environment.
* guix/scripts/pack.scm (self-contained-tarball): Add #:entry-point and
warn when it's true.
(squashfs-image): Add #:entry-point and honor it.
(docker-image): Add #:entry-point and honor it.
(%options, show-help): Add '--entry-point'.
(guix-pack): Honor '--entry-point' and pass #:entry-point to BUILD-IMAGE.
* gnu/tests/docker.scm (run-docker-test): Test 'docker run' with the
default entry point.
(build-tarball&run-docker-test): Pass #:entry-point to 'docker-image'.
* doc/guix.texi (Invoking guix pack): Document it.
* gnu/tests/singularity.scm (run-singularity-test)["singularity run"]:
New test.
(build-tarball&run-singularity-test): Pass #:entry-point to
'squashfs-image'.
* guix/scripts/pack.scm (%options, show-help): Add "--root".
(guix-pack): Honor it.
* tests/guix-pack.sh: Test it.
* doc/guix.texi (Invoking guix pack): Document it.
Fixes <https://bugs.gnu.org/34913>.
* guix/scripts/pack.scm (squashfs-image)[build]: Use
'relative-file-name' when creating SYMLINKS.
* guix/scripts/pack.scm (guix-pack): Pass #:relative-symlinks? #t when
PACK-FORMAT is 'squashfs.
* gnu/packages/aux-files/run-in-namespace.c (exec_with_proot): New
function.
(main): When 'clone' fails, call 'rm_rf'.
[PROOT_PROGRAM]: When 'clone' fails, call 'exec_with_proot'.
* guix/scripts/pack.scm (wrapped-package): Add #:proot?.
[proot]: New procedure.
[build]: Compile with -DPROOT_PROGRAM when PROOT? is true.
* guix/scripts/pack.scm (%options): Set the 'relocatable?' value to
'proot when "-R" is passed several times.
(guix-pack): Pass #:proot? to 'wrapped-package'.
* tests/guix-pack-relocatable.sh: Use "-RR" on Intel systems that lack
user namespace support.
* doc/guix.texi (Invoking guix pack): Document -RR.
In practice the error was not triggered because
'package-transitive-propagated-inputs' currently returns the empty list
for these two packages.
* guix/scripts/pack.scm (gcrypt-sqlite3&co): Remove labels from the
result.
* gnu/system/vm.scm (gcrypt-sqlite3&co): Likewise.
The previous '--verbosity' option was misleading and rarely what users
were looking for. The new option provides a consistent way to choose
whether or not to display the build log.
* guix/scripts/build.scm (show-build-options-help): Remove "--verbosity"
and add "--debug".
(set-build-options-from-command-line): Use the 'debug key of OPTS for
#:verbosity.
(%standard-build-options): Change "verbosity" to "debug". Use
'string->number*' instead of 'string->number'.
(%default-options): Change 'verbosity to 'debug and add a 'verbosity
key.
(show-help): Add '--verbosity'.
(%options): Likewise, and change '--quiet' to set the 'verbosity key of
RESULT.
(guix-build): Use 'with-status-verbosity' instead of parameterizing
CURRENT-BUILD-OUTPUT-PORT, honor the 'verbosity key of OPTS, and remove
'quiet?'.
* guix/scripts/environment.scm (show-help, %options): Add '--verbosity'.
(%default-options): Add 'debug'.
(guix-environment): Honor the 'verbosity key of OPTS.
* guix/scripts/pack.scm (%default-options): Add 'debug.
(%options, show-help): Add '--verbosity'.
(guix-pack): Honor the 'verbosity key of OPTS.
* guix/scripts/package.scm (%default-options): Add 'debug.
(show-help, %options): Add '--verbosity'. Mark '--verbose' as
deprecated and change it to set 'verbosity.
(guix-package): Honor the 'verbosity key of OPTS and remove 'verbose?'.
* guix/scripts/pull.scm (%default-options): Add 'debug.
(show-help, %options): Add '--verbosity'.
(guix-pull): Honor the 'verbosity key of OPTS.
* guix/scripts/system.scm (show-help, %options): Add '--verbosity'.
(%default-options): Add 'debug.
(guix-system): Honor the 'verbosity key of OPTS.
* guix/scripts/archive.scm (%default-options): Add 'debug,
'print-build-trace?, 'print-extended-build-trace?, and
'multiplexed-build-output?.
(show-help, %options): Add '--verbosity'.
(export-from-store): Remove call to 'set-build-options-from-command-line'.
(guix-archive): Wrap body in 'with-status-verbosity'. Add call to
'set-build-options-from-command-line.
* doc/guix.texi (Common Build Options): Document '--verbosity' and
'--debug'.
(Additional Build Options): Adjust description of '--quiet'.
* guix/scripts/pack.scm (squashfs-image)[database]: New variable.
[build]: Add (gnu build install) to the closure. Call
'install-database-and-gc-roots' when DATABASE is true, and invoke
mksquashfs once more.
* tests/pack.scm ("squashfs-image + localstatedir"): New test.
* guix/docker.scm (build-docker-image): Add #:database parameter.
Create /var/guix/db, /var/guix/profiles, etc. when DATABASE is true.
* guix/scripts/pack.scm (docker-image): Export. Remove #:deduplicate?
parameter. Define 'database' and pass it to 'docker-image'.
* tests/pack.scm (test-assertm): Recompile the derivation of
%BOOTSTRAP-GUILE.
("docker-image + localstatedir"): New test.
This is another way to address <https://bugs.gnu.org/32184>, which was
previously addressed in commit 19c924af4f.
* gnu/build/install.scm (register-closure): Move to...
* gnu/build/vm.scm (register-closure): ... here. New procedure.
* guix/scripts/pack.scm (self-contained-tarball)[build]: Remove
now unneeded 'with-extensions' form and custom (guix config) module.
* tests/guix-pack.sh: Revert the strategy from
commit 19c924af4f.
* tests/pack.scm ("self-contained-tarball"): Likewise.
* guix/scripts/pack.scm (store-database): New procedure.
(self-contained-tarball): Use it when LOCALSTATEDIR? is true.
Remove 'schema' and add 'database'.
[build]: Pass DATABASE to 'populate-single-profile-directory'.
(squashfs-image): Remove #:deduplicate? parameter.
[build]: Remove (gnu build install) and (guix config) from the imported
modules. Remove 'with-extensions'.
* gnu/build/install.scm (populate-single-profile-directory): Remove
#:deduplicate?, #:register?, and #:schema; add #:database. Remove call
to 'register-closure' and simply copy DATABASE instead.
Fixes a regression introduced in
48b444304e whereby "guix pack -C none
hello" would fail with:
ERROR: In procedure string-join:
In procedure string-join: Wrong type argument in position 1: #f
builder for `/gnu/store/…-tarball-pack.tar.drv' failed with exit code 1
* guix/scripts/pack.scm (self-contained-tarball): Adjust for when
'compressor-command' returns #f.
This allows for more accurate status tracking and parsing of extended
build traces.
* guix/status.scm (multiplexed-output-supported?): New procedure.
(print-build-event): Don't print \r when PRINT-LOG? is true.
Adjust 'build-log' handling for when 'multiplexed-output-supported?'
returns true.
(bytevector-index, split-lines): New procedures.
(build-event-output-port)[%build-output-pid, %build-output]
[%build-output-left]: New variables.
[process-line]: Handle "@ build-output" traces.
[process-build-output]: New procedure.
[write!]: Add case for when %BUILD-OUTPUT-PID is true. Use
'bytevector-index' rather than 'string-index'.
(compute-status): Add #:derivation-path->output-path. Use it.
* tests/status.scm ("compute-status, multiplexed build output"):
New test.
("build-output-port, UTF-8")
("current-build-output-port, UTF-8 + garbage"): Adjust to new
'build-log' output.
* guix/scripts/build.scm (set-build-options-from-command-line):
Pass #:multiplexed-build-output?.
(%default-options): Add 'multiplexed-build-output?'.
* guix/scripts/environment.scm (%default-options): Likewise.
* guix/scripts/pack.scm (%default-options): Likewise.
* guix/scripts/package.scm (%default-options): Likewise.
* guix/scripts/pull.scm (%default-options): Likewise.
* guix/scripts/system.scm (%default-options): Likewise.
Threaded compression makes archives non-deterministic: the result depends on
the number of threads used for compressing. See <https://bugs.gnu.org/31015>.
This reverts commit 63102406f2.
Until now, running "guix pack -s i686-linux" on an x86_64-linux machine,
for instance, would use an x86_64 guile for module derivations. This
was OK until now, but would break when passing "--localstatedir" due to
the introduction of guile-sqlite3: we'd be using the i686 guile-sqlite3
along with the x86_64 guile.
* guix/scripts/pack.scm (guix-pack): Pass the 'system option from OPTS
to 'package-derivation'.
Fixes a regression introduced in
c45477d2a1.
Reported by Christopher Baines <mail@cbaines.net>.
* guix/scripts/pack.scm (not-config?, guile-sqlite3&co): New variables.
(self-contained-tarball)[not-config?]: Remove.
[build]: Use GUILE-SQLITE3&CO for 'with-extensions'.
(squashfs-image)[libgcrypt]: New variable.
[build]: Use 'source-module-closure', 'make-config.scm', and
'with-extensions'.
(docker-image)[not-config?]: Remove.
Without this change, running guix pack fails as (guix sets) is missing when
compiling (guix build store-copy).
* guix/scripts/pack.scm (docker-image): Move (guix build store-copy) to within
the source-module-closure call.
* gnu/build/install.scm (register-closure): Add #:reset-timestamps? and
and #:schema; honor them. Rewrite in terms of 'register-path'.
(populate-single-profile-directory): Add #:schema and honor it. Make
/var/guix/profiles and /var/guix/gcroots.
* gnu/build/vm.scm (root-partition-initializer): Pass
#:reset-timestamps? to 'register-closure'.
* gnu/system/vm.scm (not-config?): New procedure.
(guile-sqlite3&co): New variable.
(expression->derivation-in-linux-vm)[config]: New variable.
[builder]: Use 'with-extensions'.
(iso9660-image)[schema, config]: New variables.
Wrap build expression in 'with-extensions'; add 'sql-schema' call.
Remove GUIX from INPUTS.
(qemu-image)[schema, config]: New variables.
Wrap body in 'with-extensions'.
(system-docker-image)[not-config?]: Remove.
[config]: Use 'make-config.scm'.
[schema]: New variable.
[build]: Use 'with-extensions'. Add call to 'sql-schema'. Remove GUIX
from INPUTS.
* gnu/system/file-systems.scm (%store-prefix): Check whether
'%store-prefix' is defined.
* guix/scripts/pack.scm (self-contained-tarball)[not-config?]
[libgcrypt, schema]: New variables.
[build]: Wrap in 'with-extensions'. Adjust imported module list to use
'make-config.scm' for (guix config).
The previous implementation of 'read-reference-graph' was good enough
for many use cases, but it discarded the graph structure, which is
useful information in some cases.
* guix/build/store-copy.scm (<store-info>): New record type.
(read-reference-graph): Rewrite to return a list of <store-info>.
(closure-size, populate-store): Adjust accordingly.
* gnu/services/base.scm (references-file): Adjust accordingly.
* gnu/system/vm.scm (system-docker-image): Likewise.
* guix/scripts/pack.scm (squashfs-image, docker-image): Likewise.
* tests/gexp.scm ("gexp->derivation #:references-graphs"): Likewise.
Previously things like:
guix pack -R guile guile-json
would lead to a profile without Guile-JSON, and to an 'etc/profile' that
does not define 'GUILE_LOAD_PATH'.
* guix/scripts/pack.scm (wrapped-package)[build]: Link top-level entries
of PACKAGE into OUTPUT.
* guix/scripts/pack.scm (self-contained-tarball)[build](symlink->directives):
Do not add a 'directory' directive for "/". Previously, as soon as we
were using '-S /bin=bin' or similar, we would add every entry a second
time in the tarball; this would translate as hard links in the tarball,
which tar < 1.30 sometimes fails to extract.
Pass symlinks defined in DIRECTIVES to 'tar'.
* gnu/packages/aux-files/run-in-namespace.c: New file.
* Makefile.am (AUX_FILES): Add it.
* guix/scripts/pack.scm (<c-compiler>): New record type.
(c-compiler, bootstrap-c-compiler, c-compiler-compiler): New procedures.
(self-contained-tarball): Use
'relative-file-name' for the SOURCE -> TARGET symlink.
(docker-image): Add 'defmod' to please Geiser.
(wrapped-package, map-manifest-entries): New procedures.
(%options, show-help): Add --relocatable.
(guix-pack): Honor it.
Fixes a regression introduced in
aad16cc196.
Reported by Julien Lepiller.
* guix/scripts/pack.scm (guix-pack)[manifest-from-args]: In
'match-lambda', add clause for single packages.
* tests/guix-pack.sh: Add test for '-e'.
Previously they would silently be ignored.
* guix/scripts/pack.scm (guix-pack)[manifest-from-args]: Add 'store'
parameter. Call 'options->transformation' and use it.
Move 'with-store' and 'parameterize' around the 'let'.
* tests/guix-pack.sh: Add test using '--with-source'.
* guix/scripts/pack.scm: Do not autoload (gnu packages base).
Previously, we could end up with 'canonical-package' as an unbound
variable because of the incorrect autoload spec.
* guix/docker.scm (build-docker-image): Rename "path" argument to
"prefix" to reflect the fact that it is used as a prefix for the
symlink targets. Add the "paths" argument, and remove the "closure"
argument, since it is now redundant. Add a "transformations"
argument.
* guix/scripts/pack.scm (docker-image): Read the profile's reference
graph and provide its paths to build-docker-image via the new "paths"
argument.
* guix/scripts/pack.scm (bootstrap-xz): New variable.
(%options) <--bootstrap>: New option.
(show-help): Document the new --bootstrap option.
(guix-pack): When --bootstrap is specified, use the bootstrap Guile,
tar, and xz to build the pack, and do not use any profile hooks or
locales.
* doc/guix.texi (Invoking guix pull): Document the new --bootstrap
option.
* tests/guix-pack.sh: New file.
* Makefile.am (SH_TESTS): Add guix-pack.sh.
* gnu/packages/package-management.scm (guix) <inputs>: Add util-linux.
Ludovic Courtès