This fixes a regression in 'retrieve-files*' introduced in
896fec476f, whereby (guix scripts offload)
would not read the initial sexp now sent by the remote host via
'store-export-channel'. This would effectively prevent file retrieval
entirely when offloading.
* guix/ssh.scm (retrieve-files*): New procedure, like former
'retrieve-files' but with an extra #:import parameter.
(retrieve-files): Rewrite in terms of 'retrieve-files*'.
(file-retrieval-port): Make private.
* guix/scripts/offload.scm (transfer-and-offload): Pass #:import to
'retrieve-files*'.
(retrieve-files*): Remove.
This was harmless but non-compliant and unnecessary.
* guix/scripts/publish.scm <top level>: Rename "Guix-Nar-Compression" to
"X-Nar-Compression" as should have always been.
(render-nar, nar-response-port): Adjust accordingly.
(strip-headers): New procedure.
(sans-content-length, with-content-length): Use it.
This adjusts the workaround for <http://bugs.gnu.org/21093> so that it's
not limited to a single content-type.
* guix/scripts/publish.scm (render-nar/cached): Add the 'x-raw-file'
header on the response.
(render-content-addressed-file): Likewise.
(with-content-length): Remove the 'x-raw-file' header.
(http-write): Instead of dispatching on 'application/octet-stream',
check whether 'x-raw-file' is set to determine whether to spawn a
thread.
This saves 18 bytes on each 404 narinfo response.
* guix/scripts/publish.scm (render-narinfo): Pass #:phrase to
'not-found'.
(render-narinfo/cached): Likewise.
* guix/scripts/build.scm (numeric-extension?, tarball-base-name): New
procedures, formerly in 'package-with-source'.
(transform-package-source)[new-sources]: Look for '=' in URI. Each
element of the list of now a (PKG VERSION SOURCE) tuple.
Pass VERSION to 'package-with-source'.
(package-with-source): Add 'version' parameter and honor it.
* tests/scripts-build.scm ("options->transformation, with-source, PKG=URI")
("options->transformation, with-source, PKG@VER=URI"): New tests.
* doc/guix.texi (Package Transformation Options): Document the new
forms.
Fixes <https://bugs.gnu.org/29761>.
Reported by Martin Castillo <castilma@uni-bremen.de>.
* guix/scripts/gc.scm (argument->verify-options): New procedure.
(%options) ["verify"]: Adjust to use it.
* tests/guix-gc.sh: Add test.
* guix/scripts/offload.scm (check-machine-status): New procedure.
(guix-offload): Call it when the argument is "status".
* doc/guix.texi (Daemon Offload Setup): Document it.
* guix/scripts/system.scm (show-help): Add expression option.
(%options): Ditto.
(guix-system): Allow commands taking a file as an argument to use an
expression instead.
(process-action): Read operating-system from expression or file.
* doc/guix.texi (Invoking guix system): Introduce the expression option.
* gnu/bootloader/extlinux.scm (install-extlinux): Factorize bootloader
writing in a new procedure write-file-on-device defined in (gnu build
bootloader).
* gnu/build/bootloader.scm: New file.
* gnu/local.mk (GNU_SYSTEM_MODULES): Add new file.
* gnu/system/vm.scm (qemu-img): Adapt to import and use (gnu build bootloader)
module during derivation building.
* gnu/scripts/system.scm (bootloader-installer-derivation): Ditto.
* guix/scripts/lint.scm (check-vulnerabilities): Also check for CVEs
listed as mitigated in the package properties.
* tests/lint.scm ("cve: known safe from vulnerability"): New test.
* guix/scripts/offload.scm (build-machines): Comment out
'(set! %fresh-auto-compile #t)' since with Guile 2.2.3 it could lead to
an actual rebuild of everything that gets loaded from there on. See
<https://bugs.gnu.org/29226>.
* guix/ui.scm (load*): Likewise.
Until now it would print the name of each store item being copied, which
was verbose and unhelpful.
* guix/scripts/system.scm (copy-closure): Use 'progress-reporter/bar'
and 'call-with-progress-reporter'.
(guix-system): Parameterize 'current-terminal-columns'.
* guix/scripts/system.scm (copy-item): Add 'references' argument and
remove 'references*' call. Turn into a non-monadic procedure.
(copy-closure): Remove initial call to 'references*'. Only pass ITEM to
'topologically-sorted*' since that's equivalent. Compute the list of
references corresponding to TO-COPY and pass it to 'copy-item'.
Reported at <https://bugs.gnu.org/27943>
by Danny Milosavljevic <dannym@scratchpost.org>.
* guix/scripts/lint.scm (%distro-directory): New variable.
(check-patch-file-names): Add check for the file name length.
* tests/lint.scm ("patches: file name too long"): New test.
This reverts commit 5f93d97005.
'guix pull' would fail because (guix self) needs 'scheme-files'
from (guix discovery), which was not exported until now.
This mitigates <https://bugs.gnu.org/27284>.
* guix/self.scm: New file.
* Makefile.am (MODULES): Add it.
* build-aux/build-self.scm (libgcrypt, zlib, gzip, bzip2, xz)
(false-if-wrong-guile, package-for-current-guile, guile-json)
(guile-ssh, guile-git, guile-bytestructures): Remove.
(build): Rewrite to simply delegate to 'compiled-guix'.
* gnu/packages.scm (%distro-root-directory): Rewrite to try different
directories.
* guix/discovery.scm (guix): Export 'scheme-files'.
* guix/scripts/pull.scm (build-and-install): Split into...
(install-latest): ... this. New procedure. And...
(build-and-install): ... this, which now takes a monadic value argument.
(indirect-root-added): Remove.
(guix-pull): Call 'add-indirect-root'. Call 'build-from-source' and
pass the result to 'build-and-install'.
Suggested by Marius Bakke <mbakke@fastmail.com>
in <https://bugs.gnu.org/29177>.
* guix/scripts/refresh.scm (all-packages): Pass #:select? to
'fold-packages'.
* guix/scripts/lint.scm (call-with-networking-fail-safe): New procedure.
(with-networking-fail-safe): New macro.
(current-vulnerabilities*): Rewrite in terms of 'with-networking-fail-safe'.
* guix/scripts/challenge.scm (summarize-report-list): New procedure.
(guix-challenge): Use it.
* doc/guix.texi (Invoking guix challenge): Adjust command output in
example.
Fixes <https://bugs.gnu.org/28731>.
Reported by Jan Nieuwenhuizen <janneke@gnu.org>.
* guix/scripts/substitute.scm (at-most): New procedure.
(http-multiple-get): Use it to send at most 1000 requests at once.
Among other things, this removes (guix utils), (guix ui), (guix config),
etc. from the closure of (guix build download), as was the case since
798648515b.
* guix/utils.scm (<progress-reporter>, call-with-progress-reporter):
Move to...
* guix/progress.scm: ... here. New file.
* Makefile.am (MODULES): Add it.
* guix/build/download.scm (current-terminal-columns)
(nearest-exact-integer, duration->seconds, seconds->string)
(byte-count->string, progress-bar, string-pad-middle)
(rate-limited, progress-reporter/file, dump-port*)
(time-monotonic): Move to progress.scm.
* guix/scripts/download.scm: Adjust accordingly.
* guix/scripts/substitute.scm: Likewise.
Fixes <https://bugs.gnu.org/28756>.
* guix/utils.scm (filtered-port): Close the 'input' port for the current
process, and close it upon exit in the child process.
* guix/scripts/substitute.scm (progress-substitution): Display "\n\n" after
the reporter has finished.
Previously we would call 'machine-load' once per machine, which was very
costly when there were many machines. Now we arrange to call it only
once on average (when all the machines have the same 'speed' value).
* guix/scripts/offload.scm (random-seed, shuffle): New procedures.
(choose-build-machine)[machines+slots+loads]: Rename to...
[machines+slots]: ... this. Remove load from the tuples therein.
[undecorate]: Adjust accordingly.
[machine-less-loaded-or-faster?]: Remove.
[machine-faster?]: New procedure.
Sort MACHINES+SLOTS according to 'machine-faster?'. Call
'machine-load?' as the last thing.
* guix/scripts/lint.scm (check-for-updates): New procedure.
(%checkers): Add it.
* guix/scripts/refresh.scm (importer-modules, %updaters): Move
from here ...
* guix/upstream.scm: ... to here.
Fixes <https://bugs.gnu.org/28664>.
Reported by Maxim Cournoyer <maxim.cournoyer@gmail.com>.
* guix/scripts/publish.scm (bake-narinfo+nar): When COMPRESSION is
'none, write NAR instead of doing nothing.
(make-request-handler): Use 'render-nar/cached' for /nar URLs with no
compression.
* tests/publish.scm ("with cache", "with cache, uncompressed"): Adjust
expected result accordingly.
This is a follow-up to commit fb1db38547.
* guix/scripts/build.scm (lazy-util): Remove procedure.
(options->things-to-build): Remove special case for files with ".json"
extension.
* guix/utils.scm (<progress-reporter>): New record type.
(call-with-progress-reporter): New procedure.
* guix/build/download.scm (dump-port*, rate-limited, progress-reporter/file):
New procedures.
(ftp-fetch, http-fetch): Use 'dump-port*'.
(progress-proc): Remove procedure.
* guix/scripts/substitute.scm (progress-report-port): Rewrite in terms of
<progress-reporter>.
(process-substitution): Adjust accordingly.
This allows substitutes to be downloaded from unauthorized servers, as
long as they advertise the same hash and references as one of the
authorized servers.
* guix/scripts/substitute.scm (assert-valid-narinfo): Remove.
(valid-narinfo?): Add #:verbose?. Handle each case of
'signature-case'.
(equivalent-narinfo?): New procedure.
(lookup-narinfos/diverse): Add 'authorized?' parameter and honor it.
[select-hit]: New procedure.
(lookup-narinfo): Add 'authorized?' parameter and pass it.
(process-query): Adjust callers accordingly.
(process-substitution): Remove call to 'assert-valid-narinfo'. Check
whether 'lookup-narinfo' returns true and call 'leave' if not.
* tests/substitute.scm (%main-substitute-directory)
(%alternate-substitute-directory): New variables.
(call-with-narinfo): Make 'narinfo-directory' a parameter. Call
'mkdir-p' to create it. Change unwind handler to check whether
CACHE-DIRECTORY exists before deleting it.
(with-narinfo*): New macro.
("substitute, no signature")
("substitute, invalid hash")
("substitute, unauthorized key"): Change expected error message to "no
valid substitute".
("substitute, unauthorized narinfo comes first")
("substitute, unsigned narinfo comes first")
("substitute, first narinfo is unsigned and has wrong hash")
("substitute, first narinfo is unsigned and has wrong refs")
("substitute, unsigned narinfo comes first")
("substitute, two invalid narinfos"): New tests.
* doc/guix.texi (Substitutes): Explain the new behavior.
* guix/scripts/substitute.scm (%cache-urls): Rename to...
(%default-substitute-urls): ... this.
(substitute-urls): New variable.
(guix-substitute): Use it instead of %CACHE-URLS.
* tests/substitute.scm: Likewise.
* guix/build/download.scm (http-fetch): Complete the hard-coded list of HTTP
redirection status codes.
* guix/http-client.scm (http-fetch): Likewise.
* guix/scripts/lint.scm (probe-uri): Likewise.
Fixes <http://bugs.gnu.org/27157>.
Having a finite default `max-silent-time' value for scripts such as 'guix
environment' could lead to timeouts when building subtitutes; this was
undesirable.
This change also fixes client behavior to match the documentation, which
is that by default the daemon's settings are honored.
* guix/scripts/archive.scm (%default-options): Remove max-silent-time entry.
* guix/scripts/copy.scm (%default-options): Likewise.
* guix/scripts/environment.scm (%default-options): Likewise.
* guix/scripts/pack.scm (%default-options): Likewise.
* guix/scripts/package.scm (%default-options): Likewise.
* guix/scripts/pull.scm (%default-options): Likewise.
* guix/scripts/system.scm (%default-options): Likewise.
Signed-off-by: Ludovic Courtès <ludo@gnu.org>
* gnu/bootloader.scm (<bootloader-configuration>): Deprecate "device" field in
favor of "target" field. This is mostly a renaming but also a generalization
to support UEFI targets being paths to a mounted partition instead of a device
name.
* gnu/system/examples/bare-bones.tmpl:
* gnu/system/examples/desktop.tmpl:
* gnu/system/examples/lightweight-desktop.tmpl:
* gnu/system/examples/vm-image.tmpl:
* gnu/system/install.scm:
* gnu/tests.scm:
* gnu/tests/install.scm:
* gnu/tests/nfs.scm:
* tests/system.scm: Adapt all invocations of bootloader-configuration.
* guix/scripts/system.scm (perform-action): Rename device argument to
bootloader-target.
(process-action): Adapt caller.
* doc/guix.texi (Proceeding with the Installation):
* doc/guix.texi (Bootloader Configuration): Update documentation.
* guix/scripts/download.scm (guix-download): Treat the URL argument as a local
file path when it fails on 'string->uri'. Call 'fetch' with the processed
'uri' instead of the original URL argument.
* tests/guix-download.sh: Adjust accordingly.
* guix/scripts/pull.scm (build-from-source): Add #:commit parameter.
Pass it to BUILD.
(build-and-install): Add #:commit and pass it to 'build-from-source'.
(guix-pull): Pass #:commit to 'build-and-install'.
* guix/scripts/pull.scm (%snapshot-url, with-environment-variable)
(with-PATH): Remove.
(ensure-guile-git!): New procedure.
(%repository-url): New variable.
(%default-options): Add 'repository-url' and 'ref'.
(show-help, %options): Add '--commit' and '--url'.
(temporary-directory, first-directory, interned-then-deleted)
(unpack): Remove.
(build-from-source): Rename 'tarball' to 'source'. Remove call to
'unpack'.
(build-and-install): Rename 'tarball' to 'source'.
(honor-lets-encrypt-certificates!, report-git-error): New procedures.
(with-git-error-handling): New macro.
(guix-pull)[fetch-tarball]: Remove.
Wrap body in 'with-git-error-handling'. Rewrite to use
'latest-repository-commit'.
* build-aux/build-self.scm (build): Print an error message and exit when
GUILE-GIT is #f.
* doc/guix.texi (Invoking guix pull): Mention Git. Document '--commit'
and '--branch'.
Fixes <https://bugs.gnu.org/27820>.
Reported by Hartmut Goebel <h.goebel@crazy-compilers.com>.
* guix/scripts/package.scm (%options) <"-u">: Emit a warning when ARG
starts with "-".
* gnu/bootloader.scm (menu-entry-device-mount-point): New variable. Export it.
(<menu-entry>: New field "device".
* gnu/bootloader/grub.scm (grub-confgiuration-file): Handle <menu-entry>
entries.
* gnu/bootloader/extlinux.scm (extlinux-configuration-file): Handle
<menu-entry> entries.
* gnu/system.scm (menu->entry->boot-parameters): Delete variable.
(boot-parameters->menu-entry): New variable. Export it.
(operating-system-bootcfg): Make OLD-ENTRIES a list of <menu-entry>.
* guix/script/system.scm (reinstall-bootloader): Fix bootcfg usage.
(perform-action): Fix bootcfg usage.
The '%slots' list could grow indefinitely; in practice though,
guix-daemon is likely to restart 'guix offload' often enough.
* guix/scripts/offload.scm (%slots): Remove.
(choose-build-machine): Don't 'set!' %SLOTS. Return the acquired slot
as a second value.
(process-request): Adjust accordingly. Release the returned slot after
'transfer-and-offload'.
This fixes a memory leak that can be seen by running:
(map (lambda _ (machine-load m)) (iota 1000))
* guix/scripts/offload.scm (machine-load): Add call to 'disconnect!'.
Suggested by Mark H Weaver <mhw@netris.org>.
* guix/scripts/publish.scm (nar-expiration-time): New procedure.
(render-narinfo/cached): Use it as the #:entry-expiration passed to
'maybe-remove-expired-cache-entries'.
Previously this operation was linear in the number of requests and
involved costly calls to 'string-contains'.
* guix/scripts/substitute.scm (fetch-narinfos)[hash-part->path]: New
procedure.
[handle-narinfo-response]: Use it for caching when CODE is not 200.
Fixes <https://bugs.gnu.org/25762>.
Reported by Ricardo Wurmus <rekado@elephly.net>.
* guix/scripts/package.scm (canonicalize-profile): Trim trailing slashes
from PROFILE.
Prior to this change, half of the cached narinfos would expire
immediately since they contained the number of nanoseconds instead of
the number of seconds as their date.
* guix/scripts/substitute.scm (time-monotonic) <guile-2.2>: Define, as a
workaround.
Before this change '--substitute-urls' would be ignored.
* guix/scripts/environment.scm (build-environment): Remove redundant
call to 'set-build-options-from-command-line*'.
(guix-environment): Move 'set-build-options-from-command-line' right
after 'with-store'.
Failing to do that, "%p" would be "0" when using "ProxyCommand"
in ~/.ssh/config.
* guix/scripts/copy.scm (send-to-remote-host): Default to port 22.
(retrieve-from-remote-host): Likewise.
* guix/scripts/system.scm (process-action): Pass file-system-type to ...
(perform-action): ... here. Add new keyword argument. Pass new value to ...
(system-derivation-for-action): ... here. Add new keyword argument.
Pass new value to system-disk-image.
* doc/guix.texi (disk-image): Document new option.
* gnu/build/vm.scm (estimated-partition-size): New procedure.
* gnu/system/vm.scm (expression->derivation-in-linux-vm):
Change #:disk-image-size default to 'guess.
[builder]: When DISK-IMAGE-SIZE is 'guess, use
'estimated-partition-size' and compute and estimate of the image size.
(qemu-image): Likewise.
* guix/build/store-copy.scm (file-size, closure-size): New procedures.
* guix/scripts/system.scm (%default-options): Change 'image-size' to
'guess.
* doc/guix.texi (Building the Installation Image): Remove '--image-size'
flag from example.
(Invoking guix system): Document the image size estimate.
Fixes <http://bugs.gnu.org/27262>.
Reported by Mark H Weaver <mhw@netris.org>.
* guix/scripts/package.scm (transaction-upgrade-entry): Check if ENTRY
matches 'manifest-transaction-removal-candidate?' and return TRANSACTION
if it does.
(process-actions): Move 'options->removable' from step 2 to step 1.
Fixes <http://bugs.gnu.org/27261>.
Reported by Mark H Weaver <mhw@netris.org>.
* guix/scripts/package.scm (transaction-upgrade-entry): Add call to
'warning' when NAME cannot be found in the package set.
* guix/scripts/refresh.scm (check-for-package-update): Use
'version-compare' instead of 'version>?'. When WARN? is true, print
something for the '=' and '<' cases.
Reported by Efraim Flashner.
This is a followup to a654dc4bcf.
* guix/profiles.scm (profile-derivation): Add #:allow-collisions? and
honor it.
* guix/scripts/environment.scm (inputs->profile-derivation): Pass
#:allow-collisions? #f to 'profile-derivation'.
* tests/guix-environment.sh: Test "guix environment guix".
* guix/scripts/package.scm (find-packages-by-description): Rewrite to
compute a score based on the number of regexps matched and the number of
matches for each regexp. Sort according to this score and return it as
a second value.
(process-query) <'search>: Capture the two return values of
'find-packages-by-description'. Pass #:extra-fields to
'package->recutils'.
* doc/guix.texi (Invoking guix package): Mention relevance, give an
example.
This avoids the open/fstat/close syscalls upon a cache hit that we had
with the previous idiom:
(call-with-input-file file read-derivation)
where caching happened in 'read-derivation' itself.
* guix/derivations.scm (%read-derivation): Rename to...
(read-derivation): ... this.
(read-derivation-from-file): New procedure.
(derivation-prerequisites, substitution-oracle)
(derivation-prerequisites-to-build):
(derivation-path->output-path, derivation-path->output-paths):
(derivation-path->base16-hash, map-derivation): Use
'read-derivation-from-file' instead of (call-with-input-file …
read-derivation).
* guix/grafts.scm (item->deriver): Likewise.
* guix/scripts/build.scm (log-url, options->things-to-build): Likewise.
* guix/scripts/graph.scm (file->derivation): Remove.
(derivation-dependencies, %derivation-node-type): Use
'read-derivation-from-file' instead.
* guix/scripts/offload.scm (guix-offload): Likewise.
* guix/scripts/perform-download.scm (guix-perform-download): Likewise.
* guix/scripts/publish.scm (load-derivation): Remove.
(narinfo-string): Use 'read-derivation-from-file'.
Ludovic Courtès