Reported by Arne Babenhauserheide <arne.babenhauserheide@kit.edu>.
* guix/scripts/pull.scm (unpack)[builder](compile-file*): Change to run
'compile-file' in a child process. This limits memory usage; before
that memory usage was proportional to the number of files to compile.
* guix/scripts/substitute-binary.scm (n-par-map*): New procedure.
(guix-substitute-binary): Use it instead of 'n-par-map'.
Reported by Nikita Karetnikov and Eric Bavier.
* guix/ui.scm (read/eval): New procedure.
(read/eval-package-expression): Use it.
* guix/scripts/build.scm (derivations-from-package-expressions): Rename to...
(derivation-from-expression): ... this. Accept procedures, under the
assumption that they are monadic thunk.
(show-help): Adjust accordingly.
(guix-build): Ditto.
* tests/guix-build.sh: Add test.
* doc/guix.texi (Invoking guix build): Augment description of '-e'.
* guix/scripts/substitute-binary.scm (with-timeout): Update comment to
mention the fix's commit ID.
(fetch): In the 'with-timeout' handler, close PORT only one Guile
versions < 2.0.9.39. Before that, on Guile >= 2.0.9.39, the HTTP
client would end up trying to read from a closed file descriptor.
* guix/scripts/substitute-binary.scm (%lookup-threads): New variable.
(guix-substitute-binary): Use 'n-par-map' instead of 'par-map' for
batch 'lookup-narinfo' calls.
Fixes <http://bugs.gnu.org/15756>.
* guix/profiles.scm (<manifest-pattern>): New record type.
(remove-manifest-entry): Remove.
(entry-predicate, manifest-matching-entries): New procedures.
(manifest-remove): Accept a list of <manifest-pattern>.
(manifest-installed?): Replace 'name' parameter by 'pattern', a
<manifest-pattern>.
* guix/scripts/package.scm (options->removable): Return a list of
<manifest-pattern>.
(guix-package)[process-action]: Use 'manifest-matching-entries' to
compute the list of packages to remove.
* tests/profiles.scm: New file.
* Makefile.am (SCM_TESTS): Add it.
* guix/scripts/package.scm (options->removable): New procedure.
(guix-package)[process-actions]: Use it. Rename 'remove*' to 'remove'
and 'install*' to 'install'.
* guix/ui.scm (package-specification->name+version+output): New
procedure.
* guix/scripts/package.scm (specification->package+output): Use it.
* tests/ui.scm ("package-specification->name+version+output"): New test.
* guix/scripts/package.scm (<manifest-entry>): Add 'inputs' field.
(manifest=?, lower-input): New procedure.
(profile-derivation)[builder]: Add #:log-port argument to
'union-build'.
[ensure-valid-input]: Remove.
Add each entry's inputs to the input list.
(options->installable): Return just the list of entries.
[package->manifest-entry]: Set 'inputs' field.
[canonicalize-deps]: Rename to...
[deduplicate]: ... this. Remove input fiddling.
(guix-package)[process-actions]: Use 'manifest=?' to compare the new
and old manifests. Pass directly PROF-DRV to 'show-what-to-build'.
Pass #:print-build-trace #f to 'set-build-options'.
* guix/scripts/package.scm (%user-environment-directory): Rename to...
(%user-profile-directory): ... this. Update users accordingly.
(profile-derivation): Use the term "profile" instead of "user
environment", and use "profile" as the derivation name.
* guix/scripts/package.scm (<manifest>, <manifest-entry>): New record
types.
(make-manifest, read-manifest, manifest->sexp, sexp->manifest,
read-manifest, write-manifest, remove-manifest-entry, manifest-remove,
manifest-installed?): New procedures.
(profile-derivation): Take a manifest as the second parameter. Use
'manifest->sexp'. Expect <manifest-entry> objects instead of
"tuples". Adjust callers accordingly.
(search-path-environment-variables): Changes 'packages' parameter to
'entries'. Rename 'package-in-manifest->package' to
'manifest-entry->package'; expect <manifest-entry> objects.
(display-search-paths): Rename 'packages' to 'entries'.
(options->installable): Change 'installed' to 'manifest'. Have
'canonicalize-deps' return name/path tuples instead of raw packages.
Rename 'package->tuple' to 'package->manifest-entry'. Use
<manifest-entry> objects instead of tuples.
(guix-package)[process-actions]: Likewise. Rename 'packages' to
'entries'.
[process-query]: Use 'manifest-entries' instead of
'manifest-packages'.
* guix/scripts/package.scm (find-package): Rename to...
(specification->package+output): ... this. Rename 'name' parmameter
to 'spec'. Return a package and output name instead of a tuple.
(options->installable): New procedure
(guix-package)[process-actions]: Use it, and remove corresponding
code.
* guix/scripts/package.scm (switch-to-previous-generation): New function.
(roll-back): Use the new function instead of 'switch-link'.
(show-help): Add '--delete-generations'.
(%options): Likewise.
(guix-package)[process-actions]: Add 'current-generation-number',
'display-and-delete', and 'delete-generation'. Add support for
'--delete-generations', and reindent the code.
* tests/guix-package.sh: Test '--delete-generations'.
* doc/guix.texi (Invoking guix-package): Document '--delete-generations'.
* guix/scripts/package.scm (guix-package)[process-query]: Show that a
generation is the current one if the profile points to it.
* tests/guix-package.sh: Test it.
* guix/scripts/package.scm (guix-package)[process-query]: Change
'list-generation' to not list the zeroth generation.
* tests/guix-package.sh: Test it.
* doc/guix.texi (Invoking guix package): Document it, and use the
right term when talking about generations.
* guix/scripts/package.scm (guix-package)[process-query]: Exit with 1
when a generation does not exist or the profile points to the zeroth
generation.
* tests/guix-package.sh: Test the former case.
Suggested by Andreas Enge <andreas@enge.fr>.
* guix/scripts/package.scm (guix-package)[list-generations,
list-installed]: Reverse the result of 'manifest-packages'.
* doc/guix.texi (Invoking guix package): Document the order of packages
for '--list-generations' and '--list-installed'.
Fixes <http://bugs.gnu.org/15428>.
Reported by Cyrill Schenkel <cyrill.schenkel@gmail.com>.
* guix/scripts/pull.scm (guix-pull): 'unpack' returns a single value.
* guix/derivations.scm (derivation->output-path,
derivation->output-paths): New procedures.
(derivation-path->output-path): Use 'derivation->output-path'.
(derivation-path->output-paths): Use 'derivation->output-paths'.
(derivation): Accept 'derivation?' objects as inputs. Return a single
value.
(build-derivations): New procedure.
(compiled-modules): Use 'derivation->output-paths'.
(build-expression->derivation)[source-path]: Add case for when the
input matches 'derivation?'.
[prologue]: Accept 'derivation?' objects in INPUTS.
[mod-dir, go-dir]: Use 'derivation->output-path'.
* guix/download.scm (url-fetch): Adjust to the single-value return.
* guix/packages.scm (package-output): Use 'derivation->output-path'.
* guix/scripts/build.scm (guix-build): When the argument is
'derivation-path?', pass it through 'read-derivation'.
Use 'derivation-file-name' to print out the .drv file names, and to
register them. Use 'derivation->output-path' instead of
'derivation-path->output-path'.
* guix/scripts/package.scm (roll-back): Adjust to the single-value
return.
(guix-package): Use 'derivation->output-path'.
* guix/ui.scm (show-what-to-build): Adjust to deal with 'derivation?'
objects instead of .drv file names.
* gnu/system/grub.scm (grub-configuration-file): Use
'derivation->output-path' instead of 'derivation-path->output-path'.
* gnu/system/vm.scm (qemu-image, system-qemu-image): Likewise.
* tests/builders.scm, tests/derivations.scm, tests/packages.scm,
tests/store.scm, tests/union.scm: Adjust to the new calling
convention.
* doc/guix.texi (Defining Packages, The Store, Derivations): Adjust
accordingly.
* guix/scripts/package.scm (profile-numbers): Rename to 'generation-numbers'.
(previous-profile-number): Rename to 'previous-generation-number'.
(profile-number): Rename to 'generation-number'.
(roll-back): Rename 'previous-profile' to 'previous-generation'.
* guix/scripts/substitute-binary.scm (guix-substitute-binary)["--substitute"]:
Show the Nar size, when available.
* guix/ui.scm (show-what-to-build): Add 'TODO'.
This fixes Guile version comparisons when (version) has a
vendor-specific suffix.
Reported by Andreas Enge <andreas@enge.fr>.
* guix/utils.scm (guile-version>?): New procedure.
* tests/utils.scm ("guile-version>? 1.8", "guile-version>? 10.5"): New
tests.
* guix/scripts/substitute-binary.scm (fetch, progress-report-port): Use
`guile-version>?' instead of `version>?'.
* guix/http-client.scm (when-guile<=2.0.5, http-fetch): Likewise.
* guix/scripts/substitute-binary.scm (fetch): In the `file' case, open
with the `b' flag, so that the coding cookie reading thing doesn't
lead to buffering some of the data (on 2.0.5).
* tests/utils.scm ("filtered-port, file"): Open with `r0b'. Fixes a
test failure with Guile 2.0.5 whereby the first byte of FILE would be
missing from DECOMPRESSED.
* guix/scripts/package.scm (guix-package)[process-actions](same-package?):
New procedure. Use it instead of `alist-delete' when filtering out
duplicate packages from the profile.
This fixes a bug whereby a command like
guix package -e '(@@ (gnu packages base) gnu-make-boot0)'
would only succeed when the outputs of that package are available, and
otherwise fail with "/nix/store/... is not valid".
* guix/scripts/package.scm (guix-package)[process-action](package->tuple):
Leave P as is in the result. Move `package->tuple' call from INSTALL*
to INSTALL.
* guix/gnu-maintenance.scm (latest-release): Add `ftp-close' and
`ftp-open' keyword parameters.
* guix/scripts/package.scm (ftp-open*): New variable.
(check-package-freshness): Call `latest-release' with `ftp-open*' and
a no-op procedure.
* guix/scripts/substitute-binary.scm (%regexp-exec-mutex, string->uri):
Remove.
(regexp-exec): Replace this global binding by a thread-safety wrapper.
(fields->alist): Remove `with-mutex', and directly alias `recutils->alist'.
* guix/records.scm (%recutils-field-rx): New variable.
(recutils->alist): New procedure, formerly known as `fields->alist'.
* guix/scripts/substitute-binary.scm (fields->alist): Use it.
* tests/records.scm ("recutils->alist"): New test.
* guix/derivations.scm (derivation-hash): Add comment as to why we keep
the plain `sha256' call.
* guix/scripts/download.scm (guix-download): Use `port-sha256' instead
of (compose sha256 get-bytevector-all).
* guix/scripts/refresh.scm (update-package): Likewise.
* guix/web.scm (open-socket-for-uri): New procedure.
(http-fetch): Add `port' keyword parameter; use it.
* guix/scripts/substitute-binary.scm (%random-state): New variable.
(with-timeout): Wait a little before retrying.
(fetch): Use `open-socket-for-uri', and keep a copy of the socket in
variable `port'. Close PORT upon timeout.
* guix/scripts/substitute-binary.scm (decompressed-port): Improve docstring.
(progress-report-port): New procedure.
(guix-substitute-binary)["--substitute"]: Use it to report progress.
* guix/build/download.scm: Export `progress-proc' and `uri-abbreviation'.
* guix/gnu-maintenance.scm (download-tarball): Add a 'key-download'
keyword argument and pass it to 'gnupg-verify*'. Make
'archive-type' a keyword argument.
(package-update): Add a 'key-download' keyword argument. Pass
'archive-type' and 'key-download' keyword arguments to
'download-tarball'.
* guix/gnupg.scm: Import (ice-9 i18n) and (guix ui).
(gnupg-verify*): Add a 'key-download' keyword argument and adjust
'gnupg-verify*' to use it. Make 'server' a keyword argument.
* guix/scripts/refresh.scm (show-help, %options): Add and document
'--key-download'.
(update-package): Add a 'key-download' keyword argument and pass it
to 'package-update'.
(guix-refresh): Pass 'key-download' to 'update-package'. Limit
lines to a maximum of 79 characters.
Suggested by Andreas Enge <andreas@enge.fr>.
* guix/scripts/substitute-binary.scm (with-networking): New macro.
(guix-substitute-binary): Wrap the body in `with-networking'.
* guix/scripts/package.scm (guix-package)[ensure-default-profile]: Check
whether S is true before checking its owner. Fixes the case where we
built %PROFILE-DIRECTORY just above.
* guix/scripts/package.scm (guix-package)[ensure-default-profile]: Check
the owner of %PROFILE-DIRECTORY. Report an error when the owner is
not the current user. Add `rtfm' procedure.
* doc/guix.texi (Invoking guix package): Mention the ownership test.
This fixes a bug whereby `read-response' would read more than just the
response, with the extra data going into the port's buffer; the
"bzip2 -dc" process spawned by `filtered-port' would not see the those
buffered data, which are definitely lost, and would bail out with
"bzip2: (stdin) is not a bzip2 file."
* guix/utils.scm (filtered-port): Document that INPUT must be
unbuffered.
* guix/web.scm (http-fetch): Add `buffered?' parameter. Call
`open-socket-for-uri' explicitly, and call `setvbuf' when BUFFERED? is
false. Pass the port to `http-get'. Close it upon 301/302.
* guix/scripts/substitute-binary.scm (fetch): Add `buffered?'
parameter. Pass it to `http-fetch'; honor it for `file' URIs.
(guix-substitute-binary): Call `fetch' with #:buffered? #f for port RAW.
* tests/utils.scm ("filtered-port, file"): Open FILE as unbuffered.
* guix/scripts/substitute-binary.scm (%regexp-exec-mutex): New variable.
(string->uri): New procedure.
(fields->alist): Wrap `regexp-exec' call in `with-mutex'.
* guix/ftp-client.scm (ftp-open): Let exceptions through.
* guix/scripts/package.scm (waiting): Wrap EXP in a `dynamic-wind', so
the line is always cleared.
* guix/scripts/hash.scm (guix-hash)[eof->null]: New procedure.
Use it to convert the EOF object to the empty bytevector.
* tests/guix-hash.sh: New file.
* Makefile.am (SH_TESTS): Add it.
* guix/scripts/refresh.scm (%options): Add `--key-server' and `--gpg'.
(show-help): Update accordingly.
(update-package): New procedure, formerly in `guix-refresh'.
(guix-refresh): Use it. Parameterize `%openpgp-key-server' and
`%gpg-command'.
* guix/scripts/package.scm (guix-package)[process-actions](package->tuple):
Put the output path in the tuple, not the derivation path.
* tests/guix-package.sh: Add test.
Before this, something like "guix package -i glibc" could fail because
glibc lists linux-libre-headers as a propagated input (which would be
added as a dependency in the manifest) but the linux-libre-headers
output could be unavailable, leading to an error like this:
path `/nix/store/4v2bk8sx5cm166gks3fi3q7d9zchibnk-linux-libre-headers-3.3.8' is not valid
This patch adds such dependencies as inputs of the profile derivation.
* guix/scripts/package.scm (profile-derivation): Accept package objects
in the `deps' field of an element of PACKAGES. Convert them to their
output path for BUILDER, and add them to the inputs of the
`build-expression->derivation' call.
(input->name+path): When INPUT doesn't contain a package object,
return it as is.
(guix-package)[process-actions](canonicalize-deps): Expect DEPS to
contain package objects, and leave them as is.
* guix/scripts/refresh.scm (guix-refresh): When updating a source file,
gracefully handle the case where TARBALL is #f.
Reported by Nikita Karetnikov <nikita@karetnikov.org>.
* guix/scripts/package.scm (search-path-environment-variables): Prefer
%USER-ENVIRONMENT-DIRECTORY when it points to PROFILE.
(display-search-paths): Use 3 spaces for indentation.
* guix/scripts/package.scm (guix-package)[find-package]: Add optional
parameter `output'. Use it.
[process-actions]: When computing UPGRADE, pass OUTPUT to
`find-package'.
* guix/scripts/substitute-binary.scm (filtered-port): Move to utils.scm.
(decompressed-port): Upon "none", return '() as the second value.
(guix-substitute-binary): Expect `decompressed-port' to return a list
of PIDs as its second value.
* guix/utils.scm (filtered-port): New procedure. Add case for when
INPUT is not `file-port?'.
* tests/utils.scm ("filtered-port, file", "filtered-port, non-file"):
New tests.
* guix/scripts/refresh.scm (%options): Add `--select'.
(show-help): Likewise. Augment initial help text.
(guix-refresh)[core-package?]: New procedure.
Use it when selecting packages.
This should fix `substitute-binary --query' on Guile 2.0.5.
* guix/web.scm: New file.
* Makefile.am (MODULES): Add it.
* po/POTFILES.in: Add it.
* guix/gnu-maintenance.scm (http-fetch): Remove.
(%package-list-url): Turn into a URI.
(official-gnu-packages): Add #:text? #t to `http-fetch' call.
* guix/scripts/substitute-binary.scm (fetch): Remove `http' case, and
use `http-fetch' instead.
* guix/scripts/hash.scm: New file.
* Makefile.am (MODULES): Add it.
* po/POTFILES.in: Add it.
* doc/guix.texi (Invoking guix hash): New node.
(Defining Packages): Add a cross-reference to the 'Invoking guix
hash' node.
* guix/ui.scm (define-diagnostic): New macro, which is based on the
previous version of 'warning'.
(warning, leave): Redefine using 'define-diagnostic'.
(report-error): New macro.
(install-locale): Use 'warning' instead of 'format'.
(call-with-error-handling): Adjust 'leave'.
* gnu/packages.scm (package-files): Use 'warning' instead of 'format'.
* guix/gnu-maintenance.scm (http-fetch): Use 'warning' and 'leave'.
* guix/scripts/build.scm (derivations-from-package-expressions, guix-build):
Adjust 'leave'.
* guix/scripts/download.scm (guix-download): Adjust 'leave'.
* guix/scripts/gc.scm (size->number, %options): Adjust 'leave'.
* guix/scripts/package.scm (roll-back, guix-package): Adjust 'leave'.
* po/POTFILES.in: Add 'guix/gnu-maintenance.scm'.
* guix/derivations.scm (derivation-input-output-paths): New procedure.
(derivation-prerequisites-to-build): New `use-substitutes?' keyword
argument. Change two return the list of substitutable paths as a
second argument.
* guix/ui.scm (show-what-to-build): Turn `dry-run?' into a keyword
argument. New `use-substitutes?' keyword argument. Use `fold2' and
adjust to use both return values of
`derivation-prerequisites-to-build'. Display what will/would be
downloaded.
* guix/scripts/build.scm (guix-build): Adjust accordingly.
* guix/scripts/package.scm (guix-package): Likewise.
* tests/derivations.scm ("derivation-prerequisites-to-build and
substitutes"): New test.
* guix/scripts/packages.scm (guix-package) [process-actions]: When upgrading,
use "" when REGEXP is #f.
* doc/guix.texi: update the documentation accordingly.
* guix/scripts/substitute-binary.scm (lookup-narinfo): Force CACHE when
passing it to `fetch-narinfo'.
(guix-substitute-binary): Delay calls to `open-cache'.
* guix/scripts/substitute-binary.scm (%narinfo-cache-directory,
%narinfo-ttl, %narinfo-negative-ttl): New variables.
(with-atomic-file-output, object->fields, read-narinfo,
write-narinfo, narinfo->string, string->narinfo, lookup-narinfo): New
procedures.
(fetch-narinfo): Adjust to use `read-narinfo'.
(guix-substitute-binary): Ensure the existence of
%NARINFO-CACHE-DIRECTORY. Use `lookup-narinfo' instead of
`fetch-narinfo'.
* guix/scripts/package.scm (roll-back): Use `format', not `leave' when
indicating "already at the empty profile". Fixes a regression
introduced in a2011be5df. Reported by
Nikita Karetnikov <nikita@karetnikov.org>.
This allows build outputs to be transparently downloaded from
http://hydra.gnu.org, for example.
* config-daemon.ac: Check for `gzip', `bzip2', and `xz'.
* guix/config.scm.in (%gzip, %bzip2, %xz): New variable.
* guix/scripts/substitute-binary.scm (fetch): Return SIZE as a second value.
(<narinfo>): Change `url' to `uri'.
(make-narinfo): Rename to...
(narinfo-maker): ... this. Handle relative URLs.
(fetch-narinfo): Adjust accordingly.
(filtered-port, decompressed-port): New procedures.
(guix-substitute-binary): Implement the `--substitute' case.
* tests/store.scm ("substitute query"): Use (%store-prefix) instead
of (getenv "NIX_STORE_DIR").
("substitute"): New test.
* guix/ui.scm (program-name, guix-warning-port): New variables.
(warning): New macro.
(guix-main): Parametrize PROGRAM-NAME.
* guix/scripts/build.scm, guix/scripts/download.scm,
guix/scripts/gc.scm, guix/scripts/package.scm: Adjust to use `leave'
and `warning' consistently.
* guix/scripts/substitute-binary.scm (guix-substitute-binary)["--query"]:
Emit blank lines only after the complete list of store paths has been
returned.
* guix/scripts/substitute-binary.scm: New file.
* Makefile.am (MODULES): Add it.
* nix/scripts/substitute-binary.in: New file.
* config-daemon.ac: Produce nix/scripts/substitute-binary.
* daemon.am (nodist_pkglibexec_SCRIPTS): Add
nix/scripts/substitute-binary.
* guix/store.scm (substitutable-path-info): Use the
`query-substitutable-path-infos' RPC.
* nix/nix-daemon/guix-daemon.cc (main): Honor `NIX_SUBSTITUTERS'.
* pre-inst-env.in: Set `NIX_SUBSTITUTERS'.
* test-env.in: Leave `NIX_SUBSTITUTERS' unchanged. Set
`GUIX_BINARY_SUBSTITUTE_URL, and create
$NIX_STATE_DIR/substituter-data.
Run `guix-daemon' within `./pre-inst-env'.
* tests/store.scm ("substitute query"): New test.
* guix/gnu-maintenance.scm (http-fetch): Error out when DATA is #f.
* guix/scripts/package.scm (check-package-freshness): Wrap
`gnu-package?' call in `false-if-exception'.
Reported by Cyril Roelandt <tipecaml@gmail.com>.
* guix/store.scm (&nix-connection-error): New condition type.
(open-connection): Translate `system-error' during the `connect' call
into `&nix-connection-error'.
* guix/ui.scm (call-with-error-handling): Add case for `nix-connection-error?'.
* guix/scripts/package.scm (guix-package): Move `open-connection' call
within `with-error-handling'.
* guix/scripts/pull.scm (guix-pull): Likewise.
* guix/scripts/download.scm (guix-download): Move body within
`with-error-handling'.
* guix/gnu-maintenance.scm (gnu-package?): New procedure.
* guix/scripts/package.scm (waiting): New macro.
(check-package-freshness): New procedure.
(guix-package)[process-actions]: Use it.
* doc/guix.texi (Invoking guix package): Mention the feature.
* guix/scripts/package.scm (read/eval-package-expression): New
procedure.
(show-help): Add `-e'.
(%options): Likewise.
(guix-package)[process-actions]: Handle ('install . p) pairs, where P
is a package.
* tests/guix-package.sh: Add `boot_make_drv'. Use `-i $boot_make_drv'
once, and then use `-e $boot_make'.
* doc/guix.texi (Invoking guix package): Document `-e'.
* guix/ui.scm (show-what-to-build): Return (length req*).
* guix/scripts/pull.scm (guix-pull): Print an "already up to date"
message when there's nothing to build.
* guix/scripts/pull.scm: New file.
* Makefile.am (MODULES): Add it.
* doc/guix.texi (Invoking guix pull): New node.
(Invoking guix package): Add cross-ref to it.
* guix/ui.scm (config-directory): New procedure.
* scripts/guix.in: When `GUIX_UNINSTALLED' is undefined, add
$XDG_CONFIG_HOME/guix/latest to the search path.
* po/POTFILES.in: Add guix/scripts/pull.scm.
Ludovic Courtès