On a profile with 280 packages, this reduces the number of
'valid-paths?' RPCs made by 'guix package -nu' from 6K to 500.
* guix/derivations.scm (derivation-prerequisites-to-build)[built?]:
Memoize 'valid-path?' calls.
This reduces max RSS from 1.3G to 1.0G.
* guix/derivations.scm (invalidate-derivation-caches!): New procedure.
* build-aux/hydra/gnu-system.scm (hydra-jobs): Use it. Add 'format'
call.
This has little or no run-time impact and slightly reduces the memory
footprint.
* guix/derivations.scm (derivation-hash): Replace 'mlambda' with
'lambda'.
This avoids the open/fstat/close syscalls upon a cache hit that we had
with the previous idiom:
(call-with-input-file file read-derivation)
where caching happened in 'read-derivation' itself.
* guix/derivations.scm (%read-derivation): Rename to...
(read-derivation): ... this.
(read-derivation-from-file): New procedure.
(derivation-prerequisites, substitution-oracle)
(derivation-prerequisites-to-build):
(derivation-path->output-path, derivation-path->output-paths):
(derivation-path->base16-hash, map-derivation): Use
'read-derivation-from-file' instead of (call-with-input-file …
read-derivation).
* guix/grafts.scm (item->deriver): Likewise.
* guix/scripts/build.scm (log-url, options->things-to-build): Likewise.
* guix/scripts/graph.scm (file->derivation): Remove.
(derivation-dependencies, %derivation-node-type): Use
'read-derivation-from-file' instead.
* guix/scripts/offload.scm (guix-offload): Likewise.
* guix/scripts/perform-download.scm (guix-perform-download): Likewise.
* guix/scripts/publish.scm (load-derivation): Remove.
(narinfo-string): Use 'read-derivation-from-file'.
* guix/derivations.scm (derivation-prerequisites-to-build): Rename
#:substitutable? to #:substitutable-info.
[derivation-substitutable?]: Rename to...
[derivation-substitutable-info]: ... this. Return a list of <substitutable>.
Second return value is now a list of <substitutable> instead of a list
of strings.
* guix/ui.scm (show-what-to-build)[substitutable?]: Rename to...
[substitutable-info]: ... this.
Adjust to new 'derivation-prerequisites-to-build' return value type.
* tests/derivations.scm ("derivation-prerequisites-to-build and
substitutes"): Adjust.
("derivation-prerequisites-to-build and substitutes, local build"):
Likewise.
* guix/derivations.scm (substitution-oracle): Use
'substitution-path-info' instead of 'substitution-paths'. Turn SUBST
into a vhash from path to <substitutable>. Change the returned
procedure to provide a <substitutable> instead of a Boolean.
* tests/derivations.scm ("substitution-oracle and #:substitute? #f"):
Mock 'substitutable-path-info' instead of 'substitutable-paths'.
Reported by Mark H Weaver <mhw@netris.org>
at <https://lists.gnu.org/archive/html/guix-devel/2017-04/msg00457.html>.
* guix/derivations.scm (build-expression->derivation): Use a string
output port for the expression. This reverts part of
2dce88d5bb.
* tests/derivations.scm ("build-expression->derivation and builder
encoding"): New test.
On Guile 2.2.1, this yields a 5% speedup on:
guix build libreoffice xmonad certbot -n --no-substitutes --no-build-hook
* guix/derivations.scm (derivation->string): Rename to...
(derivation->bytevector): ... this. Use 'open-bytevector-output-port'
instead of 'call-with-output-string'.
(derivation-hash): Remove string-to-bytevector conversion before
'sha256' call.
(build-expression->derivation): Use 'add-data-to-store' and an
bytevector port instead of a string port for the expression.
This avoids connections to substitute servers for derivations that are
not substitutable anyway, such as profiles.
Reported by Andy Wingo.
* guix/derivations.scm (substitution-oracle): Skip derivations that do
not pass 'substitutable-derivation?'.
* tests/derivations.scm ("substitution-oracle and #:substitute? #f"):
New test.
This leads a 13% speedup on 'guix build libreoffice -d' and 18% on
'guix build gnome -d'.
* guix/derivations.scm (%derivation-cache): New variable.
(read-derivation): Use it instead of the private 'cache' variable.
(derivation): Populate %DERIVATION-CACHE before returning.
* guix/derivations.scm (write-derivation)[write-input]: Use 'display'
instead of 'write' for PATH.
Use 'simple-format' instead of 'format', and '~a' instead of '~s' for
SYSTEM and BUILDER.
* guix/derivations.scm (derivation-input<?): New procedure.
(write-derivation)[coalesce-duplicate-inputs]: Remove.
Remove calls to 'sort'.
(coalesce-duplicate-inputs): New procedure.
(derivation-hash): Sort INPUTS and use 'coalesce-duplicate-inputs'.
(derivation)[input->derivation-input]
[coalesce-duplicate-inputs]: New procedures.
Sort OUTPUTS, INPUTS, and ENV-VARS.
* tests/derivations.scm ("read-derivation vs. derivation"): New test.
Suggested by Jookia.
* guix/derivations.scm (&file-search-error): New error condition.
(search-path*): Raise it when 'search-path' returns #f.
* guix/gexp.scm (search-path*): Remove.
* guix/ui.scm (call-with-error-handling): Add case for
'file-search-error?'.
* tests/derivations.scm ("build-expression->derivation and invalid
module name"): New test.
* guix/derivations.scm (substitution-oracle): Add #:mode parameter and
honor it.
(derivation-prerequisites-to-build): Likewise.
[derivation-built?]: Take it into account.
* guix/ui.scm (show-what-to-build): Add #:mode parameter. Pass it to
'substitute-oracle' and 'derivations-prerequisites-to-build'.
* tests/derivations.scm ("derivation-prerequisites-to-build in 'check'
mode"): New test.
Before that, "guix build qt", when only qt itself is missing, would lead
'substitution-oracle' to call 'substitutable-paths' with 318 items.
Now, this is down to 6 items, because it doesn't ask about prerequisites
that are already valid.
* guix/derivations.scm (substitution-oracle)[valid-input?,
dependencies]: New procedures.
Use 'dependencies' and remove call to 'remove'.
* guix/derivations.scm (valid-derivation-input?): New procedure.
(derivation-prerequisites): Add 'cut?' parameter and honor it.
* tests/derivations.scm ("derivation-prerequisites and
derivation-input-is-valid?"): New test.
Fixes <http://bugs.gnu.org/20188>.
Reported by Mark H Weaver <mhw@netris.org>.
* guix/derivations.scm (substitution-oracle): Add 'valid?' procedure.
Remove 'valid?' items from PATHS.
Fixes <http://bugs.gnu.org/19630>.
Reported by Ricardo Wurmus <ricardo.wurmus@mdc-berlin.de>.
* guix/derivations.scm (&derivation-error,
&derivation-missing-output-error): New error conditions.
(derivation->output-path): Raise a '&derivation-missing-output-error'
if OUTPUT is not an output of DRV.
* guix/ui.scm (call-with-error-handling): Add case for
'derivation-missing-output-error?'.
(show-what-to-build): Check whether (derivation-outputs drv) is
empty.
* tests/packages.scm ("reference to non-existent output"): Add test.
This yields a 46% improvement in 'derivation-prerequisites' invocations
on the Emacs derivation.
* guix/derivations.scm (derivation-prerequisites): Add 'input-set'
variable, and use it in iterations.
Ludovic Courtès