Commit Graph

1999 Commits

Author SHA1 Message Date
Ludovic Courtès b879b3e848 substitute: Do not leak file descriptors for TLS connections.
Partially fixes <http://bugs.gnu.org/20145>.

* guix/scripts/substitute.scm (fetch, download-cache-info):
(http-multiple-get, fetch-narinfos, progress-report-port): Use
'close-connection' instead of 'close-port'.
2016-03-17 23:53:53 +01:00
Ludovic Courtès 958fb14cdb substitute: Cache transient HTTP errors for 10mn.
* guix/scripts/substitute.scm (fetch-narinfos)[handle-narinfo-response]:
Cache transient errors for 10mn.
(%narinfo-transient-error-ttl): New variable.
2016-03-17 23:53:53 +01:00
Ludovic Courtès 14d6ca3e4d lint: Do not leak file descriptors for TLS connections.
Partially fixes <http://bugs.gnu.org/20145>.

* guix/scripts/lint.scm (probe-uri): Use 'close-connection' instead of
'close-port'.
2016-03-17 23:53:53 +01:00
Ludovic Courtès 097a951e96 download: Add 'close-connection'.
Partially fixes <http://bugs.gnu.org/20145>.

* guix/build/download.scm (add-weak-reference): Remove.
(%tls-ports): New variable.
(register-tls-record-port): New procedure.
(tls-wrap): Use it instead of 'add-weak-reference'.
(close-connection): New procedure.
2016-03-17 23:53:53 +01:00
Ludovic Courtès fc3ea24bf4 substitute: Update progress for responses different from 200/404.
* guix/scripts/substitute.scm (fetch-narinfos)[handle-narinfo-response]:
Add missing call to 'update-progress!'.
2016-03-17 23:53:53 +01:00
Roel Janssen f88ce395e6 licenses: Add Apache Software License 1.1.
* guix/licenses.scm (asl1.1): New variables.

Signed-off-by: Leo Famulari <leo@famulari.name>
2016-03-17 00:49:30 -04:00
Ludovic Courtès 71e2065a38 substitute: Honor client-provided empty URL list.
Before that, 'guix build --substitute-urls=""' would lead to using the
daemon's own URL list instead of the empty list.  The 'or*' hack, which
is to blame, had become unnecessary since commit
fb4bf72be3.

Reported by Mark H Weaver <mhw@netris.org>.

* guix/scripts/substitute.scm (or*): Remove.
(%cache-urls): Use 'or' instead of 'or*'.
* tests/store.scm ("substitute query, alternating URLs"): Add test with
empty URL list.
* doc/guix.texi (Common Build Options): Mention the empty string.
2016-03-16 18:34:53 +01:00
Ludovic Courtès 23d60ba65c substitute: Honor the 'max-age' of 'Cache-Control' headers.
This allows substitute servers to tell 'guix substitute' how long they
can cache narinfo lookups.

* guix/scripts/substitute.scm (cache-narinfo!): Add 'ttl' parameter.
[cache-entry]: Honor it.
(fetch-narinfos)[handle-narinfo-response]: Check the 'Cache-Control'
header of RESPONSE and pass its 'max-age' value to 'cache-narinfo!'.
2016-03-16 15:57:47 +01:00
Ludovic Courtès 1cf7e31898 substitute: Make room for a 'ttl' field in cached entries.
* guix/scripts/substitute.scm (cached-narinfo): Expect 'narinfo' sexp
version 2 with a 'ttl' field.
(cache-narinfo!)[cache-entry]: Produce 'narinfo' sexp version 2 with a
'ttl' field.
(remove-expired-cached-narinfos)[expired?]: Read 'narinfo' sexp version 2.
2016-03-16 15:57:47 +01:00
Ludovic Courtès df061d079b build: Default to "https://mirror.hydra.gnu.org/" for substitutes.
* config-daemon.ac: Check for (gnutls) and define 'GUIX_SUBSTITUTE_URLS'.
* nix/nix-daemon/guix-daemon.cc (main): Use GUIX_SUBSTITUTE_URLS.
* guix/store.scm (%default-substitute-urls): Use 'https' when (gnutls)
is available.
* doc/guix.texi (Binary Installation): Mention mirrors
(Invoking guix-daemon): Mention mirror.hydra.gnu.org.
(Substitutes): Mention mirrors.
(Invoking guix archive): Show https URLs.
2016-03-16 11:27:37 +01:00
Ludovic Courtès 409e4ac6e3 http-client: No 'setvbuf' for non-file ports.
* guix/http-client.scm (http-fetch): Do not call 'setvbuf' on non-file
ports.
2016-03-16 11:27:37 +01:00
Ludovic Courtès 264fdedb40 grafts: Update the narinfo cache before building a derivation.
* guix/grafts.scm (references-oracle)[references*]: Add call to
'substitution-oracle'.
2016-03-14 23:34:33 +01:00
Ludovic Courtès 026ca50fa4 substitute: Keep the initial connection alive.
The connection used to fetch /nix-cache-info is now reused for the
subsequent narinfo requests.

* guix/scripts/substitute.scm (download-cache-info)[download]: Remove.
[uri, read-cache-info]: New variables.
Rewrite in terms of 'http-fetch' instead of 'fetch'.  Return an open
port in addition to a <cache-info>.
* guix/scripts/substitute.scm (http-multiple-get): Add #:port parameter
and honor it.
(fetch-narinfos)[do-fetch]: Add 'port' parameter.
Adjust to new 'download-cache-info' and 'do-fetch' signatures.
2016-03-14 23:34:33 +01:00
Ludovic Courtès d262a0f36b http-client: Add #:keep-alive? parameter.
* guix/http-client.scm (http-fetch): Add #:keep-alive? parameter and
pass it to 'http-get' or 'http-get*'.
2016-03-14 23:34:33 +01:00
Ludovic Courtès cc27dbcf4a substitute: Remove dead code.
This parameter became unused with the switch to HTTP pipelining in
commit d3a652037e.

* guix/scripts/substitute.scm (fetch): Remove #:quiet-404? and adjust
accordingly.
2016-03-14 23:34:33 +01:00
Ludovic Courtès f09aea1b58 store: 'references/substitutes' caches its results.
* guix/store.scm (%reference-cache): New variable.
(references/substitutes): Use it.
2016-03-14 23:34:33 +01:00
Ludovic Courtès 3667bb6cb0 size: Disable grafts.
* guix/scripts/size.scm (guix-size): Parametrize '%graft?'.
2016-03-14 23:34:33 +01:00
Ricardo Wurmus 5f7a1a4def build: Add Ant build system.
* guix/build-system/ant.scm: New file.
* guix/build/ant-build-system: New file.
* Makefile.am (MODULES): Add new files.
* doc/guix.texi (Build Systems): Document ant-build-system.
2016-03-14 12:36:49 +01:00
Ludovic Courtès 6a25e59514 cve: Read entire CVE databases for the current year and the past year.
The "Modified" database that we were reading is much smaller, but it
only shows CVEs modified over the past week.

* guix/cve.scm (%now, %current-year, %past-year): New variables.
(yearly-feed-uri): New procedure.
(%cve-feed-uri, %ttl): Remove.
(%current-year-ttl, %past-year-ttl): New variables.
(call-with-cve-port): Add 'uri' and 'ttl' parameters and honor them.
Add 'setvbuf' call.
(current-vulnerabilities)[read-vulnerabilities]: New procedure.
Read from both %LAST-YEAR and %CURRENT-YEAR.
2016-03-11 16:33:50 +01:00
Ludovic Courtès cf557afa2e cve: Make CPE patch level part of the version string.
* guix/cve.scm (%cpe-package-rx): Adjust to account for :PATCH-LEVEL.
(cpe->package-name): Likewise.
2016-03-11 16:33:50 +01:00
Ludovic Courtès ec278439f3 substitute: Optimize HTTP pipelining over TLS.
* guix/scripts/substitute.scm (http-multiple-get): Write the requests
to a bytevector output port before sending them.
2016-03-10 13:55:30 +01:00
Ludovic Courtès 9b7bd1b160 substitute: Add HTTPS support.
Fixes <http://bugs.gnu.org/22937>.
Reported by Chris Marusich <cmmarusich@gmail.com>.

* guix/scripts/substitute.scm (fetch): Add 'https' alongside 'http'.
Use 'open-connection-for-uri' instead of 'open-socket-for-uri'.  Call
'setvbuf' only when PORT matches 'file-port?'.
(http-multiple-get): Likewise.  Change 'base-url' parameter to
'base-uri'.
(fetch-narinfos)[do-fetch]: Add 'https' case alongside 'http'.  Pass URI
instead of URL to 'http-multiple-get'.
* doc/guix.texi (Requirements): Move GnuTLS one level higher and mention
HTTPS substitutes.
(Substitutes): Mention HTTPS and recommend it.  Explain why servers
are not authenticated.  Add "On Trusting Binaries" subsection.
2016-03-10 13:55:30 +01:00
Ludovic Courtès 204d34ff96 substitute: Error out on unsupported URL schemes.
Reported in <http://bugs.gnu.org/22937>
by Chris Marusich <cmmarusich@gmail.com>.

* guix/scripts/substitute.scm (fetch): Add 'else' case and call 'leave'.
2016-03-09 23:47:49 +01:00
Ludovic Courtès bec7f35214 ui: Do not call 'port-filename' on closed file ports.
* guix/ui.scm (call-with-error-handling)[port-filename*]: New
procedure.
Use it in the 'nar-error?' case.
2016-03-09 23:47:49 +01:00
Ludovic Courtès f1eacbafc4 upstream: Fix 'signature-urls' coalescing.
Previously, the resulting 'signature-urls' would contain N times the
same URL.

* guix/upstream.scm (coalesce-sources): Fix TWO in 'signature-urls'.
* tests/upstream.scm: New file.
* Makefile.am (SCM_TESTS): Add it.
2016-03-09 15:08:00 +01:00
Ludovic Courtès 5284339d9d guix build: Add '--quiet'.
Fixes <http://bugs.gnu.org/19772>.
Reported by Andrei Osipov <andrspv@gmail.com>.

* guix/scripts/build.scm (show-help, %options): Add --quiet.
(guix-build): Parameterize 'current-build-output-port' accordingly.
* doc/guix.texi (Invoking guix build): Use it in example.
(Additional Build Options): Document it.
2016-03-08 22:00:17 +01:00
Ludovic Courtès 16210486e6 guix system: Write the GC root on the target file system.
Fixes <http://bugs.gnu.org/22802>.
Reported by Jookia <166291@gmail.com>.

* guix/scripts/system.scm (install-grub*): Prepend TARGET to GC-ROOT.
2016-03-08 11:11:28 +01:00
Ludovic Courtès ced71ac7a7 packages: Cache the result of 'input-grafts'.
This reduces the wall-clock time of

  guix environment gnutls --pure -E true

by ~35%.

* guix/packages.scm (%graft-cache): New variable.
(input-graft): Use 'cached' to cache to %GRAFT-CACHE.
2016-03-08 00:01:13 +01:00
Ludovic Courtès 9775412ee0 packages: Cache the result of 'package->bag'.
This reduces the wall-clock time of

  guix environment gnutls --pure -E true

by ~25%.

* guix/packages.scm (%bag-cache): New variable.
(package->bag): Use 'cached' to cache things to %BAG-CACHE.
2016-03-08 00:01:13 +01:00
Ludovic Courtès 198d84b70b packages: Generalize the 'cached' macro.
* guix/packages.scm (cache): Rename to...
  (cache!): ... this.  Add 'cache' parameter, and use it.
  (cached): Add a rule to allow the cache to be specified.
2016-03-08 00:01:13 +01:00
Ludovic Courtès 322bb53c7a import: snix: Use the right 'package-name->name+version'.
Fixes a regression introduced in
1b846da8c3.

* guix/import/snix.scm: Use 'package-name->name+version' from (guix
build utils).
2016-03-06 23:16:35 +01:00
Ludovic Courtès dd78e90a4d store: 'references/substitutes' correctly handles the order of substitutes.
Before that, 'references/substitutes' would assume that
'substitutable-path-info' would return things in the same order as its
arguments, which is not the case.  Thus, it would sometimes provide
incorrect reference information, occasionally leading to infinite
loop (because dependency information would denote cycles.)

Fixes <http://bugs.gnu.org/22914>.
Reported by Eric Bavier <ericbavier@openmailbox.org>.

* guix/store.scm (references/substitutes): Make ITEMS the first argument
of the loop; match on it.  Use 'any' to find a matching substitute.
(substitutable-path-info): Clarify docstring about ordering.
2016-03-05 22:53:40 +01:00
Ludovic Courtès d4da602e4c grafts: Memoize intermediate results in 'cumulative-grafts'.
The time for:

  guix build inkscape -n --no-substitutes

goes down by 30% (in the presence of 3 replacements among all the
packages.)

* guix/grafts.scm (cumulative-grafts): Turn into a monadic procedure in
%STATE-MONAD.  Use the current state as a derivation-to-graft cache.
(graft-derivation): Call 'cumulative-grafts' within 'run-with-state'.
2016-03-05 00:19:10 +01:00
Ludovic Courtès fcadd9ff9d packages: The result of 'bag-grafts' does not contain duplicates.
* guix/packages.scm (bag-grafts): Add call to 'delete-duplicates'.
2016-03-05 00:19:10 +01:00
Ludovic Courtès c90cb5c9d8 grafts: Use dependency information from substitutes when possible.
This avoids starting derivation builds just for the sake of knowing the
references of their outputs, thereby restoring the expected behavior of
--dry-run when substitutes are available.

* guix/grafts.scm (non-self-references): Remove 'store' parameter, and
add 'references'.  Use it.  Update caller.
(references-oracle): New variable.
(cumulative-grafts): Add 'references' parameter and use it.  Update
callers.
(graft-derivation): Remove 'build-derivations' call.  Add call to
'references-oracle'.
2016-03-05 00:19:10 +01:00
Ludovic Courtès 6581ec9ab9 store: Add 'references/substitutes'.
* guix/store.scm (references/substitutes): New procedure.
* tests/store.scm ("references/substitutes missing reference info")
("references/substitutes with substitute info"): New tests.
2016-03-05 00:19:10 +01:00
Ludovic Courtès 7bfeb9df20 tests: Narinfos can specify an non-empty reference list.
* guix/tests.scm (derivation-narinfo): Add #:references and honor it.
(call-with-derivation-narinfo, call-with-derivation-substitute):
Likewise.
(with-derivation-narinfo, with-derivation-substitute): Add 'references'
keyword.
2016-03-05 00:19:10 +01:00
Ludovic Courtès c8f9f24776 guix build: Set the build options early.
This fixes a bug whereby, with grafts leading to builds very early,
build options such as --substitute-urls would not be taken into account
yet.

Reported by Andreas Enge <andreas@enge.fr>.

* guix/scripts/build.scm (guix-build): Move 'opts' to the beginning.
Use 'with-store' instead of 'open-connection'.  Call
'set-build-options-from-command-line' right after 'with-store'.
2016-03-05 00:19:10 +01:00
Ludovic Courtès 4ce783a2f9 lint: cve: Gracefully handle HTTP errors.
* guix/scripts/lint.scm (current-vulnerabilities*): New procedure.
(package-vulnerabilities): Use it.
2016-03-04 11:48:52 +01:00
Ludovic Courtès 937690f909 lint: derivation: Disable grafts, but check replacements.
* guix/scripts/lint.scm (check-derivation): Pass #:graft? #f.  When
'package-replacement' exists, compute its derivation.
2016-03-03 09:46:17 +01:00
Ludovic Courtès 94d609aba8 guix build: -S returns the replacement's source.
Reported by Mark H Weaver.

* guix/scripts/build.scm (options->derivations): When SRC and GRAFT? are
true, use the source of P's replacement.
* tests/guix-build.sh: Add test.
2016-03-03 00:18:30 +01:00
Mathieu Lirzin 1b846da8c3 utils: Use '@' for separating package names and version numbers.
This provides the ability to use numbers in package names.

Fixes <http://bugs.gnu.org/19219>.

* guix/utils.scm (package-name->name+version): New procedure.
* gnu/packages.scm (%find-package): Add a FALLBACK? keyword argument.
Use the previous method when no package is found.
(specification->package+output, specification->package): Adapt
documentation to new syntax.
* doc/guix.texi (Invoking guix package, Invoking guix import): Likewise.
* guix/ui.scm (package-specification->name+version+output): Likewise.
* guix/scripts/import/hackage.scm (show-help): Likewise.
* tests/guix-build.sh: Adapt to new syntax.
* tests/guix-lint.sh: Likewise.
* tests/guix-package.sh: Likewise.
* tests/ui.scm ("package-specification->name+version+output"): Likewise.
* tests/utils.scm ("package-name->name+version"): Likewise.
* NEWS: Mention new syntax.
2016-03-02 21:41:41 +01:00
Ludovic Courtès 637cd12543 graph: Ignore 'GUIX_BUILD_OPTIONS'.
Previously 'GUIX_BUILD_OPTIONS' would be honored, but 'guix graph' does
not support the common build options.

* guix/scripts/graph.scm (guix-graph): Use 'args-fold*' instead of
'parse-command-line'.
2016-03-02 14:47:07 +01:00
Ludovic Courtès 7573d30ff8 guix build: Move '--no-grafts' to the common build options.
* guix/scripts/build.scm (%options): Move --no-grafts to...
(%standard-build-options): ... here.
(show-help, show-build-options-help): Adjust accordingly.
* guix/scripts/archive.scm (%default-options): Add 'graft?'.
(guix-archive): Parametrize '%graft?'.
* guix/scripts/environment.scm (%default-options): Add 'graft?'.
(guix-environment): Parametrize '%graft?'.
* guix/scripts/package.scm (%default-options): Add 'graft?'.
(guix-package): Parametrize '%graft?'.
* guix/scripts/system.scm (%default-options): Add 'graft?'.
(guix-system): Parametrize 'graft?'.
* doc/guix.texi (Additional Build Options): Move --no-grafts to...
(Common Build Options): ... here.
2016-03-02 14:47:07 +01:00
Ludovic Courtès 1cbdf82d3b guix archive: Use 'with-store'.
* guix/scripts/archive.scm (guix-archive): Use 'with-store' instead of
an explicit 'open-connection'.
2016-03-02 14:47:07 +01:00
Ludovic Courtès c22a1324e6 grafts: Graft recursively.
Fixes <http://bugs.gnu.org/22139>.

* guix/grafts.scm (graft-derivation): Rename to...
(graft-derivation/shallow): ... this.
(graft-origin-file-name, item->deriver, non-self-references)
(cumulative-grafts, graft-derivation): New procedures
* tests/grafts.scm ("graft-derivation, grafted item is a direct
dependency"): Clarify title.  Use 'grafted' instead of 'graft' to refer
to the grafted derivation.
("graft-derivation, grafted item is an indirect dependency")
("graft-derivation, no dependencies on grafted output"): New tests.
* guix/packages.scm (input-graft): Change to take a package instead of
an input.
(input-cross-graft): Likewise.
(fold-bag-dependencies): New procedure.
(bag-grafts): Rewrite in terms of 'fold-bag-dependencies'.
* tests/packages.scm ("package-derivation, indirect grafts"): Comment out.
* doc/guix.texi (Security Updates): Mention run-time dependencies and
recursive grafting.
2016-03-01 16:00:46 +01:00
Ludovic Courtès f376dc3acb grafts: Consider all the outputs in the graft mapping.
Before that, outputs of a derivation could be left referring to the
ungrafted version of the derivation.

* guix/grafts.scm (graft-derivation)[outputs]: Change to a list of
name/file pairs.
* guix/grafts.scm (graft-derivation)[build]: Add 'old-outputs' variable
and use it when computing 'mapping'.  Use 'mapping' directly.
* tests/grafts.scm ("graft-derivation, multiple outputs"): New test.
2016-02-27 23:31:52 +01:00
Ludovic Courtès cd05d38812 grafts: Slight simplification.
* guix/grafts.scm (graft-derivation)[output-names]: Use
'derivation-output-names'.
2016-02-27 23:31:52 +01:00
Ludovic Courtès 82f5186650 grafts: Make sure files are not created world-writable.
* guix/build/graft.scm (rewrite-directory): Add 'umask' call.
2016-02-27 23:31:52 +01:00
Ben Woodcroft 917a2a58ec import: Add github-updater.
* guix/import/github.scm: New file.
* guix/scripts/refresh.scm (%updaters): Add %GITHUB-UPDATER.
* doc/guix.texi (Invoking guix refresh): Mention it.
* Makefile.am (MODULES): Add gnu/import/github.scm.
2016-02-26 22:06:34 -05:00