Commit Graph

13 Commits

Author SHA1 Message Date
Ludovic Courtès 101d9f3fd4 substitute-binary: Pass `filtered-port' an unbuffered port.
This fixes a bug whereby `read-response' would read more than just the
response, with the extra data going into the port's buffer; the
"bzip2 -dc" process spawned by `filtered-port' would not see the those
buffered data, which are definitely lost, and would bail out with
"bzip2: (stdin) is not a bzip2 file."

* guix/utils.scm (filtered-port): Document that INPUT must be
  unbuffered.
* guix/web.scm (http-fetch): Add `buffered?' parameter.  Call
  `open-socket-for-uri' explicitly, and call `setvbuf' when BUFFERED? is
  false.  Pass the port to `http-get'.  Close it upon 301/302.
* guix/scripts/substitute-binary.scm (fetch): Add `buffered?'
  parameter.  Pass it to `http-fetch'; honor it for `file' URIs.
  (guix-substitute-binary): Call `fetch' with #:buffered? #f for port RAW.
* tests/utils.scm ("filtered-port, file"): Open FILE as unbuffered.
2013-05-15 23:40:09 +02:00
Ludovic Courtès 0332386251 substitute-binary: Work around thread-unsafe `regexp-exec'.
* guix/scripts/substitute-binary.scm (%regexp-exec-mutex): New variable.
  (string->uri): New procedure.
  (fields->alist): Wrap `regexp-exec' call in `with-mutex'.
2013-05-14 23:53:38 +02:00
Ludovic Courtès c0cd1b3ea7 Move record utilities to (guix records).
* guix/utils.scm (define-record-type*): Move to...
* guix/records.scm: ... here.  New file.
* guix/build-system.scm, guix/packages.scm: Use it.
* guix/gnu-maintenance.scm: Likewise.
  (official-gnu-packages)[alist->record]: Remove.
* guix/scripts/substitute-binary.scm: Likewise.
  (alist->record, object->fields): Remove.
* tests/utils.scm ("define-record-type*", "define-record-type* with
  letrec* behavior", "define-record-type* & inherit",
  "define-record-type* & inherit & letrec* behavior",
  "define-record-type* & thunked", "define-record-type* & thunked &
  default", "define-record-type* & thunked & inherited"): Move to...
* tests/records.scm: ... here.  New file.
2013-05-12 15:46:16 +02:00
Ludovic Courtès e0fbbc889d substitute-binary: Support decompression from non-file ports.
* guix/scripts/substitute-binary.scm (filtered-port): Move to utils.scm.
  (decompressed-port): Upon "none", return '() as the second value.
  (guix-substitute-binary): Expect `decompressed-port' to return a list
  of PIDs as its second value.
* guix/utils.scm (filtered-port): New procedure.  Add case for when
  INPUT is not `file-port?'.
* tests/utils.scm ("filtered-port, file", "filtered-port, non-file"):
  New tests.
2013-04-29 23:25:19 +02:00
Ludovic Courtès 1c9e7d65d4 web: Factorize `http-get' hackery.
This should fix `substitute-binary --query' on Guile 2.0.5.

* guix/web.scm: New file.
* Makefile.am (MODULES): Add it.
* po/POTFILES.in: Add it.
* guix/gnu-maintenance.scm (http-fetch): Remove.
  (%package-list-url): Turn into a URI.
  (official-gnu-packages): Add #:text? #t to `http-fetch' call.
* guix/scripts/substitute-binary.scm (fetch): Remove `http' case, and
  use `http-fetch' instead.
2013-04-25 22:14:51 +02:00
Ludovic Courtès 4c7cacf117 substitute-binary: Remove expired cache entries once in a while.
* guix/scripts/substitute-binary.scm (%narinfo-expired-cache-entry-removal-delay):
  New variable.
  (obsolete?): New procedure, formerly in `lookup-narinfo'.
  (lookup-narinfo): Adjust accordingly.
  (remove-expired-cached-narinfos, maybe-remove-expired-cached-narinfo):
  New procedures.
  (guix-substitute-binary): Call `maybe-remove-expired-cached-narinfo'.
2013-04-20 15:12:24 +02:00
Ludovic Courtès e967678ed1 substitute-binary: Skip servers that use a different store prefix.
* guix/scripts/substitute-binary.scm (fetch-narinfo): Return #f when
  CACHE uses a store directory different from (%store-prefix).
2013-04-15 23:43:13 +02:00
Ludovic Courtès 0c357a088b substitute-binary: Call `open-cache' only when needed.
* guix/scripts/substitute-binary.scm (lookup-narinfo): Force CACHE when
  passing it to `fetch-narinfo'.
  (guix-substitute-binary): Delay calls to `open-cache'.
2013-04-15 23:43:10 +02:00
Ludovic Courtès eba783b7b2 substitute-binary: Add a local cache.
* guix/scripts/substitute-binary.scm (%narinfo-cache-directory,
  %narinfo-ttl, %narinfo-negative-ttl): New variables.
  (with-atomic-file-output, object->fields, read-narinfo,
  write-narinfo, narinfo->string, string->narinfo, lookup-narinfo): New
  procedures.
  (fetch-narinfo): Adjust to use `read-narinfo'.
  (guix-substitute-binary): Ensure the existence of
  %NARINFO-CACHE-DIRECTORY.  Use `lookup-narinfo' instead of
  `fetch-narinfo'.
2013-04-15 23:42:55 +02:00
Ludovic Courtès fe0cff14f6 substitute-binary: Implement `--substitute'.
This allows build outputs to be transparently downloaded from
http://hydra.gnu.org, for example.

* config-daemon.ac: Check for `gzip', `bzip2', and `xz'.
* guix/config.scm.in (%gzip, %bzip2, %xz): New variable.
* guix/scripts/substitute-binary.scm (fetch): Return SIZE as a second value.
  (<narinfo>): Change `url' to `uri'.
  (make-narinfo): Rename to...
  (narinfo-maker): ... this.  Handle relative URLs.
  (fetch-narinfo): Adjust accordingly.
  (filtered-port, decompressed-port): New procedures.
  (guix-substitute-binary): Implement the `--substitute' case.
* tests/store.scm ("substitute query"): Use (%store-prefix) instead
  of (getenv "NIX_STORE_DIR").
  ("substitute"): New test.
2013-04-12 17:31:01 +02:00
Ludovic Courtès d7c5d27795 substitute-binary: Correctly handle missing narinfos in `--query' mode.
* guix/scripts/substitute-binary.scm (guix-substitute-binary)["--query"]("have",
  "info"): Filter SUBSTITUTABLE through `narinfo?'.
2013-04-12 17:30:47 +02:00
Ludovic Courtès 462f8e9f33 substitute-binary: Fix communication of several store paths to the daemon.
* guix/scripts/substitute-binary.scm (guix-substitute-binary)["--query"]:
  Emit blank lines only after the complete list of store paths has been
  returned.
2013-04-04 21:47:55 +02:00
Ludovic Courtès f65cf81a3c Add preliminary binary substituter.
* guix/scripts/substitute-binary.scm: New file.
* Makefile.am (MODULES): Add it.
* nix/scripts/substitute-binary.in: New file.
* config-daemon.ac: Produce nix/scripts/substitute-binary.
* daemon.am (nodist_pkglibexec_SCRIPTS): Add
  nix/scripts/substitute-binary.
* guix/store.scm (substitutable-path-info): Use the
  `query-substitutable-path-infos' RPC.
* nix/nix-daemon/guix-daemon.cc (main): Honor `NIX_SUBSTITUTERS'.
* pre-inst-env.in: Set `NIX_SUBSTITUTERS'.
* test-env.in: Leave `NIX_SUBSTITUTERS' unchanged.  Set
  `GUIX_BINARY_SUBSTITUTE_URL, and create
  $NIX_STATE_DIR/substituter-data.
  Run `guix-daemon' within `./pre-inst-env'.
* tests/store.scm ("substitute query"): New test.
2013-04-03 22:44:39 +02:00