Commit Graph

68 Commits

Author SHA1 Message Date
Ludovic Courtès 27fd13c3c2
download: Work around GnuTLS bug with UTF-8 certificate file names.
Reported by Mark H Weaver <mhw@netris.org>
at <https://debbugs.gnu.org/cgi/bugreport.cgi?bug=26948#17>.

* guix/build/download.scm (set-certificate-credentials-x509-trust-file!*):
New procedure.
(make-credendials-with-ca-trust-files): Use it instead of
'set-certificate-credentials-x509-trust-file!'.
2017-05-30 13:24:55 +02:00
Ludovic Courtès 7b9ac883ea
download: Continue handshake upon TLS warning alerts.
This allows us to download from site such as
<https://fusionforge.int-evry.fr> where the server does not recognize
the server name passed via the 'server_name' extension.

* guix/build/download.scm (tls-wrap): Catch 'gnutls-error' around
'handshake'.  Upon ERROR/WARNING-ALERT-RECEIVED, print a message and
call 'handshake'.
2017-05-02 23:41:13 +02:00
Ludovic Courtès 25a49294ca
cache: Work around 'time-monotonic' bug in Guile 2.2.2.
* guix/cache.scm (time-monotonic) [guile-2.2]: New variable.
* tests/cache.scm (time-monotonic) [guile-2.2]: Likewise.
* guix/build/download.scm (time-monotonic) [guile-2.2]: Adjust comment:
it's a 2.2.2 bug.
2017-04-22 14:42:15 +02:00
Ludovic Courtès 1fe958d62a
download: Work around Guile 2.2 bug with 'time-monotonic' objects.
* guix/build/download.scm (time-monotonic) [guile-2.2]: New variable.
2017-04-22 01:25:22 +02:00
Ludovic Courtès 6dfd683dc7
download: Properly parse ftp:// URLs that lack user info.
Fixes a regression introduced in
a4f5423415.

Reported by Hartmut Goebel.

* guix/build/download.scm (ftp-fetch): Account for the case where
'uri-userinfo' returns #f.  Remove the case where it returns "" since
that cannot happen.
2017-03-28 13:30:09 +02:00
Roel Janssen a4f5423415
download: Handle username and password properties for FTP URIs.
* guix/build/download.scm (ftp-fetch): Process username and password from a URI.
2017-03-21 12:15:14 +01:00
Ludovic Courtès 4fd06a4dd1
http-client: Avoid name clash with 'open-connection-for-uri' in 2.2.0.
* guix/build/download.scm (open-connection-for-uri): Add note about
same-named binding in Guile 2.2.0.
* guix/http-client.scm: Use 'guix:open-connection-for-uri' for the
procedure coming from (guix build download).
* guix/scripts/lint.scm: Likewise.
* guix/scripts/substitute.scm: Likewise.
2017-03-18 00:39:50 +01:00
Ludovic Courtès 36626c556e
build: Require Guile >= 2.0.9.
* configure.ac: Bump requirement to 2.0.9.
* doc/guix.texi (Requirements): Adjust accordingly.
* README (Requirements): Likewise.
* build-aux/download.scm: Remove workaround for <http://bugs.gnu.org/13095>.
* guix/build/download.scm: Likewise.
(http-fetch)[post-2.0.7?]: Remove.  Remove conditional code for not
POST-2.0.7?.
* guix/http-client.scm:  Remove workaround for <http://bugs.gnu.org/13095>.
(http-fetch)[post-2.0.7?]: Remove.  Remove conditional code for not
POST-2.0.7?.
* guix/serialization.scm (read-latin1-string): Remove mention of 2.0.9.
* tests/nar.scm: Use (ice-9 control).
(let/ec): Remove.
2017-03-18 00:39:49 +01:00
Ludovic Courtès b111bceecc
download: Export '%x509-certificate-directory'.
* guix/build/download.scm (%x509-certificate-directory): Export, as
expected by (guix scripts pull) since commit
7e81d699de.
2017-03-13 21:52:38 +01:00
Ludovic Courtès 580deec5b4
download: Protect against dangling symlinks in $SSL_CERT_DIR.
Reported by Christopher Baines <mail@cbaines.net>
in <https://bugs.gnu.org/25213>.

* guix/build/download.scm (make-credendials-with-ca-trust-files): Check
whether FILE exists before calling
'set-certificate-credentials-x509-trust-file!'.
2016-12-16 18:00:01 +01:00
Ludovic Courtès bc3c41ce36
download: Verify TLS certificates unless asked not to.
Fixes <http://bugs.gnu.org/24466>.
Reported by Leo Famulari <leo@famulari.name>.

* guix/build/download.scm (%x509-certificate-directory): New variable.
(make-credendials-with-ca-trust-files, peer-certificate)
(assert-valid-server-certificate, print-tls-certificate-error): New
procedures.  Add 'print-tls-certificate-error' as an exception printer
for 'tls-certificate-error'.
(tls-wrap): Add #:verify-certificate? parameter and honor it.
(open-connection-for-uri): Likewise.
(http-fetch): Likewise.
(url-fetch): Likewise.
* guix/download.scm (url-fetch)[builder]: Pass #:verify-certificate? #f.
* guix/scripts/lint.scm (probe-uri): Add case for 'tls-certificate-error'.
(validate-uri): Likewise.
* doc/guix.texi (Invoking guix download): Mention 'SSL_CERT_DIR'.
2016-11-07 23:39:01 +01:00
Ludovic Courtès 38f1cf8a8e
download: Pass the raw file name to content-addressed mirrors.
* guix/build/download.scm (url-fetch)[content-addressed-uris]: Call
'strip-store-file-name' on FILE before passing it to 'make-url'.
2016-07-31 18:27:14 +02:00
Ludovic Courtès ab84b927ef
download: Prepare to support the 'guix publish' /file URLs.
* guix/download.scm (%content-addressed-mirrors): Add 'file' parameter
to the lambda.
* guix/build/download.scm (url-fetch)[content-addressed-uris]: Adjust
accordingly.
2016-07-20 17:02:35 +02:00
David Thompson 242ad41c01
download: Use basic authentication when userinfo is present in URI.
* guix/download.scm (url-fetch): Include (guix base64) module on the
  build-side.
* guix/build/download.scm (http-fetch): Add "Authorization" header when
  userinfo is present in the URI.
2016-06-29 08:51:41 -04:00
Ludovic Courtès b18ede2704
download: Default to a 10s connection establishment timeout.
* guix/build/download.scm (ftp-fetch): Add #:timeout and pass it to
'ftp-open'.
(http-fetch): Add #:timeout and pass it to 'open-connection-for-uri' and
in recursive calls.
(url-fetch): Add #:timeout and pass it to 'http-fetch' and 'ftp-fetch'.
2016-05-27 10:37:11 +02:00
Ludovic Courtès dab2472c6a
download: Use URI objects for content-addressed mirrors.
This fixes a bug whereby 'http-fetch' would be passed a string instead
of a URI object.

* guix/build/download.scm (url-fetch): Rename 'content-addressed-urls'
to 'content-addressed-uris', and call 'string->uri'.
2016-05-27 10:37:11 +02:00
Ricardo Wurmus 8bfd602bb0 build: Accept dates with space-padded hour field.
* guix/build/download.scm: Replace "parse-rfc-822-date" from the (web
  http) module.
2016-05-16 18:26:26 +02:00
Ludovic Courtès cd436bf05a
download: Support content-addressed mirrors.
* guix/download.scm (%content-addressed-mirrors)
(%content-addressed-mirror-file): New variables.
* guix/download.scm (url-fetch)[builder]: Define
'value-from-environment.  Pass #:hashes and
 #:content-addressed-mirrors to 'url-fetch'.
Define "guix download hashes" environment variable.
* guix/build/download.scm (url-fetch): Add #:content-addressed-mirrors
and #:hashes.
[content-addressed-urls]: New variable.
Use it.
2016-05-14 17:37:47 +02:00
Ludovic Courtès cf5e58297d substitute: Better abbreviate substitute URL in progress report.
Suggested by Danny Milosavljevic <dannym@scratchpost.org>.

* guix/build/download.scm (nar-uri-abbreviation): New procedure.
* guix/scripts/substitute.scm (process-substitution): Use it instead of
'store-path-abbreviation'.
2016-04-20 23:01:41 +02:00
Ludovic Courtès 3e31ec827a download: 'uri-abbreviation' can abbreviate the URI's basename.
* guix/build/download.scm (uri-abbreviation): Use 'ellipsis' instead of
"...".  Abbreviate the basename of PATH if needed.
2016-04-20 22:52:35 +02:00
Ludovic Courtès 967ee481e8 download: Add "%COMPAT" to the priority string.
Fixes <http://bugs.gnu.org/23311>.

* guix/build/download.scm (tls-wrap): Add 'set-session-priorities!' call.
2016-04-20 13:17:52 +02:00
Ludovic Courtès 4d276c6403 download: Add 'current-terminal-columns' parameter.
* guix/build/download.scm (current-terminal-columns): New variable.
(progress-proc): Use it instead of the hard-coded "80".
2016-04-15 00:32:18 +02:00
Ludovic Courtès 8a2154fefa download: Send an ANSI erase-in-line sequence in addition to CR.
Partly fixes <http://bugs.gnu.org/22536>.
Reported by Danny Milosavljevic <dannym@scratchpost.org>.

* guix/build/download.scm (progress-proc): Send an ANSI erase-in-line
sequence.
2016-04-15 00:32:18 +02:00
Alex Kost 82fd23b81f download: Follow HTTP 307 "Temporary Redirection".
Fixes <http://bugs.gnu.org/23275>.
Reported by Albin Söderqvist <albin@fripost.org>.

* guix/build/download.scm (http-fetch): Follow redirections upon 307.
  This is what 'binaries.openttd.org' does.
2016-04-14 10:34:37 +03:00
Ludovic Courtès 097a951e96 download: Add 'close-connection'.
Partially fixes <http://bugs.gnu.org/20145>.

* guix/build/download.scm (add-weak-reference): Remove.
(%tls-ports): New variable.
(register-tls-record-port): New procedure.
(tls-wrap): Use it instead of 'add-weak-reference'.
(close-connection): New procedure.
2016-03-17 23:53:53 +01:00
Ludovic Courtès 1b9aefa394 download: Always use AI_ADDRCONFIG when resolving host names.
* guix/build/download.scm (open-socket-for-uri): Always pass
  AI_ADDRCONFIG to 'getaddrinfo' as recommended in the fine Guile
  manual.
* guix/ftp-client.scm (ftp-open): Ditto.
2015-11-12 23:47:01 +01:00
Ludovic Courtès 60fd51222f download: Add timeout parameter for connections.
* guix/build/download.scm (ensure-uri): New procedure.
(current-http-proxy): New variable.
(open-socket-for-uri): Copy from Guile commit aaea5b2, but add #:timeout
parameter and use 'connect*' instead of 'connect'.
(open-connection-for-uri): Add #:timeout parameter and pass it to
'open-socket-for-uri'.
2015-11-12 23:47:00 +01:00
Ludovic Courtès fde1783087 download: Check whether HORIZONTAL ELLIPSIS can be encoded.
* guix/build/download.scm (ellipsis): New procedure.
  (store-path-abbreviation): Use it.
2015-10-01 11:28:58 +02:00
Ludovic Courtès 75726135ce download: Don't abbreviate things that are not store items.
Fixes a regression introduced in a8be7b9a.

* guix/build/download.scm (store-path-abbreviation): Return STORE-PATH
  if it's not an actual store path.  Fixes an out-of-range exception
  when running tests/substitute.scm and tests/store.scm.
2015-09-24 21:54:37 +02:00
Steve Sprang 9462882889 download: Fix some minor progress-logging regressions.
* guix/build/download.scm
  (string-pad-middle): Allow resulting padded string to overflow.
  (store-url-abbreviation): Remove unnecessary procedure.
  (progress-proc): Use BASENAME as default for parameter 'abbreviation'.
  (url-fetch): Display extra newlines for readability.
2015-09-23 22:04:46 -04:00
Steve Sprang 47770296d2 download: Only show hours in the elapsed time if necessary.
* guix/build/download.scm
  (seconds->string): Conditionally include hours in timestamp.

Signed-off-by: Ludovic Courtès <ludo@gnu.org>
2015-09-17 14:40:46 +02:00
Steve Sprang a8be7b9a7a substitute: Improve readability of download progress report.
* guix/build/download.scm
  (string-pad-middle, store-url-abbreviation, store-path-abbreviation):
  New procedures.
  (progress-proc): Add #:abbreviation parameter and use it.  Generate a
  better indeterminate progress string.
* guix/scripts/substitute.scm (assert-valid-narinfo): Add newlines to output.
  (process-substitution): Use byte-count->string and store-path-abbreviation.

Co-authored-by: Ludovic Courtès <ludo@gnu.org>
2015-09-16 23:50:12 +02:00
Steve Sprang eb95ace9f1 download: Avoid type errors when formatting download progress output.
* guix/build/download.scm (nearest-exact-integer): New procedure.
  (seconds->string, byte-count->string): Use it.
2015-09-14 23:36:26 -04:00
Steve Sprang 0c0a1f22ce build: Improve information density and appearance of download progress output.
* guix/build/download.scm (seconds->string): New function.
  (byte-count->string): New function.
  (progress-bar): New function.
  (throughput->string): Remove function.
  (progress-proc): Display base file name, elapsed time, and progress bar.

Signed-off-by: Ludovic Courtès <ludo@gnu.org>
2015-09-08 21:47:30 +02:00
Ludovic Courtès 4f7b564adb download: Remove spurious warning about 'https_proxy'.
* guix/build/download.scm (open-connection-for-uri)[with-https-proxy]: Warn
  about 'https_proxy' only when 'getenv' returns a non-empty string.
2015-07-17 22:31:12 +02:00
Ludovic Courtès 7623848343 download: Work around Guile small-receive-buffer bug.
Previously, code using directly (guix build download) was still affected
by <http://bugs.gnu.org/15368>.  This includes source derivations, the
'guix download' command, and (guix gnu-maintenance).

'guix substitute' was unaffected since it used (guix http-client), which
already had the fix.

* guix/http-client.scm (open-socket-for-uri): Remove.
  (http-fetch): Remove #:buffered? argument to 'open-socket-for-uri';
  use 'setvbuf' instead.
* guix/scripts/substitute.scm (fetch): Likewise.
* guix/build/download.scm (open-socket-for-uri): New procedure, taken
  from guix/http-client.scm, but without the #:buffered? parameter.
2015-05-06 10:31:11 +02:00
Ludovic Courtès c822fb8e34 download: Reinstate buffering on connection sockets.
* guix/build/download.scm (open-connection-for-uri): Reinstate call to
  'setvbuf' inadvertently removed in d17551d9.
2015-05-06 09:50:15 +02:00
Ludovic Courtès d17551d943 download: Simplify 'open-connection-for-uri' to support HTTP proxies.
Partly fixes <http://bugs.gnu.org/20402>.
Reported by Joshua Randall <jcrandall@alum.mit.edu>.

* guix/build/download.scm (open-connection-for-uri): Rewrite to be a
  small wrapper around 'open-socket-for-uri'.  This procedure was
  initially introduced in d14ecda to work around the lack of NSS modules
  during bootstrap but that has become unnecessary since 0621349, which
  introduced a bootstrap Guile that uses static NSS modules (from commit
  d3b5972.)
  On Guile >= 2.0.10, this allows the 'http_proxy' environment variable
  to be used.
2015-05-01 00:06:17 +02:00
Ludovic Courtès c9727aac40 download: Comment on lack of progress report with chunked encoding.
* guix/build/download.scm (progress-proc): Add comment.
2015-02-27 15:08:37 +01:00
Ludovic Courtès 9fbe6f1920 download: Measure and display the throughput.
* guix/build/download.scm (duration->seconds, throughput->string): New
  procedures.
  (progress-proc): Measure and display the throughput.
2015-02-27 15:08:37 +01:00
Ludovic Courtès e7620dc995 download: Abstract the receive buffer size.
* guix/build/download.scm (%http-receive-buffer-size): New variable.
  (progress-proc, tls-wrap, http-fetch): Use it.
2015-02-27 15:08:37 +01:00
Mark H Weaver c964a15d82 download: Cope with Guile 2.0.6 or earlier.
* guix/build/download.scm: Do not attempt to support relative URIs in
  "Location" headers if 'declare-relative-uri-header!' is not present.
  This is the case for Guile 2.0.6 or earlier.
2015-02-24 20:01:55 -05:00
Mark H Weaver 04dec194d8 download: Handle HTTP redirects to relative URI references.
Fixes <http://bugs.gnu.org/19840>.
Reported by Ricardo Wurmus <rekado@elephly.net>.

* guix/build/download.scm: On Guile 2.0.11 or earlier, redefine the http
  "Location" header to accept relative URIs.
  (resolve-uri-reference): New exported procedure.
  (http-fetch): Use 'resolve-uri-reference' to resolve redirections.
* guix/http-client.scm (http-fetch): Use 'resolve-uri-reference'
2015-02-19 03:32:22 -05:00
Ludovic Courtès dd8ea244f4 download: Export 'maybe-expand-mirrors'.
* guix/build/download.scm (uri-vicinity, maybe-expand-mirrors): New
  procedures.
  (url-fetch): Remove them from here.
2014-12-29 21:19:17 +01:00
Ludovic Courtès a3bf096945 lint: Add 'home-page' checker.
* guix/build/download.scm (open-connection-for-uri): Export.
* guix/scripts/lint.scm (probe-uri, check-home-page): New procedures.
  (%checkers): Add 'home-page' checker.
2014-12-28 17:57:36 +01:00
Ludovic Courtès 38bf090e7c download: Add "Accept: */*" to the headers.
Fixes downloads from https://alioth.debian.org.
Reported by John Darrington <jmd@gnu.org>.

* guix/build/download.scm (http-fetch)[headers]: Add 'Accept'.
2014-10-19 12:07:53 +02:00
Ludovic Courtès 077bd18d22 download: Use the 'SERVER NAME' TLS extension when possible.
Fixes <http://bugs.gnu.org/18526>.
Reported by Mark H. Weaver.

* guix/build/download.scm (tls-wrap): Add 'server' parameter.  Call
  'set-session-server-name!' when (gnutls) defines it.
  (open-connection-for-uri): Adjust 'tls-wrap' call accordingly.
2014-09-22 21:06:39 +02:00
Ludovic Courtès a68d976b66 download: Enlarge your receive buffer.
* guix/build/download.scm (open-connection-for-uri): Remove call to
  'setsockopt'.
* guix/http-client.scm (open-socket-for-uri)[rmem-max, buffer-size]: New
  variables.  Add call to 'setsockopt'.
2014-05-23 22:19:37 +02:00
Ludovic Courtès 395bea2a53 download: Improve progress report output.
* guix/build/download.scm (url-fetch): Make current-output-port
  unbuffered.
2014-04-16 12:25:52 +02:00
Ludovic Courtès 212ece42b0 download: Don't use 'http-get*' on Guile 2.0.10+.
* guix/build/download.scm (http-fetch)[post-2.0.7?]: Use
  'string->number' and numeric comparison.  This fixes version
  comparison with "2.0.10" and subsequent 2.0 releases.
2014-03-25 18:54:52 +01:00