This is another instance of <https://bugs.gnu.org/22966>.
The Microsoft-IIS/7.5 server at static.nvd.nist.gov would sometimes hang
when receiving our requests byte by byte.
* guix/build/download.scm (tls-wrap) [!guile-2.0]: Add 'setvbuf' call.
This ensures the timeout applies when connecting to the port returned by
PASV.
* guix/ftp-client.scm (ftp-list): Add #:timeout parameter. Use
'connect*' instead of 'connect' and pass TIMEOUT.
(ftp-retr): Likewise.
* guix/build/download.scm (ftp-fetch): Pass TIMEOUT to 'ftp-retr'.
Among other things, this removes (guix utils), (guix ui), (guix config),
etc. from the closure of (guix build download), as was the case since
798648515b.
* guix/utils.scm (<progress-reporter>, call-with-progress-reporter):
Move to...
* guix/progress.scm: ... here. New file.
* Makefile.am (MODULES): Add it.
* guix/build/download.scm (current-terminal-columns)
(nearest-exact-integer, duration->seconds, seconds->string)
(byte-count->string, progress-bar, string-pad-middle)
(rate-limited, progress-reporter/file, dump-port*)
(time-monotonic): Move to progress.scm.
* guix/scripts/download.scm: Adjust accordingly.
* guix/scripts/substitute.scm: Likewise.
* guix/build/download.scm (http-fetch): Remove 'file' parameter. Change
to return an input port and the content-length. Make public.
(url-fetch): Adjust accordingly.
This is a followup to 36626c556e.
* guix/build/download.scm (http-fetch): Rename 'port-or-bv' to 'port'.
Assume (port? port) is always true, and remove other branch.
* guix/utils.scm (<progress-reporter>): New record type.
(call-with-progress-reporter): New procedure.
* guix/build/download.scm (dump-port*, rate-limited, progress-reporter/file):
New procedures.
(ftp-fetch, http-fetch): Use 'dump-port*'.
(progress-proc): Remove procedure.
* guix/scripts/substitute.scm (progress-report-port): Rewrite in terms of
<progress-reporter>.
(process-substitution): Adjust accordingly.
* guix/build/download.scm (http-fetch): Complete the hard-coded list of HTTP
redirection status codes.
* guix/http-client.scm (http-fetch): Likewise.
* guix/scripts/lint.scm (probe-uri): Likewise.
Reported by Mark H Weaver <mhw@netris.org>
at <https://debbugs.gnu.org/cgi/bugreport.cgi?bug=26948#17>.
* guix/build/download.scm (set-certificate-credentials-x509-trust-file!*):
New procedure.
(make-credendials-with-ca-trust-files): Use it instead of
'set-certificate-credentials-x509-trust-file!'.
This allows us to download from site such as
<https://fusionforge.int-evry.fr> where the server does not recognize
the server name passed via the 'server_name' extension.
* guix/build/download.scm (tls-wrap): Catch 'gnutls-error' around
'handshake'. Upon ERROR/WARNING-ALERT-RECEIVED, print a message and
call 'handshake'.
Fixes a regression introduced in
a4f5423415.
Reported by Hartmut Goebel.
* guix/build/download.scm (ftp-fetch): Account for the case where
'uri-userinfo' returns #f. Remove the case where it returns "" since
that cannot happen.
* guix/build/download.scm (open-connection-for-uri): Add note about
same-named binding in Guile 2.2.0.
* guix/http-client.scm: Use 'guix:open-connection-for-uri' for the
procedure coming from (guix build download).
* guix/scripts/lint.scm: Likewise.
* guix/scripts/substitute.scm: Likewise.
Reported by Christopher Baines <mail@cbaines.net>
in <https://bugs.gnu.org/25213>.
* guix/build/download.scm (make-credendials-with-ca-trust-files): Check
whether FILE exists before calling
'set-certificate-credentials-x509-trust-file!'.
* guix/download.scm (url-fetch): Include (guix base64) module on the
build-side.
* guix/build/download.scm (http-fetch): Add "Authorization" header when
userinfo is present in the URI.
* guix/build/download.scm (ftp-fetch): Add #:timeout and pass it to
'ftp-open'.
(http-fetch): Add #:timeout and pass it to 'open-connection-for-uri' and
in recursive calls.
(url-fetch): Add #:timeout and pass it to 'http-fetch' and 'ftp-fetch'.
This fixes a bug whereby 'http-fetch' would be passed a string instead
of a URI object.
* guix/build/download.scm (url-fetch): Rename 'content-addressed-urls'
to 'content-addressed-uris', and call 'string->uri'.
* guix/download.scm (%content-addressed-mirrors)
(%content-addressed-mirror-file): New variables.
* guix/download.scm (url-fetch)[builder]: Define
'value-from-environment. Pass #:hashes and
#:content-addressed-mirrors to 'url-fetch'.
Define "guix download hashes" environment variable.
* guix/build/download.scm (url-fetch): Add #:content-addressed-mirrors
and #:hashes.
[content-addressed-urls]: New variable.
Use it.
Suggested by Danny Milosavljevic <dannym@scratchpost.org>.
* guix/build/download.scm (nar-uri-abbreviation): New procedure.
* guix/scripts/substitute.scm (process-substitution): Use it instead of
'store-path-abbreviation'.
Fixes <http://bugs.gnu.org/23275>.
Reported by Albin Söderqvist <albin@fripost.org>.
* guix/build/download.scm (http-fetch): Follow redirections upon 307.
This is what 'binaries.openttd.org' does.
Partially fixes <http://bugs.gnu.org/20145>.
* guix/build/download.scm (add-weak-reference): Remove.
(%tls-ports): New variable.
(register-tls-record-port): New procedure.
(tls-wrap): Use it instead of 'add-weak-reference'.
(close-connection): New procedure.
* guix/build/download.scm (open-socket-for-uri): Always pass
AI_ADDRCONFIG to 'getaddrinfo' as recommended in the fine Guile
manual.
* guix/ftp-client.scm (ftp-open): Ditto.
* guix/build/download.scm (ensure-uri): New procedure.
(current-http-proxy): New variable.
(open-socket-for-uri): Copy from Guile commit aaea5b2, but add #:timeout
parameter and use 'connect*' instead of 'connect'.
(open-connection-for-uri): Add #:timeout parameter and pass it to
'open-socket-for-uri'.
Fixes a regression introduced in a8be7b9a.
* guix/build/download.scm (store-path-abbreviation): Return STORE-PATH
if it's not an actual store path. Fixes an out-of-range exception
when running tests/substitute.scm and tests/store.scm.
* guix/build/download.scm
(string-pad-middle): Allow resulting padded string to overflow.
(store-url-abbreviation): Remove unnecessary procedure.
(progress-proc): Use BASENAME as default for parameter 'abbreviation'.
(url-fetch): Display extra newlines for readability.
* guix/build/download.scm
(string-pad-middle, store-url-abbreviation, store-path-abbreviation):
New procedures.
(progress-proc): Add #:abbreviation parameter and use it. Generate a
better indeterminate progress string.
* guix/scripts/substitute.scm (assert-valid-narinfo): Add newlines to output.
(process-substitution): Use byte-count->string and store-path-abbreviation.
Co-authored-by: Ludovic Courtès <ludo@gnu.org>
* guix/build/download.scm (seconds->string): New function.
(byte-count->string): New function.
(progress-bar): New function.
(throughput->string): Remove function.
(progress-proc): Display base file name, elapsed time, and progress bar.
Signed-off-by: Ludovic Courtès <ludo@gnu.org>
Previously, code using directly (guix build download) was still affected
by <http://bugs.gnu.org/15368>. This includes source derivations, the
'guix download' command, and (guix gnu-maintenance).
'guix substitute' was unaffected since it used (guix http-client), which
already had the fix.
* guix/http-client.scm (open-socket-for-uri): Remove.
(http-fetch): Remove #:buffered? argument to 'open-socket-for-uri';
use 'setvbuf' instead.
* guix/scripts/substitute.scm (fetch): Likewise.
* guix/build/download.scm (open-socket-for-uri): New procedure, taken
from guix/http-client.scm, but without the #:buffered? parameter.
Partly fixes <http://bugs.gnu.org/20402>.
Reported by Joshua Randall <jcrandall@alum.mit.edu>.
* guix/build/download.scm (open-connection-for-uri): Rewrite to be a
small wrapper around 'open-socket-for-uri'. This procedure was
initially introduced in d14ecda to work around the lack of NSS modules
during bootstrap but that has become unnecessary since 0621349, which
introduced a bootstrap Guile that uses static NSS modules (from commit
d3b5972.)
On Guile >= 2.0.10, this allows the 'http_proxy' environment variable
to be used.
Ludovic Courtès