Commit Graph

85 Commits

Author SHA1 Message Date
Steve Sprang 0c0a1f22ce build: Improve information density and appearance of download progress output.
* guix/build/download.scm (seconds->string): New function.
  (byte-count->string): New function.
  (progress-bar): New function.
  (throughput->string): Remove function.
  (progress-proc): Display base file name, elapsed time, and progress bar.

Signed-off-by: Ludovic Courtès <ludo@gnu.org>
2015-09-08 21:47:30 +02:00
Ludovic Courtès 4f7b564adb download: Remove spurious warning about 'https_proxy'.
* guix/build/download.scm (open-connection-for-uri)[with-https-proxy]: Warn
  about 'https_proxy' only when 'getenv' returns a non-empty string.
2015-07-17 22:31:12 +02:00
Ludovic Courtès 7623848343 download: Work around Guile small-receive-buffer bug.
Previously, code using directly (guix build download) was still affected
by <http://bugs.gnu.org/15368>.  This includes source derivations, the
'guix download' command, and (guix gnu-maintenance).

'guix substitute' was unaffected since it used (guix http-client), which
already had the fix.

* guix/http-client.scm (open-socket-for-uri): Remove.
  (http-fetch): Remove #:buffered? argument to 'open-socket-for-uri';
  use 'setvbuf' instead.
* guix/scripts/substitute.scm (fetch): Likewise.
* guix/build/download.scm (open-socket-for-uri): New procedure, taken
  from guix/http-client.scm, but without the #:buffered? parameter.
2015-05-06 10:31:11 +02:00
Ludovic Courtès c822fb8e34 download: Reinstate buffering on connection sockets.
* guix/build/download.scm (open-connection-for-uri): Reinstate call to
  'setvbuf' inadvertently removed in d17551d9.
2015-05-06 09:50:15 +02:00
Ludovic Courtès d17551d943 download: Simplify 'open-connection-for-uri' to support HTTP proxies.
Partly fixes <http://bugs.gnu.org/20402>.
Reported by Joshua Randall <jcrandall@alum.mit.edu>.

* guix/build/download.scm (open-connection-for-uri): Rewrite to be a
  small wrapper around 'open-socket-for-uri'.  This procedure was
  initially introduced in d14ecda to work around the lack of NSS modules
  during bootstrap but that has become unnecessary since 0621349, which
  introduced a bootstrap Guile that uses static NSS modules (from commit
  d3b5972.)
  On Guile >= 2.0.10, this allows the 'http_proxy' environment variable
  to be used.
2015-05-01 00:06:17 +02:00
Ludovic Courtès c9727aac40 download: Comment on lack of progress report with chunked encoding.
* guix/build/download.scm (progress-proc): Add comment.
2015-02-27 15:08:37 +01:00
Ludovic Courtès 9fbe6f1920 download: Measure and display the throughput.
* guix/build/download.scm (duration->seconds, throughput->string): New
  procedures.
  (progress-proc): Measure and display the throughput.
2015-02-27 15:08:37 +01:00
Ludovic Courtès e7620dc995 download: Abstract the receive buffer size.
* guix/build/download.scm (%http-receive-buffer-size): New variable.
  (progress-proc, tls-wrap, http-fetch): Use it.
2015-02-27 15:08:37 +01:00
Mark H Weaver c964a15d82 download: Cope with Guile 2.0.6 or earlier.
* guix/build/download.scm: Do not attempt to support relative URIs in
  "Location" headers if 'declare-relative-uri-header!' is not present.
  This is the case for Guile 2.0.6 or earlier.
2015-02-24 20:01:55 -05:00
Mark H Weaver 04dec194d8 download: Handle HTTP redirects to relative URI references.
Fixes <http://bugs.gnu.org/19840>.
Reported by Ricardo Wurmus <rekado@elephly.net>.

* guix/build/download.scm: On Guile 2.0.11 or earlier, redefine the http
  "Location" header to accept relative URIs.
  (resolve-uri-reference): New exported procedure.
  (http-fetch): Use 'resolve-uri-reference' to resolve redirections.
* guix/http-client.scm (http-fetch): Use 'resolve-uri-reference'
2015-02-19 03:32:22 -05:00
Ludovic Courtès dd8ea244f4 download: Export 'maybe-expand-mirrors'.
* guix/build/download.scm (uri-vicinity, maybe-expand-mirrors): New
  procedures.
  (url-fetch): Remove them from here.
2014-12-29 21:19:17 +01:00
Ludovic Courtès a3bf096945 lint: Add 'home-page' checker.
* guix/build/download.scm (open-connection-for-uri): Export.
* guix/scripts/lint.scm (probe-uri, check-home-page): New procedures.
  (%checkers): Add 'home-page' checker.
2014-12-28 17:57:36 +01:00
Ludovic Courtès 38bf090e7c download: Add "Accept: */*" to the headers.
Fixes downloads from https://alioth.debian.org.
Reported by John Darrington <jmd@gnu.org>.

* guix/build/download.scm (http-fetch)[headers]: Add 'Accept'.
2014-10-19 12:07:53 +02:00
Ludovic Courtès 077bd18d22 download: Use the 'SERVER NAME' TLS extension when possible.
Fixes <http://bugs.gnu.org/18526>.
Reported by Mark H. Weaver.

* guix/build/download.scm (tls-wrap): Add 'server' parameter.  Call
  'set-session-server-name!' when (gnutls) defines it.
  (open-connection-for-uri): Adjust 'tls-wrap' call accordingly.
2014-09-22 21:06:39 +02:00
Ludovic Courtès a68d976b66 download: Enlarge your receive buffer.
* guix/build/download.scm (open-connection-for-uri): Remove call to
  'setsockopt'.
* guix/http-client.scm (open-socket-for-uri)[rmem-max, buffer-size]: New
  variables.  Add call to 'setsockopt'.
2014-05-23 22:19:37 +02:00
Ludovic Courtès 395bea2a53 download: Improve progress report output.
* guix/build/download.scm (url-fetch): Make current-output-port
  unbuffered.
2014-04-16 12:25:52 +02:00
Ludovic Courtès 212ece42b0 download: Don't use 'http-get*' on Guile 2.0.10+.
* guix/build/download.scm (http-fetch)[post-2.0.7?]: Use
  'string->number' and numeric comparison.  This fixes version
  comparison with "2.0.10" and subsequent 2.0 releases.
2014-03-25 18:54:52 +01:00
Ludovic Courtès 2de227af4b download: Provide a 'User-Agent' field in HTTP requests.
Fixes <http://bugs.gnu.org/16703>.
Reported by Raimon Grau <raimonster@gmail.com>.

* guix/build/download.scm (http-fetch)[headers]: New variable.
  Pass it as #:headers or #:extra-headers to 'http-get' and
  'http-get*'.
2014-02-10 00:03:34 +01:00
Ludovic Courtès 91a7fde456 download: Don't fail when abbreviating `file://' URIs.
* guix/build/download.scm (uri-abbreviation)[elide-path]: Handle the
  case where URI has no `host' part.
2013-06-22 16:10:25 +02:00
Ludovic Courtès a85060efec substitute-binary: Report progress while downloading.
* guix/scripts/substitute-binary.scm (decompressed-port): Improve docstring.
  (progress-report-port): New procedure.
  (guix-substitute-binary)["--substitute"]: Use it to report progress.
* guix/build/download.scm: Export `progress-proc' and `uri-abbreviation'.
2013-06-20 23:41:28 +02:00
Ludovic Courtès dd9afe64b5 download: Fix premature socket close on TLS connections.
This would manifest when downloading a large file such as the Bazaar
tarball, leading to an "Error in the pull function" GnuTLS exception.

* guix/build/download.scm (add-weak-reference): New procedure.
  (tls-wrap): Add (add-weak-reference record port).
2013-05-10 01:14:25 +02:00
Ludovic Courtès 80736cdf20 download: Adjust to `http-get*' deprecation.
* guix/build/download.scm (http-fetch): Adjust to use #:streaming? when
  using Guile 2.0.8+.
2013-02-20 22:59:35 +01:00
Ludovic Courtès 67158a4900 download: Follow HTTP redirection upon 301 "moved permanently".
* guix/build/download.scm (http-fetch): Follow redirections upon 301.
  This is what `downloads.sourceforge.net' returns, for instance.
2013-01-25 21:52:08 +01:00
Ludovic Courtès 483f11589e download: Add HTTPS support.
* guix/build/download.scm: Autoload (gnutls).
  (tls-wrap): New procedure.
  (open-connection-for-uri): Add support for `https'.  Wrap the socket
  with `tls-wrap' in that case.
  (url-fetch): Add `https'.
* guix/download.scm (gnutls-derivation): New procedure.
  (url-fetch)[need-gnutls?]: New variable.
  Call `gnutls-derivation' when NEED-GNUTLS? is true, and add its output
  to the `GUILE_LOAD_PATH' env. var. in that case.
2013-01-20 22:54:36 +01:00
Ludovic Courtès 28e5560421 download: Abbreviate URLs when displaying the progress report.
* guix/build/download.scm (uri-abbreviation): New procedure.
  (ftp-fetch, http-fetch): Use it instead of `uri->string' when calling
  `progress-proc'.  Reported by Andreas Enge.
2013-01-11 15:41:58 +01:00
Ludovic Courtès e66ca1a5a8 download: Report the progress of HTTP downloads.
* guix/build/download.scm (http-fetch): Rename `bv' to `bv-or-port'.
  Use `http-get*' followed by `dump-port' when the former is available,
  and pass a progress procedure to `dump-port'.
2013-01-06 18:36:50 +01:00
Ludovic Courtès e47bac7902 download: Report the progress of FTP downloads.
* guix/build/download.scm (progress-proc): New procedure.
  (ftp-fetch): Call `ftp-size' on URI.  Use `progress-proc', and pass
  the result to `dump-port', along with #:buffer-size.
2013-01-06 18:35:23 +01:00
Ludovic Courtès 4050e5d6cf Merge branch 'master' into core-updates
Conflicts:
	build-aux/download.scm
	distro/packages/autotools.scm
	distro/packages/base.scm
	distro/packages/bootstrap.scm
	distro/packages/lsh.scm
	distro/packages/make-bootstrap.scm
	distro/packages/ncurses.scm
	distro/packages/perl.scm
	tests/derivations.scm
	tests/union.scm
2013-01-06 17:33:02 +01:00
Ludovic Courtès 4155e2a909 Update license headers of builder-side code.
Change license headers with this script:

  (use-modules (guix build utils))

  (fluid-set! %default-port-encoding "UTF-8")

  (substitute* (cons "distro/packages/ld-wrapper.scm"
		     (find-files "guix/build" "\\.scm$"))
    (("^([[:graph:]]+) This file is part of Guix." _ comment-start)
     (string-append comment-start " This file is part of GNU Guix."))
    (("^([[:graph:]]+) Guix --- Nix package management.*" _ comment-start)
     (string-append comment-start
		    " GNU Guix --- Functional package management for GNU\n"))
    (("^([[:graph:]]+) Guix is " _ comment-start)
     (string-append comment-start " GNU Guix is "))
    (("^([[:graph:]]+) along with Guix." _ comment-start)
     (string-append comment-start " along with GNU Guix."))
    (("^([[:graph:]]+) Copyright \\(C\\)" _ comment-start)
     (string-append comment-start " Copyright ©")))

* distro/packages/ld-wrapper.scm, guix/build/download.scm,
  guix/build/gnu-build-system.scm, guix/build/union.scm,
  guix/build/utils.scm: Update license headers.
2013-01-05 16:08:07 +01:00
Ludovic Courtès 3c738d6bd8 download: Correctly detect "No route to host" conditions.
* guix/build/download.scm (open-connection-for-uri): Delete addrinfos
  with the same address.  Always open SOCK_STREAM/IPPROTO_IP sockets.
  Fix the error handler's condition to determine what to do.
  Reported by Nikita Karetnikov <nikita.karetnikov@gmail.com> at
  <http://lists.gnu.org/archive/html/bug-guix/2012-12/msg00150.html>.
2012-12-17 00:14:30 +01:00
Ludovic Courtès b3a53fb361 download: Work around (web client) bug <http://bugs.gnu.org/13095>.
* guix/build/download.scm: Annihilate `shutdown' in (web client).
2012-12-14 18:07:10 +01:00
Ludovic Courtès 480943dd46 download: Keep only one slash when concatenating URIs.
* guix/build/download.scm (url-fetch)[uri-vicinity]: New procedure.
  [maybe-expand-mirrors]: Use it.
2012-11-27 21:33:54 +01:00
Ludovic Courtès 94d222ad97 download: Add support for mirror:// URLs.
* guix/download.scm (%mirrors): New variable.  Mirror lists taken from
  Nixpkgs.
  (url-fetch): New `mirrors' keyword parameter.
  [builder]: Pass it.

* guix/build/download.scm (url-fetch): New `mirrors' keyword parameter.
  [maybe-expand-mirrors]: New procedure.
  [uri]: Use it.
2012-11-13 00:23:43 +01:00
Ludovic Courtès 270246defe download: Follow HTTP redirections.
* guix/build/download.scm (http-fetch): Follow the redirection when CODE
  is 302.
2012-11-13 00:23:43 +01:00
Ludovic Courtès 62cab99c32 Add (guix download) and (guix build download).
* guix/download.scm, guix/build/download.scm: New files.
* Makefile.am (MODULES): Add them.
* tests/builders.scm ("url-fetch"): New test.
* distro/packages/bootstrap.scm (bootstrap-origin): Support
  `url-fetch'.
* guix/snix.scm (snix-derivation->guix-package): Use `url-fetch' instead
  of `http-fetch'.
2012-11-12 23:35:04 +01:00