Commit Graph

787 Commits

Author SHA1 Message Date
Ludovic Courtès 987a29ba43
Add (guix ssh) module.
* guix/scripts/offload.scm (connect-to-remote-daemon)
(store-import-channel, store-export-channel, send-files)
(retrieve-files): Move to (guix ssh).
(nonce): Add optional 'name' parameter and use it.
(retrieve-files*): New procedure.
(transfer-and-offload): Use it instead of 'retrieve-files', and add
first parameter to 'send-files'.
(assert-node-can-import): Likewise.
(assert-node-can-export): Use 'retrieve-files' instead of
'store-export-channel'.
* guix/ssh.scm: New file.
* configure.ac: Use 'GUIX_CHECK_GUILE_SSH' and define 'HAVE_GUILE_SSH'
Automake conditional.
* Makefile.am (MODULES) [HAVE_GUILE_SSH]: Add guix/ssh.scm.
2016-12-31 01:44:04 +01:00
Ludovic Courtès 7c247809ef
guix build: '--with-source' overrides the 'replacement' of a package.
* guix/scripts/build.scm (package-with-source): Set 'replacement' to #f.
* tests/scripts-build.scm ("options->transformation, with-source,
replacement"): New test.
2016-12-21 23:48:14 +01:00
Ludovic Courtès f943c317fb
environment: Add '--root' option.
* guix/scripts/environment.scm (show-help, %options): Add --root.
(register-gc-root): New procedure.
(guix-environment): Call 'register-gc-root' when OPTS has a 'gc-root'
option.
* doc/guix.texi (Invoking guix environment): Document it.
* tests/guix-environment.sh: Add tests.
2016-12-20 19:14:41 +01:00
Ricardo Wurmus d68ba5f483
guix import: Print list of expressions.
* guix/scripts/import.scm (guix-import): Print list of expressions.
2016-12-17 19:13:41 +01:00
Ricardo Wurmus 64ce53eb5e
import cran: Add "recursive" option.
* guix/scripts/import/cran.scm: (%options): Add "recursive" option.
(guix-import-cran): Handle "recursive" option.
* doc/guix.texi (Invoking guix import): Document it.
2016-12-17 19:13:41 +01:00
Ludovic Courtès a6562c7e20
profiles: Remove dependency on 'glibc-utf8-locales' for tests.
Commit 1af0860e8b added a mandatory
dependency on 'glibc-utf8-locales', which entails long rebuilds for
tests.

* guix/profiles.scm (profile-derivation): Add #:locales? parameter.
Add 'set-utf8-locale' variable.  Use it when LOCALES? is true.
(link-to-empty-profile): Pass #:locales? #f.
* guix/scripts/environment.scm (inputs->profile-derivation): Pass
  #:locales?.
* guix/scripts/package.scm (build-and-use-profile): Likewise.
* tests/packages.scm ("--search-paths with pattern"): Pass #:locales? #f.
* tests/profiles.scm ("profile-derivation")
("profile-derivation, inputs", "profile-manifest, search-paths")
("etc/profile", "etc/profile when etc/ already exists"):
("etc/profile when etc/ is a symlink"): Likewise.
2016-12-17 14:46:42 +01:00
Ludovic Courtès 99ec0cb397
refresh: Make dependency on (guix import crates) weak.
Reported by Tobias Geerinckx-Rice <me@tobias.gr>.

* guix/scripts/refresh.scm: Do not explicitly import (guix import crates).
2016-12-16 17:38:39 +01:00
Ricardo Wurmus 642339dc3f
graph: Add "list-backend" and "backend" options.
* guix/graph.scm (%graph-backends): New variable.
* guix/scripts/graph.scm (lookup-backend, list-backends): New
procedures.
(%options): Add options for "backend" and "list-backends".
(show-help): Add help texts for "backend" and "list-backend" options.
(%default-options): Add "backend" default.
(guix-graph): Pass backend argument to "export-graph".
* doc/guix.texi (Invoking guix graph): Document the new options.
2016-12-14 21:57:37 +01:00
David Craven 8ac5298786
import: Add updater for rust crates.
* guix/import/crate.scm (crate-package?, latest-release,
  %crate-updater): New variables.
* guix/scripts/refresh.scm (%updaters): Add crate updater.
* doc/guix.texi: Add crate updater to table.
2016-12-14 16:30:42 +01:00
David Craven 3e0c036584
import: Add importer for rust crates.
* guix/import/crate.scm: New file.
* guix/scripts/import/crate.scm: New file.
* guix/scripts/import.scm (importers): Add crate importer.
* tests/crate.scm: New file.
* doc/guix.texi: Add crate importer to table.
* Makefile.am (MODULES, SCM_TESTS): Add files.
2016-12-14 16:30:42 +01:00
Eric Bavier ff55fe5599
import: cpan: Add updater.
* guix/import/cpan.scm (module->dist-name): Fetch the field of interest.
(cpan-fetch): Accept release name rather than module name.
(fix-source-url): Rename to ...
(cpan-source-url): ... this.  Take metadata as parameter.
(package->upstream-name, cpan-version, cpan-package?, latest-release):
New procedures.
(cpan-module->sexp): Use cpan-version and cpan-source-url.
(%cpan-updater): New variable.
* guix/scripts/refresh.scm (%updaters): Add %CPAN-UPDATER.
2016-12-12 22:22:33 -06:00
Ludovic Courtès 27991c97e6
offload: Allow testing machines that match a regexp.
* guix/scripts/offload.scm (check-machine-availability): Add 'pred'
parameter and honor it.
(guix-offload): for the "test" sub-command, accept an extra 'regexp'
parameter.  Pass a second argument to 'check-machine-availability'.
2016-12-09 23:12:06 +01:00
Ludovic Courtès 2b513387cd
offload: Test each machine only once.
* guix/scripts/offload.scm (check-machine-availability)[build-machine=?]:
New procedure.
Add call to 'delete-duplicates'.
2016-12-09 23:02:02 +01:00
Ludovic Courtès bd8345777f
offload: Do not read ~/.ssh/known_hosts.
* guix/scripts/offload.scm (open-ssh-session): Pass #:knownhosts to
'make-session'.
2016-12-09 23:02:01 +01:00
Ludovic Courtès 8d125cfc2e
offload: Increase the connection timeout.
* guix/scripts/offload.scm (open-ssh-session): Set #:timeout to 10.
2016-12-06 01:00:11 +01:00
Ludovic Courtès 0237d79717
offload: Send the build log to the right file descriptor.
This fixes a regression introduced in
21531add32 whereby the build log would no
longer be sent to FD 4, thereby leading the daemon to not see the build
log.

* guix/scripts/offload.scm (transfer-and-offload): Parameterize
CURRENT-BUILD-OUTPUT-PORT.
2016-12-06 00:50:08 +01:00
Ludovic Courtès e11c42f297
offload: Fix plural of some messages.
* guix/scripts/offload.scm (send-files): Use 'N_' for possibly plural
message.  Write "store item" instead of "store file".
(retrieve-files): Likewise.
2016-12-05 23:26:38 +01:00
Ludovic Courtès 1d48cf948c
offload: Make the compression methods configurable.
* guix/scripts/offload.scm (<build-machine>)[compression]
[compression-level]: New fields.
(open-ssh-session): Honor them.
* doc/guix.texi (Daemon Offload Setup): Document them.
2016-12-05 23:19:38 +01:00
Chris Marusich 067a2e2de9
guix system: If the new system generation already exists, overwrite it.
Fixes <http://bugs.gnu.org/25100>.

Before that, 'guix system reconfigure' would fail if the next generation
already existed.

* guix/scripts/system.scm (switch-to-system): Use 'switch-symlink'
instead of 'symlink'.
* doc/guix.texi (Using the Configuration System, Invoking guix system):
Document the behavior.

Signed-off-by: Ludovic Courtès <ludo@gnu.org>
2016-12-05 23:19:37 +01:00
Tobias Geerinckx-Rice d8501b844b
refresh: Indicate that PACKAGE... is optional.
* guix/scripts/refresh.scm (show-help): Add brackets around PACKAGE.
2016-12-05 20:50:12 +01:00
Ludovic Courtès aebaee95cc
offload: Add "test" sub-command.
* guix/scripts/offload.scm (assert-node-repl, assert-node-has-guix)
(nonce, assert-node-can-import, assert-node-can-export)
(check-machine-availability): New procedures.
(%random-state): New variable.
(guix-offload): Add case for "test".
* doc/guix.texi (Daemon Offload Setup): Document it.  Remove obsolete
bit about remote invocation of 'guix build'.
2016-12-05 18:18:10 +01:00
Ludovic Courtès 42d07286f4
publish: Factorize 'content-length' addition.
* guix/scripts/publish.scm (with-content-length): New procedure.
(http-write) <application/octet-stream>: Use it.
2016-12-04 22:32:26 +01:00
Ludovic Courtès 463fb7d0c8
offload: Do not abort when a machine is unreachable.
* guix/scripts/offload.scm (machine-load): Wrap 'open-ssh-session' call
in 'false-if-exception'; return +inf.0 if it returns #f.
2016-12-01 23:35:11 +01:00
Ludovic Courtès 74afca5dcf
offload: Gracefully report connection failures.
* guix/scripts/offload.scm (open-ssh-session): Check the return value of
'connect!'.  Call 'leave' when it's not 'ok.
2016-12-01 23:35:10 +01:00
Ludovic Courtès 0b72475301
offload: Warn about the lack of zlib support.
* guix/scripts/offload.scm (guix-offload): Print a warning when
'zlib-support?' returns false.
2016-12-01 23:35:10 +01:00
Ludovic Courtès f3cf860635
offload: Remove redundant call to 'topologically-sorted' in 'send-file'.
* guix/scripts/offload.scm (send-files): Remove call to
'topologically-sorted'.
2016-12-01 23:35:10 +01:00
Ludovic Courtès 3676f89255
refresh: '--list-updaters' shows updater coverage.
* guix/scripts/refresh.scm (list-updaters-and-exit): Compute the
coverage ratio of each updater and print it.  Print the coverage ratio
for all the updaters.
* doc/guix.texi (Invoking guix refresh): Document it.
2016-11-30 17:35:21 +01:00
Ludovic Courtès 2fd370e816
gnu-maintenance: Add kernel.org updater.
* guix/gnu-maintenance.scm (latest-kernel.org-release): New procedure.
(%kernel.org-updater): New variable.
* guix/scripts/refresh.scm (%updaters): Add it.
2016-11-30 17:35:21 +01:00
Ludovic Courtès 4e6230ec00
refresh: Honor the selected updaters when '-u' isn't given.
Fixes a regression introduced in
e9c72306fd.

* guix/scripts/refresh.scm (check-for-package-update): Add 'updaters'
parameter and honor it.
(guix-refresh): Pass UPDATERS to 'check-for-package-update'.
2016-11-30 17:35:21 +01:00
Hartmut Goebel 3bf4280659
Merge branch 'master' into python-build-system 2016-11-29 18:47:16 +01:00
Ludovic Courtès e9c72306fd
refresh: Warn about packages that lack an updater.
* guix/upstream.scm (package-update-path): Rename to...
(package-latest-release): ... this.  Remove 'version>?' check.
(package-latest-release*): New procedure.
(package-update): Use it.
* guix/scripts/refresh.scm (lookup-updater): Rename to...
(lookup-updater-by-name): ... this.
(warn-no-updater): New procedure.
(update-package): Add #:warn? parameter and honor it.
(check-for-package-update): New procedure.
(guix-refresh)[warn?]: New variable.
Replace inline code when UPDATE? is false with a call to
'check-for-package-update'.
Pass WARN? to 'check-for-package-update' and 'update-package'.
* doc/guix.texi (Invoking guix refresh): Document it.  Fix a couple of
typos.
2016-11-29 16:12:31 +01:00
Ludovic Courtès 1cd1d8a7ea
offload: Call 'machine-load' only once per machine.
This fixes a longstanding issue where 'choose-build-machine' would make
on average O(N log(N)) calls to 'machine-load', plus an extra call for
the selected machine, instead of N calls.

* guix/scripts/offload.scm (machine-load): Add comment.
(machine-power-factor, machine-less-loaded-or-faster?): Remove.
(choose-build-machine)[machines+slots]: Rename to...
[machines+slots+loads]: ... this.
[undecorate]: Adjust accordingly.
[machine-less-loaded-or-faster?]: New procedure.
Remove extra 'machine-load' call in body.
2016-11-26 23:21:37 +01:00
Ludovic Courtès bc1ad4e334
offload: Drop 'remote-pipe'.
* guix/scripts/offload.scm (remote-pipe): Remove.
(machine-load): Use 'open-remote-pipe*' instead of 'remote-pipe'.
2016-11-25 23:44:21 +01:00
Ludovic Courtès cf283dd92e
offload: Rewrite to make direct RPCs to the remote daemon.
* guix/scripts/offload.scm (<build-machine>)[daemon-socket]: New field.
(connect-to-remote-daemon): New procedure.
(%gc-root-file, register-gc-root, remove-gc-roots, offload): Remove.
(transfer-and-offload): Rewrite using 'connect-to-remote-daemon' and
RPCs over SSH.
(store-import-channel, store-export-channel): New procedures.
(send-files, retrieve-files): Rewrite using these.
2016-11-25 23:44:20 +01:00
Ludovic Courtès e8a5db80d5
offload: Remove 'with-nar-error-handling' macro.
* guix/scripts/offload.scm (with-nar-error-handling): Remove.
(guix-offload): Use 'with-error-handling' instead.
2016-11-25 23:44:20 +01:00
Ludovic Courtès 9e76eed37f
offload: Reuse SSH session during 'transfer-and-offload'.
* guix/scripts/offload.scm (remote-pipe): Replace 'machine' parameter
with 'session'.  Remove 'open-ssh-session' call.
(register-gc-root): Replace 'machine' with 'session'.  Use '
session-get' instead of 'build-machine-name'.
(remove-gc-roots, offload, send-files, retrieve-files): Likewise.
(transfer-and-offload): Add 'open-ssh-session' call.  Handle 'offload'
errors here.
(machine-load): Add call to 'open-ssh-session'.
2016-11-25 23:44:20 +01:00
Ludovic Courtès 21531add32
offload: Use Guile-SSH instead of GNU lsh.
* guix/scripts/offload.scm (<build-machine>)[ssh-options]: Remove.
[host-key, host-key-type]: New fields.
(%lsh-command, %lshg-command, user-lsh-private-key): Remove.
(user-openssh-private-key, private-key-from-file*): New procedures.
(host-key->type+key, open-ssh-session): New procedures.
(remote-pipe): Remove 'mode' parameter.  Rewrite in terms of
'open-ssh-session' etc.  Update users.
(send-files)[missing-files]: Rewrite using the bidirectional channel
port.
Remove call to 'call-with-compressed-output-port'.
(retrieve-files): Remove call to 'call-with-decompressed-port'.
(machine-load): Remove exit status logic.
* doc/guix.texi (Requirements): Mention Guile-SSH.
(Daemon Offload Setup): Document 'host-key' and 'private-key'.  Show the
default value on each @item line.
* m4/guix.m4 (GUIX_CHECK_GUILE_SSH): New macro.
* config-daemon.ac: Use 'GUIX_CHECK_GUILE_SSH'.  Set
'HAVE_DAEMON_OFFLOAD_HOOK' as a function of that.
2016-11-25 23:44:20 +01:00
Leo Famulari de32aa74b4
Merge branch 'master' into python-build-system 2016-11-25 11:20:21 -05:00
Ludovic Courtès fac46e3f5e
lint: Add 'mirror-url' checker.
* guix/scripts/lint.scm (origin-uris): New procedure.
(check-source): Use it.
(check-mirror-url): New procedure.
(%checkers): Add 'mirror-url' checker.
* tests/lint.scm ("mirror-url")
("mirror-url: one suggestion"): New tests.
* doc/guix.texi (Invoking guix lint): Document it.
2016-11-19 18:44:53 +01:00
Ludovic Courtès e74f64b9e5
store: Add 'references*'.
* guix/store.scm (references*): New procedure.
* guix/profiles.scm (manifest-lookup-package)[references*]: Remove.
* guix/scripts/system.scm (references*): Remove.
* tests/gexp.scm ("gexp->file", "gexp->file + file-append")
("gexp->derivation", "gexp->derivation, cross-compilation")
("gexp->derivation, ungexp + ungexp-native")
("scheme-file", "text-file*", "mixed-text-file"): Remove 'references*'
instead of (store-lift references).
2016-11-19 18:44:53 +01:00
Ludovic Courtès 94d92c7796
daemon: Add "builtin:download" derivation builder.
This ensures that 1) the derivation doesn't change when Guix changes;
2) the derivation closure doesn't contain Guix and its dependencies; 3)
we don't have to rely on ugly chroot hacks.

Adapted from Nix commit 0a2bee307b20411f5b0dda0c662b1f9bb9e0e131.

* nix/libstore/build.cc (DerivationGoal::runChild): Add special case for
'isBuiltin(drv)'.  Disable chroot when 'isBuiltin(drv)'.
* nix/libstore/builtins.cc, nix/libstore/builtins.hh,
nix/scripts/download.in, guix/scripts/perform-download.scm: New files.
* guix/ui.scm (show-guix-help)[internal?]: Add 'perform-download'.
* nix/local.mk (libstore_a_SOURCES): Add builtins.cc.
(libstore_headers): Add builtins.hh.
(nodist_pkglibexec_SCRIPTS): Add 'scripts/download'.
* config-daemon.ac: Emit 'scripts/download'.
* Makefile.am (MODULES): Add 'guix/scripts/perform-download.scm'.
* tests/derivations.scm ("unknown built-in builder")
("'download' built-in builder")
("'download' built-in builder, invalid hash")
("'download' built-in builder, not found")
("'download' built-in builder, not fixed-output"): New tests.

Co-authored-by: Eelco Dolstra <eelco.dolstra@logicblox.com>
2016-11-16 18:19:47 +01:00
Ludovic Courtès 1bcc87bb68
guix download: Add '-o' option.
* guix/scripts/download.scm (download-to-file, download-to-store*): New
procedures.
(%default-options): Add 'download-proc'.
(show-help): Adjust description and document '-o'.
(%options): Add '-o'.
(guix-download): Remove 'store' variable.  Add 'fetch' and define 'path'
to as its result.
* tests/guix-download.sh: Add test.
2016-11-16 18:00:41 +01:00
Hartmut Goebel e442246a2f
lint: more packages to probably be a native input.
* guix/scripts/lint.scm (check-inputs-should-be-native package): Add python
  packages which are typically used for testing or for building the
  documentation.
2016-11-15 17:37:50 +01:00
Hartmut Goebel 891a843d51
guix: Add lint-checker for packages which should be no inputs at all.
Also refactor some common code into a new function.

Examples for these pacakges are python(2)-setuptools and python(2)-pip, which
are installed together with python itself.

* guix/scripts/lint.scm (warn-if-package-has-input): New procedure.
  (check-inputs-should-be-native package): Use it; rename and clean-up
  variables. (check-inputs-should-not-be-an-input-at-all): New procedure.
  (%checkers) Add it.
* doc/guix.texi (Python Modules): Document it.
* tests/lint.scm: ("inputs: python-setuptools should not be an input at all
  (input)", "inputs: python-setuptools should not be an input at all
  (native-input)" "inputs: python-setuptools should not be an input at all
  (propagated-input)"): Add tests.
2016-11-15 17:37:49 +01:00
Ludovic Courtès 166ba5b102
substitute: Disable HTTPS certificate verification.
Fixes a regression introduced in
9e4e431e04 as a consequence of
bc3c41ce36.
Reported by Marius Bakke <mbakke@fastmail.com>.

* guix/scripts/substitute.scm (fetch): Pass #:verify-certificate? #f to
'open-connection-for-uri' and 'http-fetch'.
(download-cache-info): Likewise.
(http-multiple-get): Add #:verify-certificate? and honor it.
(fetch-narinfos): Pass #:verify-certificate? #f.
2016-11-12 13:04:45 +01:00
Ludovic Courtès 5970e8e248
container: Pass a list of <file-system> objects as things to mount.
* gnu/build/linux-container.scm (mount-file-systems): 'mounts' is now a
list of <file-system> objects instead of a list of lists ("specs").
Add call to 'file-system->spec' as the argument to 'mount-file-system'.
(run-container, call-with-container): Adjust docstring accordingly.
* gnu/system/file-systems.scm (spec->file-system): New procedure.
* gnu/system/linux-container.scm (container-script)[script]: Call
'spec->file-system' inside gexp.
* guix/scripts/environment.scm (launch-environment/container): Remove
call to 'file-system->spec'.
* tests/containers.scm ("call-with-container, mnt namespace")
("call-with-container, mnt namespace, wrong bind mount"): Pass a list of
<file-system> objects.
2016-11-10 18:01:44 +01:00
Ludovic Courtès c169d91e5a
lint: 'cve' checker catches 'tls-certificate-error'.
Reported by Frederick Muriithi <fredmanglis@gmail.com>.

* guix/scripts/lint.scm (tls-certificate-error-string): New procedure.
(validate-uri): Use it.
(current-vulnerabilities*): Catch 'tls-certificate-error' and print a
warning.
2016-11-09 21:24:44 +01:00
Marius Bakke 81bf2ccbc4
system: Use grub from bootloader configuration.
* gnu/system/grub.scm (gnu): Export grub-configuration-grub.
* guix/scripts/system.scm (perform-action): Use it.
(define-module): Don't import (gnu packages grub).

Co-authored-by: Danny Milosavljevic <dannym@scratchpost.org>
2016-11-08 14:35:05 +00:00
Ludovic Courtès 64b8695cd8
guix download: Add '--no-check-certificate' option.
* guix/download.scm (download-to-store): Add #:verify-certificate?
parameter and honor it.
* guix/scripts/download.scm (%default-options): Add
'verify-certificate?' key.
(show-help, %options): Add '--no-check-certificate'.
(guix-download): Pass #:verify-certificate to 'download-to-store'.
* doc/guix.texi (Invoking guix download): Document it.
2016-11-07 23:39:01 +01:00
Ludovic Courtès bc3c41ce36
download: Verify TLS certificates unless asked not to.
Fixes <http://bugs.gnu.org/24466>.
Reported by Leo Famulari <leo@famulari.name>.

* guix/build/download.scm (%x509-certificate-directory): New variable.
(make-credendials-with-ca-trust-files, peer-certificate)
(assert-valid-server-certificate, print-tls-certificate-error): New
procedures.  Add 'print-tls-certificate-error' as an exception printer
for 'tls-certificate-error'.
(tls-wrap): Add #:verify-certificate? parameter and honor it.
(open-connection-for-uri): Likewise.
(http-fetch): Likewise.
(url-fetch): Likewise.
* guix/download.scm (url-fetch)[builder]: Pass #:verify-certificate? #f.
* guix/scripts/lint.scm (probe-uri): Add case for 'tls-certificate-error'.
(validate-uri): Likewise.
* doc/guix.texi (Invoking guix download): Mention 'SSL_CERT_DIR'.
2016-11-07 23:39:01 +01:00