Commit Graph

3798 Commits

Author SHA1 Message Date
Ludovic Courtès 10b2834f82
offload: Adjust 'test' and 'status' to the latest changes.
This is a followup to ed7b44370f71126087eb953f36aad8dc4c44109f;
following that commit, 'guix offload test' and 'guix offload status'
would abort with a backtrace instead of clearly diagnosing a missing
'guix' command on the build machine.

* guix/scripts/offload.scm (assert-node-has-guix): Call 'leave' when
NODE is not an inferior.  Remove 'catch' blocks for 'node-repl-error'.
(check-machine-availability): Invoke 'assert-node-has-guix' first.
(check-machine-status): Print a warning when 'remote-inferior' returns #f.
2018-12-25 17:13:43 +01:00
Ludovic Courtès ed7b44370f
offload: Use (guix inferior) instead of (ssh dist node).
Using inferiors and thus 'guix repl' simplifies setup on build
machines (no need to worry about GUILE_LOAD_PATH etc.)

Furthermore, the 'guix repl -t machine' protocol running in a remote
pipe addresses several issues with the current implementation of nodes
and RREPLs in Guile-SSH: fewer round trips, doesn't leave a 'guile
--listen' process behind it, stateless (since a new process is started
each time), more efficient (the SSH channel can be reused), more
reliable (no 'pgrep', 'pkill', and shellology; see
<https://github.com/artyom-poptsov/guile-ssh/issues/11> as an example.)

* guix/ssh.scm (inferior-remote-eval): New procedure.
(send-files): Use it instead of 'make-node' and 'node-eval'.
* guix/scripts/offload.scm (node-guile-version): New procedure.
(node-free-disk-space, transfer-and-offload, node-load)
(choose-build-machine, assert-node-has-guix): Use 'remote-inferior'
instead of 'make-node' and 'inferior-eval' instead of 'node-eval'.
(assert-node-can-import, assert-node-can-export): Likewise, and add
'session' parameter.
(check-machine-availability): Likewise, and add calls to
'close-inferior' and 'disconnect!'.
(check-machine-status): Likewise.
* doc/guix.texi (Daemon Offload Setup): Remove bit related to 'guile' in
$PATH and $GUILE_LOAD_PATH; mention 'guix' alone.
2018-12-24 16:06:32 +01:00
Ludovic Courtès af15fe13b6
ssh: Add 'remote-inferior'.
* guix/inferior.scm (<inferior>)[close]: New field.
(port->inferior): New procedure.
(open-inferior): Rewrite in terms of 'port->inferior'.
(close-inferior): Honor INFERIOR's 'close' field.
(inferior-eval-with-store): Add FIXME comment.
* guix/ssh.scm (remote-inferior): New procedure.
2018-12-24 16:06:32 +01:00
Efraim Flashner c39491829a
scripts: refresh: Allow searching recursively.
* guix/scripts/refresh.scm (refresh-recursive, list-transitive): New
procedures.
(show-help): Document it.
(guix-refresh): Add flags and checks for new options.
* doc/guix.texi (Invoking guix refresh): Document new options.
2018-12-24 12:19:17 +02:00
Arun Isaac 0865d8a8f6
guix: lint: Check for source URIs redirecting to GitHub.
* guix/scripts/lint.scm (check-github-uri): New procedure.
(%checkers): Add it.
* doc/guix.texi (Invoking guix lint): Document it.
* tests/lint.scm ("github-url", "github-url: one suggestion"): New tests.
2018-12-24 08:15:17 +05:30
Ludovic Courtès 62b845c5e2
offload: Display the normalized load in 'guix offload status' output.
Fixes a regression introduced in
bbe66a530a whereby the actual
load (non-normalized) would be displayed.

* guix/scripts/offload.scm (check-machine-status): Add call to
'normalized-load'.
2018-12-23 22:14:15 +01:00
Ludovic Courtès bdf860c2e9
database: Use "write-ahead log" mode and set a long "busy timeout".
This should avoid "database is locked" errors when there's a lot of
concurrency, for instance when offloading simultaneously a lot of
builds.

* guix/store/database.scm (call-with-database): Add two 'sqlite-exec'
calls to set 'journal_mode' and 'busy_timeout'.
2018-12-21 23:50:13 +01:00
Ludovic Courtès b96e05aefd
offload: Recognize build failures due to lack of disk space.
Previously, if a remote build would fail due to lack of disk space, this
would be considered a permanent failure and thus cached as a build
failure if the local daemon runs with '--cache-failures'.

* guix/scripts/offload.scm (transfer-and-offload): Upon
'nix-protocol-error?' call 'node-free-disk-space' and return 1 instead
of 100 if the result if lower than 10 MiB.
2018-12-21 23:50:13 +01:00
Ludovic Courtès 63b0c3eacc
offload: Skip machines that are low on disk space.
Fixes <https://bugs.gnu.org/33378>.

* guix/scripts/offload.scm (node-free-disk-space): New procedure.
(%minimum-disk-space): New variable.
(choose-build-machine): Call 'node-free-disk-space' and take it into
account in addition to LOAD.
(check-machine-status): Display the free disk space.
2018-12-21 23:50:13 +01:00
Ludovic Courtès bbe66a530a
offload: Decompose 'machine-load' into simpler procedures.
* guix/scripts/offload.scm (machine-load): Remove.
(node-load, normalized-load): New procedures.
(choose-build-machine): Call 'open-ssh-session' and 'make-node' from
here; pass the node to 'node-load'.
(check-machine-status): Use 'node-load' instead of 'machine-load'.  Call
'disconnect!' on SESSION.
2018-12-21 23:50:13 +01:00
Ricardo Wurmus 10a1cacb16
import: cran: Try import via CRAN if package is not on Bioconductor.
* guix/import/cran.scm (fetch-description): Return #F on failure.
(cran->guix-package): Retry from CRAN on failure to fetch description from
bioconductor.
2018-12-20 09:42:22 +01:00
Ludovic Courtès 40cc850aeb
download: 'built-in-builders*' relies on the functional cache.
The previous caching strategy, which used STORE as an 'eq?' key, would
no longer work when the functional cache is used because subsequent
store values are not 'eq?'.

* guix/download.scm (built-in-builders*): Rewrite in terms of
'mcached'.
2018-12-19 23:52:25 +01:00
Ludovic Courtès 87b711d200
utils: Memoize 'absolute-dirname'.
* guix/utils.scm (absolute-dirname): Wrap in 'mlambda'.
2018-12-19 23:52:25 +01:00
Ludovic Courtès 0b1be8fd57
packages: Turn 'cache!' into a single-value-return cache.
* guix/packages.scm (cache!): Assume THUNK returns a single value.
(cached): Likewise.
2018-12-19 23:52:24 +01:00
Ludovic Courtès b334674fe5
Use 'mapm' instead of 'sequence' + 'map'.
Previously we'd use the (sequence M (map P L)) idiom just because 'mapm'
was slower (not specialized for the given monad).  This is no longer the
case since commit dcb95c1fc9.

* guix/gexp.scm (lower-inputs): Use (mapm M P L) instead of (sequence
M (map P L)).
(lower-references, gexp->sexp, imported-files): Likewise.
* guix/profiles.scm (profile-derivation): Likewise.
* guix/scripts/environment.scm (inputs->requisites): Likewise.
* guix/scripts/system.scm (copy-closure): Likewise.
2018-12-19 23:52:24 +01:00
Ludovic Courtès 73b0ebdd5e
store: Add 'GUIX_PROFILING' support for the object cache.
* guix/store.scm (profiled?): New procedure.
(record-operation): Use it.
(record-cache-lookup!): New procedure.
(lookup-cached-object): Use it.
2018-12-19 23:52:24 +01:00
Ricardo Wurmus 80eebee9f7
ui: Report profile hooks separately.
* guix/ui.scm (profile-hook-derivation?): New procedure.
(show-what-to-build): Distinguish among BUILD derivations that match
'profile-hook-derivation?'.  Report them separately.
* guix/status.scm (hook-message): New procedure.
(print-build-event): Display profile hooks with readable hook name.
* guix/profiles.scm (info-dir-file, ghc-package-cache-file,
ca-certificate-bundle, glib-schemas, gtk-icon-themes, gtk-im-modules,
xdg-desktop-database, xdg-mime-database, fonts-dir-file, manual-database):
Augment derivation with "type" and "hook" properties.
2018-12-19 23:46:38 +01:00
Julien Lepiller 6e8986c838
build: Add dune-build-system.
* guix/build/dune-build-system.scm,
  guix/build-system/dune.scm: New files.
* Makefile.am (MODULES): Add them.
* doc/guix.texi (Build Systems): Document dune-build-system.
* guix/build-system/ocaml.scm (lower, default-findlib, default-ocaml): Export
them.
(package-with-explicit-ocaml): Also transform packages built with
dune-build-system.
2018-12-18 22:16:23 +01:00
swedebugia bd5a81f9b8
guix system: Clarify the three strategies available.
* guix/scripts/system.scm (show-help): Clarify the three choices.
* doc/guix.texi (Invoking guix system): Add 3 @cindex to make on-error=strategy
easier to find.

Signed-off-by: Ludovic Courtès <ludo@gnu.org>
2018-12-18 16:04:25 +01:00
Timothy Sample 6616a35202
ui: Report file names in 'system-error' exceptions from 'execlp'.
Fixes <https://bugs.gnu.org/33755>.

* guix/ui.scm (apply-formals): New macro.
(execlp): New error-reporting wrapper.

Signed-off-by: Ludovic Courtès <ludo@gnu.org>
2018-12-18 16:04:25 +01:00
Ludovic Courtès 9b9de08477
publish: Add a 'Cache-Control' header on /nar responses.
Fixes <https://bugs.gnu.org/33721>.
Reported by Chris Marusich <cmmarusich@gmail.com>.

* guix/scripts/publish.scm (render-nar/cached): Add #:ttl and honor it.
(make-request-handler): Pass #:ttl to 'render-nar/cached'.
* tests/publish.scm ("with cache, uncompressed"): Pass "--ttl=42h" to
'guix publish'.  Check 'Cache-Control' on narinfo response and on nar
response.
2018-12-17 23:33:42 +01:00
Ludovic Courtès a93c160631
environment: Support package transformation options.
Fixes <https://bugs.gnu.org/33776>.
Reported by Adrien Guilbaud <adrien.guilbaud@inria.fr>.

* guix/scripts/environment.scm (show-help): Add call to
'show-transformation-options-help'.
(%options): Add %TRANSFORMATION-OPTIONS.
(options/resolve-packages): Add 'store' parameter.
[transform, package->manifest-entry*]: New procedures.
Use 'package->manifest-entry*' instead of 'package->manifest-entry'.
(guix-environment): Move definition of 'manifest' within 'with-store'.
* tests/guix-environment.sh: Add test.
2018-12-17 23:33:42 +01:00
Julien Lepiller cce654fabd
import: Update opam importer.
* guix/import/opam.scm: Update importer for opam 2.
* tests/opam.scm: Update tests for the opam 2 importer.
2018-12-17 21:43:51 +01:00
Ludovic Courtès adb158b739
deduplication: Gracefully handle ENOSPC raised by 'link' calls.
Reported by Andreas Enge <andreas@enge.fr>
in <https://bugs.gnu.org/33676>.

* guix/store/deduplication.scm (replace-with-link): Catch ENOSPC around
'get-temp-link'.  Do nothing when 'get-temp-link' throws ENOSPC.  Move
code to restore PARENT's permissions outside of 'catch'.
* tests/store-deduplication.scm ("deduplicate, ENOSPC"): New test.
2018-12-14 12:07:24 +01:00
Ludovic Courtès 60b04024f8
substitute: Ignore irrelevant narinfo signatures.
Fixes <https://bugs.gnu.org/33733>.

Fixes a bug whereby 'guix substitute' would accept narinfos whose
signature does not cover the StorePath/NarHash/References tuple.

* guix/scripts/substitute.scm (narinfo-sha256)[%mandatory-fields]: New
variable.
Compute SIGNED-FIELDS; return #f unless each of the %MANDATORY-FIELDS
is among SIGNED-FIELDS.
 * tests/substitute.scm ("query narinfo with signature over nothing")
("query narinfo with signature over irrelevant bits"): New tests.
2018-12-14 00:02:59 +01:00
Eric Bavier cba7ddcf60
refresh: Account for overlapping updater coverage.
* guix/scripts/refresh.scm (list-updaters-and-exit): Do not assume updater
predicates are disjoint.  Track covered packages directly.
2018-12-10 19:59:01 -06:00
Ricardo Wurmus af12790bdd
guix: Add support for channel dependencies.
* guix/channels.scm (<channel-metadata>): New record.
(read-channel-metadata, channel-instance-dependencies): New procedures.
(latest-channel-instances): Include channel dependencies; add optional
argument PREVIOUS-CHANNELS.
(channel-instance-derivations): Build derivation for additional channels and
add it as dependency to the channel instance derivation.
* doc/guix.texi (Channels): Add subsection "Declaring Channel Dependencies".
* tests/channels.scm: New file.
* Makefile.am (SCM_TESTS): Add it.
2018-12-09 13:55:22 +01:00
Ludovic Courtès 6ddc63e599
guix system: Fix bootloader config file generation by 'guix system roll-back'.
Fixes <https://bugs.gnu.org/33623>.
Reported by Diego Nicola Barbato <dnbarbato@posteo.de>.

Regression introduced in commit
46c296dcc4.

* guix/scripts/system.scm (reinstall-bootloader): Add call to
'lower-object'.
2018-12-07 15:14:12 +01:00
Ludovic Courtès ed9c8eb4e5
describe: In 'channels' format, quote the channel name.
Fixes a regression introduced in
commit 8548f99549.

* guix/scripts/describe.scm (channel->sexp): Quote the channel name.
2018-12-07 15:14:12 +01:00
Ludovic Courtès 7bc5657f84
grafts: Mark as non substitutable.
* guix/grafts.scm (graft-derivation/shallow): Pass #:substitutable? to
'build-expression->derivation'.
2018-12-04 10:57:56 +01:00
Ludovic Courtès 9300e2e893
refresh: '--list-dependents' ignores deprecated packages.
* guix/scripts/refresh.scm (all-packages): Filter out deprecated packages.
2018-12-04 10:57:56 +01:00
Ludovic Courtès 0a5fa004f7
build: Default to https://ci.guix.info for substitutes.
* config-daemon.ac (guix_substitute_urls): Always default to
"https://ci.guix.info".
* doc/guix.texi (SUBSTITUTE-SERVER): Switch to ci.guix.info.
* guix/scripts/build.scm (%default-log-urls): Likewise.
* guix/scripts/substitute.scm (%default-substitute-urls): Likewise.
* guix/store.scm (%default-substitute-urls): Likewise.
2018-12-04 10:57:56 +01:00
Ludovic Courtès 0bc02becca
Remove most references to hydra.gnu.org.
* Makefile.am (assert-binaries-available): Don't mention hydra.gnu.org
in comment.
* build-aux/check-available-binaries.scm: Likewise.
* build-aux/check-final-inputs-self-contained.scm: Likewise.
* doc/guix.texi (SUBSTITUTE-SERVER): New variable.
Use it throughout instead of "mirror.hydra.gnu.org".
* doc/contributing.texi (Submitting Patches): Likewise.
* gnu/services/base.scm (hydra-key-authorization)
(guix-activation): Remove mentions of "hydra.gnu.org" in comments and
messages.
* gnu/system/install.scm (%installation-services): Likewise.
* guix/scripts/size.scm (guix-size): Likewise.
2018-12-04 10:57:56 +01:00
Ludovic Courtès 6a837b6062
etc: Add "ci.guix.info.pub" public key file.
* etc/substitutes/ci.guix.info.pub: New file.
* Makefile.am (dist_pkgdata_DATA): Add it.
* guix/self.scm (miscellaneous-files): Add
"share/guix/ci.guix.info.pub".
2018-12-04 10:57:56 +01:00
Björn Höfling b8ca5b3146
hg-fetch: Remove .hg directories of sub-repositories.
* guix/build/hg.scm (hg-fetch): Remove all .hg directories recursively.
2018-12-03 12:29:08 +01:00
Ludovic Courtès 194451347d
Merge branch 'core-updates' 2018-12-03 08:52:17 +01:00
Oleg Pykhalov 37c6f11f8d
repl: Load user's initialization file.
* guix/scripts/repl.scm (guix-repl): Load user's initialization file.
2018-12-03 08:04:51 +03:00
Ludovic Courtès 2d33776050
self: Provide UTF-8 locales when building the manual.
Fixes <https://bugs.gnu.org/33580>.
Reported by <znavko@tutanota.com>.

* guix/self.scm (info-manual)[glibc-utf8-locales]: New variable.
[build]: Add call to 'setenv' for "GUIX_LOCPATH".
2018-12-02 22:53:22 +01:00
Ludovic Courtès a3d77c51bc
git: Nicely report '--with-commit' errors.
* guix/git.scm (latest-repository-commit*): Rewrite to catch
'git-error'.
* po/guix/POTFILES.in: Add guix/git.scm.
* tests/guix-build-branch.sh: Test --with-commit errors.
2018-11-30 17:03:04 +01:00
Ludovic Courtès b18f7234aa
guix build: Add '--with-commit'.
* guix/git.scm (<git-checkout>)[commit]: New field.
(git-checkout-compiler): Honor it.
* guix/scripts/build.scm (evaluate-git-replacement-specs): Add 'proc'
parameter and honor it.
(transform-package-source-branch)[replace]: New procedure.
Adjust 'evaluate-git-replacement-specs' accordingly.
(transform-package-source-commit): New procedure.
(%transformations, %transformation-options)
(show-transformation-options-help): Add 'with-commit'.
* tests/guix-build-branch.sh: Add test.
* doc/guix.texi (Package Transformation Options): Document it.
2018-11-30 17:03:04 +01:00
Ludovic Courtès 96915a448c
guix build: Add '--with-branch' transformation option.
* guix/scripts/build.scm (evaluate-git-replacement-specs)
(transform-package-source-branch): New procedures.
(%transformations, %transformation-options): Add 'with-branch'.
(show-transformation-options-help): Likewise.
* tests/guix-build-branch.sh: New file.
* Makefile.am (SH_TESTS): Add it.
* doc/guix.texi (Package Transformation Options): Document it.
2018-11-30 17:03:04 +01:00
Ludovic Courtès 49ae3f6d89
git: Add <git-checkout> record type.
* guix/git.scm (<git-checkout>): New record type.
(latest-repository-commit*): New procedure.
(git-checkout-compiler): New gexp compiler.
2018-11-30 17:03:04 +01:00
Ludovic Courtès 35cb37ea9c
git: 'latest-repository-commit' logs its progress.
* guix/git.scm (latest-repository-commit): Add #:log-port and honor it.
2018-11-30 17:03:03 +01:00
Marius Bakke 25ca46985c
Merge branch 'master' into core-updates 2018-11-29 15:56:49 +01:00
Ludovic Courtès d09ce3f91b
docker: Hide scary-looking but harmless tar failure messages.
* guix/docker.scm (build-docker-image): Wrap "tar --delete" invocation
in 'with-error-to-port'.
2018-11-29 09:30:00 +01:00
Ludovic Courtès 13bcc6b45f
download: Add berlin.guixsd.org as a content-addressed mirror.
* guix/download.scm (%content-addressed-mirrors)[guix-publish]: New
procedure.
Use it for "mirror.hydra.gnu.org" and add "berlin.guixsd.org" too.
2018-11-28 22:25:34 +01:00
Ludovic Courtès a52ae1b662
download: Make (guix base16) a soft dependency.
Fixes <https://bugs.gnu.org/33542>.
Reported by Mark H Weaver <mhw@netris.org>.

* guix/download.scm (%content-addressed-mirrors): Autoload (guix base16).
2018-11-28 22:25:34 +01:00
Tim Gesthuizen e91152e9f2
guix hash: Fix version and help messages
ca71942445 changes the name of the executable to
be displayed by the --version and --help commands of `guix hash` to
"gcrypt hash".
This is reverted by this commit.

* guix/scripts/hash.scm (show-help): Change string literals

Signed-off-by: Ludovic Courtès <ludo@gnu.org>
2018-11-28 10:39:58 +01:00
Ludovic Courtès fe5db4eb03
channels: Add 'checkout->channel-instance'.
* guix/channels.scm (checkout->channel-instance): New procedure.
2018-11-28 10:39:58 +01:00
Ludovic Courtès 94c0e61fe7
inferior: Add 'inferior-eval-with-store'.
* guix/inferior.scm (inferior-eval-with-store): New procedure, with code
formerly in 'inferior-package-derivation'.
(inferior-package-derivation): Rewrite in terms of
'inferior-eval-with-store'.
* tests/inferior.scm ("inferior-eval-with-store"): New test.
2018-11-28 10:39:58 +01:00