Commit Graph

3792 Commits

Author SHA1 Message Date
Ludovic Courtès bdf860c2e9
database: Use "write-ahead log" mode and set a long "busy timeout".
This should avoid "database is locked" errors when there's a lot of
concurrency, for instance when offloading simultaneously a lot of
builds.

* guix/store/database.scm (call-with-database): Add two 'sqlite-exec'
calls to set 'journal_mode' and 'busy_timeout'.
2018-12-21 23:50:13 +01:00
Ludovic Courtès b96e05aefd
offload: Recognize build failures due to lack of disk space.
Previously, if a remote build would fail due to lack of disk space, this
would be considered a permanent failure and thus cached as a build
failure if the local daemon runs with '--cache-failures'.

* guix/scripts/offload.scm (transfer-and-offload): Upon
'nix-protocol-error?' call 'node-free-disk-space' and return 1 instead
of 100 if the result if lower than 10 MiB.
2018-12-21 23:50:13 +01:00
Ludovic Courtès 63b0c3eacc
offload: Skip machines that are low on disk space.
Fixes <https://bugs.gnu.org/33378>.

* guix/scripts/offload.scm (node-free-disk-space): New procedure.
(%minimum-disk-space): New variable.
(choose-build-machine): Call 'node-free-disk-space' and take it into
account in addition to LOAD.
(check-machine-status): Display the free disk space.
2018-12-21 23:50:13 +01:00
Ludovic Courtès bbe66a530a
offload: Decompose 'machine-load' into simpler procedures.
* guix/scripts/offload.scm (machine-load): Remove.
(node-load, normalized-load): New procedures.
(choose-build-machine): Call 'open-ssh-session' and 'make-node' from
here; pass the node to 'node-load'.
(check-machine-status): Use 'node-load' instead of 'machine-load'.  Call
'disconnect!' on SESSION.
2018-12-21 23:50:13 +01:00
Ricardo Wurmus 10a1cacb16
import: cran: Try import via CRAN if package is not on Bioconductor.
* guix/import/cran.scm (fetch-description): Return #F on failure.
(cran->guix-package): Retry from CRAN on failure to fetch description from
bioconductor.
2018-12-20 09:42:22 +01:00
Ludovic Courtès 40cc850aeb
download: 'built-in-builders*' relies on the functional cache.
The previous caching strategy, which used STORE as an 'eq?' key, would
no longer work when the functional cache is used because subsequent
store values are not 'eq?'.

* guix/download.scm (built-in-builders*): Rewrite in terms of
'mcached'.
2018-12-19 23:52:25 +01:00
Ludovic Courtès 87b711d200
utils: Memoize 'absolute-dirname'.
* guix/utils.scm (absolute-dirname): Wrap in 'mlambda'.
2018-12-19 23:52:25 +01:00
Ludovic Courtès 0b1be8fd57
packages: Turn 'cache!' into a single-value-return cache.
* guix/packages.scm (cache!): Assume THUNK returns a single value.
(cached): Likewise.
2018-12-19 23:52:24 +01:00
Ludovic Courtès b334674fe5
Use 'mapm' instead of 'sequence' + 'map'.
Previously we'd use the (sequence M (map P L)) idiom just because 'mapm'
was slower (not specialized for the given monad).  This is no longer the
case since commit dcb95c1fc9.

* guix/gexp.scm (lower-inputs): Use (mapm M P L) instead of (sequence
M (map P L)).
(lower-references, gexp->sexp, imported-files): Likewise.
* guix/profiles.scm (profile-derivation): Likewise.
* guix/scripts/environment.scm (inputs->requisites): Likewise.
* guix/scripts/system.scm (copy-closure): Likewise.
2018-12-19 23:52:24 +01:00
Ludovic Courtès 73b0ebdd5e
store: Add 'GUIX_PROFILING' support for the object cache.
* guix/store.scm (profiled?): New procedure.
(record-operation): Use it.
(record-cache-lookup!): New procedure.
(lookup-cached-object): Use it.
2018-12-19 23:52:24 +01:00
Ricardo Wurmus 80eebee9f7
ui: Report profile hooks separately.
* guix/ui.scm (profile-hook-derivation?): New procedure.
(show-what-to-build): Distinguish among BUILD derivations that match
'profile-hook-derivation?'.  Report them separately.
* guix/status.scm (hook-message): New procedure.
(print-build-event): Display profile hooks with readable hook name.
* guix/profiles.scm (info-dir-file, ghc-package-cache-file,
ca-certificate-bundle, glib-schemas, gtk-icon-themes, gtk-im-modules,
xdg-desktop-database, xdg-mime-database, fonts-dir-file, manual-database):
Augment derivation with "type" and "hook" properties.
2018-12-19 23:46:38 +01:00
Julien Lepiller 6e8986c838
build: Add dune-build-system.
* guix/build/dune-build-system.scm,
  guix/build-system/dune.scm: New files.
* Makefile.am (MODULES): Add them.
* doc/guix.texi (Build Systems): Document dune-build-system.
* guix/build-system/ocaml.scm (lower, default-findlib, default-ocaml): Export
them.
(package-with-explicit-ocaml): Also transform packages built with
dune-build-system.
2018-12-18 22:16:23 +01:00
swedebugia bd5a81f9b8
guix system: Clarify the three strategies available.
* guix/scripts/system.scm (show-help): Clarify the three choices.
* doc/guix.texi (Invoking guix system): Add 3 @cindex to make on-error=strategy
easier to find.

Signed-off-by: Ludovic Courtès <ludo@gnu.org>
2018-12-18 16:04:25 +01:00
Timothy Sample 6616a35202
ui: Report file names in 'system-error' exceptions from 'execlp'.
Fixes <https://bugs.gnu.org/33755>.

* guix/ui.scm (apply-formals): New macro.
(execlp): New error-reporting wrapper.

Signed-off-by: Ludovic Courtès <ludo@gnu.org>
2018-12-18 16:04:25 +01:00
Ludovic Courtès 9b9de08477
publish: Add a 'Cache-Control' header on /nar responses.
Fixes <https://bugs.gnu.org/33721>.
Reported by Chris Marusich <cmmarusich@gmail.com>.

* guix/scripts/publish.scm (render-nar/cached): Add #:ttl and honor it.
(make-request-handler): Pass #:ttl to 'render-nar/cached'.
* tests/publish.scm ("with cache, uncompressed"): Pass "--ttl=42h" to
'guix publish'.  Check 'Cache-Control' on narinfo response and on nar
response.
2018-12-17 23:33:42 +01:00
Ludovic Courtès a93c160631
environment: Support package transformation options.
Fixes <https://bugs.gnu.org/33776>.
Reported by Adrien Guilbaud <adrien.guilbaud@inria.fr>.

* guix/scripts/environment.scm (show-help): Add call to
'show-transformation-options-help'.
(%options): Add %TRANSFORMATION-OPTIONS.
(options/resolve-packages): Add 'store' parameter.
[transform, package->manifest-entry*]: New procedures.
Use 'package->manifest-entry*' instead of 'package->manifest-entry'.
(guix-environment): Move definition of 'manifest' within 'with-store'.
* tests/guix-environment.sh: Add test.
2018-12-17 23:33:42 +01:00
Julien Lepiller cce654fabd
import: Update opam importer.
* guix/import/opam.scm: Update importer for opam 2.
* tests/opam.scm: Update tests for the opam 2 importer.
2018-12-17 21:43:51 +01:00
Ludovic Courtès adb158b739
deduplication: Gracefully handle ENOSPC raised by 'link' calls.
Reported by Andreas Enge <andreas@enge.fr>
in <https://bugs.gnu.org/33676>.

* guix/store/deduplication.scm (replace-with-link): Catch ENOSPC around
'get-temp-link'.  Do nothing when 'get-temp-link' throws ENOSPC.  Move
code to restore PARENT's permissions outside of 'catch'.
* tests/store-deduplication.scm ("deduplicate, ENOSPC"): New test.
2018-12-14 12:07:24 +01:00
Ludovic Courtès 60b04024f8
substitute: Ignore irrelevant narinfo signatures.
Fixes <https://bugs.gnu.org/33733>.

Fixes a bug whereby 'guix substitute' would accept narinfos whose
signature does not cover the StorePath/NarHash/References tuple.

* guix/scripts/substitute.scm (narinfo-sha256)[%mandatory-fields]: New
variable.
Compute SIGNED-FIELDS; return #f unless each of the %MANDATORY-FIELDS
is among SIGNED-FIELDS.
 * tests/substitute.scm ("query narinfo with signature over nothing")
("query narinfo with signature over irrelevant bits"): New tests.
2018-12-14 00:02:59 +01:00
Eric Bavier cba7ddcf60
refresh: Account for overlapping updater coverage.
* guix/scripts/refresh.scm (list-updaters-and-exit): Do not assume updater
predicates are disjoint.  Track covered packages directly.
2018-12-10 19:59:01 -06:00
Ricardo Wurmus af12790bdd
guix: Add support for channel dependencies.
* guix/channels.scm (<channel-metadata>): New record.
(read-channel-metadata, channel-instance-dependencies): New procedures.
(latest-channel-instances): Include channel dependencies; add optional
argument PREVIOUS-CHANNELS.
(channel-instance-derivations): Build derivation for additional channels and
add it as dependency to the channel instance derivation.
* doc/guix.texi (Channels): Add subsection "Declaring Channel Dependencies".
* tests/channels.scm: New file.
* Makefile.am (SCM_TESTS): Add it.
2018-12-09 13:55:22 +01:00
Ludovic Courtès 6ddc63e599
guix system: Fix bootloader config file generation by 'guix system roll-back'.
Fixes <https://bugs.gnu.org/33623>.
Reported by Diego Nicola Barbato <dnbarbato@posteo.de>.

Regression introduced in commit
46c296dcc4.

* guix/scripts/system.scm (reinstall-bootloader): Add call to
'lower-object'.
2018-12-07 15:14:12 +01:00
Ludovic Courtès ed9c8eb4e5
describe: In 'channels' format, quote the channel name.
Fixes a regression introduced in
commit 8548f99549.

* guix/scripts/describe.scm (channel->sexp): Quote the channel name.
2018-12-07 15:14:12 +01:00
Ludovic Courtès 7bc5657f84
grafts: Mark as non substitutable.
* guix/grafts.scm (graft-derivation/shallow): Pass #:substitutable? to
'build-expression->derivation'.
2018-12-04 10:57:56 +01:00
Ludovic Courtès 9300e2e893
refresh: '--list-dependents' ignores deprecated packages.
* guix/scripts/refresh.scm (all-packages): Filter out deprecated packages.
2018-12-04 10:57:56 +01:00
Ludovic Courtès 0a5fa004f7
build: Default to https://ci.guix.info for substitutes.
* config-daemon.ac (guix_substitute_urls): Always default to
"https://ci.guix.info".
* doc/guix.texi (SUBSTITUTE-SERVER): Switch to ci.guix.info.
* guix/scripts/build.scm (%default-log-urls): Likewise.
* guix/scripts/substitute.scm (%default-substitute-urls): Likewise.
* guix/store.scm (%default-substitute-urls): Likewise.
2018-12-04 10:57:56 +01:00
Ludovic Courtès 0bc02becca
Remove most references to hydra.gnu.org.
* Makefile.am (assert-binaries-available): Don't mention hydra.gnu.org
in comment.
* build-aux/check-available-binaries.scm: Likewise.
* build-aux/check-final-inputs-self-contained.scm: Likewise.
* doc/guix.texi (SUBSTITUTE-SERVER): New variable.
Use it throughout instead of "mirror.hydra.gnu.org".
* doc/contributing.texi (Submitting Patches): Likewise.
* gnu/services/base.scm (hydra-key-authorization)
(guix-activation): Remove mentions of "hydra.gnu.org" in comments and
messages.
* gnu/system/install.scm (%installation-services): Likewise.
* guix/scripts/size.scm (guix-size): Likewise.
2018-12-04 10:57:56 +01:00
Ludovic Courtès 6a837b6062
etc: Add "ci.guix.info.pub" public key file.
* etc/substitutes/ci.guix.info.pub: New file.
* Makefile.am (dist_pkgdata_DATA): Add it.
* guix/self.scm (miscellaneous-files): Add
"share/guix/ci.guix.info.pub".
2018-12-04 10:57:56 +01:00
Björn Höfling b8ca5b3146
hg-fetch: Remove .hg directories of sub-repositories.
* guix/build/hg.scm (hg-fetch): Remove all .hg directories recursively.
2018-12-03 12:29:08 +01:00
Ludovic Courtès 194451347d
Merge branch 'core-updates' 2018-12-03 08:52:17 +01:00
Oleg Pykhalov 37c6f11f8d
repl: Load user's initialization file.
* guix/scripts/repl.scm (guix-repl): Load user's initialization file.
2018-12-03 08:04:51 +03:00
Ludovic Courtès 2d33776050
self: Provide UTF-8 locales when building the manual.
Fixes <https://bugs.gnu.org/33580>.
Reported by <znavko@tutanota.com>.

* guix/self.scm (info-manual)[glibc-utf8-locales]: New variable.
[build]: Add call to 'setenv' for "GUIX_LOCPATH".
2018-12-02 22:53:22 +01:00
Ludovic Courtès a3d77c51bc
git: Nicely report '--with-commit' errors.
* guix/git.scm (latest-repository-commit*): Rewrite to catch
'git-error'.
* po/guix/POTFILES.in: Add guix/git.scm.
* tests/guix-build-branch.sh: Test --with-commit errors.
2018-11-30 17:03:04 +01:00
Ludovic Courtès b18f7234aa
guix build: Add '--with-commit'.
* guix/git.scm (<git-checkout>)[commit]: New field.
(git-checkout-compiler): Honor it.
* guix/scripts/build.scm (evaluate-git-replacement-specs): Add 'proc'
parameter and honor it.
(transform-package-source-branch)[replace]: New procedure.
Adjust 'evaluate-git-replacement-specs' accordingly.
(transform-package-source-commit): New procedure.
(%transformations, %transformation-options)
(show-transformation-options-help): Add 'with-commit'.
* tests/guix-build-branch.sh: Add test.
* doc/guix.texi (Package Transformation Options): Document it.
2018-11-30 17:03:04 +01:00
Ludovic Courtès 96915a448c
guix build: Add '--with-branch' transformation option.
* guix/scripts/build.scm (evaluate-git-replacement-specs)
(transform-package-source-branch): New procedures.
(%transformations, %transformation-options): Add 'with-branch'.
(show-transformation-options-help): Likewise.
* tests/guix-build-branch.sh: New file.
* Makefile.am (SH_TESTS): Add it.
* doc/guix.texi (Package Transformation Options): Document it.
2018-11-30 17:03:04 +01:00
Ludovic Courtès 49ae3f6d89
git: Add <git-checkout> record type.
* guix/git.scm (<git-checkout>): New record type.
(latest-repository-commit*): New procedure.
(git-checkout-compiler): New gexp compiler.
2018-11-30 17:03:04 +01:00
Ludovic Courtès 35cb37ea9c
git: 'latest-repository-commit' logs its progress.
* guix/git.scm (latest-repository-commit): Add #:log-port and honor it.
2018-11-30 17:03:03 +01:00
Marius Bakke 25ca46985c
Merge branch 'master' into core-updates 2018-11-29 15:56:49 +01:00
Ludovic Courtès d09ce3f91b
docker: Hide scary-looking but harmless tar failure messages.
* guix/docker.scm (build-docker-image): Wrap "tar --delete" invocation
in 'with-error-to-port'.
2018-11-29 09:30:00 +01:00
Ludovic Courtès 13bcc6b45f
download: Add berlin.guixsd.org as a content-addressed mirror.
* guix/download.scm (%content-addressed-mirrors)[guix-publish]: New
procedure.
Use it for "mirror.hydra.gnu.org" and add "berlin.guixsd.org" too.
2018-11-28 22:25:34 +01:00
Ludovic Courtès a52ae1b662
download: Make (guix base16) a soft dependency.
Fixes <https://bugs.gnu.org/33542>.
Reported by Mark H Weaver <mhw@netris.org>.

* guix/download.scm (%content-addressed-mirrors): Autoload (guix base16).
2018-11-28 22:25:34 +01:00
Tim Gesthuizen e91152e9f2
guix hash: Fix version and help messages
ca71942445 changes the name of the executable to
be displayed by the --version and --help commands of `guix hash` to
"gcrypt hash".
This is reverted by this commit.

* guix/scripts/hash.scm (show-help): Change string literals

Signed-off-by: Ludovic Courtès <ludo@gnu.org>
2018-11-28 10:39:58 +01:00
Ludovic Courtès fe5db4eb03
channels: Add 'checkout->channel-instance'.
* guix/channels.scm (checkout->channel-instance): New procedure.
2018-11-28 10:39:58 +01:00
Ludovic Courtès 94c0e61fe7
inferior: Add 'inferior-eval-with-store'.
* guix/inferior.scm (inferior-eval-with-store): New procedure, with code
formerly in 'inferior-package-derivation'.
(inferior-package-derivation): Rewrite in terms of
'inferior-eval-with-store'.
* tests/inferior.scm ("inferior-eval-with-store"): New test.
2018-11-28 10:39:58 +01:00
Ludovic Courtès d4aa147eec
ui: 'show-what-to-build' reports grafts separately.
* guix/ui.scm (graft-derivation?): New procedure.
(show-what-to-build): Distinguish among BUILD derivations that match
'graft-derivation?'.  Report them separately.
2018-11-28 10:39:58 +01:00
Ludovic Courtès af1f1c38fb
status: Report grafting derivations specially.
* guix/status.scm (print-build-event): In 'build-started' event handler,
check the properties of DRV and handle 'graft' derivations specially.
2018-11-28 10:39:58 +01:00
Ludovic Courtès 64fd1c01bc
grafts: Record metadata as derivation properties.
* guix/grafts.scm (graft-derivation/shallow): Pass #:properties to
'build-expression->derivation'.
* tests/grafts.scm ("graft-derivation, grafted item is a direct
dependency"): Check the value returned by 'derivation-properties'.
2018-11-28 10:39:58 +01:00
Ludovic Courtès 8856f409d1
derivations: Add properties.
* guix/derivations.scm (derivation): Add #:properties parameter.
[user+system-env-vars]: Honor it.
(derivation-properties): New procedure.
(build-expression->derivation): Add #:properties and pass it to
'derivation'.
* guix/gexp.scm (gexp->derivation): Likewise.
* tests/derivations.scm ("derivation-properties"): New test.
* tests/gexp.scm ("gexp->derivation properties"): New test.
* doc/guix.texi (Derivations, G-Expressions): Adjust accordingly.
2018-11-28 10:39:58 +01:00
Efraim Flashner db08ea4087
pack: List the available formats.
* guix/scripts/pack.scm (show-formats): New variable.
(%options, show-help): Add 'list-formats' option.
2018-11-27 19:23:39 +02:00
Oleg Pykhalov 01262f1ece
describe: Delete 'directory' argument from 'display-checkout-info'.
This commit follows 1255400faa.

* guix/scripts/describe.scm (display-checkout-info): Delete 'directory'
argument.
2018-11-27 20:09:03 +03:00