Commit Graph

3835 Commits

Author SHA1 Message Date
Ludovic Courtès 7489207ff7
refresh: Turn on warnings when '--manifest' is used.
* guix/scripts/refresh.scm (guix-refresh): Set WARN? when '-m' is used.
2019-01-11 12:14:38 +01:00
Ludovic Courtès fca43e14f7
refresh: Refactor option handling and '--recursive'.
This allows us to combine '--recursive' with other options (-u, -m,
etc.), turns off warnings when '--recursive' is used, and avoids the
hazards of I/O in the presence of multithreading.

* guix/scripts/refresh.scm (options->packages): New procedure, with code
formerly in 'guix-refresh'.
(refresh-recursive): Remove.
(guix-refresh)[keep-newest, core-package?, args-packages, packages]:
Remove.
[warn?]: Set to #f when RECURSIVE? is true.
Call 'options->packages' in monadic context.
2019-01-11 12:14:38 +01:00
Ricardo Wurmus bf6fb59a33
build-system: texlive: Do not hide build output.
* guix/build/texlive-build-system.scm (compile-with-latex): Use "nonstopmode"
instead of "batchmode".
2019-01-10 22:43:09 +01:00
Ricardo Wurmus ac5c9f6ba6
build-system: texlive: Do not truncate lines.
* guix/build/texlive-build-system.scm (configure): Set environment variables
to prevent build output lines from being truncated.
2019-01-10 22:43:09 +01:00
Ricardo Wurmus d4d9a1ece0
gnu: Update texlive packages.
* guix/build-system/texlive.scm (%texlive-tag, %texlive-revision): Update to
texlive-2018.2, revision 49435.
* gnu/packages/tex.scm (texlive-dvips, texlive-generic-unicode-data,
texlive-generic-dehyph-exptl, texlive-generic-hyph-utf8, texlive-fontname,
texlive-fonts-cm, texlive-tex-plain, texlive-latex-base,
texlive-latex-graphics, texlive-latex-graphics, texlive-latex-oberdiek,
texlive-latex-tools, texlive-latex-l3kernel, texlive-latex-l3packages,
texlive-latex-fontspec, texlive-latex-amsmath, texlive-latex-amscls,
texlive-latex-babel, texlive-latex-cyrillic, texlive-latex-eqparbox,
texlive-latex-ifplatform, texlive-latex-etoolbox, texlive-latex-galois,
texlive-latex-polyglossia, texlive-tex-texinfo, texlive-latex-changebar,
texlive-latex-fancyhdr, texlive-latex-overpic, texlive-latex-parskip,
texlive-metapost, texlive-latex-ucs, texlive-generic-pdftex,
texlive-latex-media9, texlive-latex-ocgx2, texlive-latex-koma-script,
texlive-generic-listofitems, texlive-bibtex, texlive-context-base): Update
hashes.
(texlive-latex-fontspec)[arguments]: Remove custom build target.
(texlive-latex-dinbrief)[arguments]: Add build phase "fix-encoding-error".
(texlive-latex-xkeyval): New variable.
(texlive-latex-pstool)[source]: Fetch from new location.
[build-system]: Use trivial-build-system.
[arguments]: Write simple builder.
[propagated-inputs]: Add texlive-latex-l3kernel, texlive-latex-tools, and
texlive-latex-xkeyval.
[synopsis]: Fix typo.
2019-01-10 22:43:09 +01:00
Ludovic Courtès 787da810a0
Add (guix deprecation).
* guix/deprecation.scm: New file.
* Makefile.am (MODULES): Add it.
* po/guix/POTFILES.in: Add it.
2019-01-10 14:59:22 +01:00
Arun Isaac 40fa21c22e
guix: lint: Warn only if GitHub URI is not same as the package URI.
* guix/scripts/lint.scm (check-github-url): Warn only if the GitHub URI
obtained after following redirects is not same as the original URI.
* tests/lint.scm ("github-url: already the correct github url"): New test.
2019-01-10 15:39:35 +05:30
Ludovic Courtès 804b9b18ac
build-system/asdf: 'package-with-build-system' accesses inputs lazily.
Fixes a bug whereby we would, at the top-level (while evaluation
lisp.scm package definitions), attempt to access package inputs.
Because of circular dependencies, this could lead to unbound variables
as reported by Arun Isaac <arunisaac@systemreboot.net> at
<https://debbugs.gnu.org/cgi/bugreport.cgi?bug=33801#56>.

* guix/build-system/asdf.scm (package-with-build-system)[transform]
(new-propagated-inputs): Turn into a procedure.
Adjust user accordingly.
2019-01-09 15:01:40 +01:00
Ludovic Courtès 6e54e488c6
self: Drop support for Guix < 0.15 on Guile 2.0.
* guix/self.scm (false-if-wrong-guile, package-for-guile): Remove.
(specification->package): Remove "guile2.0-" variants.
(compiled-guix): #:guile-for-build defaults to (default-guile).
Use 'specification->package' instead of 'package-for-guile'.
(guile-for-build): Remove.
(guix-derivation): Use 'default-guile' instead of 'guile-for-build'.
Check whether we're running on Guile 2.0 with PULL-VERSION < 1.
2019-01-09 14:47:53 +01:00
Ludovic Courtès 76832d3420
Remove most uses of the _IO*F constants.
These constants, for use with 'setvbuf', were deprecated in Guile 2.2
and disappeared in Guile 3.0.  Here we keep these constants in
build-side code where removing them is not feasible.

* guix/build/download-nar.scm (download-nar): Adjust 'setvbuf' calls to
the Guile 2.2+ API.
* guix/build/download.scm (open-socket-for-uri): Likewise.
(open-connection-for-uri, url-fetch): Likewise.
* guix/build/make-bootstrap.scm (make-stripped-libc): Likewise.
* guix/build/union.scm (setvbuf) [guile-2.0]: New conditional wrapper.
(union-build): Adjust to new API.
* guix/ftp-client.scm (ftp-open, ftp-list, ftp-retr): Likewise.
* guix/http-client.scm (http-fetch): Likewise.
* guix/inferior.scm (proxy): Likewise.
* guix/scripts/substitute.scm (fetch, http-multiple-get): Likewise.
* guix/self.scm (compiled-modules): Likewise.
* guix/ssh.scm (remote-daemon-channel, store-import-channel)
(store-export-channel): Likewise.
* guix/ui.scm (initialize-guix): Likewise.
* tests/publish.scm (http-get-port): Likewise.
* guix/store.scm (%newlines): Adjust comment.
2019-01-09 14:47:53 +01:00
Ludovic Courtès c3d9bca48a
download: Remove Guile 2.0 workaround.
* guix/build/download.scm: Remove Guile 2.0 workaround.
2019-01-09 14:47:53 +01:00
Ludovic Courtès a65177a657
maint: Remove 'cond-expand' forms for Guile 2.0.
Note: Leave 'cond-expand' forms used in the build-side modules that can
run on %BOOTSTRAP-GUILE, which is currently Guile 2.0.

* guix/build/compile.scm: Move 'use-modules' clause from 'cond-expand'
to 'define-module' form.
(%default-optimizations): Remove 'cond-expand'.
* guix/build/download.scm (tls-wrap): Remove 'cond-expand'.
* guix/build/syscalls.scm: Remove 'cond-expand' form around
'%set-automatic-finalization-enabled?!' and
'without-automatic-finalization'.
* guix/inferior.scm (port->inferior): Remove 'cond-expand'.
* guix/scripts/pack.scm (wrapped-package)[build]: Remove 'cond-expand'.
* guix/status.scm (build-event-output-port): Remove 'cond-expand'.
* guix/store.scm (open-inet-socket): Remove 'cond-expand'.
* guix/ui.scm (install-locale): Remove 'cond-expand'.
* tests/status.scm ("current-build-output-port, UTF-8 + garbage"):
Remove 'cond-expand'.
* tests/store.scm ("current-build-output-port, UTF-8 + garbage"):
Remove 'cond-expand'.
2019-01-09 14:47:53 +01:00
Ludovic Courtès a21a906fcd
pull: Document '--system'.
Fixes <https://bugs.gnu.org/34010>.
Reported by Alex Kost <alezost@gmail.com>.

This is a followup to 5923102f7b.

* guix/scripts/pull.scm (show-help): Add '--system'.
2019-01-08 18:38:15 +01:00
Ludovic Courtès 08fdee3911
self: Compress Info files.
Fixes <https://bugs.gnu.org/33993>.
Reported by Adonay Felipe Nogueira <adfeno@hyperbola.info>.

* guix/self.scm (info-manual): Compress Info files.
2019-01-08 18:38:15 +01:00
Ludovic Courtès efff32452a
gexp: 'gexp->script' does not emit load-path expression when unnecessary.
This removes two elements from %LOAD-PATH and %LOAD-COMPILED-PATH of the
'guix' command and thus further reduces the number of 'stat' calls it
makes.

* guix/gexp.scm (load-path-expression): Return #f when MODULES and
EXTENSIONS are both empty.
(gexp->script): Don't emit anything when SET-LOAD-PATH is #f.
2019-01-07 23:50:56 +01:00
Ludovic Courtès 49c35bbb71
self: Move all modules into a single directory.
This halves the number of elements in %LOAD-PATH and %LOAD-COMPILED-PATH
and halves the number of 'stat' calls as reported by:

  env -i $(type -P guix) build -e '(@ (gnu packages base) coreutils)' -nd

* guix/self.scm (node-source+compiled, guile-module-union): New
procedures.
(guix-command): Remove 'compiled-modules' parameter.  Remove
'source-directories' and 'object-directories' variables and add
'module-directory'.  Change command so that it adds nothing but
MODULE-DIRECTORY to %LOAD-PATH and %LOAD-COMPILED-PATH.
(whole-package): Remove #:compiled-modules.  Assume MODULES contains
'share/guile/site' and 'lib/guile' and adjust code accordingly.
(compiled-guix): When PULL-VERSION is 1, use 'node-source+compiled'
only.  Remove #:compiled-modules argument to 'whole-package'.
* guix/channels.scm (whole-package-for-legacy): Add 'module+compiled'
and pass it to 'whole-package'.
2019-01-07 23:50:56 +01:00
Julien Lepiller f31ce9ecf1
import: opam: Parse comments.
* guix/import/opam.scm: Add comment support in parser.
2019-01-07 22:05:51 +01:00
Julien Lepiller 755e6d4a0a
import: opam: Add updater.
* guix/import/opam.scm (%opam-updater): New variable.
2019-01-07 22:05:45 +01:00
Julien Lepiller 6090b0beb0
import: opam: Add recursive option.
* guix/script/import/opam.scm: Add recursive option.
* guix/import/opam.scm (opam->guix-package): return two values.
(opam-recursive-import): New variable.
2019-01-07 22:05:43 +01:00
Ludovic Courtès 54ddb6a154
lint: Avoid 'dirname' call at the top level.
* guix/scripts/lint.scm (%distro-directory): Wrap in 'mlambda'.
(check-patch-file-names): Adjust accordingly.
2019-01-07 14:41:00 +01:00
Ludovic Courtès 46cf4cd676
Remove (guix build pull).
This module had been unused since commit
5f93d97005 (Oct. 2017).

* guix/build/pull.scm: Delete.
* Makefile.am (MODULES): Remove.
2019-01-07 14:40:59 +01:00
Ludovic Courtès b5f8c2c885
hydra: Compute jobs in an inferior.
Previously we would rely on auto-compilation of all the Guix modules.
The complete evaluation would take ~15mn on berlin.guixsd.org and
require lots of RAM.  This approach should be faster since potentially
only part of the modules are rebuilt.  Furthermore, as a side-effect, it
builds the derivations that 'guix pull' uses.

* build-aux/hydra/gnu-system.scm: Remove 'eval-when' form.
(hydra-jobs): New procedure.
* gnu/ci.scm (package->alist, qemu-jobs, system-test-jobs)
(tarball-jobs): Return strings for the 'license' field.
* guix/self.scm (compiled-guix)[*cli-modules*]: Add (gnu ci).
2019-01-06 21:41:07 +01:00
Ludovic Courtès 012bf5c4c0
lint: Rename checker to 'github-url'.
* guix/scripts/lint.scm (%checkers): Rename 'github-uri' to 'github-url'
to match the documentation.
2019-01-06 00:22:05 +01:00
Ludovic Courtès c070d1423f
git-download: Use 'invoke'.
* guix/build/git.scm (git-fetch): Use 'invoke' instead of 'system*' for
"git submodule update".
2019-01-06 00:22:05 +01:00
Ludovic Courtès 18524466bb
git-download: 'git-fetch' really returns #f upon error.
This allows the fallback code in (guix git-download) to actually run.
Regression introduced in commit 329dabe13b.

Fixes <https://bugs.gnu.org/33911>.
Reported by Björn Höfling <bjoern.hoefling@bjoernhoefling.de>.

* guix/build/git.scm (git-fetch): Guard against 'invoke-error?' and
really return #f upon failure.
2019-01-06 00:22:05 +01:00
Ludovic Courtès a173f09811
ui: It's 2019 now!
* guix/ui.scm (show-version-and-exit): Change year to 2019.
2019-01-05 00:08:04 +01:00
Ludovic Courtès ec651f2562
guix build: Honor '--system' for file-like objects and gexps.
Fixes a bug whereby "guix build -f file.scm -s SYSTEM" would not honor
SYSTEM when 'file.scm' returns a gexp or a file-like object.

* guix/scripts/build.scm (options->derivations): Pass #:system to
'run-with-store' in the 'file-like?' and 'gexp?' cases.
2019-01-05 00:08:04 +01:00
Ludovic Courtès 9ec154f51f
gexp: Lowering a <computed-file> honors SYSTEM and TARGET.
* guix/gexp.scm (computed-file-compiler): Pass #:system and #:target to
'gexp->derivation'.
* tests/gexp.scm ("lower-object, computed-file, #:system"): New test.
2019-01-05 00:08:04 +01:00
Ricardo Wurmus 2a13642b65
import: cran: Default to 'cran repo.
* guix/import/cran.scm (cran-recursive-import): Default to 'cran repo.
2019-01-03 08:34:41 +01:00
Ricardo Wurmus 7bb6420c5a
import: cran: Abort if no description could be fetched.
* guix/import/cran.scm (cran->guix-package): Only proceed if a valid
description could be fetched.
2019-01-03 08:34:41 +01:00
Ricardo Wurmus 632ea817b8
import: cran: Use HTTPS.
* guix/import/cran.scm (%cran-url): Use HTTPS.
2019-01-03 08:34:41 +01:00
Ricardo Wurmus b3d0617a55
import: cran: Download tarballs only once.
* guix/import/cran.scm (download): New procedure.
(fetch-description, description->package): Use it.
2019-01-03 08:34:41 +01:00
Eric Bavier 789fc77bef
refresh: github: updates for origins using 'git-fetch'.
* guix/import/github.scm (updated-github-url): Respond with the repository url
for the 'git-fetch' fetch method.
(github-package?): Simplify boolean expression.
(github-repository, github-user-slash-repository): Strip trailing ".git" from
project if present.
(latest-release)<origin-github-uri>: Recognize a 'git-reference'.
2018-12-31 19:00:23 -06:00
Efraim Flashner c180017b6f
lint: Check for unstable tarballs.
* guix/scripts/lint.scm (check-source-unstable-tarball): New procedure.
(%checkers): Add it.
* tests/lint.scm ("source-unstable-tarball", "source-unstable-tarball:
source #f", "source-unstable-tarball: valid", "source-unstable-tarball:
package named archive", "source-unstable-tarball: not-github",
"source-unstable-tarball: git-fetch"): New tests.
* doc/guix.texi (Invoking guix lint): Document it.
2018-12-27 14:55:54 +02:00
Ludovic Courtès 5923102f7b
pull: Add '--system'.
* guix/scripts/pull.scm (%options): Add '--system'.
(guix-pull): Honor it.
* doc/guix.texi (Invoking guix pull): Document it.
2018-12-27 12:08:25 +01:00
Ludovic Courtès 0ef595b996
offload: Remove unnecessary locking on machine slots.
This extra level of locking turned out to be unnecessary.

* guix/scripts/offload.scm (with-machine-lock): Remove.
(machine-lock-file): Remove.
(acquire-build-slot): Remove surrounding 'with-machine-lock'.
2018-12-26 18:40:49 +01:00
Ludovic Courtès 7f4d102c2f
offload: Remove the "machine choice" lock.
This lock was unnecessary and it led to a contention when many 'guix
offload' processes are polling for available machines.

* guix/scripts/offload.scm (machine-choice-lock-file): Remove.
(choose-build-machine): Remove surrounding 'with-file-lock (machine-lock-file)'.
2018-12-26 18:40:49 +01:00
Ludovic Courtès 10b2834f82
offload: Adjust 'test' and 'status' to the latest changes.
This is a followup to ed7b44370f71126087eb953f36aad8dc4c44109f;
following that commit, 'guix offload test' and 'guix offload status'
would abort with a backtrace instead of clearly diagnosing a missing
'guix' command on the build machine.

* guix/scripts/offload.scm (assert-node-has-guix): Call 'leave' when
NODE is not an inferior.  Remove 'catch' blocks for 'node-repl-error'.
(check-machine-availability): Invoke 'assert-node-has-guix' first.
(check-machine-status): Print a warning when 'remote-inferior' returns #f.
2018-12-25 17:13:43 +01:00
Ludovic Courtès ed7b44370f
offload: Use (guix inferior) instead of (ssh dist node).
Using inferiors and thus 'guix repl' simplifies setup on build
machines (no need to worry about GUILE_LOAD_PATH etc.)

Furthermore, the 'guix repl -t machine' protocol running in a remote
pipe addresses several issues with the current implementation of nodes
and RREPLs in Guile-SSH: fewer round trips, doesn't leave a 'guile
--listen' process behind it, stateless (since a new process is started
each time), more efficient (the SSH channel can be reused), more
reliable (no 'pgrep', 'pkill', and shellology; see
<https://github.com/artyom-poptsov/guile-ssh/issues/11> as an example.)

* guix/ssh.scm (inferior-remote-eval): New procedure.
(send-files): Use it instead of 'make-node' and 'node-eval'.
* guix/scripts/offload.scm (node-guile-version): New procedure.
(node-free-disk-space, transfer-and-offload, node-load)
(choose-build-machine, assert-node-has-guix): Use 'remote-inferior'
instead of 'make-node' and 'inferior-eval' instead of 'node-eval'.
(assert-node-can-import, assert-node-can-export): Likewise, and add
'session' parameter.
(check-machine-availability): Likewise, and add calls to
'close-inferior' and 'disconnect!'.
(check-machine-status): Likewise.
* doc/guix.texi (Daemon Offload Setup): Remove bit related to 'guile' in
$PATH and $GUILE_LOAD_PATH; mention 'guix' alone.
2018-12-24 16:06:32 +01:00
Ludovic Courtès af15fe13b6
ssh: Add 'remote-inferior'.
* guix/inferior.scm (<inferior>)[close]: New field.
(port->inferior): New procedure.
(open-inferior): Rewrite in terms of 'port->inferior'.
(close-inferior): Honor INFERIOR's 'close' field.
(inferior-eval-with-store): Add FIXME comment.
* guix/ssh.scm (remote-inferior): New procedure.
2018-12-24 16:06:32 +01:00
Efraim Flashner c39491829a
scripts: refresh: Allow searching recursively.
* guix/scripts/refresh.scm (refresh-recursive, list-transitive): New
procedures.
(show-help): Document it.
(guix-refresh): Add flags and checks for new options.
* doc/guix.texi (Invoking guix refresh): Document new options.
2018-12-24 12:19:17 +02:00
Arun Isaac 0865d8a8f6
guix: lint: Check for source URIs redirecting to GitHub.
* guix/scripts/lint.scm (check-github-uri): New procedure.
(%checkers): Add it.
* doc/guix.texi (Invoking guix lint): Document it.
* tests/lint.scm ("github-url", "github-url: one suggestion"): New tests.
2018-12-24 08:15:17 +05:30
Ludovic Courtès 62b845c5e2
offload: Display the normalized load in 'guix offload status' output.
Fixes a regression introduced in
bbe66a530a whereby the actual
load (non-normalized) would be displayed.

* guix/scripts/offload.scm (check-machine-status): Add call to
'normalized-load'.
2018-12-23 22:14:15 +01:00
Ludovic Courtès bdf860c2e9
database: Use "write-ahead log" mode and set a long "busy timeout".
This should avoid "database is locked" errors when there's a lot of
concurrency, for instance when offloading simultaneously a lot of
builds.

* guix/store/database.scm (call-with-database): Add two 'sqlite-exec'
calls to set 'journal_mode' and 'busy_timeout'.
2018-12-21 23:50:13 +01:00
Ludovic Courtès b96e05aefd
offload: Recognize build failures due to lack of disk space.
Previously, if a remote build would fail due to lack of disk space, this
would be considered a permanent failure and thus cached as a build
failure if the local daemon runs with '--cache-failures'.

* guix/scripts/offload.scm (transfer-and-offload): Upon
'nix-protocol-error?' call 'node-free-disk-space' and return 1 instead
of 100 if the result if lower than 10 MiB.
2018-12-21 23:50:13 +01:00
Ludovic Courtès 63b0c3eacc
offload: Skip machines that are low on disk space.
Fixes <https://bugs.gnu.org/33378>.

* guix/scripts/offload.scm (node-free-disk-space): New procedure.
(%minimum-disk-space): New variable.
(choose-build-machine): Call 'node-free-disk-space' and take it into
account in addition to LOAD.
(check-machine-status): Display the free disk space.
2018-12-21 23:50:13 +01:00
Ludovic Courtès bbe66a530a
offload: Decompose 'machine-load' into simpler procedures.
* guix/scripts/offload.scm (machine-load): Remove.
(node-load, normalized-load): New procedures.
(choose-build-machine): Call 'open-ssh-session' and 'make-node' from
here; pass the node to 'node-load'.
(check-machine-status): Use 'node-load' instead of 'machine-load'.  Call
'disconnect!' on SESSION.
2018-12-21 23:50:13 +01:00
Ricardo Wurmus 10a1cacb16
import: cran: Try import via CRAN if package is not on Bioconductor.
* guix/import/cran.scm (fetch-description): Return #F on failure.
(cran->guix-package): Retry from CRAN on failure to fetch description from
bioconductor.
2018-12-20 09:42:22 +01:00
Ludovic Courtès 40cc850aeb
download: 'built-in-builders*' relies on the functional cache.
The previous caching strategy, which used STORE as an 'eq?' key, would
no longer work when the functional cache is used because subsequent
store values are not 'eq?'.

* guix/download.scm (built-in-builders*): Rewrite in terms of
'mcached'.
2018-12-19 23:52:25 +01:00
Ludovic Courtès 87b711d200
utils: Memoize 'absolute-dirname'.
* guix/utils.scm (absolute-dirname): Wrap in 'mlambda'.
2018-12-19 23:52:25 +01:00