Commit Graph

316 Commits

Author SHA1 Message Date
Ludovic Courtès b5f8c2c885
hydra: Compute jobs in an inferior.
Previously we would rely on auto-compilation of all the Guix modules.
The complete evaluation would take ~15mn on berlin.guixsd.org and
require lots of RAM.  This approach should be faster since potentially
only part of the modules are rebuilt.  Furthermore, as a side-effect, it
builds the derivations that 'guix pull' uses.

* build-aux/hydra/gnu-system.scm: Remove 'eval-when' form.
(hydra-jobs): New procedure.
* gnu/ci.scm (package->alist, qemu-jobs, system-test-jobs)
(tarball-jobs): Return strings for the 'license' field.
* guix/self.scm (compiled-guix)[*cli-modules*]: Add (gnu ci).
2019-01-06 21:41:07 +01:00
Ludovic Courtès 65ff85dcee
hydra: evaluate: Add the checkout to the store.
* build-aux/hydra/evaluate.scm <top level>: Add call to 'add-to-store'.
Use that as the 'file-name' attribute.  Call 'primitive-load' in a
directory excursion to SOURCE.
2019-01-06 21:41:07 +01:00
Ludovic Courtès 59fb5c1cdb
hydra: Move job definitions to (gnu ci).
* build-aux/hydra/gnu-system.scm: Move code to...
* gnu/ci.scm: ... here.  New file.
* gnu/local.mk (GNU_SYSTEM_MODULES): Add it.
2019-01-06 21:41:07 +01:00
Ludovic Courtès afb82831fa
build-self: Don't clobber the output port.
The newline is meant to follow the spinner's traces so it must go to the
error port as well.

* build-aux/build-self.scm (build): Send newline to the error port.
2019-01-06 21:41:07 +01:00
Ludovic Courtès dd8356e3c6
hydra: Use short timeouts for the QEMU jobs and system tests.
These builds shouldn't take too long.  Sometimes the guest kernel
crashes and the VM gets stuck; we should be able to terminate those jobs
quickly.

* build-aux/hydra/gnu-system.scm (qemu-jobs)[->alist]: Add
'max-silent-time' and 'timeout'.
(system-test-jobs)[test->thunk]: Likewise.
2018-12-29 01:05:17 +01:00
Danny Milosavljevic 07c791c110
hydra: Increase image sizes for USB image and Flash image.
* build-aux/hydra/gnu-system.scm (qemu-jobs) <usb-image>: Increase from
1024 MiB to 1500 MiB.
<flash-image>: Increase from 1024 MiB to 1500 MiB.
2018-12-09 00:49:54 +01:00
Ludovic Courtès 6eac835f17
maint: update-guix-package: Really register GC roots.
Previously we'd pass a relative file name to 'add-indirect-root', which
the daemon would interpret as relative to "/".  Consequently, checkouts
were not protected from GC.

* build-aux/update-guix-package.scm (main): Pass an absolute file name
to 'add-indirect-root'.
2018-12-07 15:14:12 +01:00
Ludovic Courtès e3c03952be
maint: update-NEWS: Don't produce full package lists.
The lists of new and upgraded packages in 'NEWS' had become way too long
and redundant with what 'guix pull' reports.

* build-aux/update-NEWS.scm (write-packages-added): Don't print ADDED.
(write-packages-updates)[important, table, latest, noteworthy]: New
variables.
Print NOTEWORTHY rather than all of UPGRADED.
(main): Print PREVIOUS-VERSION and NEW-VERSION.
2018-12-05 00:08:34 +01:00
Ludovic Courtès 0bc02becca
Remove most references to hydra.gnu.org.
* Makefile.am (assert-binaries-available): Don't mention hydra.gnu.org
in comment.
* build-aux/check-available-binaries.scm: Likewise.
* build-aux/check-final-inputs-self-contained.scm: Likewise.
* doc/guix.texi (SUBSTITUTE-SERVER): New variable.
Use it throughout instead of "mirror.hydra.gnu.org".
* doc/contributing.texi (Submitting Patches): Likewise.
* gnu/services/base.scm (hydra-key-authorization)
(guix-activation): Remove mentions of "hydra.gnu.org" in comments and
messages.
* gnu/system/install.scm (%installation-services): Likewise.
* guix/scripts/size.scm (guix-size): Likewise.
2018-12-04 10:57:56 +01:00
Marius Bakke d02bb02f7d
Merge branch 'master' into core-updates 2018-10-17 20:47:11 +02:00
Ludovic Courtès 276f368051
tests: "make check-system" produces colored output.
* build-aux/run-system-tests.scm (run-system-tests): Use 'with-status-report'.
2018-10-09 18:54:12 +02:00
Marius Bakke 2817ac3c18
Merge branch 'master' into core-updates 2018-09-20 13:37:58 +02:00
Clément Lassieur 6ffffb3c96
hydra: Remove useless 'parameterize'.
* build-aux/hydra/gnu-system.scm (manifests->packages): Remove 'parameterize'.
2018-09-14 09:01:07 +02:00
Leo Famulari d763940711
Merge branch 'master' into core-updates 2018-09-13 13:32:39 -04:00
Ludovic Courtès ac4d2ec81a
build-self: Emit a useful message upon error.
* build-aux/build-self.scm (build): Raise a '&message' condition instead
of calling 'error'.
2018-09-13 10:29:59 +02:00
Ludovic Courtès 78c9058dde
build-self: Add a dummy (git) module to 'compute-guix-derivation'.
Fixes a regression introduced in
aed0a59405.

* build-aux/build-self.scm (build-program)[fake-git]: New variable.
Use it as an imported module.
2018-09-13 10:29:59 +02:00
Clément Lassieur 88bfabf111
hydra: Add support for manifests.
* build-aux/hydra/gnu-system.scm (arguments->manifests, manifests->packages):
New procedures.
(hydra-jobs): Add a "manifests" subset.
* doc/guix.texi (Continuous Integration): Update accordingly.
2018-09-13 10:15:04 +02:00
Marius Bakke 0aeb134850
Merge branch 'master' into core-updates 2018-09-09 17:40:35 +02:00
Ludovic Courtès 3ffcad7df3
build-self: Accomodate upgrades from May 2016 and earlier.
Reported by Paul Garlick <pgarlick@tourbillion-technology.com>.

Fixes a bug whereby "old" Guix instances (before mid-May 2018) would
fail to pull to current master.

Specifically, Paul reported being unable to upgrade from
6f84dc4314 (a commit that predates that
addition of the 'bootstrap' phase on gnu-build-system on 'master' and
that also predates the new 'guix pull'.)

* build-aux/build-self.scm (guile-gcrypt)[arguments]: New field.
2018-09-06 12:01:39 +02:00
Ludovic Courtès ca71942445
Switch to Guile-Gcrypt.
This removes (guix hash) and (guix pk-crypto), which now live as part of
Guile-Gcrypt (version 0.1.0.)

* guix/gcrypt.scm, guix/hash.scm, guix/pk-crypto.scm,
tests/hash.scm, tests/pk-crypto.scm: Remove.
* configure.ac: Test for Guile-Gcrypt.  Remove LIBGCRYPT and
LIBGCRYPT_LIBDIR assignments.
* m4/guix.m4 (GUIX_ASSERT_LIBGCRYPT_USABLE): Remove.
* README: Add Guile-Gcrypt to the dependencies; move libgcrypt as
"required unless --disable-daemon".
* doc/guix.texi (Requirements): Likewise.
* gnu/packages/bash.scm, guix/derivations.scm, guix/docker.scm,
guix/git.scm, guix/http-client.scm, guix/import/cpan.scm,
guix/import/cran.scm, guix/import/crate.scm, guix/import/elpa.scm,
guix/import/gnu.scm, guix/import/hackage.scm,
guix/import/texlive.scm, guix/import/utils.scm, guix/nar.scm,
guix/pki.scm, guix/scripts/archive.scm,
guix/scripts/authenticate.scm, guix/scripts/download.scm,
guix/scripts/hash.scm, guix/scripts/pack.scm,
guix/scripts/publish.scm, guix/scripts/refresh.scm,
guix/scripts/substitute.scm, guix/store.scm,
guix/store/deduplication.scm, guix/tests.scm, tests/base32.scm,
tests/builders.scm, tests/challenge.scm, tests/cpan.scm,
tests/crate.scm, tests/derivations.scm, tests/gem.scm,
tests/nar.scm, tests/opam.scm, tests/pki.scm,
tests/publish.scm, tests/pypi.scm, tests/store-deduplication.scm,
tests/store.scm, tests/substitute.scm: Adjust imports.
* gnu/system/vm.scm: Likewise.
(guile-sqlite3&co): Rename to...
(gcrypt-sqlite3&co): ... this.  Add GUILE-GCRYPT.
(expression->derivation-in-linux-vm)[config]: Remove.
(iso9660-image)[config]: Remove.
(qemu-image)[config]: Remove.
(system-docker-image)[config]: Remove.
* guix/scripts/pack.scm: Adjust imports.
(guile-sqlite3&co): Rename to...
(gcrypt-sqlite3&co): ... this.  Add GUILE-GCRYPT.
(self-contained-tarball)[build]: Call 'make-config.scm' without
 #:libgcrypt argument.
(squashfs-image)[libgcrypt]: Remove.
[build]: Call 'make-config.scm' without #:libgcrypt.
(docker-image)[config, json]: Remove.
[build]: Add GUILE-GCRYPT to the extensions  Remove (guix config) from
the imported modules.
* guix/self.scm (specification->package): Remove "libgcrypt", add
"guile-gcrypt".
(compiled-guix): Remove #:libgcrypt.
[guile-gcrypt]: New variable.
[dependencies]: Add it.
[*core-modules*]: Remove #:libgcrypt from 'make-config.scm' call.
Add #:extensions.
[*config*]: Remove #:libgcrypt from 'make-config.scm' call.
(%dependency-variables): Remove %libgcrypt.
(make-config.scm): Remove #:libgcrypt.
* build-aux/build-self.scm (guile-gcrypt): New variable.
(make-config.scm): Remove #:libgcrypt.
(build-program)[fake-gcrypt-hash]: New variable.
Add (gcrypt hash) to the imported modules.  Adjust load path
assignments.
* gnu/packages/package-management.scm (guix)[propagated-inputs]: Add
GUILE-GCRYPT.
[arguments]: In 'wrap-program' phase, add GUILE-GCRYPT to the search
path.
2018-09-04 17:25:11 +02:00
Marius Bakke 70dc8db8e7
Merge branch 'master' into core-updates 2018-09-03 19:20:06 +02:00
Ludovic Courtès fe634eaf93
Add (guix describe) and use it to initialize '%package-search-path'.
* guix/describe.scm: New file.
* Makefile.am (MODULES): Add it.
* gnu/packages.scm (%default-package-module-path): New variable.
(%package-module-path): Honor 'package-path-entries'.
* build-aux/update-NEWS.scm (main): Use %DEFAULT-PACKAGE-MODULE-PATH
instead of (last (%package-module-path)).
2018-09-02 16:43:10 +02:00
Ludovic Courtès 20cbd4f5f4
compile-as-derivation: Adjust %load-compiled-path.
* build-aux/compile-as-derivation.scm: Set not only %load-path but also
%load-compiled-path.  This can avoid problems where a stale .go is
loaded.
2018-09-02 15:43:51 +02:00
Ludovic Courtès 1428bce348
build-self: Default to Guile 2.2 when building a standalone Guix.
* build-aux/build-self.scm (build): Change #:guile-version to default to
"2.2" when PULL-VERSION is greater than 0.
2018-07-27 11:32:59 +02:00
Ludovic Courtès 8bece84022
gnu: guile: Remove version 2.2.2.
* gnu/packages/guile.scm (guile-2.2.2): Remove.
* guix/self.scm (guile-for-build): Remove special case for "2.2.2".
(guix-derivation): Likewise.
* build-aux/build-self.scm (build): Likewise.
2018-07-27 11:28:15 +02:00
Ludovic Courtès 790c3e019a
build-self: Inherit the daemon connection from the parent process.
Fixes <https://bugs.gnu.org/31892>.
Reported by Vagrant Cascadian <vagrant@debian.org>.

* build-aux/build-self.scm (build): Define 'port' and wrap 'open-pipe*'
call in 'with-input-from-port'.
(build-program): Use 'port->connection' or 'open-connection' instead of
'with-store.'
2018-06-26 14:27:57 +02:00
Ludovic Courtès 1f1d76a178
build-self: Avoid recompilations of 'compute-guix-derivation'.
* build-aux/build-self.scm (build-program)["compute-guix-derivation"]:
Honor the SOURCE command-line argument.  Add a VERSION command-line
argument and honor it.
(build): Pass VERSION to BUILD.
2018-06-21 23:54:15 +02:00
Ludovic Courtès 7af5c2a248
self: Define derived '-directory' variables in config.scm.
This is a followup to d6b5aa0b03.

* guix/self.scm (%config-variables): Remove %CONFIG-DIRECTORY,
%STATE-DIRECTORY, %STORE-DATABASE-DIRECTORY, and %STORE-DIRECTORY.
(make-config.scm): Define them here.
* build-aux/build-self.scm (%config-variables, make-config.scm):
Likewise.
2018-06-18 23:56:16 +02:00
Ludovic Courtès d6b5aa0b03
build-self: Do not rely on '%store-database-directory'.
Guix'es older than one week don't have this variable so requiring it
would break things for them.

* build-aux/build-self.scm (%config-variables): Remove
'%store-database-directory'.
2018-06-18 23:25:53 +02:00
Ludovic Courtès e8cb9c01c6
build: Remove checks for 'nix-instantiate'.
* guix/import/snix.scm (open-nixpkgs): Use "nix-instantiate"
unconditionally.
* configure.ac: Remove check for 'nix-instantiate'.
* guix/config.scm.in (%nix-instantiate): Remove.
* guix/self.scm (%dependency-variables): Remove '%nix-instantiate'.
(make-config.scm): Remove it from the generated "config.scm".
* build-aux/build-self.scm (%dependency-variables, make-config.scm):
Likewise.
2018-06-18 23:19:51 +02:00
Ludovic Courtès 45779fa676
self: Make (guix config) generation really stateless.
Previously the %CONFIG-VARIABLES list would be generated based on what
the current (guix config) contains.  Thus, it would include
'%guix-register-program', which we recently removed, because
existing (guix config) most likely contained that variable.  Since its
value could differ from machine to machine, the build farm could be
building a different config.scm, thereby preventing people from getting
substitutes.

* guix/self.scm (%config-variables): Turn into a white list instead of
taking all the remaining variables from the current (guix config).
* build-aux/build-self.scm (%config-variables): Likewise.
2018-06-18 23:19:51 +02:00
Ludovic Courtès 7b0449712e
hydra: Make sure we add an absolute file name to the load path.
Previously, under Cuirass, we'd add "build-aux/cuirass/../hydra/../..".
But then, commit c45477d2a1 added this
in (gnu system vm):

         (local-file (search-path %load-path
                                  "guix/store/schema.sql"))

Since 'search-path' would not return an absolute file name, 'local-file'
would try to resolve that relative to "gnu/system", leading to:

  ERROR: In procedure force:
  In procedure canonicalize-path: No such file or directory: "/gnu/store/…-guix-ce573b1/gnu/system/build-aux/cuirass/../hydra/../../guix/store/schema.sql

* build-aux/hydra/gnu-system.scm <top level>: Canonicalize DIR before
adding it to %LOAD-PATH.
2018-06-15 10:04:59 +02:00
Ludovic Courtès ea0a06cee2
Remove 'guix-register' and its traces.
* Makefile.am (SH_TESTS): Remove tests/guix-register.sh.
* build-aux/pre-inst-env.in (GUIX_REGISTER): Remove.
* gnu/build/install.scm (directives): Remove outdated comment.
* gnu/build/vm.scm (root-partition-initializer): Update comment.
* gnu/packages/package-management.scm (guix-register): Remove.
* guix/config.scm.in (%sbindir, %guix-register-program): Remove.
* guix/scripts/system.scm (install): Adjust docstring.
* guix/self.scm (make-config.scm): Remove #:guix.  Do not generate
%sbindir and %guix-register-program.
(specification->package): Remove "guix".
* nix/guix-register/guix-register.cc: Remove.
* nix/libstore/store-api.cc (decodeValidPathInfo): Remove.
* nix/libstore/store-api.hh (decodeValidPathInfo): Remove declaration.
* nix/local.mk (sbin_PROGRAMS, guix_register_SOURCES)
(guix_register_CPPFLAGS, guix_register_LDFLAGS): Remove.
* tests/guix-register.sh: Remove.
2018-06-14 11:17:00 +02:00
Ludovic Courtès e740e8b27d
hydra: guix-modular: Build the full Guix.
* build-aux/hydra/guix-modular.scm (build-job): Pass #:pull-version to
BUILD.
2018-06-10 23:32:00 +02:00
Ludovic Courtès 75e24d7b0e
pull: Install the new Guix in a profile.
* guix/scripts/pull.scm (%pull-version): New variable.
(build-from-source): Pass #:pull-version to BUILD.
(whole-package-for-legacy, derivation->manifest-entry): New procedure.
(build-and-install): Rewrite in terms of 'build-and-use-profile'.
* guix/scripts/system.scm (maybe-suggest-running-guix-pull)[latest]:
Switch to "/current".
* scripts/guix.in (augment-load-paths!): Remove use of
~/.config/guix/latest.
* build-aux/compile-as-derivation.scm: Replace "/guix/latest/" with
"/current/share/guile/site/X.Y"
* guix/scripts.scm (warn-about-old-distro)[age]: Check "/current"
instead of "/latest".
* doc/guix.texi (Invoking guix pull): Document it.
* doc/contributing.texi (Running Guix Before It Is Installed): Remove
footnote about abusing ~/.config/guix/latest.
2018-06-09 12:02:27 +02:00
Ludovic Courtès 8a0d9bc8a3
self: Produce a complete package with the 'guix' command.
* guix/self.scm (guix-command): New procedure.
(compiled-guix): Add #:pull-version parameter.
[command, package]: New variables.
Honor PULL-VERSION.
(guix-derivation): Add #:pull-version and pass it to 'compiled-guix'.
* build-aux/build-self.scm (build-program): Add #:pull-version
parameter.  Pass it to 'guix-derivation'.
(build): Add #:pull-version and pass it to 'build-program'.
* build-aux/compile-as-derivation.scm: Pass #:pull-version to BUILD.
2018-06-09 12:02:27 +02:00
Ludovic Courtès 9e7b87ddc5
build: 'test-env' does not 'guix download' non-existent files.
Reported by uniq10.

* build-aux/test-env.in: Check whether FILE exists before invoking 'guix
download'.
2018-05-13 23:16:16 +02:00
Ludovic Courtès d6680dccb0
hydra: Switch back to "-" instead of "@" in package job names.
Fixes a regression introduced in ede121de42.
Reported by Mark H Weaver <mhw@netris.org>.

* build-aux/hydra/gnu-system.scm (job-name): Pass "-" as the second
argument to 'package-full-name'.
2018-05-12 15:58:56 +02:00
Ludovic Courtès 479c0e3f60
build: Fix possible race condition in 'test-env'.
Reported by Mark H Weaver <mhw@netris.org>.

* build-aux/test-env.in: Add 'guile' invocation before 'guix download'
invocation to make sure 'guix-daemon' is listening.
2018-04-22 23:50:49 +02:00
Ludovic Courtès df154c05dc
hydra: Fix uses of 'eval-when'.
* build-aux/hydra/gnu-system.scm: Use (expand load eval) in 'eval-when'
clause so as to not poison a cat and to ensure %load-compiled-path is
properly set when the 'use-modules' clauses are expanded.
* build-aux/hydra/guix.scm: Likewise.
2018-04-09 00:00:58 +02:00
Ludovic Courtès 98cda1b997
hydra: guix-modular: Honor the 'systems' argument.
* build-aux/hydra/guix-modular.scm (hydra-jobs)[systems]: Define as in
gnu-system.scm.
Honor SYSTEMS.
2018-04-08 22:07:47 +02:00
Ludovic Courtès f27a712840
build-self: Add missing 'close-pipe' call.
* build-aux/build-self.scm (build): Call 'close-pipe'.
2018-04-08 21:11:37 +02:00
Ludovic Courtès 16c28becf2
hydra: guix-modular: Explicitly build for the latest 2.2.
* build-aux/hydra/guix-modular.scm (build-job): Pass #:guile-version to
'build'.
2018-04-08 19:57:31 +02:00
Ludovic Courtès f6dfb8bb08
hydra: Clear '%load-compiled-path' for gnu-system.scm.
On hydra.gnu.org GUILE_LOAD_COMPILED_PATH happened to be empty so we
were fine.  However on GuixSD with Cuirass, we could have a conflicting
non-empty path and .go files for some of the (guix …) and (gnu …)
modules would be taken from there.

* build-aux/hydra/gnu-system.scm: Set %load-compiled-path.
2018-04-08 17:57:13 +02:00
Ludovic Courtès 9fe9594611
hydra: 'guix-modular' jobs use the new 'build-self'.
* build-aux/hydra/guix-modular.scm: Remove 'eval-when' form that fiddled
with '%load-path'.
(build-job): Load 'build-aux/build-self.scm' and call the procedure it
returns.
* build-aux/hydra/evaluate.scm: The 'build-things' replacement no longer
calls 'exit'.
2018-04-08 17:48:33 +02:00
Ludovic Courtès 351f384e57
cuirass: Add job specs for the modular Guix.
* build-aux/cuirass/guix-modular.scm: New file.
* Makefile.am (EXTRA_DIST): Add it.
2018-04-08 17:48:33 +02:00
Ludovic Courtès a1ec7540be
cuirass: Factorize hydra-to-cuirass CI job translation.
* build-aux/cuirass/gnu-system.scm: Move code to...
* build-aux/cuirass/hydra-to-cuirass.scm: ... here, and include it.
* Makefile.am (EXTRA_DIST): Add 'build-aux/cuirass/hydra-to-cuirass.scm'
and 'build-aux/cuirass/gnu-system.scm'.
2018-04-08 17:48:33 +02:00
Ludovic Courtès 2cfc8d6964
build: Add 'as-derivation' target.
* build-aux/compile-as-derivation.scm: New file.
* Makefile.am (as-derivation): New target.
(EXTRA_DIST): Add compile-as-derivation.scm.
2018-04-08 17:48:25 +02:00
Ludovic Courtès f0527ce3a4
build-self: Use (guix self).
This mitigates <https://bugs.gnu.org/27284>.

* build-aux/build-self.scm (libgcrypt, zlib, gzip, bzip2, xz)
(false-if-wrong-guile, package-for-current-guile, guile-json)
(guile-ssh, guile-git, guile-bytestructures, matching-guile-2.2): Remove.
(%dependency-variables, %persona-variables, %config-variables): New
variables.
(make-config.scm, load-path-expression, gexp->script)
(build-program): New procedures.
(build): Rewrite to simply delegate to 'guix-derivation'.
2018-04-08 17:41:08 +02:00
Ludovic Courtès 4debffadb7
maint: Adjust 'check-final-inputs-self-contained' for glibc:static.
This change is necessary to cope with
6dff905e51.

* build-aux/check-final-inputs-self-contained.scm (final-inputs): Change
'match' pattern to match (LABEL PACKAGE OUTPUT).
2018-03-25 00:26:13 +01:00