Commit Graph

25 Commits

Author SHA1 Message Date
Ludovic Courtès 13512e1b8f
git-download: 'git-predicate' returns #f on Git errors.
Fixes a regression introduced in
aed0a59405 whereby 'git-predicate' would
throw to 'git-error instead of returning #f as the docstring says.

* guix/git-download.scm (git-predicate): Return #f upon 'git-error'.
2018-09-14 11:23:08 +02:00
Ludovic Courtès aed0a59405
git-download: Rewrite 'git-predicate' using Guile-Git.
Fixes <https://bugs.gnu.org/27925>.

* guix/git-download.scm (files->directory-tree)
(directory-in-tree?): Remove.
(git-file-list): New procedures.
(git-predicate): Use it instead of opening a pipe to 'git'.  Remove
directory tree hack and rely exclusively on inode/device numbers.
2018-09-13 00:20:42 +02:00
Eric Bavier be95bcf088
git-download: Fix recursive checkouts.
* guix/git-download.scm (git-fetch)<build>: Fix match on INPUTS, which may
contain package outputs.
2018-03-18 20:00:44 -05:00
Ludovic Courtès 37ce440dcf
download: Download a nar when a VCS checkout fails.
Fixes <https://bugs.gnu.org/28709>.

* guix/build/download-nar.scm: New file.
* Makefile.am (MODULES): Add it.
* guix/cvs-download.scm (cvs-fetch)[zlib, config.scm, modules]: New
variables.
[build]: Use MODULES.  Add call to 'download-nar'.
* guix/git-download.scm (git-fetch): Likewise.
* guix/hg-download.scm (hg-fetch): Likewise.
2017-10-19 23:21:49 +02:00
Ludovic Courtès 228a3982df
git-download: Remove call to 'canonicalize-path'.
* guix/git-download.scm (git-predicate): Remove call to
'canonicalize-path' since this could lead to discrepancies.  For
instance it broke 'make update-guix-package' since it passes a
non-canonical directory name.
2017-07-30 17:22:13 +02:00
Christopher Baines f135b4ae83
git-download: Speed up 'git-predicate'.
Adjust 'git-predicate' to use data structures that perform better when used
with git repositories with a large number of files.

Previously when matching either a regular file or directory, 'git-predicate'
would search a list with a length equal to the number of files in the
repository. As a search operation happens for roughly every file in the
repository, this meant that the time taken to use 'git-predicate' to traverse
all the files in a repository was roughly exponential with respect to the
number of files in the repository.

Now, for matching regular files or symlinks, 'git-predicate' uses a vhash
using the inode value as the key. This should perform roughly in constant
amount of time, instead of linear with respect to the number of files in the
repository.

For matching directories, 'git-predicate' now uses a tree structure stored in
association lists. To check if a directory is in the tree, the tree is
traversed from the root. The time complexity of this depends on the shape of
the tree, but it should be an improvement on searching through the list of all
files.

* guix/git-download.scm (files->directory-tree, directory-in-tree?): New
procedures.
(git-predicate): Compute DIRECTORY-TREE.  Turn INODES into a vhash.
Adjust body of lambda accordingly.

Co-authored-by: Ludovic Courtès <ludo@gnu.org>
2017-07-25 23:24:16 +02:00
Christopher Baines 151cb9738a
git-download: Fix 'git-predicate' to use absolute paths.
git ls-files will return paths relative to the repository directory. This
commit prepends the repository directory to those paths when calling lstat,
such that 'git-predicate' works if the current working directory is not the
repository directory.

* guix/git-download.scm (git-predicate): Prepend repository directory to the
  file path when calling lstat.

Signed-off-by: Ludovic Courtès <ludo@gnu.org>
2017-05-30 18:08:37 +02:00
Ludovic Courtès ba2260dbbc
git-download: Fix 'git-predicate' file membership.
Previously, it the predicate would return #t for "m4/ChangeLog" if
"ChangeLog" (in the top-level directory) was in FILES.  This commit
fixes the ambiguity.

* guix/git-download.scm (git-predicate): Add 'inodes' variable.  Use it
to determine file membership.
2017-05-07 18:08:11 +02:00
Mathieu Lirzin 6554be68b4
git-download: Add 'git-predicate'.
* guix/git-download.scm (git-predicate): New procedure.
* gnu/packages/package-management.scm (current-guix): Use it.
(make-git-predicate): Remove.
2017-02-09 13:56:54 -06:00
Mathieu Lirzin 5c6a30c511
git download: Remove redundant argument in 'gexp->derivation' call.
* guix/git-download.scm (git-fetch): Call 'gexp->derivation' with only one
'#:local-build?' keyword argument.
2017-01-11 18:55:26 +01:00
David Craven ee17a9e06e
git-download: Add some helpers.
* guix/git-download.scm (git-version, git-file-name): New variables.
2017-01-04 11:44:07 +01:00
Ludovic Courtès c0b2d08bf4
git-download: Use a single download script for all derivations.
That way, we have only one /gnu/store/…-git-download instead of one
/gnu/store/…-PACKAGE-checkout-builder for each package.

This is a followup to ced200328c.

* guix/git-download.scm (git-fetch)[build]: Get the URL, commit, and
recursive parameter using 'getenv'.  Pass #:script-name and #:env-vars
arguments to 'gexp->derivation'.
2016-12-31 18:35:29 +01:00
Ludovic Courtès e9b046fdda
download: Use 'with-imported-modules'.
* guix/cvs-download.scm (cvs-fetch): Use 'with-imported-modules' instead
of the #:modules argument of 'gexp->derivation'.
* guix/download.scm (url-fetch): Likewise.
* guix/git-download.scm (git-fetch): Likewise.
* guix/hg-download.scm (hg-fetch): Likewise.
* guix/svn-download.scm (svn-fetch): Likewise.
2016-07-12 22:47:08 +02:00
Ludovic Courtès 6b44a09747 download: Disable offloading for downloads.
* guix/download.scm (url-fetch): Use #:local-build? #t.
* guix/git-download.scm (git-fetch): Likewise.
2015-09-05 22:11:03 +02:00
Ludovic Courtès 4a6aeb670f derivations: Add #:substitutable?, distinguished from #:local-build?.
Fixes <http://bugs.gnu.org/18747>.

* guix/derivations.scm (substitutable-derivation?): Rewrite to check for
  "allowSubstitutes".
  (derivation): Add #:substitutable? parameter.
  [user+system-env-vars]: Honor it.
  (build-expression->derivation): Add #:substitutable? and honor it.
* guix/gexp.scm (gexp->derivation): Likewise.
* tests/derivations.scm ("derivation-prerequisites-to-build and substitutes,
  non-substitutable build"): Use #:substitutable? instead of #:local-build?.
  ("substitutable-derivation?", "derivation-prerequisites-to-build and
  substitutes, local build"): New tests.
* guix/download.scm (url-fetch): Adjust comment.
* guix/git-download.scm (git-fetch): Likewise.
* guix/build-system/gnu.scm (gnu-build, gnu-cross-build): Use #:substitutable?
  instead of #:local-build?.
* doc/guix.texi (Derivations, G-Expressions): Adjust accordingly.
2015-07-03 00:30:55 +02:00
Ludovic Courtès 1590e8a1dd packages: Refer to the native tools when handling sources and downloads.
* guix/packages.scm (patch-and-repack)[build]: Change most #$ to #+.
* guix/cvs-download.scm (cvs-fetch)[build]: Likewise.
* guix/download.scm (url-fetch)[builder]: Likewise.
* guix/git-download.scm (git-fetch)[build]: Likewise.
* guix/svn-download.scm (svn-fetch)[build]: Likewise.
2015-04-15 10:19:47 +02:00
Ludovic Courtès f220a83848 packages: Convert source derivations to monadic style.
* guix/packages.scm (origin->derivation): Take body from
  'package-source-derivation', and change it to monadic style.  Expect
  METHOD to a monadic procedure.
  (package-source-derivation): Define in terms of 'origin->derivation'.
* guix/download.scm (url-fetch): Remove 'store' argument.  Remove
  'guile-for-build' variable.  Turn into a monadic procedure.
* guix/git-download.scm (git-fetch): Likewise.
* guix/svn-download.scm (svn-fetch): Likewise.
* tests/builders.scm (url-fetch*): New procedure.
  Change tests to call 'url-fetch*' instead of 'url-fetch'.
* tests/packages.scm ("package-source-derivation, snippet"): Remove
  'store' parameter of 'fetch' and change it to use 'interned-file'
  instead of 'add-to-store'.
* gnu/packages/bootstrap.scm (bootstrap-origin)[boot]: Remove 'store'
  parameter.
2015-01-14 14:42:10 +01:00
Ludovic Courtès e87f0591f3 monads: Move '%store-monad' and related procedures where they belong.
This turns (guix monads) into a generic module for monads, and moves the
store monad and related monadic procedures in their corresponding
module.

* guix/monads.scm (store-return, store-bind, %store-monad, store-lift,
  text-file, interned-file, package-file, package->derivation,
  package->cross-derivation, origin->derivation, imported-modules,
  compiled, modules, built-derivations, run-with-store): Move to...
* guix/store.scm (store-return, store-bind, %store-monad, store-lift,
  text-file, interned-file): ... here.
  (%guile-for-build): New variable.
  (run-with-store): Moved from monads.scm.  Remove default value for
  #:guile-for-build.
* guix/packages.scm (default-guile): Export.
  (set-guile-for-build): New procedure.
  (package-file, package->derivation, package->cross-derivation,
  origin->derivation): Moved from monads.scm.
* guix/derivations.scm (%guile-for-build): Remove.
  (imported-modules): Rename to...
  (%imported-modules): ... this.
  (compiled-modules): Rename to...
  (%compiled-modules): ... this.
  (built-derivations, imported-modules, compiled-modules): New
  procedures.
* gnu/services/avahi.scm, gnu/services/base.scm, gnu/services/dbus.scm,
  gnu/services/dmd.scm, gnu/services/networking.scm,
  gnu/services/ssh.scm, gnu/services/xorg.scm, gnu/system/install.scm,
  gnu/system/linux-initrd.scm, gnu/system/shadow.scm, guix/download.scm,
  guix/gexp.scm, guix/git-download.scm, guix/profiles.scm,
  guix/svn-download.scm, tests/monads.scm: Adjust imports accordingly.
* guix/monad-repl.scm (default-guile-derivation): New procedure.
  (store-monad-language, run-in-store): Use it.
* build-aux/hydra/gnu-system.scm (qemu-jobs): Add explicit
  'set-guile-for-build' call.
* guix/scripts/archive.scm (derivation-from-expression): Likewise.
* guix/scripts/build.scm (options/resolve-packages): Likewise.
* guix/scripts/environment.scm (guix-environment): Likewise.
* guix/scripts/system.scm (guix-system): Likewise.
* doc/guix.texi (The Store Monad): Adjust module names accordingly.
2015-01-14 13:34:52 +01:00
Ludovic Courtès 347e17b474 download: Allow use of substitutes.
See <https://bugs.gnu.org/18747> for the original report.

* guix/download.scm (url-fetch): Comment out #:local-build? argument.
* guix/git-download.scm (git-fetch): Likewise.
* guix/svn-download.scm (svn-fetch): Likewise.
2014-10-16 23:40:10 +02:00
Ludovic Courtès 0d5a559f0f build-system: Introduce "bags" as an intermediate representation.
* guix/build-system.scm (<build-system>)[build, cross-build]: Remove.
  [lower]: New field.
  (<bag>): New record type.
  (make-bag): New procedure.
* guix/packages.scm (bag-transitive-inputs, bag-transitive-build-inputs,
  bag-transitive-host-inputs, bag-transitive-target-inputs,
  package->bag): New procedures.
  (package-derivation): Use it; use the bag, apply its build procedure,
  etc.
  (package-cross-derivation): Likewise.
* gnu/packages/bootstrap.scm (raw-build, make-raw-bag): New procedure.
  (%bootstrap-guile): Use them.
* guix/build-system/trivial.scm (lower): New procedure.
  (trivial-build, trivial-cross-build): Remove 'source' parameter.  Pass
  INPUTS as is.
  (trivial-build-system): Adjust accordingly.
* guix/build-system/gnu.scm (%store, inputs-search-paths,
  standard-search-paths, expand-inputs, standard-inputs): Remove.
  (gnu-lower): New procedure.
  (gnu-build): Remove 'source' and #:implicit-inputs? parameters.
  Remove 'implicit-inputs' and 'implicit-search-paths' variables.  Get
  the source from INPUT-DRVS.
  (gnu-cross-build): Likewise.
  (standard-cross-packages): Remove call to 'standard-packages'.
  (standard-cross-inputs, standard-cross-search-paths): Remove.
  (gnu-build-system): Remove 'build' and 'cross-build'; add 'lower'.
* guix/build-system/cmake.scm (lower): New procedure.
  (cmake-build): Remove 'source' and #:cmake parameters.  Use INPUTS and
  SEARCH-PATHS as is.  Get the source from INPUTS.
* guix/build-system/perl.scm: Likewise.
* guix/build-system/python.scm: Likewise.
* guix/build-system/ruby.scm: Likewise.
* gnu/packages/cross-base.scm (cross-gcc): Change "cross-linux-headers"
  to "linux-headers".
  (cross-libc)[xlinux-headers]: Pass #:implicit-cross-inputs? #f.
  Likewise.  In 'propagated-inputs', change "cross-linux-headers" to
  "linux-headers".
* guix/git-download.scm (git-fetch): Use 'standard-packages' instead of
  'standard-inputs'.
* tests/builders.scm ("gnu-build-system"): Remove use of
  'build-system-builder'.
  ("gnu-build"): Remove 'source' and #:implicit-inputs? arguments to
  'gnu-build'.
* tests/packages.scm ("search paths"): Adjust to new build system API.
  ("package-cross-derivation, no cross builder"): Likewise.
* doc/guix.texi (Build Systems): Add paragraph on bags.
2014-10-05 21:58:42 +02:00
Ludovic Courtès bdb36958fb gnu: Split (gnu packages base), adding (gnu packages commencement).
* gnu/packages/base.scm (gnu-make-boot0, diffutils-boot0,
  findutils-boot0, %boot0-inputs, nix-system->gnu-triplet, boot-triplet,
  binutils-boot0, gcc-boot0, perl-boot0, linux-libre-headers-boot0,
  texinfo-boot0, %boot1-inputs, glibc-final-with-bootstrap-bash,
  cross-gcc-wrapper, static-bash-for-glibc, glibc-final,
  gcc-boot0-wrapped, %boot2-inputs, binutils-final, libstdc++,
  gcc-final, ld-wrapper-boot3, %boot3-inputs, bash-final, %boot4-inputs,
  guile-final, gnu-make-final, ld-wrapper, coreutils-final, grep-final,
  %boot5-inputs, %final-inputs, canonical-package, gcc-toolchain,
  gcc-toolchain-4.8, gcc-toolchain-4.9): Move to...
* gnu/packages/commencement.scm: ... here.  New file.
* gnu-system.am (GNU_SYSTEM_MODULES): Add it.
* build-aux/check-final-inputs-self-contained.scm: Adjust accordingly.
* gnu/packages/cross-base.scm: Likewise.
* gnu/packages/make-bootstrap.scm: Likewise.
* guix/build-system/cmake.scm (cmake-build): Likewise.
* guix/build-system/gnu.scm (standard-packages, gnu-build,
  gnu-cross-build): Likewise.
* guix/build-system/perl.scm (perl-build): Likewise.
* guix/build-system/python.scm (python-build): Likewise.
* guix/build-system/trivial.scm (guile-for-build): Likewise.
* guix/download.scm (url-fetch): Likewise.
* guix/gexp.scm (default-guile): Likewise.
* guix/git-download.scm (git-fetch): Likewise.
* guix/monads.scm (run-with-store): Likewise.
* guix/packages.scm (default-guile): Likewise.
* guix/scripts/package.scm (guix-package): Likewise.
* guix/scripts/refresh.scm: Likewise.
* guix/svn-download.scm (svn-fetch): Likewise.
* tests/builders.scm (%bootstrap-inputs, %bootstrap-search-paths):
  Likewise.
* tests/packages.scm ("GNU Make, bootstrap"): Likewise.
* tests/guix-package.sh: Likewise.
* gnu/services/base.scm: Use 'canonical-package' instead of xxx-final.
* gnu/services/xorg.scm: Likewise.
* gnu/system/vm.scm: Likewise.
* guix/scripts/pull.scm (guix-pull): Likewise.
2014-08-28 09:21:54 +02:00
Ludovic Courtès 6119ebf194 git-download: Rewrite using gexps.
* guix/git-download.scm (git-package): New procedure.
  (git-fetch): Use it.  Remove 'git-for-build'.
  Use a gexp and 'gexp->derivation'.
* guix/download.scm (gnutls-package): Fix docstring.
2014-08-23 22:58:09 +02:00
Ludovic Courtès 6750877f46 git-download: Support recursive clones.
* guix/git-download.scm (<git-reference>)[recursive?]: New field.
  (git-fetch): Add 'inputs' variable.  Add it to the #:inputs argument
  of 'build-expression->derivation'.  Augment builder with call to
  'set-path-environment-variable', and pass #:recursive? to
  'git-fetch'.
* guix/build/git.scm (git-fetch): Add #:recursive? parameter.  Pass
  --recursive when RECURSIVE? is true, and delete all the '.git' files.
2014-06-27 23:04:48 +02:00
Ludovic Courtès 56c72822a8 download: Perform derivations locally.
* guix/download.scm (url-fetch): Pass #:local-build? #t to
  'build-expression->derivation'.
* guix/git-download.scm (git-fetch): Likewise.
2014-03-06 21:42:24 +01:00
Ludovic Courtès 9b5b5c1740 Add (guix git-download).
* guix/git-download.scm, guix/build/git.scm: New files.
* Makefile.am (MODULES): Add them.
* guix/packages.scm (<origin>): Fix comment for 'method' field.
2014-02-21 23:49:53 +01:00