guix-devel/guix
Christopher Baines f135b4ae83
git-download: Speed up 'git-predicate'.
Adjust 'git-predicate' to use data structures that perform better when used
with git repositories with a large number of files.

Previously when matching either a regular file or directory, 'git-predicate'
would search a list with a length equal to the number of files in the
repository. As a search operation happens for roughly every file in the
repository, this meant that the time taken to use 'git-predicate' to traverse
all the files in a repository was roughly exponential with respect to the
number of files in the repository.

Now, for matching regular files or symlinks, 'git-predicate' uses a vhash
using the inode value as the key. This should perform roughly in constant
amount of time, instead of linear with respect to the number of files in the
repository.

For matching directories, 'git-predicate' now uses a tree structure stored in
association lists. To check if a directory is in the tree, the tree is
traversed from the root. The time complexity of this depends on the shape of
the tree, but it should be an improvement on searching through the list of all
files.

* guix/git-download.scm (files->directory-tree, directory-in-tree?): New
procedures.
(git-predicate): Compute DIRECTORY-TREE.  Turn INODES into a vhash.
Adjust body of lambda accordingly.

Co-authored-by: Ludovic Courtès <ludo@gnu.org>
2017-07-25 23:24:16 +02:00
..
build syscalls: Add network-interface-running? 2017-07-12 16:15:30 +02:00
build-system build-system: texlive: Build union in configure phase. 2017-07-09 17:07:27 +02:00
import import: cpan: Updater returns a list of URLs. 2017-07-24 17:35:26 +02:00
scripts offload: Fix potential file descriptor and memory leak. 2017-07-25 23:24:16 +02:00
store store: Support 'ssh://' URIs in 'GUIX_DAEMON_SOCKET'. 2017-04-21 19:17:36 +02:00
tests tests: Move HTTP server to (guix tests http). 2016-11-16 18:00:41 +01:00
base16.scm utils: Move base16 procedures to (guix base16). 2017-03-16 22:50:14 +01:00
base32.scm base32: Export the base32 charsets. 2017-07-21 17:03:24 +02:00
base64.scm base64: Turn into a regular Guile module. 2017-05-25 14:25:17 +02:00
build-system.scm build-system: Bags record their system and target. 2014-10-05 21:58:42 +02:00
cache.scm cache: Work around 'time-monotonic' bug in Guile 2.2.2. 2017-04-22 14:42:15 +02:00
combinators.scm Add (guix memoization). 2017-01-28 18:55:20 +01:00
config.scm.in pull: Install (guix config) module to override the user's one. 2016-07-20 22:53:02 +02:00
cpio.scm linux-initrd: Produce cpio archives with zeroed timestamps, etc. 2015-06-24 18:05:03 +02:00
cve.scm cve: Use a more compact format for the list of package/versions. 2016-05-28 01:07:12 +02:00
cvs-download.scm download: Use 'with-imported-modules'. 2016-07-12 22:47:08 +02:00
derivations.scm derivations: Introduce 'read-derivation-from-file'. 2017-06-12 17:53:51 +02:00
discovery.scm discovery: Recurse into directories pointed to by a symlink. 2017-07-03 23:51:23 +02:00
docker.scm pack: Add '--target'. 2017-03-18 00:38:41 +01:00
download.scm download: Add OpenBSD mirrors. 2017-07-19 01:42:08 +02:00
elf.scm elf: Add missing argument in 'elf-segment'. 2015-03-30 15:21:11 +02:00
ftp-client.scm ftp-client: Allow custom username and password for FTP servers. 2017-03-21 12:14:05 +01:00
gcrypt.scm Add (guix gcrypt). 2015-01-26 21:58:12 +01:00
gexp.scm gexp: Slightly improve error reporting for 'local-file'. 2017-07-20 18:08:00 +02:00
git-download.scm git-download: Speed up 'git-predicate'. 2017-07-25 23:24:16 +02:00
git.scm guix: git: Stop using libgit2-shutdown. 2017-07-01 16:32:42 +02:00
gnu-maintenance.scm gnu-maintenance: 'gnu-package?' uses 'eq?' memoization. 2017-01-28 18:55:20 +01:00
gnupg.scm ui: Rename '_' to 'G_'. 2017-05-03 16:16:17 +02:00
grafts.scm derivations: Introduce 'read-derivation-from-file'. 2017-06-12 17:53:51 +02:00
graph.scm graph: Add Cypher backend. 2017-05-11 16:17:49 +02:00
hash.scm Use (ice-9 binary-ports) instead of (rnrs io ports). 2016-10-19 15:54:10 +02:00
hg-download.scm download: Use 'with-imported-modules'. 2016-07-12 22:47:08 +02:00
http-client.scm ui: Rename '_' to 'G_'. 2017-05-03 16:16:17 +02:00
licenses.scm licenses: Add MirOS license. 2017-07-20 19:55:21 +05:30
memoization.scm memoization: Micro-optimize code produced by 'define-cache-procedure'. 2017-03-16 22:50:15 +01:00
modules.scm modules: Raise an error when a dependency could not be found. 2017-05-25 14:25:17 +02:00
monad-repl.scm monad-repl: Close connection when leaving the monad REPL. 2016-01-13 17:43:21 +01:00
monads.scm monads: Add a template and specialization mechanism for monadic procedures. 2017-05-02 23:41:13 +02:00
nar.scm ui: Rename '_' to 'G_'. 2017-05-03 16:16:17 +02:00
packages.scm packages: Mark 'replacement' as an "innate" field. 2017-07-03 23:51:22 +02:00
pk-crypto.scm publish: The public and private keys are now SRFI-39 parameters. 2017-03-22 14:05:59 +01:00
pki.scm Use (ice-9 binary-ports) instead of (rnrs io ports). 2016-10-19 15:54:10 +02:00
profiles.scm profiles: Remove workaround for an old Guile 'scandir' bug. 2017-07-20 11:57:13 +02:00
records.scm records: Slight simplification. 2017-04-17 23:31:28 +02:00
scripts.scm scripts: Do not create the config directory. 2017-05-13 00:03:46 +02:00
search-paths.scm search-paths: Allow specs with #f as their separator. 2017-01-23 22:23:41 +01:00
serialization.scm serialization: Remove Guile < 2.0.9 workaround. 2017-04-17 23:31:28 +02:00
sets.scm Add (guix sets). 2015-01-11 23:50:01 +01:00
ssh.scm ssh: Improve error reporting when 'send-files' fails. 2017-06-04 23:00:32 +02:00
store.scm store: Rewrite 'store-path-hash-part' to not use regexps. 2017-07-21 17:03:25 +02:00
svn-download.scm guix: Add download-svn-to-store. 2017-06-15 17:03:14 +02:00
tests.scm store: 'GUIX_DAEMON_SOCKET' can now be a URI. 2017-04-21 17:23:37 +02:00
ui.scm ui: package->recutlis: Remove duplicated package names in dependencies. 2017-07-02 13:14:16 +08:00
upstream.scm ui: Rename '_' to 'G_'. 2017-05-03 16:16:17 +02:00
utils.scm utils: 'compressed-file?' matches ".lzma" files. 2017-05-15 23:55:10 +02:00
workers.scm scripts: Set thread names. 2017-05-28 23:13:39 +02:00
zlib.scm zlib: Don't rely on EBADF being ignored by 'fport_close'. 2017-03-15 15:19:53 +01:00