Commit Graph

15 Commits

Author SHA1 Message Date
Ludovic Courtès 00753f7038
publish: Add '--cache' and '--workers'.
Fixes <http://bugs.gnu.org/26201>.
Reported by <dian_cecht@zoho.com>.

These options allow nars to be "baked" off-line and cached instead of
being compressed on the fly.  As a side-effect, this allows us to
provide a 'Content-Length' header for nars.

* guix/scripts/publish.scm (show-help, %options): Add '--cache' and
'--workers'.
(%default-options): Add 'workers'.
(nar-cache-file, narinfo-cache-file, run-single-baker): New procedures.
(single-baker): New macro.
(render-narinfo/cached, bake-narinfo+nar)
(render-nar/cached): New procedures.
(make-request-handler): Add #:cache and #:pool parameters and honor
them.
(run-publish-server): Likewise.
(guix-publish): Honor '--cache' and '--workers'.
* tests/publish.scm ("with cache"): New test.
* doc/guix.texi (Invoking guix publish): Document it.
2017-04-18 23:18:41 +02:00
Ludovic Courtès cdd7a7d210
publish: Make the nar URL prefix a parameter.
* guix/scripts/publish.scm (narinfo-string): Add #:nar-path and honor it.
(render-narinfo): Likewise.
(make-request-handler): Likewise.
(run-publish-server): Likewise.
* tests/publish.scm ("custom nar path"): New test.
2017-03-22 14:05:59 +01:00
Ludovic Courtès ab2a74e4db
publish: The public and private keys are now SRFI-39 parameters.
* guix/scripts/publish.scm (%default-options): Add 'public-key-file' and
'private-key-file'.
(lazy-read-file-sexp): Remove.
(%private-key, %public-key): Turn into SRFI-39 parameters.
(signed-string, render-narinfo): Adjust accordingly.
(guix-publish): Honor 'public-key-file' and 'private-key-file' from
OPTS.  Use 'parameterize'.
* guix/pk-crypto.scm (read-file-sexp): New procedure.
* tests/publish.scm: Initialize '%public-key' and '%private-key'.
2017-03-22 14:05:59 +01:00
Ludovic Courtès a5c376034f
tests: Make sure threads use separate output ports.
* tests/publish.scm (with-separate-output-ports): New macro.
<top level>: Use it when spawning new thread.
("/*.narinfo with compression"): Likewise.
2016-08-02 18:07:19 +02:00
Ludovic Courtès 2c7b48c2fb
tests: Work around Guile bug with unbuffered custom binary input ports.
Reported by Chris Marusich <cmmarusich@gmail.com>
at <http://debbugs.gnu.org/cgi/bugreport.cgi?bug=24060#19>

* tests/publish.scm (http-get-port): Remove 'setvbuf' call for the
response port.
2016-08-02 18:07:19 +02:00
Ludovic Courtès 089b167812
publish: Do not compress already-compressed files.
* guix/scripts/publish.scm (narinfo-string): Force %NO-COMPRESSION when
STORE-PATH matches 'compressed-file?'.
* guix/utils.scm (compressed-file?): New procedure.
* tests/publish.scm ("/*.narinfo for a compressed file"): New test.
2016-08-01 17:58:56 +02:00
Ludovic Courtès 37402ecb43
tests: Use unbuffered input ports when decompressing from 'guix publish'.
Fixes <http://bugs.gnu.org/24060>.
Reported by Chris Marusich <cmmarusich@gmail.com>.

* tests/publish.scm (http-get-port): Explicitly call
'open-socket-for-uri' and add calls to 'setvbuf'.
2016-07-27 12:45:01 +02:00
Ludovic Courtès ff6638d112
publish: Handle '/file' URLs, for content-addressed files.
* guix/scripts/publish.scm (render-content-addressed-file): New procedure.
(http-write): Add 'application/octet-stream' case.
(make-request-handler): Add /file/NAME/sha256/HASH URLs.
* tests/publish.scm ("/file/NAME/sha256/HASH")
("/file/NAME/sha256/INVALID-NIX-BASE32-STRING")
("/file/NAME/sha256/INVALID-HASH"): New tests.
* doc/guix.texi (Invoking guix publish): Mention the /file URLs.
2016-07-20 16:54:31 +02:00
Ludovic Courtès 4a1fc562ae
publish: Add '--compression'.
* guix/scripts/publish.scm (show-help, %options): Add '--compression'.
(<compression>): New record type.
(%no-compression, %default-gzip-compression): New variables.
(%default-options): Add 'compression' key.
(narinfo-string): Add #:compression parameter and honor it.
(render-narinfo): Likewise.
(render-nar): Likewise.
<top level>: Add call to 'declare-header!'.
(swallow-zlib-error): New macro.
(nar-response-port): New procedure.
(http-write): Add call to 'force-output'.  Use 'nar-response-port'
instead of 'response-port'.  Use 'swallow-zlib-error'.
(make-request-handler): Add #:compression parameter and honor it.  Add
"nar/gzip" URL handler.
(run-publish-server): Add #:compression parameter and honor it.
(guix-publish): Honor --compression.
* tests/publish.scm (http-get-port, wait-until-ready): New procedures.
<top level>: Run main server with "-C0".  Call 'wait-until-ready'.
("/nar/gzip/*", "/*.narinfo with compression"): New tests.
* doc/guix.texi (Invoking guix publish): Document it.
2016-07-19 00:07:12 +02:00
Ludovic Courtès 93961f0298
publish: Encore URIs that appear in narinfos.
Fixes <http://bugs.gnu.org/21888>.
Reported by iyzsong@member.fsf.org (宋文武).

* guix/scripts/publish.scm (narinfo-string): Use
'encode-and-join-uri-path' instead of 'string-append' to compute URL.
* tests/publish.scm ("/*.narinfo with properly encoded '+' sign"):
("/nar/ with properly encoded '+' sign"): New tests.
2016-06-08 19:07:58 +02:00
Mathieu Lirzin a9edb211e7 build: Add a Guile custom test driver using SRFI-64.
Before that '.log' files for scheme tests were fragmented and not
included in test-suite.log.  This unifies the semantics of SRFI-64 API
with Automake test suite.

* build-aux/test-driver.scm: New file.
* Makefile.am (SCM_LOG_DRIVER, AM_SCM_LOG_DRIVER_FLAGS): New variables.
(SCM_LOG_COMPILER, AM_SCM_LOG_FLAGS): Delete variables.
(AM_TESTS_ENVIRONMENT): Set GUILE_AUTO_COMPILE to 0.
* test-env.in: Silence guix-daemon.
* doc/guix.texi (Running the Test Suite): Describe how to display the
detailed results.  Bug reports require only 'test-suite.log' file.
* tests/base32.scm, tests/build-utils.scm, tests/builders.scm,
tests/challenge.scm, tests/cpan.scm, tests/cpio.scm, tests/cran.scm,
tests/cve.scm, tests/derivations.scm, tests/elpa.scm,
tests/file-systems.scm, tests/gem.scm, tests/gexp.scm,
tests/gnu-maintenance.scm, tests/grafts.scm, tests/graph.scm,
tests/gremlin.scm, tests/hackage.scm, tests/hash.scm,
tests/import-utils.scm, tests/lint.scm, tests/monads.scm, tests/nar.scm,
tests/packages.scm, tests/pk-crypto.scm, tests/pki.scm,
tests/profiles.scm, tests/publish.scm, tests/pypi.scm,
tests/records.scm, tests/scripts-build.scm, tests/scripts.scm,
tests/services.scm, tests/sets.scm, tests/size.scm, tests/snix.scm,
tests/store.scm, tests/substitute.scm, tests/syscalls.scm,
tests/system.scm, tests/ui.scm, tests/union.scm, tests/upstream.scm,
tests/utils.scm: Don't exit at the end of test groups.
* tests/containers.scm: Likewise.  Use 'test-skip' instead of exiting
with error code 77.
2016-04-03 14:19:09 +02:00
Ludovic Courtès 0043558082 publish: Do not publish nars for invalid store items.
Before that, /nar requests could succeed if the requested store item
exists but is invalid (although such requests were unlikely because the
corresponding narinfo request would have failed.)

* guix/scripts/publish.scm (render-nar): Add 'store' parameter.  Use
'valid-path?' instead of 'file-exists?'.
(make-request-handler): Adjust 'render-nar' call accordingly.
* tests/publish.scm ("/nar/invalid"): New test.
2016-02-25 12:44:41 +01:00
Ludovic Courtès c74f0cb2b8 tests: Prevent 'http_proxy' from breaking Web server tests.
* tests/lint.scm: Add call to 'unsetenv' to remove 'http_proxy'.
* tests/publish.scm: Likewise.
2016-01-22 00:02:52 +01:00
Alex Kost 9cced52610 tests: Fix 'publish' test for nix-base32 format.
This is a followup to commit 3b307162e8.

* tests/publish.scm ("/*.narinfo"): Use 'bytevector->nix-base32-string',
  not 'bytevector->base32-string'.
2015-07-18 17:05:28 +03:00
David Thompson aff8ce7c74 scripts: Add 'publish' command.
* guix/scripts/publish.scm: New file.
* po/guix/POTFILES.in: Add it.
* tests/publish.scm: New file.
* Makefile.am (MODULES): Add script module.
  (SCM_TESTS): Add test module.
* doc/guix.texi ("Invoking guix publish"): New node.
2015-04-04 14:16:43 -04:00