offload: Prevent the '.drv' and build result from being GC'd.

Before that, there was a small time window during which the GC could
wipe the .drv (before 'guix build' has been called), or the build
result (before 'retrieve-files' has started.)

* guix/scripts/offload.scm (remote-pipe): Add #:quote? parameter and
  honor it.
  (%gc-root-file): New variable.
  (register-gc-root, remove-gc-root): New procedures.
  (offload): Adjust comment.  Run 'guix build' with '-r %GC-ROOT-FILE'.
  (transfer-and-offload): Call 'register-gc-root' before
  sending (derivation-file-name DRV).  Call 'remove-gc-root' after the
  call to 'offload' or 'retrieve-files'.
  (send-files): Call 'remote-pipe' with #:quote? #f.
  (retrieve-files): Likewise.

union: Rewrite to be faster; handle symlink/directory conflicts.

* guix/build/union.scm: Rewrite; only 'file=?' remains unchanged.  Remove
  'tree-union' and 'delete-duplicate-leaves' exports.  Merge inputs in a
  breadth-first fashion.  Follow symlinks for purposes of making decisions
  about the merge.

* tests/union.scm: Remove tests of 'tree-union' and 'delete-duplicate-leaves'.

guix package: 'search-path-environment-variables' traverses module tree once.

* guix/scripts/package.scm (search-path-environment-variables)[manifest-entry->package]:
  Use 'find-best-packages-by-name' instead of 'find-packages-by-name'.
  On a profile with 140 packages, this reduces execution time of this
  procedure from 5.8 seconds to 2.9 seconds (50% improvement.)

guix package: Register non-default profiles as GC roots.

* guix/scripts/package.scm (maybe-register-gc-root): New procedure.
* tests/guix-package.sh (profile): Grep the output of "guix gc
  --list-live" in a couple of places.

pki: Keep ACL in native sexp format to speed up 'authorized-key?'.

* guix/pki.scm (acl-entry-sexp, acl-sexp): Remove.
  (public-keys->acl, current-acl): Return a native sexp.
  (acl->public-keys, authorized-key?): Expect ACL to be a native sexp.
* guix/scripts/archive.scm (authorize-key): Convert ACL to
  canonical-sexp when writing it.

substitute-binary: Avoid consing 'regexp-exec' arguments.

* guix/scripts/substitute-binary.scm (regexp-exec): Change formals
  to (rx str . rest).

substitute-binary: Avoid reloading the ACL repeatedly.

* guix/scripts/substitute-binary.scm (guix-substitute-binary) <--query>:
  Cache the result of (current-acl); pass it to 'valid-narinfo?' calls.
  This saves 12% wall-clock time for "guix build emacs -n".

Use 'signature-case' in (guix nar) and 'substitute-binary'.

* guix/nar.scm (restore-file-set)[assert-valid-signature]: Rewrite in
  terms of 'signature-case'.
* guix/scripts/substitute-binary.scm (narinfo-signature->canonical-sexp):
  Call 'leave' instead of 'raise' when SIGNATURE is invalid.
  (&nar-signature-error, &nar-invalid-hash-error): Remove.
  (assert-valid-signature): Add 'narinfo' parameter; remove 'port'.
  Rewrite in terms of 'signature-case' and 'leave'.  Mention NARINFO's
  URI in error messages.  Adjust caller.
  (narinfo-sha256): New procedure.
  (assert-valid-narinfo): Use it.
  (valid-narinfo?): Rewrite using 'narinfo-sha256' and
  'signature-case'.
* tests/substitute-binary.scm (assert-valid-signature,
  test-error-condition): Remove.
  ("corrupt signature data", "unauthorized public key", "invalid
  signature"): Remove.

pki: Add 'signature-case' macro.

* guix/pki.scm (%signature-status): New procedure.
  (signature-case): New macro.
* tests/pki.scm (%secret-key, %alternate-secret-key): New variables.
  ("signature-case valid-signature", "signature-case invalid-signature",
  "signature-case hash-mismatch", "signature-case unauthorized-key",
  "signature-case corrupt-signature"): New tests.

substitute-binary: Notify of valid signatures.

* guix/scripts/substitute-binary.scm (assert-valid-narinfo): Add
  #:verbose? parameter; when true, write "found valid signature".
  (valid-narinfo?): Pass #:verbose? #f.

guix archive: Make sure $sysconfdir/guix exists in '--authorize'.

* guix/scripts/archive.scm (authorize-key): Add 'mkdir-p' call.
  Reported by Alex Sassmannshausen <alex.sassmannshausen@gmail.com>.

offload: Exit with code 100 upon build failures.

* guix/scripts/offload.scm (transfer-and-offload): Exit with code 100
  upon build failure.

substitute-binary: Defer narinfo authentication and authorization checks.

* guix/scripts/substitute-binary.scm (narinfo-signature->canonical-sexp):
  Catch 'gcry-error' around 'string->canonical-sexp' call, and re-raise
  as a SRFI-35 &message and &nar-signature-error.
  (narinfo-maker): Handle when SIGNATURE is #f or an invalid canonical
  sexp.
  (&nar-signature-error, &nar-invalid-hash-error): New variables.
  (assert-valid-signature): Use them.  Expect 'signature' to be a
  canonical sexp.
  (read-narinfo): Remove authentication and authorization checks.
  (%signature-line-rx): New variable.
  (assert-valid-narinfo, valid-narinfo?): New procedures.
  (guix-substitute-binary): Wrap body in 'with-error-handling'.
  [valid?]: New procedure.
  <--query>: Show only store items of narinfos that match
  'valid-narinfo?'.
  <--substitute>: Call 'assert-valid-narinfo'.
* tests/substitute-binary.scm (test-error*): Use 'test-equal'.
  (%keypair): Remove.
  (%public-key, %private-key): Load from signing-key.{pub,sec}.
  (signature-body): Add #:public-key parameter.
  (call-with-narinfo): New procedure.
  (with-narinfo): New macro.
  ("corrupt signature data", "unauthorized public key", "invalid
  signature"): Make the first argument to 'assert-valid-signature' a
  canonical sexp.
  ("invalid hash", "valid read-narinfo", "valid write-narinfo"):
  Remove.
  ("query narinfo with invalid hash", "query narinfo signed with
  authorized key", "query narinfo signed with unauthorized key",
  "substitute, invalid hash", "substitute, unauthorized key"): New
  tests.

substitute-binary: Store the cache's URI in the local cached narinfo.

* guix/scripts/substitute-binary.scm (<narinfo>)[uri-base]: New field.
  (narinfo-maker): Pass CACHE-URL as the 'uri-base' value.
  (string->narinfo): Add 'cache-uri' parameter.
  (lookup-narinfo)[cache-entry]: Switch to version 1.  Add 'cache-uri'
  field.  Adjust body accordingly.
  (remove-expired-cached-narinfos): Switch to version 1 by default.

substitute-binary: Support the Signature field of a narinfo file.

* guix/scripts/substitute-binary.scm (<narinfo>): Add the 'signature'
  and 'contents' fields.
  (narinfo-signature->canonical-sexp): New function.
  (narinfo-maker): Add the 'signature' argument and use it.
  (assert-valid-signature): New function.
  (read-narinfo): Support the Signature field.
  (write-narinfo): Use 'narinfo-contents'.
  (%allow-unauthenticated-substitutes?): New variable.
* guix/base64.scm, tests/base64.scm, tests/substitute-binary.scm: New files.
* Makefile.am (SCM_TESTS): Add tests/base64.scm and
  tests/substitute-binary.scm.
  (MODULES): Add guix/base64.scm.
* test-env.in: Set 'GUIX_ALLOW_UNAUTHENTICATED_SUBSTITUTES'.

nar: Clarify that 'assert-valid-signature' accepts a string.

* guix/nar.scm (assert-valid-signature): Improve the wording.

Add (guix svn-download).

* guix/svn-download.scm, guix/build/svn.scm: New files.
* Makefile.am (MODULES): Add them.

Signed-off-by: Ludovic Courtès <ludo@gnu.org>

Merge branch 'core-updates'

offload: Remove erroneous 'close-pipe' call.

* guix/scripts/offload.scm (send-files): Remove 'close-pipe' call from
  'guard' handler ('pipe' here referred to Guile's 'pipe' procedure.)

offload: Wait for the processes involved in 'guix archive --missing'.

* guix/scripts/offload.scm (send-files): Keep the second return value of
  'filtered-port'.  Call 'waitpid' on it.