substitute: Download from unauthorized sources that provide the right content.

This allows substitutes to be downloaded from unauthorized servers, as
long as they advertise the same hash and references as one of the
authorized servers.

* guix/scripts/substitute.scm (assert-valid-narinfo): Remove.
(valid-narinfo?): Add #:verbose?.  Handle each case of
'signature-case'.
(equivalent-narinfo?): New procedure.
(lookup-narinfos/diverse): Add 'authorized?' parameter and honor it.
[select-hit]: New procedure.
(lookup-narinfo): Add 'authorized?' parameter and pass it.
(process-query): Adjust callers accordingly.
(process-substitution): Remove call to 'assert-valid-narinfo'.  Check
whether 'lookup-narinfo' returns true and call 'leave' if not.
* tests/substitute.scm (%main-substitute-directory)
(%alternate-substitute-directory): New variables.
(call-with-narinfo): Make 'narinfo-directory' a parameter.  Call
'mkdir-p' to create it.  Change unwind handler to check whether
CACHE-DIRECTORY exists before deleting it.
(with-narinfo*): New macro.
("substitute, no signature")
("substitute, invalid hash")
("substitute, unauthorized key"): Change expected error message to "no
valid substitute".
("substitute, unauthorized narinfo comes first")
("substitute, unsigned narinfo comes first")
("substitute, first narinfo is unsigned and has wrong hash")
("substitute, first narinfo is unsigned and has wrong refs")
("substitute, unsigned narinfo comes first")
("substitute, two invalid narinfos"): New tests.
* doc/guix.texi (Substitutes): Explain the new behavior.

substitute: Make substitute URLs a SRFI-39 parameter.

* guix/scripts/substitute.scm (%cache-urls): Rename to...
(%default-substitute-urls): ... this.
(substitute-urls): New variable.
(guix-substitute): Use it instead of %CACHE-URLS.
* tests/substitute.scm: Likewise.

graft: Correctly replace references near the end of the scan buffer.

Fixes <http://bugs.gnu.org/28212>.
Reported by Leo Famulari <leo@famulari.name>.

* guix/build/graft.scm (replace-store-references): When I >= END, check
whether WRITTEN > END and call 'get-bytevector-n!' when it is.
* tests/grafts.scm (buffer-size): New variable.
("replace-store-references, <http://bugs.gnu.org/28212>"): New test.

gnu: bootloader: Deprecate "device" field in favor of "target".

* gnu/bootloader.scm (<bootloader-configuration>): Deprecate "device" field in
favor of "target" field.  This is mostly a renaming but also a generalization
to support UEFI targets being paths to a mounted partition instead of a device
name.
* gnu/system/examples/bare-bones.tmpl:
* gnu/system/examples/desktop.tmpl:
* gnu/system/examples/lightweight-desktop.tmpl:
* gnu/system/examples/vm-image.tmpl:
* gnu/system/install.scm:
* gnu/tests.scm:
* gnu/tests/install.scm:
* gnu/tests/nfs.scm:
* tests/system.scm: Adapt all invocations of bootloader-configuration.
* guix/scripts/system.scm (perform-action): Rename device argument to
bootloader-target.
(process-action): Adapt caller.
* doc/guix.texi (Proceeding with the Installation):
* doc/guix.texi (Bootloader Configuration): Update documentation.

Merge branch 'master' into core-updates

guix download: Support retrieving local file without the URI scheme.

* guix/scripts/download.scm (guix-download): Treat the URL argument as a local
file path when it fails on 'string->uri'.  Call 'fetch' with the processed
'uri' instead of the original URL argument.
* tests/guix-download.sh: Adjust accordingly.

Merge branch 'master' into core-updates

gexp: 'ungexp-splicing' properly accounts for nested native inputs.

Previously, (gexp-native-inputs #~#$@(list #~#+foo)) would return '().

This is a followup to 5b14a7902c58d9fb7923f9e16871f549fbe59b6e.

* guix/gexp.scm (gexp-inputs)[add-reference-inputs]: In the list case,
remove 'if' around 'fold-right'.  In 'map' lambda, always inherit N?.
* tests/gexp.scm ("gexp list splicing + ungexp-splicing"): New test.

syscalls: Add network-interface-running?

* guix/build/syscalls.scm (network-interface-running?): New variable.
Export it.
* tests/syscalls.scm: Add test.

Co-authored-by: John Darrington <jmd@gnu.org>

Merge branch 'master' into core-updates

discovery: Recurse into directories pointed to by a symlink.

Reported by Christopher Baines <mail@cbaines.net>
and Alex Kost <alezost@gmail.com>
at <https://lists.gnu.org/archive/html/guix-devel/2017-06/msg00290.html>.

* guix/discovery.scm (scheme-files): When ENTRY is a symlink that
doesn't end in '.scm', call 'stat' and recurse if it points to a
directory.
* tests/discovery.scm ("scheme-modules recurses in symlinks to
directories"): New test.

Merge branch 'master' into core-updates

guix package: 'guix package -r PKG -u' does not upgrade PKG.

Fixes <http://bugs.gnu.org/27262>.
Reported by Mark H Weaver <mhw@netris.org>.

* guix/scripts/package.scm (transaction-upgrade-entry): Check if ENTRY
matches 'manifest-transaction-removal-candidate?' and return TRANSACTION
if it does.
(process-actions): Move 'options->removable' from step 2 to step 1.

profiles: Add 'manifest-transaction-removal-candidate?'.

* guix/profiles.scm (manifest-transaction-removal-candidate?): New
procedure.
* tests/profiles.scm ("manifest-transaction-removal-candidate?"): New
test.

Merge branch 'master' into core-updates

daemon: '--listen' can be passed several times, can specify TCP endpoints.

* nix/nix-daemon/guix-daemon.cc (DEFAULT_GUIX_PORT): New macro.
(listen_options): New variable.
(parse_opt): Push back '--listen' options to LISTEN_OPTIONS.
(open_unix_domain_socket, open_inet_socket)
(listening_sockets): New functions.
(main): Use it.  Pass SOCKETS to 'run'.
* nix/nix-daemon/nix-daemon.cc (matchUser): Remove.
(SD_LISTEN_FDS_START): Remove.
(acceptConnection): New function.
(daemonLoop): Rewrite to take a vector of file descriptors, to select(2)
on them, and to call 'acceptConnection'.
(run): Change to take a vector of file descriptors.
* tests/guix-daemon.sh: Add test.

environment: Disable profile collision checks.

Reported by Efraim Flashner.
This is a followup to a654dc4bcf7c8e205bdefa1a1d5f23444dd22778.

* guix/profiles.scm (profile-derivation): Add #:allow-collisions? and
honor it.
* guix/scripts/environment.scm (inputs->profile-derivation): Pass
 #:allow-collisions? #f to 'profile-derivation'.
* tests/guix-environment.sh: Test "guix environment guix".

profiles: Catch and report collisions in the profile.

* guix/profiles.scm (&profile-collision-error): New error condition.
(manifest-transitive-entries, manifest-entry-lookup, lower-manifest-entry)
(check-for-collisions): New procedures.
(profile-derivation): Add call to 'check-for-collisions'.
* guix/ui.scm (call-with-error-handling): Handle '&profile-collision-error'.
* tests/profiles.scm ("collision", "collision of propagated inputs")
("no collision"): New tests.

profiles: Manifest entries keep a reference to their parent entry.

* guix/profiles.scm (<manifest-entry>)[parent]: New field.
(package->manifest-entry): Add #:parent parameter.  Fill out the
'parent' field of <manifest-entry>; pass #:parent in recursive calls.
* guix/profiles.scm (sexp->manifest)[sexp->manifest-entry]: New
procedure.  Use it for version 3.
* tests/profiles.scm ("manifest-entry-parent"): New procedure.
("read-manifest")[entry->sexp]: Add 'manifest-entry-parent' to the
result.

profiles: Represent propagated inputs as manifest entries.

* guix/profiles.scm (package->manifest-entry): Turn DEPS into a list of
manifest entries.
(manifest->gexp)[entry->gexp]: Call 'entry->gexp' on DEPS.
Bump version to 3.
(sexp->manifest)[infer-dependency]: New procedure.
Use it for versions 1 and 2.  Parse version 3.
(manifest-inputs)[entry->gexp]: New procedure.
Adjust to 'dependencies' being a list of <manifest-entry>.
* tests/profiles.scm ("packages->manifest, propagated inputs")
("read-manifest"): New fields.