offload: Ignore EEXIST when registering a .drv as a GC root.

Fixes <http://bugs.gnu.org/18115>.
Reported by Mark H Weaver <mhw@netris.org>.

* guix/scripts/offload.scm (register-gc-root)[script]: Wrap 'symlink'
  call in "catch 'system-error", and ignore EEXIST errors.

offload: Ignore unreachable machines.

Fixes <http://bugs.gnu.org/18070>.
Reported by Andreas Enge <andreas@enge.fr>.

* guix/scripts/offload.scm (remote-pipe): Augment docstring.
  (machine-load): Return +inf.0 instead of 1 if MACHINE does not respond
  or responds badly.

guix {system,offload}: Improve reporting of syntax errors.

* guix/scripts/system.scm (read-operating-system) <catch handler>: Add
  case for 'syntax-error'.  Correct message for default case.
* guix/scripts/offload.scm (build-machines) <catch handler>: Add case
  for 'syntax-error'.
* tests/guix-system.sh: New file.
* Makefile.am (SH_TESTS): Add it.

Merge branch 'master' into core-updates

offload: '{send,receive}-files' wait for completion of the transfer.

Fixes situations where the remote 'guix build' is invoked before the
.drv has been completely copied, as reported at
<https://lists.gnu.org/archive/html/guix-devel/2014-04/msg00174.html>.

In some cases 'send-files' would return before the other end is done
importing the files, and so the subsequent 'guix build' invocation would
just miss the .drv file it refers to.

* guix/utils.scm (call-with-decompressed-port): Don't close PORT.
  (call-with-compressed-output-port): Likewise.
* tests/utils.scm ("compressed-output-port + decompressed-port"): Adjust
  accordingly.
* guix/scripts/offload.scm (send-files): Add explicit (close-pipe pipe)
  call.
  (retrieve-files): Likewise.

offload: Better synchronize with remote invocation of 'guix archive --missing'.

* guix/scripts/offload.scm (send-files)[missing-files]: Call 'waitpid'
  after reading all of MISSING.

offload: Remove all the GC roots in case of multiple-output derivations.

* guix/scripts/offload.scm (remove-gc-root): Rename to...
  (remove-gc-roots): ... this.
  [builder]: Use 'scandir' and remove all the files starting with
  %GC-ROOT-FILE.
  (transfer-and-offload): Adjust to renaming; remove
  'false-if-exception' wraps.

offload: Bail out when failing to register a GC root on the build machine.

* guix/scripts/offload.scm (register-gc-root): Call 'leave' when
  'close-pipe' returns non-zero.

offload: Prevent the '.drv' and build result from being GC'd.

Before that, there was a small time window during which the GC could
wipe the .drv (before 'guix build' has been called), or the build
result (before 'retrieve-files' has started.)

* guix/scripts/offload.scm (remote-pipe): Add #:quote? parameter and
  honor it.
  (%gc-root-file): New variable.
  (register-gc-root, remove-gc-root): New procedures.
  (offload): Adjust comment.  Run 'guix build' with '-r %GC-ROOT-FILE'.
  (transfer-and-offload): Call 'register-gc-root' before
  sending (derivation-file-name DRV).  Call 'remove-gc-root' after the
  call to 'offload' or 'retrieve-files'.
  (send-files): Call 'remote-pipe' with #:quote? #f.
  (retrieve-files): Likewise.

offload: Exit with code 100 upon build failures.

* guix/scripts/offload.scm (transfer-and-offload): Exit with code 100
  upon build failure.

offload: Remove erroneous 'close-pipe' call.

* guix/scripts/offload.scm (send-files): Remove 'close-pipe' call from
  'guard' handler ('pipe' here referred to Guile's 'pipe' procedure.)

offload: Wait for the processes involved in 'guix archive --missing'.

* guix/scripts/offload.scm (send-files): Keep the second return value of
  'filtered-port'.  Call 'waitpid' on it.

offload: Allow one transfer in each direction simultaneously.

* guix/scripts/offload.scm (transfer-and-offload): Use 'upload' lock
  instead of 'bandwidth' around 'send-files' calls, and 'download' lock
  around 'retrieve-files' call.

offload: Disable SSH-level compression.

* guix/scripts/offload.scm (remote-pipe): Remove '-z' lsh command line
  argument.  This makes transfers almost an order of magnitude slower.
  OpenSSH's ssh(1) man page notes: "Compression is desirable on modem lines
  and other slow connections, but will only slow down things on fast
  networks."  See also
  <http://www.spikelab.org/blog/transfer-largedata-scp-tarssh-tarnc-compared.html>.

offload: Compress files being sent/retrieved.

* guix/scripts/offload.scm (send-files): Add "xz -dc |" to the remote
  pipe command.  Pass PIPE through 'call-with-compressed-output-port'.
  Remove 'close-pipe' call.
  (retrieve-files): Add "| xz -c" to the remote pipe command.  Pass PIPE
  through 'call-with-decompressed-port'.  Remove 'close-pipe' call.

Merge branch 'master' into core-updates

offload: Send build logs to file descriptor 4.

* guix/scripts/offload.scm (with-error-to-port): New macro.
  (remote-pipe): Add #:error-port parameter.  Use 'with-error-to-port'
  around 'open-pipe*' call.
  (build-log-port): New procedure.
  (offload): Change #:log-port to default to (build-log-port).  Call
  'remote-pipe' with #:error-port LOG-PORT.

Merge branch 'master' into core-updates

offload: Fix 'choose-build-machine' for several machines.

* guix/scripts/offload.scm (choose-build-machine)[undecorate]: Turn into
  a two-argument procedure.

offload: Convert the port number to a string when invoking lsh.

* guix/scripts/offload.scm (remote-pipe, send-files): Pass the result of
  'build-machine-port' to 'number->string'.