~ruther/guix-local

ref: 8a9d928f57bd37bc2ff2742bee280962ffb9bc21 guix-local/guix/scripts/offload.scm -rw-r--r-- 26.8 KiB
4b00f343 — Ludovic Courtès 12 years ago
offload: Prevent the '.drv' and build result from being GC'd.

Before that, there was a small time window during which the GC could
wipe the .drv (before 'guix build' has been called), or the build
result (before 'retrieve-files' has started.)

* guix/scripts/offload.scm (remote-pipe): Add #:quote? parameter and
  honor it.
  (%gc-root-file): New variable.
  (register-gc-root, remove-gc-root): New procedures.
  (offload): Adjust comment.  Run 'guix build' with '-r %GC-ROOT-FILE'.
  (transfer-and-offload): Call 'register-gc-root' before
  sending (derivation-file-name DRV).  Call 'remove-gc-root' after the
  call to 'offload' or 'retrieve-files'.
  (send-files): Call 'remote-pipe' with #:quote? #f.
  (retrieve-files): Likewise.
36b5851d — Ludovic Courtès 12 years ago
offload: Exit with code 100 upon build failures.

* guix/scripts/offload.scm (transfer-and-offload): Exit with code 100
  upon build failure.
07fb21b2 — Ludovic Courtès 12 years ago
offload: Remove erroneous 'close-pipe' call.

* guix/scripts/offload.scm (send-files): Remove 'close-pipe' call from
  'guard' handler ('pipe' here referred to Guile's 'pipe' procedure.)
6c41cce0 — Ludovic Courtès 12 years ago
offload: Wait for the processes involved in 'guix archive --missing'.

* guix/scripts/offload.scm (send-files): Keep the second return value of
  'filtered-port'.  Call 'waitpid' on it.
236e6648 — Ludovic Courtès 12 years ago
offload: Allow one transfer in each direction simultaneously.

* guix/scripts/offload.scm (transfer-and-offload): Use 'upload' lock
  instead of 'bandwidth' around 'send-files' calls, and 'download' lock
  around 'retrieve-files' call.
3dfd8af5 — Ludovic Courtès 12 years ago
offload: Disable SSH-level compression.

* guix/scripts/offload.scm (remote-pipe): Remove '-z' lsh command line
  argument.  This makes transfers almost an order of magnitude slower.
  OpenSSH's ssh(1) man page notes: "Compression is desirable on modem lines
  and other slow connections, but will only slow down things on fast
  networks."  See also
  <http://www.spikelab.org/blog/transfer-largedata-scp-tarssh-tarnc-compared.html>.
8b7af637 — Ludovic Courtès 12 years ago
offload: Compress files being sent/retrieved.

* guix/scripts/offload.scm (send-files): Add "xz -dc |" to the remote
  pipe command.  Pass PIPE through 'call-with-compressed-output-port'.
  Remove 'close-pipe' call.
  (retrieve-files): Add "| xz -c" to the remote pipe command.  Pass PIPE
  through 'call-with-decompressed-port'.  Remove 'close-pipe' call.
1eefbb26 — Mark H Weaver 12 years ago
Merge branch 'master' into core-updates
d81195bf — Ludovic Courtès 12 years ago
offload: Send build logs to file descriptor 4.

* guix/scripts/offload.scm (with-error-to-port): New macro.
  (remote-pipe): Add #:error-port parameter.  Use 'with-error-to-port'
  around 'open-pipe*' call.
  (build-log-port): New procedure.
  (offload): Change #:log-port to default to (build-log-port).  Call
  'remote-pipe' with #:error-port LOG-PORT.
0562dbe5 — Ludovic Courtès 12 years ago
Merge branch 'master' into core-updates
1a8ea0a1 — Ludovic Courtès 12 years ago
offload: Fix 'choose-build-machine' for several machines.

* guix/scripts/offload.scm (choose-build-machine)[undecorate]: Turn into
  a two-argument procedure.
3c0e6e60 — Ludovic Courtès 12 years ago
offload: Convert the port number to a string when invoking lsh.

* guix/scripts/offload.scm (remote-pipe, send-files): Pass the result of
  'build-machine-port' to 'number->string'.
cecd72d5 — Ludovic Courtès 12 years ago
offload: Allow build machines to specify a port number.

* guix/scripts/offload.scm (<build-machine>): Add 'port' field.
  (remote-pipe, send-files): Use lsh's '-p' option when invoking it.
e06f7865 — Ludovic Courtès 12 years ago
Merge branch 'master' into core-updates
714084e6 — Ludovic Courtès 12 years ago
offload: Honor absolute build timeouts.

* guix/scripts/offload.scm (offload): Remove default value for
  'build-timeout'.  Pass '--timeout' to the remote 'guix build'
  process.
  (transfer-and-offload, process-request): Remove default value for
  'build-timeout'.
59f704df — Ludovic Courtès 12 years ago
offload: Move macro definitions before use.

* guix/scripts/offload.scm (lock-file, unlock-file, with-file-lock,
  with-machine-lock, machine-slot-file, acquire-build-slot,
  release-build-slot): Move definitions above their first use.
88da0b68 — Ludovic Courtès 12 years ago
offload: Distinguish between 'decline' and 'postpone'.

* guix/scripts/offload.scm (transfer-and-offload): New procedure, with
  core formerly in 'process-request'.
  (choose-build-machine): Remove 'requirements' parameter.
  (process-request): Reply 'decline' when none of MACHINES matches the
  requirements, and 'postpone' when MACHINES are busy.
5cc569dc — Ludovic Courtès 12 years ago
offload: Prevent locked files from being GC'd.

* guix/scripts/offload.scm (%slots): New variable.
  (choose-build-machine): Add SLOT to '%slots'.
7df3ade1 — Ludovic Courtès 12 years ago
offload: Fix thinko.

* guix/scripts/offload.scm (choose-build-machine)[machine+slots]: Use
  'filter-map', not 'filter'.
d652b851 — Ludovic Courtès 12 years ago
offload: Make 'parallel-builds' a hard limit.

* guix/scripts/offload.scm (machine-choice-lock-file,
  machine-slot-file, acquire-build-slot, release-build-slot): New
  procedures.
  (choose-build-machine): Operate with (machine-choice-lock-file)
  taken.  Acquire a build slot for each of MACHINES.  Release those not
  used.
Next