Merge branch 'master' into core-updates

offload: '{send,receive}-files' wait for completion of the transfer.

Fixes situations where the remote 'guix build' is invoked before the
.drv has been completely copied, as reported at
<https://lists.gnu.org/archive/html/guix-devel/2014-04/msg00174.html>.

In some cases 'send-files' would return before the other end is done
importing the files, and so the subsequent 'guix build' invocation would
just miss the .drv file it refers to.

* guix/utils.scm (call-with-decompressed-port): Don't close PORT.
  (call-with-compressed-output-port): Likewise.
* tests/utils.scm ("compressed-output-port + decompressed-port"): Adjust
  accordingly.
* guix/scripts/offload.scm (send-files): Add explicit (close-pipe pipe)
  call.
  (retrieve-files): Likewise.

tests: Avoid buffering in 'fcntl-flock' tests.

Partially fixes <http://bugs.gnu.org/17212>.

* tests/utils.scm ("fcntl-flock wait", "fcntl-flock non-blocking"): Open
  files in binary mode, with no buffering.

utils: Add 'call-with-decompressed-port' and 'call-with-compressed-output-port'.

* guix/utils.scm (call-with-decompressed-port,
  call-with-compressed-output-port): New procedures.
* tests/utils.scm ("compressed-output-port + decompressed-port"):
  Rewrite to use them.

utils: Add 'filtered-output-port' and 'compressed-output-port'.

* guix/utils.scm (filtered-output-port, compressed-output-port): New
  procedures.
* tests/utils.scm ("compressed-output-port + decompressed-port"): New
  test.

utils: Add 'decompressed-port' and 'compressed-port'.

* guix/utils.scm (decompressed-port, compressed-port): New procedures.
* guix/scripts/substitute-binary.scm (decompressed-port): Remove.
  (guix-substitute-binary): Pass a symbol or #f as the first argument to
  'decompress-port'.
* tests/utils.scm ("compressed-port, decompressed-port, non-file"): New
  test.

utils: 'filtered-port' doesn't leave dangling processes behind.

* guix/utils.scm (filtered-port): Make sure the 'execl' child process
  always exits, and does (primitive-_exit 1) upon execution failure.
  Use 'primitive-_exit' in the 'dump-port' child process.
* tests/utils.scm ("filtered-port, does not exist"): New test.

Merge branch 'master' into core-updates

utils: Add a non-blocking option for 'fcntl-flock'.

* guix/utils.scm (F_SETLK): New variable.
  (fcntl-flock): Add 'wait?' keyword parameter; honor it.
* tests/utils.scm ("fcntl-flock non-blocking"): New test.

tests: Rewrite 'fcntl-lock' test.

* tests/utils.scm (temp-file): New variable.
  ("fcntl-flock"): Rewrite to actually test whether the child process
  waits for the lock to be released.  The previous test was wrong
  because (1) it expected F_SETLK semantics, not F_SETLKW, and (2) it
  got EBADF because of a mismatch between the open mode and the lock
  style.

Merge branch 'master' into core-updates

utils: Add 'fcntl-flock'.

* guix/utils.scm (%struct-flock, F_SETLKW, F_xxLCK): New variables.
  (fcntl-flock): New procedure.
* tests/utils.scm ("fcntl-flock"): New test.

Merge branch 'master' into core-updates

Conflicts:
	guix/packages.scm

utils: Add 'string-replace-substring'.

* guix/utils.scm (string-replace-substring): New procedure.  Based on
  code by Mark H. Weaver.
* tests/utils.scm ("string-replace-substring"): New test.

utils: Add `guile-version>?', and use it.

This fixes Guile version comparisons when (version) has a
vendor-specific suffix.

Reported by Andreas Enge <andreas@enge.fr>.

* guix/utils.scm (guile-version>?): New procedure.
* tests/utils.scm ("guile-version>? 1.8", "guile-version>? 10.5"): New
  tests.
* guix/scripts/substitute-binary.scm (fetch, progress-report-port): Use
  `guile-version>?' instead of `version>?'.
* guix/http-client.scm (when-guile<=2.0.5, http-fetch): Likewise.

substitute-binary: Try hard to avoid port buffering.

* guix/scripts/substitute-binary.scm (fetch): In the `file' case, open
  with the `b' flag, so that the coding cookie reading thing doesn't
  lead to buffering some of the data (on 2.0.5).
* tests/utils.scm ("filtered-port, file"): Open with `r0b'.  Fixes a
  test failure with Guile 2.0.5 whereby the first byte of FILE would be
  missing from DECOMPRESSED.

substitute-binary: Pass `filtered-port' an unbuffered port.

This fixes a bug whereby `read-response' would read more than just the
response, with the extra data going into the port's buffer; the
"bzip2 -dc" process spawned by `filtered-port' would not see the those
buffered data, which are definitely lost, and would bail out with
"bzip2: (stdin) is not a bzip2 file."

* guix/utils.scm (filtered-port): Document that INPUT must be
  unbuffered.
* guix/web.scm (http-fetch): Add `buffered?' parameter.  Call
  `open-socket-for-uri' explicitly, and call `setvbuf' when BUFFERED? is
  false.  Pass the port to `http-get'.  Close it upon 301/302.
* guix/scripts/substitute-binary.scm (fetch): Add `buffered?'
  parameter.  Pass it to `http-fetch'; honor it for `file' URIs.
  (guix-substitute-binary): Call `fetch' with #:buffered? #f for port RAW.
* tests/utils.scm ("filtered-port, file"): Open FILE as unbuffered.

Move record utilities to (guix records).

* guix/utils.scm (define-record-type*): Move to...
* guix/records.scm: ... here.  New file.
* guix/build-system.scm, guix/packages.scm: Use it.
* guix/gnu-maintenance.scm: Likewise.
  (official-gnu-packages)[alist->record]: Remove.
* guix/scripts/substitute-binary.scm: Likewise.
  (alist->record, object->fields): Remove.
* tests/utils.scm ("define-record-type*", "define-record-type* with
  letrec* behavior", "define-record-type* & inherit",
  "define-record-type* & inherit & letrec* behavior",
  "define-record-type* & thunked", "define-record-type* & thunked &
  default", "define-record-type* & thunked & inherited"): Move to...
* tests/records.scm: ... here.  New file.

Merge branch 'core-updates'

substitute-binary: Support decompression from non-file ports.

* guix/scripts/substitute-binary.scm (filtered-port): Move to utils.scm.
  (decompressed-port): Upon "none", return '() as the second value.
  (guix-substitute-binary): Expect `decompressed-port' to return a list
  of PIDs as its second value.
* guix/utils.scm (filtered-port): New procedure.  Add case for when
  INPUT is not `file-port?'.
* tests/utils.scm ("filtered-port, file", "filtered-port, non-file"):
  New tests.