substitute-binary: Try hard to avoid port buffering.

* guix/scripts/substitute-binary.scm (fetch): In the `file' case, open
  with the `b' flag, so that the coding cookie reading thing doesn't
  lead to buffering some of the data (on 2.0.5).
* tests/utils.scm ("filtered-port, file"): Open with `r0b'.  Fixes a
  test failure with Guile 2.0.5 whereby the first byte of FILE would be
  missing from DECOMPRESSED.

substitute-binary: Don't pretend to report download progress on Guile 2.0.5.

* guix/scripts/substitute-binary.scm (progress-report-port): On Guile
  2.0.5, return PORT directly and emit a warning.

Rename (guix web) to (guix http-client).

* guix/web.scm: Rename to...
* guix/http-client.scm: ... this.
* guix/gnu-maintenance.scm, guix/scripts/substitute-binary.scm,
  Makefile.am, po/POTFILES.in: Update accordingly.

substitute-binary: Directly replace the global `regexp-exec'.

* guix/scripts/substitute-binary.scm (%regexp-exec-mutex, string->uri):
  Remove.
  (regexp-exec): Replace this global binding by a thread-safety wrapper.
  (fields->alist): Remove `with-mutex', and directly alias `recutils->alist'.

substitute-binary: Increase the default timeout.

* guix/scripts/substitute-binary.scm (%fetch-timeout): Set to 5 seconds.

records: Add `recutils->alist' for public consumption.

* guix/records.scm (%recutils-field-rx): New variable.
  (recutils->alist): New procedure, formerly known as `fields->alist'.
* guix/scripts/substitute-binary.scm (fields->alist): Use it.
* tests/records.scm ("recutils->alist"): New test.

substitute-binary: Avoid dangling connections to the server.

* guix/web.scm (open-socket-for-uri): New procedure.
  (http-fetch): Add `port' keyword parameter; use it.
* guix/scripts/substitute-binary.scm (%random-state): New variable.
  (with-timeout): Wait a little before retrying.
  (fetch): Use `open-socket-for-uri', and keep a copy of the socket in
  variable `port'.  Close PORT upon timeout.

Merge branch 'master' into core-updates

Conflicts:
	gnu/packages/make-bootstrap.scm

substitute-binary: Report progress while downloading.

* guix/scripts/substitute-binary.scm (decompressed-port): Improve docstring.
  (progress-report-port): New procedure.
  (guix-substitute-binary)["--substitute"]: Use it to report progress.
* guix/build/download.scm: Export `progress-proc' and `uri-abbreviation'.

substitute-binary: Provide feedback when the server is unresponsive.

* guix/scripts/substitute-binary.scm (%fetch-timeout): New variable.
  (with-timeout): New macro.
  (fetch): Add `timeout?' keyword parameter.  Enclose `http-fetch' call
  in `with-timeout'.
  (guix-substitute-binary): Call `fetch' with #:timeout? #f.

substitute-binary: Don't cache .narinfo lookups when lacking networking.

* guix/scripts/substitute-binary.scm (lookup-narinfo): Don't cache
  NARINFO when CACHE is #f.

substitute-binary: Gracefully exit upon networking errors.

Suggested by Andreas Enge <andreas@enge.fr>.

* guix/scripts/substitute-binary.scm (with-networking): New macro.
  (guix-substitute-binary): Wrap the body in `with-networking'.

substitute-binary: Pass `filtered-port' an unbuffered port.

This fixes a bug whereby `read-response' would read more than just the
response, with the extra data going into the port's buffer; the
"bzip2 -dc" process spawned by `filtered-port' would not see the those
buffered data, which are definitely lost, and would bail out with
"bzip2: (stdin) is not a bzip2 file."

* guix/utils.scm (filtered-port): Document that INPUT must be
  unbuffered.
* guix/web.scm (http-fetch): Add `buffered?' parameter.  Call
  `open-socket-for-uri' explicitly, and call `setvbuf' when BUFFERED? is
  false.  Pass the port to `http-get'.  Close it upon 301/302.
* guix/scripts/substitute-binary.scm (fetch): Add `buffered?'
  parameter.  Pass it to `http-fetch'; honor it for `file' URIs.
  (guix-substitute-binary): Call `fetch' with #:buffered? #f for port RAW.
* tests/utils.scm ("filtered-port, file"): Open FILE as unbuffered.

substitute-binary: Work around thread-unsafe `regexp-exec'.

* guix/scripts/substitute-binary.scm (%regexp-exec-mutex): New variable.
  (string->uri): New procedure.
  (fields->alist): Wrap `regexp-exec' call in `with-mutex'.

Move record utilities to (guix records).

* guix/utils.scm (define-record-type*): Move to...
* guix/records.scm: ... here.  New file.
* guix/build-system.scm, guix/packages.scm: Use it.
* guix/gnu-maintenance.scm: Likewise.
  (official-gnu-packages)[alist->record]: Remove.
* guix/scripts/substitute-binary.scm: Likewise.
  (alist->record, object->fields): Remove.
* tests/utils.scm ("define-record-type*", "define-record-type* with
  letrec* behavior", "define-record-type* & inherit",
  "define-record-type* & inherit & letrec* behavior",
  "define-record-type* & thunked", "define-record-type* & thunked &
  default", "define-record-type* & thunked & inherited"): Move to...
* tests/records.scm: ... here.  New file.

substitute-binary: Support decompression from non-file ports.

* guix/scripts/substitute-binary.scm (filtered-port): Move to utils.scm.
  (decompressed-port): Upon "none", return '() as the second value.
  (guix-substitute-binary): Expect `decompressed-port' to return a list
  of PIDs as its second value.
* guix/utils.scm (filtered-port): New procedure.  Add case for when
  INPUT is not `file-port?'.
* tests/utils.scm ("filtered-port, file", "filtered-port, non-file"):
  New tests.

Merge branch 'master' into core-updates

Conflicts:
	Makefile.am
	gnu/packages/autotools.scm
	gnu/packages/guile.scm
	gnu/packages/python.scm
	gnu/packages/shishi.scm
	guix/gnu-maintenance.scm
	guix/scripts/build.scm
	guix/scripts/gc.scm
	guix/scripts/package.scm
	guix/scripts/substitute-binary.scm
	guix/ui.scm
	nix/nix-daemon/guix-daemon.cc
	test-env.in
	tests/nar.scm
	tests/store.scm

web: Factorize `http-get' hackery.

This should fix `substitute-binary --query' on Guile 2.0.5.

* guix/web.scm: New file.
* Makefile.am (MODULES): Add it.
* po/POTFILES.in: Add it.
* guix/gnu-maintenance.scm (http-fetch): Remove.
  (%package-list-url): Turn into a URI.
  (official-gnu-packages): Add #:text? #t to `http-fetch' call.
* guix/scripts/substitute-binary.scm (fetch): Remove `http' case, and
  use `http-fetch' instead.

substitute-binary: Remove expired cache entries once in a while.

* guix/scripts/substitute-binary.scm (%narinfo-expired-cache-entry-removal-delay):
  New variable.
  (obsolete?): New procedure, formerly in `lookup-narinfo'.
  (lookup-narinfo): Adjust accordingly.
  (remove-expired-cached-narinfos, maybe-remove-expired-cached-narinfo):
  New procedures.
  (guix-substitute-binary): Call `maybe-remove-expired-cached-narinfo'.

substitute-binary: Skip servers that use a different store prefix.

* guix/scripts/substitute-binary.scm (fetch-narinfo): Return #f when
  CACHE uses a store directory different from (%store-prefix).