satlab: nginx: use sliced caching for Android build and Drive downloads

Align the Satlab nginx config with caching-backend
(crrev.com/c/8078920, crrev.com/c/8007508), plus a Drive-specific
location (see below).

Remove the separate /download/android-build/ location. It cached
whole files because the downloader used to not support range
requests. This made large Android artifacts (multi-GB images) fail
to cache because of nginx's 1GB temp file limit, and concurrent
requests for the same file blocked each other. The downloader now
supports range requests for Android builds and Google Drive, so
these requests can use sliced caching too. This also lets Satlab
cache artifacts downloaded from Google Drive.

Google Drive downloads use a dedicated /download/drive/ location, the
same as /download/ except that "expires max" is only set for pinned
revisions (?rev=...). The downloader redirects requests without a
revision to a pinned one with "Cache-Control: no-store", and
"expires" would otherwise overwrite that header and let clients cache
the redirect.

Change the nginx slice size to 32m in all locations (was 10m for the
catch-all location, 30m for CQ and /download/). 32MB is a multiple of
common SSD/NVMe erase block sizes (4MB/8MB) and filesystem allocation
units, which avoids write amplification at slice boundaries. Cached
slices are keyed by range, so existing cache entries will be
refetched once after the update.

Re-add the X-Cache-Secondary header in the /download/ locations. The
location's own add_header directives were overriding the server-level
one, hiding the cache status of /download/ responses.

Requires a downloader image with range read support for Android
builds and Google Drive (crrev.com/c/8078920, crrev.com/c/8614341),
and Drive revision pinning (crrev.com/c/8669761).

BUG=b:566966058
TEST=manual

Change-Id: I78c609dc865d7edfeb871ab7430491241b968416
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/satlab/+/8671458
Reviewed-by: Przemysław Filipek <filipek@google.com>
Tested-by: Jacek Klimkowicz <klimkowicz@google.com>
Reviewed-by: Rafał Gaweł <rafalgawel@google.com>
Commit-Queue: Jacek Klimkowicz <klimkowicz@google.com>
1 file changed
tree: 61ade0909727a869142dcabf40264e542ea93d3e
  1. cfg/
  2. os-dependent/
  3. src/
  4. .gitignore
  5. Makefile
  6. OWNERS