Fix Infrastructure Manager to wait for full instance configuration before running tests

For Infrastructure Manager deployments, the waitForInstancesReady
function polls each instance's metadata every 30 seconds looking for a
key named windows_machine_<instanceName>_status. This key is set by the
cel_agent running on each VM: when cel_agent starts, it runs all asset
resolvers (domain join, AD setup, RDS setup, etc.), and only after all
resolvers complete successfully does it set this metadata key to "ready"
(or "error" if something fails). The watcher keeps polling until all
instances have status "ready", at which point it returns and the test
can proceed. This ensures the test doesn't start until all configuration
(including RDS setup) is complete on all VMs, unlike the previous
approach which only checked if cel_agent had started (too early).

Tested: Integration tests
Bug: 417756034
Change-Id: I42051706c87bed13babf57c95fa75e81426ca593
Reviewed-on: https://chromium-review.googlesource.com/c/enterprise/cel/+/7477654
Reviewed-by: Jonathan Lee <jonathanjlee@google.com>
4 files changed
tree: 17c8e52c04ca50d79beafc8c26a19cf1ad545a0a
  1. build/
  2. docs/
  3. examples/
  4. go/
  5. infra/
  6. resources/
  7. schema/
  8. scripts/
  9. test/
  10. .gitignore
  11. .style.yapf
  12. .vpython3
  13. build.py
  14. CODE_OF_CONDUCT.md
  15. codereview.settings
  16. CONTRIBUTING.md
  17. go.mod
  18. go.sum
  19. LICENSE
  20. navbar.md
  21. OWNERS
  22. PRESUBMIT.py
  23. README.md
  24. VERSION
  25. WATCHLISTS
README.md

Chrome Enterprise Lab

Chrome Enterprise Lab is a set of tools for building enterprise experimental labs quickly and easily. The labs so built can be used for system level end-to-end testing of Google Chrome/Chromium. These system tests are required to avoid enteprise regression under some complex enterprise software & networking configurations, such as Microsoft Active Directory.

Have a peek at the Design document.

Also have a peek at the Code of Conduct.

Most of the code is in Go. See Guide to code to get a head start on the code in this repository.

Building

Prerequisites

  • Go: Download Go from the Go download page. This project requires Go 1.23.4. Using other golang build might cause build or test failures.

  • Depot Tools: Used for managing the checkout and the contributor workflow.

  • Protocol Buffers Compiler: Protocol buffers are used extensively for wranging all the data that needs to be shuttled around. Run protoc --version to check the protoc version. The protoc version should be at least 3.5.1. If protoc is not installed, install it from command line:

    sudo apt install -y protobuf-compiler
    
  • python: python should be on most systems. check by running python --version if that doesn't work try python3 --version if you have python3 install this to make it your primary version: sudo apt-get install python-is-python3

  • pip: python‘s package manager. make sure you have it by typing pip in your terminal. if you don’t: sudo apt-get install pip

  • absl-py: This Python package is used by tests. Install it by running pip install absl-py.

Get The Source

  1. Setting GO environment variables by adding below lines into ~/.bashrc file

    export GOPATH=$HOME/go
    export GOBIN=$GOPATH/bin
    export PATH=$PATH:$GOBIN
    
  2. Clone this repository:

    Assumes that $GOPATH is a single path and not a : delimited list.

    mkdir -p ${GOPATH}/src/chromium.googlesource.com/enterprise
    cd ${GOPATH}/src/chromium.googlesource.com/enterprise
    git clone https://chromium.googlesource.com/enterprise/cel
    cd cel
    

Build It

  1. Pull dependencies

    python3 build.py deps --install
    
  2. Use the build script:

    python3 build.py build
    

    If you see an error message like google/protobuf/descriptor.proto: File not found, see the protoc prerequisites (specifically the includes step).

  3. Also make sure that all unit tests for the Go source files are passing.

    python3 build.py test
    

Contributing

See CONTRIBUTING.

Release

The CELab deployment binary and its Python test framework is uploaded into CIPD as a package.

Here are the general steps to release the package.

  1. After the code review, merge in the change
  2. Wait for windows CI build finished (https://ci.chromium.org/p/celab/g/Main%20Console/console), click on the build(either Windows or Linux) you'd like to release.
  3. Run upload_to_cipd with the link to binaries in the CI build (instructions here: https://chromium.googlesource.com/enterprise/cel/+/refs/heads/main/scripts/cipd)
  4. (Optional) Check the output of that command and update the infra version in the .vpython file for your app