Support full file JS formatting and apply it to deterministic.js
The goal is to support `git cl format --full --js deterministic.js`.
--full causes the command to format the entire file, not only the lines
changed by the current CL. This is useful for one-off instances, it
shouldn't be used most of the time.
There are various ingredients at play:
1) An unsolved issue: currently, the command only takes effect if the
file was already modified. This is an issue in a different layer (most
likely third_party/clang-format/script) and deserves separate
investigation and fixing.
2) After making artificial changes to deterministic.js to account for
(1), the command still fails with:
```
Problem while looking for clang-format in Chromium source tree:
File does not exist in either path: [
'.../buildtools/linux64-format/clang-format',
'.../buildtools/linux64/format/clang-format',
'.../buildtools/linux64/clang-format'
]
```
This CL fixes that issue by adding the GCS deps from [1]. The original
buildtools/ dependency apparently becomes redundant, so it is removed.
Note that `gclient sync -D` is not smart enough to delete the leftover
files - because the outer directory was kept - so I rm-ed the entire
directory manually and ran `gclient sync`.
3) At this point, the command works but reformats replacement constants
like `{{WPR_TIME_SEED_TIMESTAMP}}` into `{{ WPR_TIME_SEED_TIMESTAMP }}`,
which does not match the corresponding go code. One option would be to
disable formatting for those specific deterministic.js lines with
`clang-format off` and `clang-format on` comments. Instead, I opted for
removing the braces everywhere. The code keeps handling the braces
version too for backwards compatibility.
[1] https://source.chromium.org/chromium/chromium/src/+/main:DEPS;l=715-770;drc=96c536897c703e01b2d89b97182d057f9eb6d322
Change-Id: Ibb0ef846ffb02802406c0acaa0a7eda6df43620b
Reviewed-on: https://chromium-review.googlesource.com/c/webpagereplay/+/7720623
Reviewed-by: Elad Alon <eladalon@chromium.org>
Web Page Replay (WprGo) is a performance testing tool written in Golang for recording and replaying web pages. WprGo is currently used in Telemetry for Chrome benchmarking purposes. This requires go 1.8 and above. This has not been tested with earlier versions of go. It is supported on Windows, MacOS and Linux.
For performance tests, this tool is generally not used directly. Instead, we use these instructions to record, and these instructions to replay.
The first way is to use go install. This will use your default GOPATH, which is typically $HOME/go:
go install go.chromium.org/webpagereplay@latest
The second way is to download the repository. Don't just git clone! Use depot_tools to set everything up correctly for you.
mkdir webpagereplay cd webpagereplay fetch webpagereplay cd webpagereplay
Terminal 1:
Start wpr in record mode. By default wpr uses both rsa and ecdsa certificates, or provide your certificate directly with --https-cert-file and --https-key-file parameters.
cd path/to/webpagereplay go run src/wpr.go record --http-port=8080 --https-port=8081 /tmp/archive.wprgo
...
Ctrl-C
Terminal 2:
google-chrome-beta --user-data-dir=$foo \ --host-resolver-rules="MAP *:80 127.0.0.1:8080,MAP *:443 127.0.0.1:8081,EXCLUDE localhost" --ignore-certificate-errors-spki-list=PhrPvGIaAMmd29hj8BCZOq096yj7uMpRNHpn5PDxI6I=,2HcXCSKKJS0lEXLQEWhpHUfGuojiU0tiT5gOF9LP6IQ=
... wait for record servers to start
Terminal 1:
Start wpr in replay mode.
cd path/to/webpagereplay go run src/wpr.go replay --http-port=8080 --https-port=8081 /tmp/archive.wprgo
Terminal 2:
google-chrome-beta --user-data-dir=$bar \ --host-resolver-rules="MAP *:80 127.0.0.1:8080,MAP *:443 127.0.0.1:8081,EXCLUDE localhost" --ignore-certificate-errors-spki-list=PhrPvGIaAMmd29hj8BCZOq096yj7uMpRNHpn5PDxI6I=,2HcXCSKKJS0lEXLQEWhpHUfGuojiU0tiT5gOF9LP6IQ=
... wait for replay servers to start
load the page
You will need a Linux host machine and an android device.
adb reverse tcp:8080 tcp:8080 adb reverse tcp:8081 tcp:8081
build/android/adb_chrome_public_command_line --host-resolver-rules="MAP *:80 127.0.0.1:8080,MAP *:443 127.0.0.1:8081,EXCLUDE localhost" \ --ignore-certificate-errors-spki-list=PhrPvGIaAMmd29hj8BCZOq096yj7uMpRNHpn5PDxI6I=,2HcXCSKKJS0lEXLQEWhpHUfGuojiU0tiT5gOF9LP6IQ=
WebPageReplay uses self signed certificates for Https requests. To make Chrome trust these certificates, you can use --ignore-certificate-errors-spki-list like above. If that doesn't work, you may try installing a test certificate authority as a local trust anchor. Note: Please do this with care because installing the test root CA compromises your machine. This is currently only supported on Linux and Android.
Installing the test CA. Specify a --android-device-id if you'd like to install the root CA on an android device.
cd path/to/webpagereplay go run src/wpr.go installroot
Uninstall the test CA. Specify a --android-device-id if you'd like to remove the root CA from an android device.
cd path/to/webpagereplay go run src/wpr.go removeroot
cd path/to/webpagereplay go run src/wpr.go replay --https-port=8081 --https-to-http-port=8082 \ /tmp/archive.wprgo
google-chrome-beta --user-data-dir=$foo \ --host-resolver-rules="MAP *:443 127.0.0.1:8081,EXCLUDE localhost" \ --ignore-certificate-errors-spki-list=PhrPvGIaAMmd29hj8BCZOq096yj7uMpRNHpn5PDxI6I=,2HcXCSKKJS0lEXLQEWhpHUfGuojiU0tiT5gOF9LP6IQ= \ --proxy-server=http=https://127.0.0.1:8082 \ --trusted-spdy-proxy=127.0.0.1:8082
httparchive.go is a convenient script to inspect a wprgo archive. Use ls,cat and edit. Options are available to specify request url host (--host) and path (--full-path).
E.g.
cd path/to/webpagereplay go run src/httparchive.go ls /tmp/archive.wprgo --host=example.com --full-path=/index.html
httparchive.go also provides methods to alter a wprgo archive. Use commands such as add, merge and trim. These will add new traffic, merge two archives to create a third, or trim request response pairs by host (--host) or path (--full-path). See --help for more information.
E.g.
cd path/to/webpagereplay go run src/httparchive.go trim /tmp/archive.wprgo --host=example.com /tmp/trimmed.wprgo
Run all tests in a specific file. Use ‘-v’ flag to show results. Note: proxy_test requires more includes than just proxy.go.
cd path/to/webpagereplay/src/webpagereplay go test archive_test.go archive.go go test transformers_test.go transformers.go go test proxy_test.go proxy.go transformers.go archive.go
Run all tests in webpagereplay module.
cd path/to/webpagereplay/src/webpagereplay go test -run ''
Or
cd path/to/webpagereplay go test -v go.chromium.org/webpagereplay/src/webpagereplay
wpr_public_hash.txt is generated from wpr_cert.pem using the command below.
openssl x509 -noout -pubkey -in wpr_cert.pem | \ openssl pkey -pubin -outform der | \ openssl dgst -sha256 -binary | \ base64
The run_benchmark and record_wpr tools will build and invoke WPR from this directory if they are run with the --use-local-wpr flag.
You can file bugs here. Patches welcome! Please run scripts/upload_new_binaries.py after making changes to go code.