Add globalSetup and addGlobalTearDown APIs for cross-suite setup and teardown (#2714)
Adds `globalSetup()` and `addGlobalTearDown()` APIs in `package:test_api` / `package:test` to execute shared global setup scripts once on the host test runner, memoizing results across all test suites and executing registered teardowns when the test runner closes.
Addresses #2225
The main motivation is to support setup shared across multiple test suites (`_test.dart` files). Examples include:
* Precompiling a binary used by multiple end-to-end tests
* Provisioning test databases or starting local test servers
### Example
```dart
// test/setup_server.dart
import 'dart:io';
import 'package:test/global.dart';
Future<Map<String, dynamic>> setUp() async {
var server = await HttpServer.bind('localhost', 0);
addGlobalTearDown(() async => await server.close(force: true));
return {'port': server.port};
}
// test/service_test.dart
import 'package:test/test.dart';
void main() {
test('connects to server', () async {
final config = await globalSetup(
Uri(path: '/test/setup_server.dart'),
) as Map;
final port = config['port'] as int;
// ...
});
}
```
### Key Features
* **`globalSetup(Uri uri)`**:
* Runs the Dart file at `uri` in an isolate on the host runner and returns its JSON-encodable result.
* Memoizes the JSON-encoded result so the script only executes once per canonical URI, while returning an independent decoded value to each caller.
* Must be called from within a running test or a `setUp()` / `setUpAll()` callback.
* Bootstraps isolates via an in-memory `data:` URI using `Isolate.spawnUri` (sharing architecture with `spawnHybridUri`), avoiding filesystem compilation or runner-specific compiler hooks.
* **`addGlobalTearDown(FutureOr<void> Function() callback)`** (exported from `package:test/global.dart`):
* Called from within a setup script to register cleanup tasks.
* Executed in reverse registration order (LIFO) when the test runner closes.
* Best-effort cleanup: all registered teardowns run even if an earlier teardown throws, with errors reported to stderr and runner exiting with code 1.
* 30-second per-isolate timeout to prevent hanging teardowns.
* **URI Resolution**:
* **Root-relative** (`Uri(path: '/test/setup.dart')`): Resolved relative to package root (`pubspec.yaml`).
* **`package:` URIs** (`Uri.parse('package:my_pkg/test_helpers.dart')`): Resolved synchronously via package config.
* **Relative URIs** (`Uri(path: 'setup.dart')` or `Uri(path: '../setup.dart')`): Resolved relative to the executing test suite's directory.
* **`file:` URIs**: Resolved as absolute filesystem paths.
* Fragments are automatically removed for consistent compilation caching. Relative, root-relative, and file URIs strictly prohibit query parameters.
* **Cross-Platform Support**:
* Dispatches over `test_channel`, allowing browser, Node, and VM tests alike to trigger host-side setup.
* **Standalone Execution Support**:
* Tests run directly via `dart <test_file>.dart` execute setup scripts using a standalone scaffolding fallback and run teardowns on process exit.
* **Flutter Compatibility**:
* Works out of the box with `flutter test` without runner-level code changes.
Welcome! package:test is the standard testing library for Dart and Flutter. If you have questions about Dart testing, please see the docs for package:test. package:test_api and package:test_core are implementation details and generally not user-facing.
package:checks is a relatively new library for expressing test expectations. It's a more modern version of package:matcher and features a literate API.
| Package | Description | Issues | Version |
|---|---|---|---|
| checks | A framework for checking values against expectations and building custom expectations. | ||
| fake_async | Fake asynchronous events such as timers and microtasks for deterministic testing. | ||
| matcher | Support for specifying test expectations via an extensible Matcher class. Also includes a number of built-in Matcher implementations for common cases. | ||
| test | A full featured library for writing and running Dart tests across platforms. | ||
| test_api | The user facing API for structuring Dart tests and checking expectations. | ||
| test_core | A basic library for writing tests and running them on the VM. | ||
| test_descriptor | An API for defining and verifying files and directory structures. | ||
| test_process | Test processes: starting; validating stdout and stderr; checking exit code. |