Release Process

For each release, we will fill a new Milestone, and an Issue with a Checklist.

Example Milestone and Issue.

Over the last month before the release target date, we'll discuss the checklist and milestone progress in our weekly calls, and fine tune timing and scope.

Pre-release checklist

This is a checklist of things that should be done in the weeks leading to the release.

• [ ] Verify that the milestone and checklist are complete
• [ ] Verify with component owners that they're ready for release
• [ ] Verify that the semver breakages (listed by the build-test job) are acceptable
• [ ] Take a bird-eye view at:
  • [ ] READMEs
  • [ ] Documentation
  • [ ] Coverage (coveralls and codecov)
  • [ ] Performance / Memory / Size benchmarks
  • [ ] Cargo.toml files
    • [ ] All dependencies from the workspace should use workspace = true rather than their version number or path
    • [ ] Cargo.toml files need to specify versions for each non-workspace entry in dependencies, or use workspace = true.
    • [ ] Ensure that any new packages have version number 0.0.0, this will making bumping during the release easier.
    • [ ] Ensure that the Cargo.toml file includes a set of fields consistent with other ICU4X packages.
      • These should mostly use workspace inheritance
• [ ] Run RUSTDOCFLAGS="--no-run --nocapture --test-builder clippy-driver -Z unstable-options" cargo +nightly test --doc --all-features --no-fail-fast and fix relevant Clippy issues in the docs (deprecated APIs, unused imports, etc.)
• [ ] Run cargo +nightly fmt -- --config=format_code_in_doc_comments=true --config=doc_comment_code_block_width=80 to prettify our docs
• [ ] Run cargo update for each Cargo.lock file to update our CI to freshest dependencies. A helpful snippet is find . -name Cargo.lock | while read lockfile; do cd $(dirname $lockfile); cargo update; done, though it is best run from examples/ since you may have other lockfiles in target/cargo-semver-checks directories.
• [ ] Go through ffi/capi/tests/missing_apis.txt and verify that it is empty. If it is not, component owners should either add FFI APIs, add rust_link annotations, or allowlist the relevant APIs as having been punted to the future. In case of unstable APIs, it is okay to leave things in the missing_apis file for now, see unicode-org#7181.
• [ ] Verify that ffi/capi depends on a released (not Git) version of Diplomat. Get it published (ask manishearth or sffc) otherwise.
• [ ] Ensure that landed PRs all have decent changelog entries (see changelog.md)
• [ ] Go through the last two months of open PRs. For PRs which affect the release (new features, optimizations, etc, as opposed to internal docs, tooling, etc), use your judgement to determine if (a) this PR has a chance of making it into the release and (b) if we should try to get it in. If so, comment and ask the authors/reviewers if they can try and get it in by the release timeline. If they say yes, add it to the milestone, likely with milestone-non-blocking. Some heuristics to apply to make this determination:
  • If the PR is blocked on discussion, it is unlikely to make it into the release, unless the WG has already completed all release-relevant discussions.
  • If the PR is a major feature, it is probably not worth trying to make it into the release, except perhaps as unstable.
  • If the PR is a bugfix or a docs fix, we should try to get it into the release.
  • If the PR is a small new API, we should try to get it into the release as long as it has already been noticed by the people who are likely to care about the new APIs. Consider pinging additional people.
• [ ] Draft the text for the GitHub release and circulate to the WG at least 18 hours in advance of the release, but ideally sooner. This text will be sent to GitHub subscribers and can also be used for the mailing list email and blog post.
• [ ] Consider making earlier drafts of the changelog (see changelog.md), noting a Git commit that the changelog is accurate up to.

Release steps

Once the release checklist is complete, the assigned release driver will perform the following steps, in order:

• [ ] Land the changelog (see changelog.md)
• [ ] Go through the prerelease checklist again, ensuring that no problems were reintroduced in the PRs that landed since the opening of the checklist. (Things like doc prettification will likely need to be rerun!)
• [ ] Quickly go through the list of open PRs to ensure that there are no new release-relevant PRs since the last time you checked this list.
• [ ] Prepare a PR with updated versions
  • [ ] Remove all -dev prelease tags from Cargo.tomls
  • [ ] Update all ICU4X crate versions
    • [ ] Update the workspace version to the new version
    • [ ] Some icu_* crates do not follow the ICU4X versioning scheme: icu_codepointtrie_builder, icu_pattern, and icu_experimental. Be sure to give them an appropriate version based on the changelog. Major releases are always paired with a 0.x.0 release of icu_experimental.
    • [ ] Reset baked data versions: Make sure all non-experimental entries in COMPONENTS in tools/make/bakeddata/src/main.rs use REPO_VERSION and not some override.
      • [ ] Make sure experimental is using the version for icu_experimental chosen above.
    • [ ] Find all ICU4X component/provider crates that have an overridden version in their Cargo.toml and reset it to version.workspace = true.
    • [ ] Update all ICU4X ~ dependencies in the toplevel Cargo.toml's workspace.dependencies.
    • [ ] Update icu_locale_core, icu_provider, and icu_pattern's non-~ dependency in Cargo.toml's workspace.dependencies
      • Note: These are in a different section and easy to miss!
  • [ ] Update all relevant utils versions
    • [ ] The changelog should already mention the new versions of each util crate that needs to be published. Use those, and double-check that they are accurate.
    • [ ] For utils that have had substantiative changes (new APIs, etc), update them in workspace.dependencies. When unsure, just update. See “Updating the dependency specification of a util” below for more information.
• [ ] Get this PR reviewed and checked in.
• [ ] Perform the release.
  • The best way to do this is to use cargo workspaces publish --from-git --no-remove-dev-deps.
  • If the release fails at some point and needs a fix, make a PR for the fix and get it merged. You may bypass requirements if necessary, but prefer to wait for most CI and an approval.
  • cargo publish does not like cyclic dev-deps and will fail on some crates.
    • cargo workspaces publish will automatically edit out dev-deps to handle this. Unfortunately, it dirties the tree when it does this. Hopefully that issue is fixed.
    • If not, you can instead perform a release by running cargo publish on individual folders (going back to cargo workspaces publish after the problematic crate is published)
    • When required to remove a dev-dependency, make a commit that replaces it with a path dependency, and then cargo publish from the clean git tree.
    • Be sure to push such commits to main (you may need to temporarily change branch protection to do so: do not change the ruleset, just remove the default branch from our branch protection ruleset).
    • It is very important to only publish from commits that are reachable from main (or other repo branches). It is less important to ensure that the publishes are not “dirty”, but it is ideal to try and maintain that.
• [ ] Add the icu4x-release group as owners to each brand new crate that was published
  • cargo owner -a github:unicode-org:icu4x-release
• [ ] Tag the Release with the text drafted above
• [ ] Upload FFI artifacts to the release
  • Monitor the Artifacts Build GitHub Action for the release tag. Verify that ffi-libs completed successfully.
  • When it is done, manually trigger the release-ffi-libs job. It should upload the artifacts generated by ffi-libs to the release you created in the previous step.
• [ ] Update and publish FFI packages
  • [ ] Dart
    • [ ] update version in ffi/dart/pubspec.yaml
    • [ ] update the artifacts tag in ffi/dart/lib/src/hook_helpers/hashes.dart to the tag created above, and run regenerate_hashes.dart
    • [ ] get this checked in, then cd ffi/dart && dart pub publish
  • [ ] JS
    • [ ] update version in icu4x/ffi/npm/package.json
    • [ ] get this checked in, then cd ffi/npm && npm publish
• [ ] Create a branch named release/x.y including the release tag and FFI commits and push it to the upstream
• [ ] Update the website
  • [ ] In the icu4x-docs repo, run node tools/github-to-astro.ts and follow the instructions
• [ ] Announce the release to public
  • [ ] (All releases) Blog post on Unicode blog (email comms@unicode.org)
  • [ ] (All releases) Update https://www.unicode.org/releases/ (email Ken Whistler)
  • [ ] (All releases) Email icu4x-announce@unicode.org (see https://groups.google.com/a/unicode.org/g/icu4x-announce)
  • [ ] (Major releases only) Submit to This Week In Rust
• Keep the main branch relatively stable for 7-14 days following the release to make things easier in case a patch release is needed.
  • It's okay to land smaller or incremental changes, but avoid breaking changes during this period.

Patch Releases

The ICU4X TC may decide to make a patch release of an ICU4X component on an old release stream, such as to fix a regression in behavior. To make a patch release:

• Fix the issue on the main branch. Get it reviewed and landed.
  • Include an update to the changelog.
  • Try to include a bump to the crate version (so that future patch releases know they are starting on a newer version)
  • If possible, avoid mixing functional changes with generated files (e.g. data or FFI) in the commit that lands on the main branch.
• If your release also requires uplifting patches to a utils crate (and other crates not versioned with ICU4X), update their Cargo.tomls on main to reflect the version you wish to publish, to simplify things for people making ICU4X major/minor releases in the future. In this case, try to avoid publishing the util from main: it's fine if there have already been out-of-cycle util releases on main, but if this is the first util release since the last ICU4X release, cherry pick just the necessary changes onto the release branch.
• Check out the release/x.y branch and make a PR. On this branch:
  • Cherry-pick the functional change from the main branch
    • If doing multiple cherry-picks, please use one PR per cherry-pick.
  • Cherry-pick the changelog update if it was a separate commit
  • Regenerate code / data if necessary
  • Make a commit updating the version number of the component if necessary (this should have already been done in the commit cherry-picked from main, but it's not always possible)
  • Make a PR targeting release/x.y and have a team member review it.
  • Make sure your PR is based on the top of that branch.
  • Make sure CI passes, or that CI failures are understandable issues (e.g. some CI-used URL changed) that would not affect the correctness of the release.
• Perform the release. It is crucial that the commit that cargo publish was run from is available in the release/x.y branch, as Cargo includes that commit in the published crate and it is useful for supply-chain security and tracking.
  • Perform cargo publish --dry-run to catch many (but not all) of the reasons a publish may fail.
    • In particular, if the patch release contains multiple crates, dry run publishes will often not succeed.
  • Merge the PR(s) into release/x.y
  • Update release/x.y locally and check it out
  • Run cargo publish in the crate(s)
  • If cargo publish fails, make a new PR to fix the release branch and get it merged with review. You may admin-merge minor fixes that are mostly tweaking versions in Cargo.toml.
• Create and push a tag of the format ind/icu_collator@1.3.3 (for icu_collator patch version 1.3.3)
  • It is not necessary to create ind/icu_collator_data@1.3.3 so long as the component has a tag

Publishing utils

Our utils/ crates do not follow the same versioning scheme as the rest of ICU4X, and may experience more frequent releases.

In general, if you ever cut a new release of a utils/ crate, all icu4x crates depending on new behavior should have their Cargo.toml updated to the latest version, by updating the version specification in the workspace Cargo.toml‘s workspace.dependencies section. However, there are times when you don’t have to, see below.

While code may compile using our local path dependencies, when publishing we must ensure that it will pull in the correct published version of a utils crate.

When cutting new ICU4X releases, make sure all utilities with changes have had a new release containing those changes. To do so, go through the utils/ folder and check the history of each crate since the last version bump. Bear in mind that some folders like yoke/ contain multiple crates (yoke/derive/), and to keep derive-crates' versions in sync with their crates.

If there are no changes, ensure that the current version of the crate is the version in use in ICU4X's components. If not, make sure that ICU4X is not relying on any features since the release that is in use. In case of such reliance, update the version in use in the ICU4X components.

If there are changes, go through the changes and determine if they are breaking or not. For breaking changes, perform a breaking version bump (x.y.z to x+1.0.0, and 0.x.y to 0.x+1.0) and update all of ICU4X's components to use the new version.

For non breaking changes, perform a non-breaking version bump (x.y.z to x.y.z+1 or x.y+1.0 based on the size of the changes; 0.x.y to 0.x.y+1). Then, determine if the introduced functionality is being relied upon by ICU4X (assume it is if this is tricky to determine). If it is, update the version in use by the ICU4X components, otherwise it is fine to not do so.

This can all be done in a separate PR to chunk out the work but there should be no changes to utils between this PR landing and the overall ICU4X version bump. After landing the PR, as usual, cargo publish should be run on the updated utils.

Updating the dependency specification of a util

We share dependency specifications for all workspace crates in the [workspace.dependencies] section of Cargo.toml.

It is a nice-to-have feature that updating ICU4X does not require one to update every util. This reduces impact especially to people who need to audit new dependencies or vendor code, allowing more flexibility on ICU4X updates.

However, this carries risks. We do not currently CI for mininal-versions (See #2966), which means that a commit may inadvertently introduce a dependency on a newly introduced feature, which we then miss when we perform a release.

To allow for this, we follow the rule that EVERY [workspace.dependencies] entry that diverges from its current version MUST have a # Current version: comment after it, like so:

databake = { version = "0.2.0", path = "utils/databake", default-features = false } # Current version: 0.2.1

This MUST be kept up to date when releases are performed.

By default, when a util is being released, its workspace.dependencies should be updated. The following is a set of heuristics for when this rule does and doesn't need to be followed, applied in order.

Firstly, if before the release the util was already diverging, you SHOULD update to at least the already-released version. For example, if ICU4X 2.2 depends on databake = 0.2.0 and releases databake@0.2.1, ICU4X 2.3 should update workspace.dependencies to at least databake = 0.2.1. This simplifies things when it comes to the subsequent heuristics: one doesn't have to go trawl through multiple versions worth of history. There is a decent benefit for an ICU4X release to not require an update of all utils to newly released versions, but that benefit is less for util versions that have been published for a few months. We encourage users who need audits and vendoring to perform periodic rolling updates of their dependencies to reduce impact of ICU4X releases.

You MUST update if any part of ICU4X or ICU4X utils depends on APIs or behavior in the new version.

You MAY choose to not update when the changes are only:

• Code style
• Docs updates
• Minor optimizations

You SHOULD update for soundness fixes.

You SHOULD update for major bug fixes.