Previous change logs can be found at CHANGELOG-3.4.

The minimum recommended etcd versions to run in production are 3.2.28+, 3.3.18+, and 3.4.2+.

v3.5.0 (2021-06)

See code changes and v3.5 upgrade guide for any breaking changes.

Again, before running upgrades from any previous release, please make sure to read change logs below and v3.5 upgrade guide.

Breaking Changes

  • go.etcd.io/etcd Go packages have moved to go.etcd.io/etcd/{api,pkg,raft,client,etcdctl,server,raft,tests}/v3 to follow the Go modules conventions
  • go.etcd.io/clientv3/snapshot SnapshotManager class have moved to go.etcd.io/clientv3/etcdctl. The method snapshot.Save to download a snapshot from the remote server was preserved in 'go.etcd.io/clientv3/snapshot`.
  • `go.etcd.io/client' package got migrated to ‘go.etcd.io/client/v2’.
  • Changed behavior of clientv3 API MemberList.
    • Previously, it is directly served with server's local data, which could be stale.
    • Now, it is served with linearizable guarantee. If the server is disconnected from quorum, MemberList call will fail.
  • gRPC gateway only supports /v3 endpoint.
    • Deprecated /v3beta.
    • curl -L http://localhost:2379/v3beta/kv/put -X POST -d '{"key": "Zm9v", "value": "YmFy"}' doesn't work in v3.5. Use curl -L http://localhost:2379/v3/kv/put -X POST -d '{"key": "Zm9v", "value": "YmFy"}' instead.
  • etcd --experimental-enable-v2v3 flag remains experimental and to be deprecated.
    • v2 storage emulation feature will be deprecated in the next release.
    • etcd 3.5 is the last version that supports V2 API. Flags --enable-v2 and --experimental-enable-v2v3 are now deprecated and will be removed in etcd v3.6 release.
  • etcd --experimental-backend-bbolt-freelist-type flag has been deprecated. Use etcd --backend-bbolt-freelist-type instead. The default type is hashmap and it is stable now.
  • etcd --debug flag has been deprecated. Use etcd --log-level=debug instead.
  • Remove embed.Config.Debug.
  • etcd --log-output flag has been deprecated. Use etcd --log-outputs instead.
  • etcd --logger=zap --log-outputs=stderr is now the default.
  • etcd --logger=capnslog flag value has been deprecated.
  • etcd --logger=zap --log-outputs=default flag value is not supported..
    • Use etcd --logger=zap --log-outputs=stderr.
    • Or, use etcd --logger=zap --log-outputs=systemd/journal to send logs to the local systemd journal.
    • Previously, if etcd parent process ID (PPID) is 1 (e.g. run with systemd), etcd --logger=capnslog --log-outputs=default redirects server logs to local systemd journal. And if write to journald fails, it writes to os.Stderr as a fallback.
    • However, even with PPID 1, it can fail to dial systemd journal (e.g. run embedded etcd with Docker container). Then, every single log write will fail and fall back to os.Stderr, which is inefficient.
    • To avoid this problem, systemd journal logging must be configured manually.
  • etcd --log-outputs=stderr is now the default.
  • etcd --log-package-levels flag for capnslog has been deprecated. Now, etcd --logger=zap --log-outputs=stderr is the default.
  • [CLIENT-URL]/config/local/log endpoint has been deprecated, as is etcd --log-package-levels flag.
    • curl http://127.0.0.1:2379/config/local/log -XPUT -d '{"Level":"DEBUG"}' won't work.
    • Please use etcd --logger=zap --log-outputs=stderr instead.
  • Deprecated etcd_debugging_mvcc_db_total_size_in_bytes Prometheus metric. Use etcd_mvcc_db_total_size_in_bytes instead.
  • Deprecated etcd_debugging_mvcc_put_total Prometheus metric. Use etcd_mvcc_put_total instead.
  • Deprecated etcd_debugging_mvcc_delete_total Prometheus metric. Use etcd_mvcc_delete_total instead.
  • Deprecated etcd_debugging_mvcc_txn_total Prometheus metric. Use etcd_mvcc_txn_total instead.
  • Deprecated etcd_debugging_mvcc_range_total Prometheus metric. Use etcd_mvcc_range_total instead.
  • Main branch /version outputs 3.5.0-pre, instead of 3.4.0+git.
  • Changed proxy package function signature to support structured logger.
    • Previously, NewClusterProxy(c *clientv3.Client, advaddr string, prefix string) (pb.ClusterServer, <-chan struct{}), now NewClusterProxy(lg *zap.Logger, c *clientv3.Client, advaddr string, prefix string) (pb.ClusterServer, <-chan struct{}).
    • Previously, Register(c *clientv3.Client, prefix string, addr string, ttl int), now Register(lg *zap.Logger, c *clientv3.Client, prefix string, addr string, ttl int) <-chan struct{}.
    • Previously, NewHandler(t *http.Transport, urlsFunc GetProxyURLs, failureWait time.Duration, refreshInterval time.Duration) http.Handler, now NewHandler(lg *zap.Logger, t *http.Transport, urlsFunc GetProxyURLs, failureWait time.Duration, refreshInterval time.Duration) http.Handler.
  • Changed pkg/flags function signature to support structured logger.
    • Previously, SetFlagsFromEnv(prefix string, fs *flag.FlagSet) error, now SetFlagsFromEnv(lg *zap.Logger, prefix string, fs *flag.FlagSet) error.
    • Previously, SetPflagsFromEnv(prefix string, fs *pflag.FlagSet) error, now SetPflagsFromEnv(lg *zap.Logger, prefix string, fs *pflag.FlagSet) error.
  • ClientV3 supports grpc resolver API.
  • Turned on --pre-vote by default. Should prevent disrupting RAFT leader by an individual member.
  • ETCD_CLIENT_DEBUG env: Now supports log levels (debug, info, warn, error, dpanic, panic, fatal). Only when set, overrides application-wide grpc logging settings.
  • Embed Etcd.Close() needs to called exactly once and closes Etcd.Err() stream.
  • Embed Etcd does not override global/grpc logger be default any longer. If desired, please call embed.Config::SetupGlobalLoggers() explicitly.
  • Embed Etcd custom logger should be configured using simpler builder NewZapLoggerBuilder.
  • Client errors of context cancelled or context deadline exceeded are exposed as codes.Canceled and codes.DeadlineExceeded, instead of codes.Unknown.

Storage format changes

Security

Metrics, Monitoring

See List of metrics for all metrics per release.

Note that any etcd_debugging_* metrics are experimental and subject to change.

etcd server

Package runtime

Package embed

Package clientv3

Package lease

Package wal

etcdctl v3

gRPC gateway

  • gRPC gateway only supports /v3 endpoint.
    • Deprecated /v3beta.
    • curl -L http://localhost:2379/v3beta/kv/put -X POST -d '{"key": "Zm9v", "value": "YmFy"}' does work in v3.5. Use curl -L http://localhost:2379/v3/kv/put -X POST -d '{"key": "Zm9v", "value": "YmFy"}' instead.
  • Set enable-grpc-gateway flag to true when using a config file to keep the defaults the same as the command line configuration.

gRPC Proxy

Auth

API

Package netutil

tools/etcd-dump-metrics

Dependency

Platforms

Release

Go

Project Governance

  • The etcd team has added, a well defined and openly discussed, project governance.