SUSE Support

Here When You Need Us

etcd snapshots showing 0B size in the Rancher UI

This document (000021447) is provided subject to the disclaimer at the end of this document.

Environment

Rancher managed RKE2 cluster

Situation

When viewing etcd snapshots in the Rancher UI for a downstream RKE2 cluster, the snapshots show a size of 0 B.

Resolution

The workaround is to modify the rke2 lease and make the lease-holder match the rke2-etcd lease. The following one-liner can be used for implementing this workaround:
kubectl get lease -n kube-system rke2-etcd -o jsonpath='{.spec.holderIdentity}' | xargs -I {} kubectl patch lease -n kube-system  --patch='{"spec":{"holderIdentity":"{}"}}' --type=merge rke2; kubectl get lease -n kube-system -o custom-columns='name:.metadata.name,holder:.spec.holderIdentity' | grep rke2
After this is done, if the rke2 or rke2-etcd ConfigMap objects have an annotation with a differing holderIdentity value, please modify that to match as well.

After the workaround is implemented, please wait some time for the snapshots to reconcile.

Ultimately, the solution will be to upgrade to the version where this bug is fixed.
 

Cause

One reason this can happen is a mismatch between the leaseholder for the rke2 and rke2-etcd leases There is also an annotation in the rke2 and rke2-etcd ConfigMap objects that should be checked.
kubectl get cm -n kube-system rke2 -o yaml | grep holderIdentity
kubectl get cm -n kube-system rke2-etcd -o yaml | grep holderIdentity
kubectl get lease -n kube-system rke2
kubectl get lease -n kube-system rke2-etcd

This is a known bug: https://github.com/rancher/rke2/issues/5866
 

Additional Information

If there is no lease-holder mismatch, some other potential issues that could cause this problem are mentioned in the following knowledge base articles:

Disclaimer

This Support Knowledgebase provides a valuable tool for SUSE customers and parties interested in our products and solutions to acquire information, ideas and learn from one another. Materials are provided for informational, personal or non-commercial use within your organization and are presented "AS IS" WITHOUT WARRANTY OF ANY KIND.

  • Document ID:000021447
  • Creation Date: 09-May-2024
  • Modified Date:01-Jun-2024
    • SUSE Rancher

< Back to Support Search

For questions or concerns with the SUSE Knowledgebase please contact: tidfeedback[at]suse.com

tick icon

SUSE Support Forums

Get your questions answered by experienced Sys Ops or interact with other SUSE community experts.

tick icon

Support Resources

Learn how to get the most from the technical support you receive with your SUSE Subscription, Premium Support, Academic Program, or Partner Program.

tick icon

Open an Incident

Open an incident with SUSE Technical Support, manage your subscriptions, download patches, or manage user access.