Reproduce method:
When you use a dynamic field to set the color and save it, it works normally. However, if you click on the color setting again, it will be overwritten by the default value. Clicking save at this time will prove that it is not just an error display because the color value has actually changed.
Dyn. Data: Color picker Customer field on Custom Taxonomy by MetaBox.io plugin.
I simply recorded a video of how to reproduce it, please contact me if you need it.
Unfortunately, I cannot reproduce the issue. Would you be so kind as to provide the mentioned screencast (best using https://jam.dev) showing and explaining what’s happening?