By “accidental changes” do you refer to this?
Because if you do, i don’t think it’s related. you implement this feature without having this issue by making the dragging work only on the unit word or so and not on selected text (just like in chrome, elementor, pinegrow, webflow etc…)