You are right, that will not work on Hover, since those controls are handled a bit differently than the others. The same issue was reported here:
and it’s already in our bug tracker.
For now, I suggest you use other controls to change color. You can use “Color” control below, and it will change the SVG color, and it works on hover.