I have a brand new install, 1.9.4, and something odd is going on.
In the builder itself, the underline isn’t showing unless I set the header to display: flex AND add the underline via CSS. In all other cases it won’t show up.
I reproduced the issue and added it to our bug tracker. We have changed the display value due to another problem and apparently need to look at this again
I reported this same issue here accidentally as a duplicate of this one. Adding in keywords for search-ability as what I was searching for initially was “typography decoration” that did not turn up this post earlier.