Hi Dustin,
I put it as “WAIT” while discussing with the team 
The filter was designed to filter results on the current page instead of navigation.
Example:
In shop page:
- without any filter, the result will display all products
- filter Category A, and the result will display all products which have Category A checked
In Category C archive page:
- without choosing any filter, the result will display all products which have Category C checked
- filter Category A, and the result will display all products which have Category C and Category A checked (one product can have multiple categories)
If you wish visitors can easily navigate to a different category, you can just create a Term query loop with a term link on it.
Hope this explanation makes sense.
Regards,
Jenn