You can use the nth-child pseudo selector for this. Basically the filters are in a list (li) and you can use nth-child for selecting an individual list item in the list. For example in the above image you can use nth-child(2) to give the gold colored background on the second item.
In Bricks click on the filter element and in the top left hand corner copy its ID as you will need it. Then go down to the custom CSS section and use the ID with nth-child to color each button with an individual background color eg (my ID was #brxe-cmjpuj):