Hey Josh,
Today is another one of those days where I learned something new (I never used flexible content with bricks before) 
To prevent the duplicated sections, you need to add a condition to them and compare if {acf_get_row_layout} equals your flexible_layout_name.
If you have two possible layouts (e.g. text + image and text + text), the setup looks like this:
Flexible Content Loop on the container:
Layout conditions:
‘text_text’ and ‘text_image’ are the layout names of each layout:
Best regards,
timmse