Optical brighteners work by absorbing ultra violet light and turning it into a visible blue light, which masks any yellowing of clothes that can occur over time. Clothes appear whiter & brighter, but they're not actually any cleaner.
These synthetic chemicals can only work by sticking to fabrics that they have been exposed to. The problem with this, is that they also bond to your skin. In a lot of cases, it's not fragrances and dyes that causes skin irritation, but optical brighteners.
Many of the artificial chemicals classed as optical brighteners can be toxic to aquatic life and have been known to cause mutations in bacteria. They're slow to biodegrade, so once they have been washed into our rivers & seas, they hang around polluting our waterways for a long, long time.