Why can customs unions lead to higher prices for consumers?

Although trading blocs such as the EU (which is a customs union) promote free trade without tariffs (taxes on imports) within the borders of the union, it still places tariffs on imports that are external to the trading bloc. This means that the act of being in a customs union leads to trade diversion which means consumers in a country like the UK will tend to import more from countries inside the EU, even though the producers outside the bloc may actually charge lower prices but because of the tariffs imposed on them they become relatively more expensive.
Therefore, trading blocs can be seen as protectionist in nature to external countries that in relation to some products leads to higher prices for consumers.