What is the difference between public goods and externalities?

Public goods are non-excludable and non-rival - i.e. you cannot be stopped from consuming the good and this does not affect others' consumption
Externalities are benefits/costs to a third party outside the market transaction
A public good such as knowledge could have positive externalities, but they are not the same thing