The distribution of continents is thought to influence the temperature of the underlying mantle. Over geological timescales, insulation effects generate a build-up of heat that may cause increased magmatism, such as flood basalt volcanism, and ultimately rift the continents, causing them to break apart and new ocean basins to form. Here we use analyses of the major element geochemistry of lava samples collected from oceanic crust in the Atlantic and Pacific oceans to quantify the effect of continental insulation. The lavas formed at mid-ocean ridges following continental rifting and break up, and preserve a record of upper mantle temperatures over the past 170 Myr. We find that samples from the Pacific Ocean - formed more than 2,000 km from the nearest continental margin - do not record raised mantle temperatures. In contrast, samples from the Atlantic Ocean that formed close to the margin of the rifted continent reveal an upper mantle temperature immediately after continental rifting that was up to 150C higher than the present-day average; mantle temperatures remained high for 60-70 Myr. We conclude that the Atlantic thermal anomaly was created by continental insulation, and persisted in the mantle beneath the Atlantic Ocean long after the continental fragments had dispersed.