Simulations suggest Neanderthals were on the brink of extinction by the time our ancestors arrived on the Iberian Peninsula.