One theory, according to Time magazine, is that the symbol of the rabbit stems from the ancient Pagan tradition believed to have started the celebration of Easter – the festival of Eostre.