Usually not big on denying people's self reports on their preferences/experiences but my only explanation for people saying babies are easier than toddlers is sleep deprivation induced memory loss
I have 3 toddlers right now and it's a lot easier than it was at any point for the last 5 years