Direct Answer
The speaker uses babies as a picture of what it looks like to be free from certain sins, so the “most adult” sin is whichever trait in you most contradicts those baby-like qualities (malice, deceit, hypocrisy, envy, slander). To “go back” would mean stripping those patterns away and returning to the “sincere milk of the word” as your starting point for growth.