The other day I was talking to one of my friends who stopped watching pony years ago. He still likes the characters, but quit the show entirely. I asked him what his tipping point was, and his response was post-library destruction.
It's a sentiment I've heard a few times from people that lost interest. Something about this moment and the rise of Twilight's future castle turned them off from Friendship is Magic. While Twilight had become an alicorn princess previously, she still lived in her basic little tree house with a more down-to-earth life. After that it really does go full princess mode.
There are a lot of moments throughout the series that "lost" people. If you could go back in time and change one major occurence, what would it be?
