What did I do wrong?

Did I not gather enough villagers maybe?