Centennial Charmed Universe - why did the Elders let Cole and the Underworld take over San Francisco?

Centennial Charmed is one of my favorite episodes of the Paige era but I never understood why beings so powerful just let the Source, Cole, other demons take over above ground at least in San Francisco. The Elders did nothing to stop the Manor from being taken over by evil. There's no Power of three; Prue and Paige in the Centennial Charmed universe were dead.

Phoebe was coerced into being evil to save Piper and Leo