It's simple. The Jedi knew there was a 'Darth Sidious' who controlled the entire Separatist movement, thanks to Count Dooku. As soon as Palp confessed to Anikan to be that sith lord, he immediately told the second strongest and in power jedi he could find, Windu. Now knowing someone that powerful is also in control of the Galactic Rep. he couldn't possible wait till the next morning to arrest him, let alone in public to make a scene. He takes a team with him to arrest Palp, he strikes. That's it. Not because he wasn't a jedi, but because he was the dark lord they were looking for who was in control of the Separatist, not to mention the treason charges. Nothing more.