I did this quest yesterday by letting them attack me plus spending at least a minute on each wave. But looking on how others did and couldn't finish this I am beginning to think that it was just a dumb luck xd

Just gotten here and have same problem. I managed to get to the third wave few times but thats it.