Yeah, I feel your pain. Running on a 580 and a 6core at 4,5 GHz, cpu usage is averaging around 40%, gpu usage averages around 30%, still in heavy fighting fps is around 20. No matter what settings.
Managed to improve my fps by enabling sparse grid transperancy aa in nvidia inspector (aa improving fps = logic), so now i get 20 fps during a fight, not while running randomly in a middle of nowhere. Best part is the ingame fps monitor that says that it's my gpu that limits my fps, while the usage on the gpu drops when fps drops
But I bet that someone who is "maxing this game on a riva tnt2 and never drops below 110 fps" will come here soon enough.