I got poor download speed, then I remembered that I've seen that SWTOR Unleashed and patching aren't a good combination so I restarted and now it's up to 6MB/s. I'm not going to risk restarting again to see if SWTOR Unleashed was the culprit, but it could be worth a shot if you use SWTOR Unleashed and see slow downloads.
Edit: Actually just after I posted that it slowed down, so I tried just clicking the pause button and then resumed and it sped up again.