Have you tried courting gifts? I don't know if that's it or not, just a suggestion.
I got it on with him when my affection was around 5k, took all the flirt options, never said Sith-y things to him. After we had "fun" and I gave him a courting gift he started pursuing me for more date type things.