Actually, I believe those affection points gained are not final until the conversation finishes and the quest is complete. You will even get a pop-up on completion giving the amount of experience, affection, LS/DS points, and currency gained during all quest dialogue after the conversation has concluded and the quest is complete. Yes, you can cancel and restart the conversations and still get the affection point pop-ups, but they don't tally and finalize until the conversation ends successfully. Should get some Bioware clarification on it, though.