I have tried using the key, then exiting the stronghold. the key is still there.
Seems that, to me, the key only vanished when I logged out of game while still inside my friends stronghold. In that case, when I logged in again, still being inside his stronghold, then the key vanished.
After testing that, I logged out of game at fleet, with the key on my SH window showing, and logged back in: Key is still there.
Seems to me that the problem is related to logging out of game while still in friends stronghold.
The glitch can be worked around by friend. He has to delete your key and give you another one.