using a cognac bottle for scotch

i have a 350ml bottle of remy martin vsop nearing its end, and i would like to use it to store scotch in. since the vsop bottle is a cork top, im worried that it might contaminate the scotch when i transfer it to the remy martin bottle.

is there reason for me to get worried about aromas left in the cork?
and if so, is there a way for me to get rid of the aromas left on the cork? any tips?