Libertine holds key to cosmetic (5)
I believe the answer is:
rouge
'to cosmetic' is the definition.
(rouge is a kind of cosmetic)
'libertine holds key' is the wordplay.
'libertine' becomes 'roue' (roue is a kind of libertine).
'holds' is an insertion indicator.
'key' becomes 'g' (musical key).
'roue' enclosing 'g' is 'ROUGE'.
(Other definitions for rouge that I've seen before include "struggle at Eton" , "Red face powder" , "Ferric oxide powder" , "Face paint" , "Cosmetic powder" .)