His Excellency leaves infidels' capital (6)
I believe the answer is:
athens
'capital' is the definition.
(capital of Greece)
'his excellency leaves infidels'' is the wordplay.
'his excellency' becomes 'HE' (abbreviation).
'leaves' is a deletion indicator.
'infidels'' becomes 'heathens'' ('heathen' can be a synonym of 'infidel').
'heathens' with 'he' removed is 'ATHENS'.
(Other definitions for athens that I've seen before include "Two thousand and four Olympics city" , "Hasten to European capital" , "Hasten to European city" , "Plato's birthplace" , "Hasten to Greek city" .)