I think that if you used the language of privilege, rather than white privilege, you will not run into the difficulties you fear.

xiphias and I do stand in danger of losing our white privilege, because it's entirely possible that Jews will at some time again be defined as "not-white." Syrians are another example of people who have been white sometimes not white others -- seriously, the Supreme COurt ruled on the question three times in 20 years around the turn of the century.