A Question About Epiphenomenalism

Say that epiphenomenalist accepts causal theory of reference.
Then zombie philosopher in the zombie-world by ‘conscious experience’ is referring to something by which it was causally affected. As there is no conscious experience in the zombie-world, what it is affected by  and refers to by ‘conscious experience’, then, is something physical.
But as the causal relations are same in our world, it appears that whatever zombie philosopher is referring to by ‘conscious experience’, we are referring to the same thing by ‘conscious experience’. So, seems that epiphenomenalist can’t after all accept causal theory of reference, as that would mean that by ‘conscious experience’ she is referring to something physical.

So, what kind of grounding of reference does epiphenomenalist buy?
Can zombies refer to conscious experience at all?