All the comments that follow appear to me to be true of my English. No guarantee that they apply to others'. Comments welcome.
Thinking about examples that I feel more and less happy with, I'd say that there are some prosodic constraints on -ee.
In the best examples, -ee attaches to a root that has (i) two syllables, (ii) has, or can have, primary on the first syllable, (iii) has no long vowel in the second syllable. Your examples assignee, employee, and refugee all fit this description, as do nominee, advisee, and conferee.
Note that, in the last two, stress shifts from its normal locus (adVISE, conFER) to the first (ADvisEE, CONferEE). This is why (ii) says "can have", not just "has". Some prefixes seem less able to accommodate this stress shift: forgetee is completely impossible for me, and impugnee, repressee are pretty iffy.
Nondisyllabic roots are (un)acceptable depending, again, in part at least, on prosodic factors. Two unstressed syllables before ee is problematic: witness such contrasts as convertee (imaginable) ~ controvertee (not), or preposee (imaginable) ~ presupposee (not). Similarly, one long syllable is also uncomfortable, as in your massagee. But examinee and eliminee are both fine for me, with the extra syllable occurring before the main stress.
When it comes to monosyllabic roots, if the root has a long vowel, I find the examples slightly better than if the vowel is short. I'm fine with slayee (a Buffy example), callee (cited above), freeee (it's about time we had a quadruple vowel in English), but not so happy with givee, dropee, kickee. To make the latter three vaguely acceptable, I find myself inserting a glottal stop between root and suffix (kick'ee, etc.).
That said, though, phonology is clearly not the whole story. I'm fine with confutee but unhappy with confusee, unhappy with refutee but fine, of course, with refugee. Or maybe I'm just tired...