short answer:
Yes, it's fine. It's an abbreviation for 'the party consisting of you and me'. (The rhyme is merely a happy coincidence.)
long answer:
“Me and her are married.”
Schoolmarms object that this is ungrammatical, but this kind of thing is said all the time by native speakers of English. Now, whatever usage native speakers make of a language (sufficiently often, of course, not just as an occasional joke), is BY DEFINITION correct! So, let’s not let the tail wag the dog. It is up to us to explain the acceptability of the sentence, not to ‘explain away’ the sentence by simply declaring it to be ungrammatical (or by declaring the proposed alternative as being "pretentious sounding, but correct").
There's nothing wrong with being a pedant, but there is something wrong about being wrong.
The two nominative expressions "Me and her" and "She and I" are NOT synonymous. They are both grammatical, but have not only different meanings, but also, of course, different registers. The former, being an abbreviation for "The party consisting of me and her" has lower register, simply by virtue of being an abbreviation. The notion of "party" implies "togetherness", and "togetherness" is an essential part of what is being conveyed. (If abbreviations are not recognized and taken into account as such, an endless train of anomalies and paradoxes results.) Perhaps consideration of an old joke will make it even clearer.
old joke: A. "She and I are married." B: "To each other?"; but "Me and her are married." leaves no invitation for such a rejoinder.
To give another example:
The two sentences, "Me and her are going shopping this afternoon." and "She and I are going shopping this afternoon." are not synonymous. The former clearly implies that we are going together, but the latter leaves open the possibility that separate trips are intended.
“Me and her are going shopping this afternoon.” is simply and abbreviation for, “The members of the party consisting of me and her want to go shopping together this afternoon.” - perfectly grammatical, but who wants to say such a mouthful when, “Me and her are going shopping this afternoon.” is a perfectly understandable abbreviation of it?
This also shows that the shoot-from-the-hip advice of omitting part of a sentence in order to analyze what is left is just more schoolmarm mush-headed thinking, much like killing the golden goose in order to find out what makes it work. We don’t say, “Me am going shopping this afternoon.” because there is no question of togetherness. But we do say, "It’s me." in answer to the query, "Who is it?" because the person asking has no way of knowing how many people are involved, and so "party" mode is defaulted to (much like a set of exactly one element in Mathematics - called a ‘singleton’). Of course, if one is making an entrance, "I", followed by an exalted name, is appropriate: "It is I, Don Quixote!" – or if the passage is part of an exalted speech, the nominative is appropriate, as in the Shakespearian line, "We few, we happy few, we band of brothers;".
Captions, such as for photographs, are similarly explained. Everyone recognizes in their gut that "Susan and me" is correct and appropriate as a caption (being an abbreviation for, "Hey, everybody, this is a photograph of (the party consisting of) Susan and me!"), and that "Susan and I" feels like fingernails scratching a blackboard, because for such a photograph the "togetherness" motif is primary, not secondary as in a sentence such as "Me and her are married." - and the construction "Susan and I" negates that togetherness, just as a random photograph in the wild might happen to capture both a bear and a squirrel at the same time.
Note that telling a complete item from an abbreviated item can require context, and the two might not be synonymous. For example ‘gas’ as a complete item refers to natural gas, but as an abbreviation refers to gasoline. Another case requiring context is the expression “I could care less.”. As a complete item it means that the speaker does care to some degree, but as an abbreviation it means “If you think I could care less, you are mistaken.” – which is a much more powerful put-down than “I couldn’t care less.” – similar to the difference between “Hell, no.” and merely “No.”