It's probably a bad idea to take grammar advice from someone who would answer "Yes" to the question "Do you want tea or coffee?" :-) This sort of thing is just an exercise in interpreting natural language as strict propositional logic, which can be a fun past-time for some species of nerds, but isn't really relevant to the question at hand, which is about correct usage.
@Marcin above, I think, has it basically right: either is probably fine.
Personally I agree with you and use the "and" construction. To me, it feels as if the "and" is being used to describe a set of things, and then the "larger" is like a function that applies to that set.
So I'd say:
"The larger of the two"
"The larger of A and B"
"The largest of A, B, C and D"
"The largest of the Beatles"
What's nice here is that you preserve substitutability amongst the different ways of describing the set, i.e.:
"A and B are the two"
"A, B, C, and D are the Beatles"
Notice that if you use the "or" construction, this doesn't work out:
"The larger of the two"
"The larger of A or B"
"The largest of A, B, C, or D"
"The largest of the Beatles
"A or B are the two" //wrong!
"A, B, C, or D are the Beatles" //wrong!
But of course ultimately what trumps everything is what sounds right to and is used by English speakers, and as @Macin says both ways seem to be ok. But to me this sort of reasoning can break the tie. :-)