I think there are two sides to your question: whether the "contradictions" you state actually pose a problem, and whether it is even physically possible to have an infinity of something.
To the former, the answer is that though your description of what one could do with infinite apples "seems absurd," it is nonetheless perfectly valid. If indeed I have an infinity of apples (for simplicity let us assume it is the smallest infinity, countably many apples - a set of cardinality aleph-null), then I could take countably many apples from that set and make as many infinite sets of apples as I want - in theory. It is just a mathematical fact that one can remove countably many numbers from a countably infinite set, and if one does it right, one will get two infinite sets. And one can do that countably many times, getting an infinite number of infinite sets of apples.
However, I think the philosophically relevant issue in your question is whether this could even be possible in practice. Sure, theoretically infinitely many apples isn't a logically incoherent concept, but you want to know if it can ever happen.
Well, the shape of the Universe is presently an open question, but what looks most likely is that it is flat and infinite in extent. It is a physically coherent (and presently quite popular) theory to suggest that the Universe is infinite in size, and by corollary (assuming the cosmological principle of homogeneity and isotropy holds), that there is an infinite amount of matter in existence.
However, it's important to note that this is not a useful fact. Because of limitations such as the speed of light and the Planck scale, which both limit the amount of information we can receive, it really isn't possible to have, within one's range of observation, infinitely many things. This is a twofold argument. First, if the objects have finite size, they must take up infinite volume and thus extend to infinitely far away. Thus, in finite time (which is the only sort of time we have), we will never have information about all of the objects (in fact, we will always have information from almost none of them, mathematically speaking) since there will never be enough time for light to carry their information two us. Second, if the objects are infinitesimal small (if that is even possible), they will be effectively unobservable due to the way the Planck scale works, so we won't even see them.
So the summary is that infinity is physically coherent but not useful on any remotely familiar scale.