tl;dr- You're right, this is a common misunderstanding. While perhaps not particularly mature, calling someone a "bloody idiot" can be a comment outside of the context of a debate. Rather than being a fallacy or non-fallacy, such a statement doesn't exist within the context of the debate at all.
Within the context of the debate, it's essentially a non-statement. As discussed in @CortAmmon's answer, this contextual confusion.
Analogy: Multiplexing serial communications
Most computers can be said to have a single serial port through which they access the internet. All of the data that comes in is like a stream of 0's and 1's, then gets streamed to various endpoints through multiplexing:
.
The above multiplexer description is flat, though in practice multiplexing tends to be a hierarchical process. Anyway, details aside, it's then important for the communication protocol to sort incoming data into the various endpoints that ought to receive it.
With computers, multiplexing tends to require precise framing. Part of this is that ambiguities can be exploited, allowing hackers to get undesired effects:
.
With humans, our language tends to be far less precise, so while we must still multiplex our communications, the routing process involves a lot more inference.
As given in the question statement, Person B doesn't intend for the string "You're a bloody idiot." to be parsed within the context of the argument. Since that string should never be routed to the argument in the first place, it has no existence within the argument; it's neither a fallacy nor a non-fallacy.
At most, Person A can say (in awkwardly precise language):
I've interpreted your prior message as being within the context of our debate, though I have not found a non-fallacious interpretation of it, such that I believe that your thinking is in error. Specifically, I believe that your assertion that I'm a "bloody idiot" has some bearing on the issue being discussed, which is an error (fallacy) on your part.
But to be more concise, they might simply say:
That's a fallacy.
I suppose that a correct response might be to say:
No, I wasn't continuing the debate. I'm making a comment outside the context of the debate in which I'm expressing my belief that you're a bloody idiot. By this, I mean [elaboration].
Or to be more concise (if not a bit terse):
No, I wasn't arguing with you; I'm just calling you an idiot.
Not to say that this is polite or acceptable in moderated venues, e.g. on StackExchange, however it'd be logically correct.
On moderated venues, it may simply be easier to stop responding. This also has drawbacks in cases due to the public nature of such forums, though I guess various constraints limit how much can be effectively communicated anyway.
Detail on the multiplexing analogy
Revisiting this answer, I'm not sure how obvious my multiplexing analogy may've been, so this section is meant to explain it a bit.
The basic point is that a "fallacy" is an invalid construction within a debate:
A fallacy is the use of invalid or otherwise faulty reasoning, or "wrong moves" in the construction of an argument.
-"Fallacy", Wikipedia [references and formatting omitted]
So, obviously, something must exist within a debate to be a fallacy within it. For example, someone can turn on a TV, but if that's not part of a debate, it can't be a fallacy or non-fallacy within it.
The main point of this post is that folks might be confused due to the assumption that all words in an exchange in which a debate occurs are necessarily part of the debate. But, that's the basic mistake.
Instead, people can exchange words, including having one-or-more debates, without all words being part of all of the on-going dialogs. This is like how a computer can play music from the internet while a user is checking their email; the email app and music app are both streaming info over the same serial line, but the music doesn't exist in the email app nor do the emails exist in the music app.