Here's what Google is telling me:
- "mono-channel" returns ~500,000 results
- "monochannel" returns ~100,000 results
- "multi-channel" returns ~19,000,000 results
- "multichannel" returns ~13,000,000 results
I would say that to describe a configuration involving multiple channels, use either "multichannel" or "multi-channel". Do not use both in the same paper; you want to maintain consistency.
To describe a configuration involving a single channel, "mono-channel" is acceptable, but "monochannel" is almost certainly non-standard (even my browser underlines it as a spelling error).
Personally, I would use "single-channel" rather than "mono-channel", as single is sort of the complement to multiple. Using the prefix mono- implies that you also want to be using the term stereo-, which is more specific than multi-.