Every formal question presupposes a domain of discourse from which valid answers can be drawn. Whether or not it is made explicit, a question like this appears to be seeking a mathematical answer. Giving a color-based answer violates the implied domain of discourse, and suggests that the person answering has either accidentally or deliberately misinterpreted the question.
Of course, there are situations where "outside the box" thinking is required, or desirable. In such a case the implied domain of discourse is the "box" and ignoring it is "thinking outside of it."
Formal questions, however, restrict the realm of possible answers by necessity. In a formal system, every recognized condition must be well-defined within that system. That cannot be guaranteed without a restricted domain.