It's a little more complicated than that. But you are starting to ask some good questions about the axiomatic method of reasoning. Let's clarify some ideas.
Today, we use formal systems to reason deductively quite frequently. In fact, there are now logical frameworks that provide a mechanism for choosing all sorts of axioms at the first-order and higher-order by allowing us to treat logical systems as first-class objects in our computations. So, understanding how we choose axioms and what that means to reasoning is very important.
When we build a system of reasoning, we indeed must choose axioms. One of the most famous choices of axioms in the history of logic and mathematics is the parallel postulate of Euclid's axiomatic method is elements. For a couple of thousand years, there was widespread belief both that the axiom might be reducible to the other axioms and that the interpretation of the axiom was immutable. Of course, the end to all of this attention was the realization that it was not reducible, but that there were parallel interpretations. The product were various non-Euclidean geometries.
In reverse mathematics, we often find ourselves searching for new axioms. Often times, we can show using mathematical logic that something cannot be proved with the current set of axioms. Thus, a search for a new axiom to arrive at a sure conclusion from or about the system drive the adoption of a new axiom. The transition of ZF to ZFC exemplifies this happening.
But what you're asking about is weakening a system of logic by rejecting or removing an axiom by presuming it's not true. A historical example of this is the development of intuition logic (IL) from classical logic by rejecting an axiom. IL rejects not one, but two axioms, LEM and DN.
So, when you reject an axiom, you change your system of logic which is usually just phrased as 'having a different logic'. There are all sorts of logics and the interesting one's are the non-classical ones. Click on Bumble's response to 'In how many and which ways can a logic be non-classical? Are there systems for organizing them?' (PhilSE) for more information.
Now, interestingly, one axiom in reasoning that you can reject is law of non-contradiction. Another is the principle of bivalence. From WP:
In logic, the semantic principle (or law) of bivalence states that every declarative sentence expressing a proposition (of a theory under inspection) has exactly one truth value, either true or false. A logic satisfying this principle is called a two-valued logic or bivalent logic.
So, when you start rejecting axioms in a formal system, you can indeed wind up with issues regarding contradictions and tautologies, and whether or not the system of logic is even useful becomes a question. Famously, changing your logic system can result in occurrences of the principle of explosion or other related outcomes. In fact, there is a class of logics where contradiction is even allowed, those are the paraconsistent logics.