6 of one, half a dozen of the other. The computer, because it does so many different things for many different people, and because applications of that computer abstract all the nitty-gritty hardware from the "user experience", is thought of in many different metaphorical or analagous contexts. These various analogies we use to describe computers call for differing prepositions when describing the tasks, and sometimes two prepositions, coined in the context of a particular computer analogy, become commonly used.
Personally, I think "log in/out" is the better term. This is because I think of the computer as a part of the networked IT system, and when I enter my username and password, the action of me beginning work is "logged in" to the computer and the system behind it. I think of it similar to if I punched a time clock every day, and "clocked in" and "clocked out".
However, "log on and log off" could have similar analagous meaning to someone: the terms "sign on" and "sign off" have been common in radio and television for decades before the personal computer entered the scene, and in the context of going "on air" vs "off air", or "on duty" vs "off duty", these acts would translate to the computer pretty seamlessly.
There are many other terms used. For instance, I see "sign in" and "sign out" a lot in websites; GMail and the BofA website use these terms to refer to their security session management. This sounds just as logical as "logging in/out" to me.