2

The following argument concludes that the common understanding of ChatGPT (trained on text, receives online users' text questions, etc.) is not supported by the science. What criticisms are there of the argument which show it is unsound (one or more false premises) or invalid (breaks the rules of formal logical deduction)? The argument is:

In November 2022 AI research firm OpenAI released its online "text" chatbot ChatGPT. The following great interest was based on the popular and expert-led conviction that ChatGPT did, or with suitable (multi-billion-dollar) development might, understand the text questions online users type.

ChatGPT exists inside the electronics of computers. The research I conducted indicates that the science of the physics and chemistry of electronics doesn't support the idea that ChatGPT could understand online users' text questions. The reason is as follows. The only place text exists is on user keyboards and display screens. Everything else is electromagnetic radiation (WiFi, Bluetooth, fiber optics) and electrons (in wires and semiconductors).

ChatGPT never gets the text questions online users type. All it gets is electrons. The internet is made of electronic storage. All that's stored on the internet is electrons. There's no text on the internet. The ubiquitous claim that ChatGPT is trained on text stored on the internet is simply false.

Relevant quoted and referenced statements of OpenAI and respected media allegedly falsely describing ChatGPT can be found in Does ChatGPT understand text (PDF).

Roddus
  • 721
  • 3
  • 15
  • Comments have been moved to chat; please do not continue the discussion here. Before posting a comment below this one, please review the purposes of comments. Comments that do not request clarification or suggest improvements usually belong as an answer, on [meta], or in [chat]. Comments continuing discussion may be removed. – Philip Klöcking Feb 25 '24 at 19:26
  • 12
    A completely bizarre analysis. You might as well argue that there is no text in a printed book, only splodges of black ink splattered over the paper. – Michael Kay Feb 25 '24 at 23:15
  • 3
    "All that's stored on the internet is electrons." All that comes into your eyes is electrical impulses generated by your retina / optic nerve. I think you are getting hung up on the word "text" as a physical thing rather than information, in whatever physical forms it may take or be transformed into. – user4574 Feb 26 '24 at 03:26
  • Nobody who understands what LLMs are has ever genuinely thought they understand anything... They are designed to predict what the next word would be based off of the input and training set. That's clearly not the same as understanding the underlying concepts. Just ask it to do some basic maths and you'll see this. That being said, this analysis is ridiculous... – ScottishTapWater Feb 26 '24 at 10:51
  • Check out Hinton's take on this issue. – Ajax Feb 26 '24 at 11:34
  • I think this should be reopened. – CriglCragl Feb 26 '24 at 12:37
  • 1
    @CriglCragl Thanks for the support. I've edited the question and removed references to myself and also made the question more formal. Hopefully that will meet the tests. I was looking for criticisms of the argument. I wanted to be told them and to understand them. So hopefully the question will be reopened. – Roddus Feb 29 '24 at 06:10

9 Answers9

15

The argument is irrelevant. You can apply the same logic to say that humans don't understand text, since text doesn't exist in your brain. There are lots of software applications that read text- if you integrated one with ChatGPT, would that mean the integrated system suddenly 'understood' text in a way ChatGPT did not? You can tell Rodsmith from me that the argument is nonsense, and he should present himself at my study if he requires further clarification.

Marco Ocram
  • 20,914
  • 1
  • 12
  • 64
  • You say, "There are lots of software applications that read text". I find this is one of the strangest things much like the denialism following disconfirming instances of phlogiston theory. My background is in software development and I found it really difficult to overcome the view of computers as symbol (i.e. text) processing systems. Understanding Turing machines helped. They internally manipulate text (0, 1, x, y, ::, etc.). Everyone says computers are Turing machines. They're not. Computers internally manipulate electrons. Electrons are not text. They're sub-atomic particles. – Roddus Feb 24 '24 at 21:39
  • 6
    You cannot use vehicles to go from one place to another because vehicles are nothing else than protons and electrons. Would you agree? Probably not. Similary with computers. They don't internally manipulate electrons. We build our computers, for some historic reasons, as electronic circuits that use various effects of electricity but all our computers manipulate are their own internal state. The computer is only concerned with this state and not the underlying physical processes we use to build them upon. – Gábor Feb 24 '24 at 21:48
  • 9
    @Roddus you are fixated on the visual appearance of text, as if it were important. It's not. Blind people can learn to read without seeing text. The symbols are just that- symbols. When people say that CGPT is trained on text, they don't mean it is trained on the visual appearance of printed text. – Marco Ocram Feb 24 '24 at 22:20
  • @Marco Ocram I agree I'm fixated on shape. I press certain keyboard keys then on my screen see the text shapes. I click "Send". If this is an email then the human recipient sees the same shapes appear on their screen. I had a certain meaning in my mind. They react to the shapes. They now have the same meaning in their mind. Now if ChatGPT does not react to shape, and it doesn't, then how could it possibly come to get my meaning? How could it understand my question? The shape, via sensation, activates the meaning. No shape then no activation. No activation then no understanding. – Roddus Feb 25 '24 at 01:20
  • 1
    Suppose then that I compose a delightful sonnet in my head- where is the text of the sonnet? – Marco Ocram Feb 25 '24 at 07:11
  • Can you understand speech? No text, no visual appearance, just modulated audio. – keshlam Feb 25 '24 at 22:33
  • @Roddus the brain also does not manipulate any text. The main issue is that you are confusing "input" and "processing". ChatGPT do not have eyes, so they get the input from other sources (as blind people can read braille, or can listen to podcasts, or can imagine the text in spoken form). The processing of that input (a totally separated process) is pretty much using electrons in neural networks in both cases (brains and A.I.). The mechanism by which text comes to the neural network is absolutely irrelevant, and (to the scope of the question) the processing is the only important part. – Guilherme Feb 26 '24 at 05:41
  • Also, you are mixing "text" and "language". What ChatGPT and our brains processes is language (unless you are thinking about text visually, and BTW I.A.s can do that to, just ask one to generate an image of a sign with something written on it). The fixation on text is pretty much removing the role of the brain and putting it on your eyes, which really messes up the entire argument. And BTW again, you can download and install Copilot and use the camera of your smartphone as eyes. It can read any text. The process is fundamentally the same, the difference is that one is metal and other is meat. – Guilherme Feb 26 '24 at 05:51
  • 3
    "You can tell Rodsmith from me…" I suspect you already know this, but for other readers it might be worth noting that the OP actually is Rodsmith (as can be seen in their bio page). The question is a strange attempt at self-promotion. – Michael MacAskill Feb 26 '24 at 05:59
  • @Roddus What does the fact that computers manipulate elections has to do with them not being Turing machines? Like a Turing machine is the abstract idea behind a computer and usually a computer fails it by practical means such as limited memory and processing power, but if you were to ignore that the analogy still fits, doesn't it? – haxor789 Feb 27 '24 at 10:10
  • @Roddus Also no if you sent an email the recipient does NOT have to see the same text that you had sent. Actually you're not sending text but you encode text to a bunch of numbers and the recipients computer looks up these numbers in a table and replaces them with prerendered images. And they can differ quite a lot, might google "font types" or what if the other person uses a different lookup table altogether and transliterates that to Cyrillic or Chinese or whatnot where you get what the other person is sending but wouldn't be able to read your own language. – haxor789 Feb 27 '24 at 10:21
12

ChatGPT never gets the text questions online users type. All it gets is electrons. […] The ubiquitous claim that ChatGPT is trained on text stored on the internet is simply bonkers.

This argument is based entirely on a misinterpretation of what the word "text" means in this context.

In computing, "text" means a sequence of codes which are designed to represent written words, essentially by using one code for each character in the writing which is to be represented. It is obvious that ChatGPT is trained not on text in the sense of matter physically arranged into written words, but on text in the computing sense.

You're a software developer, so presumably you're familiar with the use of the word "text" to mean a sequence of character codes. It's not clear why you chose to interpret the word "text" in a way that produces a "bonkers" claim instead of a sensible one.

The argument says nothing about ChatGPT's ability or inability to understand these sequences of codes, so there is no need for me to say anything about that either.

By the way, I notice that the idea that symbols are shapes rather than other types of patterns seems to be very important to you. In this question from 2019, you asked why Alan Turing described computers as manipulating "symbols," given that the things that they manipulate are actually voltages and not shapes. In this question and this question, both from 2022, you point out that when people do what they call a Turing test in real life, the machine almost always receives electronic pulses instead of shapes, and you assert that "[w]ithout exposure to the shapes, the machine could not possibly understand the questions."

Why do you think that the distinction between shapes and other kinds of codes is important? When people listen to each other speaking, they don't hear any shapes, they only hear information about frequencies and amplitudes; does that mean that it is impossible for any human to ever understand speech? If a person has never seen any shapes representing words in their life, does that mean that that person is unable to understand language at all?

It seems like you keep repeating the argument "computers don't receive shapes as input, therefore computers can't understand anything" over and over, presumably hoping that someone will eventually agree with your argument (or at least understand it), but you've never given any reasoning for why you think the argument is valid. The argument seems like a non sequitur to me.

Tanner Swett
  • 930
  • 4
  • 11
  • Electronic engineers know that no text (ABC123...) is received by or processed inside computers. I argue that ChatGPT is said to receive such linguistic text. Why? Because, it is trained on text ("books", "articles",...) found on the internet. That's what's commonly said. But you argue that the term "text" when applied to computers means groups of electrons (clocked or in semiconductor traps, etc.). I've worked with assembler and C and not heard of his meaning. But assuming it exists, have Sam Altman and others explained that by "text" they mean something totally different from everyone else? – Roddus Feb 24 '24 at 23:03
  • (Cont...) "I want to distinguish between shapes and voltages for a semantic reason. We understand the meanings of shapes (as in reading a textbook). We don't understand voltages (have no clocked-voltage sensing modality). – Roddus Feb 24 '24 at 23:35
  • (Cont...) You say "It seems like you keep repeating the argument "computers don't receive shapes as input, therefore computers can't understand anything" over and over... [it seems] like a non sequitur". My argument is a bit different. Searle's CRA says computers just receive interpretable shapes, and such shapes in themselves mean nothing (interpretation being extrinsic), and all a computer gets is the shapes, hence no computer could understand anything. I reject this argument concluding computers, once realizing principles of knowledge acquisition via sensory apparatus, could understand. – Roddus Feb 24 '24 at 23:46
  • 5
    @Roddus "I've worked with assembler and C and not heard of his meaning." – You've never heard anyone describe a file as containing "plain text" before? The official documentation for C, Python, and HTML all describe those languages as consisting of or handling "text," meaning sequences of character codes, not text in a physical or visual form. If you've never heard the word "text" used in the way that those sources use it, I can only say that your experience has evidently been very different from mine. I apologize for not responding to your other comments immediately; I'm short on time. – Tanner Swett Feb 25 '24 at 02:27
7

First of all - LLM has no conscience so one can argue they do not understand anything at all. Does a flashlight understand that you switch it on when you push a button on it? No, it's just turned on cause there is electricity that starts flowing. If we continue this line of reasoning why does it matter at all in which form LLM process data? We shouldn't anthropomorphize LLMs as consciously understanding. They have no idea they even exist not to mention text understanding.

On a more technical note - the paper you provide leans quite heavily on Searle's CRA, but does not address counterarguments that have been made against it. The thing is reference, and semantics are not intrinsic qualities of arbitrary symbols. They emerge from the relationship between internal states of an agent and its external environment, through the intercession of the senses. Symbols must be "grounded" in sensorimotor interaction with the world to take on meaning.

The Chinese room, for example, cannot claim genuine semantics because it lacks grounding - its symbols derive from rules given by external designers, not internal perception and action. The manipulated symbols are not meaningful.

By relying on intrinsic semantics, the CRA ignores the actual relationship between representation and reference that gives rise to true semantics. That's a weakness.

Groovy
  • 637
  • 7
  • 2
    +1 Out of curiousity, what constitutes "genuine semantics"? – J D Feb 24 '24 at 15:54
  • I agree with the need of grounding via sensorimotor apparatus, and Harnad's arguments are well known. But I see the CRA as having two conclusions. The first is sound (has true premises) but the second isn't. The first argues no computation, the process, could understand what it receives and internally manipulates because these items ("symbols" meaning text) have no intrinsic semantics. They do not carry, contain or indicate their meanings. But the second argument has two false premises: (1) that computers, the hardware, manipulates text, (2) and do nothing else. The CRA does ignore grounding. – Roddus Feb 24 '24 at 21:22
  • (Cont...) but I don't understand "the actual relationship between representation and reference that gives rise to true semantics". If by "representation" one means an internal structure, and by "reference" one means things in the outside world, then some relationship between the first and the second seems to be the classical idea of how words mean things (I might have this wrong). However a full understanding of semantics will presumably explain why solipsism can't be rebutted. There might be no external world at all. Semantics will need to encompass more than just the inner-outer relation. – Roddus Feb 24 '24 at 21:31
  • @J D, "Out of curiousity, what constitutes "genuine semantics"?". In short, I think a place to start considering semantics is with understanding (a process), knowledge (a structure) and intentionality (a property of the structure). Of course this needs explanation. Geoff Hinton recently said "semantics, whatever that is". Greg Brockman said as much. They don't know. Yet thinking, the semantic process yielding understanding, is key to intelligence. In other words, the field of AI research doesn't really know what intelligence is. But structure, process and property seems a place to start. – Roddus Feb 25 '24 at 01:34
  • Or maybe understanding has nothing to do with thinking, maybe it’s not verbal but visual? Do you see what I mean? – Yuri Zavorotny Feb 26 '24 at 08:43
  • @Yuri Zavorotny sure, understanding text must be visual to some extent, since the text is observed with the eyes, and understanding text is a reaction to the text shape. But to say the the understanding is purely visual I think is going too far. For instance a person might say the understand what "rotten egg gas" means but when smelling it says they don't know what the smell is, so do they understand what "rotten egg gas" means? Does understanding involve various sensory modalities? Does it involve grounding in the world in the sense of multiple sensory modalities? – Roddus Mar 02 '24 at 02:32
  • @Roddus -- thanks for your question. I think humans have evolved a capacity that is not present in other species: we can visualize a simulation of how things work in reality. This is how we understand things -- by assembling and visualizing their interactive models. Language, whether spoken or written, was evolved as a communication tool. We want the other person to understand us -- that is, we want them to see in their mind what we see in ours. Unfortunately, we cannot telepathically transmit our understanding to the other person's visual cortex. So, instead, we describe it with words [...] – Yuri Zavorotny Mar 02 '24 at 05:26
  • [...] hoping that the other person will be able to reconstruct the model that we are describing in their mind, like a puzzle. And if they do -- and only if they do -- the communication becomes successful. Often this process is iterative (a.k.a. dialog), with the listener asking for the missing puzzle pieces, and the speaker describing them to the listener, going back and forth until the listener completes the puzzle. See what I mean? – Yuri Zavorotny Mar 02 '24 at 05:34
  • @Yuri Zavorotny Yep I see what you mean. It makes good sense, specially the dialogue part to correct any perceived possible misunderstanding or lack of understanding. AI researchers often seem to have ignored that dialectical process. – Roddus Mar 05 '24 at 03:36
7

People are quick to point out that AI is merely electricity passing through some wires and transistors and such and to perform basic mathematical operations, and therefore AI is not intelligent like humans are.

But these arguments generally seem to take the form of "AI is like this, therefore it's not like humans in terms of understanding", without actually addressing what humans are like. That's comparing 2 things by only analysing 1 of those things. Considering the mechanics of human intelligence (as best we understand it) doesn't seem to end up in favour of those saying AI doesn't "understand" or aren't "conscious". Human brains are, after all, a bunch of connected neurons passing electrical signals between one another, in some chemical soup. Some might say there's more to human consciousness than our brains, but we have no evidence of this, and every indication that consciousness is strongly and directly tied to the physical parts of our brain.

It's probably fair to say the physical parts of the brain is more than just computation (which arguably set it apart from modern AI), although we're not currently able to say which mechanisms in the brain makes a human conscious, and how it does so. Even the people who says the mind exists beyond the brain still can't describe the mechanism for understanding or consciousness within such a mind (nor describe how such a mind would interact with the brain on a mechanical or logical level).

Knowing a thing or two about AI doesn't make someone an expert in neuroscience, yet some people act as if it does, as if they have a perfect understanding of the mechanisms of "consciousness" and "understanding" in humans, for them to be justified in saying AI isn't conscious.

This "it's all electrons" seems to be taking a few steps up on the scale of smart-sounding nonsense.

One could say "ChatGPT never gets the text questions, because it just gets electrons", but what does it even mean to "get" something? You could similarly say that your brain doesn't "get" the things you see or hear because all it receives is electric signals from your ears and eyes via your nerves, and the brain itself works largely by electric signals being passed between neurons.

* I briefly scanned through the paper. It does mention the need to "chose[sic] a place to start trying to understand the mind", but it's hard to make sense of the gibberish that follows as anything that could be said to differentiate human minds from AI.

NotThatGuy
  • 9,207
  • 1
  • 18
  • 32
  • saying "it's all electrons" seems to be taking a few steps up on the scale of smart-sounding nonsense". Well at least it's smart-sounding. But the point is that ChatGPT gets electrons, that's all. Virtually everyone says that ChatGPT does or might understand what it gets. I press my keyboard's character shapes. I click Send. Electrons or EMR go out. A human email recipient sees the same shapes. They react to them and understand what was in my mind. ChatGPT receives electrons. It doesn't react to shape. How does it know what was in my mind when I pressed my keyboard keys? – Roddus Feb 25 '24 at 01:59
  • @Roddus From some brief research, when you "see" something, the input your brain receives via nerves is ions. The input ChatGPT receives is electrons. Those are similar enough that I don't think you can say one understands and the other doesn't based on that. That's why the "it's just electrons" argument fundamentally doesn't work. – NotThatGuy Feb 25 '24 at 19:48
  • 2
    @Roddus "How does [ChatGPT] know what was in my mind" - how do you know a person knows what's in your mind? They respond appropriately and you can ask them questions to investigate. You can do the same with ChatGPT. From that, you might conclude its dynamic long-term memory is functionally non-existent, and it often doesn't respond appropriately. But that applies to some extent to people too, and the next generation of AI might not have those problems, at which point it would be much harder to argue that AI doesn't "know" things. – NotThatGuy Feb 25 '24 at 19:48
  • @Roddus You don't think in letter shapes you think in the images that are encoded in these letter shapes. The encoding is different you see letter shapes while ChatGPT encodes .... well technically strings of chars, which stripped down are just patterns of 1s and 0s so think of it as something like Morse Code. Actually Morse code is a pretty good example because you don't think of it as beeps and pauses, it could also be light flashes and darkness it could be a person blinking or keeping their eyes open... There are million ways to impliment the concept. – haxor789 Feb 29 '24 at 10:10
  • @haxor789 This is the basic problem as I see it: When 2 humans communicate by email, the sender has a certain understanding in their mind. They press keyboard keys inscribed with certain shapes. The shapes don't contain or indicate the understanding (per Searle's CRA etc). Electronic transmission occurs. The receiver's device displays the same shapes on its screen. The receiver sees the displayed shapes and understands what they mean. There is a meeting of minds. The receiver understands the message. ChatGPT never sees the sender's shapes. How does it understand the message? – Roddus Mar 02 '24 at 01:56
  • @NotThatGuy I've just responded to haxor789 with what I think is the problem with the idea that ChatGPT could understand an online user's text message. The gist is that ChatGPT never sees the message in the first place, so how could it undersatnd what it means? – Roddus Mar 02 '24 at 02:02
  • @NotThatGuy on your argument that our brains receive electrical neural pulses yet we understand the text we see, the point I think is that we don't understand the meanings of our neural pulses. What we understand is the meaning of external things, text characters, and we understand these by way of reaction to their shapes. But ChatGPT has no access to the external shapes. I really can't see how ChatGPT could "get to" the meanings inside the sender's mind without reacting to the shapes the sender pressed, and which would be redisplayed on a human receiver's device. – Roddus Mar 02 '24 at 02:23
  • @Roddus It doesn't: https://chat.stackexchange.com/rooms/151671/discussion-on-question-by-roddus-does-or-could-chatgpt-understand-text – haxor789 Mar 02 '24 at 16:53
5

First of all, I'm involved in NLP, so thanks for the paper. Some questions that come through here are malformed grammatically, so it's always a pleasure to find a semantically apropos, well cited question.

Obviously, it all hinges on what it means "to understand". The questions about the transformer model and their implications about semantics is a serious philosophical matter if you are willing to move past the claim computers only twiddle bits. The gulf between AGI and AI is best outlined in the recent book The Myth of Artificial Intelligence by Erik J. Larson (GB). Essentially, LLMs and other ML strategies have human-like abilities, but they don't rise to the level of general intelligence.

The point Larson makes is that big data methods like LLMs (which are statistical language models that rely on massive corpora) are inductive techniques and that whatever general intelligence is, it at a minimum is something that integrates deductive, inductive, and abductive methods. As he points out, there isn't even a theory to put to empirical test. In theory then, it's open philosophical as well as scientific question if general intelligence can be exhibited by non-biological systems of computation. (See the SEP's Physical Computation for a thorough treatment of what computation is best described as.)

Thus, it's strong to claim that computers do not understand with general intelligence. Note, there is no firm argument that computers cannot understand. John Searle's argument is considered by many thinkers to be severely deficient. For a good introduction to the criticisms thereof, see Preston & Bishop's Views into the Chinese Room. If you believe that the systems response carries weight for instance, then there is no serious argument that general intelligence cannot emerge in a significantly complex, non-biological system as far as I have found.

Does ChatGPT only get and process electrons? No, this claim is based on a category mistake. Electrons are ontological primitives in the discourse of electrical theory. ChatGPT is a statistical engine that processes vectorized, arbitrary token sequences based on frequency. Anyone conflating those two domains of discourse doesn't understand the basics of how reductionism in scientific theory (SEP) works. In fact, human understanding is built on processing morphemes (mostly) deterministically and contextually in the bigger picture of discourse (See dynamic semantics (SEP) for some insights there), and LLMs do not have access to that sort facility, even if you plunk an LLM into langchain and add some lightweight prompt engineering, use embedding strategies, etc.

Be cautious of anyone who philosophically abuses metaphysical reductionism and who couldn't explain in plain English what a transformer is. If you really are intrigued by the topic, I would encourage you to also read natural language ontology (SEP), categories (SEP), and concepts (SEP). At a minimum, the mimicry of language with formal semantics and human-level intelligence and natural language use are easy to confuse, but also easy with the right technical background to expose as distinct categories.

See also:

J D
  • 26,214
  • 3
  • 23
  • 98
  • Comments have been moved to chat; please do not continue the discussion here. Before posting a comment below this one, please review the purposes of comments. Comments that do not request clarification or suggest improvements usually belong as an answer, on [meta], or in [chat]. Comments continuing discussion may be removed. – Geoffrey Thomas Mar 07 '24 at 10:41
2

I see from OP's profile that they wrote the paper!
It defines what they understand by "text".

4.1 What is text?

Text is written or printed language such as we find in textbooks. It comprises instances of shapes. But not every instance of a shape is text. A shape is text if it has been assigned a meaning, or interpretation. This is often done by a community.

Such shapes might be atomic such as A, B, C, D and 1, 2, 3, 4, that is, letters of an alphabet and digits. Or they might be such shapes sequenced into words, multi-digit numerals, and so on.

Types of text characters include the just-mentioned words and numerals, and also punctuation marks, special characters such as %, &, @, mathematical symbols, +, -, ^, = and others. In short, the shapes one finds imprinted on the top surfaces of keys of a computer or typewriter keyboard.

To understand text is to know the meanings of its shapes. Understanding text presupposes learning the meanings of the shapes. This is what many enthusiasts think ChatGPT does. They say it's rained on text scraped from the internet and that this training results in ChatGPT learning the meanings of the text shapes.

The last sentence undermines the whole premise of the paper:

The text shapes are not what is transmitted to ChatGPT. It is the electronic standard representations of the text, and the shapes of the letters are defined in the webpage style section (or overridden by the reader). ChatGPT won't care what font the webpage's author preferred the text to be read by humans. It is irrelevant. It is still text.

It is a symbolic representation of text, as defined more than 60 years ago with ASCII, and updated more recently with variants of Unicode. But it is still a symbology of the text, and not the actual shapes.

So ChatGPT is not sent shapes, but a representation of the text which allows it to process the semantics. Human readers are not sent shapes either, but instructions as to what shapes their browser should use, to make the text readable by humans. But ChatGPT does not convert the text representation to shapes, and then apply an OCR (optical character reader) to convert the shapes back to a simpler representation of text which it can process, which it was sent in the first place.

The author might just as well have written a paper explaining that Morse code wasn't about text, because it wasn't transmitting actual "letters" and "words" but pulses.

But they overlook the semantics of what the pulses, electrons, whatever carry, which is contextual (pun intended).

Weather Vane
  • 394
  • 2
  • 8
  • And there's a second error that's probably even bigger: "this training results in ChatGPT learning the meanings of the text shapes." Nobody ever claimed that ChatGPT learned the meaning of anything. Some tabloid papers might have written something like that but nobody involved with the system ever claimed anything even remotely similar. ChatGPT has no connection to the meaning at all. – Gábor Feb 26 '24 at 09:36
2

  1. The question whether ChatGPT understands text has the pragmatic answer "Yes": Every user can confirm this by entering text into the text-interface of ChatGPT.

  2. The deeper question is: Does ChatGPT understand the meaning of text?

    Here is the answer from ChatGPT itself:

    As an AI language model, I don't "understand" text in the same way humans do. Instead, I process and generate text based on patterns and associations learned from vast amounts of data during training. When you ask a question or input text, I analyze it, identify patterns, and generate responses based on those patterns. While I can provide coherent and contextually relevant responses, I don't have consciousness or subjective understanding like humans do. My responses are generated algorithmically based on statistical patterns in the data I've been trained on.

    The answer shows what ChatGPT is able to achieve and where it (i.e. it's designer team) locates its limit.

Jo Wehler
  • 30,912
  • 3
  • 29
  • 94
1

The old "It is nothing but a stochastic parrot" argument. Now it is true the main building blocks of ChatGPT and other LLMs do just that: User their neural net to predict on what word comes next. Not much intelligence there.

But then again: This is also a big building block on how our brain works: We lookup things in our own neural net: You are presented some new problem and immediately (without "thinking") some ideas come to your mind. People call it "intuition" or "experience". You hear a word and you find one that rhymes with it and you call it "creativity". Is there conscience? No. We do not know how we arrived at our "intuition" and neither does the LLM.

Now it is easy to see how these building blocks can be arranged to "mimic" conscience: Feed it some input and let it reflect on it. Let it checks the consistency of its results. Let it create new associations from that. Feed it back and let it reflect on it again, etc, etc..

Is this conscience? Well it would certainly look a bit like it. You could ask such a system what it was "thinking" and it could truthfully tell you that it was currently reflecting on some logical errors it found in the writings of Thomas Aquinas and then used its time to search the internet to find places where someone else spotted that but was not successful in finding much.

I think even a system that is based purely on text can get pretty far. Can you understand our 3D reality? At least such a system can reason about all its properties as it knows mathematics. Just as the human mathematician can reason about abstract 5D spaces. But can it understand what it means to enjoy a sunset on the beach when it has never "seen" a sunset? Leaving aside the fact that newer LLMs are "multi modal" and have images, video, sound as input and output, even the text can get you pretty far. All the works of literature that write about what humans feel. The understanding of biochemistry that governs the human body will also help. Sure this kind of intelligence is a true "alien" intelligence. It is the first alien intelligence mankind contacts.

And then the last part: The argument that these systems do not have a "world" they live in where they can collect their own experiences. Now this ignores that there is reinforcement learning.

A chess computer learns by playing endless games of chess. It has a 8 by 8 checkerboard world. Soon the coding assistance software will not only "autocomplete" your code but will write programs, then compile then, run and test them, profile their performance and learn from that "experience". It has an abstract world of "software and programming" where it navigates and learns.

The "universe" of these systems is quite different from ours. So there will always be a bridge where we do not understand them and they do not understand us. But I find it quite arrogant to assume that THEY are limited where actually WE are. THEY can create new universes where they "live" that we can't.

As Wittgenstein famously said:

if a lion could speak, we could not understand him’ because a lion’s form of life is so alien to ours that we cannot seriously claim to know what the lion means by what the lion says.

So the main fallacy of the critique on AI is that they confuse "intelligence" and "human intelligence".

And all of this is not science fiction: All the above either exists or is actively worked on and literally trillions of dollars of budget are directed that way.

So the answer to the question is:

Can it ChatGPT understand text:

  • currently not but within the next few month most likely yes.
  • yes. but it will never be able to understand it the same way as a human. as much as we can not understand the lion.
mond
  • 149
  • 1
  • 4
  • The Chomsky reference is great, and the points are also great. For the conclusion, "within the next few month most likely yes ... [but not] the same way as a human". I think that LLMs need sensors and different algorithms in order to understand text (won't be LLMs anymore), and the understanding will be predicated on what we do: looking at text. But the connective relationship between storage locations is spot on. How the connections are established and followed will be different. But I think the understandings such as of "tree" and of "mud" can easily be the same as a human's. – Roddus Feb 25 '24 at 00:03
  • But to add to this, I think the machine's sensor and effector apparatus generally will need to be much the same as ours, since understanding "mud" includes its smell, feel, taste, viscosity etc., and one could also say includes the ability to avoid slipping when moving over mud. – Roddus Feb 25 '24 at 02:28
1

The answer is pretty simple and short:

The argument is false. Here's why:

The presence or absence of text in the guts of computing machines is completely irrelevant. What matters is information. Shannon information. That's what human language is about and what LLM models, and deep learning models in general, deal with.

You better coherently establish the meaning of understanding if you're gonna argue computers don't have it. Since I don't suppose the author successfully and satisfactorily defined it, his argument is doomed.

nir
  • 4,786
  • 16
  • 27