Category: AI

Gottfried Wilhelm Leibnitz (1646-1716), who famously invented calculus at the same time as Isaac Newton (Newton got the credit; we got the Leibnitz notation, thankfully), also believed that if we could discover the correct way of representing our world in symbols, we could calculate the world using logic and mathematics. He also invented binary mathematics and made significant advances in logic to this end.

Stephen Wolfram is on a similar mission to make the world computable. For about 40 years, he has been working diligently on this project, starting with Mathematica, then several years ago expanding that to the Wolfram Programming Language, and more recently to the Wolfram Physics Project. Along the way, Wolfram has expanded on the ideas of von Neumann, Turing, and Godel by describing the principle of computational equivalence, and by carefully differentiating between the traditional, formula-based methods of math and physics, and computational or algorithmic methods.

Large Language Models (LLMs) like GPT-4 (ChatGPT) provide text that sounds like their training data, given prompts that they can use to predict what text most likely comes next. They neither encodes things about the world based on knowledge of how the world works for possible later computation (Leibnitz), nor do they compute answers to questions about the world by feeding knowledge into a compute engine (Wolfram). They simply predict what they think a human would write based on about 1500-3000 words of written prompt.

What do these have to do with the AI Knowledge Problem?

They represent three fundamentally different ways of dealing with it.

Leibnitz, based on the belief of his time that the world was inherently knowable, started with quantifying knowledge about the world. He believed that if we could solve the problem of describing the world in a consistent, logical, and rigorous manner, we could then perform computation based on that knowledge to make new discoveries. Alas, Leibnitz was burdened with living in a time before computing machines were available for him to fully experiment with his ideas.

Wolfram, who is burdened with the additional knowledge of the past 300 years, most specifically that of Godel, who showed that there is a limit to what is knowable within any system and that nothing is 100% knowable, chose to start with computation. He seems to believe fundamentally that if we build a good enough compute system, then we can feed it knowledge and get good answers. He has had some success with this approach. Most of us use his technology, even if we don’t know it. Wolfram Alpha, his “search engine” built on Wolfram Language, powers big chunks of Siri and Bing.

LLMs basically ignore the knowledge problem altogether. The most generous thing that can be said about LLMs and knowledge is that they regress to the mean of the training data, and if the training data overwhelmingly supports one hypothesis, the LLM will likely reproduce that hypothesis and some supporting statements. Of course, it can be prompted in a way to argue against a prevailing hypothesis. Unless the OpenAI safety engineers have decided that doing so causes a problem.

So we have a few choices about how to approach the AI Knowledge Problem.

Option 1 – The Leibnitz Paradigm. We can start with knowledge, properly quantified and qualified, and add compute and a friendly interface. The challenges here are several, not the least of which is that there are fundamental limitations on knowability which will require us to qualify our knowledge with a measure of uncertainty. Luckily, we have developed the mathematics necessary to do this. There are societal barriers as well, especially that those who profit from certainty do not welcome uncertainty, and they certainly do not gladly accept falsification. But to advance our knowledge, we need both – to acknowledge uncertainty and to welcome falsification.

Option 2 – The Wolfram Paradigm. We can start with computation and add knowledge and a friendly interface. This has the advantage of appearing to be useful more quickly by providing answers earlier in the process of making the world computable. But we have a GIGO (Garbage In, Garbage Out) problem when we scrape the internet for the knowledge to feed our compute engine. Without a rigorous understanding of the quality and certainty of the information going into the compute engine, we cannot assess the quality of the outputs. The risk here is that we attribute rigor to the process based on the computation, but forget that the computation depends on the rigor of the data.

Option 3 – The ChatGPT Paradigm. We can ignore the knowledge problem and use AI and human intervention to ensure that the outputs of our tools conform to some vaguely-defined standard of acceptability. There are many problems here. First, what is the standard of acceptability? If we are building tools to solve problems, as OpenAI claims in their advertising, then we risk overlooking solutions to difficult problems because of an arbitrary standard of acceptability, which may very well rest upon the belief that “we already know the right answer.” Second, if OpenAI continues to refuse to make their tool Open, then the users will not be able to assess the rigor of the output at all. Third, and perhaps most fundamentally, LLMs regress to the mean of the training dataset. As LLMs get more capable, the training data is becoming something like “a weighted representation of the entire internet.” The predictable result of LLMs becoming authoritative sources is that we continue to divide into our increasingly polarized camps and throw experts at each other. Even if we escape the polarization, we are still regressing to the mean of what we already think we know, which is a terrible way of inventing new solutions to difficult problems.

Which Option do you think holds the most promise for helping us solve our most difficult problems?

Artificial Intelligence has a knowledge problem. The latest wave of big AI tools, Large Language Models (LLMs) like ChatGPT/GPT4 and Llama, don’t encode actual knowledge about the world. They are only models that predict what words should come next, based on many billions of words of training data. LLMs usually sound like they are providing “correct” answers to our prompts, but this is merely a side-effect of the fact that LLMs are trained on data that contains enough correct information that it usually ends up sounding correct.

We are already seeing people getting frustrated with LLMs for “hallucinating” or “making things up” when the LLM generates text that is wrong or nonsensical. In Europe, government regulators are hard at work trying to mitigate potentially harmful uses and outputs of LLMs. OpenAI, the company behind ChatGPT, is working hard to make sure their product is safe, and at the same time, they are contributing to the problem by selling ChatGPT as a problem-solving tool.

A Very Brief History of AI

In the 1940s and 1950s, mathematicians and computer pioneers (most of whom were mathematicians) got excited about the potential of machines to think like humans. In the first wave of AI, this looked primarily like programming computers to play chess and checkers. These programs are search algorithms – they build a tree of possible moves and then search it for winning moves based on rules that assign values to different board positions. It soon became apparent that because of the number of possible positions in chess (about 10^44), we wouldn’t have enough computing power to search deeply enough to beat humans at chess for a very long time, and researchers moved on to other problems.

The second wave of AI in the 1960s-1980s consisted largely of heuristics engines and expert systems. Heuristics engines use a set of shortcuts or rules of thumb to quickly find a “good enough” solution to a complex problem that might not be solvable with traditional methods. Most anti-virus software uses a heuristics engine to identify suspicious behaviors. Expert systems encode knowledge about a specific domain of expertise in a rigorous way and use formal logic to solve problems within that domain. Examples include programs that can generate and verify mathematical proofs and programs for medical diagnosis. These systems were very highly developed by the early 1980s but fell out of favor for two main reasons: (1) they were extremely expensive to develop and maintain, and (2) they behaved unpredictably when faced with inputs that didn’t map directly to the underlying knowledge.

In the 1990s, computing power caught up to the chess problem enough to allow Deep Blue to beat Gary Kasparov, the top human player of the era. It was a landmark moment in AI, but looking back, probably said more about the weakness of human chess players than the strength of computers. We still can’t compute the full game tree of chess, or even exactly how many legal, reachable board positions there are.

By the early 2000s, large pattern recognition problems started to gain favor, as the combination of increasing computing power and improved neural network methods captured the imagination of researchers and programmers. Significant advances were made in techniques that would eventually prove fruitful in areas like voice recognition, handwriting recognition, facial and object recognition, and language translation. In 2012, a major neural network advance made image recognition practical, and neural networks have dominated the development of AI since then.

In the 2010s- present, the advent of compute-as-a-service enabled by cloud computing and the use of parallel processing GPUs to implement neural networks have lowered the barriers to AI development. LLMs were initially made possible by this new compute environment, and are gradually being moved onto smaller platforms. What we haven’t seen yet is a return to the knowledge-based approaches of the 70s and 80s.

A Call to Action

As we are poised to become dependent on LLMs to enhance our productivity, it will be increasingly urgent that we build a knowledge base under these tools. If we fail to do so, we risk granting authority to LLMs that are fundamentally untethered from actual knowledge of the world.

In future posts, we will examine the human knowledge problem, ponder what a return to knowledge-based AI might look like, and consider how incorporating a bottom-up knowledge representation might help us solve some of the biggest problems in scientific research today.

With all of the recent advancements in AI, especially the November release of ChatGPT and last week’s upgrade of the underlying GPT engine to version 4, there is a lot of ongoing discussion about AI Safety.

AI Safety is a term that has come to mean a lot of things, and as such, it requires some further elaboration so we know what we are actually talking about. According to Wikipedia (Mar 21, 2023): “AI safety is an interdisciplinary field concerned with preventing accidents, misuse, or other harmful consequences that could result from artificial intelligence (AI) systems. It encompasses machine ethics and AI alignment, which aim to make AI systems moral and beneficial, and AI safety encompasses technical problems including monitoring systems for risks and making them highly reliable. “

For this post, I’ll be having a conversation with ChatGPT 4 to provide an introduction to one concept that lives within AI Alignment. The concept is called “Provably Beneficial AI” and has been proposed, refined, and advocated primarily by Stuart Russell of the University of California at Berkeley, where he is the Director of the Center for Human-Compatible AI, a professor of Computer Science, and the Smith-Zadeh Chair in Engineering. He literally wrote the book on AI.

I did the interview twice, once with the GPT 3.5 engine and once with the GPT 4 engine. GPT 4 generally provided more detail in its responses, with a strong tendency to use numbered lists. GPT 3.5 was more conversational, but provided less detail. I used the same prompts for both, and since I did GPT 3.5 first, the GPT 4 prompts might feel a little clunky, but I wanted to be able to compare the two and get a feel for the differences. In the interview below, I am S: and ChatGPT-4 is GPT.

Enjoy!

---

S: Briefly explain Russell’s concept of “provably beneficial AI”

GPT: Russell’s concept of “provably beneficial AI” refers to the development of artificial intelligence systems that can be mathematically proven to act in ways that are beneficial to humans. This concept aims to ensure that AI systems consistently work towards the best interests of humanity, avoiding unintended negative consequences.

In order to achieve this, the AI must be designed with certain principles in mind, such as uncertainty about its objectives and deference to human preferences. By constructing AI systems that acknowledge their own uncertainty and learn from human input, we can work towards creating AI that remains aligned with human values and provides benefits without causing harm.

---

S: What is the strongest critique of Russell’s “provably beneficial AI?”

GPT: One of the strongest critiques of Russell’s “provably beneficial AI” concept is the challenge of defining and quantifying human values and preferences. Human values are complex, diverse, and often contradictory, making it difficult to create a universally accepted model that can guide AI systems in a provably beneficial way. Some specific concerns include:

1. Value alignment: Aligning AI systems with human values is challenging because values can vary significantly between individuals, cultures, and over time. It is unclear how an AI system can be designed to account for these variations and adapt to changing preferences without risking unintended consequences.
2. Incompleteness and ambiguity: Human preferences are often incomplete or ambiguous, which makes it hard for AI systems to make decisions that are guaranteed to be beneficial. A provably beneficial AI needs a way to handle such situations while remaining aligned with human values.
3. Preference manipulation: A learning AI system designed to be deferential to human preferences may inadvertently encourage humans to express preferences that are not in their long-term best interests, leading to potential manipulation of the AI system for short-term gains.
4. Moral dilemmas and ethical trade-offs: Real-world situations often involve moral dilemmas and ethical trade-offs that can’t be easily resolved. Designing a provably beneficial AI that can navigate these complex scenarios in a way that respects diverse perspectives and ethical considerations is a significant challenge.
5. Technical difficulties: The development of provably beneficial AI requires significant advancements in AI safety research, including robust methods for ensuring AI systems learn and follow human values, even as they become more capable. This is a difficult technical problem that has not yet been fully solved.

---

S: Didn’t Russell specify a big data approach to defining human values and objectives by essentially looking at what decisions humans made historically and inferring the underlying values and objectives?

GPT: Yes, Stuart Russell did suggest an approach to defining human values and objectives by examining the decisions humans have made historically, with the idea that AI systems can learn these values and preferences through a big data analysis. This approach is part of the broader concept of “inverse reinforcement learning” (IRL), where AI systems learn to infer the underlying objectives or values that drive human decision-making by observing human behavior.

However, despite the promise of this big data approach, it is not without challenges and limitations. Some concerns include:

1. Biases and inaccuracies: Historical data can be biased or incomplete, reflecting cultural, social, or political norms of a particular time period. These biases may not accurately represent the values and objectives of all humans, potentially leading to AI systems that reinforce existing inequalities or injustices.
2. Changing values: Human values and preferences evolve over time, and what might have been considered acceptable in the past might not be considered acceptable today. AI systems need to be adaptable and able to recognize these shifts to remain beneficial.
3. Overgeneralization: Inferring human values and objectives from historical data can lead to overgeneralization, where AI systems may not be able to distinguish between genuinely shared values and preferences that vary between individuals, cultures, or contexts.
4. Incompleteness and ambiguity: As mentioned earlier, human preferences can be incomplete or ambiguous, making it difficult to infer clear, universally applicable objectives from historical data.
5. Ethical dilemmas and trade-offs: Learning human values from historical data does not necessarily provide AI systems with a clear framework for resolving complex ethical dilemmas or making trade-offs between competing values.

Overall, while Russell’s big data approach to defining human values and objectives offers a potential path forward, it is not without significant challenges that need to be addressed in order to develop truly provably beneficial AI systems.

---

S: Isn’t there a more fundamental problem with this big data approach? Specifically, that there are many cases where humans were forces to choose a course of action from a set of possibilities, all of which were bad options and not aligned with our values. For example, it is hard to argue that Truman’s decision to use nuclear weapons against Japan was his “preference,” but neither was a mass invasion of the Japanese home islands. In fact, all really hard ethical decisions are between bad alternatives, so using a big data approach to categorize the decision that was made as a “preference” seems to be very problematic, if we want AI to avoid making the same types of mistakes as humans.

GPT: You raise a valid concern about the limitations of the big data approach to defining human values and objectives. Indeed, there are situations where humans face difficult ethical decisions and must choose between suboptimal options, none of which align perfectly with their values. In these cases, inferring preferences from historical data may not accurately represent human values and could lead to AI systems making similar mistakes or perpetuating undesirable outcomes.

This fundamental problem highlights the importance of considering additional factors when designing provably beneficial AI:

1. Context sensitivity: AI systems should be able to understand the broader context of human decision-making and recognize that certain decisions may have been made under duress or constrained by limited options, rather than as a true reflection of human preferences.
2. Ethical reasoning: AI systems should be equipped with robust ethical reasoning capabilities that allow them to navigate complex moral dilemmas, even when historical data may not provide clear guidance on the best course of action.
3. Human guidance and collaboration: Developing AI systems that work closely with humans and actively seek their input in situations with ambiguous or conflicting preferences can help ensure that the AI’s actions remain aligned with human values.
4. Ongoing learning and adaptation: AI systems should be able to learn from new information, societal shifts, and evolving human values, allowing them to update their understanding of human preferences and make better decisions over time.

In summary, while the big data approach to inferring human values and objectives has its limitations, it can still be a useful starting point for understanding human preferences. However, to develop truly provably beneficial AI, it is crucial to combine this approach with other techniques that address the inherent complexity and nuance of human decision-making and values.

---

S: Human guidance and collaboration seems to imply that moral values are relative to who we ask. Is Russell’s approach inherently one of moral relativism?

GPT: Stuart Russell’s approach to AI value alignment does not explicitly endorse moral relativism, but it does recognize the diversity and complexity of human values, preferences, and cultural norms. By emphasizing the importance of human guidance and collaboration, Russell’s approach aims to create AI systems that respect and adapt to this diversity, rather than adhering to a single, predefined set of moral absolutes.

In practice, this means that AI systems designed using Russell’s approach may exhibit a degree of moral relativism, as they attempt to align with the values of the humans they interact with or the societies they operate within. However, the goal is not to embrace moral relativism for its own sake, but rather to ensure that AI systems are beneficial and aligned with human values, which can be diverse and even contradictory.

It’s important to note that balancing between moral relativism and moral absolutism is a challenge when designing AI systems. To create provably beneficial AI, researchers and developers must grapple with the complexities of human values and ethics, and strike a balance that respects diverse perspectives while still upholding some level of universally accepted moral principles.

S: How might we decide what are universally accepted moral principles? Who should help decide that? It seems like the greatest philosophical minds of all time have been working on this problem since before Plato, and we haven’t reached any agreement yet.

GPT: You are correct that determining universally accepted moral principles is a complex and longstanding challenge, one that philosophers have been grappling with for centuries. While there may never be a complete consensus on universally accepted moral principles, there are some approaches to consider when determining which principles to incorporate into AI systems:

1. Cross-cultural consensus: Analyzing moral principles and values that are widely shared across different cultures and societies can help identify common ethical themes that may serve as a basis for universally accepted principles.
2. Interdisciplinary collaboration: Involving experts from various fields, including philosophy, ethics, sociology, psychology, and computer science, can help develop a more comprehensive understanding of moral principles and their applicability to AI systems.
3. Public involvement and deliberation: Engaging a diverse range of stakeholders, including the general public, in discussions and deliberations about moral principles and AI ethics can help ensure that multiple perspectives are considered and that the resulting principles are more widely accepted.
4. Iterative refinement: Recognizing that universally accepted moral principles may evolve over time, it is important to establish ongoing processes for revising and updating these principles as societal values and norms change.
5. International frameworks and guidelines: Building on existing international ethical frameworks, guidelines, and conventions related to human rights, environmental sustainability, and other global concerns can provide a foundation for developing universally accepted moral principles for AI systems.

Ultimately, determining universally accepted moral principles for AI is a collaborative and ongoing process that requires input from a diverse range of stakeholders, from philosophers and ethicists to policymakers and the general public. While achieving perfect consensus may be impossible, striving for a shared understanding of core moral principles can help guide the development of AI systems that are more broadly beneficial and aligned with human values.

---

Overall, not a bad summary of the goals and some of the potential pitfalls of Russell’s approach.

The latest version of ChatGPT, (based on GPT-4) is about a week old, and already something funny is happening.

A Good Start

People are finding some pretty cool ways to use ChatGPT to help with their normal work.

One user gave ChatGPT a description of his company’s mission and a rough job description; GPT wrote a formal job description, and it was posted to a job board minutes later.

Another user had GPT interview him and write content based on the interview.

One friend had it draft a wealth management plan.

Another had it build the financial policy manual for his company, including lists of reports and deliverables, deadlines, and cost estimates.

People have figured out that they can get GPT to change its writing style by providing examples, and that they can even have it to write legal documents.

Several people have had it draft marketing emails and web site copy to sell their products.

For the first few days, it seemed the sky was the limit with this amazing new tool.

And Some Frustration

In the last couple of days, I’ve increasingly been hearing about problems. Users are getting frustrated with the tool misbehaving and doing unexpected things.

In one case, for a user who was having it create budgets and financial plans, it started changing hourly rates and budget lines on its own. When confronted, it apologized profusely for the mistake.

Other users have been frustrated that it seems to change its mind if you ask it the same question more than once.

One guy was puzzled when he asked it to cut and paste some text into a new response, but it wrote a new paragraph on the same topic instead.

And of course, some users have tricked it into doing nonsensical things – like writing a set of instructions for patching a hole in the wall of a house made entirely of cheese.

What’s Happening?

What’s happening here? Is GPT misbehaving? Making mistakes? Deliberately undermining its users?

Without getting into the technical explanation for each of the types of “errors” above (and there are relatively simple explanations based on how GPT works for each of them), I think there is a meta-error happening. And we, the users, are the ones making it.

We have been so impressed with our initial experiences that we are treating GPT as something like an experienced Executive Assistant and expecting that level of performance from it. It is performing well enough that when it misses the mark, we are getting frustrated with it in the same way we would if it were human.

The problem is, GPT isn’t built to be an Executive Assistant. It is built to be a language model. What that means is that it does one thing very well – it predicts what word should come next given the previous couple thousand words, based on a corpus of about 500 billion words of written language (for more on how it does that, see my previous blog post here.).

It’s worth repeating for emphasis: the only thing GPT does is generate text that looks convincing by modeling a statistical process to predict what word should come next.

What it doesn’t do is… everything else. It doesn’t build plans. It doesn’t apply logic. It doesn’t understand meaning. It doesn’t understand consequences. It doesn’t do rigorous research. It doesn’t formulate new ideas.

But GPT’s text generation model is good enough that it looks like it does those things. The training data includes many, many examples of those types of writing, so it produces output that is convincing and might even be reasonably accurate. But any accuracy or inaccuracy GPT provides is a side effect of what is in the training data. 

So, if you ask GPT for a financial plan, it generates text that looks like a financial plan. If you ask it to explain the logic of Zeno’s Paradox, it generates text that looks like an essay on that topic. If you ask it to explain what it means to be alive, it generates text that looks like an answer to that question. If you ask it to provide references on a certain topic, it generates text that looks like a list of references (some of them are decent references, and some of them are rubbish). 

What we are seeing is a disconnect between what we expect GPT to do and what it is actually capable of doing. This disconnect seems to be present in large part because GPT sounds so human that we expect it to behave like a human. GPT may be a victim of its own success. But it turns out that humans do a lot more than generate streams of words that sound human. We are expecting way too much from GPT. 

It is important to understand what GPT does, and what it doesn’t do. The reactions of some pretty smart people experimenting with the tool suggest that together, we (the users and OpenAI) haven’t done well at communicating the limitations of GPT. I hope we become more educated users soon – before we start treating GPT as if it were an authority on any important topic.

Perhaps GPT has passed the Turing Test, at least well enough that we are beginning to treat it as if it were an underperforming co-worker. It will be interesting to see what happens over the next few months.

What is ChatGPT?

ChatGPT has been in the news a lot recently, and we’re going to hear a lot more about it in the near future. GPT stands for Generative Pre-trained Transformer, meaning that it uses a transformer network (a type of Deep Learning neural network) trained on a very large corpus of text to produce human-like text based on a prompt. GPT is a type of Large Language Model (LLM) and there are several other competing models in various states of development. One way to think of this is that GPT is a neural network implementation of a predictive text algorithm.

ChatGPT adds an interactive use interface to the front end of the GPT-3 model, allowing users to quickly and easily interact with the tool, and even allowing users to fine-tune how it responds based on previous interactions and examples of different writing styles. Give ChatGPT a prompt, and it will build an answer, an essay, or even a joke on a given topic by “analyzing” what has been written before in its training data and any data you have provided, and giving you a set of predictive text on the topic.

This post will break down what a predictive text algorithm is generally, what neural networks are and how they work, why ChatGPT is built using a neural network, and some background on ChatGPT’s development and how the underlying models were trained. The next posts will begin to look at some of the problems with LLMs, including GPT, and with ChatGPT specifically.

Predictive Text Algorithms

One way to conceptualize a predictive text algorithm is to imagine looking at every book that has ever been published (GPT also uses a much larger body of internet text, which presents additional problems that I’ll look at in a future post) and building a table of probabilities for every common English word (there are about 40,000 or them) that captures the odds that of what the next word will be. We get a table of 40,000 x 40,000 words with each cell filled with a probability representing how often in the training text the second word follows the first. As you might imagine, many of the entries will be zeroes.

With this table in hand, we can “prompt” the table by giving it a keyword, and then hop through the table by picking the entry with the highest probability of being the next word. If we ask our model to generate text based on the prompt “cat” we might get something like “cat through shipping variety is made the aid emergency can the.” Hardly a useful sentence.

In order to get an output that makes more sense (seems more like something a human would write), we might extend our table to include the probabilities for the most likely next word given the previous two words. Our output in this case would be a little better, but now our table has 64 Trillion (40,000^3 entries), rather than just 1.6 Billion (40,000^2). If we want to continue to improve the output of our algorithm, we need to continue it for greater depth of the-words-that-came-before. The problem with this approach is that it becomes too computationally expensive before it starts to produce good quality text. (For example, if we want to consider the previous 13 words, still not enough to produce a coherent essay-length output, we need a table with as one entry for each atom of hydrogen in 100 stars the size of the sun.) This problem is complicated by the fact that we also don’t have enough written text to fill out much of the table at that depth.

Because of these limitations, we need to find a better approach. We need an approach that is both more computationally efficient, and one that can achieve reasonable results using the training data we do have. GPT (and all Large Language Models, LLMs) uses a neural network to estimate the probabilities that we don’t have the computational capacity and necessary volume of training data to build traditionally.

Neural Networks

The idea of Neural Networks dates back to 1943, when Warren McCulloch and Walter Pitts formulated a simple model of networks of neurons and attempted to analyze the model mathematically. The first discussion of using neural networks as a possible approach to artificial general intelligence (AGI, or teaching machines to think “like” humans) was in the 1948 paper Intelligent Machinery by Alan Turing.  In 1949, Donald Hebb argued that such networks would be capable of learning. Computer simulations of neural networks began in the 1950s, but it soon became apparent that we didn’t have enough computing power to model enough neurons at a fast enough speed to do anything interesting. Neural Networks were put on the back burner of computing and AI for a few decades. They were studied rigorously again in the 1980s and 90s, but researchers decided that the only interesting things they could do were things that could be done more simply with probabilistic models. Neural networks finally broke out of the background between 2009 and 2012 when Swiss researchers successfully applied recurrent neural networks and deep feed-forward neural networks to win a series of international pattern recognition and machine learning competitions.

Neural networks work by simulating the type of computation that biological neurons perform. You can think of a simulated neuron as having a set of inputs (analogous to the connections (dendrites) coming to a biological neuron from other neurons); a function that converts those inputs to an output value (analogous to the body of the neuron); and a set of output connections that carries that output to some number of other neurons. Each output connection we can think of as a signal with a particular strength, or weight. In a neural network, these units are usually organized into layers, which are frequently (but not always) two-dimensional arrays of interconnected neurons. A number of layers are connected in ways that generate the type of output we are seeking. There are many different types of connection topographies within layers, many different types of layer architectures, and many different ways of “stacking” the layers. ChatGPT, which is built on GPT 3, uses something like 10 million neurons with about 175 billion connections arranged in about 400 layers.

Training the neural network is done by feeding it lots of data, and providing it feedback in the form of a difference signal from the desired output (at the final output layer, or sometimes at a layer close to the final layer). The network then adjusts the weights across the entire network in order to try to reduce the distance between the actual output and the desired output. As it turns out, it takes about as many training data runs as there are connections in the network to “fully” train the network. In the case of GPT-3, that means about 175 billion words of training data.

GPT-3 was trained on about 500 billion words of text. (Technically, tokens, which differ from words a little bit, but not enough to matter for our purposes.) The sources of that data are described in the table below (from Wikipedia):

Common Crawl is a dataset produced by the Common Crawl non-profit organization that contains over 3 billion we pages (as of Oct 2022). WebText2 is a corpus containing all Reddit submissions from 2005 through April 2022. Books1 and Books2 contain digitized published books. You may notice that the data sets are weighted withing the model in a way that isn’t the same as their token count. *Note: for a more detailed account of exactly how ChatGPT works, and an excellent overview of the underlying technologies, see this post by Stephen Wolfram: What Is ChatGPT Doing … and Why Does It Work?—Stephen Wolfram Writings

One of the things that neural networks are good at is generalizing patterns from very large, but incomplete data sets, more quickly and sometimes more usefully than a brute-force algorithm can accomplish. And that’s what ChatGPT does – it is a neural network that generalizes patterns from a large corpus of text and iteratively predicts what word should come next given the previous words (including the text it generated). By doing so, it has gotten to a level of capability that looks like useful human-written text in many ways, though it still has some important and significant limitations.

In the next post, I’ll look at some of the problems with GPTs and LLMs generally, and some of the specific issues with OpenAI’s approach to ChatGPT (and the underlying GPT-3 and -4 models).

Update:

As I was editing this post, OpenAI has released GPT-4 to the public. OpenAI has chosen not to reveal the specifics of the neural network implementation of GPT-4, but has promised that it will be “safer and more useful” than previous versions. Rumors, which have been denied by OpenAI’s founder, suggest that the model is on the order of 1000x bigger than GPT-3.