Generative AI in a Nutshell

So how does it actually work?

An LLM (large language model) is an Artificial Neural Network. Basically a bunch of numbers, or parameters, connected to each other, similar to how our brain is a bunch of neurons, or brain cells, connected to each other.

Internally, Neural Networks only deal with numbers. You send in numbers, and depending on how the parameters are set, other numbers come out. But any kinds of content, such as text or images, can be represented as numbers. So Neural Networks can really be used for any kinds of media. For example, self-driving cars use neural networks to process visual input from cameras and other sensors, outputting control signals like “adjust steering 5 degrees right” or “apply 20% braking force”.

LLMs are Neural Networks that are optimized for understanding and generating text. You may have heard the term “token” and “token limits”. A token is a small chunk of text, typically a word or part of word. Internally, LLMs read and generate tokens. What you see is words and sentences. Technically, they work with tokens rather than words, but I’ll just call it “words” in this chapter to make it easier to understand.

Let’s say I write “Dogs are”. When I send that to an LLM, that gets converted to numbers, processed by the neural network, and then the resulting numbers are converted back into text. In this case the output is ”animals”. So we get “Dogs are animals”.

How did it decide on the word “animals?” It calculates probabilities for all possible next words based on input you gave it plus the data it was trained on (see next chapter for more info on training). It then selects a word, balancing probability with some randomness for creative diversity.

So an LLM is basically a “guess the next word” machine.

There’s actually more going on than just statistical text prediction - there is some level of understanding happening. Remember the “Attention is All You Need” paper mentioned in the previous chapter? One of its key concepts there was the attention mechanism.

Think about how you read a sentence - your brain automatically focuses on the important words and kind of glazes over the less important ones (like “the”). That’s basically what attention does - it helps the AI figure out which parts of the text really matter. For example, when processing the phrase “bat flew at night,” the attention mechanism helps the model focus more on the word “flew” to understand that “bat” refers to the animal rather than the sports equipment. Understanding the relationships and dependencies between words is a key to understanding human language.

The interesting part is if we take the output and combine it with the input and send it through the neural network again, it will continue adding new words. And when we loop that, we get sentences and paragraphs. That’s what your AI client is doing when it responds to your prompts.