How LLMs Learn and Respond — The Training and Inference Process of Large Language Models

LLM training is the process of adjusting billions of parameters to better predict the next token by reading vast amounts of text, and inference is the process of generating tokens one by one that follow the input using fixed parameters.

In the previous article, we explored the relationship between artificial intelligence, machine learning, deep learning, and generative AI from a historical and classification perspective. In that classification, Large Language Models (LLMs) were a type of generative AI that generates text based on deep learning.

This time, let’s delve one step further into the classification.

When you enter a question into services like ChatGPT or Claude, a natural answer appears within seconds. From the screen, it looks as if the model reads the question, searches stored knowledge, thinks like a human, and then writes a sentence.

However, what happens internally is slightly different from this description. During the training phase, LLMs compress statistical patterns of language into parameters, and during the inference phase, they use those parameters and the current context to calculate the probability of the next token. Once a token is selected, it is appended to the input, and the same calculation is repeated to form a sentence.

This article covers the following topics:

The process by which raw text becomes training data
What tokens and parameters are, and what the model learns
How forward propagation, loss calculation, and backpropagation occur in pre-training
Why supervised fine-tuning and preference alignment are necessary
How prefill, KV cache, and decoding operate in inference
The principles by which temperature, Top-k, and Top-p change responses
Differences between training, fine-tuning, RAG, and inference

The Two Stages to Distinguish First: Training and Inference

The most important starting point for understanding LLMs is to separate training and inference.

Training is the process of changing the model’s parameters. It calculates the difference between the model’s predicted next token and the actual next token, and gradually adjusts the weights in a direction that reduces that error. By repeating this process across countless sentences and batches, the model learns how words are used, sentence structures, relationships between concepts, and various problem-solving patterns.

Inference is the process of using a trained model. Even when a user’s prompt is entered, the model’s parameters do not change during typical inference. It merely calculates the next token using already fixed parameters and the current conversational context.

The differences between the two processes are summarized in the table below:


Category	Training	Inference
Purpose	Forms capabilities by reducing next token prediction error	Generates output tokens matching the input
Parameters	Continuously updated	Generally fixed
Main Input	Large-scale training data and ground truth tokens	User prompt and conversational context
Calculation Direction	Forward and backward propagation	Mainly forward propagation
Parallel Processing	Many tokens and batches processed in parallel	Prefill is parallel, generation is sequential per token
Cost Characteristics	Massive computation and time required for a single training run	Repeated per request, so operational scale determines cost
Result	Checkpoint or completed model	Generated output such as text or code

Training is akin to creating a textbook and undergoing long-term training, while inference is like taking an exam and using what has already been learned to write answers. However, there’s a caveat in this analogy. Humans can accumulate experience even while taking exams, but a typical LLM API call does not automatically train the model’s base parameters once a request is finished.

Step 1: Collecting Text Alone Is Not Enough for Training

The first step in LLM training is data preparation.

While various texts such as web documents, books, academic papers, code, Q&A data, and conversational data can be collected, feeding raw collected text directly into training can lead to quality and safety issues.

The same document might be duplicated across multiple sites, and it could be mixed with automatically generated spam, meaningless strings, personal information, copyrighted material, malicious code, and harmful content. If data from a specific language, region, or viewpoint is excessively abundant, that bias will also be reflected in the model’s behavior.

Therefore, the following tasks are crucial in a real data pipeline:

Removal of duplicate or similar documents
Document quality assessment and spam filtering
Detection of personal and sensitive information
Adjustment of language and domain ratios
Removal of excessively short or corrupted documents
Recording data sources, usage rights, and lineage
Separation of data for training, validation, and evaluation

Even with the same model architecture, different training data compositions and refinement methods can lead to significantly different results. In LLM development, data work is not just simple preprocessing but a core design process that determines the model’s knowledge scope, style, bias, and safety.

Step 2: Breaking Sentences into Tokens

Neural networks cannot directly compute sentences themselves. Tokenization is first required to convert text into numbers.

A token is the basic unit that a model processes text. A single token does not always mean a single word or a single character. Frequently appearing words can become one token, while rare words might be split into multiple parts. Korean particles and endings, English word prefixes and suffixes, spaces, and punctuation marks are also broken down into different units according to the tokenizer’s rules.

For example, the following sentence can be divided into multiple tokens depending on the model and tokenizer:

생성형 AI는 문장을 만듭니다.

Conceptually, it could take a form similar to this:

[생성, 형, AI, 는, 문장, 을, 만듭니다, .]

The actual segmentation results vary by model. What’s important is that each token is converted into an integer ID from a vocabulary.

[1842, 731, 9274, 118, 5321, 84, 22017, 13]

The model converts these IDs into embedding vectors. An embedding is a representation of a token as a numerical array of hundreds or thousands of dimensions. As training progresses, tokens used in similar contexts will have related directional representations, and the model calculates the relationships between words and concepts in this vector space.

Tokenization also affects cost. Even sentences with the same meaning can have different token counts depending on the language and expression style, and most LLM services calculate costs and usage based on input and output tokens.

Step 3: Learning Language by Predicting the Next Token

The pre-training objective of many autoregressive LLMs is surprisingly simple:

Given the preceding tokens, predict the next token.

Let’s assume the following sentence is in the training data:

인공지능은 데이터를 통해 패턴을 학습한다.

The model can learn several prediction problems at once, such as:

인공지능은 → 데이터를
인공지능은 데이터를 → 통해
인공지능은 데이터를 통해 → 패턴을
인공지능은 데이터를 통해 패턴을 → 학습한다

Even without humans manually labeling each sentence with ground truth, the token one position ahead in the original text can be used as the correct answer. This is called self-supervised learning. It is a crucial reason why vast amounts of internet-scale text could be utilized as training data.

However, the model doesn’t simply memorize the frequency of preceding and succeeding words. To accurately predict the next token in various contexts, it must incorporate grammar, style, factual relationships, document structure, code rules, and conceptual associations into its internal representations.

For example, predicting the next token after “The capital of South Korea is” requires incorporating geographical information, and predicting the code after the start of a function requires tracking programming syntax and the roles of preceding variables. This is why multiple capabilities emerge from the simple objective of predicting the next token.

Step 4: Transformer Calculates Context

Once token IDs are converted into embeddings, they pass through Transformer layers.

The core of the Transformer, proposed in 2017, is attention. Attention calculates how much other tokens within the context should be referenced when processing the current token.

Consider the following sentence:

민수는 노트북을 가방에 넣었다. 그것은 새 제품이었다.

To determine what “it” refers to, one must look at the relationship between “laptop” and “bag” mentioned earlier. Self-attention creates Query, Key, and Value vectors for each token and uses the similarity between Query and Key to calculate how much of which token’s Value to mix in.

Conceptually, it’s like this:

Attention(Q, K, V) = softmax(QKᵀ / √d) V

Multiple attention heads can focus on different relationships. Some heads might capture grammatical relationships, while others might capture references or thematic relationships between distant words.

Each Transformer layer includes not only attention but also feed-forward networks, residual connections, and normalization. By passing through these layers multiple times, token representations transform from simple word vectors to representations that reflect the current context.

The final output layer generates logits, which are scores for all tokens in the vocabulary. Applying softmax yields the probability distribution of the next token.

서울 0.72
부산 0.08
대한민국 0.05
도시 0.03
기타 토큰 0.12

During the training phase, the model is modified so that this distribution assigns a high probability to the actual correct token.

Step 5: Calculate Error and Update Parameters

The difference between the model’s predicted probability and the actual next token is called loss. Cross-entropy loss is typically used for next token prediction.

If the correct answer is “Seoul” and the model assigned a probability of 0.72 to “Seoul,” the loss is relatively small. If it assigned only 0.01, the loss is large.

After calculating the loss, backpropagation begins. Using the chain rule of differentiation, the gradient is calculated to determine how much the output error was influenced by each layer and parameter.

Then, the optimizer uses the gradient to gradually update the parameters.

새 파라미터 = 기존 파라미터 - 학습률 × 기울기

Actual training involves techniques such as Adam-family optimizers, learning rate schedules, weight decay, and gradient clipping. While a single equation is simple, repeating this calculation across billions of parameters and large batches requires many GPUs or AI accelerators and distributed training technologies.

The training process roughly repeats in the following loop:

데이터 배치 준비
→ 순전파
→ 다음 토큰 확률 계산
→ 손실 계산
→ 역전파
→ 파라미터 업데이트
→ 다음 배치

Periodically, the model’s parameters and optimizer state are saved as checkpoints. This is for recovery in case of training failure, evaluating the model at specific points, and using it as a starting point for subsequent fine-tuning.

Pre-training Alone Is Not Enough to Become a Good Conversational AI

A model that has completed pre-training knows many language and knowledge patterns, but it is not necessarily a conversational tool that reliably follows user instructions.

This is because the goal of accurately predicting the next token in web documents is not the same as the goal of “answer questions accurately and safely.” A pre-trained model might continue with another question after a question, generate unnecessarily long documents, or respond directly to harmful requests.

Therefore, post-training follows pre-training.

Supervised Fine-tuning

In supervised fine-tuning (SFT), instruction-response pairs are used.

사용자: 다음 문장을 세 줄로 요약해줘.
도우미: <좋은 요약 답변>

By training on this data, the model learns to distinguish between user and assistant roles and generate answers that fit the instruction format, rather than simply continuing internet sentences.

Aligning with Human Preferences

Many questions do not have a single correct answer. Even if two answers are factually correct, one might be clearer and more helpful.

In the RLHF method, humans rank multiple responses by preference, a reward model is trained to predict these preferences, and then reinforcement learning can be applied to the language model to receive higher rewards.

Methods like DPO directly optimize the policy model using pairs of preferred and dispreferred responses. Although the implementation differs, the goal is similar: to make the model better adhere to human-preferred utility, honesty, safety, and style.

While pre-training builds the model’s foundational capabilities and broad knowledge representation, post-training adjusts how those capabilities are expressed in a conversational product.

Inference Step 1: Tokenize the Prompt and Read It All at Once

Let’s assume a user sends the following question to a trained model:

LLM의 학습과 추론 차이를 두 문장으로 설명해줘.

The server tokenizes the prompt and sends the token IDs to the model. If there are system prompts, previous conversations, search results, and tool execution results, this information is also included in the context in a defined format.

The initial processing of the entire input prompt by the model is called prefill. During prefill, input tokens pass through Transformer layers, and Key and Value vectors are calculated for each position.

These results are stored in the KV cache. This is to avoid recalculating the Key and Value for the entire previous prompt every time a new token is generated.

Prefill can process input tokens relatively in parallel. Therefore, inputting a very long document might increase the time before the first token appears and increase memory usage, but it does not necessarily generate all input tokens sequentially, one character at a time.

Inference Step 2: Generate Next Tokens One by One

Once prefill is complete, decoding begins.

The model calculates the logits and probability distribution of the next token based on the current context. The decoder selects one token from that distribution.

For example, the first output token candidates might be:

학습은 0.58
LLM의 0.18
두 0.07
모델 0.05
기타 0.12

If “training is” is selected, this token is appended to the context, and the next token is calculated again.

입력 프롬프트
→ “학습은” 생성
→ “학습은 모델의” 생성
→ “학습은 모델의 파라미터를” 생성
→ 계속 반복

This process is called autoregressive generation. Since each new token depends on all previously generated tokens, output generation is inherently sequential.

The KV cache reuses the attention Key and Value of previous tokens to reduce redundant calculations. However, as the output lengthens, the cache grows, and the context that needs to be referenced at each step also increases. This is why long contexts and long answers increase latency and memory usage.

The model finishes generation when it produces an end-of-sequence token, reaches a set maximum number of tokens, or encounters a stop string specified by the service.

Why Responses Differ for the Same Question

The model doesn’t just output a single next token; it calculates a probability distribution over the entire vocabulary. Which token is selected depends on the decoding strategy.

Greedy Selection

At each step, the token with the highest probability is selected. Results are relatively consistent but can be repetitive or overly predictable.

Temperature

Temperature controls the sharpness of the probability distribution.

Low temperature: Focuses more on high-probability tokens, leading to stable results.
High temperature: Lower-probability candidates are more likely to be selected, leading to diverse results.

A higher temperature does not mean the model becomes smarter. It’s a setting that changes the balance between diversity and predictability.

Top-k

Only the top k tokens with the highest probabilities are kept as candidates; the rest are excluded.

Top-p

Only candidates whose cumulative probability reaches p when summed in descending order of probability are used. The number of candidates changes dynamically based on the context.

For accurate code or structured responses, low temperature and restrictive decoding might be advantageous, while for idea generation or style variation, more diverse sampling can be helpful.

Does the Model Learn Anew During Inference?

During typical inference requests, the model’s parameters do not change.

If a user says “My name is Minsu” during a conversation, the model can refer to that information within the current context. However, once the conversation disappears from the context, the base model has not permanently learned about a user named Minsu.

A service might store conversation history in a separate database, re-insert it into the prompt using a memory function, or review it later as training data. But this is different from parameters being immediately updated during a single inference process.

The same principle applies when the model generates multi-step explanations during inference. The model can use more output tokens to break down intermediate problems, but its base parameters are not newly learned in that process.

Where Do Fine-tuning and RAG Fit In?

In LLM applications, training, fine-tuning, and RAG are often confused.

Fine-tuning

Fine-tuning is the process of further training an existing model with specific data to change its parameters. It can involve updating all parameters or just small additional parameters, like LoRA.

It is useful for reflecting specific styles, output formats, and recurring task patterns in the model’s behavior. However, using it as a substitute for a database to frequently update the latest facts can be inefficient in terms of management and accuracy.

RAG

Retrieval-Augmented Generation (RAG) is a method of searching for documents related to a question, inserting them into the prompt, and then performing inference. It does not change the model’s base parameters.

사용자 질문
→ 관련 문서 검색
→ 질문과 문서를 프롬프트로 구성
→ LLM 추론
→ 답변 생성

It is useful for providing knowledge that changes frequently, such as internal documents, product manuals, and policies, where source and recency are important.

Prompt Engineering

Prompt engineering is also the process of designing the input during the inference phase. It clarifies roles, instructions, examples, and output formats to help the model use its existing capabilities in a desired way. It does not directly change the model’s parameters.


Method	Parameter Change	External Document Use	Best Suited Purpose
Pre-training	Yes	Used as training data	Forms foundational language capabilities and broad representations
Fine-tuning	Yes	Used as additional training data	Adjusts style, format, and specific behavioral patterns
RAG	No	Searched and used during inference	Provides up-to-date facts, internal knowledge, and evidence
Prompt Engineering	No	Can be directly included in input	Controls instructions and output format

—

Why Training Is Expensive and Inference Becomes an Operational Issue

Training requires not only forward propagation but also backpropagation and optimizer state calculations. Parameters, gradients, and optimizer states must be kept in memory, and large batches of data must be processed across multiple devices.

Therefore, a single large-scale pre-training run requires immense computational power, electricity, high-speed networks, and storage devices.

Inference does not involve backpropagation, so for a single request, it is lighter than training. However, in a service, countless user requests continuously arrive. Input length, output length, number of concurrent users, and response time targets determine the overall operational cost.

The following metrics are important in an inference system:

Time taken for the first token to appear
Number of tokens generated per second thereafter
Number of requests that can be processed simultaneously
GPU memory and KV cache usage per request
Cost per input and output token
Level of quantization that can be applied while maintaining response quality

Using smaller models, quantizing parameters to lower precision, batching requests dynamically, and efficiently managing the KV cache can reduce inference costs. Conversely, indiscriminately using the largest model and longest context might lead to greater latency and cost increases than quality benefits.

Why Does Next Token Prediction Alone Produce Natural-Sounding but Incorrect Answers?

An LLM is not a database that directly judges factual accuracy but a model that calculates the probability of the next token based on context.

Even when encountering questions not present or rare in training data, conflicting information, recent events, or ambiguous prompts, the model might continue generating plausible tokens rather than stopping. The fact that a sentence sounds natural is not the same as the content being true.

To reduce hallucinations, system design is more important than simply increasing model size.

Provide up-to-date or internal materials via RAG
Require sources and evidence in answers
Delegate calculations and lookups to verifiable tools
Verify important results with rules and separate validation models
Apply a policy of responding “I don’t know” when appropriate
Repeatedly measure accuracy and failure types with evaluation data

Understanding the principles of LLM training helps us understand why the expectation that “the model has learned a lot, so it will always tell the truth” is dangerous.

Connecting the Entire Process at Once

The lifecycle of an LLM can be simplified as follows:

원시 데이터 수집
→ 정제·중복 제거·품질 관리
→ 토큰화
→ 사전 학습
→ 평가와 체크포인트 선택
→ 지도 미세 조정
→ 선호도 정렬과 안전성 조정
→ 배포
→ 프롬프트 입력
→ 프리필과 KV 캐시 생성
→ 토큰별 디코딩
→ 결과 검증과 사용자 응답

In training, the error with the ground truth token is backpropagated to change parameters. In inference, with parameters fixed, the next token is selected using the prompt and all tokens generated so far.

Understanding this difference makes it easier to answer the following questions:

Why doesn’t a single conversation permanently train the model?
Why do long prompts increase the initial response time?
Why are long outputs generated slowly, token by token?
Why does changing the temperature alter the answer to the same question?
Why might RAG be more suitable than fine-tuning for up-to-date knowledge?
Why does a natural-sounding sentence not necessarily imply factual accuracy?

An LLM is neither a search engine that stores knowledge sentence by sentence and retrieves it, nor a system that fixes its own parameters with each request.

During the training phase, it compresses patterns from large-scale data into parameters, and during the inference phase, it repeatedly generates the next token using those parameters and the current context.

Accurately understanding this simple principle is the starting point for not exaggerating the capabilities and limitations of LLMs, and for correctly deploying training, fine-tuning, RAG, and prompt design where needed.