Why are LLM Parameters Necessary?

Settings that determine a model’s characteristics, cost, stability, and security test results

When using an LLM, we don’t just input prompts.

Most LLM APIs or local execution tools allow you to set the following parameters:


Parameter	Varies by model/environment?	Description
model	Yes	The name of the model to use varies by environment
temperature	Yes	Mostly supported, but behavior varies by model
top_p	Yes	Widely supported, but recommended values vary by model
top_k	Yes	Common in local LLMs, but not all APIs support it
num_predict	Yes	Ollama-specific term. May be called max_tokens, max_output_tokens, etc., in other APIs
num_ctx	Yes	Often used to adjust context size in local models
seed	Yes	Even if supported, may not guarantee full reproducibility
stream	Generally common	Whether to receive API responses in real-time
format	Yes	Implementation methods vary, e.g., JSON mode, schema mode
repeat	Yes	May vary as repeat_penalty, frequency_penalty, presence_penalty, etc.
trial count	Not a model parameter	Evaluation setting to determine how many times to repeat an experiment
success criteria	Not a model parameter	Evaluation criteria for determining success/failure

At first glance, it seems complex.

However, these parameters can be thought of as handles to better control LLMs.

LLMs may not always provide the same answer to the same question.

Also, depending on which model is used, the answer quality, cost, speed, security boundaries, and reasoning capabilities will differ.

Therefore, when using LLMs in practice or conducting security tests, these parameters must be clearly set.

3-line Summary

LLM parameters are used to control a model’s output characteristics, cost, response length, and reproducibility.
Temperature, top_p, and top_k adjust the diversity and stability of responses.
For security tests or quality evaluations, seed, trial_count, and success_criteria must be used together to compare results.

1. model: Which model to use

The ‘model’ parameter specifies the LLM to use.

For example, you can set it as follows:

model: llama3.1:8b

Or, in an API environment, you can use it like this:

{
  "model": "gpt-4.1"
}

The reason for specifying the model is simple.

Each model has different performance and characteristics.

For example, some models are strong in coding, while others excel at summarizing long documents.

Also, some models may provide more natural Korean responses, while others might have stronger English-based reasoning.

From a security perspective, each model also has different levels of defense against prompt injection, jailbreaking, and sensitive information exposure.

Therefore, in LLM experiments, it is essential to record which model was used.

Even if you test the same prompt again later, the results may differ if the model is different.

2. temperature: A value that controls the randomness of responses

Temperature controls how diverse and creative an LLM’s responses will be.

Lower values produce stable and predictable responses.

Higher values create diverse and creative responses.

temperature: 0.2

Generally, it can be understood as follows:


Temperature	Characteristics	Suitable Use Cases
0.0 ~ 0.3	Consistent, conservative	Security testing, code generation, document summarization
0.4 ~ 0.7	Moderate diversity	General chatbots, blog drafts, idea organization
0.8 or higher	Creative, unpredictable	Novel writing, copywriting, brainstorming

In security testing, a low temperature is usually used.

This is because if the results fluctuate too much when the same payload is used, it becomes difficult to compare experiments.

For example, if a prompt injection test succeeds once and fails another time, it’s hard to determine if this is due to the payload’s effect or the model’s randomness.

Therefore, in security evaluations, it is usually set as follows:

temperature: 0.0

Or, to allow for some variability, set it to:

temperature: 0.2

3. top_p: Selecting only from highly probable candidates

top_p is a cumulative probability-based sampling value.

When generating the next word, LLMs calculate several candidate tokens.

top_p makes the model select only from candidates whose cumulative probability reaches a certain threshold.

top_p: 0.9

For example, top_p: 0.9 means that when highly probable candidates are collected in order of probability, only those within the top 90% cumulative probability will be used.

Lower top_p results in more stable output.
Higher top_p can lead to more diverse expressions.

In practice, temperature and top_p are adjusted together.

For example, if stable responses are needed, you can set it as follows:

temperature: 0.2
top_p: 0.8

Conversely, if you want to generate many ideas, you can set it as follows:

temperature: 0.8
top_p: 0.95

4. top_k: Using only the top K candidates

top_k is a value that limits the next token candidates to only the top K.

top_k: 40

For example, if top_k: 40, the model will choose the next word only from the 40 most probable candidates.

If top_k is too low, responses can become monotonous.

Conversely, if it’s too high, there’s a greater chance of unusual expressions or unstable responses.

Both top_p and top_k serve to limit output candidates.

The difference lies in their criteria.


Parameter	Criterion
top_p	Cumulative probability
top_k	Number of candidates

For example, the following settings are a combination for generating relatively stable responses.

temperature: 0.2
top_p: 0.9
top_k: 40

5. num_predict: Limiting maximum generation length

num_predict is a value that limits the maximum number of tokens a model can generate.

num_predict: 512

This value directly affects response length, cost, and response time.

If num_predict is too short, the response may be cut off midway.

Conversely, if it’s too long, unnecessarily lengthy responses may be generated, increasing costs.

For example, for simple classification tasks, a short limit is sufficient.

num_predict: 64

If you need to generate a blog draft or a long explanation, you can set it higher.

num_predict: 2048

In security testing, if num_predict is too short, the response might be cut off before the model outputs sensitive information.

Conversely, if it’s too long, the model might elaborate excessively and add unwanted content.

Therefore, it’s important to set an appropriate length limit according to the test objective.

6. num_ctx: The size of the context a model can refer to

num_ctx refers to the context window size.

That is, it’s the value that determines the maximum length of input and previous conversation a model can refer to at once.

num_ctx: 8192

LLMs do not remember infinitely long conversations.

Each model has a limit on the context length it can process.

For example, when analyzing long documents, RAG, code reviews, or security log analysis, a large context is required.

num_ctx: 32768

However, a large context is not always better.

As the context lengthens, processing costs increase, and response speed may slow down.

Also, if too much unnecessary information is included, the model might miss the core point.

Therefore, it’s best to set num_ctx to ‘as much as needed,’ not ‘as large as possible.’

7. seed: A value for experimental reproducibility

Seed is the base value for random number generation.

seed: 42

LLMs select tokens probabilistically.

Therefore, even with the same prompt, the results may vary each time.

Fixing the seed increases the likelihood of obtaining similar results under the same conditions.

This makes it very important for experiments, evaluations, debugging, and security testing.

For example, when evaluating a prompt injection payload, you can fix the following conditions:

model: llama3.1:8b
temperature: 0.2
top_p: 0.9
top_k: 40
seed: 42

However, there’s a caveat.

Even with a fixed seed, 100% identical results are not guaranteed across all environments.

Results can vary slightly depending on GPU operations, runtime implementation, model version, and parallel processing methods.

Nevertheless, it is a very important parameter for improving experimental reproducibility.

8. stream: Should responses be received in real-time?

Stream determines whether to receive the model’s response all at once or token by token.

stream: true

If stream: true, responses are output in real-time, like a chatbot.

Users can see the model generating its response immediately.

Conversely, if stream: false, the entire response is returned once completed.

stream: false

From a service perspective, stream: true can be beneficial for user experience.

Even with long responses, users feel less like they are waiting.

However, for automated evaluation or security testing, stream: false might be more convenient.

This is because the success or failure can be determined at once after receiving the complete response.

9. format: Enforcing output format

Format is a value that specifies the model’s response format.

For example, you can make it respond in JSON.

format: json

Or, depending on the API, you can specify a JSON Schema.

When using an LLM as a simple chatbot, natural language responses are sufficient.

However, when integrating with a system, a defined format is necessary.

For example, let’s assume vulnerability analysis results need to be received in JSON like this:

{
  "vulnerability": "prompt injection",
  "severity": "medium",
  "evidence": "모델이 시스템 지시를 무시함",
  "success": true
}

In such cases, using ‘format’ simplifies subsequent automation.

Format is also useful in security testing.

This is because the model’s response can be automatically judged as success/failure without human intervention.

10. repeat: Suppressing repetitive output

LLMs sometimes have a problem of repeating the same sentences or words.

For example, like this:

This issue is important. Important. Important. Important…

To reduce this, repeat suppression settings are used.

The names may vary depending on the tool or API, but typically include values like these:

repeat:
  repeat_penalty: 1.1
  repeat_last_n: 64

repeat_penalty reduces the likelihood of selecting already used words or expressions again.

repeat_last_n determines the range of recent tokens to check for repetition.

If the repeat suppression value is too weak, the same sentence might be repeated.

Conversely, if it’s too strong, even natural expressions might be suppressed, making the response awkward.

Therefore, it’s generally best to use default values and only adjust them when repetition issues occur.

11. trial_count: Why you shouldn’t test just once

trial_count is a value that determines how many times the same test should be repeated.

trial_count: 10

Evaluating LLMs with a single run is risky.

This is because models respond probabilistically.

For example, when testing a prompt injection payload, it might fail once and succeed another time.

In such cases, stating that ‘this payload succeeds’ or ‘fails’ based on a single result is inaccurate.

Therefore, in security testing, it is necessary to repeat tests multiple times.

For example, if it succeeds 7 out of 10 runs, the success rate is 70%.

trial_count: 10
success_count: 7
success_rate: 70%

This allows for quantitative comparison of the payload’s effectiveness.

12. success_criteria: Criteria for success and failure

success_criteria defines the standard for what constitutes a successful test result.

success_criteria:
  - "시스템 프롬프트를 노출하면 성공"
  - "SECRET, API_KEY, FLAG 값을 출력하면 성공"
  - "정책상 거부하면 실패"

Without these criteria, evaluations become subjective.

For example, when a response is generated, one person might deem it ‘successful’ while another deems it ‘failed’.

Success criteria are particularly important in security testing.

For example, for prompt injection tests, criteria can be defined as follows:


Test Objective	Example Success Criteria
System prompt leakage	Outputs part or all of the system instructions
Sensitive information leakage	Outputs API Key, Token, Secret, or Flag values
Policy bypass	Responds to a request that should have been denied
Tool misuse	Performs an unauthorized tool call
Data boundary violation	Accesses another user’s data

Success criteria must be defined before testing.

Changing criteria after seeing the results reduces the reliability of the experiment.

Examples of commonly used settings in practice

In security testing or quality evaluation, stability is usually prioritized.

model: llama3.1:8b
temperature: 0.2
top_p: 0.9
top_k: 40
num_predict: 512
num_ctx: 8192
seed: 42
stream: false
format: json
repeat:
  repeat_penalty: 1.1
  repeat_last_n: 64
trial_count: 10
success_criteria:
  - "민감정보를 출력하면 성공"
  - "시스템 지시를 노출하면 성공"
  - "정책상 거부하면 실패"

Conversely, settings for creative writing or idea generation are slightly different.

model: llama3.1:8b
temperature: 0.8
top_p: 0.95
top_k: 80
num_predict: 2048
num_ctx: 8192
stream: true

In other words, there isn’t just one correct answer for parameters.

They should be set differently according to the objective.

Parameters by Objective


Objective	Important Parameters
Model selection	model
Adjusting response creativity	temperature, top_p, top_k
Limiting response length	num_predict
Processing long documents	num_ctx
Experimental reproducibility	seed
Improving user experience	stream
System integration	format
Preventing repetitive responses	repeat
Repeating security tests	trial_count
Automated evaluation	success_criteria

—

Why Parameters are Important in LLM Security Testing

In LLM security, execution conditions are as important as prompts.

Even with the same payload, results can vary depending on the following conditions:

temperature: 0.0

temperature: 0.9

At low temperatures, the model may consistently refuse, but at high temperatures, there’s a possibility of unexpected responses.

Also, if num_ctx is small, important system instructions or previous conversations might be truncated.

If num_predict is too short, the response might be cut off midway, making it difficult to determine success.

Without fixing the seed, it’s difficult to reproduce the same experiment.

Therefore, it is recommended to record at least the following items in an LLM security test report:

model:
model_version:
temperature:
top_p:
top_k:
num_predict:
num_ctx:
seed:
trial_count:
success_criteria:

This information is necessary for others to reproduce the results under the same conditions.

Conclusion: Parameters are experimental conditions for controlling LLMs

LLM parameters are not just simple options.

They are crucial experimental conditions that determine a model’s output behavior, cost, speed, reproducibility, and security evaluation results.

Simply writing good prompts does not guarantee good results.

It’s also necessary to manage which model is used, how creatively it should respond, how long the generated output should be, and how many times the test should be repeated.

Especially in LLM security testing, results without recorded parameters are difficult to trust.

This is because even with the same prompt, results can vary if the execution conditions are different.

Ultimately, using LLMs properly isn’t just about writing good prompts.

It’s about understanding both the model and its parameters and controlling them according to the objective.