Presidio is a tool that finds sensitive sections, such as personal information, in text. Regression testing is a check to ensure that examples previously processed correctly continue to be correct even after code or configuration changes. If you have extended Korean detection, it is important to first fix what kind of information was found and at what location, rather than just “something was found.”
Simply installing a Korean model and passing language=”ko” does not validate both the configuration and detection quality. The model, recognizer, and analysis engine must use the same language, and the returned start and end positions must match the correct answer. This article examines that connection using practice markers like virtual number-AB12, not actual personal information. It is not an example of measuring the overall detection rate for Korean names, addresses, or contact information. Official Multilingual Configuration
First, Match the Language Settings in Three Places
Following a single analysis request, there are three connections to check:
- NLP Engine: Passes ko input to the Korean model.
- Recognizer Registration: Recognizers to be applied to Korean input are registered with supported_language=”ko”.
- AnalyzerEngine: Supported languages include ko, and analysis requests also use language=”ko”.
NLP is the process of dividing sentences into tokens or finding entities like names. Regular expression recognizers find specific string formats. Since their roles are different, you need to separately confirm the existence of a Korean NLP model and the selection of a Korean regular expression recognizer. Even if a regular expression itself seems language-agnostic, surrounding contextual words and recognizer selection are affected by language settings. Language-specific Recognizer Settings
spaCy’s ko_core_news_sm is a pipeline for Korean news text. The sm in the model name means it’s a small model family and does not guarantee detection for all types of personal information. Check the model version and spaCy compatibility range together. In this article, Presidio Analyzer 2.2.364, spaCy 3.8.16, and ko_core_news_sm 3.8.0 were used in an isolated Python 3.12 environment. spaCy Korean Models, 3.8.0 Official Model Metadata
Creating a Reproducible Environment
Creating a virtual environment in a new practice directory allows you to test without changing packages in existing applications. The following commands download public packages and models, so an internet connection is required, but the analysis examples do not call external services.
python3.12 -m venv .venv
.venv/bin/python -m pip install presidio-analyzer==2.2.364 spacy==3.8.16
.venv/bin/python -m spacy download ko_core_news_sm
.venv/bin/python -m pip show presidio-analyzer spacy ko-core-news-smThe model download command verifies the selected model version. In this article’s execution, 3.8.0 was installed. If a different model version is selected later, the same test should be re-run, and the results and installed version should be recorded together for comparison. If you need to preserve the entire environment, save the pip freeze results as a dependency file, but do not disclose installation URLs containing internal repository credentials.
Connecting the Korean Engine and Practice Recognizer
Save the following content as ko_analyzer.py. DEMO_ID is a practice-only entity type created in this article. It does not replicate actual phone number, account number, or organization identifier rules.
from presidio_analyzer import AnalyzerEngine, Pattern, PatternRecognizer, RecognizerRegistry
from presidio_analyzer.nlp_engine import SpacyNlpEngine, NerModelConfiguration
from presidio_analyzer.predefined_recognizers import SpacyRecognizer
def build_analyzer():
# Explicitly link Korean model labels and Presidio entity names.
ner = NerModelConfiguration(
model_to_presidio_entity_mapping={
"PS": "PERSON", "LC": "LOCATION", "OG": "ORGANIZATION",
},
labels_to_ignore=["DT", "TI", "QT"],
)
nlp = SpacyNlpEngine(
models=[{"lang_code": "ko", "model_name": "ko_core_news_sm"}],
ner_model_configuration=ner,
)
nlp.load()
registry = RecognizerRegistry(supported_languages=["ko"])
registry.add_recognizer(SpacyRecognizer(supported_language="ko"))
registry.add_recognizer(PatternRecognizer(
name="KoreanDemoIdRecognizer",
supported_entity="DEMO_ID",
supported_language="ko",
patterns=[Pattern(
name="연습 표식",
regex=r"가상번호-[A-Z]{2}d{2}(?![A-Za-z0-9])",
score=0.8,
)],
context=["연습", "식별자"],
))
return AnalyzerEngine(
nlp_engine=nlp, registry=registry, supported_languages=["ko"],
)PS, LC, OG are entity labels used by this Korean model. They are mapped to Presidio’s PERSON, LOCATION, ORGANIZATION, and date, time, quantity labels not used in this example are excluded. If the model changes, the labels must also be checked. Adding a mapping does not mean that name detection quality has improved; it means different naming schemes have been connected. Model Labels, Presidio spaCy Engine Implementation
The practice regular expression ends with (?![A-Za-z0-9]). This is a condition to prevent a match if an English letter or number follows a two-digit number. Without it, an error might occur where only “virtual number-AB12” is found from “virtual number-AB123”. The Korean postposition “를” (reul) is allowed. Such boundary conditions are better revealed in tests that check positions rather than just string types.
Figure: Configuration connection check + synthetic marker type/position check · Reconstructed based on official documentation
Checking Entity Types and Correct Segments Together
Save test_ko_regression.py in the same directory. Correct segments are not automatically generated from the recognizer’s regex results. Using the results directly as correct answers could lead to the same bug being counted twice, causing the test to pass.
import unittest
from ko_analyzer import build_analyzer
# end is a Python string position that does not include the last character.
CASES = [
("조사 경계", "연습 식별자 가상번호-AB12를 확인합니다.", [("DEMO_ID", 7, 16)]),
("기호 경계", " 가상번호-CD34!", [("DEMO_ID", 2, 11)]),
("형식 불일치", "연습 식별자 가상번호-AB123", []),
("탐지 대상 없음", "식별자는 다음 문서에 설명되어 있습니다.", []),
]
class KoreanRegressionTests(unittest.TestCase):
@classmethod
def setUpClass(cls):
cls.analyzer = build_analyzer()
def test_language_model_and_recognizers_are_connected(self):
self.assertEqual(set(self.analyzer.supported_languages), {"ko"})
self.assertEqual(set(self.analyzer.nlp_engine.get_supported_languages()), {"ko"})
active = self.analyzer.registry.get_recognizers(
language="ko", entities=["DEMO_ID"],
)
self.assertTrue(any(r.name == "KoreanDemoIdRecognizer" for r in active))
self.assertEqual(self.analyzer.nlp_engine.nlp["ko"].lang, "ko")
def test_exact_type_and_span(self):
for name, text, expected in CASES:
with self.subTest(case=name):
found = self.analyzer.analyze(
text=text, language="ko", entities=["DEMO_ID"],
score_threshold=0.5, return_decision_process=True,
)
actual = sorted((r.entity_type, r.start, r.end) for r in found)
self.assertEqual(actual, expected)
for result in found:
self.assertTrue(text[result.start:result.end].startswith("가상번호-"))
self.assertIsNotNone(result.analysis_explanation)
def test_wrong_language_fails(self):
with self.assertRaises(ValueError):
self.analyzer.analyze(
text="가상번호-AB12", language="en", entities=["DEMO_ID"],
)
if __name__ == "__main__":
unittest.main()start and end are character positions in the string in this Python example, not UTF-8 byte positions. Verify that text[start:end] extracts the exact marker. Examples with emojis at the beginning reveal byte/character confusion that is easily missed with examples containing only Korean/English. If results are sent to clients in other languages, the string indexing scheme must also be separately checked for consistency.
entities=[“DEMO_ID”] narrows the scope of this test. Even if SpacyRecognizer is registered, this analysis does not measure PERSON detection quality. It only verifies that the NLP model is actually loaded, the Korean recognizer is selected, and the correct segments of the practice markers are maintained.
Reading Execution Results and Failure Causes
The execution command is as follows:
.venv/bin/python -m unittest -v test_ko_regression.pyThis code was executed in the isolated environment of the version mentioned above. All three tests passed, and the output was “Ran 3 tests” and “OK”. This included four synthetic sentences and a check for incorrect language requests. These are not results using actual personal information or real business documents.
If a connection check fails, first verify the supported languages and registered recognizers. If model loading fails, check model installation and spaCy compatibility versions. If the type is correct but the segment check fails, look at regex boundaries, input preprocessing, and character indices. Lowering the threshold is not a common remedy for these three problems.
return_decision_process=True helps to identify which recognizers and patterns produced a given result. However, the decision process describes what was found, and does not automatically explain all reasons for what was not found. Detailed explanations should only be checked with synthetic examples, and original input or trace logs from actual inputs should not be publicly disclosed. Decision Process Explanation
Extending Small Tests to Real Quality Checks
If these four sentences pass, then next, expand with normal sentences, changes in delimiters, changes in postpositions and spaces, overlapping detection segments, etc. Model-based items like names and addresses should be checked with separate synthetic ground truths. Repeatedly increasing only one sentence that matches a regular expression does not help find various omissions.
Comparing ground truth and predictions as identical tuples of entity type, start, and end allows counting TP, FP, FN based on exact segment criteria. Based on this, precision is calculated as the ratio of correct answers among detected items, and recall as the ratio of found items among correct answers. To compare results, you must first decide whether a slightly different segment counts as a failure, or if overlapping segments count as partial correct answers. Do not group scores from different criteria as the same detection rate. Official Evaluation Guidelines
The value of minimum regression testing is not in certifying that Korean is handled well overall. It lies in leaving an executable record of what settings and what synthetic examples need to be maintained. Even text that passes automatic detection cannot guarantee safety from re-identification or customer attribution, so a separate human review stage for meaning and context should be maintained before public release.
Leave a Reply