[Hands-on] “Too Many Logs, I’m Drowning!” 🤮 Extracting Only ‘Anomalous Behavior’ from Vast System Logs with AI! 📉

Jul 14, 2026

—

Hello, security analysts who love efficiency! 🕵️‍♂️

Last time, we executed malware in a sandbox and collected logs with ProcMon, didn’t we?

But when you actually open the collected log files… 100,000 lines, 200,000 lines… you’re often overwhelmed by an endless feast of text. 😱

“When am I supposed to read all of this? How do I find only what the hacker did here?”

Finding a ‘needle’ in this ‘haystack’ with human eyes is almost impossible. But for our AI assistant, it’s a piece of cake! 🥣

Today, we will conduct a log filtering and anomaly detection practice where we feed vast system logs (Syslog, ProcMon, Firewall Log, etc.) to AI and instruct it to “ignore normal things and pick out only the strange ones!”

1. 🚧 Realistic Problem: “AI, read all of this” (Context Limit)

Before we start, there’s an important point. AI models like ChatGPT or Claude have a limit on the number of characters (Token Limit) they can read at once. If you paste an entire 1GB log file, they’ll spit out “Too long!”

So, we need a ‘smart strategy’.

Preprocessing: Filter out clearly normal behaviors (e.g., default Windows system operations) first using Python or similar tools, or
Chunking: Split logs into chunks of 100-200 lines and input them, or
Key Section Extraction: Extract and input only logs from suspicious timeframes (e.g., 1 minute immediately after malware execution).

Today’s practice will proceed by requesting AI to analyze ‘key section logs (approx. 50-100 lines)’.

2. 📝 [Practice Data] Example of Confusing Logs

Let’s assume the logs you scraped from ProcMon are mixed as shown below. Normal and malicious entries are intertwined.

[Log Sample (Simulation)]

Plaintext

[INFO] 14:00:01 svchost.exe (PID: 892) - Network Connect 52.1.1.1:443 (Microsoft Update)
[INFO] 14:00:02 chrome.exe (PID: 4120) - WriteFile C:UsersUserAppDataLocalGoogleCache...
[WARN] 14:00:05 powershell.exe (PID: 5521) - Process Create -Command "powershell -W Hidden -Enc aW1wb3J0..."
[INFO] 14:00:06 explorer.exe (PID: 1204) - ReadFile C:WindowsSystem32shell32.dll
[CRIT] 14:00:08 malware.exe (PID: 5521) - RegSetValue HKCUSoftwareMicrosoftWindowsCurrentVersionRunEvil
[INFO] 14:00:10 kakaotalk.exe (PID: 3002) - Network Receive 203.21.1.1:8080
[WARN] 14:00:12 cmd.exe (PID: 6601) - Process Create "vssadmin.exe Delete Shadows /All /Quiet"

Now, let’s take this block of text to the AI.

3. 🧠 [Prompt] “Ignore Normal, Find Only Anomalies”

If you simply say “analyze this,” the AI will try to explain every line. You need to strongly demand “Filtering”.

📋 Log Analysis Specific Prompt

[Role]

You are a Senior Threat Hunter at a Security Operations Center (SOC).

[Task]

Analyze the system logs (Log Data) provided below and extract only ‘anomalous and threatening behaviors (Anomalies)’.

[Filtering Rules – Important!]

1. Whitelist: Strictly ignore and do not include common user/system behaviors such as web browser cache writes, Windows updates, and messenger communications in the results.

2. Detection Targets:

– Obfuscated script execution (Encoded Command)

– Registry auto-run registration (Persistence)

– Attempts to delete backups and disable system recovery

– Suspicious network connections from unknown processes

[Output Format]

Organize the analysis results into a table with the following format:

| Time | Process | Behavior Summary | Threat Level (High/Medium/Low) | Basis for Judgment |

[Log Data]

(Paste the Log Sample above here)

4. 🕵️‍♂️ [Results] ‘Real Threats’ Filtered by AI

Following our instructions, AI removes all noise like chrome.exe, explorer.exe, and svchost.exe, leaving only the core findings.

🤖 AI Analysis Results (Example)

🚨 Threat Detection Report

Excluding common system behaviors (Chrome cache, Windows update, KakaoTalk, etc.), 3 log entries suspected of being security threats have been identified.

| Time | Process | Behavior Summary | Threat Level | Basis for Judgment |

| — | — | — | — | — |

| 14:00:05 | powershell.exe | Obfuscated command execution | High (Critical) | The -W Hidden -Enc options are primarily used to execute malicious code while evading user detection. |

| 14:00:08 | malware.exe | Startup program registration | High (Critical) | Writing a value to the HKCU…Run key is a typical persistence technique to ensure malware continues to run after reboot. |

| 14:00:12 | cmd.exe | Shadow copy deletion | High (Critical) | vssadmin Delete Shadows is one of the most common preliminary actions performed by ransomware to prevent file recovery. |

💡 Overall Opinion

This system is strongly suspected of being infected with ransomware or a backdoor. In particular, the execution of vssadmin suggests the need for immediate isolation.

5. 💡 Pro Tip: Building a ‘Pipeline’ for Large-Volume Log Processing

What if there are hundreds of thousands of lines of logs?

In such cases, a hybrid approach linking Python scripts with AI APIs is necessary.

Python (1st Filter): First remove “Known Good” processes (e.g., System, MsMpEng.exe) using logic like findstr or grep. This can reduce data volume by 90%.
AI (2nd Detailed Analysis): Send only the remaining 10% of ‘ambiguous logs’ to the AI and ask, “Is this malicious?”

💬 Prompt Tip:

“I know how to use Python. Write me a Python preprocessing script that excludes processes with Microsoft signatures from the ProcMon.csv file and leaves only the remaining suspicious processes.”

If you request this, AI will even create a preprocessing tool for you! 🛠️

🎉 Conclusion: Don’t Look at the Forest, Look at the Rotten Tree!

Today, we learned how to avoid drowning in a ‘flood of logs’ and instead, use AI as a filter to extract only ‘threats’.

Before: Pondering “Is this normal?” while manually reviewing 10,000 lines of logs. 😵
After: Responding based on 5 lines of key threats summarized by AI. 😎

This is the essence of Security Operations Center (SOC) automation and the working method of a smart analyst.

Now, we have analyzed malware’s code (static), behavior (dynamic), and logs (traces). All the puzzle pieces are gathered.

Next time, we will put all these pieces together and conclude with [Final Practice] “Automated Generation of Expert-Level Malware Analysis Reports”. Stay tuned! 👋