Recent Updates
Introduction: CTI and IoCs in My Home Lab
As part of my home projects, specifically Home Network Guardian and Łukasz's Cybersecurity Projects 2024/2025 I have implemented advanced network monitoring mechanisms based on the concepts of Cyber Threat Intelligence (CTI) and Indicators of Compromise (IoC).
CTI is organized, analytical knowledge about threats that enables the proactive detection, prediction, and response to potential attacks. In turn, IoC are technical indicators, such as malicious domains, IP addresses, or file hashes, that point to the presence of a cyber threat.
My home lab, built on a Raspberry Pi 5, has been expanded with systems that automatically collect data on IoC from DNS traffic and analyze it using the VirusTotal platform.
System Logic and Workflow
The core of this solution is a three-stage, automated process designed to identify and flag potential threats within the home network's DNS traffic.
1. Passive DNS Monitoring (The "Listener")
The first component, the Passive DNS service, acts as a silent observer. It continuously monitors the network for DNS queries and their corresponding replies. By capturing and parsing the `dns.log` file in real time, it extracts crucial information like the queried domain, the source IP of the request, and the response IP. This data is then logged into a central MySQL database. This process allows for the creation of a comprehensive, historical record of all DNS activity on the network without actively interfering with traffic.
2. Domain Scanning (The "Hunter")
The second component is the DNS Scanner, a Python script that runs at regular intervals. It connects to the database and retrieves a list of all domains that have not yet been scanned. For each new domain, it initiates a scan request to the VirusTotal service via a dedicated API endpoint. This stage is crucial as it automates the process of checking for known malicious domains, turning raw network data into actionable intelligence.
3. Threat Analysis (The "Analyzer")
The third component is the VirusTotal API Service. This Flask-based service is responsible for handling the requests from the scanner. It receives a domain, communicates with the official VirusTotal API, and retrieves a detailed report. This report includes information about the domain's reputation and any known malicious associations. The service then stores this valuable threat intelligence in the database, linking it back to the original DNS query.
The subjects of analysis are exclusively DNS queries that have been marked as "reply." This means these domains successfully resolved the query and were not blocked at an earlier stage by standard blocklists (e.g., those maintained by CERT Polska, which can be found at this address: https://cert.pl/lista-ostrzezen/). The analysis of these queries allows for the detection of new, as yet unknown threats that might bypass existing defenses.
For example, a typical entry from the log file, which is then analyzed by the system, looks like this:
Sep 6 18:48:56 dnsmasq[1]: reply userlocation.googleapis.com is 142.250.180.106
Sep 6 18:48:56 dnsmasq[1]: reply userlocation.googleapis.com is 142.251.143.74
This entire workflow creates a robust and scalable system for proactive threat detection, turning passive network logs into an active defense mechanism.
4. Automated DNS Blocking with Pi-hole (Integration In Progress)
While Pi-hole is already active in my network as the primary DNS sinkhole for blocking ads and trackers, the next phase of this project is to integrate it with the VirusTotal data. This will allow for the automatic feeding of known malicious domains from the database directly into Pi-hole's blocklists. This step moves beyond static blocklists, creating a dynamic and responsive defense layer that automatically adapts to new threats identified by the scanner.
5. Dynamic Analysis with Suricata (Integration In Progress)
Similarly, Suricata is a key component of my network's security posture, providing powerful Intrusion Detection (IDS) capabilities. The current work focuses on building an automated pipeline that will dynamically generate Suricata rules based on the `positives_total_scans` metric from the VirusTotal analysis. This orchestration will allow the system to forward low-risk but suspicious domains for deeper behavioral analysis, logging detailed alerts to a centralized platform like Graylog for further forensic investigation.
System Architecture: Docker Components
The entire system is orchestrated using Docker Compose, which defines and manages a multi-container application. Below are the key services, their related files, and their functions:
- `mysqldb`: This container hosts the MySQL database. It serves as the central repository for storing all DNS query records captured by `dns_log_processor` and the detailed threat reports from VirusTotal.
- `dns_log_processor`: Built using `Dockerfile.dns`, this container runs the `dns_log_processor.py` script. Its main task is to continuously monitor the `dns.log` file and parse DNS queries that have a "reply" status. It then inserts this data, including the domain, response IP, and source IP, into the `mysqldb` database.
- `virustotal_scanner`: This is a custom Flask-based web service built with `Dockerfile.virustotal`. It runs the `virus_total.py` script which acts as an API. The service receives domains, queries the official VirusTotal API, and then stores the resulting analysis in the database.
- `dns_scanner`: Built with `Dockerfile.scanner`, this container runs the `dns_scanner.py` script. Its purpose is to automate scanning. It periodically connects to the database, retrieves a list of new domains that have not yet been scanned, and sends them to the `virustotal_scanner` service for analysis.
This modular, containerized approach ensures that each component can be managed independently, making the system robust, scalable, and easy to maintain.
Threat Analysis in Practice: Example Query
To demonstrate the system's effectiveness, a simple SQL query can be run to retrieve a summary of the scanned domains. The query below selects information from a database view, providing an at-a-glance overview of which domains are clean and which have been flagged as malicious.
SELECT url, scan_date, positives_total_scans, permalink FROM v_scanned_domains;
The results clearly show how the system successfully identifies and logs both legitimate and potentially malicious domains. The `positives_total_scans` column indicates the number of security vendors that flagged the URL, while the `permalink` provides a direct link to the full VirusTotal report for a deep-dive analysis.
This example demonstrates that even URLs associated with legitimate services might be flagged if they are part of a larger chain of redirects or have been involved in past malicious activities, highlighting the importance of continuous monitoring.
| URL | Scan Date | Positives / Total Scans | Permalink |
|---|---|---|---|
| consent-pref.trustarc.com | 2023-05-09T11:38:28 | 1 / 98 | Link to Report |
| google-http-relay-safebrowsing.fastly-edge.com | 2023-05-09T17:32:02 | 2 / 98 | Link to Report |
| prod.detectportal.prod.cloudops.mozcp.net | 2023-05-09T02:49:29 | 1 / 98 | Link to Report |
| jqv4-cizz-waw02-orangepl-isp.1.oca.nflixvideo.net | 2023-05-09T11:47:28 | 1 / 98 | Link to Report |
Automated Threat Response: The Next Frontier
While the current system provides a robust platform for collecting and analyzing threat intelligence, the next logical step is to transition from a passive monitoring solution to a fully automated threat response mechanism. This evolution will introduce dynamic, rule-based logic to actively neutralize threats based on the CTI data collected.
This orchestrated defense strategy will be governed by a predefined threshold based on the VirusTotal analysis results. The `positives_total_scans` metric will serve as the key decision-making factor, creating a tiered response system:
-
Tier 1: Deeper Analysis via Suricata
Domains that have not exceeded a low-risk threshold (e.g., fewer than 3 positives) will not be immediately blocked. Instead, they will be dynamically forwarded to Suricata—an advanced Intrusion Detection System (IDS). This will allow for deeper behavioral analysis within the network traffic. The data from Suricata, including detailed alert information, will be centralized in Graylog for comprehensive logging and analyst review. This approach enables the investigation of suspicious, but not yet confirmed, threats, preventing potential false positives while still providing valuable forensic data. -
Tier 2: Immediate DNS-Level Blocking
Domains that meet or exceed a critical threat threshold (e.g., 3 or more positives) will be automatically and immediately blocked at the DNS level. This is a high-impact countermeasure designed to prevent any further communication with a known malicious host. By integrating this action directly with the DNS resolver, the system proactively protects all devices on the network, preventing data exfiltration, malware downloads, or other malicious activities from reaching their target.
This automated, two-tiered response pipeline represents a significant leap forward, transforming the home network into an intelligent, self-defending system. It automates the correlation of threat intelligence with active countermeasures, minimizing response time and significantly enhancing the network's overall resilience against sophisticated cyber threats.
Configuration and Source Files
All configuration files, including the `docker-compose.yml` that orchestrates the entire system, can be found in the dedicated GitHub repository.