How to Check Domains in a Log: Track Domains in Server Log Files

Quick answer: First find out which log holds the domains: web access logs record requested hosts and referrers, mail logs record sender and recipient domains, and DNS logs record queried names. Then extract the domains with simple commands, count them, and look up any unknown ones before deciding whether they need action.

Server logs record almost everything a server does, and many of those records include domain names. Knowing how to check domains in a log helps site owners spot unknown domains pointing at their server, find which sites send traffic, investigate odd mail activity and catch problems early.

This guide explains where domains appear in common logs, how to extract and count them, and how to keep track of them over time.

Key takeaways

• Different logs record domains in different places, so the first step is choosing the right log.
• Web logs only record the requested host name if the log format includes it.
• A few standard command-line tools can extract and count domains quickly.
• Unknown domains should be checked before any action is taken.
• Logs contain personal data, so they should be kept only as long as needed.

Where domains appear in server logs

How to check domains in a log, step by step

The six steps below go from making sure the data exists to tracking it over time.

Step 1: Make sure the log records the host name

The standard "combined" log format used by Apache and nginx records the visitor's IP address, the request, the status code, the referrer and the browser. It does not record which domain was requested, because one server often hosts several sites.

To add it, Apache can log the host with %v (the server name of the site that answered) or %{Host}i (the Host header the visitor sent). Debian-based systems already include a "vhost_combined" format that starts each line with the site name. In nginx, the $host variable can be added to a custom log format.

Step 2: Find the right files

Common locations include /var/log/apache2/ or /var/log/httpd/ for Apache, /var/log/nginx/ for nginx, and /var/log/mail.log or /var/log/maillog for mail. Older logs are often compressed into .gz files by log rotation, and the zgrep command searches those without unpacking them.

Step 3: Extract and count the domains

The commands below run on Linux and macOS. Each one prints a count next to each domain, with the busiest first.

1. For logs that start with the site name (vhost_combined), count requests per domain: cut -d' ' -f1 access.log | cut -d: -f1 | sort | uniq -c | sort -rn | head -20
2. For the referring domains in a combined log: awk -F'"' '{print $4}' access.log | awk -F/ '{print $3}' | sort | uniq -c | sort -rn | head -20
3. For sender domains in a Postfix mail log: grep -o 'from=<[^>]*@[^>]*>' /var/log/mail.log | sed 's/.*@//; s/>$//' | sort | uniq -c | sort -rn | head -20
4. To check one specific domain: grep -ci "example.com" access.log, which prints how many lines mention it.

The output is a simple ranked list. A result such as 412 www.example.com followed by 37 old-brand.example shows that most traffic goes to the main site, while a second, older domain still receives some requests. That second line might be a forgotten domain worth renewing, or a sign that links still point to an old name.

Windows servers running IIS keep logs in a similar way, and the host name is recorded when the "cs-host" field is switched on in the logging settings.

Step 4: Look for anything unexpected

Once the list is ready, a few patterns deserve attention:

• Unknown host names: if requests arrive for a domain the site owner does not run, someone may have pointed their domain at the server's IP address, sometimes to copy its content.
• Strange referrers: spam domains sometimes appear as referrers to attract clicks from people reading analytics reports.
• Unexpected mail domains: a sudden spike of mail from or to one domain can signal a compromised account or a spam run.
• Odd DNS lookups: repeated lookups of random-looking names can point to malware on a device.

Step 5: Check the unknown domains

Before blocking anything, find out what each unknown domain is. A WHOIS Lookup shows when it was registered and through which registrar. A DNS Lookup shows where it points, which confirms whether it really targets the server. If an IP address from the same lines looks suspicious, the IP Blacklist Checker shows whether it appears on known spam or abuse lists.

Step 6: Track domains over time

One check is a snapshot, while tracking shows trends. A simple approach is to save each day's domain counts to a dated file and compare them with the previous day. A new domain near the top of the list is worth a look.

For larger setups, log management tools can collect logs from many servers, extract domains automatically and send alerts when something new appears.

For a domain that should never appear in requests at all, a web server rule can reject requests for unknown host names. That keeps other people's domains from serving the site's content.

Logs and privacy

Logs often contain personal data, such as IP addresses and email addresses, which data protection laws like the GDPR treat as personal information. Good practice is to keep logs only as long as they are needed, limit who can read them, and rotate old files automatically.

Next steps

Knowing how to check domains in a log is a small skill with a big payoff: it turns raw log lines into a clear list of who is talking to the server. When an unfamiliar name turns up, start with a WHOIS Lookup to see who registered it. For turning messy URLs from logs into clean company domains, see the guide on extracting a registrable domain from a URL.

Frequently asked questions