FAQ: Firewalls: What am I seeing?

This document answers the question: I've seen <something> on my firewall; what does it mean? Firewall administrators regularly see strange behaviour showing up in their logfiles. This document describes some of the common things seen on these firewalls, and what they mean.

Note that this document is intended both for owners of personal firewalls as well as corporate firewalls.

0. Information about this FAQ

Version 0.1.7, August 25, 1999 You can get this document from:
http://www.robertgraham.com/pubs/firewall-seen.html (HTML)
This is an early work in progress. I have a lot more things I want to add to this document, but I haven't gotten around to it yet. If you have any suggestions, please e-mail me.

1. What does port number ZZZZ mean?

Port numbers are divided into three ranges: In reality, machines start assigning "dynamic" ports starting at 1024. We also see strangeness, such as Sun starting their RPC parts at 32768.

Where to get a more complete list of port info:

ftp://ftp.isi.edu/in-notes/iana/assignments/port-numbers
"Assigned Numbers" RFC, the official source for port assignments.
http://advice.networkice.com/advice/Exploits/Ports/
Database of portnumbers, hyperlinked to various exploits on those port numbers.
/etc/services
On UNIX systems, the file /etc/services contains a list of commonly used UNIX port number assignements. On Windows NT, this file is located in %systemroot%/system32/drivers/etc/services.

1.1 What are some common incoming TCP/UDP probes against my firewall?

Here is a list of common scans we see these days:
0 Commonly used to help determine the operating system, in which case the destination IP address will be 0.0.0.0 and the ACK bit will be set.
7Echo A common DoS attack is an echo-loop, where the attacker forges a UDP from one machine and sends it to the other, then both machines bounce packets off each other as fast as they can (see also chargen). Another common thing seen is TCP connections to this port by DoubleClick. They use a product called "Resonate Global Dispatch" that connects to this port on DNS servers in order to locate the closest one.
11sysstat This is a UNIX service that will list all the running processes on a machine and who started them. This gives an intruder a huge amount of information that might be used to compromise the machine, such as indicating programs with known vulnerabilities or user accounts. It is similar the contents that can be displayed with the UNIX "ps" command.
19chargenThis is a service that simply spits out characters. The UDP version will respond with a packet containing garbage characters whenever a UDP packet is received. On a TCP connection, it spits out a stream of garbage characters until the connection is closed. Hackers can take advantage of IP spoofing for denial of service attacks. Forging UDP packets between two chargen servers, or a chargen and echo can overload links as the two servers attempt to infinitely bounce the traffic back and forth. Likewise, the "fraggle" DoS attack broadcasts a packet destined to this port with a forged victim address, and the victim gets overloaded with all the responses.
21FTPThe most common attack you will see are hackers/crackers looking for "open anonymous" FTP servers. These are servers with directories that can be written to and read from. Hackers/crackers use these machines as way-points for transfering warez (pirated programs) and pr0n (intentionally mispelled word to avoid search engines classifying this document).
23Telnet The intruder is looking for a remote login to UNIX. Most of the time intruders scan for this port simply to find out more about what operating system is being used. In addition, if the intruder finds passwords using some other technique, they will try the passwords here.
25SMTPSpammers are looking for SMTP server that allow them to "relay" spam. Since spammers keep getting their accounts shut down, they use dial-ups to connect to high bandwidth e-mail servers, then send a single message to the relay with multiple addresses. The relay then forwards to all the victims. There are also numerous holes in many SMTP servers that hackers/crackers can exploit to break in.
53DNSDNS. Hackers/crackers may be attempting to do zone transfers (TCP), to spoof DNS (UDP), or even hide other traffic since port 53 is frequently neither filtered nor logged by firewalls.
67 and 68bootp
DHCP
Bootp/DHCP over UDP. Firewalls hooked to DSL and cable-modem lines see a ton of these sent to the broadcast address 255.255.255.255. These are machines that are asking to for an address assignement from a DHCP server. You could probably hack into them by giving them such an assignment and specifying yourself as the local router, then execute a wide range of "man-in-the-middle" attacks. The client requests configuration on a broadcast to port 68 (bootps). The server broadcasts back the response to port 67 (bootpc). The response uses some type of broadcast because the client doesn't yet have an IP address that can be sent to.
69TFTP(over UDP). Many servers support this protocol in conjunction with BOOTP in order to download boot code to the system. However, they are frequently misconfigured to provide any file from the system, such as password files. They can also be used to write files to the system.
110POPThere are numerous security holes in POP services (POP is used by clients accessing e-mail on their servers).
111sunrpc
portmap
rpcbind
Sun RPC PortMapper/RPCBIND. Access to portmapper is the first step in scanning a system looking for all the RPC services enabled, such as rpc.mountd, NFS, rpc.statd, rpc.csmd, rpc.ttybd, etc. If the intruder finds the appropriate service enabled, s/he will then run an exploit against the port where the service is running.
113identd
auth
This is a protocol that runs on many machines that identifies the user of a TCP connection. In standard usage this reveals a LOT of information about a machine that hackers can exploit. However, it used by a lot of services by loggers, especially POP, IMAP, SMTP, and IRC servers. In general, if you have any clients accessing these services through a firewall, you will see incoming connection attempts on this port. Note that if you block this port, clients will perceive slow connections to e-mail servers on the other side of the firewall.
119NNTP
news
Network News Transfer Protocol, carries USENET traffic. This is the port used when you have a url like news://comp.security.firewalls. Attempts on this port are usually by people hunting for open USENET servers. Most ISPs retrict access to their news servers to only their customers. Open news servers allow posting and reading from anybody, and are used to access newgroups blocked by someone's ISP, to post anonymously, or to post spam.
137NetBIOS
name service
nbtstat
(UDP)This is normal traffic. When a machine you are communicating with attempts to resolve your IP address into a name (for logging purposes), it calls the function 'gethostbyaddr()'. On UNIX, this function uses DNS and possibly NIS. On Windows, this function attempts DNS and NetBIOS. Thus, the average Internet server will frequently be pinged by packets sent to port 137. Note that Windows machines also use a source port of 137 as well as a desination port of 137. UNIX machines will often use high-numbered ports.

You will see these packets any time someone is attempting to resolve your name, way may also result in unknown machines sending these packets to your site. For example, if someone is using your IP address as part of a decoy scan, then the victim will send NetBIOS packets at you in order to resolve your name.

143IMAPSame security idea as POP above. Note that for awhile, there was a Linux worm (admw0rm) that would spread by compromising port 143, so a lot of scans on this port are actually from innocent people.
161SNMP (UDP) A very common port that intruders probe for. SNMP allows for remote management of devices. All the configuration and performance information is stored in a database that can be retrieved or set via SNMP. Many managers mistakeningly leave this available on the Internet. Crackers will first attempt to use the default passwords "public" and "private" to access the system, they may then attempt to "crack" the password by trying all combinations.
162SNMP trap Probably a misconfiguration.
177xdmcp Numerous hacks may allow access to an X-Window console, it needs port 6000 open as well in order to really succeed.
535CORBA
IIOP
(UDP) If you are on a cable-modem or DSL VLAN, then you may see broadcasts to this port. CORBA is an object-oriented remote procedure call (RPC) system. It is highly likely that when you see these broadcasts, you can use the information to hack back into the systems generating these broadcasts.
635mountdLinux mountd bug. This is a popular bug that people are scanning for. Most scans on this port are UDP-based, but they are increasingly TCP-based (mountd runs on both ports simultaneously).
1024----- Many people ask the question what this port is used for. The answer is that this is the first port number in the dynamic range of ports. Many applications don't care what port they use for a network connection, so they ask the operating system to assign the "next freely available port". In point of fact, they as for port 0, but are assigned one starting with port 1024. This means the first application on your system that requests a dynamic port will be assigned port 1024. You can test this fact by booting your computer, then in one window open a Telnet session, and in another window run "netstat -a". You will see that the Telnet application has been assigned port 1024 for its end of the connection. As more applications request more and more dynamic ports, the operating system will assign increasingly higher port numbers. Again, you can watch this effect with 'netstat' as your browse the Internet with your web browser, as each web-page requires a new connection.
1025----- See port 1024.
1026----- See port 1024.
1027----- See port 1024.
1114SQL This is rarely probed by itself, but is almost always seen as part of the sscan script.
1243Sub-7Trojan Horse (TCP). This is a commonly seen scan looking for systems compromised by this trojan. Sub-Seven scans are becoming very frequent, primarily due to an easy-to-use scanner built-in to the client.
1080SOCKS This protocol tunnels traffic through firewalls, allowing many people behind the firewall access to the Internet through a single IP address. In theory, it should only tunnel inside traffic out towards the Internet. However, it is frequently misconfigured and allows hackers/crackers to tunnel their attacks inwards, or simply bounce through the system to other Internet machines, masking their attacks as if they were coming from you. WinGate, a popular Windows personal firewall, is frequently misconfigured this way. This is often seen when joining IRC chatrooms.
2049NFS The NFS program usually runs at this port. Normally, access to portmapper is needed to find which port this service runs on, but since most installations run NFS on this port, hackers/crackers can bypass NFS and try this port directly.
3128squid This is the default port for the "squid" HTTP proxy. An attacker scanning for this port is likely searching for a proxy server they can use to surf the Internet anonymously. You may see scans for other proxies at the same time, such as at port 8000/8001/8080/8888. Another cause of scans at this port, for a similar reason, is when users enter chatrooms. Others users (or the servers themselves) will attempt to check this port to see if the user's machines supports proxying.
5632pcAnywhereYou may see lots of these, depending on the sort of segment you are on. When a user opens PCAnywhere, it scans the local Class C range looking for potential agents. Hackers/crackers also scan looking for open machines, so look at the source address to see which it is.
13223PowWowThe "PowWow" chat program from Tribal Voice. It allows users to open up private chat connections with each other on this port. The program is very agressive at trying to establish the connection and will "camp" on the TCP port waiting for a response. This causes a connection attempt at regular intervalls like a heartbeat. This can be seen by dial-up users who inherit IP addresses from somebody who was chatting with other people: it will appear as if many different people are probing that port. The protocol uses the letters "OPNG" as the first four bytes of its connection attempt. more
30100NetSphereTrojan Horse (TCP). This is a commonly seen scan looking for systems compromised by this trojan.
31337 "elite" and Back Orifice. This number means "elite" in hacker/cracker spelling (3=E, 1=L, 7=T). Lots of hacker/cracker backdoors run at this port, but the most important is Back Orifice. This is probably the single most popular scan nowdays on the Internet.
31789Hack-a-tack UDP traffic on this port is currently being seen due to the "Hack-a-tack" RAT (Remote Access Trojan).
33434-33600traceroute If you see a series of UDP packets within this port range (and only within thisrange), then it is probably indicative of traceroute. See traceroute for more info.
41508Inoculan Inoculan on UDP. Older versions of Inoculan apparantely generate huge quantities of UDP traffic directed at subnets in order to discover each other. More info can be found at http://www.circlemud.org/~jelson/software/udpsend.html and http://www.ccd.bnl.gov/nss/tips/inoculan/index.html. Thanks to Jerry Leslie, NeoNET < leslie at clio dot rice dot edu>

1.2 What do the following source ports mean?

Ports 1-5 are indicative of a script called 'sscan'

Ports 1-1024 are for reserved services, and almost never appear as the source. There are some exceptions, such as when connections come from NAT machines. See section 1.9 for some more details.

Ports after 1024 are the ones most commonly seen.

1.3 I'm seeing attempts on the same set of ports from widely varying sources all over the Internet.

This is due to a "decoy" scan, such as in 'nmap'. One of them is the attacker, the others are not.

Forensics and protocol analysis can be used to track down who this is. For example, if you ping each of the systems, you can match up the TTL fields in those responses with the connection attempts. This will at least point a finger at a decoy scan. (The TTLs should match; if not, then they are being spoofed). [Newer versions of scanner now randomize the attackers own TTL, making it harder to weed them out].

You can also attempt to go back further in your logs, looking for all the decoy addresses or people from the same subnets. You will often see that the attacker has actually connected to you recently, while the decoyed addresses haven't.

1.4 What are Trojan Horse probes?

The first stage of a Trojan Horse attack is to get the program on a user's machine. Typical techniques are:

  • post the program to newsgroups claiming to be some other program
  • spam mailing lists with the attached program
  • post program to websites
  • send via instant messenger programs and chat systems (ICQ, AIM, IRC, etc.)
  • forge e-mail from the ISP (like AOL) with a hoax message asking somebody to run a program (such as a software update).
  • copy to startup folder via "File and Print Sharing".

The next stage of the attack is to scan the Internet looking for machines that might be compromised. The problem is that most of the techniques outlined above don't tell the cracker/hacker where ther victim machine is. Therefore, the cracker/hacker must scan the Internet looking for the machines they might have compromised.

This leads the condition where owners of firewalls (including personal firewalls) regularly see "probes" directed at their machines from crackers/hackers looking for these machines. However, if the machine hasn't been compromised, then these probes are not a problem. The probes cannot compromise the machine by themselves. Adminstrators can usually ignore these "attacks".

Typical ports used by these probes are listed below. In order to tell if your machine might be running one of these trojans, run the program "netstat -r" on your machine. Look for the ports that might be "listening" for incoming connections.
555Phase Zero
1243Sub-7, SubSeven
3129Masters Paradise
6969GateCrasher
21544GirlFriend
12345NetBus
23456EvilFtp
30100NetSphere
31789Hack'a'Tack
37337BackOrifice, and many others
50505Sockets de Troie

1.9 DNS packets from low numbered ports

Q: I've seen many DNS requests from many low port numbers below 1024. Aren't they supposed to be reserved? Aren't they supposed to use 1024-65535 range?
A: These are coming from machines behind NAT firewalls. A NAT doesn't necessarily have the concept of reserved port numbers. thanks to Ryan Russell Ryan.Russell at sybase dot com

Q: My filters reject incoming packets with source ports below 1024, so the DNS lookups are failing.
A: Don't filter that way. Lots of firewalls have similar rules, but this is somewhat "misguided" since hackers/crackers can forge whatever ports they want.

Q: Are these NAT firewalls doing it incorrectly?
A: Not in theory, but in practice it will result in failures. The "correct" way would be more strictly control DNS traffic in any case (such as essentially "proxying" DNS and forcing out through port 53).

Q: I thought DNS lookup was supposed to use a random source port above 1024?
A: In practice, your average DNS client will use a non-reserved port. However, a lot of implementations use a source port of 53. In any case, the NAT issue is completely separate because it completely changes the entire 'socket' (IP address + port combo).

1.10 Immediately upon dialing up to my ISP, my personal firewall starts alarming me about probes against port X.

This is very common. The cause is that somebody hung up just before you dialed in and your ISP assigned you the same IP address. You are now seeing the remnents of communication with the previous person.

A typical example is chat programs. If someone simply hangs up, then everyone who was chatting with that person will attempt to still send traffic to them. Some programs take a long time to timeout. Typical programs that show this behavior are PowWow and ICQ.

Another example is on-line, multiple games. You might see such traffic from gaming providers like MPlayer, or maybe from unknown servers (Quake servers litter the Internet). These games are typically UDP based, so there is no concept of a connection that can be dropped. They also are quite agressive at maintaining connections, in order to make a good user experience. Some game ports that you might see are:
7777Unreal, Klingon Honor Guard
22450Sin
26000Quake
26900Hexen 2
26950HexenWorld
27015Half-life
27500QuakeWorld
27910Quake 2
28910Heretic 2

Make sure that you carefully figure out the correct side of the connection. For example, an ICQ server runs on port 4000, and the client chooses a random high-numbered port. That means you will see UDP packets from port 4000 going to the random port. In other words, don't go looking in a port database trying to figure what that random, high-numbered port means. The significant port is the source.

1.11 IRC servers are probing me.

One of the most popular applications is "chat", like IRC. One feature of chat programs is that they reveal the IP address of the people you are chatting with. One problem with chatrooms is that people enter the rooms "anonymously" and play around, either by disrupting conversations with offtopic comments and flamebait, or by "flooding" the servers or other clients in an attempt to kicked them off.

Therefore, both servers and clients are implementing measures to stop "anonymous" use of chatrooms. In particular, they check people entering chatrooms in order to see if they are "proxying" through some other connection. The most popular of such probes is SOCKS. The assumption is that if the IP address of where you are coming from supports SOCKS, then it is possible that you have a complete separate machine and are only going through the indicated machine in order to hide your true identity.

At the same time, crackers/hackers will scan people's machines in order to determine if they are running some sort of server that can be bounced through. Again, by checking for SOCKS, the attacker hopes to find somebody that has left SOCKS open, such as a home user implementing connection sharing using SOCKS, but accidentally configured it so that anybody on the Internet has access to it.

2. ICMP

See http://www.isi.edu/in-notes/iana/assignments/icmp-parameters for a list of all the Type/Code parameters.

A common question is whether the firewall administrator should filter ICMP traffic. The answer is generally "yes": while ICMP is an integral part of communication, a lot of ICMP packets are harmful from a security context. Many firewall admins block all ICMP.

The following ICMP packets are USEFUL and probably won't cause harm by allowing them through the firewall:

MTU Path Discovery (Type 3, Code 4)
Needs to receive messages "Fragmentation Needed and Don't Fragment was Set". This allows TCP stacks to find out the maxim sized packet that can go through without fragmenting. This is a big problem on international links, which fragment at around 500 bytes, then the fragments get stopped by firewalls.
outgoing pings (outgoing Type 8, incoming Type 3)
This allow machines on the inside to ping machines on the outside -- lots of applications do this. Note that there are some backdoors that attempt to use ICMP packets as transport for information. This isn't a big concern -- if you've been compromised already, there are many other ways to open covert channels.
incoming TTL exceeded
This allows inside people to do traceroutes, and spoofed packets don't cause DoS.
Any outgoing ICMP
The theory is that outgoing ICMP packets are not very dangerous, and they help a lot in communication.

Hopefully people will give me feedback for others. For more information on this, you may want to consult "Protect and Survive Using IBM Firewall 3.1 for AIX", IBM publication SG24-2577-02. See http://www.redbooks.ibm.com for more info. I disagree with it, though.

2.0 Type = 0 (Echo Reply)

The sender is responding to a ping from your address. This could be because:
Someone's ping that person
Somebody behind the firewall is sending pings to the target.
Automated ping
Lots of applications use pings for various purposes, such as to see if their communication partner is alive, or to measure the response time. A big cause of this is VitalSign's Net.Medic, which sends pings of various sizes in order to measure link speed.
Decoy Ping Sweep
Somebody is using your IP address as a decoy in a ping sweep, so you are seeing the responses.

2.3 Type = 3 (Destination Unreachable)

The exact code is important in the Unreachable packet.

Note that Unreachables sometimes play a part in defeating SYN floods. This means that if a host you are talking to is under SYN flood attack, you will not be able to reach them if you block incoming Unreachables.

2.3.a No other communication with the sender

In some cases, you will receive destination unreachable packets from hosts you have never heard of. The most common cause of this is a "decoy scan". An attacker is sending spoofed packets a target using possibly hundreds of source addresses, including one that is the real address. The hacker's theory is that the victim won't wade through all the decoys in order to pin them down.

The best way to solve this is to examine the actual packets as described below. Try to discover is the pattern looks like what one would see in a decoy scan. For example, look for alternating port numbers in TCP or UDP headers contained within the ICMP portion of the packet.

2.3.3 Type = 3, Code = 3 (Destination Port Unreachable)

This packet is sent by a SERVER when a CLIENT tries to connect to a UDP port that isn't running. For example, if you try to send an SNMP packet to port 161, but the machine doesn't support the SNMP service, you will get back an ICMP Desination Port Unreachable packet.

Protocol Decode

The first thing to debug this problem is to check the port numbers within the packet. You probably need to use a sniffing utility as firewalls tend not to log the information. This technique relies upon the fact that ICMP messages include the IP and UDP headers of the original packet. Here is a hex dump of an ICMP unreachable:

 00 00 BA 5E BA 11 00 60 97 07 C0 FF 08 00 45 00
 00 38 6F DF 00 00 80 01 B4 12 0A 00 01 0B 0A 00
 01 C9 03 03 C2 D2 00 00 00 00 45 00 00 47 07 F0
 00 00 80 11 1B E3 0A 00 01 C9 0A 00 01 0B 08 A7 
 79 19 00 33 B8 36                              
Where the bytes 03 03 are the type/code for the ICMP packet. The last 8 bytes of the packet are the original UDP header, which decodes as:
08A7
UDP Source Port = 2215
May be dynamically allocated, so no always important.
7919
UDP Destination Port = 31001
This is very important, it meant the person was originally attempting to contact a service on port 31001.
0033
UDP Length = 51
The length of the original UDP data might be important.
B836
UDP Checksum = 0xB836
The checksum may or may not be important

Analysis

Here are some reasons why you may be seeing this:

Decoy UDP Scans
Somebody may be scanning the person who sent you the ICMP packet. They are forging the source as one of your IP addresses. They will in reality forge lots of different source addresses so that they victim can't be sure who it really is. If you receive large numbers of these packets from the same source in a short time frame, then this is a likely bet. Check the UDP Destination Port field. If it is constantly changing, then this is a very likely scenario.
Stale DNS
A client may send a DNS request to your server, which takes a long time to resolve. By the time your DNS server responds, the client has already forgotten about you and closed the UDP port assigned to receive your response. Check the UDP Source Port field to see if it equals 53. If so, then this is a likely occurance. Why does this happen? The server may be resolving a recursive query, but its own query packet was lost, so it had to time out and try again. By the time it gets back to the client, it has timed out. Many client applications (especially on Windows) do their own DNS resolution, meaning that they must create their own socket to do so. If they passed the request onto the OS, it is likely the OS would simply have left the socket open.
Multi-response DNS
Another variation is when the client receives multiple responses to the same request. It receives the first response, then closes the socket. Subsequent responses will be dropped. There other variations on this problem. A Sun machine connected with multiple NICs on the same Ethernet will assign both NICs the same MAC address, causing it to receive two copies of every frame, then send multiple responses. Likewise, a poorly written client program (it has been claimed that some DNS resolvers are multi-threaded, but not thread safe) sometimes send out multiple requests, then close the socket on the first response. However, there may be an attempt at DNS spoofing, where a hacker is attempting to corrupt the resolver's cache by sending both a recursive query and a response.
NetBIOS Resolution
If the receiver of the ICMP packets is a Windows machine, look to see if the UDP Destination Port is 137. In this case, the cause of this is the Windows system trying to execute the 'gethostbyaddr()' function, which attempts to resolve the IP address into a name using both DNS and NetBIOS. The DNS request gets sent to a DNS server somewhere (and not sent to the target), but the NetBIOS request gets sent directly to the target. If the target doesn't support NetBIOS, then it will send back an ICMP unreachable.
Traceroute
Most traceroute programs (with the exception of Windows tracert.exe) send UDP packets to closed ports. This causes a sequence of back-to-back ICMP Port Unreachable packets to be sent back to the machine doing the traceroute. Thus, if you are seeing these ICMP packets on your firewall, then somebody inside might be doing a traceroute. You may also see TTL exceeded as well.

2.3.3 Type = 3, Code = 4 (Fragmentation Needed and Don't Fragment was Set)

These are sent by routers attempting to forward IP datagrams that are marked "DF" (Don't Fragment).

Why? Both IP and TCP fragment data, but in different ways. TCP is vastly more efficient at fragmentation than IP. Therefore, stacks attempt to find the "Path MTU (Maximum Transmission Unit)". This ICMP message is sent during that process.

Let's consider ALICE talking to BOB. Both are on Ethernets (max frame size = 1500 bytes), but some intervening link limits the maximum IP packet size to 600 bytes. This means all IP packets sent will be fragmented by the routers on that link into 3 fragments. Since it is much more efficient to fragment at the TCP layer, the TCP stack will attempt to discover the MTU. It does this by setting the "DF" (Don't Fragment) bit in all its packets. As soon as it hits a router than cannot forward a packet that large, the router will send back this ICMP error message. From that, the TCP stack will know how to fragment correctly.

You should probably let these packets through the firewall. Otherwise, the intended recipient will have a hung connection as small packets get through to set up the connection, but the large packets are mysteriously dropped. A common result from this are people who see web pages that are only halfway returned.

Path MTU Discovery is becoming more and more integrated into communication. For example, IPsec needs this functionality.

2.4 Type = 4 (Source Quench)

These packets are supposed to be transmitted by routers/destination when traffic level exceeds a certain threshold. Many systems today, however, do not generate them. The reason is that we now believe that simple packet loss is the best indication of congestions (since the only reason packets are dropped, in practice, is congestion).

In general, the rules for source quenches are now (RFC 1122):

  • Routers SHOULD NOT not generate them.
  • Hosts MAY generate them.
  • Hosts SHOULD honor them.
  • Firewalls SHOULD discard them.

However, hosts still react to Source Quenches by slowing communication, so they can be used as a denial of service. Firewalls should filter these out. If a DoS is suspected, the source address of the packets will be meaningless, because the IP addresses are spoofed.

Source quenches have been known to sent by some SMTP servers.

2.8 Type = 8 (Echo a.k.a PING)

These are ping request packets. They are used all over the place; it may indicate hostile intent of someone trying to scan your computer, but it may be part of the normal network functionality. See Type = 0 (Echo Response) above for more info.

Lots of network management "scanners" will precede a scan using a special ping packet. These include ISS scanner, WhatsUp monitor, and others. This will be visible in the payload of the scanner. Most firewalls don't log this payload, so you may need to use some sort of sniffer to capture them or some time of Intrusion Detection System to flag them.

Note that blocking incoming PINGs does not mean a hacker can't scan the network. There are many other ways of doing this. For example, TCP ACK scanning becoming popular -- they usually get through the firewall, and they illicit a response from the target system.

2.11 Type = 11 (Time Exceeded In Transit)

This probably doesn't indicate an attack from a hacker/cracker.

2.11.0 Type = 11, Code = 0 (TTL Exceeded In Transit)

This can be caused by a number of things. If somebody from your site is doing traceroutes out to the Internet, you will see lots of TTL exceeded responses from routers. This is how tracroute works: forces the routers to generate TTL exceeded messages in order to find them.

Another common reason firewall administrators see this is due to routing loops developing in the Internet. Route flapping (constant route changes) is a common problem, and will often briefly result in a loop. This means that while a IP packet is heading towards it destination, the packet gets misrouted to a router that it previously visited it. The packet then gets routed in a circle infinately -- or it would be, if the routers didn't decrement the TTL field each time and discard the packet once that value hit zero.

Another cause of this is distance. Many machines start with a default TTL of 127 (Windows) or even lower. Routers will often decrement the TTL more than by one in order to reflect slow lines like dialups or transcontinental links. Therefore, a site might not be reachable with a low initial TTL. In addition, some hackers/crackers like to make their site unreachable through this method.

2.11.1 Type = 11, Code = 1 (Fragment Reassembly Time Exceeded)

When sending fragmented IP datagrams, the sender of this message never received all the fragments. Normally, most TCP/IP traffic shouldn't even be fragmented. You will only see this if the traffic is both fragmented AND there congestion somewhere between you and the target.

2.12 Type = 12 (Parameter Problem)

This probably indicates an attack. There are a number of fingerprinting techniques that will generate these packets.

3. IP

3.1 What are source routed packets?

Source route is an option in the IP header that allows the sender to override some or all of the routing decisions. Normally, routers between the source and destination decide how to route the packet.

There are a couple of network management uses of this packet, such as testing to see if two computers can talk to each other. A network manager at point A may send a packet to B through point C. This tells A if B & C can talk to each other.

The same technique can be used to evade firewalls, subvert trust relationships, and communicate with machines using "private" address (10.x.x.x, 192.168.x.x, 172.[16-31].x.x).

Let's say you are a hacker/cracker on the Internet and you want to talk to some machines behind a firewall who use 10.x.x.x as their IP addresses. Since the routers on the Internet do not know where this subnet is located, they will drop your packets. However, you put a loose source route option in the IP packet and tell all the Internet routers to first forward to the firewall. Since the firewall straddles both the Internet and the private network, it will know how to forward the packet appropriately. Thus, you can carry on a conversation with the victim by bouncing all packets through the firewall.

This can be used with IP spoofing. You pretend to be a router (like the firewall mentioned above) and pretend that somebody else is bouncing packets through you. Thus, pick some random machine on the Internet (ALICE) as the spoofee, then send packets from ALICE to your victim BOB. BOB will think the packets are coming from ALICE, but in reality they are coming from you. This masks the real source of the attack.

This is even better if you know that BOB trusts ALICE. IP addresses are often used as part of authentication. Let's say the firewall has a rule allowing all traffic from ALICE into the network. By forging all IP packets to be from ALICE (but being source routed through your own machine), then you get free access to the victim network.

More and more core Internet routers are disabling source routed packets. They slow down routing anyway, but they are a huge security risk. There is also no real need for them. Managers should do the same and disable source routing everywhere: on firewalls, on routers, and even on end-nodes so that they won't even accept incoming source routed packets.

3.2 I'm seeing the IP address 255.255.255.255 in my reject log

This is happening a lot these days as more and more people use DSL or cable-modem connections. The reason is that unlike point-to-point connections (like T-1, frame relay, etc.), these new high speed technologies drop you onto an ATM VLAN, which is a single broadcast domains. In fact, many cable-modem users are seeing multiple megabytes of traffic per day simply from such broadcasts.

You must remember that such packets MUST be local. Routers (generally) refuse to forward packets with the IP address of 255.255.255.255. This address is known as a "local broadcast" for this reason: it never travels past the local segment (or these days, the local "virtual" segment).

This is rarely something to be concerned about, though, because usually it advertises something about the person sending the traffic that can be used to hack them. It is only rarely a packet that is trying to scan you for information.

It should be noted that with todays ATM networks, the source of the broadcast may not even be in the same state as you are; they may be hundreds of miles away. The word "local" means in terms of the network topology, not distance.

3.3 How do I track down the owner of an IP address?

Remember that IP addresses can be spoofed, so that the "owner" of an IP address may be innocent. Increasingly, attacks are coming from compromised machines. The owner of the IP may actually be grateful! Both of these statements come to the same conclusion: be polite and professional.

Many companies have established the e-mail address "abuse@example.com" (replace "example" with the proper company). This e-mail role is for both e-mail abuse (such as spam) as well as for network abuse. When you find the owner of the IP address, you should probably compose a message including the evidence of the attack.

Registrar Databases

In the past, all the IP address owners were kept by the Internic. A database built from that information is at http://ipindex.dragonstar.net/. There are now 3 official registrars for North America, Asia, and Europe. Unfortunately, you will have to query each individual database. However, if you start with the North America registrar, it will tell you if the address belongs to one of the other three. The three registrars are:

North America
ARIN (American Registry for Internet Numbers)

http://www.arin.net/whois/

Europe
RIPE (Reseaux IP Europeens)

http://www.ripe.net/db/whois.html

Asia and Pacific
APNIC (Asia Pacific Network Information Centre)

http://www.apnic.net/apnic-bin/whois.pl

traceroute

Running traceroute will often find at least the ISP who is hosting the IP address. A reverse DNS lookup on the actual IP address is easy to spoof, but the route to the machine will reveal who is hosting the possible intruder.

Common IP addresses

Many attacks are now coming from cable-modem subscribers in the 24.x.x.x range. These are probably from machines who have been compromised by a Remote Access Trojan (RAT). (While hackers/crackers frequently use dial-up lines because they don't care if their account gets canceled, few users want to have their cable-modem accounts canceled).

Another important range is the "private address" ranges of 10.x.x.x, 192.168.x.x, and 172.16.x.x-172.31.x.x.

4. Stuff doesn't work

4.1 Installing a firewall causes slow connections to POP and SMTP services

This is because the POP and SMTP servers are trying to establish an identd/AUTH connection back to the client. These reverse-connections are blocked, and it takes a while before the servers timeout and continue.

The identd/AUTH service identifies the user of the TCP connection (user name, process id, etc.). When the e-mail server accepts the incoming TCP connection, before sending the greetings, it will first attempt to gather information via the identd protocol. This consists of a TCP connection in the reverse direction. In other words, when I connect to my e-mail server, my e-mail server attempts to connect back to me on port 113, the identd port. My e-mail connection just sits there until the e-mail server resolves the identd information.

The problem comes about because the firewall silently drops the SYN packet. The e-mail server is expecting an immediate SYN-ACK (identd supported) or RST (identd not supported), but when the firewall drops the packet it keeps trying until the connection times out.

Note that the e-mail server doesn't care if I don't support identd, and indeed most people don't on their clients. It just wants an immediate response one way or the other. The firewall blocks that. This is why some personal firewalls for Windows (like BlackICE Defender from my company) contain default rules that allow identd/AUTH to pass through. Windows doesn't reveal the information that UNIX does, and opening it up gives the immediate response these servers are looking for.

To solve this problem:

  • reconfigure the e-mail server to stop querying identd info
  • reconfigure the firewall to RST all those connections
  • reconfigure the firewall to allow this protocol, but this would be a BAD IDEA because identd/AUTH reveals a HUGE amount of information about your UNIX machines.

Note that this means you should be seeing lots of dropped incoming connection attempts at port 113 in your log files because of this.

5. Programs

5.1 traceroute

The program "traceroute" is based upon a very intelligent hack by Van Jacobson (also famous for other nifty kludges). Every IP packet has a time-to-live (TTL) field that indicates how many hops the packet can travel before being dropped. This field is needed because routers sometimes get misconfigured and will forward packets in a continuous: i.e. Alice fowards the packet to Bob who fowards it to Charlene who mistakenly forwards it back to Alice.

Therefore, each router decrements (subtracts 1) from the TTL field. When each reaches zero, the router who currently has the packet will simply "drop" it (not forward it on). When a router drops a packet, it sends a message back to the sender informing for this. This message is called an ICMP "TLL Exceeded in Transit".

The nifty thing about this is that the router uses its own IP address as the source address of the ICMP message. Therefore, if you send a packet to a target but with a TTL of only 1, the first router will receive the packet, decrement the field to 0, drop it, then send back the ICMP notification. This informs you of the first router along the route (which you probably knew anyway).

The same goes for an initial TTL of 2. The first router gets it, decrements to 1, then forwards to the second router along the route. This router then decrements to 0, drops the packet, and sends back and error ICMP message.

By continuing this process, you eventually end up with the list of routers between yourself and the target.

Versions of traceroute

There are various versions of the traceroute program. In particular, the Windows program "tracert.exe" uses pings as the packet it sends to the target. Therefore, you might see ICMP Echoes on your firewall.

The most popular "traceroute" program for UNIX programs sends UDP datagrams to port 33434 for the first packet sent, then increases this port number by one for each successive packet. This means that you will never see port 33434 on your firewall, but you will start to see successive ones starting at higher port numbers. Traceroute programs typically send 3 packets for each hop (in case some get dropped). Therfore, if somebody is 10 hops away, the first port you will see is 33434 + 3*10 = 33464.

Symptoms

Firewall administrators should learn the symptoms of traceroute activity.

port scans in 33434-33600
A brief sequential "port scan" in this range usually indicates a traceroute for a UNIX machine, as explained in this section.
incoming TTL exceeded
If someone inside the network is attempting a traceroute, then you'll see these incoming packets. Many admins allow these through the firewall.
outgoing TTL exceeded
This indicates that somebody is tracerouting you. This doesn't necessarily indicate hostile activtiy, but somebody is scanning you. These should be blocked by the firewall.
outgoing ICMP port unreachable
When a traceroute successfully hits a target, it will generate back-to-back "ICMP port unreachable" messages (probably 3 in a row).

Other

Some traceroutes are designed to bypass firewalls. See http://www.packetfactory.net/firewalk/firewalk-final.html for more information.

5.2 sscan

The 'sscan' tool has become a popular scanning tool on the Internet. It not only "port scans" but attempts to discover some common vulnerabilities. There are several versions of sscan, and it is very configurable, so matching an exact fingerprint to this program may be difficult. The 'sscan' program is derived from the older 'mscan' tool.

A sscan goes through several phases:

TCP ACK pings
The program will attempt to see if the host is reachable by scanning for the most common services, namely ports 23/telnet, 25/smtp, 110/pop3, 143/imap4, 80/http. This phase is easily detected because both the source and destination port are the same.
connection attempts
Connection attempts are made to several services in order to see if they are available. This is highly configurable. Typically configured probes are those above, as well as 111/rpc, 6000/x-windows, 79/finger, 53/dns, 31337/elite, 139/netbios,smb, 21/ftp, 1114/msql, 1/tcpmux
OS fingerprint
sscan contains a basic OS fingerprinting technique, easily detected because it uses source ports 1-5. The fingerprinting is not as complete as the techniques used by Queso or nmap.
vulnerability assessment
It then looks at the ports that are open and checks the banners that might indicate a vulnerable version of one of the services. It also scans for a range of known vulnerable CGI scripts.
script execution
Depending upon what it finds, it can further launch configured scripts against the system.

Example

The following is a record pulled from an intrusion detection system.

ports=1 22 23 25 53 79 110 111 143 1114 2766 6000 31337

Unfortunately, the system consolidates alerts, discards duplicates, and keeps the port numbers in sort order. In a real scan, several of the ports would have duplicate connection attempts, and port 1/tcpmux would be one of the last probes, not one of the first.

More info

See CERT: Incident Note IN-99-01 (http://www.cert.org/incident_notes/IN-99-01.html) for more information about sscan.

[fin] Firewall Seen FAQ 1
Site hosted by Angelfire.com: Build your free website today!