Capturing TCP packets with particular flag combinations (SYN-ACK, URG-ACK, etc.)

There are 8 bits in the control bits section of the TCP header:

CWR | ECE | URG | ACK | PSH | RST | SYN | FIN

Let's assume that we want to watch packets used in establishinga TCP connection.Recall that TCP uses a 3-way handshake protocolwhen it initializes a new connection; the connection sequence withregard to the TCP control bits is

1) Caller sends SYN

2) Recipient responds with SYN, ACK

3) Caller sends ACK

Now we're interested in capturing packets that have only theSYN bit set (Step 1).Note that we don't want packets from step 2(SYN-ACK), just a plain initial SYN.What we need is a correct filterexpression fortcpdump.

Recall the structure of a TCP header without options:

 0                            15                              31
-----------------------------------------------------------------
|          source port          |       destination port        |
-----------------------------------------------------------------
|                        sequence number                        |
-----------------------------------------------------------------
|                     acknowledgment number                     |
-----------------------------------------------------------------
|  HL   | rsvd  |C|E|U|A|P|R|S|F|        window size            |
-----------------------------------------------------------------
|         TCP checksum          |       urgent pointer          |
-----------------------------------------------------------------

A TCP header usually holds 20 octets of data, unless options arepresent.The first line of the graph contains octets 0 - 3, thesecond line shows octets 4 - 7 etc.

Starting to count with 0, the relevant TCP control bits are containedin octet 13:

 0             7|             15|             23|             31
----------------|---------------|---------------|----------------
|  HL   | rsvd  |C|E|U|A|P|R|S|F|        window size            |
----------------|---------------|---------------|----------------
|               |  13th octet   |               |               |

Let's have a closer look at octet no. 13:

                |               |
                |---------------|
                |C|E|U|A|P|R|S|F|
                |---------------|
                |7   5   3     0|

These are the TCP control bits we are interestedin.We have numbered the bits in this octet from 0 to 7, right toleft, so the PSH bit is bit number 3, while the URG bit is number 5.

Recall that we want to capture packets with only SYN set.Let's see what happens to octet 13 if a TCP datagram arriveswith the SYN bit set in its header:

                |C|E|U|A|P|R|S|F|
                |---------------|
                |0 0 0 0 0 0 1 0|
                |---------------|
                |7 6 5 4 3 2 1 0|

Looking at thecontrol bits section we see that only bit number 1 (SYN) is set.

Assuming that octet number 13 is an 8-bit unsigned integer innetwork byte order, the binary value of this octet is

00000010

and its decimal representation is

   7     6     5     4     3     2     1     0
0*2 + 0*2 + 0*2 + 0*2 + 0*2 + 0*2 + 1*2 + 0*2  =  2

We're almost done, because now we know that if only SYN is set,the value of the 13th octet in the TCP header, when interpretedas a 8-bit unsigned integer in network byte order, must be exactly 2.

This relationship can be expressed as

tcp[13] == 2

We can use this expression as the filter for tcpdump in orderto watch packets which have only SYN set:

tcpdump -i xl0 tcp[13] == 2

The expression says "let the 13th octet of a TCP datagram havethe decimal value 2", which is exactly what we want.

Now, let's assume that we need to capture SYN packets, but wedon't care if ACK or any other TCP control bit is set at thesame time.Let's see what happens to octet 13 when a TCP datagramwith SYN-ACK set arrives:

     |C|E|U|A|P|R|S|F|
     |---------------|
     |0 0 0 1 0 0 1 0|
     |---------------|
     |7 6 5 4 3 2 1 0|

Now bits 1 and 4 are set in the 13th octet.The binary value ofoctet 13 is

     00010010

which translates to decimal

   7     6     5     4     3     2     1     0
0*2 + 0*2 + 0*2 + 1*2 + 0*2 + 0*2 + 1*2 + 0*2   = 18

Now we can't just use 'tcp[13] == 18' in the tcpdump filterexpression, because that would select only those packets that haveSYN-ACK set, but not those with only SYN set.Remember that we don't careif ACK or any other control bit is set as long as SYN is set.

In order to achieve our goal, we need to logically AND thebinary value of octet 13 with some other value to preservethe SYN bit.We know that we want SYN to be set in any case,so we'll logically AND the value in the 13th octet withthe binary value of a SYN:

          00010010 SYN-ACK              00000010 SYN
     AND  00000010 (we want SYN)   AND  00000010 (we want SYN)
          --------                      --------
     =    00000010                 =    00000010

We see that this AND operation delivers the same resultregardless whether ACK or another TCP control bit is set.The decimal representation of the AND value as well asthe result of this operation is 2 (binary 00000010),so we know that for packets with SYN set the followingrelation must hold true:

( ( value of octet 13 ) AND ( 2 ) ) == ( 2 )

This points us to the tcpdump filter expression

     tcpdump -i xl0 'tcp[13] & 2 == 2'

Note that you should use single quotes or a backslashin the expression to hide the AND ('&') special characterfrom the shell.