P4 - Packet Capture and Analysis

Week 13 · 100 points · about 8 hours · submit in Canvas

Overview

You have spent the semester reading about headers. Now you are going to parse real ones. Given a packet capture, your program walks each frame down the stack: Ethernet, then ARP or IPv4, then TCP or UDP, and reports what it finds. The main capture is the "day in the life of a web request" from section 6.7 of the textbook, with the book's own IP addresses: a laptop joins the network with DHCP, ARPs for its gateway, looks up www.google.com, and fetches a page over HTTP. This time you pull it out of the bytes yourself.

Then you open the same capture in Wireshark and check your numbers against it. Any disagreement is a bug in your parser, and finding it is the point of the assignment.

Fork the starter repository using the Use this template button and name your copy cs425-p4.

DANGER

Your code must compile on GitHub Codespaces and Onyx. If it compiles on only one of them you will receive a zero even if it works on the other.

Learning Outcomes

• 5.2 Trace an end-to-end request through ARP, DHCP, switching, and routing
• 1.1 Explain encapsulation and how a message is transformed as it moves down and up the protocol stack
• 7.2 Produce code free of memory leaks and out-of-bounds accesses

Background

The captures

Download p4-captures.zip and unzip it in the root of your repository. It holds five captures and one expected output:

File	What is in it
day-in-the-life.pcap	24 frames: DHCP, an IPv6 frame, ARP, DNS, an HTTP fetch with one corrupted segment and its retransmission, and one traceroute probe with the router's ICMP time exceeded reply
day-in-the-life-be.pcap	The same 24 frames, written by a big endian machine
snaplen.pcap	The same 24 frames, captured with a snaplen of 96, so long frames were cut short
truncated.pcap	day-in-the-life.pcap with the file cut off in the middle of frame 13
malformed.pcap	23 frames, most of them broken in one specific way
malformed.txt	What your program must print for malformed.pcap

The capture was taken on the laptop. That matters in one place: frames the laptop receives arrive padded to the 60 byte Ethernet minimum, and frames it sends are captured before the network card pads them.

The pcap format

A pcap file is a 24-byte file header followed by one record per captured frame. Every record is a 16 byte record header followed by the frame's bytes.

Offset	Size	File header field
0	4	Magic number, 0xa1b2c3d4
4	2	Major version, 2
6	2	Minor version, 4
8	4	Time zone offset, always 0
12	4	Timestamp accuracy, always 0
16	4	Snaplen, the most bytes kept of any one frame
20	4	Link type, 1 for Ethernet

Offset	Size	Record header field
0	4	Timestamp, seconds since the epoch
4	4	Timestamp, microseconds
8	4	Captured length, the number of frame bytes that follow
12	4	Original length, the frame's length on the wire

The pcap headers are in the byte order of the machine that wrote the file, and the magic number is how you tell. Read the first four bytes as little endian: if you get 0xa1b2c3d4 the file is little endian. Read them as big endian: if you get 0xa1b2c3d4 it is big endian. Anything else is not a pcap file. The frames themselves are always in network order (big endian), whoever wrote the file.

Do not read the pcap headers by casting a pointer to a struct, and do not use ntohs on them. Build every value a byte at a time with shifts. That way the byte order is a decision you make and not an accident of the machine you happen to be on, and you never read an unaligned field.

The captured length is less than the original length when the capture was taken with a small snaplen (tcpdump -s 96). That frame is not broken, the capture just did not keep all of it. Your program decodes the headers it has and does not pretend to have the rest.

TIP

Wireshark saves in pcapng by default, which is a different format. If you save a capture to test with, pick Wireshark/tcpdump/... - pcap in the Save As dialog. A pcapng file starts with the bytes 0a 0d 0d 0a.

Task 1 - Complete the program

Your program is built as ./build/release/myapp and must take this command line:

Usage: myapp [-s] <capture>

  -s         print only the flow summary
  <capture>  a pcap file of Ethernet frames

Use getopt for the parsing. The output format below is fixed because your program is graded by running it on the captures and comparing what it prints against the expected output, character for character. Match it exactly.

One line per frame

For every frame, print the frame number (starting at 1), the time in seconds since the first frame with exactly six decimal places, and a description:

7 0.021101 ARP reply 68.85.2.1 is-at 00:22:5f:4d:5e:6f

The description is one of these:

ARP request who-has <target IP> tell <sender IP>
ARP reply <sender IP> is-at <sender MAC>
ARP op=<opcode>
TCP <src IP>:<src port> > <dst IP>:<dst port> [<flags>] seq=<n> ack=<n> win=<n> len=<n> cksum=<status>
UDP <src IP>:<src port> > <dst IP>:<dst port> len=<n> cksum=<status>
IPv4 <src IP> > <dst IP> proto=<n>
IPv4 <src IP> > <dst IP> proto=<n> fragment
IPv4 <src IP> > <dst IP> proto=<n> bad-header-cksum
Ethernet type=0x<4 lowercase hex digits>
malformed Ethernet
malformed ARP
malformed IPv4
malformed TCP
malformed UDP

Here is a stretch of the HTTP exchange from day-in-the-life.pcap so you can see the formatting:

10 0.040210 TCP 68.85.2.101:49152 > 64.233.169.105:80 [S] seq=3127851020 ack=0 win=64240 len=0 cksum=ok
11 0.068433 TCP 64.233.169.105:80 > 68.85.2.101:49152 [S.] seq=2215467233 ack=3127851021 win=1050 len=0 cksum=ok
12 0.068512 TCP 68.85.2.101:49152 > 64.233.169.105:80 [.] seq=3127851021 ack=2215467234 win=502 len=0 cksum=ok
13 0.068790 TCP 68.85.2.101:49152 > 64.233.169.105:80 [P.] seq=3127851021 ack=2215467234 win=502 len=77 cksum=ok

The details:

• MAC addresses are six lowercase hex pairs separated by colons. IP addresses are dotted quads. Every number is decimal unless the format says otherwise.
• seq and ack are the raw 32-bit values from the header, not the relative numbers Wireshark shows by default. win is the raw window field, unscaled.
• TCP flags are the letters F S R P U E W (FIN, SYN, RST, PSH, URG, ECE, CWR), in that order, for each flag that is set, followed by a . if ACK is set. A segment with no flags at all prints none.
• len is the number of payload bytes according to the headers: for TCP the IPv4 total length minus the IPv4 header length minus the TCP header length, and for UDP the UDP length minus 8. Never work it out from the frame length. A frame padded to 60 bytes does not have 6 bytes of payload.
• cksum is ok or bad after you verify the TCP or UDP checksum over the pseudo header and the whole segment. It is none for a UDP datagram whose checksum field is zero, which means the sender did not compute one. It is partial when the capture did not keep the whole segment, since you cannot check bytes you do not have.

What counts as malformed

A frame is never a reason to stop. A frame that is broken gets a malformed line and your program moves on to the next one. Check in this order, and report the first problem you find:

1. Ethernet. Fewer than 14 bytes. Otherwise dispatch on the EtherType: 0x0806 is ARP, 0x0800 is IPv4, and anything else gets an Ethernet type= line.
2. ARP. Fewer than 28 bytes, or not IPv4 over Ethernet (hardware type 1, protocol type 0x0800, address lengths 6 and 4).
3. IPv4. Fewer than 20 bytes, a version other than 4, an IHL under 5, an IHL that runs past the captured bytes, or a total length shorter than the header. A total length longer than the captured bytes is malformed too, unless the frame was cut short by the snaplen (captured length less than original length).
4. IPv4 header checksum. If it does not verify, print the bad-header-cksum line and do not decode any further.
5. Fragments. If the more fragments flag is set or the fragment offset is not zero, print the fragment line. Only the first fragment has a transport header, and reassembly is not part of this project.
6. Protocol. 6 is TCP, 17 is UDP, and anything else gets an IPv4 ... proto= line. The transport header starts IHL × 4 bytes into the datagram, not 20.
7. TCP. Fewer than 20 bytes captured, a data offset under 5, or a header that runs past the captured bytes.
8. UDP. Fewer than 8 bytes captured, a UDP length under 8, or a UDP length longer than the IPv4 payload.

Bytes past the end of the IPv4 total length are Ethernet padding. Ignore them.

The flow summary

After the last frame print a blank line, then one line for each TCP or UDP flow, in the order the flows were first seen:

TCP 68.85.2.101:49152 <> 64.233.169.105:80 packets=13 bytes=4403

A flow is the conversation between two endpoints over one protocol, in both directions, so A:x > B:y and B:y > A:x are the same flow. Side A is the source of the first packet of the flow you saw. packets counts every frame that got a TCP or UDP line, including ones with a bad or partial checksum, and bytes is the sum of their original lengths. Those are the numbers Wireshark's Conversations window shows, which is what makes Task 4 possible.

With -s, print only the flow lines: no frame lines and no blank line.

Exit status

• Exit 0 when the whole capture was read.
• Exit 1 when the command line is wrong.
• Exit 2 when the file cannot be read or is not a well formed pcap file. Print every frame up to the problem, then a message on stderr that says what was wrong and at which frame, and skip the summary.
• Print the usage message and exit 0 when run with no arguments at all. This is what make leak runs, so it has to be a clean, successful path.

The file is not well formed when it is shorter than the 24 byte header, the magic number is wrong in both byte orders, the major version is not 2, the link type is not 1, a record header or a record's data runs past the end of the file, or a record claims to have captured more bytes than were on the wire. A pcapng file should get a message that says it is pcapng, because that is the mistake people actually make. Truncated files are common in real life (the disk filled up, or someone hit Ctrl-C), and a parser that trusts the lengths it reads is how a capture file becomes an exploit, so never read a byte you have not checked is there.

Task 2 - Design for testability

Every function that parses bytes must take a pointer and a length and must not look past that length. Split src/lab.h into three layers:

1. The pcap reader. Opens a capture that is already in memory and hands out one record at a time. It never touches a file, so a test can give it any bytes it likes, including a file header cut off after 7 bytes.
2. The header parsers. One function each for Ethernet, ARP, IPv4, TCP and UDP that takes a pointer and a length and fills in a struct, or reports that the header does not fit or is not legal. Plus the Internet checksum, and the formatting of addresses and flags. No I/O and no allocation in here.
3. Decoding and reporting. A function that walks one frame down the stack using the parsers and produces its line, a flow table, and a function that runs a whole capture and writes the report to a FILE *.

main.c reads the file into memory, parses the command line, and calls layer 3. That is all it does.

The payoff is that every malformed header in the rubric is a unit test you write with a byte array, and you can test the report itself by pointing it at a tmpfile() and reading back what it wrote.

Task 3 - Testing

Add Unity tests for every function you declare in src/lab.h.

make check

Beyond the happy path, make sure you have tests for:

• A pcap file in each byte order, and a file cut off inside the file header, inside a record header, and inside a record's data
• Each of the malformed cases in Task 1, one byte array each
• An IPv4 header with options, so the transport header is not at byte 20
• A frame with Ethernet padding, and a frame cut short by the snaplen
• A checksum over an odd number of bytes, and one whose sum carries more than once
• A flow seen in both directions

Then run your program on all five captures. malformed.pcap has to match malformed.txt exactly:

./build/release/myapp malformed.pcap | diff - malformed.txt

Task 4 - Cross-check against Wireshark

Install Wireshark on your own machine (it is free) and open day-in-the-life.pcap. Wireshark is a GUI, so this part does not happen in Codespaces or on Onyx. Download the capture to your laptop.

1. Open Statistics → Conversations, and look at the TCP and UDP tabs. Compare the Packets and Bytes columns against your flow summary.
2. Wireshark does not check TCP and UDP checksums unless you ask it to. Turn on Edit → Preferences → Protocols → TCP → Validate the TCP checksum if possible (on macOS, Preferences is under the Wireshark menu), and the same setting under UDP, then find the frame Wireshark flags.
3. Wireshark shows relative sequence numbers. Turn off Relative sequence numbers in the same TCP preferences, or look at the Sequence Number (raw) field, and compare a few frames against your output.
4. Open snaplen.pcap and look at what Wireshark says about frame 15.

Put the results in your README (see Task 7). If your numbers and Wireshark's disagree, fix your parser, and say in the README what the bug was. That paragraph is worth more to me than a table that matched the first time.

TIP

You can capture your own traffic in Codespaces, which runs Linux and gives you sudo:

sudo apt-get update && sudo apt-get install -y tcpdump
sudo tcpdump -i eth0 -w mine.pcap port 80 &
curl -s http://example.com > /dev/null
sudo pkill tcpdump
./build/release/myapp mine.pcap

Expect the segments your machine sent to show cksum=bad. That is not a bug in your parser. The network card computes the checksum after the capture has already copied the segment (checksum offload), so the capture sees a placeholder. It is a good thing to know before you spend an hour chasing it.

Task 5 - Coverage

make clean
make all
make report

Fix your tests until you have 100% coverage with everything passing. As in P0, you may only exclude branches originating from system or library calls.

Task 6 - Leak and crash check

• Run make leak, then make leak-test.
• Fix every leak and every crash. Address Sanitizer reports a read past the end of a buffer as a crash, which is exactly the bug a malformed frame is designed to find.

Task 7 - Replace README.md

Replace README.md following this example. Your Design section should explain the three layers from Task 2. Add a Wireshark section with:

1. A table with one row per flow: your packets and bytes next to Wireshark's.
2. Any disagreement you found, what caused it, and how you fixed it.
3. Short answers to these, using frame numbers from your output:
   • Why do the first DHCP messages come from 0.0.0.0 and go to 255.255.255.255?
   • The ARP request is 42 bytes and the reply is 60. Why?
   • Which frame has a bad checksum, and what did TCP do about it? Name the frames.
   • What is frame 24, and what made the router send it?

Task 8 - Continuous integration

• Push everything to GitHub and confirm the CI run is green.
• Run the Create Submission Report Via GitHub Action workflow.
• Download submission-report.docx once it completes.

Submitting

1. Download submission-report.docx from GitHub and submit it to Canvas.
2. Check your submission in Canvas so you know it arrived intact.

Rubric

#	Criterion	Points
1	pcap files read in both byte orders, with truncated and malformed files rejected with exit 2 and no read past the buffer	15
2	Ethernet decoded and dispatched on EtherType, and ARP requests and replies reported	10
3	IPv4 decoded, including options, the header checksum, fragments, and the total length rather than the frame length	15
4	TCP and UDP decoded, flags formatted, and checksums verified over the pseudo header	20
5	Flow summary correct, with the Wireshark section of the README showing it agrees	15
6	Parsing separated from file I/O, with every parser bounded by a length	5
7	Unity tests cover every function in lab.h with 100% coverage	10
8	No memory leaks or crashes under make leak-test; README.md replaced; CI green on the last push	10