Basic concepts on the architecture

Winpcap is an architecture that adds to the operating systems of the Win32 family the ability to capture the data of a network using the network adapter of the machine. Moreover, it provides to the applications a high level API that makes simpler the use of its low-level capabilities. It is subdivided into three separate components: a packet

capture device driver, a low-level dynamic library and a high level static library. This architecture can be used to create new capture tools for Windows, but also to make the porting of UNIX applications, since it is compatible with the BPF-libpcap architecture. In this chapter I will describe the structure and the basic principle of the architecture,

and I will explain some concepts that will be useful in the rest of the book.

Note: I will use the term packet even though frame is more accurate, since the cap-ture process is done at the data-link layer and the data-link header is included in the frames received.

Structure of the capture stack

To capture packets transferred by a network, a capture application needs to interact directly with the network hardware. For this reason the operating system should offer a set of capture primitives to communicate and receive data directly from the network adapter. Goal of these primitives is basically to capture the packets from the network (hiding the interaction with the network adapter), and transfer them to the calling pro-grams. This is heavily system dependent so the implementation is quite different in the various operating systems. The packet capture section of the kernel should be quick and efficient because it must be able to capture packets also on high-speed LANs with heavy traffic, limiting losses of packets and using a small amount of sys-tem resources. It should also be general and flexible in order to be used by different types of applications (analyzers, network monitors, network test applications...). The user-level capture application receives packets from the system, interprets and processes them, and outputs them to the user in a comprehensible and productive way. It should be easy to use, system independent, modular and expandable, so to support the higher number of protocols. Moreover is should allow to increase the number of decoded protocols in a simple way. These features are essential because of the high number of network protocols currently available and the speed they change, so com-pleteness and expandability are important. WinDump.exe is only the highest part of a packet capture stack that is composed by a module that runs at the kernel level and one that runs at user level. These two mod-ules have different purposes and are independent and isolated one from another. The first runs at ring 0 on Intel based machines, while the second runs at ring 3 like a nor-mal Windows program. The kernel part is Windows specific and it is very different according to various Windows flavors. The user-level part is very similar to the UNIX

implementation and it is the same under Win95 and WinNT. The structure of the cap-ture stack from the network   dapter to an application like WinDump is shown in the At the lowest level there is the network adapter. It is used to capture the packets that circulate in the network. During a capture the network adapter usually works in a

particular mode (‘promiscuous mode’) that forces it to accept all the packets instead of the ones directed to it only.

Packet Capture Driver is the lowest level software module of the capture stack. It is the part that works at kernel level and interacts with the network adapter to obtain the packets. It supplies the applications a set of functions used to read and write data from the network at data-link level. next figure.

User level: WinDump program and the libpcap library

The WinDump program is the higher part of the capture stack and manages the in-teraction between the user and the system. It gets the user’s inputs (for example, which packets must be captured and in which way they must be showed to the user) from the command line and outputs the results on the screen. The WinDump executa-ble running on a Windows platform is identical to the TcpDump executable running on a UNIX workstation from the user viewpoint. This means that the two programs have almost the same input parameters and the same output format. TcpDump on UNIX makes use of a library for the packet capture process, called Packet Capture library, or pcap library, or libpcap, a system-independent interface for user-level packet capture. Libpcap provides a set of functions independent from the

hardware and the operating system that an application can use to capture packets from a network. It is important to remember that original libpcap is a general-purpose cap-ture library, and is used by TcpDump, as well as several other network tools and ap-plications. TcpDump does not interact with hardware directly, but uses the functions exported by libpcap to capture packets, set packet filters and communicate with the network adapter. Basically it is system-independent and can easily be ported to any system on which the libpcap library is available. For this reason the WinDump project includes a full porting of the libpcap library toward the Win32 platform. Furthermore,

the pcap library is not limited to be used with WinDump, and we think that it can be very useful to people that want to convert network monitors and analyzers from the UNIX world, and it can be also a powerful base to create new network tools for the Win32 environment. Therefore, we tried to maintain the structure of the original ver-sion

in our implementation. We modified only the section of libpcap that communi-cates with the kernel, implementing the interaction with the NDIS packet capture driver. An important characteristic of libpcap for Win32 is that there is only a version that works both on Windows 95, 98, NT and 2000. This can be obtained by putting a dynamic link library, called PACKET.DLL, between libpcap and the capture driver, so that the system calls to the driver are the same in the various Windows environ-ments. In this way a network tool based on libpcap will work on Windows 95 and

Windows NT without any modifications. The porting of TcpDump, once a working implementation of libpcap for Win32 was ready, was not too difficult. Only few improvements were introduced in Win-Dump, to handle the difference in the names of the network adapters and the dynamic kernel buffer.

Kernel level: the NDIS packet capture driver

The basic role of the kernel part of the capture stack is to take the link-layer pack-ets from the network and to transfer them to the application level without modifications. We implemented it as a kernel driver (packet.sys) under Windows NT and as a Virtual Device Driver (packet.vxd) under Windows 95. Applications can have access to the capture driver with read and write primitives and can consider the network adapter somehow similar to a normal file, reading or writing the data that comes from the network. The capture driver can be used by any Win32 application that needs to

capture packets, and is not limited to the use with WinDump or Analyzer. The interac-tion with the packet capture driver usually passes through the PACKET.DLL dynamic link library. The DLL implements a set of functions that make simpler the communi-cation with the driver, avoiding the use of things such system calls or IOCTLs.

The packet capture driver interacts with the network adapter’s device drivers through NDIS, that is a part of the network code of Win32. NDIS is responsible of the management of the various network adapters and of the communication between the adapters and the software portions that implement the protocols. A basic network capture driver can be quite simple. It needs only to read the pack-ets from the network driver and copy them to the application. However, in order to obtain acceptable performances, substantial improvements need to be done to this ba-sic

structure. The most important are:

 To limit packet loss, the driver should be able to store the incoming packets in a buffer because the user-level application could not be ready to process them at their arrival. Buffered packets will be transferred as soon as the application will be ready.

 In order to minimize the number of context switch between the application (that runs in user mode) and the driver (that runs in kernel mode), it should be possible to transfer several packets from the buffer to the application using a single read call.

 The user-level application must receive only the packets it is interested in, usually a subset of the whole network traffic. An application must be able to specify the type of packets it wants (for example the packets generated by a

particular host) and the driver will send to it only these packets. In other words the application must be able to set a filter on the incoming packets, receiving only the subset of them that satisfy the filter. A packet filter is simply function

with a boolean returned value applied on a packet. If the returned value is TRUE, the driver copies the packet to the application, otherwise the packet is ignored.

The implementation of these features and the architecture of the driver were in-spired

by the BSD Packet Filter (BPF) of the UNIX kernel, of which we make a brief

description in the next paragraph.

The packet filter

The filter decides if a packet should be accepted and copied to the listening appli-cation. Most applications using BPF reject far more packets than those accepted, therefore good performance of the packet filter is critical for a good over-all perform-ance. The packet filter is a function with boolean output that is applied to a packet. If the value of the function is true the kernel copies the packet to the application; if it is false the packet is ignored. BPF packet filter is quite more complex, because it deter-mines not only if the packet should be kept, but also the number of bytes to keep. This feature is very useful if the capture application does not need the whole packet. For example, a capture application is often interested only in the headers and not in the data of a packet. Such an application can set a filter that accepts only the firsts bytes of the packet so that only the headers are copied. This action speeds up the capture

process and decrease the loss probability because it decreases the number of byte to copy from the driver to the application and decreases the space needed to store the packet into the buffer. Historically there have been two approaches to the filter abstraction: a boolean ex-pression tree and a directed acyclic control flow graph or CFG. More details about them can be found in [1]. These two models of filtering are computationally equiva-lent,

i.e., any filter that can be expressed in one can be expressed in the other. How-ever implementation is very different: the tree model can be easily mapped into code for a stack machine while the CFG model can be mapped into a register machine code. BPF creators choose CFG because it can be implemented more efficiently on modern computers that are register based. The BPF pseudo-machine is a virtual proc-essor, and its abstraction consists of an accumulator, an index register (x), a scratch memory store, and an implicit program counter. It is able to execute load and store

instructions, branches, arithmetic instructions and so on. Therefore, a UNIX applica-tion that wants to set a filter on the incoming packets has to build a program for the pseudo-machine and send it to BPF through an IOCTL call. BPF executes the filter program on each packet and discards the ones that do not satisfy the filter. The BPF

pseudo-machine has some nice features:

 It is protocol independent. This means that the kernel has not to be modified to

add new protocol support.

 It is general enough to handle unforeseen uses.

 It is optimized for speed, to have good performances.

 

Interaction with NDIS

NDIS (Network Driver Interface Specification) is a set of specifics that defines the communication between a network adapter (or, better, the driver that manages it) and the protocol drivers (IP, IPX...). Main NDIS purpose is to act as a wrapper that allows protocol drivers to send and receive packets onto a network (LAN or WAN) without

caring either the particular adapter or the particular Win32 operating system. NDIS supports three types of network drivers:

1. Network interface card or NIC drivers. NIC drivers directly manage net-work interface cards, referred to as NICs. The NIC drivers interface directly to the hardware at their lower edge and at their upper edge present an interface to

allow upper layers to send packets on the network, to handle interrupts, to re-set the NIC, to halt the NIC and to query and set the operational characteristics of the driver. A NIC driver cannot communicate with user-mode applications,

but only with NDIS intermediate drivers or protocol drivers. NIC drivers can be either miniports or legacy full NIC drivers.

 Miniport drivers implement only the hardware-specific operations nec-essary to manage a NIC, including sending and receiving data on the NIC. Operations common to all lowest level NIC drivers, such as syn-chronization,

is provided by NDIS. Miniports do not call operating sys-tem routines directly; their interface to the operating system is NDIS. A miniport does not keep track of bindings. It merely passes packets up to NDIS and NDIS makes sure that these packets are passed to the cor-rect protocols.

 Full NIC drivers have been written to perform both hardware-specific operations and all the synchronization and queuing operations usually done by NDIS. Full NIC drivers, for instance, maintain their own bind-ing

information for indicating received data.

2. Intermediate drivers. Intermediate drivers interface between an upper-level driver such as a legacy transport driver and a miniport. To the upper-level driver, an intermediate driver looks like a miniport. To a miniport, the inter-mediate

driver looks like a protocol driver. An intermediate protocol driver can layer on top of another intermediate driver although such layering could have a negative effect on system performance. A typical reason for developing

an intermediate driver is to perform media translation between an existing leg-acy transport driver and a miniport that manages a NIC for a new media type unknown to the transport driver. For instance, an intermediate driver could

translate from LAN protocol to ATM protocol. An intermediate driver cannot communicate with user-mode applications, but only with other NDIS drivers.

3. Transport drivers or protocol drivers. A protocol driver implements a net-work protocol stack such as IPX/SPX or TCP/IP, offering its services over one or more network interface cards. A transport driver services application-layer

clients at its upper edge and connects to one or more NIC driver(s) or interme-diate NDIS driver(s) at its lower edge.

Next figure shows a sample NDIS structure with two capture stacks on the same network adapter: one with the NIC driver and a protocol driver, the other with the NIC driver, an intermediate driver and a protocol driver.

The packet capture driver needs to communicate both with the network drivers (to get data from the net) and with user-level applications (to provide them the packets), so it is implemented in the NDIS structure as a protocol driver. This allows it to be independent from the network hardware, thus working with all the network interfaces

supported by Windows. Notice however that the packet capture driver works at the moment only on Ethernet adapters, loopback adapters and on some WAN connections due to limits imposed by the driver's and filter's architecture. Notice also that a WAN connection is usually seen by the protocol drivers as an Ethernet NIC, and every re-ceived

packet has a fake Ethernet header created by NDIS. This allows to the protocol drivers written for Ethernet to work on WAN connections without any change, but implies also that specific packets like PPP NCP-LCP are not seen by the protocol drivers, because the PPP connection is virtualized. This means that the packet driver

cannot capture this kind of packets. Notice that the various Win32 operating systems have different versions of NDIS:

the version of NDIS under Windows 95 is 3.0, while Windows NT has NDIS 4 and Windows 2000 has NDIS 5. NDIS 4 and 5 are supersets of NDIS 3, therefore a driver written to work with NDIS 3 (usually) works also with NDIS 4 and 5. The packet cap-ture driver is written for NDIS 3, but works also with the more recent versions of NDIS. This means that the interaction between the driver and NDIS is the same under Windows 95/98 and under Windows 2000.

A protocol driver that communicates with lower level NDIS drivers uses NDIS-provided functions. For instance, a protocol driver must call NdisSend or NdisSend-Packetsto send a packet or packets to a lower level NDIS driver.

Lower level drivers, on the other hand, communicate with the protocol driver in an asynchronous way. They indicate the arrival of a new packet by calling a callback  function of the protocol driver and passing a pointer to a lookahead buffer, its size, and the total size of the received packet. The name of this callback function in the

packet capture driver is Packet_tap. The behavior of the packet driver is however quite different from the one of a standard protocol driver. In fact:

 The packet driver receives and processes all the packets that are currently transferred in the network. Such a behavior can be obtained setting the adapter in ‘promiscuous mode’, i.e. forcing it to accept all the packets from the net-work. A standard protocol driver manages only packets (unicast and multicast) directed to or coming it and the broadcast ones.

 The packet driver does not implement a protocol, but it stores the packets and transfers them as they are, with their timestamp and their length, to the applications. A standard protocol driver removes the various headers from the

packets and passes to the applications only the data. Packet driver leaves the headers and copies them to the applications with the captured data. Note that in UNIX implementation, BPF is called before the protocol stack, di-rectly by the network interface’s driver. This is not possible for the packet capture driver, that is a part of the protocol stack. That is why the packet capture driver is not able to capture PPP specific packets, because it does not work at hardware level, but on top of NDIS. Having the precedence on NDIS would imply changes in the kernel

or in the NIC drivers, which is not possible in Windows.