PCAP Next Generation Dump File Format. PCAP Next Generation Dump File Format. PCAP- Dump. File. Format. 2Status of this Memo. This document is an Internet- Draft and is.
Section 1. 0 of RFC2. Internet- Drafts are working documents of the Internet Engineering. Task Force (IETF), its areas, and its working groups. Note that other groups may also distribute working documents as. Internet- Drafts. Internet- Drafts are draft documents valid for a maximum of six months. It is inappropriate to use Internet- Drafts as reference material or to cite. The list of current Internet- Drafts can be accessed at. The list of Internet- Draft Shadow Directories can be accessed at. This Internet- Draft will expire on August 3. Copyright Notice. Copyright (C) The Internet Society (2. All Rights Reserved. Abstract. This document describes a format to dump captured packets on a file. This format is extensible and it is currently proposed for implementation in the libpcap/Win. Pcap packet capture library. Development / LibpcapFileFormat. FrontPage; RecentChanges; FindPage; HelpContents; LibpcapFileFormat; Immutable Page; Comments; Info; Attachments. There's a next generation pcap file format documented at the pcapng. A list of publicly available pcap files / network traces that can be downloaded for free. Table of Contents. General File Structure. General Block Structure. Block Types. 2. 3. Block Hierarchy and Precedence. Block Definition. Section Header Block (mandatory)3. Interface Description Block (mandatory)3. Packet Block (optional)3. Simple Packet Block (optional)3. Name Resolution Block (optional)3. Interface Statistics Block (optional)4. Experimental Blocks (deserved to a further investigation)5. Other Packet Blocks (experimental)5. Compression Block (experimental)5. Encryption Block (experimental)5. Fixed Length Block (experimental)5. Directory Block (experimental)5. Traffic Statistics and Monitoring Blocks (experimental)5. Event/Security Block (experimental)6. Most important open issues. A. Packet Block Flags Word§. Intellectual Property and Copyright Statements. Objectives. The problem of exchanging packet traces becomes more and more critical every day; unfortunately, no standard solutions exist for this task right now. One of the most accepted packet interchange formats is the one defined by libpcap, which is rather old and does not fit for some of the nowadays applications particularly from the extensibility point of view. This document proposes a new format for dumping packet traces. The following goals are being pursued. Extensibility: aside of some common functionalities, third parties should be able to enrich the information embedded in the file with proprietary extensions, which will be ignored by tools that are not able to understand them. Portability: a capture trace must contain all the information needed to read data independently from network, hardware and operating system of the machine that made the capture. Merge/Append data: it should be possible to add data at the end of a given file, and the resulting file must still be readable. General File Structure General Block Structure. A capture file is organized in blocks, that are appended one to another to form the file. All the blocks share a common format, which is shown in Figure 1. Basic block structure.. Block Type |. Block Total Length |. Block Body /. Block Total Length |. The fields have the following meaning. Block Type (3. 2 bits): unique value that identifies the block. Values whose Most Significant Bit (MSB) is equal to 1 are reserved for local use. They allow to save private data to the file and to extend the file format. Block Total Length: total size of this block, in bytes. For instance, the length of a block that does not have body is 1. Block Body: content of the block. Block Total Length: total size of this block, in bytes. This field is duplicated for permitting backward file navigation. This structure, shared among all blocks, makes easy to process a file and to skip unneeded or unknown blocks. Some blocks can contain other blocks inside (nested blocks). Some of the blocks are mandatory, i. The General Block Structure allows defining other blocks if needed. A parser that does non understand them can simply ignore their content. Block Types. Blocks currently defined are the following. Section Header Block: it defines the most important characteristics of the capture file. Interface Description Block: it defines the most important characteristics of the interface(s) used for capturing traffic. Packet Block: it contains a single captured packet, or a portion of it. Simple Packet Block: it contains a single captured packet, or a portion of it, with only a minimal set of information about it. Name Resolution Block: it defines the mapping from numeric addresses present in the packet dump and the canonical name counterpart. Capture Statistics Block: it defines how to store some statistical data (e. The following blocks instead are considered interesting but the authors believe that they deserve more in- depth discussion before being defined. Additional Packet Blocks. Compression Marker Block. Encryption Marker Block. Fixed Length Marker Block. Directory Block. Traffic Statistics and Monitoring Blocks. Event/Security Blocks. Currently standardized Block Type codes are specified in Appendix 1. TODO). Block Hierarchy and Precedence. The file must begin with a Section Header Block. However, more than one Section Header Block can be present on the dump, each one covering the data following it till the next one (or the end of file). A Section includes the data delimited by two Section Header Blocks (or by a Section Header Block and the end of the file), including the first Section Header Block. In case an application cannot read a Section because of different version number, it must skip everything until the next Section Header Block. Note that, in order to properly skip the blocks until the next section, all blocks must have the fields Type and Length at the beginning. This is a mandatory requirement that must be maintained in future versions of the block format. Figure 2. File structure example: the Section Header Block. Section Header that covers the whole file. The second one contains three headers, and is normally the result of file concatenation. An application that understands only version 1. Section Header. +- +- +- +- +- +- +- +- +- +- +- +- +- +- +- +- +- +- +- +- +- +- +- +- +- +- +- +- +- +- +- +- +. SHB v. 1. 0 | Data |. Typical configuration with a single Section Header Block. Section - -|- - 2nd Section - -|- - 3rd Section - -|. SHB v. 1. 0 | Data | SHB V1. Data | SHB V1. 0 | Data |. Configuration with three different Section Header Blocks. File structure example: the Section Header Block. NOTE: TO BE COMPLETED with some examples of other blocks. Data format. Data contained in each section will always be saved according to the characteristics (little endian / big endian) of the dumping machine. This refers to all fields that are saved as numbers and that span over two or more bytes. The approach of having each section saved in the native format of the generating host is more efficient because it avoids translation of data when reading / writing on the host itself, which is the most common case when generating/processing capture dumps. TODO Probably we have to specify something more here. Is what we're saying enough to avoid any kind of ambiguity? Block Definition. This section details the format of the body of the blocks currently defined. Section Header Block (mandatory)The Section Header Block is mandatory. It identifies the beginning of a section of the capture dump file. Its format is shown in Figure 3. Section Header Block format.. Byte- Order Magic |. Major Version | Minor Version |. Section Length |. Options (variable) /. Section Header Block format. The meaning of the fields is. Byte- Order Magic: magic number, whose value is the hexadecimal number 0x. A2. B3. C4. D. This number can be used to distinguish sections that have been saved on little- endian machines from the ones saved on big- endian machines. Major Version: number of the current mayor version of the format. Current value is 1. This value should change if the format changes in such a way that tools that can read the new format could not read the old format (i. Minor Version: number of the current minor version of the format. Current value is 0. This value should change if the format changes in such a way that tools that can read the new format can still automatically read the new format but code that can only read the old format cannot read the new format. Section Length: 6. Section Header Block itself. This field can be used to skip the section, for faster navigation inside large files. Section Length equal - 1 (0xffffffffffffffff) means that the size of the section is not specified, and the only way to skip the section is to parse the blocks that it contains. Options: optionally, a list of options (formatted according to the rules defined in Section 4. Options) can be present. Adding new block types or options would not necessarily require that either Major or Minor numbers be changed, as code that does not know about the block type or option could just skip it; only if skipping a block or option does not work should the minor version number be changed. The block type of the Section Header Block is the integer corresponding to the 4- char string "\r\n\n\r". This particular value is used for 2 reasons. This number is used to detect if a file has been trasferred via FTP from a machine to another without the proper ASCII conversion. In this case, the value if this field will differ from the standard one ("\r\n\n\r") and the reader can detect a possibly corrupted file. This value is palindromic, so that the reader is able to recognize the Section Header Block regardless of the endianess of the section. The endianess is recognized by reading the Byte Order Magic, that is located 8 bytes after the Block Type. Aside from the options defined in Section 4. Options, the following options are valid within this block. Name. Code. Length. Description. Hardware. An UTF- 8 string containing the description of the hardware used to create this section. Operating System. Win. Pcap: Handling offline dump files. In this lession we are going to learn how to handle packet capture to a file (dump to file). Win. Pcap offers a wide range of functions to save the network traffic to a file and to read the content of dumps - - this lesson will teach how to use all of these functions. We'll see also how to use the kernel dump feature of Win. Pcap to obtain high- performance dumps (NOTE: At the moment, due to some problems with the new kernel buffer, this feature has been disabled). The format for dump files is the libpcap one. This format contains the data of the captured packets in binary form and is a standard used by many network tools including Win. Dump, Ethereal and Snort. Saving packets to a dump file. First of all, let's see how to write packets in libpcap format. The following example captures the packets from the selected interface and saves them on a file whose name is provided by the user.#include "pcap. PCAP_ERRBUF_SIZE]. PCAP_SRC_IF_STRING, NULL, & alldevs, errbuf) == - 1). Error in pcap_findalldevs: %s\n", errbuf). No description available)\n"). No interfaces found! Make sure Win. Pcap is installed.\n"). Enter the interface number (1- %d): ",i). Interface number out of range.\n"). PCAP_OPENFLAG_PROMISCUOUS. NULL. errbuf. ) ) == NULL). Unable to open the adapter. Win. Pcap\n", d- > name). NULL). fprintf(stderr,"\n. Error opening output file\n"). Press Ctrl+C to stop..\n", d- > description). As you can see, the structure of the program is very similar to the ones we have seen in the previous lessons. The differences are: a call to pcap_dump_open() is issued once the interface is opened. This call opens a dump file and associates it with the interface. The parameters of pcap_dump() are in 1- 1 correspondence with the parameters of pcap_handler(). Reading packets from a dump file. Now that we have a dump file available, we can try to read its content. The following code opens a Win. Pcap/libpcap dump file and displays every packet contained in the file. The file is opened with pcap_open_offline(), then the usual pcap_loop() is used to sequence through the packets. As you can see, reading packets from an offline capture is nearly identical to receiving them from a physical interface. This example introduces another function: pcap_createsrcsrc(). This function is required to create a source string that begins with a marker used to tell Win. Pcap the type of the source, e. This step is not required when pcap_findalldevs_ex() is used (the returned values already contain these strings). However, it is required in this example because the name of the file is read from the user input.#include < stdio. LINE_LEN 1. 6void dispatcher_handler(u_char *, conststructpcap_pkthdr *, const u_char *). PCAP_ERRBUF_SIZE]. PCAP_BUF_SIZE]. if(argc != 2){. PCAP_SRC_FILE. NULL. NULL. argv[1]. errbuf. Error creating a source string\n"). PCAP_OPENFLAG_PROMISCUOUS. NULL. errbuf. ) ) == NULL). Unable to open the file %s.\n", source). NULL). void dispatcher_handler(u_char *temp. VOID)temp. 1. printf("%ld: %ld (%ld)\n", header- > ts. LINE_LEN) == 0) printf("\n"). The following example has the same purpose of the last one, but pcap_next_ex() is used instead of the pcap_loop() callback method.#include < stdio. LINE_LEN 1. 6int main(int argc, char **argv). PCAP_ERRBUF_SIZE]. PCAP_BUF_SIZE]. struct pcap_pkthdr *header. PCAP_SRC_FILE. NULL. NULL. argv[1]. errbuf. Error creating a source string\n"). PCAP_OPENFLAG_PROMISCUOUS. NULL. errbuf. ) ) == NULL). Unable to open the file %s.\n", source). LINE_LEN) == 0) printf("\n"). Error reading the packets: %s\n", pcap_geterr(fp)). Writing packets to a dump file with pcap_live_dump. NOTE: At the moment, due to some problems with the new kernel buffer, this feature has been disabled. Recent versions of Win. Pcap provide a further way to save network traffic to disk, the pcap_live_dump() function. Zero means no limit for both these values. Notice that the program can set a filter (with pcap_setfilter(), see the tutorial Filtering the traffic) before calling pcap_live_dump() to define the subset of the traffic that will be saved. The dump process goes on asynchronously until the maximum file size or the maximum amount of packets has been reached. The application can wait or check the end of the dump with pcap_live_dump_ended(). Beware that if the sync parameter is nonzero, this function will block your application forever if the limits are both 0.#include < stdlib. At the moment the kernel dump feature is not supported in the driver. PCAP_ERRBUF_SIZE]. Win. Pcap kernel- level dump faeature.\n"). Usage: %s [adapter] | dump_file_name max_size max_packs\n", argv[0]). Where: max_size is the maximum size that the dump file will reach (0 means no limit)\n"). Where: max_packs is the maximum number of packets that will be saved (0 means no limit)\n\n"). Error in pcap_findalldevs: %s\n", errbuf). No description available)\n"). No interfaces found! Make sure Win. Pcap is installed.\n"). Enter the interface number (1- %d): ",i). Interface number out of range.\n"). NULL). fprintf(stderr,"\n. Error opening adapter\n"). Unable to start the dump, %s\n", pcap_geterr(fp)). NULL). fprintf(stderr,"\n. Error opening adapter\n"). Unable to start the dump, %s\n", pcap_geterr(fp)). TRUE). pcap_close(fp). The difference between pcap_live_dump() and pcap_dump(), apart from the possibility to set limits, is performance. Win. Pcap NPF driver (see NPF driver internals manual) to write dumps from kernel level, minimizing the number of context switches and memory copies. Obviously, since this feature is currently not available on other operating systems, pcap_live_dump() is Win. Pcap specific and is present only under Win. Previous. Next > > >. Copyright (c) 2. 00. Politecnico di Torino. Copyright (c) 2. 00. CACE Technologies. All rights reserved.
0 Comments
Leave a Reply. |
AuthorWrite something about yourself. No need to be fancy, just an overview. Archives
October 2016
Categories |