Wireshark can't open netmon files from Microsoft Message Analyzer using non-standard "reserved for pcap" media types
This issue was migrated from bug 10556 in our old bug tracker.
Original bug information:
Reporter: Francesco Pretto
Status: RESOLVED FIXED
Product: Wireshark
Component: Capture file support (libwiretap)
OS: All
Platform: All
Version: Git
Attachments:
windows-client.cap: Failing cap file from Microsoft Message Analyzer
Activity
Created attachment 13154 Failing cap file from Microsoft Message Analyzer
Build Information: Version 1.12.1 (v1.12.1-0-g01b65bf from master-1.12)
Copyright 1998-2014 Gerald Combs <gerald@wireshark.org> and contributors. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
Compiled (64-bit) with GTK+ 2.24.23, with Cairo 1.10.2, with Pango 1.34.0, with GLib 2.38.0, with WinPcap (4_1_3), with libz 1.2.5, with SMI 0.4.8, with c-ares 1.9.1, with Lua 5.2, without Python, with GnuTLS 3.1.22, with Gcrypt 1.6.0, without Kerberos, with GeoIP, with PortAudio V19-devel (built Sep 16 2014), with AirPcap.
Running on 64-bit Windows 8.1, build 9600, with WinPcap version 4.1.3 (packet.dll version 4.1.0.2980), based on libpcap version 1.0 branch 1_0_rel0b (20091008), GnuTLS 3.1.22, Gcrypt 1.6.0, without AirPcap. Intel(R) Core(TM) i5-3450 CPU @ 3.10GHz, with 8143MB of physical memory.
Built using Microsoft Visual C++ 10.0 build 40219
Wireshark is Open Source Software released under the GNU General Public License.
Check the man page and http://www.wireshark.org for more information.
Opening it in WireShark gives this error: ----------------------------------- The capture file has a packet with a network type that Wireshark doesn't support. (netmon: converted pcap network type 134 unknown or unsupported) ----------------------------------- Editcap also refuses to convert it to pcap format with this error: ----------------------------------- C:\Program Files\Wireshark>editcap.exe -F libpcap windows-client.cap d:\test.pca p -v File windows-client.cap is a Microsoft NetMon 2.x capture file. editcap: Can't open or create d:\test.pcap: That file format doesn't support per -packet encapsulations -----------------------------------added enhancement libwiretap versiondev labels
-
The Network Monitor 3.4 documentation says of the "Media Type" field for packets: The following table shows the currently defined media types. Type Values Ethernet 1 Tokenring 2 FDDI 3 ATM 4 1394 5 WiFi 6 Tunneling interfaces 7 Wireless WAN 8 Raw IP Frames 9 Reserved for PCap Link Layer types 0xE000—0xEFFF Unsupported PCAP Link Layer type 0xE000 Linux Cooked Mode 0xE071 NetEvent 0xFFE0 Netmon Network Info Ex 0xFFFB Netmon PayloadHeader 0xFFFC Netmon Network Info 0xFFFD Netmon DNS Cache 0xFFFE Netmon Filter 0xFFFF The media types for which Wireshark would report "netmon: converted pcap network type XXX unknown or unsupported" are the ones in the "Reserved for PCap Link Layer types" range; the manual does not explicitly say so, but values in that range are to be interpreted as if you subtracted 0xE000 from them and then interpreted the result as a pcap/pcap-ng file link-layer type. THe pcap/pcap-ng link-layer type 134 is reserved for Juniper, so, apparently, Microsoft is using it for some other purpose here. A quick hack to look at the raw packet data doesn't show any obvious frame type, so you'd probably have to ask Microsoft how that file should be interpreted by a program reading it and provide that information to us if you want to be able to read those files in Wireshark. -
(In reply to Francesco Pretto from comment #0) > Editcap also refuses to convert it to pcap format with this error: ...which means that it's impossible to convert that file to pcap format, and it will always be impossible to convert that file to pcap format, as not all frames have the same link-layer header type, and pcap doesn't support that. You'd have to convert it to pcap-ng format, *if* we can add support for those frame formats. -
(In reply to Guy Harris from comment #1) > THe pcap/pcap-ng link-layer type 134 is reserved for Juniper, so, > apparently, Microsoft is using it for some other purpose here. The particularity of this dump is that it's taken using the "Network Tunnel Traffic and Unencrypted IPSE" profile of Microsoft Message Analyzer. From my understanding, it's a dumping scenario where the packets are the ones circulating in tunneled networks like PPP and or VPN links and that are effectively filtered by the firewall (inbound/outbound), so they are unencrypted. I guess in the *.cap file all these IP packets are encapsulated in "virtual" link-layer frames. I'm not sure this information is sufficient. If you give me an hint I can try to formulate a request to Microsoft Message Analyzer devs (which is new and under heavy develompent). > You'd have to convert it to pcap-ng format, *if* we can > add support for those frame formats. Ok, clear. -
(In reply to Francesco Pretto from comment #3) > The particularity of this dump is that it's taken using the "Network Tunnel > Traffic and Unencrypted IPSE" profile of Microsoft Message Analyzer. From my > understanding, it's a dumping scenario where the packets are the ones > circulating in tunneled networks like PPP and or VPN links and that are > effectively filtered by the firewall (inbound/outbound), so they are > unencrypted. I guess in the *.cap file all these IP packets are encapsulated > in "virtual" link-layer frames. > > I'm not sure this information is sufficient. If you give me an hint I can > try to formulate a request to Microsoft Message Analyzer devs (which is new > and under heavy development). - "What is the format of the virtual link-layer frames?" - "Can we assume this format for all MMA files with this media type?" -
Ok, I already got an answer from Message Analyzer developers [1]. They state it was maybe a mistake to use type 134 for their "virtual" frame and they request feedback to choose a better id, if needed. You may want to reply them there? Also they point how to find the specification of the frame type that could probably probably called "WFPCapture". Without installing Message Analyzer, you can just download MessageAnalyzer64.msi from [2]. After you can uncompress the plain text parser also in linux with 7-zip in command line: --------------------------------- $ 7z e MessageAnalyzer64.msi _7bb0731df3dd39478605b2d018601589 $ mv _7bb0731df3dd39478605b2d018601589 WFPCapture.opn --------------------------------- The description language used is understandable, but not completly trivial, for example: --------------------------------- endpoint WFPFrame[binary Address] accepts MessageV4 accepts MessageV6 accepts Message2V4 accepts Message2V6 accepts AuthMessageV4 accepts AuthMessageV6 accepts DriverInfo accepts CalloutInfo accepts CalloutNotifyInfo accepts DriverError accepts CalloutError accepts Discard; message MessageV4 { IPv4Address SourceAddress with Visualization{AliasName = "Address"}; IPv4Address DestinationAddress with Visualization{AliasName = "Address"}; byte Protocol; short PayloadLength; binary Payload; [...] --------------------------------- And the Payload of MessageV4 can be effectively the "header" of a IPV4 packet as defined in Network Monitor 3.4 format. [1] https://social.technet.microsoft.com/Forums/en-US/e806a807-ccf2-43f4-9bac-0a2006d75c74/linklayer-frames-format-in-microsoftpefwpfmessageprovider-dumps?forum=messageanalyzer [2] http://download.microsoft.com/download/2/8/3/283DE38A-5164-49DB-9883-9D1CC432174D/MessageAnalyzer64.msi -
(In reply to Francesco Pretto from comment #5) >endpoint WFPFrame[binary Address] > accepts MessageV4 > accepts MessageV6 > accepts Message2V4 > accepts Message2V6 > [...] > Also this means there are more than one media types used, as confirmed by MA devs. For example the more common is Message2V4 that is 0xE085. 0xE085 - 0xE000 = 133. 134 is 0xE086 and is Message2V6: that is described as: message Message2V6 { IPv6Address SourceAddress with Visualization{AliasName = "Address"}; IPv6Address DestinationAddress with Visualization{AliasName = "Address"}; byte Protocol; long FlowContext; short PayloadLength; binary Payload; } -
For your information this is also true for WiFi packets exported from Message Analyzer. Those have a Media Type of 0xE080 (128) which means they are currently decoded as TZSP by Wireshark.
After manually editing the ".cap" file from Message Analyzer to force packet type to 0x0006, Wireshark correctly sets the header information but seems to offset payload dissection by 1 byte which makes it unable to successfully process the contents.
-
I had discussions with some people at Microsoft today and there is some interest in getting Wireshak able to handle the Message Analyzer file format. This seems to be because they are only releasing captures for their protocols in Message Analyzer.
I have a contact there and some links and should be able to get documents to help me convert it using editcap etc or even to convince Wireshark to read it in by massaging it to pcap-ng format.
-
(In reply to Richard Sharpe from comment #9) > I had discussions with some people at Microsoft today and there is some > interest in getting Wireshak able to handle the Message Analyzer file > format. This seems to be because they are only releasing captures for their > protocols in Message Analyzer. > > I have a contact there and some links and should be able to get documents to > help me convert it using editcap etc or even to convince Wireshark to read > it in by massaging it to pcap-ng format. Well, it seems that if you export captures from Message Analyzer (MA) as CAP files from the version from 2014 I am using, everything works. That is, I exported a capture from MA and Wireshark had no problems reading it and displaying the packets correctly. I was using Version 1.4 copyright in 2014 (Build 4.0.8112.0). Wireshark 2.2.7. So, the only thing left to do is to figure out if Wireshark can be taught to read .matp files. -
(In reply to Richard Sharpe from comment #10) > (In reply to Richard Sharpe from comment #9) > > I had discussions with some people at Microsoft today and there is some > > interest in getting Wireshak able to handle the Message Analyzer file > > format. This seems to be because they are only releasing captures for their > > protocols in Message Analyzer. > > > > I have a contact there and some links and should be able to get documents to > > help me convert it using editcap etc or even to convince Wireshark to read > > it in by massaging it to pcap-ng format. > > Well, it seems that if you export captures from Message Analyzer (MA) as CAP > files from the version from 2014 I am using, everything works. > > That is, I exported a capture from MA and Wireshark had no problems reading > it and displaying the packets correctly. > > I was using Version 1.4 copyright in 2014 (Build 4.0.8112.0). > > Wireshark 2.2.7. > > So, the only thing left to do is to figure out if Wireshark can be taught to > read .matp files. However, I see that the cap file I exported has the same version as the capture provided above and I get the same issue as reported above for that capture file. -
Change 23386 had a related patch set uploaded by Michael Mann: Add support for WPFCapture "formats" from Microsoft Analyzer.
-
Change 23386 merged by Michael Mann: Add support for WPFCapture "formats" from Microsoft Analyzer.