Yesterday, on the Snort-Sigs mailing list, we had a report of a potential false-negative in an older Snort rule. While he was unable to provide a full packet capture at the time, the author of the email was able to provide a copy-paste of the packet data. A lot of times, Alex Kirk takes point on these complaints, but he was still trying to catch up from his jaunt down to Brazil to speak at Hacker2Hacker. So I grabbed the data and worked on the issue. I thought it might be interesting for folks to know how we approach reports like this.
So the issue was with the following rule:
alert tcp $EXTERNAL_NET any -> $SQL_SERVERS 1433 (msg:"SQL SA bruteforce login attempt TDS v7/8"; flow:to_server,established; content:"|10|"; depth:1; content:"|00 00|"; depth:2; offset:34; content:"|00 00 00 00|"; depth:4; offset:64; pcre:"/^.{12}(\x00|\x01)\x00\x00(\x70|\x71)/smi"; byte_jump:2,48,little,from_beginning; content:"s|00|a|00|"; within:4; distance:8; nocase; reference:bugtraq,4797; reference:cve,2000-1209; reference:nessus,10673; classtype:suspicious-login; sid:111113543;)And the attack pcap is as follows:
0000 00 14 bf 52 fe 40 00 d0 2b 77 75 01 08 00 45 20 ...R.@.. +wu...E0010 00 bc 1e 56 40 00 6c 06 xx xx 79 0b 50 ce xx xx ...V@.l. xxy.P.xx0020 xx 7a 08 2b 05 99 a4 51 cc 4d b1 be 2b 43 50 18 xz.+...Q .M..+CP.0030 ff ff 3d 81 00 00 10 01 00 94 00 00 01 00 8c 00 ..=..... ........0040 00 00 01 00 00 71 00 00 00 00 00 00 00 07 d0 19 .....q.. ........0050 00 00 00 00 00 00 e0 03 00 00 20 fe ff ff 04 08 ........ .. .....0060 00 00 56 00 06 00 62 00 02 00 66 00 01 00 68 00 ..V...b. ..f...h.0070 00 00 68 00 0e 00 00 00 00 00 84 00 04 00 8c 00 ..h..... ........0080 00 00 8c 00 00 00 00 1c 25 5b 6f ff 00 00 00 00 ........ %[o.....0090 8c 00 00 00 44 00 57 00 44 00 57 00 34 00 44 00 ....D.W. D.W.4.D.00a0 73 00 61 00 b3 a5 xx 00 xx 00 2e 00 xx 00 xx 00 s.a...x. x...x.x.00b0 xx 00 2e 00 xx 00 xx 00 xx 00 2e 00 31 00 32 00 x...x.x. x...1.2.00c0 32 00 4f 00 44 00 42 00 43 00 2.O.D.B. C.So, the first thing I wanted to do was to take a quick look see to check if the packet should alert. It was kind of sloppy (this cost me some time), but here is what I did:
Looking at the rule, it requires content:|10| at depth 1. As it turns out, there is only one 0x10 in the pcap, so I just assumed this was the begining of the packet payload (lazy). As it turns out, I was right. So I took each portion of the detection in the rule and laid it out and compared it to the packet:
Original packet data, serialized:
10 01 00 94 00 00 01 00 8c 00 00 00 01 00 00 71 00 00 00 00 00 00 0007 d0 19 00 00 00 00 00 00 e0 03 00 00 20 fe ff ff 04 08 00 00 56 0006 00 62 00 02 00 66 00 01 00 68 00 00 00 68 00 0e 00 00 00 00 00 8400 04 00 8c 00 00 00 8c 00 00 00 00 1c 25 5b 6f ff 00 00 00 00 8c 0000 00 44 00 57 00 44 00 57 00 34 00 44 00 73 00 61 00 b3 a5 xx 00 xx00 2e 00 xx 00 xx 00 xx 00 2e 00 xx 00 xx 00 xx 00 2e 00 31 00 32 0032 00 4f 00 44 00 42content:"|10|"; depth: 1;10
content:"|00 00|"; depth: 2; offset: 34;00 00
content:"|00 00|"; depth: 4; offset: 64;00 00 00 00
pcre:"/^.{12}(\x00|\x01)\x00\x00(\x70|\x71)/smi";10 01 00 94 00 00 01 00 8c 00 00 00 01 00 00 71
byte_jump:2,48,little,from_beginning;62 00 [Read little endian, decimal: 98]
content:"s|00|a|00|";44 00 57 00
"D" 00 "W" 00
So, a note. I totally messed the last match up, because I failed to notice that the content match had a within: 4; distance:8; set of modifiers. So at this point, I thought there was a problem with the rule. So I decided to hand decode the pcap. Nothing says dedication like hand decoding packets in VI, but I was free for a while, and for some reason very motivated to nail down the issue. The original author was actually very awesome in this regard, because he provided an excellent link to a reference that detailed the protocol, you can find it at http://www.freetds.org/tds.html#login7.
So...at first I didn't know what the first 8 bytes were, so I cleverly wrote:
10 01 00 94 00 00 01 00 I have no idea what this does
This is fine, you don't have to know everything, but don't forget that you don't know it, because if you get stuck later, its an avenue to explore. Then I got down to actually working on the decoding of the login data. Here is the full decode that I did:
[Login Packet Decode]Total Packet Size [4]: 8c 00 00 00 4TDS Version [4]: 01 00 00 71 8Packet Size [4]: 00 00 00 00 12Client Version Program [4]: 00 00 00 07 16PID of Client [4]: d0 19 00 00 20Connection ID [4]: 00 00 00 00 24Option Flags 1 [1]: e0 25Option Flags 2 [1]: 03 26Sql Type Flags [1]: 00 27reserved flags [1, mbz]: 00 28time zone [4]: 20 fe ff ff 32Collation Info [4]: 04 08 00 00 36Position of client hostname [2] 56 00 [86 decimal] 38Hostname length [2] 06 00 40Position of username [2]: 62 00 [98 decimal] 42Username length [2]: 02 00 44Position of password [2]: 66 00 [102 decimal] 46Password length [2]: 01 00 48Position of app name [2]: 68 00 [104 decimal] 50Length of app name [2]: 00 00 52Position of server name [2]: 68 00 [104 decimal] 54Length of server name [2]: 0e 00 56Int16 [2, mbz] [2]: 00 00 58Int16 [2, mbz] [2]: 00 00 60Position of library name [2]: 84 00 [132 decimal] 62Length of library name [2]: 04 00 64Position of language [2]: 8c 00 [132 decimal] 66Length of language [2]: 00 00 68Position of database name [2]: 8c 00 [132 decimal] 70Length of database name [2]: 00 00 72Mac address of the client [6]: 00 1c 25 5b 6f ff 78Position of auth portion [2]: 00 00 80NT Auth Length [2]: 00 00 82Next position [2]: 8c 00 [132 decimal] 84Int16 [2, mbz]: 00 00 86Hostname [n(6)]: 44 00 57 00 44 00 57 00 34 00 44 00 98 (DWDW4D)Username [n(2)]: 73 00 61 00 102 (sa)Password [n(1)]: b3 a5 104 (encrypted)Server Name [n(14)]: xx 00 xx 00 2e 00 xx 00 xx 00 xx 00 2e 00 xx 00 xx 00 xx 00 2e 00 31 00 32 00 32 00 132 (xx.xxx.xxx.122)Library Name [n(4)]: 4f 00 44 00 42 00 43 00 140 (ODBC)So the numbers on the far right are a running count of the offset from the begining of the TDS Login Packet data fields. I did this because all of the provided offsets (Position of....) are in relation to the begining of the Login Packet fields, and if I have to continuously recalculate where I am I will eventually screw it up. So I do a little extra work to be sure I know what I'm looking at.
Next I recheck the snort detection methodology using the decoded information so I understand what it is that the rule is trying to do. When I do this, I finally notice that the last content match actually has additional modifers to it. As I review each check, I make notes next to the checks to tell me what was going on:
content:"|10|"; depth: 1; [Not immediately apparent what this is, as it is part of the undescribed header]content:"|00 00|"; depth: 2; offset: 34; [Checking the Sql Flags and the Reserved flags are 00 00]content:"|00 00 00 00|"; depth:4; offset:64; [Checking the 4 must-be-zero bytes at offset 58 and 60]pcre:"/^.{12}(\x00|\x01)\x00\x00(\x70|\x71)/smi"; [Verifying that we have an appropriate version field at offset 8]byte_jump:2,48,little,from_beginning; [Grab the offset of Username, jump the offset from begining of packet value here is 62 hex, 98 decimal]content:"s|00|a|00|"; within 4; distance: 8; [Check for username "sa", adjusting for 8 byte header]OK, now we're working, but I want to know what the |10| is for, so I look around and find the TDS header specification a little above the login spec, so I take a moment and break that down:
[TDS Packet Header]Packet type: 10 (TDS 7.0 login packet)Last Packet indicator: 01Packet Size: 00 94Unknown: 00 00 01 00So now I can describe in plain language what the rule is trying to do. First, check to make sure this packet is a TDS login packet (as it turns out, 10 is valid for 7.0 and 8.0). Then check to make sure that fields that are known to be set as |00| for TDS login packets are indeed set to |00|. This ensures that we are looking at the correct kind of packet, by verifying that these structures are located in the correct place. Now that we've done some basic checking, we have enough information to justify calling the PCRE engine and checking that the version is set correctly. Observant readers will note that the PCRE allows for four variants, but the only valid variants are 00 00 00 70 and 01 00 00 71. The original rule writer determined that this was an acceptable false positive risk and chose to write the rule in this way. He could also have done a full four byte OR between the two values, but that doesn't substantially impact performance or accuracy, so I'm not concerned with changing it now. Finally, you grab the offset to the Username field. This is at a known location 48 bytes from the begining of the payload. We know that this field is 2 bytes long, and written in little endian. We then move the DOE pointer that many bytes from the begining of the file. Finally, we check for the unicode string "sa" 8 bytes from where the DOE is located. We do this because we know that the TDS header is of a fixed size of 8 bytes, and all offset values are off by 8 when you calculate them from the begining of the payload.
So now I know the detection should have triggered on this pcap. I also know that there is thresholding in the rule, and that there may be some issues with that. But I have a hard time checking that with just this pcap that isn't really a pcap. So I decide to decode the Ethernet, IP and TCP headers to ensure that they line up with the rule (basically checking that the dst port is 1433):
[Layer 2/3 HEADERS]Eth00 14 bf 52 fe 40 dst00 d0 2b 77 75 01 src08 00 typeIP45 Ver 4, Header size20 TOS00 bc Total length1e 56 ID40 00 Flags and frag info6c TTL06 Protocol (TCP)xx xx Checksum79 0b 50 ce 121.11.80.206xx xx xx 7a xx.xx.xx.122TCP08 2b src 209105 99 dst 1433 (Correct port for rule)a4 51 cc 4d checksumb1 be 2b 43 ack number50 18 hdr len/reservered/flags (ack/psh set flags consistent with stream state)ff ff window size3d 81 tcp checksum00 00 urgent pointerA couple of notes on this. First, the dst port was indeed 1433, so we're good there. But I wanted to point out something the original author of the email did that was very, very clever and important. He was careful to obfuscate the destination IP address so we don't know what network or company we're discussing. But he went further than that, and also obfuscated the checksum field so we couldn't use that as a check as we tried to work out what the IP address was prior to obfuscation. Very nice. I also noticed that the source IP address was left in, so I had to whois it. Turns out its an address in the Chinanet AS, so I'm assuming this is a live attack capture.
So...now I've pretty much completely decoded the packet, and am pretty certain it isn't a problem with a rule. But I hit up our bugtracking and search for old bugs that involve this SID. I had noticed that the revision number of the rule was 4, so I was hoping there was some evolution of the rule that would indicate something that might help me, but all the modifications were either process driven (standardizing the order in which modifiers come after the content: option) or adding documentation. But there was a PCAP that was built by Alex Kirk when they were first doing research on it. So I grabbed it and tested it against every snort rule we had:
Snort Test Suite v.0.3.0Alerts:1:3273:4 SQL sa brute force failed login unicode attempt Alerts: 931:3543:4 SQL SA brute force login attempt TDS v7/8 Alerts: 93So the first thing that caught my eye here was that I had a new alert, so I grepped over the rules file to see what was going on there:
alert tcp $SQL_SERVERS 1433 -> $EXTERNAL_NET any (msg:"SQL sa brute force failed login unicode attempt"; flow:from_server,established; content:"L|00|o|00|g|00|i|00|n|00| |00|f|00|a|00|i|00|l|00|e|00|d|00| |00|f|00|o|00|r|00| |00|u|00|s|00|e|00|r|00| |00|'|00|s|00|a|00|'|00|"; threshold:type threshold, track by_src, count 5, seconds 2; reference:bugtraq,4797; reference:cve,2000-1209; reference:nessus,10673; classtype:unsuccessful-user; sid:3273; rev:4;)OK, very cool. This indicated to me that I had an attack pcap against an actual server that responded correctly, so my confidence in the rule grew. Now I was wondering what impact the thresholding was having on the rule, so I copied the rule into my local.rules file. I then made a copy of the rule and removed the thresholding. This would allow me to see how many alerts of each were being generated. I used the local.rules file for two reasons. One, I was going to modify a rule, and I don't want to accidentally leave a non-published rule in my testing rule set, and two, it takes a long time to load up every snort rule, so I just load the two I want and things are much quicker. Here is what my local.rules looked like:
alert tcp $EXTERNAL_NET any -> $SQL_SERVERS 1433 (msg:"SQL SA brute force login attempt TDS v7/8"; flow:to_server,established; content:"|10|"; depth:1; content:"|00 00|"; depth:2; offset:34; content:"|00 00 00 00|"; depth:4; offset:64; pcre:"/^.{12}(\x00|\x01)\x00\x00(\x70|\x71)/smi"; byte_jump:2,48,little,from_beginning; content:"s|00|a|00|"; within:4; distance:8; nocase; threshold:type threshold, track by_src, count 5, seconds 2; reference:bugtraq,4797; reference:cve,2000-1209; reference:nessus,10673; classtype:suspicious-login; sid:3543; rev:4;)alert tcp $EXTERNAL_NET any -> $SQL_SERVERS 1433 (msg:"SQL SA brute force login attempt TDS v7/8"; flow:to_server,established; content:"|10|"; depth:1; content:"|00 00|"; depth:2; offset:34; content:"|00 00 00 00|"; depth:4; offset:64; pcre:"/^.{12}(\x00|\x01)\x00\x00(\x70|\x71)/smi"; byte_jump:2,48,little,from_beginning; content:"s|00|a|00|"; within:4; distance:8; nocase; reference:bugtraq,4797; reference:cve,2000-1209; reference:nessus,10673; classtype:suspicious-login; sid:1;)I then reran the test against the same pcap:
Snort Test Suite v.0.3.0Alerts:1:1:0 SQL SA brute force login attempt TDS v7/8 Alerts: 4701:3543:4 SQL SA brute force login attempt TDS v7/8 Alerts: 94That's a good result as well. I have 94 alerts on the thresholding rule, but 470 alerts on the rule with only the base detection. This tells me that the thresholding is behaving correctly. I've pretty much gone through everything that I can on this end. To recap the process:
- Did a quick eyeball check, screwed it up and thought there was a problem.
- Did a much more focused check after decoding the packet, discovered that the core functionality was fine.
- Checked the layer two and three headers, gave a thumbs up to checksum obfuscation, but didn't see anything problematic there.
- Checked our bug system, pulled the research notes and retested the rules using the original test pcap.
- Pulled the thresholding out of the rule to verify that that was working correctly.Everything looked good on our end, so I reported back my findings to the original author. He indicated that the attacks were coming in roughly every 2.5 seconds, which would not trigger the threshold of 5 every 2 seconds (threshold:type threshold, track by_src, count 5, seconds 2;). But this is what we love about Snort, because a quick copy paste to the local.rules file and changing the threshold to 1800 seconds will certainly give him enough alerts to deal with.
So that is how we approach things from a problem rule perspective. Hopefully there is something in here you can apply to your own rule writing and troubleshoot, or at least you know that your reports don't just go into void and that we actually have a process in place to deal with them. Let us know if you have any questions.