HI Ted, This is really looking amazing. Thank you so much for working on this. I wanted to ask whether you’ve tried the BitWriter to write a Boolean value? I’ve done that in UDFs. — C
> On Jan 3, 2018, at 14:22, Ted Dunning <[email protected]> wrote: > > Added exploded flag fields per Charles suggestion. I couldn't figure out > how to get a boolean value, however. Will push to the pull request shortly. > > 0: jdbc:drill:zk=local> select src_port, dst_port, tcp_session, tcp_ack, > tcp_flags, tcp_parsed_flags from > dfs.root.`/Users/tdunning/tmp/synscan.pcap` where tcp_flags_ack>0 limit 10; > > *+-----------+-----------+-----------------------+-------------+------------+-------------------+* > > *| **src_port ** | **dst_port ** | ** tcp_session ** | ** tcp_ack > ** | **tcp_flags ** | **tcp_parsed_flags ** |* > > *+-----------+-----------+-----------------------+-------------+------------+-------------------+* > > *| *53 * | *36050 * | *6346604732028469374 * | *-581795047 * | *18 > * | *ACK|SYN * |* > > *| *113 * | *36050 * | *2591649313338334401 * | *-581795047 * | *20 > * | *ACK|RST * |* > > *| *80 * | *36050 * | *-9031405983396365775 * | *-581795047 * | *18 > * | *ACK|SYN * |* > > *| *22 * | *36050 * | *7738739733723725373 * | *-581795047 * | *18 > * | *ACK|SYN * |* > > *| *25 * | *36050 * | *1547454657516057906 * | *-581795047 * | *20 > * | *ACK|RST * |* > > *| *31337 * | *36050 * | *2135088345829856782 * | *-581795047 * | *20 > * | *ACK|RST * |* > > *| *53 * | *36050 * | *6346604732028469374 * | *-581795047 * | *18 > * | *ACK|SYN * |* > > *| *113 * | *36061 * | *1324234686720796562 * | *-598572519 * | *20 > * | *ACK|RST * |* > > *| *80 * | *36050 * | *-9031405983396365775 * | *-581795047 * | *18 > * | *ACK|SYN * |* > > *| *70 * | *36050 * | *-3325173385040189964 * | *-581795047 * | *20 > * | *ACK|RST * |* > > *+-----------+-----------+-----------------------+-------------+------------+-------------------+* > > 10 rows selected (0.422 seconds) > > 0: jdbc:drill:zk=local> select src_port, dst_port, tcp_session, tcp_ack, > tcp_flags, tcp_parsed_flags from > dfs.root.`/Users/tdunning/tmp/synscan.pcap` where tcp_flags_rst>0 limit 10; > > *+-----------+-----------+-----------------------+-------------+------------+-------------------+* > > *| **src_port ** | **dst_port ** | ** tcp_session ** | ** tcp_ack > ** | **tcp_flags ** | **tcp_parsed_flags ** |* > > *+-----------+-----------+-----------------------+-------------+------------+-------------------+* > > *| *113 * | *36050 * | *2591649313338334401 * | *-581795047 * | *20 > * | *ACK|RST * |* > > *| *25 * | *36050 * | *1547454657516057906 * | *-581795047 * | *20 > * | *ACK|RST * |* > > *| *31337 * | *36050 * | *2135088345829856782 * | *-581795047 * | *20 > * | *ACK|RST * |* > > *| *113 * | *36061 * | *1324234686720796562 * | *-598572519 * | *20 > * | *ACK|RST * |* > > *| *70 * | *36050 * | *-3325173385040189964 * | *-581795047 * | *20 > * | *ACK|RST * |* > > *+-----------+-----------+-----------------------+-------------+------------+-------------------+* > > 5 rows selected (0.278 seconds) > > On Tue, Jan 2, 2018 at 4:34 PM, Ted Dunning <[email protected]> wrote: > >> >> As predicted, those sequence and ack numbers were wrong. >> >> I now have a pull request that is much closer, in fact ready for review. >> >> See https://github.com/apache/drill/pull/1080 >> >> *Charles, * >> >> Is this what you were looking to do? >> >> *Everybody,* >> >> On a side issue, there was a problem building this that required a change >> to drill/exec/jdbc-all/pom.xml to increase the maximum allowable size for >> the jdbc-all jar. That doesn't seem right, but it seemed to be a problem >> even before I made a change. >> >> >> On Tue, Jan 2, 2018 at 11:35 AM, Ted Dunning <[email protected]> >> wrote: >> >>> >>> Charles, >>> >>> This got me excited enough to just code something up. >>> >>> Here is my current output: >>> >>> *| *-9112490598188246883 * | *120782885 * | *191 * | *NS|ECE (ECN >>> capable)|URG|ACK|PSH|RST|SYN * |* >>> >>> *| *3418479551590665689 * | *120782885 * | *191 * | *NS|ECE (ECN >>> capable)|URG|ACK|PSH|RST|SYN * |* >>> >>> *| *9404749046357206 * | *120782885 * | *191 * | *NS|ECE (ECN >>> capable)|URG|ACK|PSH|RST|SYN * |* >>> >>> *| *7971558575805030122 * | *120782885 * | *191 * | *NS|ECE (ECN >>> capable)|URG|ACK|PSH|RST|SYN * |* >>> >>> *| *-4666599101156419893 * | *120782885 * | *191 * | *NS|ECE (ECN >>> capable)|URG|ACK|PSH|RST|SYN * |* >>> >>> *| *-672627876006511342 * | *-1846673370 * | *49 * | *ECE >>> (Congestion experienced)|URG|SYN * |* >>> >>> *| *6346604732028469374 * | *1611798224 * | *106 * | *CWR|ECE >>> (ECN capable)|ACK|RST * |* >>> >>> *| *-9031405983396365775 * | *1611798224 * | *201 * | * >>> NS|CWR|ACK|SYN * |* >>> >>> *| *7738739733723725373 * | *1611798224 * | *219 * | * >>> NS|CWR|URG|ACK|RST|SYN * |* >>> >>> *| *6346604732028469374 * | *1611798224 * | *106 * | *CWR|ECE >>> (ECN capable)|ACK|RST * |* >>> >>> *| *-9031405983396365775 * | *1611798224 * | *201 * | * >>> NS|CWR|ACK|SYN * |* >>> >>> *| *7738739733723725373 * | *1611798224 * | *219 * | * >>> NS|CWR|URG|ACK|RST|SYN * |* >>> >>> >>> I am not confident of the ack number being correct and will check against >>> your reference. What is new here is the decoded flags column. It might be >>> reasonable to have specific columns for the most important flags "ACK, RST, >>> SYN" since all access is lazy anyway. >>> >>> What do you think? >>> >>> Can some or all of the pcap file you included be distributed as a unit >>> test? >>> >>> >>> >>> On Mon, Jan 1, 2018 at 12:31 PM, Charles Givre <[email protected]> wrote: >>> >>>> Hello all, >>>> I was playing with the PCAP functionality in Drill and I wanted to add >>>> the TCP flags to the data that Drill is returning. I was also interested >>>> in adding the TCP Sequence and Ack numbers as well. I noticed that the >>>> code as written currently has a function in Packet.java which returns the >>>> TCP Sequence number, however this was never added to the schema, so I added >>>> that and rebuilt Drill, however, it doesn’t seem to be returning the >>>> correct result. The file I was querying is attached to this email, and >>>> should in all cases return a sequence number of zero. >>>> >>>> Questions: >>>> 1. Could someone please take a look at the code for the tcp_sequence >>>> and see if I did something wrong, or if the offset is not being calculated >>>> correctly >>>> 2. I’m trying to figure out the offsets for the various TCP flags. I >>>> would think that the offset should be PacketConstants.ETHER_HEADER_LENGTH >>>> + getIPHeaderLength() +13 to get the word that has the flags and then from >>>> there, access the individual bits. However, this doesn’t seem to work. >>>> What am I missing? >>>> Thanks and Happy New Year! >>>> - C >>> >>> >>> >>
