Re: Continuous packet capture

[Jon Polacheck]

What I get out the the example;

Wheel::Run is used to create tasks (as opposed to processes or sessions)
Filter::Reference is used to create a communications channel between the 
tasks

What I am still wondering is if the communications channel would buffer 
or act as a fifo for the data coming out of the enque task and if 
prioritizing would be needed to make sure that the enque task would not 
drop any packets.
_______________________________________________
Alex _
Fri, 13 Mar 2009 15:49:37 -0700

heres an example of producer/consumer with wheel::run
 http://poe.perl.org/?POE_Cookbook/Child_Processes_3




________________________________
From: Jonathan S. Polacheck <jpola...@texasmutual.com>
To: poe@perl.org
Sent: Friday, March 13, 2009 1:13:55 PM
Subject: Re: Continuous packet capture


ch...@fedde.us wrote;

Your first step is going to be to make sure your code is 'use strict; use
warnings;' clean.  I'm not saying that your code is not, but since I don't
see the strictures I'm making the knee jerk comment.  Second,  POE might be
a fine way to go.  There are lots of components available that'll make
coding this up easier.   Third, what's wrong with using one of the packages
you list?

and jpola...@texasmutual.com replied;

strict and warnings is definitely a good idea. But if I port the whole
thing to POE, I guess I'll start using them on that code.  The two included
scripts are just a sort of "proof of concept" I did to get things going.  I
ultimately may not be able to pull this off in Perl, but it's what I know
so I starting there.

As for the "packages", if you are referring to Infinistrream, Gigastor,
etc, the "problem" in my view is that they are both proprietary and
expensive.  I believe that continuous packet capture will become a standard
way of doing things, supplanting ad-hoc capture (pcap, wireshark, etc), at
least in any production environment large enough to require staff
knowledgeable to use such tools.  That, I think, is the time when open
source solutions break into to market.

We have Infinistreams in the production environment I work at.  We have the
lowest end devices at our remote sites. We paid in excess of $10k each for
them, plus ongoing support contract costs.  They work find, but the vendor
(now Netscout) has dropped them from the product line (no replacements, no
further updates, end-of-life on the horizon).  Netscout has a track record
of going for the high end of the market with product development and
pricing to match.

So I decided that wireshark should evolve to include cpc capabilities.  I
sent my code to wireshark-dev and was roundly ignored (no time for Perl
programmers, perhaps).  But no matter, communities are where you find them.
So I tried the POE list, and here we are.

I agree that there are lots of components.  Here's where my POE solution
stands;

use POE;
use strict;
use warnings;
use Net::Pcap;
use POE::Component::Pcap;
use Data::Hexdumper qw( hexdump );
use Data::Dumper;
use lib 't';

my $dev = "eth0";
my $i = 0;

POE::Session->create(
      inline_states => {
            _start      => \&start,
            got_packet  => \&got_packet,
      },

);

POE::Kernel->run;

sub start {

      #diag "[POE:start] spawning new Pcap session ", $_[&SESSION]->ID, "
on device $dev";
      POE::Component::Pcap->spawn(
            Alias => 'pcap',
            Device => $dev,
            Dispatch => 'got_packet',
            Session => $_[&SESSION],
      );

      $_[&KERNEL]->post(pcap => open_live => $dev, 1514, 1);
      $_[&KERNEL]->post(pcap => 'run');

}

# sub stop {
#     #diag "[POE:stop]";
#     $_[&KERNEL]->post(pcap => 'shutdown');
# }

sub got_packet {
      #diag "[POE:got_packet]";
#     $i++;
#     print "got_packet run $i\n";

      my $packets = $_[&ARG0];

      # process the first packet only
      process_packet(@{ $packets->[0] });

      # send a message to stop the capture
      #    $_[&KERNEL]->post(pcap => 'shutdown');
}

sub process_packet {
#     my ($pkt) = $_[1];
      my $results = hexdump( data => $_[1]
            , number_format => 'C',
      );
#     print Dumper($header);
      print $results;
}

#&start;
exit;

So I have the hexdump moved into "process_packet" and out of "got_packet".
And it;'s easy enough to change "print $results" to a SQL insert statement.
But I think I need a que or fifo (HEAP?) to hold the packets and a priority
on "get_packet" to make sure it keeps up with POE::Component::Pcap and the
incoming traffic.  Or is there a better way?

Thanks for your interest,