Re: [pmacct-discussion] Only packets from router to netflow server

2016-08-19 Thread Paolo Lucente

Hi Mattias,

From what i read so far I believe the pesky bit here is that you are using
pmacctd (which is the libpcap-based daemon) rather than nfacctd (which is
the NetFlow collector daemon, which collects and analyses/dissects NetFlow
packets). 

Cheers,
Paolo


On Fri, Aug 19, 2016 at 12:37:39PM +, Mattias Larsson wrote:
> Hi Markus,
> 
> Not sure what you mean with that the server does NOT accept/process the
> packets due to it target to another MAC address.
> 
> I thought the pmacctd used the libpcap the same way that tcpdump does and
> analyses packets. But with tcpdump I have to use -vvv the all of the packet.
> 
> This is what I get when i'm writing to plain text-file.
> 
> SRC_IP,DST_IP,SRC_PORT,DST_PORT,PROTOCOL,TOS,PACKETS,FLOWS,BYTES
> 192.168.1.1,172.16.0.100,52043,2055,udp,0,10,1,2416
> 
> 192.168.1.1 = router
> 172.16.0.100 = Netflow-server (not same server where I'm running pmacct on)
> 
> My server with pmacct has an interface (eth2) without any ip configurations
> connected to the same switch as the netflow-server. The server recieves all
> udp/2055 packets from the switch (SPAN)
> 
> Iptables are disabled on the server.
> 
> 
> /Mattias
> 
> 
> On Fri, Aug 19, 2016 at 1:00 PM Markus Weber  wrote:
> 
> > Hi Matthias,
> >
> > could it be that your hosts does NOT accept/process the packets as those
> > are targeted to another MAC address? If you run wireshark/tcpdump the
> > interface to put into promiscuous mode to get them ...
> >
> > If all have the same dst mac just change your interface facing the SPAN
> > port to it.
> >
> >
> > Other than that: any host "firewall" rules active?
> >
> >
> > Markus
> >
> >
> > On 19.08.2016 11:21, Jentsch, Mario wrote:
> >
> > Hi Mattias,
> >
> >
> >
> > do you have a drawing of your setup? I have to admit that it is unclear to
> > me…
> >
> >
> >
> > Thanks,
> >
> > Mario
> >
> >
> >
> > *From:* pmacct-discussion [mailto:pmacct-discussion-boun...@pmacct.net
> > ] *On Behalf Of *Mattias Larsson
> > *Sent:* Thursday, August 18, 2016 1:36 PM
> > *To:* pmacct-discussion@pmacct.net
> > *Subject:* [pmacct-discussion] Only packets from router to netflow server
> >
> >
> >
> >
> >
> > I use a SPAN port on my switch to capture all netflow (udp 2055) packets
> > and send it to a interface where my pmacct server has one extra interface
> > connected to.
> >
> >
> >
> > But when I look on the traffic/packets that pmacctd genereates it seems
> > only be the IP packets between my router and netflow server. It seems it
> > not decodes the cisco netflow payload/data.
> >
> >
> >
> > When I do a tcpdump on the interface and look at it with wireshark I can
> > see see the flows.
> >
> >
> >
> > Any suggestion what I'm doing wrong?
> >
> >
> >
> > Thanks in advance!
> >
> >
> > Mattias
> >
> >
> > ___
> > pmacct-discussion mailing listhttp://www.pmacct.net/#mailinglists
> >
> >
> > ___
> > pmacct-discussion mailing list
> > http://www.pmacct.net/#mailinglists

> ___
> pmacct-discussion mailing list
> http://www.pmacct.net/#mailinglists


___
pmacct-discussion mailing list
http://www.pmacct.net/#mailinglists

Re: [pmacct-discussion] Only packets from router to netflow server

2016-08-19 Thread Mattias Larsson
Hi Markus,

Not sure what you mean with that the server does NOT accept/process the
packets due to it target to another MAC address.

I thought the pmacctd used the libpcap the same way that tcpdump does and
analyses packets. But with tcpdump I have to use -vvv the all of the packet.

This is what I get when i'm writing to plain text-file.

SRC_IP,DST_IP,SRC_PORT,DST_PORT,PROTOCOL,TOS,PACKETS,FLOWS,BYTES
192.168.1.1,172.16.0.100,52043,2055,udp,0,10,1,2416

192.168.1.1 = router
172.16.0.100 = Netflow-server (not same server where I'm running pmacct on)

My server with pmacct has an interface (eth2) without any ip configurations
connected to the same switch as the netflow-server. The server recieves all
udp/2055 packets from the switch (SPAN)

Iptables are disabled on the server.


/Mattias


On Fri, Aug 19, 2016 at 1:00 PM Markus Weber  wrote:

> Hi Matthias,
>
> could it be that your hosts does NOT accept/process the packets as those
> are targeted to another MAC address? If you run wireshark/tcpdump the
> interface to put into promiscuous mode to get them ...
>
> If all have the same dst mac just change your interface facing the SPAN
> port to it.
>
>
> Other than that: any host "firewall" rules active?
>
>
> Markus
>
>
> On 19.08.2016 11:21, Jentsch, Mario wrote:
>
> Hi Mattias,
>
>
>
> do you have a drawing of your setup? I have to admit that it is unclear to
> me…
>
>
>
> Thanks,
>
> Mario
>
>
>
> *From:* pmacct-discussion [mailto:pmacct-discussion-boun...@pmacct.net
> ] *On Behalf Of *Mattias Larsson
> *Sent:* Thursday, August 18, 2016 1:36 PM
> *To:* pmacct-discussion@pmacct.net
> *Subject:* [pmacct-discussion] Only packets from router to netflow server
>
>
>
>
>
> I use a SPAN port on my switch to capture all netflow (udp 2055) packets
> and send it to a interface where my pmacct server has one extra interface
> connected to.
>
>
>
> But when I look on the traffic/packets that pmacctd genereates it seems
> only be the IP packets between my router and netflow server. It seems it
> not decodes the cisco netflow payload/data.
>
>
>
> When I do a tcpdump on the interface and look at it with wireshark I can
> see see the flows.
>
>
>
> Any suggestion what I'm doing wrong?
>
>
>
> Thanks in advance!
>
>
> Mattias
>
>
> ___
> pmacct-discussion mailing listhttp://www.pmacct.net/#mailinglists
>
>
> ___
> pmacct-discussion mailing list
> http://www.pmacct.net/#mailinglists
___
pmacct-discussion mailing list
http://www.pmacct.net/#mailinglists

Re: [pmacct-discussion] Only packets from router to netflow server

2016-08-19 Thread Markus Weber

Hi Matthias,

could it be that your hosts does NOT accept/process the packets as those 
are targeted to another MAC address? If you run wireshark/tcpdump the 
interface to put into promiscuous mode to get them ...


If all have the same dst mac just change your interface facing the SPAN 
port to it.



Other than that: any host "firewall" rules active?

Markus

On 19.08.2016 11:21, Jentsch, Mario wrote:


Hi Mattias,

do you have a drawing of your setup? I have to admit that it is 
unclear to me…


Thanks,

Mario

*From:*pmacct-discussion [mailto:pmacct-discussion-boun...@pmacct.net] 
*On Behalf Of *Mattias Larsson

*Sent:* Thursday, August 18, 2016 1:36 PM
*To:* pmacct-discussion@pmacct.net
*Subject:* [pmacct-discussion] Only packets from router to netflow server

I use a SPAN port on my switch to capture all netflow (udp 2055) 
packets and send it to a interface where my pmacct server has one 
extra interface connected to.


But when I look on the traffic/packets that pmacctd genereates it 
seems only be the IP packets between my router and netflow server. It 
seems it not decodes the cisco netflow payload/data.


When I do a tcpdump on the interface and look at it with wireshark I 
can see see the flows.


Any suggestion what I'm doing wrong?

Thanks in advance!


Mattias



___
pmacct-discussion mailing list
http://www.pmacct.net/#mailinglists



___
pmacct-discussion mailing list
http://www.pmacct.net/#mailinglists

Re: [pmacct-discussion] Only packets from router to netflow server

2016-08-19 Thread Jentsch, Mario
Hi Mattias,

do you have a drawing of your setup? I have to admit that it is unclear to me…

Thanks,
Mario

From: pmacct-discussion [mailto:pmacct-discussion-boun...@pmacct.net] On Behalf 
Of Mattias Larsson
Sent: Thursday, August 18, 2016 1:36 PM
To: pmacct-discussion@pmacct.net
Subject: [pmacct-discussion] Only packets from router to netflow server


I use a SPAN port on my switch to capture all netflow (udp 2055) packets and 
send it to a interface where my pmacct server has one extra interface connected 
to.

But when I look on the traffic/packets that pmacctd genereates it seems only be 
the IP packets between my router and netflow server. It seems it not decodes 
the cisco netflow payload/data.

When I do a tcpdump on the interface and look at it with wireshark I can see 
see the flows.

Any suggestion what I'm doing wrong?

Thanks in advance!

Mattias
___
pmacct-discussion mailing list
http://www.pmacct.net/#mailinglists

Re: [pmacct-discussion] collecting large number of netflows

2016-08-19 Thread Jentsch, Mario
Sounds like you have already have the DB server hardware.
It may be a good idea to simulate the data flows to and from your DB. Some 
scripts that insert data at different constant rates and/or intermittent as it 
comes from nfacctd normally generate the input to the DB. At the same time you 
prepare the next steps of processing with the fake data. This should reveal 
bottlenecks and give you the chance to address them before they appear in the 
live system.
E.g. using multiple Netflow collectors that write to the same tablespace may 
lock each other and decreasing insert performance. Same applies for reading the 
written data for further processing. Reducing the locks can be challenging, 
splitting the tablespace with partitioning or per collector separated inbound 
tables can help.

Good luck!
Mario

> -Original Message-
> From: pmacct-discussion [mailto:pmacct-discussion-boun...@pmacct.net]
> On Behalf Of Stephen Clark
> Sent: Thursday, August 18, 2016 2:24 PM
> To: pmacct-discussion@pmacct.net
> Subject: Re: [pmacct-discussion] collecting large number of netflows
> 
> On 08/17/2016 08:38 AM, Jentsch, Mario wrote:
> > Hey Steve,
> >
> > that question can't be answered without a lot of assumptions about the
> details of your project and we made the experience that even with project
> details it is a hard thing to predict due to the nature of network traffic
> patterns. Pmacct (namely nfacctd) can handle that number of flows - even
> with only one instance - and is most probably not the bottleneck. If it is
> possible what you plan to do, depends on questions like "how many records
> per timebin do you have after aggregation in nfacctd" - this is what your
> backend DB has to handle and "how is this data processed later on?" - this
> has more or less impact on DB performance and the time it takes to create
> reports or feed any user interfaces.
> >
> > Regards,
> > Mario
> Hi Mario,
> 
> Thanks for the response. We will be collecting data from about 200 probes.
> This
> is a new endeavor so I guess we be learning on the fly. We are planning on
> using
> fsrc sampling feature set at 20 flows per minute with inserts only into a
> postgresql 9.4 DB running on CentOS 6.8 in VMware on a hefty Cisco UCS
> system.
> 
> Regards,
> Steve
> >> -Original Message-
> >> From: pmacct-discussion [mailto:pmacct-discussion-
> boun...@pmacct.net]
> >> On Behalf Of Stephen Clark
> >> Sent: Thursday, August 04, 2016 5:01 PM
> >> To: pmacct-discussion@pmacct.net
> >> Subject: [pmacct-discussion] collecting large number of netflows
> >>
> >> Hi List,
> >>
> >> I am looking to collect a large number of netflow records, on the order of
> a
> >> 100
> >> million a day,
> >> and store them in a postgres DB. Has anyone done this or something
> similar
> >> using
> >> pmacct?
> >>
> >> Thanks,
> >> Steve
> >>
> >>
> 
> 
> ___
> pmacct-discussion mailing list
> http://www.pmacct.net/#mailinglists

___
pmacct-discussion mailing list
http://www.pmacct.net/#mailinglists