Re: [pmacct-discussion] Pmacct - conntrack - netflow v9

[rightkicktech.gmail.com]

Hi Paolo,

I was thinking if there is a way to capture the nflog-prefix with uacct. I was 
not able to find any relevant uacct key.

Probing conntrack and filtering the events is indeed interesting. I will try to 
see if the probing can be done in a way to save the long lasting sessions in 
small incremental steps.


On September 19, 2016 1:55:55 PM EEST, Paolo Lucente  wrote:
>
>Hi Alex,
>
>Inline:
>
>On Wed, Sep 14, 2016 at 12:55:00PM +0300, Abi Askushi wrote:
>
>> > > 1. Is there a pmacct plugin to get traffic flows from connection
>tracking
>> > > system, like ulogd2 with NFCT plugin?
>> >
>> > Not being familiar with this, can you elaborate what it does? An
>example
>> > would be much appreciated.
>> >
>> 
>> This is done using ulogd2 running with NFCT plugin. Then ulogd probes
>> events from connection tracking system (events can be filtered:
>destroy,
>> new, etc) and can print or store the flows in DB. The flows that can
>be
>> fetched are like the output of command "conntrack -L". The pro of
>this
>> approach is that you get the real source and destination when have to
>deal
>> with NATed traffic.
>> A very nice example is at
>>
>https://home.regit.org/2014/02/logging-connection-tracking-event-with-ulogd/.
>> The negative side of this approach is how to handle long lasting
>sessions
>> that are not fetched (at least I didn't figure out how to do that) in
>case
>> the device is rebooted, resulting in lost accounting traffic.
>
>I see, tt may be something potentially interesting. Do you think
>this is something you can contribute upon?
>
>> > > 2. NFLOG + uacctd: is there any way to aggregate/filter collected
>packets
>> > > with uacctd as received from NFLOG, according to the fwmark value
>set
>> > with
>> > > MARK at iptables ? If no, is there any recommended alternate
>approach?
>> >
>> > No, as i suspect this MARK action does not really mark/stamp the
>packet
>> > itself but mangles with an external header. But knowing more
>precisely
>> > what this MARK does, we can certainly make it an item we can tag
>upon,
>> > or more. Again, i'm not a master of ULOG/NFLOG and hence i'd need
>(your)
>> > support.
>> >
>> As you said, the MARK is an association that is done from netfilter
>and it
>> does not affect packet header.
>> This means that I'm left with the option to alter packet header to be
>able
>> to tag it.
>> Can you recommend which packet header to alter to be able to tag?
>
>I'd say the IP ToS field may be the most intuitive/easy one.
>Unfortunately
>it is rather intrusive, ie. you may be interested in the original ToS
>value. 
>
>Cheers,
>Paolo

-- 
Sent from my Android device with K-9 Mail. Please excuse my brevity.___
pmacct-discussion mailing list
http://www.pmacct.net/#mailinglists

Re: [pmacct-discussion] Pmacct - conntrack - netflow v9

[Abi Askushi]

Hi Paolo,

My comments inline.

Thanx,
Alex

On Tue, Sep 13, 2016 at 1:04 PM, Paolo Lucente  wrote:

>
> Hi Alex,
>
> Inline:
>
> On Sun, Sep 11, 2016 at 11:45:44PM +0300, Abi Askushi wrote:
>
> > 1. Is there a pmacct plugin to get traffic flows from connection tracking
> > system, like ulogd2 with NFCT plugin?
>
> Not being familiar with this, can you elaborate what it does? An example
> would be much appreciated.
>

This is done using ulogd2 running with NFCT plugin. Then ulogd probes
events from connection tracking system (events can be filtered: destroy,
new, etc) and can print or store the flows in DB. The flows that can be
fetched are like the output of command "conntrack -L". The pro of this
approach is that you get the real source and destination when have to deal
with NATed traffic.
A very nice example is at
https://home.regit.org/2014/02/logging-connection-tracking-event-with-ulogd/.
The negative side of this approach is how to handle long lasting sessions
that are not fetched (at least I didn't figure out how to do that) in case
the device is rebooted, resulting in lost accounting traffic.


> > 2. NFLOG + uacctd: is there any way to aggregate/filter collected packets
> > with uacctd as received from NFLOG, according to the fwmark value set
> with
> > MARK at iptables ? If no, is there any recommended alternate approach?
>
> No, as i suspect this MARK action does not really mark/stamp the packet
> itself but mangles with an external header. But knowing more precisely
> what this MARK does, we can certainly make it an item we can tag upon,
> or more. Again, i'm not a master of ULOG/NFLOG and hence i'd need (your)
> support.
>
As you said, the MARK is an association that is done from netfilter and it
does not affect packet header.
This means that I'm left with the option to alter packet header to be able
to tag it.
Can you recommend which packet header to alter to be able to tag?



>
> > 3. pmacctd Netflow v9 exports: when collecting flows with nfacctd
> generated
> > with pmacctd+nfprobe plugin, the interface index (in_iface, out_iface)
> was
> > showing always 0. Am I missing sth?
>
> Did you read the QUICKSTART document section "Quickstart guide to setup a
> NetFlow agent/probe"? Towards the end it starts speaking about interfaces,
> direction and tags. It essentially says: libpcap is detached from the OS
> and hence has no concept of interfaces and such; you need to issue a tag,
> ie. basing on source/destination MAC address, in order to populate the
> interface and/or direction fields of a generated NetFlow/IPFIX packet. Let
> me know if the case is you are already doing this and it's not working; if
> not (your config suggests you are not) here is a pointer to the doc:
>
> https://github.com/pmacct/pmacct/blob/master/QUICKSTART
>
> Seems I've missed that. Thank you for pointing out.


> ULOG/NFLOG is instead integrated in the Linux OS and hence would return you
> interfaces no problem.
>
Agree.


>
> Cheers,
> Paolo
>
>
> ___
> pmacct-discussion mailing list
> http://www.pmacct.net/#mailinglists
>
___
pmacct-discussion mailing list
http://www.pmacct.net/#mailinglists

Re: [pmacct-discussion] Pmacct - conntrack - netflow v9

[Paolo Lucente]

Hi Alex,

Inline:

On Sun, Sep 11, 2016 at 11:45:44PM +0300, Abi Askushi wrote:

> 1. Is there a pmacct plugin to get traffic flows from connection tracking
> system, like ulogd2 with NFCT plugin?

Not being familiar with this, can you elaborate what it does? An example
would be much appreciated.

> 2. NFLOG + uacctd: is there any way to aggregate/filter collected packets
> with uacctd as received from NFLOG, according to the fwmark value set with
> MARK at iptables ? If no, is there any recommended alternate approach?

No, as i suspect this MARK action does not really mark/stamp the packet
itself but mangles with an external header. But knowing more precisely
what this MARK does, we can certainly make it an item we can tag upon,
or more. Again, i'm not a master of ULOG/NFLOG and hence i'd need (your)
support.

> 3. pmacctd Netflow v9 exports: when collecting flows with nfacctd generated
> with pmacctd+nfprobe plugin, the interface index (in_iface, out_iface) was
> showing always 0. Am I missing sth?

Did you read the QUICKSTART document section "Quickstart guide to setup a
NetFlow agent/probe"? Towards the end it starts speaking about interfaces,
direction and tags. It essentially says: libpcap is detached from the OS
and hence has no concept of interfaces and such; you need to issue a tag,
ie. basing on source/destination MAC address, in order to populate the
interface and/or direction fields of a generated NetFlow/IPFIX packet. Let
me know if the case is you are already doing this and it's not working; if
not (your config suggests you are not) here is a pointer to the doc:

https://github.com/pmacct/pmacct/blob/master/QUICKSTART

ULOG/NFLOG is instead integrated in the Linux OS and hence would return you
interfaces no problem.

Cheers,
Paolo


___
pmacct-discussion mailing list
http://www.pmacct.net/#mailinglists