Re: [pmacct-discussion] Splitting In and Out traffic, and others questions

2014-06-25 Thread Paolo Lucente
Hi Raphael,

Addressing the last open point:

On Tue, Jun 24, 2014 at 01:00:37PM +0200, Raphael Mazelier wrote:

 So I am supposed to use v9 sql schema ? (I think tag is far more
 clear than agent_id).

Yes, agree and would recommend so. At least run sql_table_version: 9
and sql_table_type: bgp for the basic styling (ie. tags in the 'tag'
field rather than 'agent_id', etc.) then you can customize your table
(ie. which fields to include/exclude) with sql_optimize_clauses: true.

Cheers,
Paolo


___
pmacct-discussion mailing list
http://www.pmacct.net/#mailinglists


Re: [pmacct-discussion] Splitting In and Out traffic, and others questions

2014-06-25 Thread Raphael Mazelier

Hi Paolo,

Yes it work. Time to another query, in another thread for clarity :)
To the other point I stuck with my pretag filter based on interface.

Regards,


Le 25/06/2014 09:26, Paolo Lucente a écrit :

Hi Raphael,

Addressing the last open point:

On Tue, Jun 24, 2014 at 01:00:37PM +0200, Raphael Mazelier wrote:


So I am supposed to use v9 sql schema ? (I think tag is far more
clear than agent_id).


Yes, agree and would recommend so. At least run sql_table_version: 9
and sql_table_type: bgp for the basic styling (ie. tags in the 'tag'
field rather than 'agent_id', etc.) then you can customize your table
(ie. which fields to include/exclude) with sql_optimize_clauses: true.

Cheers,
Paolo




--
Raphael Mazelier
AS39605

___
pmacct-discussion mailing list
http://www.pmacct.net/#mailinglists


Re: [pmacct-discussion] Splitting In and Out traffic, and others questions

2014-06-24 Thread Raphael Mazelier

Hi Mario,

Well I'm using inline ipfix from Juniper.
Reading the juniper doc it seems that the ipv4-template does not include 
'direction' field.


I will stay with my current solution, using interface in and out.

Regards,



Le 24/06/2014 13:42, Jentsch, Mario a écrit :

Hi Raphael,

looks like the field direction is not set in your netflow v? data.

Depending on your devices that export the netflow data another way may be to 
export ingress and egress to different collector instances.

I can't say anything to the sql_plugin setup...

Regards,
Mario


-Original Message-
From: pmacct-discussion [mailto:pmacct-discussion-boun...@pmacct.net]
On Behalf Of Raphael Mazelier
Sent: Dienstag, 24. Juni 2014 13:01
To: pmacct-discussion@pmacct.net
Subject: Re: [pmacct-discussion] Splitting In and Out traffic, and others
questions

Hi Mario,

I try a pretag.map like this :

set_tag=100 ip=0.0.0.0/0 direction=0
set_tag=200 ip=0.0.0.0/0 direction=1

Unfortunately that did not work as expected :/

All my flow are tagged 100 (in) and so injected in my in table.

It's strange because quoting Paolo from another thread


You can use pre-tagging (pre_tag_map) to do it. How simple or how tricky
this is depends on the NetFlow version and exporter: 1) NetFlow v9 and

IPFIX

have a direction field (0 = ingress, 1 = egress)


This is exactly what I wanted.

To my other point, adding tag field in aggregate directive solve my
problem. This value is correctly reported to the agent_id sql column.

btw, I ve read in the changelog that the agent_id column was renamed
by tag in the last version.


SQL plugins: agent_id, agent_id2 fields renamed to tag, tag2. Issued SQL
 table schema #9 for agent_id backward compatibility. Renaming

agent_id2

 to tag2 is going to be disruptive to existing deployments instead.


So I am supposed to use v9 sql schema ? (I think tag is far more clear
than agent_id).

Thks.


Le 24/06/2014 10:32, Jentsch, Mario a écrit :

Hey Raphael,

we use the 1st tag to distinguish ingress and egress of IPv4 and IPv6:

! tag=1  - inbound IPv4 traffic
! tag=2  - outbound IPv4 traffic
! tag=3  - inbound IPv6 traffic
! tag=4  - outbound IPv6 traffic
!
set_tag=1 ip=0.0.0.0/0 direction=0 filter='ip'
set_tag=2 ip=0.0.0.0/0 direction=1 filter='ip'
set_tag=3 ip=0.0.0.0/0 direction=0 filter='ip6'
set_tag=4 ip=0.0.0.0/0 direction=1 filter='ip6'
set_tag=0 ip=0.0.0.0/0
!

This may also work for your setup...

Regards,
Mario


-Original Message-
From: pmacct-discussion [mailto:pmacct-discussion-

boun...@pmacct.net]

On Behalf Of Raphael Mazelier
Sent: Montag, 23. Juni 2014 14:31
To: pmacct-discussion@pmacct.net
Subject: [pmacct-discussion] Splitting In and Out traffic, and others

questions


Hi Paolo, All,

First I would thank you Paolo for this great piece of software !
Thanks to my predecessor (hi Pym) I already have a working pmacctd
installation which doing accounting on my network :)

I have some questions tough :

I have enabled inbound accounting in my network.
I want to distinguish in and out traffic.
For now I make something like this, using pre_tag filter :

# more /etc/pmacct/pretag.map
set_tag=100 ip=158.58.176.2 in=527
set_tag=100 ip=158.58.176.2 in=528
set_tag=100 ip=158.58.176.2 in=530
...

set_tag=200 ip=158.58.176.2 out=527
set_tag=200 ip=158.58.176.2 out=528
set_tag=200 ip=158.58.176.2 out=530
...

# more /etc/pmacct/nfacctd.conf

...
pre_tag_filter[in_hour]: 100
pre_tag_filter[out_hour]: 200
...

! sql outbound by hour
sql_refresh_time[out_hour]: 300
sql_history[out_hour]: 5m
sql_history_roundoff[out_hour]: m
sql_table[out_hour]: netflow_out_hour_%Y%m%d_%H
sql_table_schema[out_hour]: /etc/pmacct/netflow_out_hour.schema

! sql inbound by hour
sql_refresh_time[in_hour]: 300
sql_history[in_hour]: 5m
sql_history_roundoff[in_hour]: m
sql_table[in_hour]: netflow_in_hour_%Y%m%d_%H
sql_table_schema[in_hour]: /etc/pmacct/netflow_in_hour.schema


It's working well, but I wonder if it exists another, more clear/simpler
method ? because I have to maintain the pretag.map.
Or perhaps I could mix In an Out flux in the sql table (but make the
table much bigger).

Side question about pretag filter ? the tag field in sql is always at
'0' ? This is not blocking but I wonder why ?

Another question about BGP src_as and dst_as fields :
Depending on the direction the src_as or the dst_as are correclty
filled, but not the other which is always '0' ? I would assume that it
will be my As number ? Should I have to deal with network filter ?


I have many other questions, but for now I think that is sufficient :)

best,


--
Raphael Mazelier
AS39605














___
pmacct-discussion mailing list
http://www.pmacct.net/#mailinglists


___
pmacct-discussion mailing list
http://www.pmacct.net/#mailinglists



___
pmacct-discussion mailing list
http://www.pmacct.net

Re: [pmacct-discussion] Splitting In and Out traffic, and others questions

2014-06-23 Thread Paolo Lucente
Hi Raphael,

Thanks for your kind words about the pmacct project. 

In-line:

On Mon, Jun 23, 2014 at 02:30:35PM +0200, Raphael Mazelier wrote:

 It's working well, but I wonder if it exists another, more
 clear/simpler method ? because I have to maintain the pretag.map.
 Or perhaps I could mix In an Out flux in the sql table (but make the
 table much bigger).

For sure you have to maintain a map to say what is input and what
is output - would be great to find a way that is as most static as
possible for you. What comes to mind for the purpose - all depends
whether you have downstream ASNs, get at least a BGP feed or get
src_as and dst_as populated from NetFlow, get MAC addresses from
NetFlow, etc. - is you can use ASNs, IP prefixes, MAC addresses or
interfaces (this last one is what you are doing at present). 

For example, should you not have downstream ASNs and get src_as and
dst_as correctly populated by your router(s) via NetFlow you could
simply match input traffic as dst_as=0 and output traffic as src_as=0
in your pre_tag_map.

 Side question about pretag filter ? the tag field in sql is always
 at '0' ? This is not blocking but I wonder why ?

Is 'tag' part of your aggregation scheme, ie. 'aggregate' keyword
in your config? If not, then that's the reason and zero is simply
the default value imposed to the field in the SQL schema. 

 Another question about BGP src_as and dst_as fields :
 Depending on the direction the src_as or the dst_as are correclty
 filled, but not the other which is always '0' ? I would assume that
 it will be my As number ? Should I have to deal with network filter
 ?

Correct, when the ASN is zero then it's traffic delivered to/sourced
by your own IP address space. You won't see your own ASN number being
filled in - just like you don't see it in your own BGP routing table.
But you can make some tricks, ie. use a networks_map, to do that. Let
me know if interested.

Cheers,
Paolo

___
pmacct-discussion mailing list
http://www.pmacct.net/#mailinglists