Re: [openstack-dev] Change in openstack/neutron[master]: Add method to get iptables traffic counters

2013-07-23 Thread Sylvain Afchain
Hi Brian,

I just realized that my example in the comment of the review was not so good.
The currently implementation of the driver already does what you propose.

The driver addresses this case:

Two labels rules matches a packet; a CIDR of a rule overlaps another one:

iptables -N test1
iptables -N test2
iptables -A test2
iptables -A OUTPUT -j test1

iptables -A test1 -d 8.8.8.0/27 -j test2
iptables -A test1 -d 8.8.8.0/24 -j test2

I could remove the mark, and add a constraint on the plugin's side to avoid the 
overlapping.

Thoughts?

Thanks,

Sylvain.


- Original Message -
From: Brian Haley brian.ha...@hp.com
To: Sylvain Afchain sylvain.afch...@enovance.com
Cc: openstack-dev@lists.openstack.org
Sent: Monday, July 22, 2013 10:30:32 PM
Subject: Re: Change in openstack/neutron[master]: Add method to get iptables 
traffic counters

Sylvain,

Something like this would require no marking:

# iptables -N test2
# iptables -N test3
# iptables -A test3
# iptables -A test2 -d 9.9.9.9/32 -j RETURN
# iptables -A test2 -d 10.10.10.10/32 -j RETURN
# iptables -A test2 -j test3
# iptables -A OUTPUT -j test2

# ping -I eth0 -r 9.9.9.9
PING 9.9.9.9 (9.9.9.9) from 16.1.1.40 eth0: 56(84) bytes of data.
^C
--- 9.9.9.9 ping statistics ---
2 packets transmitted, 0 received, 100% packet loss, time 1007ms

# iptables-save -c | grep test
:test2 - [0:0]
:test3 - [0:0]
[3198:403274] -A OUTPUT -j test2
[2:168] -A test2 -d 9.9.9.9/32 -j RETURN
[0:0] -A test2 -d 10.10.10.10/32 -j RETURN
[3196:403106] -A test2 -j test3
[3196:403106] -A test3

# iptables -L test2 -v -x -n
Chain test2 (1 references)
pkts  bytes target prot opt in out source
destination
   2  168 RETURN all  --  *  *   0.0.0.0/0
9.9.9.9
   00 RETURN all  --  *  *   0.0.0.0/0
10.10.10.10
3182   401554 test3  all  --  *  *   0.0.0.0/0
0.0.0.0/0

 iptables -L test3 -v -x -n
Chain test3 (1 references)
pkts  bytes target prot opt in out source
destination
3182   401554all  --  *  *   0.0.0.0/0
0.0.0.0/0

And I seems similar to your cut/paste from below.

Thoughts?

-Brian

On 07/22/2013 03:55 AM, Sylvain Afchain wrote:
 Hi Brian,
 
 Thanks for your reply.
 
 1. This isn't something a tenant should be able to do, so should be 
 admin-only,
 correct?
 
 Correct.
 
 2. I think it would be useful for an admin to be able to add metering rules 
 for
 all tenants with a single command.  This gets back to wanting to pre-seed an 
 ini
 file with a set of subnets, then add/subtract from it later without 
 restarting
 the daemon.
 
 I agree with you, could be a future enhancement.
 
 3. I think it would be better if you didn't mark the packets, for performance
 reasons.  If you were marking them on input to be matched by something on 
 output
 I'd feel different, but for just counting bytes we should be able to do it
 another way.  I can get back to you next week on figuring this out.
 
 Ok, I'll take a look too.
 
 Thanks.
 
 Sylvain.
 
 - Original Message -
 From: Brian Haley brian.ha...@hp.com
 To: Sylvain Afchain sylvain.afch...@enovance.com
 Cc: openstack-dev@lists.openstack.org
 Sent: Friday, July 19, 2013 11:47:41 PM
 Subject: Re: Change in openstack/neutron[master]: Add method to get iptables 
 traffic counters
 
 Hi Sylvain,
 
 Sorry for the slow reply, I'll have to look closer next week, but I did have
 some comments.
 
 1. This isn't something a tenant should be able to do, so should be 
 admin-only,
 correct?
 
 2. I think it would be useful for an admin to be able to add metering rules 
 for
 all tenants with a single command.  This gets back to wanting to pre-seed an 
 ini
 file with a set of subnets, then add/subtract from it later without restarting
 the daemon.
 
 3. I think it would be better if you didn't mark the packets, for performance
 reasons.  If you were marking them on input to be matched by something on 
 output
 I'd feel different, but for just counting bytes we should be able to do it
 another way.  I can get back to you next week on figuring this out.
 
 Thanks,
 
 -Brian
 
 On 07/18/2013 04:29 AM, Sylvain Afchain wrote:
 Hi Brian,

 For iptables rules, see below

 Yes the only way to setup metering labels/rules is the neutronclient. I 
 agree with you about the future
 enhancement.

 Regards,

 Sylvain

 - Original Message -
 From: Brian Haley brian.ha...@hp.com
 To: Sylvain Afchain sylvain.afch...@enovance.com
 Cc: openstack-dev@lists.openstack.org
 Sent: Thursday, July 18, 2013 4:58:26 AM
 Subject: Re: Change in openstack/neutron[master]: Add method to get iptables 
 traffic counters

 Hi Sylvain,

 I think I've caught-up with all your reviews, but I still did have some
 questions on the iptables rules, below.

 One other question, and maybe it's simply a future enhancement, but is the 
 only
 way to setup these meters using neutronclient?  I think being able to 
 specify
 these in 

Re: [openstack-dev] Change in openstack/neutron[master]: Add method to get iptables traffic counters

2013-07-22 Thread Brian Haley
Sylvain,

Something like this would require no marking:

# iptables -N test2
# iptables -N test3
# iptables -A test3
# iptables -A test2 -d 9.9.9.9/32 -j RETURN
# iptables -A test2 -d 10.10.10.10/32 -j RETURN
# iptables -A test2 -j test3
# iptables -A OUTPUT -j test2

# ping -I eth0 -r 9.9.9.9
PING 9.9.9.9 (9.9.9.9) from 16.1.1.40 eth0: 56(84) bytes of data.
^C
--- 9.9.9.9 ping statistics ---
2 packets transmitted, 0 received, 100% packet loss, time 1007ms

# iptables-save -c | grep test
:test2 - [0:0]
:test3 - [0:0]
[3198:403274] -A OUTPUT -j test2
[2:168] -A test2 -d 9.9.9.9/32 -j RETURN
[0:0] -A test2 -d 10.10.10.10/32 -j RETURN
[3196:403106] -A test2 -j test3
[3196:403106] -A test3

# iptables -L test2 -v -x -n
Chain test2 (1 references)
pkts  bytes target prot opt in out source
destination
   2  168 RETURN all  --  *  *   0.0.0.0/0
9.9.9.9
   00 RETURN all  --  *  *   0.0.0.0/0
10.10.10.10
3182   401554 test3  all  --  *  *   0.0.0.0/0
0.0.0.0/0

 iptables -L test3 -v -x -n
Chain test3 (1 references)
pkts  bytes target prot opt in out source
destination
3182   401554all  --  *  *   0.0.0.0/0
0.0.0.0/0

And I seems similar to your cut/paste from below.

Thoughts?

-Brian

On 07/22/2013 03:55 AM, Sylvain Afchain wrote:
 Hi Brian,
 
 Thanks for your reply.
 
 1. This isn't something a tenant should be able to do, so should be 
 admin-only,
 correct?
 
 Correct.
 
 2. I think it would be useful for an admin to be able to add metering rules 
 for
 all tenants with a single command.  This gets back to wanting to pre-seed an 
 ini
 file with a set of subnets, then add/subtract from it later without 
 restarting
 the daemon.
 
 I agree with you, could be a future enhancement.
 
 3. I think it would be better if you didn't mark the packets, for performance
 reasons.  If you were marking them on input to be matched by something on 
 output
 I'd feel different, but for just counting bytes we should be able to do it
 another way.  I can get back to you next week on figuring this out.
 
 Ok, I'll take a look too.
 
 Thanks.
 
 Sylvain.
 
 - Original Message -
 From: Brian Haley brian.ha...@hp.com
 To: Sylvain Afchain sylvain.afch...@enovance.com
 Cc: openstack-dev@lists.openstack.org
 Sent: Friday, July 19, 2013 11:47:41 PM
 Subject: Re: Change in openstack/neutron[master]: Add method to get iptables 
 traffic counters
 
 Hi Sylvain,
 
 Sorry for the slow reply, I'll have to look closer next week, but I did have
 some comments.
 
 1. This isn't something a tenant should be able to do, so should be 
 admin-only,
 correct?
 
 2. I think it would be useful for an admin to be able to add metering rules 
 for
 all tenants with a single command.  This gets back to wanting to pre-seed an 
 ini
 file with a set of subnets, then add/subtract from it later without restarting
 the daemon.
 
 3. I think it would be better if you didn't mark the packets, for performance
 reasons.  If you were marking them on input to be matched by something on 
 output
 I'd feel different, but for just counting bytes we should be able to do it
 another way.  I can get back to you next week on figuring this out.
 
 Thanks,
 
 -Brian
 
 On 07/18/2013 04:29 AM, Sylvain Afchain wrote:
 Hi Brian,

 For iptables rules, see below

 Yes the only way to setup metering labels/rules is the neutronclient. I 
 agree with you about the future
 enhancement.

 Regards,

 Sylvain

 - Original Message -
 From: Brian Haley brian.ha...@hp.com
 To: Sylvain Afchain sylvain.afch...@enovance.com
 Cc: openstack-dev@lists.openstack.org
 Sent: Thursday, July 18, 2013 4:58:26 AM
 Subject: Re: Change in openstack/neutron[master]: Add method to get iptables 
 traffic counters

 Hi Sylvain,

 I think I've caught-up with all your reviews, but I still did have some
 questions on the iptables rules, below.

 One other question, and maybe it's simply a future enhancement, but is the 
 only
 way to setup these meters using neutronclient?  I think being able to 
 specify
 these in an .ini file would be good as well, which is something I'd want to 
 do
 as a provider, such that they're always there, and actually not visible to 
 the
 tenant.

 On 07/11/2013 10:04 AM, Sylvain Afchain wrote:
 Hi Brian,

 You're right It could be easier with your approach to get and keep the 
 traffic counters.

 I will add a new method to get the details of traffic counters of a chain.
 https://review.openstack.org/35624

 Thoughts?

 Regards,

 Sylvain.

 - Original Message -
 From: Sylvain Afchain sylvain.afch...@enovance.com
 To: Brian Haley brian.ha...@hp.com
 Cc: openstack-dev@lists.openstack.org
 Sent: Thursday, July 11, 2013 11:19:39 AM
 Subject: Re: Change in openstack/neutron[master]: Add method to get 
 iptables traffic counters

 Hi Brian,

 First thanks for the reviews and your detailed email.

 Second I will update 

Re: [openstack-dev] Change in openstack/neutron[master]: Add method to get iptables traffic counters

2013-07-19 Thread Brian Haley
Hi Sylvain,

Sorry for the slow reply, I'll have to look closer next week, but I did have
some comments.

1. This isn't something a tenant should be able to do, so should be admin-only,
correct?

2. I think it would be useful for an admin to be able to add metering rules for
all tenants with a single command.  This gets back to wanting to pre-seed an ini
file with a set of subnets, then add/subtract from it later without restarting
the daemon.

3. I think it would be better if you didn't mark the packets, for performance
reasons.  If you were marking them on input to be matched by something on output
I'd feel different, but for just counting bytes we should be able to do it
another way.  I can get back to you next week on figuring this out.

Thanks,

-Brian

On 07/18/2013 04:29 AM, Sylvain Afchain wrote:
 Hi Brian,
 
 For iptables rules, see below
 
 Yes the only way to setup metering labels/rules is the neutronclient. I agree 
 with you about the future
 enhancement.
 
 Regards,
 
 Sylvain
 
 - Original Message -
 From: Brian Haley brian.ha...@hp.com
 To: Sylvain Afchain sylvain.afch...@enovance.com
 Cc: openstack-dev@lists.openstack.org
 Sent: Thursday, July 18, 2013 4:58:26 AM
 Subject: Re: Change in openstack/neutron[master]: Add method to get iptables 
 traffic counters
 
 Hi Sylvain,

 I think I've caught-up with all your reviews, but I still did have some
 questions on the iptables rules, below.

 One other question, and maybe it's simply a future enhancement, but is the 
 only
 way to setup these meters using neutronclient?  I think being able to specify
 these in an .ini file would be good as well, which is something I'd want to 
 do
 as a provider, such that they're always there, and actually not visible to 
 the
 tenant.

 On 07/11/2013 10:04 AM, Sylvain Afchain wrote:
 Hi Brian,

 You're right It could be easier with your approach to get and keep the 
 traffic counters.

 I will add a new method to get the details of traffic counters of a chain.
 https://review.openstack.org/35624

 Thoughts?

 Regards,

 Sylvain.

 - Original Message -
 From: Sylvain Afchain sylvain.afch...@enovance.com
 To: Brian Haley brian.ha...@hp.com
 Cc: openstack-dev@lists.openstack.org
 Sent: Thursday, July 11, 2013 11:19:39 AM
 Subject: Re: Change in openstack/neutron[master]: Add method to get 
 iptables traffic counters

 Hi Brian,

 First thanks for the reviews and your detailed email.

 Second I will update the blueprint specs. as soon as possible, but for 
 example it will look like that:

 Chain FORWARD (policy ACCEPT 0 packets, 0 bytes)
 pkts  bytes target prot opt in out source   
 destination 
55   245 metering-r-aef1456343  all  --  *  *   
 0.0.0.0/00.0.0.0/0   /* jump to rules the label aef1456343 */   
  
55   245 metering-r-badf566782  all  --  *  *   
 0.0.0.0/00.0.0.0/0   
 
 So are these two used to separate out what you don't want to count from what 
 you
 want to count?  Seems the jump to the r-aef1456343 will filter, then the
 r-badf566782 will count per-subnet?  I'm just trying to understand why you're
 splitting the two up.
 
 No here, there are two rules only because there are two labels. In the chain 
 of each 
 label you will find the label's rules. 
 
 Chain metering-l-aef1456343 (1 references)  /* the chain for the label 
 aef1456343 */
 pkts  bytes target prot opt in out source   
 destination 

 Chain metering-l-badf566782 (1 references)  /* the chain for the label 
 badf566782 */
 pkts  bytes target prot opt in out source   
 destination  
 
 These two chains aren't really doing anything, and I believe their 
 packet/byte
 counts would be duplicated in the calling rules, correct?  If that's the 
 case I
 don't see the reason to jump to them.  Our performance person always reminds 
 me
 when I increase path length by doing things like this, so removing 
 unnecessary
 things is one of my goals.  Of course we're doing more work here to count
 things, but that needs to be done.
 
 I recently change this(according to your remarks on iptables accounting), so 
 now there is a 
 rule which is used to count the traffic for a label. A mark is added one this 
 rule to be 
 sure to not count it twice. You can check the metering iptables drivers.
 https://review.openstack.org/#/c/36813/
 
 Chain metering-r-aef1456343 (1 references)
 pkts  bytes target prot opt in out source   
 destination 
20 100 RETURN all  --  *  *   0.0.0.0/0   
 !10.0.0.0/24  /* don't want to count this traffic */   
00 RETURN all  --  *  *   0.0.0.0/0   
 !20.0.0.0/24  /* don't want to count this traffic */   
25  145 metering-l-aef1456343  all  --  *  *   0.0.0.0/0 
0.0.0.0/0   

Re: [openstack-dev] Change in openstack/neutron[master]: Add method to get iptables traffic counters

2013-07-18 Thread Sylvain Afchain
Hi Brian,

For iptables rules, see below

Yes the only way to setup metering labels/rules is the neutronclient. I agree 
with you about the future
enhancement.

Regards,

Sylvain

- Original Message -
From: Brian Haley brian.ha...@hp.com
To: Sylvain Afchain sylvain.afch...@enovance.com
Cc: openstack-dev@lists.openstack.org
Sent: Thursday, July 18, 2013 4:58:26 AM
Subject: Re: Change in openstack/neutron[master]: Add method to get iptables 
traffic counters

 Hi Sylvain,

 I think I've caught-up with all your reviews, but I still did have some
 questions on the iptables rules, below.
 
 One other question, and maybe it's simply a future enhancement, but is the 
 only
 way to setup these meters using neutronclient?  I think being able to specify
 these in an .ini file would be good as well, which is something I'd want to do
 as a provider, such that they're always there, and actually not visible to the
 tenant.
 
 On 07/11/2013 10:04 AM, Sylvain Afchain wrote:
 Hi Brian,
 
 You're right It could be easier with your approach to get and keep the 
 traffic counters.
 
 I will add a new method to get the details of traffic counters of a chain.
 https://review.openstack.org/35624
 
 Thoughts?
 
 Regards,
 
 Sylvain.
 
 - Original Message -
 From: Sylvain Afchain sylvain.afch...@enovance.com
 To: Brian Haley brian.ha...@hp.com
 Cc: openstack-dev@lists.openstack.org
 Sent: Thursday, July 11, 2013 11:19:39 AM
 Subject: Re: Change in openstack/neutron[master]: Add method to get iptables 
 traffic counters
 
 Hi Brian,
 
 First thanks for the reviews and your detailed email.
 
 Second I will update the blueprint specs. as soon as possible, but for 
 example it will look like that:
 
 Chain FORWARD (policy ACCEPT 0 packets, 0 bytes)
 pkts  bytes target prot opt in out source   
 destination 
55   245 metering-r-aef1456343  all  --  *  *   0.0.0.0/0 
0.0.0.0/0   /* jump to rules the label aef1456343 */
55   245 metering-r-badf566782  all  --  *  *   0.0.0.0/0 
0.0.0.0/0   

 So are these two used to separate out what you don't want to count from what 
 you
 want to count?  Seems the jump to the r-aef1456343 will filter, then the
 r-badf566782 will count per-subnet?  I'm just trying to understand why you're
 splitting the two up.

No here, there are two rules only because there are two labels. In the chain of 
each 
label you will find the label's rules. 

 Chain metering-l-aef1456343 (1 references)   /* the chain for the label 
 aef1456343 */
 pkts  bytes target prot opt in out source   
 destination 
 
 Chain metering-l-badf566782 (1 references)   /* the chain for the label 
 badf566782 */
 pkts  bytes target prot opt in out source   
 destination  

 These two chains aren't really doing anything, and I believe their packet/byte
 counts would be duplicated in the calling rules, correct?  If that's the case 
 I
 don't see the reason to jump to them.  Our performance person always reminds 
 me
 when I increase path length by doing things like this, so removing unnecessary
 things is one of my goals.  Of course we're doing more work here to count
 things, but that needs to be done.

I recently change this(according to your remarks on iptables accounting), so 
now there is a 
rule which is used to count the traffic for a label. A mark is added one this 
rule to be 
sure to not count it twice. You can check the metering iptables drivers.
https://review.openstack.org/#/c/36813/

 Chain metering-r-aef1456343 (1 references)
 pkts  bytes target prot opt in out source   
 destination 
20 100 RETURN all  --  *  *   0.0.0.0/0   
 !10.0.0.0/24  /* don't want to count this traffic */   
00 RETURN all  --  *  *   0.0.0.0/0   
 !20.0.0.0/24  /* don't want to count this traffic */   
25  145 metering-l-aef1456343  all  --  *  *   0.0.0.0/0  
   0.0.0.0/0 /* count the remaining traffic */ 

 This has excluded (!) certain subnets, then allowed anything else to pass 
 through.

Correct, I don't want to count the traffic in/out for these network and I want 
to count anything else.

 Chain metering-r-badf566782 (1 references)
 pkts  bytes target prot opt in out source   
 destination 
00 metering-l-badf56678  all  --  *  *   0.0.0.0/0
 30.0.0.0/24 /* want to count only this */

 This is only counting packets going to a certain subnet.

Correct.

 I'm still trying to get all these changes running in devstack, so sorry for 
 all
 the questions.

No problem, and thanks for your reviews.

Sylvain.

 Thanks,

 -Brian

 
 Of course the in/out interfaces will be set in order to get the ingress or 
 the egress 

Re: [openstack-dev] Change in openstack/neutron[master]: Add method to get iptables traffic counters

2013-07-17 Thread Brian Haley
Hi Sylvain,

I think I've caught-up with all your reviews, but I still did have some
questions on the iptables rules, below.

One other question, and maybe it's simply a future enhancement, but is the only
way to setup these meters using neutronclient?  I think being able to specify
these in an .ini file would be good as well, which is something I'd want to do
as a provider, such that they're always there, and actually not visible to the
tenant.

On 07/11/2013 10:04 AM, Sylvain Afchain wrote:
 Hi Brian,
 
 You're right It could be easier with your approach to get and keep the 
 traffic counters.
 
 I will add a new method to get the details of traffic counters of a chain.
 https://review.openstack.org/35624
 
 Thoughts?
 
 Regards,
 
 Sylvain.
 
 - Original Message -
 From: Sylvain Afchain sylvain.afch...@enovance.com
 To: Brian Haley brian.ha...@hp.com
 Cc: openstack-dev@lists.openstack.org
 Sent: Thursday, July 11, 2013 11:19:39 AM
 Subject: Re: Change in openstack/neutron[master]: Add method to get iptables 
 traffic counters
 
 Hi Brian,
 
 First thanks for the reviews and your detailed email.
 
 Second I will update the blueprint specs. as soon as possible, but for 
 example it will look like that:
 
 Chain FORWARD (policy ACCEPT 0 packets, 0 bytes)
 pkts  bytes target prot opt in out source   
 destination 
55   245 metering-r-aef1456343  all  --  *  *   0.0.0.0/0  
   0.0.0.0/0   /* jump to rules the label aef1456343 */
55   245 metering-r-badf566782  all  --  *  *   0.0.0.0/0  
   0.0.0.0/0   

So are these two used to separate out what you don't want to count from what you
want to count?  Seems the jump to the r-aef1456343 will filter, then the
r-badf566782 will count per-subnet?  I'm just trying to understand why you're
splitting the two up.

 Chain metering-l-aef1456343 (1 references)/* the chain for the label 
 aef1456343 */
 pkts  bytes target prot opt in out source   
 destination 
 
 Chain metering-l-badf566782 (1 references)/* the chain for the label 
 badf566782 */
 pkts  bytes target prot opt in out source   
 destination  

These two chains aren't really doing anything, and I believe their packet/byte
counts would be duplicated in the calling rules, correct?  If that's the case I
don't see the reason to jump to them.  Our performance person always reminds me
when I increase path length by doing things like this, so removing unnecessary
things is one of my goals.  Of course we're doing more work here to count
things, but that needs to be done.

 Chain metering-r-aef1456343 (1 references)
 pkts  bytes target prot opt in out source   
 destination 
20 100 RETURN all  --  *  *   0.0.0.0/0   
 !10.0.0.0/24  /* don't want to count this traffic */   
00 RETURN all  --  *  *   0.0.0.0/0   
 !20.0.0.0/24  /* don't want to count this traffic */   
25  145 metering-l-aef1456343  all  --  *  *   0.0.0.0/0   
  0.0.0.0/0  /* count the remaining traffic */ 

This has excluded (!) certain subnets, then allowed anything else to pass 
through.

 Chain metering-r-badf566782 (1 references)
 pkts  bytes target prot opt in out source   
 destination 
00 metering-l-badf56678  all  --  *  *   0.0.0.0/0 
30.0.0.0/24 /* want to count only this */

This is only counting packets going to a certain subnet.

I'm still trying to get all these changes running in devstack, so sorry for all
the questions.

Thanks,

-Brian

 
 Of course the in/out interfaces will be set in order to get the ingress or 
 the egress traffic.
 
 I agree with you I could add a single rule to the chain of the label and get 
 the traffic of the first entry, though I found this approach less generic. 
 I mean, to be forced to add a rule at the top of a chain to get its traffic. 
 My approach is I don't want the counters of a specific rule but I want to 
 count
 the traffic going through the chain.
 
 Thoughts?
 
 Regards,
 
 Sylvain.
 
 - Original Message -
 From: Brian Haley brian.ha...@hp.com
 To: sylvain afchain sylvain.afch...@enovance.com
 Cc: openstack-dev@lists.openstack.org
 Sent: Thursday, July 11, 2013 2:30:24 AM
 Subject: Re: Change in openstack/neutron[master]: Add method to get iptables 
 traffic counters
 
 On 07/08/2013 01:10 PM, Sylvain Afchain (Code Review) wrote:
 Sylvain Afchain has posted comments on this change.

 Change subject: Add method to get iptables traffic counters
 snip
 --
 To view, visit https://review.openstack.org/35624
 
 Hi Sylvain,
 
 Instead of trying to ask questions directly in the review itself (since it 
 will mess-up formatting) I'll just send this to you and the list 

Re: [openstack-dev] Change in openstack/neutron[master]: Add method to get iptables traffic counters

2013-07-11 Thread Sylvain Afchain
Hi Brian,

First thanks for the reviews and your detailed email.

Second I will update the blueprint specs. as soon as possible, but for example 
it will look like that:

Chain FORWARD (policy ACCEPT 0 packets, 0 bytes)
pkts  bytes target prot opt in out source   
destination 
   55   245 metering-r-aef1456343  all  --  *  *   0.0.0.0/0
0.0.0.0/0   /* jump to rules the label aef1456343 */
   55   245 metering-r-badf566782  all  --  *  *   0.0.0.0/0
0.0.0.0/0   

Chain metering-l-aef1456343 (1 references)  /* the chain for the label 
aef1456343 */
pkts  bytes target prot opt in out source   
destination 

Chain metering-l-badf566782 (1 references)  /* the chain for the label 
badf566782 */
pkts  bytes target prot opt in out source   
destination 

Chain metering-r-aef1456343 (1 references)
pkts  bytes target prot opt in out source   
destination 
   20 100 RETURN all  --  *  *   0.0.0.0/0   
!10.0.0.0/24  /* don't want to count this traffic */   
   00 RETURN all  --  *  *   0.0.0.0/0   
!20.0.0.0/24  /* don't want to count this traffic */   
   25  145 metering-l-aef1456343  all  --  *  *   0.0.0.0/0 
   0.0.0.0/0/* count the remaining traffic */ 

Chain metering-r-badf566782 (1 references)
pkts  bytes target prot opt in out source   
destination 
   00 metering-l-badf56678  all  --  *  *   0.0.0.0/0   
 30.0.0.0/24 /* want to count only this */


Of course the in/out interfaces will be set in order to get the ingress or the 
egress traffic.

I agree with you I could add a single rule to the chain of the label and get 
the traffic of the first entry, though I found this approach less generic. 
I mean, to be forced to add a rule at the top of a chain to get its traffic. My 
approach is I don't want the counters of a specific rule but I want to count
the traffic going through the chain.

Thoughts?

Regards,

Sylvain.

- Original Message -
From: Brian Haley brian.ha...@hp.com
To: sylvain afchain sylvain.afch...@enovance.com
Cc: openstack-dev@lists.openstack.org
Sent: Thursday, July 11, 2013 2:30:24 AM
Subject: Re: Change in openstack/neutron[master]: Add method to get iptables 
traffic counters

On 07/08/2013 01:10 PM, Sylvain Afchain (Code Review) wrote:
 Sylvain Afchain has posted comments on this change.
 
 Change subject: Add method to get iptables traffic counters
snip
 --
 To view, visit https://review.openstack.org/35624

Hi Sylvain,

Instead of trying to ask questions directly in the review itself (since it will 
mess-up formatting) I'll just send this to you and the list since I had some 
questions on the traffic counter changes you've been doing.

First, thanks for working on this, it's definitely something I'm interested in, 
and I'm trying to review all your changes.

Second, do you have more than just the short description from the blueprint for 
how the iptables chains/rules will look like when created?  I'm still a little 
confused with this change (above) and how it's matching chains to get 
packet/byte statistics.  I'm thinking it can be done within a single chain so 
that you can do an 'iptables -L $chain' call to get just what you need, instead 
of parsing the entire table.

For example, I did something similar in Nova (out of tree), and it used a 
single chain per-VM, such that you could get it's statistics with a single 
iptables call like:

(sorry if this wraps)
$ sudo iptables -t mangle -L nova-meter-output-91 -n -v -x [-Z]
Chain nova-meter-output-91 (1 references)
pkts  bytes target prot opt in out source   
destination 
  805210 247931149all  --  *  *   0.0.0.0/0
0.0.0.0/0/* inst-91 packets transmitted total */ 
   15510   964648 all  --  *  *   0.0.0.0/0
x.y.0.0/16
   21282  3075403 all  --  *  *   0.0.0.0/0
x.z.0.0/16
   [...]

None of the rules in the chain has a jump target, so they simply count 
packets/bytes as they pass through, and the chain is called from a single 
location based on IP address, so in iptables-save format it looks like this:

-A nova-meter-output -s $my_ip/32 -i bridge1 -j nova-meter-output-91 
-A nova-meter-output-91 -m comment --comment inst-91 packets transmitted total
-A nova-meter-output-91 -d x.y.0.0/16
-A nova-meter-output-91 -d x.z.0.0/16
[...]

Obviously with Neutron, and doing this at the router egress, things change, but 
I think it could still be done in a single OUTPUT chain in the filter table.

Thoughts?

-Brian

___
OpenStack-dev mailing list