On Wed, 25 Nov 2009, Cesar Delgado wrote:
> Venu,
>
> On Nov 25, 2009, at 9:49 AM, venugopal iyer wrote:
>
>>
>> Hi, Cesar:
>>
>> On Tue, 24 Nov 2009, Cesar Delgado wrote:
>>
>>> Venugopal,
>>>
>>> I'm sorry if these sounds like basic questions. I really appreciate the
>>> patience and the help. Replies in-line.
>>>
>>> On Nov 24, 2009, at 9:29 AM, venugopal iyer wrote:
>>>
>>>>
>>>>
>>>> Hi, Cesar:
>>>>
>>>> On Mon, 23 Nov 2009, Cesar Delgado wrote:
>>>>
>>>>> I'm setting up a server to go to a hosting site where I have a 1Mbps
>>>>> pipe. From what I read I know I can't set the limit to this as the
>>>>> lowest setting is ~1.2Mbps and this is something that's getting worked on
>>>>> in Crossbow2. I am seeing some strange behavior.
>>>>>
>>>>> FIrst I have a question on flowadm's show-usage command. When I try to
>>>>> run show-prop with the name of a flow I get an error. The flow exists.
>>>>> What am I doing wrong?
>>>>>
>>>>> root at myhost:~# flowadm show-usage -f /var/log/net.log http-flow
>>>>> flowadm: invalid flow: '(null)'
>>>>
>>>> This is a bug, I have submitted
>>>>
>>>> 6904427 flowadm show-usage doesn't work with a flow name
>>>
>>> Thanks for submitting that. I haven't been able to find a link to the
>>> bugtracker for Crossbow. Could you please send me the URL?
>>
>> I think it should show up on
>> http://bugs.opensolaris.org/bugdatabase/index.jsp soon.
>>
>>>>
>>>>
>>>>>
>>>>>
>>>>> Ok, now for my problem. I have the following setting:
>>>>>
>>>>> root at myhost:~# flowadm show-flowprop http-flow
>>>>> FLOW PROPERTY VALUE DEFAULT POSSIBLE
>>>>> http-flow maxbw 1.228 -- 1228k
>>>>> http-flow priority medium -- medium
>>>>>
>>>>> I ran a test hitting the webserver and I see this:
>>>>
>>>> I have the following flow
>>>>
>>>> # flowadm show-flow FLOW LINK IPADDR PROTO
>>>> LPORT RPORT DSFLD
>>>> tcp-flow <link> -- tcp -- -- --
>>>>
>>>>
>>>> # flowadm show-flowprop tcp-flow
>>>> FLOW PROPERTY VALUE DEFAULT POSSIBLE
>>>> tcp-flow maxbw 1.228 -- 1228K tcp-flow
>>>> priority -- -- ?
>>>>
>>>> When I send TCP traffic (am using a traffic generator - netperf, to
>>>> this machine from a peer) for about 2 mins.
>>>>
>>>> On the peer the traffic generator (sender) says I am capped to about
>>>> 1.14 Mbps.
>>>>
>>>> Size Size Size Time Throughput bytes bytes bytes secs.
>>>> 10^6bits/sec
>>>>
>>>> 49152 49152 49152 120.49 1.14
>>>>
>>>> Now, when I try show-usage during the traffic flow on
>>>> the machine with the above flow in place (receiver), I am seeing:
>>>>
>>>> # flowadm show-usage -s 11/24/2009 -f /var/tmp/tcpflow
>>>> FLOW START END RBYTES OBYTES BANDWIDTH
>>>> tcp-flow 08:51:48 08:52:08 3428658 107802 1.414 Mbp
>>>> tcp-flow 08:52:08 08:52:28 3431198 107802 1.415 Mbp
>>>> tcp-flow 08:52:28 08:52:48 3434614 107888 1.417 Mbp
>>>> tcp-flow 08:52:48 08:53:08 3443298 107802 1.420 Mbp
>>>> tcp-flow 08:53:08 08:53:28 3444324 107802 1.420 Mbp
>>>> tcp-flow 08:53:28 08:53:48 1376806 43576 0.568 Mbps
>>>>
>>>> ...
>>>>
>>>> I think the difference you see is likely to be because of the time
>>>> period when the stats are written to the file (the bandwidth is computed
>>>> for every 20 seconds period which might not be exactly in
>>>> sync with the bandwidth enforcement period in the kernel) and also
>>>> could be because of rounding up etc. But, if you look at the entire
>>>> duration, it averages to about the configured limit (in the above
>>>> example, I think it is about 1.275 Mbps for the 2 min duration)
>>>
>>> The way I'm testing it is setting up Apache and then moving down a file
>>> with `wget`. The use case for this machine is an Apache based app that
>>> serves large files to customers. That is why I think a `wget` is more
>>> telling of "real" performance than netperf. I'm running the test again and
>>> on the client side I am seeing usage over the maxbw limit I have set.
>>> `wget` is reporting about 2Mbps transfer rate which is much closer to what
>>> I was seeing in the show-usage statistics.
>>>
>>
>>> [cdelgado at Bluegene tmp]$ wget sol/myfile.dat
>>> --10:01:30-- http://sol/myfile.dat
>>> Resolving sol... 192.168.69.104
>>> Connecting to sol|192.168.69.104|:80... connected.
>>> HTTP request sent, awaiting response... 200 OK
>>> Length: 1048576000 (1000M)
>>> Saving to: `myfile.dat'
>>>
>>> 5% [==> ] 55,530,974
>>> 267K/s eta 60m 44s
>>>
>>>
>>>>
>>>> BTW, setting a maxbw for a link (dladm) doesn't really impact the
>>>> flow as the bandwidth for both are independent.
>>>
>>> Thank you for this clarification but I still don't understand how I can be
>>> seeing ~2Mbps transfer if both the link and the flow are both capped at
>>> 1.2Mbps.
>>>
>>
>> Can you try with a higher bandwidth, say 100 Mbps and see what the results
>> are when compared to wget's output?
>>
>> Also, another way of manually checking would be to do a
>> # kstat -c flow -n http-flow
>>
>> before and after the wget run and see how many bytes (rbytes) the
>> kernel has seen for that flow (assuming there isn't any other traffic
>> going over the flow), and then determine the bandwidth (you might need
>> to get the duration of the wget run pretty close to get the
>> right bandwdith value).
>>
>> -venu
>
> I changed the flow to be 100Mbps.
>
> root at myhost:/tmp# flowadm show-flowprop -p maxbw http-flow
> FLOW PROPERTY VALUE DEFAULT POSSIBLE
> http-flow maxbw 100 -- 100
>
> I also removed the maxbw=1.228 I had set on the link. When I changed this
> value on the link I lost all network to the machine. Had to go in and
> manually reboot it. The network came up fine.
>
> I created two files one before wget and one after with no other network
> traffic except the ssh session. I created a 1 GB file to transfer. The
> output is a bit confusing to me.
>
> root at myhost:/tmp# ls http-flow-*
> http-flow-1.txt http-flow-2.txt
> root at myhost:/tmp# cat http-flow-*
> module: unix instance: 0
> name: http-flow class: flow
> crtime 188662.418688816
> ierrors 0
> ipackets 4680
> obytes 182520974
> oerrors 0
> opackets 127521
> rbytes 308138
> snaptime 189431.882363589
>
> module: unix instance: 0
> name: http-flow class: flow
> crtime 188662.418688816
> ierrors 0
> ipackets 31262
> obytes 1281785975
> oerrors 0
> opackets 895533
> rbytes 2062678
> snaptime 189595.122347726
>
> So this means the network only passed ~1.6 MB of data?
is this on the outbound side or inbound?
looks like it has a total of 8794120008 bits in the outbound side
(obytes) and 14036320 bits on the inbound (rbytes). What is the duration
93 seconds? So the outbound would be ~94 Mbps and inbound 150 Kbps.
>
> The wget command was telling me it was getting ~80Mbps which is under the
> threshold of the flow.
>
> [cdelgado at Bluegene tmp]$ time wget sol/myfile.dat
> --13:21:49-- http://sol/myfile.dat
> Resolving sol... 192.168.69.104
> Connecting to sol|192.168.69.104|:80... connected.
> HTTP request sent, awaiting response... 200 OK
> Length: 1048576000 (1000M)
> Saving to: `myfile.dat'
>
> 100%[==========================================================>]
> 1,048,576,000 10.5M/s in 93s
>
> 13:23:23 (10.7 MB/s) - `myfile.dat' saved [1048576000/1048576000]
>
>
> real 1m33.701s
> user 0m0.899s
> sys 0m23.874s
>
> Maybe a value of 100Mbps is too high for this machine. I might try with
> 50Mbps to see what wget says.
>
OK,
-venu
> -Cesar
>
>
>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>>
>>>>> root at myhost:~# flowadm show-usage -s 11/23/2009,01:32:22 -e
>>>>> 11/23/2009,01:46:22 -f /var/log/net.log | grep -v "0 Mbps\|^FLOW"
>>>>> http-flow 01:32:22 01:32:42 1512 2571 0.001 Mbps
>>>>> ssh-flow 01:32:42 01:33:02 1818 3578 0.002 Mbps
>>>>> http-flow 01:33:02 01:33:22 66917 3165136 1.292 Mbp
>>>>> ssh-flow 01:33:02 01:33:22 3618 5344 0.003 Mbps
>>>>> http-flow 01:33:22 01:33:42 117947 5713018 2.332 Mbp
>>>>> ssh-flow 01:33:22 01:33:42 4182 3020 0.002 Mbps
>>>>> http-flow 01:33:42 01:34:02 118998 5685520 2.321 Mbp
>>>>> ssh-flow 01:33:42 01:34:02 11616 9924 0.008 Mbps
>>>>> http-flow 01:34:02 01:34:22 117084 5725664 2.337 Mbp
>>>>> http-flow 01:34:22 01:34:42 119130 5725168 2.337 Mbp
>>>>> http-flow 01:34:42 01:35:02 114180 5725168 2.335 Mbp
>>>>> http-flow 01:35:02 01:35:22 109230 5725664 2.333 Mbp
>>>>> http-flow 01:35:22 01:35:42 116160 5725168 2.336 Mbp
>>>>> http-flow 01:35:42 01:36:02 119262 5725168 2.337 Mbp
>>>>> http-flow 01:36:02 01:36:22 119196 5725664 2.337 Mbp
>>>>> http-flow 01:36:22 01:36:42 117216 5725168 2.336 Mbp
>>>>> http-flow 01:36:42 01:37:02 119394 5722636 2.336 Mbp
>>>>> http-flow 01:37:02 01:37:22 119526 5725168 2.337 Mbp
>>>>> http-flow 01:37:22 01:37:42 119460 5725168 2.337 Mbp
>>>>> http-flow 01:37:42 01:38:02 119460 5725664 2.338 Mbp
>>>>> http-flow 01:38:02 01:38:22 119724 5725168 2.337 Mbp
>>>>> http-flow 01:38:22 01:38:42 119724 5725168 2.337 Mbp
>>>>> http-flow 01:38:42 01:39:02 119130 5722636 2.336 Mbp
>>>>> http-flow 01:39:02 01:39:22 118866 5725168 2.337 Mbp
>>>>> http-flow 01:39:22 01:39:42 116490 5725664 2.336 Mbp
>>>>> http-flow 01:39:42 01:40:02 119790 5725168 2.337 Mbp
>>>>> http-flow 01:40:02 01:40:22 117678 5725168 2.337 Mbp
>>>>> http-flow 01:40:22 01:40:42 118668 5725664 2.337 Mbp
>>>>> http-flow 01:40:42 01:41:02 117414 5725168 2.337 Mbp
>>>>> http-flow 01:41:02 01:41:22 119790 5725168 2.337 Mbp
>>>>> http-flow 01:41:22 01:41:42 119813 5720510 2.336 Mbp
>>>>> http-flow 01:41:42 01:42:02 119394 5725664 2.338 Mbp
>>>>> http-flow 01:42:02 01:42:22 119724 5722272 2.336 Mbp
>>>>> http-flow 01:42:22 01:42:42 119526 5725664 2.338 Mbp
>>>>> http-flow 01:42:42 01:43:02 119196 5722140 2.336 Mbp
>>>>> http-flow 01:43:02 01:43:22 119394 5725664 2.338 Mbp
>>>>> http-flow 01:43:22 01:43:42 119658 5725168 2.337 Mbp
>>>>> http-flow 01:43:42 01:44:02 119064 5725168 2.337 Mbp
>>>>> http-flow 01:44:02 01:44:22 113256 5676668 2.315 Mbp
>>>>> ssh-flow 01:44:02 01:44:22 18414 49646 0.027 Mbps
>>>>> http-flow 01:44:22 01:44:42 118206 5725664 2.337 Mbp
>>>>> http-flow 01:44:42 01:45:02 117282 5722140 2.335 Mbp
>>>>> ssh-flow 01:44:42 01:45:02 4698 3544 0.003 Mbps
>>>>> http-flow 01:45:02 01:45:22 118536 5688284 2.322 Mbp
>>>>> ssh-flow 01:45:02 01:45:22 4092 3198 0.002 Mbps
>>>>> http-flow 01:45:22 01:45:42 119130 5725168 2.337 Mbp
>>>>> ssh-flow 01:45:22 01:45:42 1980 1478 0.001 Mbps
>>>>>
>>>>> That's above the flow's maxbw parameter. After that I tried to change
>>>>> the maxbw of the link with dladm and that brought the bandwidth down but
>>>>> still not down to 1.2 Mbps.
>>>>>
>>>>> root at myhost:~# dladm show-linkprop -p maxbw e1000g0
>>>>> LINK PROPERTY PERM VALUE DEFAULT POSSIBLE
>>>>> e1000g0 maxbw rw 1.228 -- --
>>>>>
>>>>> root at myhost:~# flowadm show-usage -s 11/23/2009,01:46:02 -e
>>>>> 11/23/2009,01:46:22 -f /var/log/net.log | grep -v "0 Mbps\|^FLOW"
>>>>> http-flow 01:46:02 01:46:22 119394 5725168 2.337 Mbp
>>>>> ssh-flow 01:46:02 01:46:22 4680 2980 0.003 Mbps
>>>>> http-flow 01:46:22 01:46:42 94314 4520316 1.845 Mbp
>>>>>
>>>>> Any ideas or is there a subtlety that I'm missing and the behavior is
>>>>> correct?
>>>>>
>>>>> Thanks for the help.
>>>>>
>>>>> -Cesar
>>>>>
>>>>> _______________________________________________
>>>>> crossbow-discuss mailing list
>>>>> crossbow-discuss at opensolaris.org
>>>>> http://mail.opensolaris.org/mailman/listinfo/crossbow-discuss
>>>>>
>>>
>>>
>
>
