[Flent-users] Re: flaws and features of flent

[Sebastian Moeller]

Hi Dave,

'but folk in the 3rd world, (like, rural America) will love it.'

You know this, but for folks in the game since after the cold war ended. 3rd 
world denotes states that are neither 'affiliated' to the 'western' USA-lead 
block or to the the USSR-lead Warsaw-block states. The meaning as 
'less-developed' is of more recent origin and I am not sure how 
'less-developed' countries call themselves

Regards
Sebastian


On 16 January 2023 02:52:45 CET, Dave Taht  wrote:
>A rather long, meandering piece:
>
>https://blog.cerowrt.org/post/flaws_in_flent/
>
>-- 
>This song goes out to all the folk that thought Stadia would work:
>https://www.linkedin.com/posts/dtaht_the-mushroom-song-activity-698135607352320-FXtz
>Dave Täht CEO, TekLibre, LLC
>___
>Flent-users mailing list -- flent-users@flent.org
>To unsubscribe send an email to flent-users-le...@flent.org

-- 
Sent from my Android device with K-9 Mail. Please excuse my brevity.___
Flent-users mailing list -- flent-users@flent.org
To unsubscribe send an email to flent-users-le...@flent.org

[Flent-users] Re: flaws and features of flent

[Sebastian Moeller]

Hi Dave,

On 16 January 2023 02:52:45 CET, Dave Taht  wrote:
>A rather long, meandering piece:
>
>https://blog.cerowrt.org/post/flaws_in_flent/

I think 'Flent’s default sampling rate is too low' has a conceptually simple 
solution, record dats at high sample rates and optionally downsample before 
plotting... However with the range of network speeds flent covers picking a 
sane default is going to be a challenge. With almost all endhosts sporting 
gigabit ethernet interfaces and beyond I am unsure whether basing the sample 
rate on to-be-detected interface rate would help (it might serve to give an 
upper bound for sampling rate).

regards
Sebastian

>

-- 
Sent from my Android device with K-9 Mail. Please excuse my brevity.
___
Flent-users mailing list -- flent-users@flent.org
To unsubscribe send an email to flent-users-le...@flent.org

Re: [Flent-users] [tohojo/flent] Customized test displaying error (#203)

[Sebastian Moeller]

Hi Toke,


> On Apr 1, 2020, at 12:09, Toke Høiland-Jørgensen  
> wrote:
> 
> 
> flent-users  writes:
> 
> > Hi Toke,
> >
> > that sounds great!
> > Question: Are the marking parameters in decimal? So can I for example
> > just take the decimal values from
> > https://www.cisco.com/c/en/us/td/docs/switches/datacenter/nexus1000/sw/4_0_4_s_v_1_3/qos/configuration/guide/n1000v_qos/n1000v_qos_6dscpval.html
> > or related tables?
> 
> I'd actually suggest you just use the symbolic values (e.g., 'CS0').

Probably decent advise for RFC endorsed named DSCPs, but I want to try 
a all 6bit DSCP variant at one point in time and hence need numerical inputs as 
well.

> Flent will make sure to convert them to decimal values for the tools
> that don't understand the symbolic ones. This is the list of supported
> symbolic values:
> 
> https://github.com/tohojo/flent/blob/master/flent/runners.py#L105

Ah, I guess that list could be extended and that you take patches?

> 
> You'll notice the values differ slightly from what's in that Cisco
> table, but that's just because the values Flent uses are for the whole
> TOS field; so if you bitshift them two bits to the right, they match the
> table you referenced :)

Ah, fair enough, all I need to know which convention is used ;)

Best Regards
Sebastian


> —
> You are receiving this because you commented.
> Reply to this email directly, view it on GitHub, or unsubscribe.
> 
> ___
> Flent-users mailing list
> Flent-users@flent.org
> http://flent.org/mailman/listinfo/flent-users_flent.org


___
Flent-users mailing list
Flent-users@flent.org
http://flent.org/mailman/listinfo/flent-users_flent.org

Re: [Flent-users] [tohojo/flent] RRUL Upload plot - why such low granularity? (#185)

[Sebastian Moeller]

Hi Toke,


> On Sep 29, 2019, at 21:08, Toke Høiland-Jørgensen  
> wrote:
> 
> flent-users  writes:
> 
> > Hi Toke,
> >
> >> On Sep 29, 2019, at 20:44, Toke Høiland-Jørgensen 
> >>  wrote:
> >> 
> >> Rich Brown  writes:
> >> 
> >> > Sorry for the delay in responding... The higher granularity makes much
> >> > better plots (see below).
> >> 
> >> Great!
> >> 
> >> > Using `-m 2048,2048` I don't see a whole lot of load on my Mac 2.5 GHz
> >> > Intel Core i7 at 7mbps/768kbps. Thanks.
> >> 
> >> No, don't expect it would. The CPU usage thing will hit you at high
> >> rates (I was testing on a gigabit link).
> >> 
> >> I'm not sure we can realistically pick an option that works well for
> >> both slow and fast links. So we may have to add a switch; and then the
> >> problem becomes what to use for defaults...
> >
> > How about make it not a "switch" but a numeric parameter, and
> > default to the current default value by principle of least
> > surprise?
> 
> Well, by "switch" I just meant "new command line option".

Oops, sorry, I read this as a switch to toggle between say the default 
(which seems system dependent not petperf dependent, no?) and a value for low 
bandwidth links, say 1460...

> The obvious
> form of that would be, as you say, just the ability to set the netperf
> xfer size directly...

I guess that would be the best compromise, if the user knows about a 
links asymmetry the send and receive buffers can be sized accordingly like 16K, 
1600 for DOCSIS down-/upload...

I also wonder whether flent should not try to also record and parse /proc/stat 
to detect CPU overload (so the user could be warned/informed to increase the 
xfer size if there is danger of being CPU bound)?


Best Regards
Sebastian


> —
> You are receiving this because you commented.
> Reply to this email directly, view it on GitHub, or mute the thread.
> 
> ___
> Flent-users mailing list
> Flent-users@flent.org
> http://flent.org/mailman/listinfo/flent-users_flent.org


___
Flent-users mailing list
Flent-users@flent.org
http://flent.org/mailman/listinfo/flent-users_flent.org

Re: [Flent-users] [tohojo/flent] Unable to use flent-gui from 83d14d68 (#167)

[Sebastian Moeller]

Thank you very much, both Toke ans Pete:

QT_API=pyside2 ./run-flent --gui
did the trick, now I guess I either write a small wrapper around run-flent or 
simply put the export in my .bashrc.

Anyway with this the GUI runs and loads data files again.

Again Best Regards & Thanks


> On Jul 16, 2019, at 09:53, Toke Høiland-Jørgensen  wrote:
> 
> Sebastian Moeller  writes:
> 
>> Mmmh, so I installed qtpy (and dependencies) I now get:
>> 
>> ./run-flent --gui
>> Started Flent 1.9.9-git-43e16d5 using Python 3.7.4.
>> Initialised matplotlib v3.1.1 on numpy v1.16.4.
>> GUI loaded. Using Qt through pyqt5 v5.12.3.
>> objc[13628]: Class FIFinderSyncExtensionHost is implemented in both 
>> /System/Library/PrivateFrameworks/FinderKit.framework/Versions/A/FinderKit 
>> (0x7fff9b6ec3d8) and 
>> /System/Library/PrivateFrameworks/FileProvider.framework/OverrideBundles/FinderSyncCollaborationFileProviderOverride.bundle/Contents/MacOS/FinderSyncCollaborationFileProviderOverride
>>  (0x12662cf50). One of the two will be used. Which one is undefined.
>> ERROR: Error while loading data file: '> connectSlotsByName> returned a result with an error set'. Skipping.
>> 
>> The GUI opens again, but I still see the same lock-up that Pete
>> initially saw. Going to bed for now, maybe I find time to poke this
>> tomorrow...
> 
> Well, you'll still need to switch from PyQt5 to PySide2 - installing the
> latter is not enough, sadly, you'll also need to either set the QT_API
> environment variable, or uninstall PyQt5...
> 
> -Toke


___
Flent-users mailing list
Flent-users@flent.org
http://flent.org/mailman/listinfo/flent-users_flent.org

Re: [Flent-users] [tohojo/flent] Unable to use flent-gui from 83d14d68 (#167)

[Sebastian Moeller]

Mmmh, so I installed qtpy (and dependencies) I now get:

./run-flent --gui
Started Flent 1.9.9-git-43e16d5 using Python 3.7.4.
Initialised matplotlib v3.1.1 on numpy v1.16.4.
GUI loaded. Using Qt through pyqt5 v5.12.3.
objc[13628]: Class FIFinderSyncExtensionHost is implemented in both 
/System/Library/PrivateFrameworks/FinderKit.framework/Versions/A/FinderKit 
(0x7fff9b6ec3d8) and 
/System/Library/PrivateFrameworks/FileProvider.framework/OverrideBundles/FinderSyncCollaborationFileProviderOverride.bundle/Contents/MacOS/FinderSyncCollaborationFileProviderOverride
 (0x12662cf50). One of the two will be used. Which one is undefined.
ERROR: Error while loading data file: ' 
returned a result with an error set'. Skipping.

The GUI opens again, but I still see the same lock-up that Pete initially saw. 
Going to bed for now, maybe I find time to poke this tomorrow...

Thanks!



> On Jul 16, 2019, at 00:16, Toke Høiland-Jørgensen  wrote:
> 
> Sebastian Moeller  writes:
> 
>> Okay, same issue on macosx 10.14.5. Now with the latest changes and PySide2 
>> installed I get:
>> 
>> 
>> ./run-flent --version
>> Flent v1.9.9-git-43e16d5.
>> Running on Python 3.7.4 (default, Jul 11 2019, 01:08:00)  [Clang 10.0.1 
>> (clang-1001.0.46.4)].
>> Using matplotlib version 3.1.1 on numpy 1.16.4.
>> Using PyQt5 version 5.12.2.
>> 
>> 
>> 
>> ./run-flent --gui
>> Started Flent 1.9.9-git-43e16d5 using Python 3.7.4.
>> ERROR: Unable to find a usable Qt version.
>> 
>> Same Qt version "worked" before the latest changes; the quotes as I
>> saw the same error as Pete.
> 
> You probably need to 'pip install QtPy' first... Will try to document
> that somewhere :)


___
Flent-users mailing list
Flent-users@flent.org
http://flent.org/mailman/listinfo/flent-users_flent.org

Re: [Flent-users] [tohojo/flent] Unable to use flent-gui from 83d14d68 (#167)

[Sebastian Moeller]

Okay, same issue on macosx 10.14.5. Now with the latest changes and PySide2 
installed I get:


./run-flent --version
Flent v1.9.9-git-43e16d5.
Running on Python 3.7.4 (default, Jul 11 2019, 01:08:00)  [Clang 10.0.1 
(clang-1001.0.46.4)].
Using matplotlib version 3.1.1 on numpy 1.16.4.
Using PyQt5 version 5.12.2.



./run-flent --gui
Started Flent 1.9.9-git-43e16d5 using Python 3.7.4.
ERROR: Unable to find a usable Qt version.

Same Qt version "worked" before the latest changes; the quotes as I saw the 
same error as Pete.

If I can do anything on my side to fix/debug this, please let me know. Also let 
me know if I am just stupid and dod something wrong...


Best Regards
Sebastian

> On Jul 15, 2019, at 23:09, Toke Høiland-Jørgensen  
> wrote:
> 
> Pete Heist  writes:
> 
> > Ah, ok, well it looks like PySide2 has a more permissive license
> > anyway, although I guess that shouldn't affect flent. I haven't
> > noticed any other problems using it, although my testing has hardly
> > been extensive.
> 
> Yeah, PySide2 are the officially supported Python bindings from upstream
> Qt, so guess it's good to support that anyway (PyQt5 predates it
> somewhat). And it's not a huge cost to carry the compatibility layer, so
> I guess I can keep it like this...
> —
> You are receiving this because you commented.
> Reply to this email directly, view it on GitHub, or mute the thread.
> 
> ___
> Flent-users mailing list
> Flent-users@flent.org
> http://flent.org/mailman/listinfo/flent-users_flent.org


___
Flent-users mailing list
Flent-users@flent.org
http://flent.org/mailman/listinfo/flent-users_flent.org

Re: [Flent-users] silly pebkac problem?

[Sebastian Moeller]

Off-Topic,

see below:

> On May 12, 2019, at 11:51, Sebastian Moeller  
> wrote:
> 
> Hi Toke,
> 
> thanks a lot, that works ;)
> 
> I want to use this in my little overhead-exploration toy project, where I use 
> 
> forced_MTU=216
> iptables -t mangle -A FORWARD -p tcp -m tcp --tcp-flags SYN,RST SYN -m 
> comment --comment "custom: Zone wan MTU fixing" -j TCPMSS --set-mss 
> ${forced_MTU}
> ip6tables -t mangle -A FORWARD -p tcp -m tcp --tcp-flags SYN,RST SYN -m 
> comment --comment "custom6: Zone wan MTU fixing" -j TCPMSS --set-mss 
> ${forced_MTU}
> 
> as openwrt custom firewall rules to clamp MSS to my desired values (to solve 
> the issue that I want bi-directional control over packet sizes). 
> Interestingly, I can not clamp egress below an MSS of 216, while ingress at 
> least allows 140

It turned out this is simply the default minimum MSS that my macos 
inherited from its FreeBSD baseline...

Best Regards
Sebastian


> 
> 
> Best Regards
>   Sebastian
> 
>> On May 12, 2019, at 00:51, Toke Høiland-Jørgensen  wrote:
>> 
>> Sebastian Moeller  writes:
>> 
>>> So I try to run:
>>> 
>>> bash-3.2$  ./run-flent --ipv4 -l 60 -H netperf-eu.bufferbloat.net 
>>> --test-parameter=download_streams=16 tcp_ndown 
>>> --remote-metadata=root@192.168.1.1 -D . -t 
>>> IPv4_SQM_cake_layer-cake_LLA-ETH_OH34_U098pct9500of40309K-D090pct49000of109924K_hms-beagle2-en0_2_wndr3700v2OpwnWrt-r9933-pppoe-wan-eth1.7_2_bridged-BTHH5aLEDE17.1.5-ptm0.7-Hvt-VDSL50_2_netperf-eu.MSS216;
>>> 
>>> and I get 
>>> 
>>> ERROR: Unable to read test config file 
>>> '/space/data_local/moeller/PRIVATE/samba/privat/MOEWE/techno_kram/CODE/flent/flent/tests/tcp_ndown.conf':
>>>  ''str' object cannot be interpreted as an integer'.
>>> bash-3.2$ date ; ping -c 10 netperf-eu.bufferbloat.net ; ./run-flent --ipv4 
>>> -l 60 -H netperf-eu.bufferbloat.net --test-parameter=upload_streams=16 
>>> tcp_nup --remote-metadata=root@192.168.1.1 -D . -t 
>>> IPv4_SQM_cake_layer-cake_LLA-ETH_OH34_U098pct9500of40309K-D090pct49000of109924K_hms-beagle2-en0_2_wndr3700v2OpwnWrt-r9933-pppoe-wan-eth1.7_2_bridged-BTHH5aLEDE17.1.5-ptm0.7-Hvt-VDSL50_2_netperf-eu.MSS216
>>> 
>>> I assum this to be a simple pilot error, but I can not figure out what
>>> I am doing wrong, so any help is appreciated.
>> 
>> Nope, that is not you, that is a bug. My bad; pushed a fix :)
>> 
>> -Toke
>> 
>> ___
>> Flent-users mailing list
>> Flent-users@flent.org
>> http://flent.org/mailman/listinfo/flent-users_flent.org
> 
> 
> ___
> Flent-users mailing list
> Flent-users@flent.org
> http://flent.org/mailman/listinfo/flent-users_flent.org


___
Flent-users mailing list
Flent-users@flent.org
http://flent.org/mailman/listinfo/flent-users_flent.org

Re: [Flent-users] silly pebkac problem?

[Sebastian Moeller]

Hi Toke,

thanks a lot, that works ;)

I want to use this in my little overhead-exploration toy project, where I use 

forced_MTU=216
iptables -t mangle -A FORWARD -p tcp -m tcp --tcp-flags SYN,RST SYN -m comment 
--comment "custom: Zone wan MTU fixing" -j TCPMSS --set-mss ${forced_MTU}
ip6tables -t mangle -A FORWARD -p tcp -m tcp --tcp-flags SYN,RST SYN -m comment 
--comment "custom6: Zone wan MTU fixing" -j TCPMSS --set-mss ${forced_MTU}

as openwrt custom firewall rules to clamp MSS to my desired values (to solve 
the issue that I want bi-directional control over packet sizes). Interestingly, 
I can not clamp egress below an MSS of 216, while ingress at least allows 140


Best Regards
Sebastian

> On May 12, 2019, at 00:51, Toke Høiland-Jørgensen  wrote:
> 
> Sebastian Moeller  writes:
> 
>> So I try to run:
>> 
>> bash-3.2$  ./run-flent --ipv4 -l 60 -H netperf-eu.bufferbloat.net 
>> --test-parameter=download_streams=16 tcp_ndown 
>> --remote-metadata=root@192.168.1.1 -D . -t 
>> IPv4_SQM_cake_layer-cake_LLA-ETH_OH34_U098pct9500of40309K-D090pct49000of109924K_hms-beagle2-en0_2_wndr3700v2OpwnWrt-r9933-pppoe-wan-eth1.7_2_bridged-BTHH5aLEDE17.1.5-ptm0.7-Hvt-VDSL50_2_netperf-eu.MSS216;
>> 
>> and I get 
>> 
>> ERROR: Unable to read test config file 
>> '/space/data_local/moeller/PRIVATE/samba/privat/MOEWE/techno_kram/CODE/flent/flent/tests/tcp_ndown.conf':
>>  ''str' object cannot be interpreted as an integer'.
>> bash-3.2$ date ; ping -c 10 netperf-eu.bufferbloat.net ; ./run-flent --ipv4 
>> -l 60 -H netperf-eu.bufferbloat.net --test-parameter=upload_streams=16 
>> tcp_nup --remote-metadata=root@192.168.1.1 -D . -t 
>> IPv4_SQM_cake_layer-cake_LLA-ETH_OH34_U098pct9500of40309K-D090pct49000of109924K_hms-beagle2-en0_2_wndr3700v2OpwnWrt-r9933-pppoe-wan-eth1.7_2_bridged-BTHH5aLEDE17.1.5-ptm0.7-Hvt-VDSL50_2_netperf-eu.MSS216
>> 
>> I assum this to be a simple pilot error, but I can not figure out what
>> I am doing wrong, so any help is appreciated.
> 
> Nope, that is not you, that is a bug. My bad; pushed a fix :)
> 
> -Toke
> 
> ___
> Flent-users mailing list
> Flent-users@flent.org
> http://flent.org/mailman/listinfo/flent-users_flent.org


___
Flent-users mailing list
Flent-users@flent.org
http://flent.org/mailman/listinfo/flent-users_flent.org

Re: [Flent-users] total throughput for rrul test on WiFi

[Sebastian Moeller]

Hi Pete,



> On Jun 26, 2018, at 00:40, Pete Heist  wrote:
> 
> The rrul test over a point-to-point WiFi link cuts the total TCP throughput 
> considerably below that of rrul_be. For example, on an 802.11n 20MHz MCS 15 
> link:
> 
> rrul_be: ~90mbit
> rrul: ~30-45mbit
> 

I believe this is simply showing the cost of using the non-aggregating 
AC_VO that also has an advantage of getting airtime, using that will 
considerably lower the achievable goodput over the wifi channel. Add to this 
that with RRUL both ends will send traffic that should be classified AC_VO and 
the half-duplex nature of current wifi specs and there might simply be much 
less total goodput available than one would fancy...

> I think this came up before, but could the standard rrul test be exceeding 
> the 802.11e spec in terms of how much bandwidth it's using for some access 
> categories?

Is there a standard for this?

> I haven’t found reference to this anywhere, but if so I’ll need to take that 
> into account when interpreting the results. (athstats on Ubiquiti’s gear 
> shows that the counters for all four categories BK, BE, VI and VO go up, so 
> they appear to map evenly with DSCP values BK, BE, CS5 and EF.)
> 
> Anyway I think the DSCP field is set to 0 in the backhaul I’m testing for, so 
> rrul tests are probably more of a curiosity in this case...
> 
> Pete
> 
> 
> ___
> Flent-users mailing list
> Flent-users@flent.org
> http://flent.org/mailman/listinfo/flent-users_flent.org


___
Flent-users mailing list
Flent-users@flent.org
http://flent.org/mailman/listinfo/flent-users_flent.org

Re: [Flent-users] irtt packet-cluster mode for estimating queueing delay

[Sebastian Moeller]

> On May 6, 2018, at 21:32, Toke Høiland-Jørgensen <t...@toke.dk> wrote:
> 
> Sebastian Moeller <moell...@gmx.de> writes:
>>> [...]
>>> Yeah, ICMP is definitely treated differently in many places... Another
>>> example is routers and switches that have a data plane implemented in
>>> hardware will reply to ICMP from the software control plane, which is
>>> way *slower* than regular forwarding...
>> 
>>  But that "should" only affect the ICMP reflector, no? I always
>>  assumed that routers will treat ICMP packets that they only pass
>>  though just like any other IP packet...
> 
> Sure, it depends on what you are pinging...

Ah, okay, I am relieved ;)

> 
>>> Also, ICMP is often subject to
>>> rate limiting... etc, etc...
>>> 
>> 
>>  If I understand posts in the nanog list correctly thee is also a
>>  trend towards limiting UDP as well.
> 
> Eh? That would mean limiting half the internet (QUIC is UDP, for
> instance)...

Exactly my sentiment when I read it, but I might have misunderstood the 
amount of limiting...

Best Regards
Sebastian


> -Toke


___
Flent-users mailing list
Flent-users@flent.org
http://flent.org/mailman/listinfo/flent-users_flent.org

Re: [Flent-users] irtt packet-cluster mode for estimating queueing delay

[Sebastian Moeller]

Hi Pete,

you could also try different packet sizes for you probes. The size ratio should 
give you an idea about the RTT difference explained by simple transit times and 
anything on top of that might be related to queueing. I am probably severely 
underestimating the scope of your question, but I believe that that also has 
the advantage that, unlike packet-tuples, will not be as sensitive to bonded 
path segments (which probably do not exist in your environment in the first 
place).

Best Regards
Sebastian

> On May 6, 2018, at 20:28, Pete Heist  wrote:
> 
> 
>> On May 6, 2018, at 5:54 PM, Toke Høiland-Jørgensen  wrote:
>> 
>> Pete Heist  writes:
>> 
>>> Channel contention delay may be estimated by the difference between
>>> the round-trip times for the strict priority VO and BE packets (0x01
>>> and 0xb9), and queueing delay between the regular vs strict priority
>>> VO packets (0xb8 and 0xb9), right?
>> 
>> I like the idea, but is there any equipment that actually implements
>> strict priority queueing within a single QoS level? Or how are you
>> planning to elicit this behavior?
> 
> The backhaul I’d like to test it on uses mostly NSM5s as the wireless devices 
> and APUs for routing, QoS, etc. The QoS scripts use the htb, sfq and prio 
> qdiscs. I’m hoping I can just add a prio qdisc / tc filter somewhere in the 
> existing rules.
> 
>>> Lastly, I’ll need to find out for sure how much impact the use of UDP
>>> with a userspace client/server (in Go) is having vs ICMP. I find it hard to 
>>> believe that I’m seeing tens of
>>> milliseconds going into userspace.
>> 
>> That does seem a bit much. Hard to tell what is the cause without a more
>> controlled experiment, though...
> 
> Actually, I think it’s impossible that userspace overhead is the problem 
> here. The irtt client and server devices are completely independent of the 
> network routing/firewalling hardware, so the CPU load on them is identical at 
> times of low and high network load.
> 
> I now added SmokePing ICMP and IRTT targets for the same LAN host, and can 
> look at that distribution after a day to judge the overhead.
> 
> I guess it’s possible that ICMP may route faster over the Internet than UDP 
> even if it isn’t being prioritized, but I would be surprised if that much 
> faster. Not quite related, but I also find this interesting:
> 
> https://perso.uclouvain.be/olivier.bonaventure/blog/html/2013/05/22/don_t_use_ping_for_delay_measurements.html
> 
> 
> ___
> Flent-users mailing list
> Flent-users@flent.org
> http://flent.org/mailman/listinfo/flent-users_flent.org


___
Flent-users mailing list
Flent-users@flent.org
http://flent.org/mailman/listinfo/flent-users_flent.org

Re: [Flent-users] DSCP / ToS

[Sebastian Moeller]

Hi Toke,


> On Feb 19, 2018, at 12:10, Toke Høiland-Jørgensen  wrote:
> 
> Pete Heist  writes:
> 
>>> On Feb 19, 2018, at 11:36 AM, Toke Høiland-Jørgensen  wrote:
>>> 
>>> Pete Heist  writes:
>>> 
 I liked the simplicity of RFC 795 (https://tools.ietf.org/html/rfc795) 
 from 1981 :) so, mostly in jest, I’ll propose a new system, call it 
 "Deferential Services", summed up by this:
 
 - Bits 0-2: Anti-precedence: 000 = normal, 111 = least important
 - Bit 3: 0 = normal delay, 1 = high delay ok
 - Bit 4: 0 = normal reliability, 1 = low reliability ok
 - Bit 5: Always 1, identifies deferential services, rarely used for 
 DiffServ in practice, I think
 - Bits 6-7 ECN
 
 Bit 5 will usually be 0 so deferential services won’t be used, and if
 it happens to be 1 for the wrong reason, 000 in bits 0-2 is still best
 effort so it won’t end up that bad in most cases. And with that I wipe
 my hands and slip out the side exit… :)
>>> 
>>> Don't forget RFC3514 compliance ;)
>>> 
>>> https://www.ietf.org/rfc/rfc3514.txt
>> 
>> :)
>> 
>> Humor aside, I do think a system based on voluntary de-prioritization
>> would be better than what we have today. Although, any system which is
>> universally respected also holds the potential for abuse, which is
>> probably why DiffServ ended up like it did in both design and
>> practice.
> 
> Sure, voluntary de-prioritisation would be awesome (people have tried
> with thinks like LEDBAT); but the people designing DiffServ are more in
> the "we must prioritise our (but no one else's) VoIP services" :/

In all fairness I always thought the VoIP application and appliances 
are the few exemplary citizens in dscp-land (which does not mean much); AFAICT 
they mostly tend to use the same small documented set of dscp markings 
(probably driven by the intent to use wifi's wmm categories well?). 
The voluntary de-prioritisation idea is interesting even though the 
devil is in the details.  For example I believe there should to be some (weak) 
guarantee of forward progress even for the back ground traffic (otherwise 
applications would need to monitor the progress themselves and switch markings 
if progress is lacking). It is one thing to say "transfer this when you find 
time" and consciously realizing that this might mean "never"; I doubt people 
consider never to be an acceptable answer ;). (I add that if say on 
LTE-networks BK packets would be free of charge, then even no delivery 
guarantee at all might be acceptable to some users).
But I digress, as far as I can tell CS1 is as close to a "universal" 
background traffic signal if I assume we will ever get (if only ISPs would 
optionally treat CS1 with lower priority at their upstream end for packets 
addressed to their customers we would have 80%? of what is to be gained by 
using a BK signal, assuming all intermediate transports did not actively screw 
up the signal). I believe Dave had data showing one ISP re-mapping almost 30% 
of packets to CS1, so this  "not-screw-up" assumption might be optimistic...

LEDBAT? great idea, sub-optimal congestion detection method ;) (at least wrong 
unless one only uses under-managed queues at the bottleneck link). As is LEDBAT 
(and BBR?) tend to punish those links that use proper AQM, not sure whether 
this could simply be ameliorated by auto-scaling the thresholds used to declare 
an observed latency increase as signal for congestion.

Best Regards
Sebastian


> 
> -Toke
> 
> ___
> Flent-users mailing list
> Flent-users@flent.org
> http://flent.org/mailman/listinfo/flent-users_flent.org


___
Flent-users mailing list
Flent-users@flent.org
http://flent.org/mailman/listinfo/flent-users_flent.org

Re: [Flent-users] DSCP / ToS

[Sebastian Moeller]

Hi Pete,


> On Feb 18, 2018, at 17:47, Pete Heist  wrote:
> 
> Change of subject...
> 
> I realized it’s the two LSBs that carry ECN, not the MSBs. :)

Really, I am always confused by this especially since I aways forget 
whether network order is big or little endian (looked it up again it is 
big-endian).

> I sometimes mistakenly passed in 46 (0x2E) for CS5 to irtt’s —dscp flag, for 
> example, But it's really 184 (0xb8) that should go into the DS (ToS) field. 
> Flent has it correct, and thankfully IRTT is doing today what Flent thinks 
> it’s doing. But what I’m thinking for IRTT’s future is:
> 
> 1) Add a new —tos parameter (possibly with a —ds synonym) that just sets the 
> value passed in, hex or dec. It will not allow text. Basically what —dscp is 
> today.
> 
> 2) Change —dscp to left shift whatever value is passed in by two bits, if 
> it’s hex or dec. If it’s text, use the same table Flent has now, except add 
> “voice-admit” (0xb0, from RFC 5865), which is the only additional value at 
> https://www.tucny.com/Home/dscp-tos, otherwise they all agree. The two LSBs 
> (ECN bits) will always be 0 in this case. Flent could additionally add the 
> "voice-admit” value (0xb0), but it’s probably not that critical. :)
> 
> This would break —dscp’s current behavior, so I would increment the version 
> number when I do it, and save it for v1.0 in case there are other breaking 
> changes.
> 
> So just some DSCP general discussion...
> 
> It occurs to me that maybe the differentiated services spec should have 
> defined official ways to _de-prioritize_ traffic (I think CS1 is sometimes 
> used for this, but unofficially).

Yes, I agree (not that this matters much ;) ), but using CS1 seems to 
be just as "common" as can be hoped for for any DSCP mark.


> Most OS and application vendors would mark their bulk traffic, and ISPs would 
> be more than happy to honor it. As it stands today I guess most ISPs ignore 
> the DSCP value from customer traffic, and DSCP is probably most useful on 
> home routers and APs, for example, to select the right WMM access category. 
> Right, or wrong?

Well, as far as I ca tell DSCPs are only ever valid inside a DSCP 
domain, so in effect one can not trust incoming DSCPs blindly. One RFC I read, 
but I forgot which, proposed to split the 6 DSCP bits evenly between endpoints 
and intermediate hops, so that the endpoints could use 3 bits to signal their 
intent, while the intermediate hops could use the other 3 bits to actually 
store their transient markings somewhere. Which sounds like a great plan if one 
is willing to accept that 8 different priority categories are probably enough 
for getting the most advantage of using different priorities (sort of assuming 
that it will lead to diminishing returns after that). That view is somewhat 
backed by WMM just using 4 priority classes, and VLAN tags also just using 3 
bits (which some switches evaluate in real-time).

> And in general, should ISPs de-prioritize traffic marked CS1 (0x10)?

Realistically they could at least offer their customers one DSCP marking which 
they will interpret as back ground/bulk. But doing that by default has issues 
as per RFC DSCPs do not carry intent between DSCP-domains and any intermediary 
hop might re-map say EF to BK (which is still RFC conform), so this is a bit 
approximate (unless everybody agrees to a) interpret CS1 as background and b) 
promises to never ever re-map anything to BK that is from outside the local 
DSCP-domain, but I digress). Well, that or people agree on the 3bit/3bit split 
so that each DSCP-domain on ingress would look at the three intent bits and 
remap the transport bits to whatever it believes to be appropriate (which as 
intent is conserved, might well be the 3 bit BK equivalent).

BUT unless and until applications start to offer their users to effortlessly 
configure DSCP markings all of this is somewhat moot ;)

Best Regards
Sebastian


___
Flent-users mailing list
Flent-users@flent.org
http://flent.org/mailman/listinfo/flent-users_flent.org

Re: [Flent-users] weir error message after update of macosx and flent

[Sebastian Moeller]

Hi Pete,


> On Feb 18, 2018, at 00:41, Pete Heist <petehe...@gmail.com> wrote:
> 
>> 
>> On Feb 17, 2018, at 11:21 PM, Toke Høiland-Jørgensen <t...@toke.dk> wrote:
>> 
>> Sebastian Moeller <moell...@gmx.de> writes:
>> 
>>> Hi Toke,
>>> 
>>> I made the mistake of updating both my os (from macos 10.12.6 to
>>> 10.13.3) and flent (to 1.2 I believe) at the same time. Now when I try
>>> to actually run a test I get the following results:
>> 
>> Well, seems you found a bug in the diffserv handling for the IRTT
>> runner; well done ;)
>> 
>> Should be fixed in git :)
> 
> Nice, this also makes me think that I could accept DSCP marking text (CS0, 
> etc) for the —dscp parameter. And that I might not have aptly named that 
> flag, since really DSCP is only 6 bits of the DS (ToS) field, but what you’re 
> passing in can be all 8 bits (or else maybe I should just mask off the two 
> ECN bits). I might make some adjustments here for v1…

Not that I have looked at the actual parameter names, but how about having 
--dscp mask out the upper two bits (if they are set) and --tos allowing to set 
the full legacy 8 bit?

Best Regards
Sebastian

> 
> 
> ___
> Flent-users mailing list
> Flent-users@flent.org
> http://flent.org/mailman/listinfo/flent-users_flent.org


___
Flent-users mailing list
Flent-users@flent.org
http://flent.org/mailman/listinfo/flent-users_flent.org

[Flent-users] macosx python is fickle

[Sebastian Moeller]

I just tried to open the gui and click the "new tab" menu item, here is the 
result:
ash-3.2$ ./run-flent --gui
Started Flent 1.2.0-git-208f903 using Python 3.6.4.
Initialised matplotlib v2.1.1 on numpy v1.14.0.
GUI loaded. Running on PyQt v5.10.
Traceback (most recent call last):
  File 
"/space/data_local/moeller/PRIVATE/samba/privat/MOEWE/techno_kram/CODE/flent/flent/gui.py",
 line 760, in activate_tab
self.update_settings(widget)
  File 
"/space/data_local/moeller/PRIVATE/samba/privat/MOEWE/techno_kram/CODE/flent/flent/gui.py",
 line 775, in update_settings
widget.update_settings(self.plotSettingsWidget.values())
  File 
"/space/data_local/moeller/PRIVATE/samba/privat/MOEWE/techno_kram/CODE/flent/flent/gui.py",
 line 2252, in update_settings
t = self.results.title
AttributeError: 'NoneType' object has no attribute 'title'
Abort trap: 6

Not sure whether that is to be expected.

Best Regards
Sebastian
___
Flent-users mailing list
Flent-users@flent.org
http://flent.org/mailman/listinfo/flent-users_flent.org

[Flent-users] weir error message after update of macosx and flent

[Sebastian Moeller]

Hi Toke,


I made the mistake of updating both my os (from macos 10.12.6 to 10.13.3) and 
flent (to 1.2 I believe) at the same time. Now when I try to actually run a 
test I get the following results:

Started Flent 1.2.0-git-94b4267 using Python 3.6.4.
Executing test environment file 
/space/data_local/user/PRIVATE/samba/privat/dir/techno_kram/CODE/flent/flent/tests/rrul_cs8.conf
Starting rrul_cs8 test. Expected run time: 310 seconds.
which: /Users/user/go/bin/netperf is not an executable file
which: /opt/jip-150326_v3.1/bin/netperf is not an executable file
which: /opt/Slicer3-3.6.2-2010-11-03-darwin-x86/netperf is not an executable 
file
which: /opt/afni/netperf is not an executable file
which: /usr/local/bin/netperf is not an executable file
which: /opt/fsl-5.0.7-macosx/bin/netperf is not an executable file
which: /opt/freesurfer-Darwin-lion-stable-pub-v5.3.0/bin/netperf is not an 
executable file
which: /opt/freesurfer-Darwin-lion-stable-pub-v5.3.0/fsfast/bin/netperf is not 
an executable file
which: /opt/freesurfer-Darwin-lion-stable-pub-v5.3.0/tktools/netperf is not an 
executable file
which: /opt/fsl-5.0.7-macosx/bin/netperf is not an executable file
which: /opt/freesurfer-Darwin-lion-stable-pub-v5.3.0/mni/bin/netperf is not an 
executable file
which: Found netperf executable at /opt/local/bin/netperf
Ping (ms) UDP CS7: Adding child IrttRunner
which: Found irtt executable at /Users/user/go/bin/irtt
Traceback (most recent call last):
  File 
"/space/data_local/user/PRIVATE/samba/privat/dir/techno_kram/CODE/flent/flent/runners.py",
 line 1500, in check
marking = "--dscp={}".format(self.marking_map[mk])
KeyError: 'CS7'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "./run-flent", line 33, in 
sys.exit(run_flent())
  File 
"/space/data_local/user/PRIVATE/samba/privat/dir/techno_kram/CODE/flent/flent/__init__.py",
 line 59, in run_flent
b.run()
  File 
"/space/data_local/user/PRIVATE/samba/privat/dir/techno_kram/CODE/flent/flent/batch.py",
 line 610, in run
return self.run_test(self.settings, self.settings.DATA_DIR, True)
  File 
"/space/data_local/user/PRIVATE/samba/privat/dir/techno_kram/CODE/flent/flent/batch.py",
 line 509, in run_test
res = self.agg.postprocess(self.agg.aggregate(res))
  File 
"/space/data_local/user/PRIVATE/samba/privat/dir/techno_kram/CODE/flent/flent/aggregators.py",
 line 233, in aggregate
measurements, metadata, raw_values = self.collect()
  File 
"/space/data_local/user/PRIVATE/samba/privat/dir/techno_kram/CODE/flent/flent/aggregators.py",
 line 121, in collect
t.check()
  File 
"/space/data_local/user/PRIVATE/samba/privat/dir/techno_kram/CODE/flent/flent/runners.py",
 line 1543, in check
self.add_child(IrttRunner, **self.runner_args)
  File 
"/space/data_local/user/PRIVATE/samba/privat/dir/techno_kram/CODE/flent/flent/runners.py",
 line 227, in add_child
c.check()
  File 
"/space/data_local/user/PRIVATE/samba/privat/dir/techno_kram/CODE/flent/flent/runners.py",
 line 1502, in check
marking = "--dscp={}".format(marking)
UnboundLocalError: local variable 'marking' referenced before assignment


Opening run-flent --gui however works as expected. I guess I made a silly 
mistake, but since I get these errors with pyhton 2.7 as well as the shown 
python 3.6 I just wanted to ask whether this "rings a bell" or whether I need 
to look deeper into my setup.



Things I did test already:
1) netperf: I used Rich Brown's betterspeedtest.sh wrapper to test whether 
netperf is operational (it is) Also:
bash-3.2$ netperf -4 -H netperf-eu.bufferbloat.net -t TCP_STREAM -l 10 -v 0 -P 
0 -D 2
Interim result:7.66 10^6bits/s over 2.053 seconds ending at 1518903857.219
Interim result:8.81 10^6bits/s over 2.023 seconds ending at 1518903859.242
Interim result:8.44 10^6bits/s over 2.112 seconds ending at 1518903861.354
Interim result:8.85 10^6bits/s over 2.133 seconds ending at 1518903863.487
Interim result:8.11 10^6bits/s over 1.680 seconds ending at 1518903865.167
   8.34 
bash-3.2$ netperf -4 -H netperf-eu.bufferbloat.net -t TCP_MAERTS -l 10 -v 0 -P 
0 -D 2
Interim result:   33.10 10^6bits/s over 2.000 seconds ending at 1518903895.743
Interim result:   38.59 10^6bits/s over 2.030 seconds ending at 1518903897.773
Interim result:   38.74 10^6bits/s over 2.001 seconds ending at 1518903899.774
Interim result:   40.12 10^6bits/s over 2.000 seconds ending at 1518903901.774
Interim result:   40.63 10^6bits/s over 1.970 seconds ending at 1518903903.744
  38.23 

2) fping:
bash-3.2$ sudo fping -D -n -c 2 127.0.0.1 -i 5
[1518903718.880975] localhost : [0], 84 bytes, 0.04 ms (0.04 avg, 0% loss)
[1518903719.885723] localhost : [1], 84 bytes, 0.07 ms (0.05 avg, 0% loss)


Best Regards
Sebastian


___
Flent-users mailing list
Flent-users@flent.org
http://flent.org/mailman/listinfo/flent-users_flent.org

Re: [Flent-users] IRTT question

[Sebastian Moeller]

Hi Toke,


> On Feb 9, 2018, at 11:38, Toke Høiland-Jørgensen <t...@toke.dk> wrote:
> 
> Toke Høiland-Jørgensen <t...@toke.dk> writes:
> 
>> Sebastian Moeller <moell...@gmx.de> writes:
>> 
>>> Hi Toke,
>>> 
>>> 
>>>> On Feb 9, 2018, at 11:05, Toke Høiland-Jørgensen <t...@toke.dk> wrote:
>>>> 
>>>> Sebastian Moeller <moell...@gmx.de> writes:
>>>> 
>>>>> Hi Toke,
>>>>> 
>>>>> 
>>>>>> On Feb 8, 2018, at 22:19, Toke Høiland-Jørgensen <t...@toke.dk> wrote:
>>>>>> 
>>>>>> Sebastian Moeller <moell...@gmx.de> writes:
>>>>>> 
>>>>>>> Hi Pete,
>>>>>>> 
>>>>>>> 
>>>>>>>> On Feb 8, 2018, at 20:49, Pete Heist <petehe...@gmail.com> wrote:
>>>>>>>> 
>>>>>>>> It should work, but the irtt server needs to be running. On your box 
>>>>>>>> where netserver is running (sounds like the Mac for you if you’re 
>>>>>>>> testing locally), run “irtt server”.
>>>>>>> 
>>>>>>> Ah, that got me a bit further:
>>>>>>> 
>>>>>>> which: Found irtt executable at /Users/smoeller/go/bin/irtt
>>>>>>> UDP RTT test: Cannot use irtt runner (Irtt connection check failed: 
>>>>>>> Error: read udp4 192.168.1.222:63322->130.243.26.64:2112: read: 
>>>>>>> connection refused
>>>>>>> ). Using netperf UDP_RR
>>>>>> 
>>>>>> Ah, that's the public server that I'm hosting. Meant to add an irtt
>>>>>> instance to that but guess I forgot. Will do so tomorrow :)
>>>>> 
>>>>>   That would be sweet, I guess I learned a lot today about what to
>>>>>   do locally so I might actually be successful then, after Pete's
>>>>>   and your generous help!
>>>> 
>>>> Right, you should be able to run irtt tests against
>>>> netperf-eu.bufferbloat.net now. Seems I did add the irtt server, but had
>>>> forgotten to open the firewall port :)
>>> 
>>> Ah, great, this got me a bit further:
>>> 
>>> UDP RTT test: Cannot use irtt runner (Irtt connection check failed:
>>> [InvalidServerRestriction] server tried to reduce interval to < 1s,
>>> from 1s to 92ns ). Using netperf UDP_RR
>> 
>> Huh, this looks like a bug? Pete?
> 
> Ah, hang on, I was running an old version of irtt on the server. Try
> now? :)
> 

Sweet:
which: Found irtt executable at /Users/smoeller/go/bin/irtt
UDP RTT test: Using irtt
Ping (ms) UDP BK: Adding child IrttRunner
UDP RTT test: Using irtt
Ping (ms) UDP BE: Adding child IrttRunner
UDP RTT test: Using irtt

Thanks a lot!


> -Toke


___
Flent-users mailing list
Flent-users@flent.org
http://flent.org/mailman/listinfo/flent-users_flent.org

Re: [Flent-users] IRTT question

[Sebastian Moeller]

Hi Toke,


> On Feb 8, 2018, at 22:19, Toke Høiland-Jørgensen <t...@toke.dk> wrote:
> 
> Sebastian Moeller <moell...@gmx.de> writes:
> 
>> Hi Pete,
>> 
>> 
>>> On Feb 8, 2018, at 20:49, Pete Heist <petehe...@gmail.com> wrote:
>>> 
>>> It should work, but the irtt server needs to be running. On your box where 
>>> netserver is running (sounds like the Mac for you if you’re testing 
>>> locally), run “irtt server”.
>> 
>>  Ah, that got me a bit further:
>> 
>> which: Found irtt executable at /Users/smoeller/go/bin/irtt
>> UDP RTT test: Cannot use irtt runner (Irtt connection check failed: Error: 
>> read udp4 192.168.1.222:63322->130.243.26.64:2112: read: connection refused
>> ). Using netperf UDP_RR
> 
> Ah, that's the public server that I'm hosting. Meant to add an irtt
> instance to that but guess I forgot. Will do so tomorrow :)

That would be sweet, I guess I learned a lot today about what to do 
locally so I might actually be successful then, after Pete's and your generous 
help!

Thanks a lot!
Sebastian

> 
> -Toke


___
Flent-users mailing list
Flent-users@flent.org
http://flent.org/mailman/listinfo/flent-users_flent.org

Re: [Flent-users] IRTT question

[Sebastian Moeller]

Hi Pete,


> On Feb 8, 2018, at 20:49, Pete Heist <petehe...@gmail.com> wrote:
> 
> It should work, but the irtt server needs to be running. On your box where 
> netserver is running (sounds like the Mac for you if you’re testing locally), 
> run “irtt server”.

Ah, that got me a bit further:

which: Found irtt executable at /Users/smoeller/go/bin/irtt
UDP RTT test: Cannot use irtt runner (Irtt connection check failed: Error: read 
udp4 192.168.1.222:63322->130.243.26.64:2112: read: connection refused
). Using netperf UDP_RR


I guess I now need to poke a hole into my firewall to allow udp port 2112 
incoming, no?

Best Regards & many thanks
Sebastian

> 
> On Linux you can use the irtt.service file in the repo with systemd to run it 
> at startup. I’m working on a Debian package to make that setup easier, and if 
> there’s interest a brew package for the Mac should be possible as well, but I 
> hope at running it manually at least gets you going for now...
> 
> Pete
> 
>> On Feb 8, 2018, at 8:32 PM, Sebastian Moeller <moell...@gmx.de> wrote:
>> 
>> So after reading the announcement, I went an tried my luck with irtt 
>> installed on macosx 10.12.6: irtt is installed and local tests seem to work, 
>> but I only get:
>> 
>> which: Found irtt executable at /Users/user/go/bin/irtt
>> UDP RTT test: Cannot use irtt runner (Irtt connection check failed: 
>> [OpenTimeout] no reply from server
>> ). Using netperf UDP_RR
>> 
>> Now, is this supposed to work for macosx at all with irtt? I tried from two 
>> different networks and both times the same irtt "demotion" to netperf (not 
>> that there is anything wrong with netperf)
>> Silly question, do I need to run irtt as root?
>> 
>> Best Regards
>> ___
>> Flent-users mailing list
>> Flent-users@flent.org
>> http://flent.org/mailman/listinfo/flent-users_flent.org
> 


___
Flent-users mailing list
Flent-users@flent.org
http://flent.org/mailman/listinfo/flent-users_flent.org

[Flent-users] IRTT question

[Sebastian Moeller]

So after reading the announcement, I went an tried my luck with irtt installed 
on macosx 10.12.6: irtt is installed and local tests seem to work, but I only 
get:

which: Found irtt executable at /Users/user/go/bin/irtt
UDP RTT test: Cannot use irtt runner (Irtt connection check failed: 
[OpenTimeout] no reply from server
). Using netperf UDP_RR

Now, is this supposed to work for macosx at all with irtt? I tried from two 
different networks and both times the same irtt "demotion" to netperf (not that 
there is anything wrong with netperf)
Silly question, do I need to run irtt as root?

Best Regards
___
Flent-users mailing list
Flent-users@flent.org
http://flent.org/mailman/listinfo/flent-users_flent.org

Re: [Flent-users] Help with RRUL / TCP upload

[Sebastian Moeller]

Hi,

I believe you need to set the step size to something bigger:
from "run-flent --help"
-s STEP_SIZE, --step-size STEP_SIZE
Measurement data point step size.

For a slow link there are simply no data from netperf for a given "sampling" 
point and hence you get the intermittent plots. As far as I know there is 
unfortunately no way to specify the step size independent for both directions, 
so for non-choppy plots in your slow direction you will also get less time 
resolution in your fast direction...

Hope that helps




> On Jul 31, 2017, at 19:36, Emmanuel Blot  wrote:
> 
> Hi all,
> 
> I’m a new flent user.
> 
> I’m trying to check my SQM settings on OpenWRT against bufferbloat.
> 
> For some reason, the TCP download and ping graphs for RRUL sessions
> give sensible results, whereas the TCP upload is broken into very
> small segments (even closer to dots actually), see the attached
> sample.
> 
> What am I doing wrong ?
> 
> command line: flent rrul -p all_scaled -l 60 -H host -o output.png
> 
> $ flent --version
> Flent v1.0.1.
> Running on Python 3.5.2 (default, Oct 11 2016, 04:59:56)  [GCC 4.2.1
> Compatible Apple LLVM 8.0.0 (clang-800.0.38)].
> Using matplotlib version 2.0.1 on numpy 1.12.1.
> Using PyQt5 version 5.9.
> 
> 
> Thanks,
> Manu.
> ___
> Flent-users mailing list
> Flent-users@flent.org
> http://flent.org/mailman/listinfo/flent-users_flent.org

___
Flent-users mailing list
Flent-users@flent.org
http://flent.org/mailman/listinfo/flent-users_flent.org