Stephen Bailey <st...@cs.uchicago.edu> writes:
> Oh, and one more topic:
>
> Do you have any thoughts for a more elegant mechanism to bind remote
> hosts to runners than runner index? It's woefully dependent on the
> test internals, and when the internals change, the runner bindings
> change.
Well, had to add that was added somewhat in a rush, so yeah, the
interface is a little rough. I'm not sure what would be a good
interface. Could match on the data set names, I suppose.
> I was sorta not horribly unhappy with how tcp_n{up,down} mapped the
> exercisers to runners -- seems like the netperfs started at 0, but
> rtt_fair_var is more complex. With my mod to support multiple
> netperfs/host, which will still only do only 1 ping/host, which is
> actually a good idea -- no point in running many pings, though I guess
> it doesn't really hurt anything -- the mapping is a bit chaotic.
>
> It seems like binding them by name & # (e.g. netperf 0, 1, 2, ....,
> ping 0, 1, ... whatever) would be much more usable.
Hmm, suppose each runner class could assign an ID based on its name and
an index, or something. It's probably better to think about the user
interface first; what is a good way to express what you want to do?
Could also be an extension to the 'host' syntax? I.e. a hostname could
be 'fromhost:tohost' which would make all runners going to that host be
remote runners from on 'fromhost'? But that would lose some granularity
compared to the current interface...
> I suppose we should be having this discussion on the flent list?
> [though not much on the flent list yet]
Yes, adding in that. Gotta start somewhere :)
>
> Thanks,
> Steph
>
> On Tue, Nov 1, 2016 at 11:18 AM, Stephen Bailey <st...@cs.uchicago.edu> wrote:
>
> I thought of that as well.... It works, but of course it's a bit messy :-).
> I'm sure you'll come up with something a bit more elegant. I don't have any
> awesome
> suggestions beyond my one already described. Of course I have no idea how
> everything is hooked together (e.g. I'm not using plots, I haven't tried the
> other users of
> rtt_fair_var ...).
>
> Another question: how do you develop flent 'in place'? I've been whacking
> away on the installed code, but I kinda hate that. How can I run it out of
> the tree I pulled
> from git?
>
> Thanks,
> Steph
>
> On Tue, Nov 1, 2016 at 1:35 AM, Toke Høiland-Jørgensen <t...@toke.dk> wrote:
>
> Yeah, you're right, forgot about that. It should be possible to fix this,
> but some care needs to be taken to avoid breaking backwards compatibility.
> I'll look into that.
>
> In the meantime, you can work around this limitation by using hostnames.
> I.e. assign multiple names to the same IP in /etc/hosts and use those names
> as the
> parameters to Flent.
>
> -Toke
>
> On 1 November 2016 04:56:04 CET, Stephen Bailey <st...@cs.uchicago.edu>
> wrote:
> >Whoops, taht was a runt message. Substantially correct, but incomplete
> >:-).
> >
> >Anyway, this specific snippet:
> >
> >DATA_SETS['TCP upload BE %s' % host]
> >
> >ends up updating the same entry of DATA_SETS multiple times (with the
> >same
> >content) when there are multiple copies of the same host string.
> >
> >Since I'm not sure of the entire vision of the rtt_fair_var_* tests I'm
> >not
> >sure what you'd want to do here. You could just tack on the 'i' from
> >the
> >enumerate to all test names ....?
> >
> >Thanks,
> > Steph
> >
> >
> >On Mon, Oct 31, 2016 at 8:52 PM, Stephen Bailey <st...@cs.uchicago.edu>
> >wrote:
> >
> >> Hi Toke,
> >>
> >> OK, that's helpful. I'm looking at rtt_fair_var_up now. It's
> >getting
> >> closer, but it seems like I can't supply the same destination host
> >multiple
> >> times (equivalent of the upload_streams=xxx). E.g. this command
> >line:
> >>
> >> flent rtt_fair_var_up --test-parameter
> >cpu_stats_hosts=localhost,10.13.241.183
> >> --format=metadata -l 10 --host=192.168.100.183 --host=192.168.100.183
> >>
> >> seems to only run one netperf session.
> >>
> >> This bit of rtt_fair_var.conf effectively uniqifies the host name
> >(which
> >> is not what one wants -- you should be able to run multiple netperfs
> >with
> >> the same endpoints (like the tcp_nup.conf case):
> >>
> >> for i,host in enumerate(HOSTS):
> >> print("testing ", i, " on ", host)
> >> control_host = CONTROL_HOSTS[i] if CONTROL_HOSTS else host
> >> local_bind = LOCAL_BIND[i] if LOCAL_BIND else None
> >> DATA_SETS['TCP upload BE %s' % host] = \
> >> {'command': find_netperf("TCP_STREAM", LENGTH, host, . . .
> >>
> >>
> >>
> >> On Fri, Oct 28, 2016 at 5:41 AM, Toke Høiland-Jørgensen
> ><t...@toke.dk>
> >> wrote:
> >>
> >>> Hi Stephen
> >>>
> >>> (replying to both your mails at once)
> >>>
> >>> > I love flent. I was one of those people you so broadly discussed
> >with
> >>> > my own little smouldering pile of Python to orchestrate netstats.
> >I
> >>> > eagerly look forward to dumping my run-netperf.py to embrace
> >flent.
> >>>
> >>> Awesome! Always good to hear when someone else is finding it useful
> >:)
> >>>
> >>> > A question (more like a ramble):
> >>> >
> >>> > How do I, as a user (versus a flent developer) add my own tests?
> >>>
> >>> I decided early on to not support this, for two reasons: (1) I'd
> >have to
> >>> support a stable interface to specify tests. And (2) one of the
> >goals of
> >>> Flent is to make tests interoperable and comparable between users,
> >so
> >>> I'd much rather encourage people to contribute tests back and
> >include
> >>> them in the main Flent sources. This is also the only way to make
> >sure
> >>> that data files can be shared between instances of Flent, since
> >loading
> >>> a data file is dependent on the test definition.
> >>>
> >>> Also, in recent versions, there are several tests that are
> >customisable
> >>> at the command line, to the point where I think you can do what you
> >want
> >>> with the existing tests (see below).
> >>>
> >>>
> >>> > I'm not sure if your vision is that test creation is a
> >>> > 'developer-level' activity (and even if it is, I think you should
> >be
> >>> > able to pick a different path for the tests outside the
> >installation
> >>> > hierarchy), or something users can do (I imagined the latter).
> >>>
> >>> No, I did intend that it should be possible for users (or, well,
> >>> advanced users, I guess) to specify their own tests. But my hope was
> >to
> >>> get people to contribute those back upstream, as I mentioned above
> >:)
> >>>
> >>> > And I wasn't sure if the batch files were a way to accomplish
> >building
> >>> > different tests (they didn't seem to be -- more like orchestration
> >of
> >>> > existing tests).
> >>>
> >>> Yes, the batch facility is for running several tests in a row and
> >doing
> >>> setup/teardown before/after.
> >>>
> >>> > If I can get the netperf stuff working the way I want, I have a
> >dream
> >>> > of adding MoonGen support too (since my test mix is netperf + ping
> >+
> >>> > MoonGen). However, that's WAY in the future at this point.
> >>>
> >>> Happy to take patches for that. Writing a parser for a new tool is
> >not
> >>> *too* complicated (depending on the tool of course; but for some
> >>> you can basically get away with just specifying a couple of regexes)
> >>>
> >>> > Ok, I eventually found tcp_n{up,down} much CLOSER to my objective
> >(TCP
> >>> > streams connecting arbitrary endpoints with CPU consumption
> >>> > measurements), but I'm still left slightly wanting.
> >>> >
> >>> > It doesn't appear these tests support multiple --host=xxx
> >arguments
> >>> > that would be required for complete arbitrary netperf connection
> >>> > 'wiring'. For example, I'd like to do this:
> >>> >
> >>> > % flent tcp_nup --length=10 --test-parameter
> >>> cpu_stats_hosts=10.13.241.183,10.13.241.184 --test-parameter
> >>> > upload_streams=2 --host=192.168.100.183 --host=192.168.200.183
> >>> >
> >>> > and this:
> >>> >
> >>> > % flent tcp_nup --length=10 --test-parameter
> >>> cpu_stats_hosts=10.13.241.183,10.13.241.184 --test-parameter
> >>> > upload_streams=2 --host=192.168.100.183
> >--remote-host=0=10.13.241.184
> >>> --host=192.168.200.183
> >>> > --remote-host=1=10.13.241.184
> >>> >
> >>> > which would basically run 1 netperf from 10.13.241.184 to each of
> >2
> >>> > different destinations. Of course you'd still only get 1 ping in
> >this
> >>> > case, but I'm actually not so concerned about the ping (or you
> >could
> >>> > instantiate 1 ping/host ...)
> >>>
> >>> What you're looking for is the rtt_fair_var* tests. Those will take
> >any
> >>> number of 'host' parameters and run a netperf instance to each.
> >Combine
> >>> that with the --remote-host and cpu_stats parameters and you should
> >be
> >>> good to go.
> >>>
> >>> > I'd also like the metadata from cpu_stats.inc to include the
> >average
> >>> > utilization for the run -- in that case the metadata would include
> >all
> >>> > the 'necessary' average data (throughput, ping latency & CPU
> >>> > consumption).
> >>>
> >>> So the logic has so far been that the metadata includes averages
> >only if
> >>> the underlying tool outputs an average as a separate piece of output
> >>> data. In all other cases, the averages are computed as needed. But I
> >can
> >>> see how having all the averages stored in the datafile would be
> >useful
> >>> for external tools that read the data file. Just haven't had anyone
> >>> requesting that before. It's easy to add though, I'll look into
> >that.
> >>>
> >>> Incidentally, the cpu stats is currently collected by a shell script
> >>> that parses /proc/stat by using awk. This works reasonably well if
> >you
> >>> have enough CPU to keep up with the polling frequency, but on
> >>> low-powered systems it can fall behind. There are C implementations
> >of
> >>> some of the other stats collection functions to deal with this, but
> >not
> >>> for cpu stats. So please keep an eye on this and let me know if it
> >>> becomes a problem; then I'll try to get around to creating a C
> >version
> >>> of that as well.
> >>>
> >>> -Toke
> >>>
> >>
> >>
>
-Toke
_______________________________________________
Flent-users mailing list
Flent-users@flent.org
http://flent.org/mailman/listinfo/flent-users_flent.org
