Re: libev and libevent

[Marc Lehmann]

On Wed, Feb 21, 2018 at 07:03:21AM +0800, Andy Green  wrote:
> > Maybe you can borrow some ideas or even some code from them.
> 
> Maybe you can fix your library headers.  Or maybe not.

Just out of curiosity, which of Harald's libraries are you refering to?

On Wed, Feb 21, 2018 at 07:26:03AM +0800, Andy Green  wrote:
> > 
> > Anyhow, I don't think this is the right place for introductory programming
> 
> You brought this subject up.

I pointed at a solution to your problem, which you claimed isn't a
solution because your prtogramming skills weren't good enough to be
familiar with it, after which I either had the choice of leaving you
or teach you about this technique i more detail. Opaque types and data
hiding is a very standard technique used in almost all libraries, so
it's not really a libev thing, thus the reference to the "introductory
programming", because that is what it is, a basic technique in C
programming.

So, if by "you brought this up" you mean "you tried to help me with my
problem" then you are spot on.

> > Quite possibly, lws_ev_io_watcher could even be identical to
> > struct ev_io, and with some work, it doesn't even have to be a pointer, and
> > lws_ev_watcher might not need to be a separate data type either.
> 
> That is the first constructive suggestion, thanks.

The thread is full of constructive suggestions, it's just that you were
unable or unwiling to understand and/or apply them.

> It adds another layer of indirection.  But actually that is probably a
> good idea.

It also incidentally shows most of the strongly worded and derisive claims
you made to be untrue. What are your thoughts about that?

> > > However you cannot compose an opaque / forward-referenced struct into
> > > another struct with type safety, because the size of the undefined thing 
> > > is
> > > unknown.
> > 
> > Yes, you can, multiple ways.
> 
> Note the word "compose" has a specific meaning here...

... a meaning which you won't explain here, so this is a useless comment,
because nobody knows what's going in on your mind. What I said is
certainly true.

> > Again, not trivial, but what you are trying to do is hardly typical.
> 
> All of these event loop things are contributed.  I don't think any of these
> extreme tricks to workaround your header clash problem will help with
> maintainability or source code clarity.

None of these "extreme" tricks as you call them are needed, either, mind you.

> > To say it bluntly, you come about as very tight-lipped about your needs
> > and a bit arrogant, when considering that your programming skills are
> > comparatively limited, given you are quite troubled by even simple
> > abstraction problems in C...
> 
> Wow!  You are living down to my expectations :-)

If by that you mean "I attack you and then you call me out, and I totally
expected that" then there is nothing surprising about it indeed.

> > Both Harald and I have tried to help you, and it would suit you well if you
> > would explain your problem a bit more and would use rational arguments
> > instead of shouting at people...
> 
> No... you preferred to attack the user and find a way that nothing needed to

You are very deluded - you used ad hominems instead of rational arguments
almost form the very beginning. I merely pointed out that your progamming
skills are lacking and your tone is arrogant, certainly in relation to your
actual skills - again, it would become you well to show a bit more humility
and learn from people who, clearly, know better how to design APIs and
implement data hiding than you do.

What you are doing is projecting your own attacks on others, but that doesn't
make it true.

Even if you can't show any humility or the will to learn, you should
certainly use rational arguments instead of hand waving, shouting, and making
stuff up just to disagree.

> change on your side, even when a comparable, more popular library has no
> problem with its header hygiene doing the same thing.

Nothing in this thread is about what any reasonable person would call
"header hygiene". The fact that libev provides a libevent-compatible API
and therefore also provides its, well, public API symbols is construed by
you to be some header hygiene problem. You clearly either don't understand
what header hygiene means or you are misusing it on purpose.

This is just another example of a non-rational made-up argument without any
merit or even content.

> As I said I will just leave it unselectable for choose-at-runtime in cmake -
> and suggest people avoid using libev.

You of course didn't say this before, but that's an entirely rational
solution if you are incapable of solving this problem yourself. I.e.
giving up when your skills are not up tp the job is probably the best
solution, also for your users.

It's also fine to suggest people to avoid using libev - libev was always
provided "as is", and certainly is a tool for experts and not beginners,
and anybody making thier event 

Re: libev and libevent

[Andy Green]

On 20/02/18 19:50, Marc Lehmann wrote:

On Tue, Feb 20, 2018 at 10:35:01AM +0800, Andy Green  wrote:

Not really, even C supports opaque types.


No, there is no "not really" about it.

C supports opaque type pointers because the size of a pointer is known
regardless of the details of what it points to.


To be pedantic, C supports opaque pointers because it is defined to do so,
not because pointers naturally all have the same size (they don't).

Anyhow, I don't think this is the right place for introductory programming


You brought this subject up.


in C, but the solution, as already mentioned, is to move libev code into
e.g. lws-libev.c, libevent code into lws-libevent.c and so on, and then
define an abstraction to those, e.g.

struct lws_ev_watcher;

lws_ev_io_watcher *
lws_ev_io_new (...);

void
lws_ev_io_start (lws_ev_watcher *, ...);

...

lws_ev_watcher is your opaque data type, and lws_ev_new etc. are the
methods used to access them.

Then you can use those in your lws_io_watcher struct:

struct lws_io_watcher {
#ifdef LWS_WITH_LIBEV
lws_ev_io_watcher *ev_watcher;
#endif

Quite possibly, lws_ev_io_watcher could even be identical to
struct ev_io, and with some work, it doesn't even have to be a pointer, and
lws_ev_watcher might not need to be a separate data type either.


That is the first constructive suggestion, thanks.  It adds another 
layer of indirection.  But actually that is probably a good idea. 
Reducing the scope of the includes if it is possible is also a good way.


There are other event-lib api types that are exposed in event-lib 
specific lws apis, eg, pass in an external loop object as the active 
event loop.  If these are all pointers then it can be solved with 
forward references.



Since you are not really telling us why you think you need this, it's hard to
tell you which solution might be best.


However you cannot compose an opaque / forward-referenced struct into
another struct with type safety, because the size of the undefined thing is
unknown.


Yes, you can, multiple ways.


Note the word "compose" has a specific meaning here...


I must have your struct definition and that means I must have your headers.


Yes, but not in the same source file as the headers for libevent. This is a
pretty standard problem with pretty standard solutions.

You can do away with pointers with some tricks, as the libev ABI has a
fixed watcher size that you could determine at compile time and reserve
enough space in your lws_io_watcher struct. This would result in virtually
no performance loss due to the indirection.

It's not trivial, but certainly possible with commonly used programming
techniques. It probbaly wouldn't buy you anything that you couldn't beat
by embedding libev, though.

The other problem, the ABI clash, can either be fixed with a custom libev
version not compiling in the event code (easy to do), or by using e.g.
shared libraries and dlopen, which allows you to open both libevent and
libev despite symbol clashes.

Again, not trivial, but what you are trying to do is hardly typical.


All of these event loop things are contributed.  I don't think any of 
these extreme tricks to workaround your header clash problem will help 
with maintainability or source code clarity.  They are just not needed 
to be considered with, eg, libuv.



Embedding libev, would enable a lot of other possible solutions as well.

A last comment on your style here - I have looked hard, but I still
couldn't find a good explanation (or any explanation) of why you need to
do it this way as opposed to separate builds or even backends for each
event library, which might be faster, more efficient, and easier to extend
than your current approach (compare e.g. how gtk+ selects backends, how
opengl/opencl work and so on).

Nothing you said explains why you need to be able to switch event
libraries at runtime, as opposed to e.g. select it at runtime by loading
the correct backend library.

To say it bluntly, you come about as very tight-lipped about your needs
and a bit arrogant, when considering that your programming skills are
comparatively limited, given you are quite troubled by even simple
abstraction problems in C...


Wow!  You are living down to my expectations :-)


Both Harald and I have tried to help you, and it would suit you well if you
would explain your problem a bit more and would use rational arguments
instead of shouting at people...


No... you preferred to attack the user and find a way that nothing 
needed to change on your side, even when a comparable, more popular 
library has no problem with its header hygiene doing the same thing.


As I said I will just leave it unselectable for choose-at-runtime in 
cmake - and suggest people avoid using libev.


-Andy

___
libev mailing list
libev@lists.schmorp.de
http://lists.schmorp.de/mailman/listinfo/libev

Re: libev and libevent

[Andy Green]

On 21/02/18 02:50, Harald Geyer wrote:

Andy Green writes:

Libuv seems to manage it just fine.


libuv used to depend on libev, so that's no surprise. It wouldn't do for
libuv to conflict it's own dependency ...


libuv is also clean against libevent, unlike libev :-)


it's not going to change no matter what I say.


That's probably true: Even if you had a convincing use case for including
libevent and libev headers in the same source file, Marc can't make it
happen: This would be forcing an API change on all users of libev. Ie make
their progamms stop compiling from one version to the next.


If someone had replied with that, I can understand it.  But they reply 
with attacks on the user.


Although we could have discussed the ins and outs of that if this was a 
good-faith discussion and not a sandbagging.  For example OpenSSL has a 
bazillion api versions but it can be managed with OPENSSL_VERSION.  If 
that's not applicable, okay, we could have arrived at that and gone away 
knowing that.



Maybe you can borrow some ideas or even some code from them.


Maybe you can fix your library headers.  Or maybe not.

-Andy


HTH,
Harald



___
libev mailing list
libev@lists.schmorp.de
http://lists.schmorp.de/mailman/listinfo/libev

Re: libev and libevent

[Harald Geyer]

Andy Green writes:
> Libuv seems to manage it just fine.

libuv used to depend on libev, so that's no surprise. It wouldn't do for
libuv to conflict it's own dependency ...

> it's not going to change no matter what I say.

That's probably true: Even if you had a convincing use case for including
libevent and libev headers in the same source file, Marc can't make it
happen: This would be forcing an API change on all users of libev. Ie make
their progamms stop compiling from one version to the next.

I'm pretty sure, you wouldn't do that to your users either...

But have a look at libelektra: https://github.com/ElektraInitiative/libelektra

They are pretty much in the same situation as you. (Users might want to
use asynchroneous IO, but might be using any event loop in their application.)
So far they only work on supporting libuv, yet they do the right thing
from the beginning: Have a separate libuv-binding, that can be installed
or not installed as the user pleases. In future additional bindings for
libev, libevent, glib, etc. can be added if somebody provides them.

And libelektra is a huge project, pulling in a few smallish extra libs
wouldn't change the memory footprint or anything much. Still they don't
do it, because forcing unnecessary dependencies on their users is wrong
for a lot of reasons.

Maybe you can borrow some ideas or even some code from them.

HTH,
Harald

-- 
If you want to support my work:
see http://friends.ccbib.org/harald/supporting/
or donate via CLAM to xASPBtezLNqj4cUe8MT5nZjthRSEjrRQXN
or via peercoin to P98LRdhit3gZbHDBe7ta5jtXrMJUms4p7w

___
libev mailing list
libev@lists.schmorp.de
http://lists.schmorp.de/mailman/listinfo/libev

Re: libev and libevent

[Marc Lehmann]

On Tue, Feb 20, 2018 at 10:35:01AM +0800, Andy Green  wrote:
> > Not really, even C supports opaque types.
> 
> No, there is no "not really" about it.
> 
> C supports opaque type pointers because the size of a pointer is known
> regardless of the details of what it points to.

To be pedantic, C supports opaque pointers because it is defined to do so,
not because pointers naturally all have the same size (they don't).

Anyhow, I don't think this is the right place for introductory programming
in C, but the solution, as already mentioned, is to move libev code into
e.g. lws-libev.c, libevent code into lws-libevent.c and so on, and then
define an abstraction to those, e.g.

   struct lws_ev_watcher;

   lws_ev_io_watcher *
   lws_ev_io_new (...);

   void
   lws_ev_io_start (lws_ev_watcher *, ...);

   ...

lws_ev_watcher is your opaque data type, and lws_ev_new etc. are the
methods used to access them.

Then you can use those in your lws_io_watcher struct:

   struct lws_io_watcher {
   #ifdef LWS_WITH_LIBEV
   lws_ev_io_watcher *ev_watcher;
   #endif

Quite possibly, lws_ev_io_watcher could even be identical to
struct ev_io, and with some work, it doesn't even have to be a pointer, and
lws_ev_watcher might not need to be a separate data type either.

Since you are not really telling us why you think you need this, it's hard to
tell you which solution might be best.

> However you cannot compose an opaque / forward-referenced struct into
> another struct with type safety, because the size of the undefined thing is
> unknown.

Yes, you can, multiple ways.

> I must have your struct definition and that means I must have your headers.

Yes, but not in the same source file as the headers for libevent. This is a
pretty standard problem with pretty standard solutions.

You can do away with pointers with some tricks, as the libev ABI has a
fixed watcher size that you could determine at compile time and reserve
enough space in your lws_io_watcher struct. This would result in virtually
no performance loss due to the indirection.

It's not trivial, but certainly possible with commonly used programming
techniques. It probbaly wouldn't buy you anything that you couldn't beat
by embedding libev, though.

The other problem, the ABI clash, can either be fixed with a custom libev
version not compiling in the event code (easy to do), or by using e.g.
shared libraries and dlopen, which allows you to open both libevent and
libev despite symbol clashes.

Again, not trivial, but what you are trying to do is hardly typical.

Embedding libev, would enable a lot of other possible solutions as well.

A last comment on your style here - I have looked hard, but I still
couldn't find a good explanation (or any explanation) of why you need to
do it this way as opposed to separate builds or even backends for each
event library, which might be faster, more efficient, and easier to extend
than your current approach (compare e.g. how gtk+ selects backends, how
opengl/opencl work and so on).

Nothing you said explains why you need to be able to switch event
libraries at runtime, as opposed to e.g. select it at runtime by loading
the correct backend library.

To say it bluntly, you come about as very tight-lipped about your needs
and a bit arrogant, when considering that your programming skills are
comparatively limited, given you are quite troubled by even simple
abstraction problems in C...

Both Harald and I have tried to help you, and it would suit you well if you
would explain your problem a bit more and would use rational arguments
instead of shouting at people...

-- 
The choice of a   Deliantra, the free code+content MORPG
  -==- _GNU_  http://www.deliantra.net
  ==-- _   generation
  ---==---(_)__  __   __  Marc Lehmann
  --==---/ / _ \/ // /\ \/ /  schm...@schmorp.de
  -=/_/_//_/\_,_/ /_/\_\

___
libev mailing list
libev@lists.schmorp.de
http://lists.schmorp.de/mailman/listinfo/libev

Re: libev and libevent

[Andy Green]

On 20/02/18 10:17, Marc Lehmann wrote:

On Tue, Feb 20, 2018 at 12:40:38AM +0800, Andy Green  wrote:

You sound confident about that, but actually having implemented it, you have 
not considered that the structures composing event lib state (handles, loop 
objects etc) must be defined in one place with types available for all event 
libs simultaneously.


Not really, even C supports opaque types.


No, there is no "not really" about it.

C supports opaque type pointers because the size of a pointer is known 
regardless of the details of what it points to.


However you cannot compose an opaque / forward-referenced struct into 
another struct with type safety, because the size of the undefined thing 
is unknown.


For example you do this:

typedef struct ev_io
{
  EV_WATCHER_LIST (ev_io)

  int fd; /* ro */
  int events; /* ro */
} ev_io;


and I do this

struct lws_io_watcher {
#ifdef LWS_WITH_LIBEV
ev_io ev_watcher;
#endif
#ifdef LWS_WITH_LIBUV
uv_poll_t uv_watcher;
#endif
#ifdef LWS_WITH_LIBEVENT
struct event *event_watcher;
#endif
struct lws_context *context;

uint8_t actual_events;
};

I must have your struct definition and that means I must have your headers.

-Andy

___
libev mailing list
libev@lists.schmorp.de
http://lists.schmorp.de/mailman/listinfo/libev

Re: libev and libevent

[Marc Lehmann]

On Tue, Feb 20, 2018 at 12:40:38AM +0800, Andy Green  wrote:
> You sound confident about that, but actually having implemented it, you have 
> not considered that the structures composing event lib state (handles, loop 
> objects etc) must be defined in one place with types available for all event 
> libs simultaneously.

Not really, even C supports opaque types.

-- 
The choice of a   Deliantra, the free code+content MORPG
  -==- _GNU_  http://www.deliantra.net
  ==-- _   generation
  ---==---(_)__  __   __  Marc Lehmann
  --==---/ / _ \/ // /\ \/ /  schm...@schmorp.de
  -=/_/_//_/\_,_/ /_/\_\

___
libev mailing list
libev@lists.schmorp.de
http://lists.schmorp.de/mailman/listinfo/libev

Re: libev and libevent

[Andy Green]

On 20/02/18 02:05, Harald Geyer wrote:

Hi!

Disclaimer: I'm not a libev developer. Just following this list as a
regular user.

Andy Green writes:

I'm proposing libevent and libev stop defining the same symbols to
different values and conflicting, the same way libev and libuv have
clean namespaces that don't conflict. It's quite basic.


Actually different projects implementing the same API is a good thing.


libev is only supporting some deprecated libevent version from what I 
understood.  It's not a good thing to have dirty, conflicting namespaces 
just for the sake of it.  Libuv seems to manage it just fine.



Usually people even come up with standards just to make this happen.


:-)


Of course you can't link a program to glibc and musllibc at the same
timer either ...


Sure, there are many cases where you can only have one of something. 
This doesn't have to be one of them as I show by [1]



is "choose at runtime" even supposed to mean? Do you link to all of
them and then control into which one the code actually calls? Do you
dl_open() the one you want to use when the application starts?


If you actually answered these questions, we might already be a step
further.


I answered this in my original reply here [2]  Did we get any further? 
No.  Because it's not going to change no matter what I say.



So I actually looked at the lws packages in debian and indeed, there
one binary depends on both libev and libuv...


[1]... which works fine.  Because libuv and libev namespaces do not 
conflict and despite the "shocked into silence" here, there is no 
problem having them in parallel.



I can't see how this can possibly work. Or rather: Any solution I can
think of requires a lot of abstraction code probably causing enough
overhead to make the exercise pointless.


The overhead is simply test a bit per event library at some key points,
accept, close, change event mask. Once the enabled lib does the accept,
the selected lib directly calls the event callback itself. The overhead
is insignificant.


[2]


But really: What's the point of choosing the event loop implementation


Lws has many projects using it, and they use different event loops.
Consider a distro that wants to include even two such projects using lws
as a dynamic lib... they must package lws, and the two projects... but
how do they configure lws to do that? They can build + package lws n
times for n event loops, or choose at runtime.


Yes, exactly. This is what they should do. (I'm maintaining library
packages for a linux distribution myself.)

Ideally there would be packages


What actually exists also works fine, and lws extends to very small 
targets like ESP32 where none of this is a concern.


https://github.com/warmcat/lws-esp32-factory


lws-common
lws-ev (depending on lws-common)
lws-uv (depending on lws-common)
app1 (depending on lsw-ev)
app2 (depending on lsw-uv)

If there is not enough event loop independed code to have lws-common,
then lws just needs to get compiled twice to produce both flavors ...

Your approach just doesn't scale. Just think about this from the


Is that so.


perspective of a user only installing app1. With your current approach
he ends up with libev and libuv installed, but one is unnecessary.
Now you add support for libevent, so he has to install that two - two
unnecessary packages. In the future you might add even more event loops,
pulling in even more packages.

This just doesn't scale well.


Currently n = 3 and no sign of more coming... libevent was only 
contributed by Samsung because they use it in Tizen or whatever.  It 
scaled to where it needed to be just fine.



at run time? If there is something for the distribution user to choose
from, then clearly multiple options are available in the distribution.
So why not just pick one at compile time?

If you really insist on making the event loop implementation selectable
(probeable?) at runtime, then I think the only viable approach is to


This already works for many years. What you're suggesting is not enough,
because each event lib requires some state in structs related to
connections.


Hence me saying the abstraction layer to do this properly would be
a nightmare.


For n = 3, it is no problem and the overhead is not measurable.  I agree 
though, if n grew much more, it would have to trigger a refactor of some 
kind on my side.



Now then, this is the only library we bind to where I have to tell CMake 
not to use -Werror on the sources where we use it, because it's not 
possible to use it with gcc without getting warnings.  This was reported 
years ago and gcc blamed by the maintainer, nothing changed.  Warnings 
for everyone on gcc was the "solution".


This is the reason I tried to discuss with libevent first, but the idea 
libev would contain a conflicting, deprecated, libevent was beyond my 
withered imagination.  If there is no willingness to clean this up in 
libev, it looks like it cannot be fixed, so I will just leave it that 
you 

Re: libev and libevent

[Harald Geyer]

Hi!

Disclaimer: I'm not a libev developer. Just following this list as a
regular user.

Andy Green writes:
> I'm proposing libevent and libev stop defining the same symbols to
> different values and conflicting, the same way libev and libuv have
> clean namespaces that don't conflict. It's quite basic.

Actually different projects implementing the same API is a good thing.
Usually people even come up with standards just to make this happen.

Of course you can't link a program to glibc and musllibc at the same
timer either ...

> >is "choose at runtime" even supposed to mean? Do you link to all of
> >them and then control into which one the code actually calls? Do you
> >dl_open() the one you want to use when the application starts?

If you actually answered these questions, we might already be a step
further.

So I actually looked at the lws packages in debian and indeed, there
one binary depends on both libev and libuv...

> >I can't see how this can possibly work. Or rather: Any solution I can
> >think of requires a lot of abstraction code probably causing enough
> >overhead to make the exercise pointless.
> 
> The overhead is simply test a bit per event library at some key points,
> accept, close, change event mask. Once the enabled lib does the accept,
> the selected lib directly calls the event callback itself. The overhead
> is insignificant.
> 
> >But really: What's the point of choosing the event loop implementation
> 
> Lws has many projects using it, and they use different event loops.
> Consider a distro that wants to include even two such projects using lws
> as a dynamic lib... they must package lws, and the two projects... but
> how do they configure lws to do that? They can build + package lws n
> times for n event loops, or choose at runtime.

Yes, exactly. This is what they should do. (I'm maintaining library
packages for a linux distribution myself.)

Ideally there would be packages
lws-common
lws-ev (depending on lws-common)
lws-uv (depending on lws-common)
app1 (depending on lsw-ev)
app2 (depending on lsw-uv)

If there is not enough event loop independed code to have lws-common,
then lws just needs to get compiled twice to produce both flavors ...

Your approach just doesn't scale. Just think about this from the
perspective of a user only installing app1. With your current approach
he ends up with libev and libuv installed, but one is unnecessary.
Now you add support for libevent, so he has to install that two - two
unnecessary packages. In the future you might add even more event loops,
pulling in even more packages.

This just doesn't scale well.

> >at run time? If there is something for the distribution user to choose
> >from, then clearly multiple options are available in the distribution.
> >So why not just pick one at compile time?
> >
> >If you really insist on making the event loop implementation selectable
> >(probeable?) at runtime, then I think the only viable approach is to
> 
> This already works for many years. What you're suggesting is not enough,
> because each event lib requires some state in structs related to
> connections.

Hence me saying the abstraction layer to do this properly would be
a nightmare.

Harald


> At the point those structs are declared, all the types must
> have been brought in from all related libs. Hence the need for headers
> for the event libs to have clean namespaces.
> 
> >move all your event loop facing code into a separate module and compile
> >this module multiple times - once per event loop supported. Then the
> >right instance of the module can be dynamically linked at run time by
> >either having the right symlinks in place or dl_open().
> 
> If you don't see this as a problem for libevent + libev to sort out,
> then not much option but keep the two impossible to select together at
> cmake, unless it can be solved at libevent.
> 
> -Andy
> 
> >HTH, arald

-- 
If you want to support my work:
see http://friends.ccbib.org/harald/supporting/
or donate via CLAM to xASPBtezLNqj4cUe8MT5nZjthRSEjrRQXN
or via peercoin to P98LRdhit3gZbHDBe7ta5jtXrMJUms4p7w

___
libev mailing list
libev@lists.schmorp.de
http://lists.schmorp.de/mailman/listinfo/libev

Re: libev and libevent

[Andy Green]

On February 19, 2018 9:01:49 PM GMT+08:00, Harald Geyer  
wrote:
>Andy Green writes:
>> For downstream maintainers like me whose users wish to be able to use
>
>> both, and where distro consumers should be able to tick all the boxes
>
>> and choose at runtime, are there any ideas what can be done to
>improve 
>> the situation?
>
>It might be a good idea to support selecting between libraries at
>compile
>time; I'm not going to comment on that.

Lws uses cmake, where the event loop support can be individually enabled and 
disabled.

>But what you are proposing here made me speechless for a while. What

I'm not proposing lws' internal workings to you.  It already does this for 
years in liuv / libev case and all works nice.

I'm proposing libevent and libev stop defining the same symbols to different 
values and conflicting, the same way libev and libuv have clean namespaces that 
don't conflict.  It's quite basic.

>is "choose at runtime" even supposed to mean? Do you link to all of
>them
>and then control into which one the code actually calls? Do you
>dl_open()
>the one you want to use when the application starts?
>
>I can't see how this can possibly work. Or rather: Any solution I can
>think of requires a lot of abstraction code probably causing enough
>overhead to make the exercise pointless.

The overhead is simply test a bit per event library at some key points, accept, 
close, change event mask.  Once the enabled lib does the accept, the selected 
lib directly calls the event callback itself.  The overhead is insignificant.

>But really: What's the point of choosing the event loop implementation

Lws has many projects using it, and they use different event loops.  Consider a 
distro that wants to include even two such projects using lws as a dynamic 
lib... they must package lws, and the two projects... but how do they configure 
lws to do that?  They can build + package lws n times for n event loops, or 
choose at runtime.

>at run time? If there is something for the distribution user to choose
>from, then clearly multiple options are available in the distribution.
>So why not just pick one at compile time?
>
>If you really insist on making the event loop implementation selectable
>(probeable?) at runtime, then I think the only viable approach is to

This already works for many years.  What you're suggesting is not enough, 
because each event lib requires some state in structs related to connections.  
At the point those structs are declared, all the types must have been brought 
in from all related libs.  Hence the need for headers for the event libs to 
have clean namespaces.

>move all your event loop facing code into a separate module and compile
>this module multiple times - once per event loop supported. Then the
>right instance of the module can be dynamically linked at run time by
>either having the right symlinks in place or dl_open().

If you don't see this as a problem for libevent + libev to sort out, then not 
much option but keep the two impossible to select together at cmake, unless it 
can be solved at libevent.

-Andy

>HTH,
>Harald

___
libev mailing list
libev@lists.schmorp.de
http://lists.schmorp.de/mailman/listinfo/libev

Re: libev and libevent

[Marc Lehmann]

On Mon, Feb 19, 2018 at 08:10:49PM +0800, Andy Green  wrote:
> Since libevent uses the preprocessor, it gets the final say; in other words
> since it defines EV_READ etc to a different value, building some software
> that can operate with both libevent and libev results in nothing working or
> not even build completing if you include the libevent bits first.

Most software choses the event model at compile time, where this causes
no issue. If you want to support both at runtime, just put the libev code
in one source file and the libevent code in another, and this problem is
solved.

> This is basically an interoperability problem between libevent and libev, or
> libev and libevent, it would be great if there is something that can be done
> to allow the happy coexistence merry-go-round to spin once more for these
> two libraries.

It's not really a problem in practise, and certainly not an obstacle for
users of these libraries.

> for libevent, but I learned there the situation is more complex than I
> expected, with libev providing some libevent compatibility by design.

Yes, libev offers the libevent (v1) api as well.

> For downstream maintainers like me whose users wish to be able to use both,

What you described doesn't seem to be an actual problem - can you give a
more detailed explanation, maybe with an example, of what you want to do
and can't?

-- 
The choice of a   Deliantra, the free code+content MORPG
  -==- _GNU_  http://www.deliantra.net
  ==-- _   generation
  ---==---(_)__  __   __  Marc Lehmann
  --==---/ / _ \/ // /\ \/ /  schm...@schmorp.de
  -=/_/_//_/\_,_/ /_/\_\

___
libev mailing list
libev@lists.schmorp.de
http://lists.schmorp.de/mailman/listinfo/libev

Re: libev and libevent

[Harald Geyer]

Andy Green writes:
> For downstream maintainers like me whose users wish to be able to use 
> both, and where distro consumers should be able to tick all the boxes 
> and choose at runtime, are there any ideas what can be done to improve 
> the situation?

It might be a good idea to support selecting between libraries at compile
time; I'm not going to comment on that.

But what you are proposing here made me speechless for a while. What
is "choose at runtime" even supposed to mean? Do you link to all of them
and then control into which one the code actually calls? Do you dl_open()
the one you want to use when the application starts?

I can't see how this can possibly work. Or rather: Any solution I can
think of requires a lot of abstraction code probably causing enough
overhead to make the exercise pointless.

But really: What's the point of choosing the event loop implementation
at run time? If there is something for the distribution user to choose
from, then clearly multiple options are available in the distribution.
So why not just pick one at compile time?

If you really insist on making the event loop implementation selectable
(probeable?) at runtime, then I think the only viable approach is to
move all your event loop facing code into a separate module and compile
this module multiple times - once per event loop supported. Then the
right instance of the module can be dynamically linked at run time by
either having the right symlinks in place or dl_open().

HTH,
Harald

-- 
If you want to support my work:
see http://friends.ccbib.org/harald/supporting/
or donate via CLAM to xASPBtezLNqj4cUe8MT5nZjthRSEjrRQXN
or via peercoin to P98LRdhit3gZbHDBe7ta5jtXrMJUms4p7w

___
libev mailing list
libev@lists.schmorp.de
http://lists.schmorp.de/mailman/listinfo/libev

Re: Libev vs. libevent benchmark (reloaded)

[Marc Lehmann]

On Tue, Apr 13, 2010 at 03:26:19PM +0200, Gabriel Kerneis 
kern...@pps.jussieu.fr wrote:
 See the attached script.  You might need to tweak it a bit (especially
 the cpufreq-set call) but otherwise, it automates completely the
 benchmark (provided you have the expected tools installed).

I tried your script, but:

a) it didn't work, as you don't specify the correct include path
b) after fixing that, the resulting binary loops internal in libevent
   (as in my previous tests), for libevent-2

So it's very likely that you compiled against the wrong set of include
files - how that would affect your timings I have no clue, (with luck, not
at all because you have it installed).

-- 
The choice of a   Deliantra, the free code+content MORPG
  -==- _GNU_  http://www.deliantra.net
  ==-- _   generation
  ---==---(_)__  __   __  Marc Lehmann
  --==---/ / _ \/ // /\ \/ /  schm...@schmorp.de
  -=/_/_//_/\_,_/ /_/\_\

___
libev mailing list
libev@lists.schmorp.de
http://lists.schmorp.de/cgi-bin/mailman/listinfo/libev

Re: Libev vs. libevent benchmark (reloaded)

[Marc Lehmann]

On Tue, Apr 13, 2010 at 03:26:19PM +0200, Gabriel Kerneis 
kern...@pps.jussieu.fr wrote:
 See the attached script.  You might need to tweak it a bit (especially
 the cpufreq-set call) but otherwise, it automates completely the
 benchmark (provided you have the expected tools installed).

As an interesting side-note, it seems libevent-2.0.4 uses the same
trick as libev w.r.t. epoll now, except that it doesn't protect against
the linux epoll fork bug (or the deliberate design, depending on
viewpoint), which explains the many reports I got that libevent-2 went
into busy loops after a fork (which I thought impossible).

-- 
The choice of a   Deliantra, the free code+content MORPG
  -==- _GNU_  http://www.deliantra.net
  ==-- _   generation
  ---==---(_)__  __   __  Marc Lehmann
  --==---/ / _ \/ // /\ \/ /  schm...@schmorp.de
  -=/_/_//_/\_,_/ /_/\_\


___
libev mailing list
libev@lists.schmorp.de
http://lists.schmorp.de/cgi-bin/mailman/listinfo/libev

Re: Libev vs. libevent benchmark (reloaded)

[Marc Lehmann]

On Tue, Apr 13, 2010 at 03:26:19PM +0200, Gabriel Kerneis 
kern...@pps.jussieu.fr wrote:
 See the attached script.  You might need to tweak it a bit (especially
 the cpufreq-set call) but otherwise, it automates completely the
 benchmark (provided you have the expected tools installed).

[As an interesting side-note, it seems libevent-2.0.4 uses the same]

Sorry, forgot to mention that this can easily explain the perceived
performance, as libev does EPOLL_CTL_MOD, while libevent does
EPOLL_CTL_ADD, which is faster in the benchmark case (it simply errors out
without writing memory), but does not support fork.

Working correctly with fork is, of course, an advertised feature of libev.

-- 
The choice of a   Deliantra, the free code+content MORPG
  -==- _GNU_  http://www.deliantra.net
  ==-- _   generation
  ---==---(_)__  __   __  Marc Lehmann
  --==---/ / _ \/ // /\ \/ /  schm...@schmorp.de
  -=/_/_//_/\_,_/ /_/\_\


___
libev mailing list
libev@lists.schmorp.de
http://lists.schmorp.de/cgi-bin/mailman/listinfo/libev

Re: Libev vs. libevent benchmark (reloaded)

[Marc Lehmann]

On Mon, Apr 19, 2010 at 01:26:33AM +0200, Marc Lehmann schm...@schmorp.de 
wrote:
 Sorry, forgot to mention that this can easily explain the perceived
 performance, as libev does EPOLL_CTL_MOD, while libevent does
 EPOLL_CTL_ADD, which is faster in the benchmark case (it simply errors out
 without writing memory), but does not support fork.

If anybody who can actually benchmark this (I still can't), you can get the
same effect by replacing:

  if (expect_true (!epoll_ctl (backend_fd, oev ? EPOLL_CTL_MOD : EPOLL_CTL_ADD, 
fd, ev)))

with:

  if (expect_true (!epoll_ctl (backend_fd, oev  oev != nev ? EPOLL_CTL_MOD : 
EPOLL_CTL_ADD, fd, ev)))

in ev_epoll.c.

-- 
The choice of a   Deliantra, the free code+content MORPG
  -==- _GNU_  http://www.deliantra.net
  ==-- _   generation
  ---==---(_)__  __   __  Marc Lehmann
  --==---/ / _ \/ // /\ \/ /  schm...@schmorp.de
  -=/_/_//_/\_,_/ /_/\_\

___
libev mailing list
libev@lists.schmorp.de
http://lists.schmorp.de/cgi-bin/mailman/listinfo/libev

Re: Libev vs. libevent benchmark (reloaded)

[Marc Lehmann]

On Mon, Apr 19, 2010 at 01:05:21AM +0200, Marc Lehmann schm...@schmorp.de 
wrote:
 Note also that you compile the libev binary with incompatible headers:
 
 libtool --mode=link gcc -O3 -fno-guess-branch-probability -g bench.c -I..  
 ../libev.la -o bench -DNATIVE -DEV_MULTIPLICITY=0 
 
 
 How it would work in native mode as opposed to instantly crashing is
 something I'd like to know :)

silly me, was just told that configure explicitly sets EV_MULTIPLICITY to
explicitly, so one mystery less.

-- 
The choice of a   Deliantra, the free code+content MORPG
  -==- _GNU_  http://www.deliantra.net
  ==-- _   generation
  ---==---(_)__  __   __  Marc Lehmann
  --==---/ / _ \/ // /\ \/ /  schm...@schmorp.de
  -=/_/_//_/\_,_/ /_/\_\

___
libev mailing list
libev@lists.schmorp.de
http://lists.schmorp.de/cgi-bin/mailman/listinfo/libev

Re: Libev vs. libevent benchmark (reloaded)

[Gabriel Kerneis]

Marc,

On Mon, Apr 12, 2010 at 05:10:29PM +0200, Marc Lehmann wrote:
 If you ran the code unmodified, how did you do that? :)

See the attached script.  You might need to tweak it a bit (especially
the cpufreq-set call) but otherwise, it automates completely the
benchmark (provided you have the expected tools installed).

Please, double-check my patching of runbench just in case.

 Now, since libevent does an enourmous number of pointer operations,
 mutex ops, syscalls etc. per watcher setup in 2.0, I find this a bit
 questionable (but it's possible).

The results are available here:
http://.pps.jussieu.fr/~kerneis/software/files/libev-vs-libevent

libevent2 is very fast, but still beaten when timeouts are involved.

Regards,
-- 
Gabriel Kerneis


bench.sh
Description: Bourne shell script
___
libev mailing list
libev@lists.schmorp.de
http://lists.schmorp.de/cgi-bin/mailman/listinfo/libev

Re: Libev vs. libevent benchmark (reloaded)

[Marc Lehmann]

On Tue, Apr 13, 2010 at 03:26:19PM +0200, Gabriel Kerneis 
kern...@pps.jussieu.fr wrote:
 On Mon, Apr 12, 2010 at 05:10:29PM +0200, Marc Lehmann wrote:
  If you ran the code unmodified, how did you do that? :)
 
 See the attached script.  You might need to tweak it a bit (especially

No, I mean the bench.c doesn't work with libvent-2.0.4 on my system, it
just hangs in the event loop. Others have reported the same, so I wonder
what you did different to make it work.

(have you strace'ed it to see if it actually does the syscalls required for
the benchmark?)

 Please, double-check my patching of runbench just in case.

I blindly trust you there :)

 The results are available here:
 http://.pps.jussieu.fr/~kerneis/software/files/libev-vs-libevent
 
 libevent2 is very fast, but still beaten when timeouts are involved.

Libev also beats it consistently with 1000 clients, just not in the 100
client case.

Still scary - maybe it does the many epoll calls at a different
time (libev delays them till event processing, libevent does it at
setup/takedown). It actually looks like a cache effect though, but thats
really weird, as libev's data structures are so much smaller.

I'll have to investigate (But first I need to have a working libevent
bench.c :).

-- 
The choice of a   Deliantra, the free code+content MORPG
  -==- _GNU_  http://www.deliantra.net
  ==-- _   generation
  ---==---(_)__  __   __  Marc Lehmann
  --==---/ / _ \/ // /\ \/ /  schm...@schmorp.de
  -=/_/_//_/\_,_/ /_/\_\

___
libev mailing list
libev@lists.schmorp.de
http://lists.schmorp.de/cgi-bin/mailman/listinfo/libev

Re: Libev vs. libevent benchmark (reloaded)

[Marc Lehmann]

On Mon, Apr 12, 2010 at 01:46:22PM +0200, Gabriel Kerneis 
kern...@pps.jussieu.fr wrote:
 or 100,000 fds with timeout and libevent emulation), libevent is faster
 than libev in terms of total time per iteration.  But libev remains
 consistently faster when considering event processing only.

Interesting.

 I ran the benchmark on an Intel Core2 Duo CPU E8500 @ 3.16GHz.  I wonder

I tried to run the benchmark myself last month, but due to changes in
libevent 2 (I used libevent-2.0.4-alpha), the current code does not run at
all. I fixed parts of it, but not all yet.

If you ran the code unmodified, how did you do that? :)

Now, since libevent does an enourmous number of pointer operations,
mutex ops, syscalls etc. per watcher setup in 2.0, I find this a bit
questionable (but it's possible).

 As an aside, Marc, did you keep the scripts you used to run the bench
 and generate the graphics?  It would make it easier to compare the
 results.

http://data.plan9.de/runbench

This is used to run the benchmark and plot the results - it requires two
binaries, bench (libev) and bench-le (libevent).

-- 
The choice of a   Deliantra, the free code+content MORPG
  -==- _GNU_  http://www.deliantra.net
  ==-- _   generation
  ---==---(_)__  __   __  Marc Lehmann
  --==---/ / _ \/ // /\ \/ /  schm...@schmorp.de
  -=/_/_//_/\_,_/ /_/\_\

___
libev mailing list
libev@lists.schmorp.de
http://lists.schmorp.de/cgi-bin/mailman/listinfo/libev