Re: libev: segmentation fault in ev_feed_event().
On Fri, Sep 08, 2023 at 02:32:46PM -0400, Olivier Langlois wrote: > but beside that, I have a hard time figuring out what could cause a > segv into that small function... Almost always, this is caused by race condiitons between threads. For example, starting or stopping watcheres from multiple threads without locking the loop. > As a side note, I think that the assert text should be: > "libev: pipe_w not active, but pipe written" We don't know if the pipe was written. What we do know is that the intent was that the pipe was not written, so the officially, the pipe was not written, but the watcher is also not active. > An exceptional occurence but theoritically possible is the following: > > ev_async_send() is called by another thread while the loop thread is > processing pending watchers and one of these watchers is calling > ev_async_stop() on the pending async watcher... While its probably a (semantic) bug to do so, it is harmless to call ev_async_send on a stopped watcher (other than losing the event, or getting a spurious event later). Stopping and freeing the watcher would be a different story, of course. > I am currently studying libev code to convince myself that libev code > handle correctly this case... The more eyes, the better. -- The choice of a Deliantra, the free code+content MORPG -==- _GNU_ http://www.deliantra.net ==-- _ generation ---==---(_)__ __ __ Marc Lehmann --==---/ / _ \/ // /\ \/ / schm...@schmorp.de -=/_/_//_/\_,_/ /_/\_\ ___ libev mailing list libev@lists.schmorp.de http://lists.schmorp.de/mailman/listinfo/libev
Re: EV_API_DECL inside ev.c
Hi, first of all, please see the discussion about compiler warnings in the documentation: http://pod.tst.eu/http://cvs.schmorp.de/libev/ev.pod#COMPILER_WARNINGS On Tue, Aug 29, 2023 at 05:07:23PM +0200, Christian Wendt he/him wrote: > libev-4.33/ev.c:2143:31: warning: ‘ev_default_loop_ptr’ initialized and > declared ‘extern’ The warning succinctly describes this line of correct code. The question that it doesn't answer is: why does it warn about it a line of code that pretty much can be found verbatim in the C standard document as an example of correct code? It looks to me to be more of a stylistic warning. > I don't see that EV_API_DECL is necessary in ev.c. Thats because you have perfect knowledge of the contents of that macro. If we hardcoded the values of all macros, indeed many things would be possible, but they are macros so they can be changed, also in ways we did not fully predict in advance. > Without EV_API_DECL, the warning would not be issued. Right, but other warnings can and will be emitted by other compilers for the changed line. You can't win, other than by writing correct code and exercising reason. (And in fact, if I remember correctly this was introduced specifically to make it compile with clang). > If this is about hiding all API in case of EV_API_STATIC, then maybe there > should be an EV_API_HIDE declared like this: There are a multitude of other options that complicate code and maintenance of the library. Given that its impossible to anticipate every compiler warning of every compiler, we opted for clear and simple code in this case, rather than hacks to avoid a spurious compiler warning that warns about perfectly correct code. -- The choice of a Deliantra, the free code+content MORPG -==- _GNU_ http://www.deliantra.net ==-- _ generation ---==---(_)__ __ ____ __ Marc Lehmann --==---/ / _ \/ // /\ \/ / schm...@schmorp.de -=/_/_//_/\_,_/ /_/\_\ ___ libev mailing list libev@lists.schmorp.de http://lists.schmorp.de/mailman/listinfo/libev
Re:
On Fri, Feb 17, 2023 at 01:39:27PM +0530, sandeep cs wrote: > Need help for libev in android system. Lots of people seem to use it on android, so there doesn't seem to be anything missing. Afte rlal, android should be able to compile C. > I see in github portal only for linux is specified. I didn't see any We can't comment on any github portal, because libev has no github portal, so you would have to contact whoever implemented that. > So please share libev library for android aarm64 if its available. It should compilöe as-is. In any case, qwe don'T have an android.mk file so cannot supply it. > If not please specify the steps for compile. I am not expert in ndk. It Unfortunately, we are also not experts in ndk - if your problem is with ndk, then this is likely the wrong group to ask. On Fri, Feb 17, 2023 at 01:41:27PM +0530, sandeep cs wrote: > So please share libev library for android aarm64 if its available. libev does work fine on android aarm64 alreadey - if you are askign for binaries, we don't distribute binaries and you would have to compile it from source. -- The choice of a Deliantra, the free code+content MORPG -==- _GNU_ http://www.deliantra.net ==-- _ generation ---==---(_)__ __ ____ __ Marc Lehmann --==---/ / _ \/ // /\ \/ / schm...@schmorp.de -=/_/_//_/\_,_/ /_/\_\ ___ libev mailing list libev@lists.schmorp.de http://lists.schmorp.de/mailman/listinfo/libev
Re: Feature request: support "-fno-exception"
Hi, and sorry for the late reply, somehow your mail slipped through moderation. On Wed, Dec 14, 2022 at 05:33:45PM +0800, Zhai Zhaoxuan wrote: > GCC & clang provide the "-fno-exception" flag to completely disable > exceptions for C++ code. I assume you mean fno-exceptions. > Some libraries allow exceptions to be disabled. libev also allows exceptions to be disabled - the question is how your compiler handles that. the sad truth is that ev++.h is written in c++ and not in a compiler-specific dialect. However, ev++.h is fully optional, libev proper does not use any c++ and should compile fine with -fno-exceptio0ns, so you sahould be able to use it directly. -- The choice of a Deliantra, the free code+content MORPG -==- _GNU_ http://www.deliantra.net ==-- _ generation ---==---(_)__ __ ____ __ Marc Lehmann --==---/ / _ \/ // /\ \/ / schm...@schmorp.de -=/_/_//_/\_,_/ /_/\_\ ___ libev mailing list libev@lists.schmorp.de http://lists.schmorp.de/mailman/listinfo/libev
Re: Need support
On Sat, Feb 04, 2023 at 04:55:48PM +0530, Sudheer Hebbale wrote: > The loop itself is started in a thread, and the main thread waits from > interrupt (ctrl-c) to terminate the program. It's not clear to me what you eman with "waits from interrupt", but in general, when you use multiple threads, you cannot simply access the same data structures without locking, which in this case, you do - you either have to access the libev loop in only one thread, and send it some signal (with an ev_async_send), or use some kind of mutex to coordinate access, or use multiple independent loops. That might or might not be the cause of your issue, but needs to be fixe din any case, and could make the problem go away. Also, threads (especially in a language such as C++) are rather difficult to use correctly, so when in doubt, I would try to avoid them and use event-based methods (which popen lends itself to naturally, start an io watcher on the file descriptor and use unbuffered I/O). > This does not happen if I do dlopen of another library (a dummy one). It is possibly that the librarey you load change the process environment. It might also not be prepared to be laoded into anything but the main thread (for example, calling sigprocmask in a threaded program is unspecified behaviour according to POSIX and might or might not do the right thing on os x, even though this is unlikely to be the issue here). -- The choice of a Deliantra, the free code+content MORPG -==- _GNU_ http://www.deliantra.net ==-- _ generation ---==---(_)__ __ __ Marc Lehmann --==---/ / _ \/ // /\ \/ / schm...@schmorp.de -=/_/_//_/\_,_/ /_/\_\ ___ libev mailing list libev@lists.schmorp.de http://lists.schmorp.de/mailman/listinfo/libev
Re: ev_io_modify usage?
On Mon, Dec 26, 2022 at 09:42:52AM -0800, Michael Stein wrote: > I'm new to libev and I see now that to support the various backends it > has to require the watcher be stopped for any modification (other than > the callback address). Well, issues can be very hairy, and people's expectations can be all over the place (for example, even "thread safe" can mean many very different things), and documenting this so it works always for everybody is a challenging thing: One could mention all limitations everywhere (good for prople only referencing the docs), or only in a central place (good for maintainers and people reading the documentatioon as a tutorial). OR something in between. However, re-reeading some parts of the docs, it definitely could be clearer what was meant by access, and maybe some guidelines are missing. We did go through the docs and made sure functions clearly are documented as safe to call on active watchers, and hopefully made it clear that, generally, no function is safe to call on active/pending watchers, and what to do about that. It's probably not optimal, but future releases will make this clearer. Greetings, -- The choice of a Deliantra, the free code+content MORPG -==- _GNU_ http://www.deliantra.net ==-- _ generation ---==---(_)__ __ __ Marc Lehmann --==---/ / _ \/ // /\ \/ / schm...@schmorp.de -=/_/_//_/\_,_/ /_/\_\ ___ libev mailing list libev@lists.schmorp.de http://lists.schmorp.de/mailman/listinfo/libev
Re: ev_io_modify usage?
On Sun, Dec 25, 2022 at 09:09:26PM -0800, Michael Stein wrote: > So I'm not sure when anyone would use ev_io_modify The documentatioon explains it: it is faster with some backends (e.g. epoll) because the file description is assumed to change. When using ev_io_Set, libev assumes that the file description might have changed. > When io_cb gets control it has nothing to send so it uses ev_io_modify > to turn off EV_WRITE: Did you stop the watcher before calling ev_io_modify? -- The choice of a Deliantra, the free code+content MORPG -==- _GNU_ http://www.deliantra.net ==-- _ generation ---==---(_)__ __ __ Marc Lehmann --==---/ / _ \/ // /\ \/ / schm...@schmorp.de -=/_/_//_/\_,_/ /_/\_\ ___ libev mailing list libev@lists.schmorp.de http://lists.schmorp.de/mailman/listinfo/libev
Re: libev: segmentation fault in ev_feed_event().
On Sat, Aug 27, 2022 at 09:09:10PM +0530, Mushtaq Khan wrote: > I see a segmentation fault ev_feed_event() called from timer_reify(). I > used to see this issue intermittently now i see this quite consistently in > my work environment. Most likely you are seeing the result of memory corruption due to a bug elsewhere in your program, e.g. from a data race between threads, reusing memory of active watchers or something like that. This can be difficult to track down - you can compile libev using -DEV_VERIFY=2 to enable very frequent checks of its data structures - this has a good chance of triggering an abor earlier, but especially data races can be hard to track down - make sure you lock your loops properly when using them in different threads, for example. -- The choice of a Deliantra, the free code+content MORPG -==- _GNU_ http://www.deliantra.net ==-- _ generation ---==---(_)__ __ __ Marc Lehmann --==---/ / _ \/ // /\ \/ / schm...@schmorp.de -=/_/_//_/\_,_/ /_/\_\ ___ libev mailing list libev@lists.schmorp.de http://lists.schmorp.de/mailman/listinfo/libev
Re: How to disable EV_FORK_ENABLE when building lib?
On Wed, Feb 09, 2022 at 09:55:11AM -0500, Olivier Langlois
wrote:
> Sorry for the silly question...
It's actually not an obvious question :)
> I usually find the answer to that type of questions by doing .configure
> --help or reading a build.txt file or something like that.
That's likely because confiure is meant to built the standard version of
libev, and disabling fork watchers would be unexpected - e.g. if we had
tests for that watcher for testsuite, it would fail to test.
However, you shoulöd be able to do what you want simply by specifyingt the
right CFLAGS, e.g.:
./configure CFLAGS=-DEV_FORK_ENABLE=0
I haven't tested this, and you would normally need to also #define this
every time you #include ev.h (although it might work without in the case
of disabling fork watchers, it will not work with anything that changes
the data structure layout).
Since the resulting libev library is incompatible with a normal libev, you
would have to be careful not to install it anywhere where other programs
might find it, though.
If your gial is simply to have a non-standard libev for your project, it
would normally be easier to embed libev, and have your own wrapper files. For
example, rxvt-unicode has two files, a header called ev_cpp.h, which looks
similar to this:
#define ECB_NO_THREADS 1
#define EV_COMPAT3 0
#define EV_USE_SELECT 1
#define EV_USE_EPOLL 1
#define EV_USE_KQUEUE 1
#define EV_USE_PORT 1
#include "ev.h"
And you would include this file whenever you normally would use ev.h.
Additionally, there is a ev_cpp.C file, which looks like this:
#include "ev_cpp.h"
#include "ev.c"
And this is the file that gets compiled into your program.
--
The choice of a Deliantra, the free code+content MORPG
-==- _GNU_ http://www.deliantra.net
==-- _ generation
---==---(_)__ __ __ Marc Lehmann
--==---/ / _ \/ // /\ \/ / schm...@schmorp.de
-=/_/_//_/\_,_/ /_/\_\
___
libev mailing list
libev@lists.schmorp.de
http://lists.schmorp.de/mailman/listinfo/libev
Re: perl EV module on Android 10 (terminal emulator)
On Sat, Jun 05, 2021 at 11:06:43AM -0400, AC wrote:
> I am running the Termux[0] terminal emulator on an Android 10 phone
> (aarch64), with perl v5.32.1.
Hi! Many apologies, but your mail seemed to be stuck in mailman for an
extended period of time :/, or I just didn't see it for soem reason.
> $ perl -MEV
>
> EV: cannot initialise libev backend. bad $ENV{LIBEV_FLAGS}? at
> /data/data/com.termux/files/usr/lib/perl5/site_perl/5.32.1/aarch64-android/EV.pm
> line 1245.
> Compilation failed in require.
> BEGIN failed--compilation aborted.
> I have also tried exporting LIBEV_FLAGS to 0, 1, 2 or 3 to no effect.
>
> How could I debug this further, to get to the root of the matter?
My first guess would be that no backends are enabled, which is quite likely
in your environment:
First, the select backend is disabled on __ANDROID (because the header
files on android are not good enough) - maybe, if termux provides an
environment different to android, it should not define that symbol (it
currently defines it).
Similarly, Ev might have trouble autodetecting other backends, in which case
you should enable them manually.
In fact, I tried it out right now, and when I enable poll and epoll
manually, the testsuite passes. You can provide the configuration using an
env variable, for automatic builds, e.g.:
export EV_EXTRA_DEFS="-DEV_USE_POLL=1 -DEV_USE_EPOLL=1 -DEV_USE_INOTIFY=1
-DEV_USE_EVENTFD=1"
--
The choice of a Deliantra, the free code+content MORPG
-==- _GNU_ http://www.deliantra.net
==-- _ generation
---==---(_)__ __ __ Marc Lehmann
--==---/ / _ \/ // /\ \/ / schm...@schmorp.de
-=/_/_//_/\_,_/ /_/\_\
___
libev mailing list
libev@lists.schmorp.de
http://lists.schmorp.de/mailman/listinfo/libev
Re: Recommended backend for Linux 5.11, and POSIX threads interop
Hi! On Sat, Aug 21, 2021 at 08:37:55AM +0200, Amirouche wrote: > I am diving in to libev. > > 1) I want to understand what is the recommended backend; The pod document > goes into a long description of how epoll is broken, but then it seems to me > EPOLL is the recommended backend on Linux: epoll is the recommended backend for performance, in general. what's broken is epoll, not libev's epoll backend, and libev tries to work around it, but one should be aware of the issues, because they can cause major slowdowns, and epoll is not faster for every use case - for example, a busy server with only active connections might well have faster throughput wiht select than with epoll. > 2) In the THREAD LOCKING EXAMPLE, the code rely on a mutex and a condvar; > Would it work better with CAS operation? Neither me nor google know what a CAS operation is w.r.t. pthreads. If you are talking about different thread systems, yes, different thread systems might work better differently, but thats probably because they are different. If CAS stands for compare-and-swap though, then, no, I don't think it would be better, as you would have to busy-wait, as CAS cannot wait. Note that pthreads mutex internally might well use a CAS operation, so the example might already use what you propose - again, that depend son your implementation. -- The choice of a Deliantra, the free code+content MORPG -==- _GNU_ http://www.deliantra.net ==-- _ generation ---==---(_)__ __ __ Marc Lehmann --==---/ / _ \/ // /\ \/ / schm...@schmorp.de -=/_/_//_/\_,_/ /_/\_\ ___ libev mailing list libev@lists.schmorp.de http://lists.schmorp.de/mailman/listinfo/libev
Re: Integrate the Glib main loop into the libev event loop (C++)
On Wed, Jul 07, 2021 at 08:59:01AM +0200, Oleksandr Kozlov wrote: > I've tried to replace std::map with std::vector, but this didn't help. > Actually, I used std::list instead of std::vector, because ev::io has > a private copy constructor, but we might need to resize a vector Right, I ould have thought you'd use pointers as you'd have a similar problem with std::map, but indeed, post-C+03 supports elements without copy constructors in some circumstances. Anyway, speed is not your problem. In any case, this is a guaranteed problöem that needs to be fixed in soemway or another. > By the way, it turned out that C-language implementation from EV::Glib > behaves in the same way: also hangs. Then I would check whether the glib watchers are actually legal. > It worth to mention, that I've tried to create ev::idle to prevent the > event loop > from sleeping. This solves the problem with hanging, but the busy loop loads > CPU for 100%. I suspect the only realistic way forward is to understand why the program is hanging. Hanging in the kernel call is unliekyl to be a problem with libev, but more likely a problem with the glib program waiting for events when that libev doesn't wait for. -- The choice of a Deliantra, the free code+content MORPG -==- _GNU_ http://www.deliantra.net ==-- _ generation ---==---(_)__ __ __ Marc Lehmann --==---/ / _ \/ // /\ \/ / schm...@schmorp.de -=/_/_//_/\_,_/ /_/\_\ ___ libev mailing list libev@lists.schmorp.de http://lists.schmorp.de/mailman/listinfo/libev
Re: Integrate the Glib main loop into the libev event loop (C++)
On Wed, Jul 07, 2021 at 12:06:37AM +0200, Oleksandr Kozlov
wrote:
> from the EV::Glib Perl module [3].
Note that this is likely going to be slow, as glib does not have the
necessary hooks to efficiently be embedded into another event loop.
> ctx.poll_fds.clear();
> int timeout = 0;
> ctx.context->query(ctx.priority, timeout, ctx.poll_fds);
> for (Glib::PollFD _fd : ctx.poll_fds) {
> int fd = poll_fd.get_fd();
> ctx.ios.try_emplace(fd, fd, to_ev_events(poll_fd.get_events()));
This looks as if you cache io watchers by fd, and could be the reason for
the problem - Glib doesn't tell you when the fd changes, so the fd is not
a unique enough primary key for your map (which really should be an array,
too).
What EV::Glib is doing is create I/O watchers in a prepare call and stop
them in the check call - you could simply push them onto a vector and clear
it out in check.
You could also try forcing libev to use select or poll as backend, which,
if Glib uses a lot of watchers, is likely more efficient, and this is a
quick way of checking whether this is your real problem - note that using
these backends for these side effects is not really supported by libev,
so you should definitely fix the c++ side, possibly followed by using
select/poll for efficiency.
--
The choice of a Deliantra, the free code+content MORPG
-==- _GNU_ http://www.deliantra.net
==-- _ generation
---==---(_)__ __ ____ __ Marc Lehmann
--==---/ / _ \/ // /\ \/ / schm...@schmorp.de
-=/_/_//_/\_,_/ /_/\_\
___
libev mailing list
libev@lists.schmorp.de
http://lists.schmorp.de/mailman/listinfo/libev
Re: libev-4.31 has just been released
descriptors that you can never remove from your epoll set, forcing you to destroy and recreate it from scratch, which is a performance disaster. That and the uneven performance you get from it. Otherwise, it would be quite fine :) A large part of my excitement w.r.t. io_uring stems from me hoping I could grandfather the epoll backend in favour of io_uring in newer kernels. Even if io_uring would be some constant factor slower on operations, in real-world performance-sensitive scenarios its improved queueing might sitll make it competitive to epoll, enough to forget epoll. Unfortunately, it's like the linux aio backend - great hgopes, destroyed by a buggy and slow implementation and the obvious nob-goal of fixing obvious bugs in favouir of cool features. > > All of this points ot being the right solution to use iouring for the > > things > > only it can do (I/O) and using epoll with a submission queue for the > > rest. > > > This is a good point. I guess that the only way to find out is to try > out those ideas. my bet isn't only on the queuing mechanic but also the > elimination of system calls for performing the I/O that io_uring can > offer. Another reason for doing it separately would be that libev might have to destroy/recreate the backend fd, which further complicates communication with the app - what happens on partial reads and so on? I think the small constant overhead of embedding io_uring into another event loop (e.g. libeio into libev) is not going to be a problem in most performance-sensitive scenarios. -- The choice of a Deliantra, the free code+content MORPG -==- _GNU_ http://www.deliantra.net ==-- _ generation ---==---(_)__ __ __ Marc Lehmann --==---/ / _ \/ // /\ \/ / schm...@schmorp.de -=/_/_//_/\_,_/ /_/\_\ ___ libev mailing list libev@lists.schmorp.de http://lists.schmorp.de/mailman/listinfo/libev
Re: libev-4.31 has just been released
Thanks for trying out the iouring backend. On Wed, Apr 28, 2021 at 11:24:49AM -0400, Olivier Langlois wrote: > I believe that in order to achieve the performance gain that io_uring > can deliver, you would need to service I/O through io_uring as well to > save on the associated system call cost instead of just using io_uring > for polling. iouring being quite a bit slower than epoll was my own experience. That and it being too buggy for general use (and obviously no movement of the maintainer to fix things) made me kind of give up on this as a libev backend. It can only work in some case,s or when you embed it into another event loop. And then you can just embed iouring into an epoll loop to get the best of both worlds. > but if adding a function specific to libev io_uring backend that would > let the watcher code perform their i/o requests through io_uring was > thinkable, that could be the performance holy grail for libev users by > only making 1 system call to service the 64 i/o operations. The obvious way to do this would be to expose request submission. There are two problems to solve though: a) how to identify/distinguish those requests from libev's own ones and b) libev can tear down and open a new iouring at any time, and this is hard to synchronise with external users. It might be possible to do some kind of iouring watcher. However, in the need, you can also use your own iouring and embed it into libev. This will also take care of extra system calls, and is rather clean and works with e.g. epoll as well, and might even be faster (certainly more correct, as iouring is too buggy as a generic event backend). > In the meantime, I took care of one of the TODO item. That is using a > single mmap() when possible. It is essentially code from liburing > adapted to libev coding style... Thanks, when I come around to implement this I will certainly take advantage of your work, although this is currently on the back burner due to the issues with iouring. Would it be possible to re-send the patch properly though? The version you sent is completely garbled because there are extra spurious newlines all over the place. > By switching from epoll backend to io_uring one, my process CPU usage > did drop from 20-30% to below 5%. It seems too good to be true! > What I suspect happening is that my socket option SO_BUSY_POLL setting > might not be honored by io_uring. That indeed sounds too good to be true. In my tests iouring is consistently slower, although I can imagine that in workloads which are very heavy on syscalls (e.g. epoll_ctl) this might change. On the other hand, epoll now has a mode where it can also queue things with few syscalls, and as much as I hate epoll, since iouring is going down the same road as linux aio (buggy, never getting fixed), it is probably the way to go for the future. On Wed, Apr 28, 2021 at 11:31:47AM -0400, Olivier Langlois wrote: > Here is a last quick sidenote concerning my CPU usage observation. > > CPU usage reported by top is now below 5% when using io_uring backend > but it seems like the CPU is spent by something else inside the kernel > as my average load did pass from 2.5 to ~3.1... I would expect that all other things being similar, that things take more cpu, as iouring seems to be vastly less efficient as epoll (e.g. its use of hash tables instead of a simple array lookup for everything is bound to slow things down). It might be possible that this is improved in future versions of the kernel, but I am doubtful that it can ever reach epoll speeds, especialy if a queueing system is used for epoll as well (which libev does not yet implement). All of this points ot being the right solution to use iouring for the things only it can do (I/O) and using epoll with a submission queue for the rest. -- The choice of a Deliantra, the free code+content MORPG -==- _GNU_ http://www.deliantra.net ==-- _ generation ---==---(_)__ __ __ Marc Lehmann --==---/ / _ \/ // /\ \/ / schm...@schmorp.de -=/_/_//_/\_,_/ /_/\_\ ___ libev mailing list libev@lists.schmorp.de http://lists.schmorp.de/mailman/listinfo/libev
Re: Using ev_async for communication in the same event loop
On Sun, Apr 18, 2021 at 01:09:03PM -0700, Arun Athrey Chandrasekaran wrote: > in an asynchronous way and not miss other events in the main event loop. Is > this expected to work? I looked at the documentation and it is not clear to > me if ev_async can be used for communication in the same event loop. Hi, there are no restrictions as to form which thread you ev_async_send, which means you can do that from within the same event loop. If you don't know whether it is the same thread or not, just use ev_async_send. If you do know that it is within the same thread that owns the event loop, it is more efficient to start an idle watcher. -- The choice of a Deliantra, the free code+content MORPG -==- _GNU_ http://www.deliantra.net ==-- _ generation ---==---(_)__ __ __ Marc Lehmann --==---/ / _ \/ // /\ \/ / schm...@schmorp.de -=/_/_//_/\_,_/ /_/\_\ ___ libev mailing list libev@lists.schmorp.de http://lists.schmorp.de/mailman/listinfo/libev
Re: libev/libuv interoperation with ev_async
On Mon, Nov 30, 2020 at 10:47:24PM +0300, Mons Anderson wrote: > > Every thread works in cooperative multitasking mode > (https://github.com/tarantool/tarantool/blob/master/src/lib/core/fiber.h#L636) > For networking with clients created a separate, net thread > (https://github.com/tarantool/tarantool/blob/master/src/box/iproto.cc#L2055) > Communication between threads works using cbus and cpipe > (https://github.com/tarantool/tarantool/blob/master/src/lib/core/cbus.h#L113) > which in fact are wrappers on top of ev_async Quite interesting! > With current state one could create a module using ev backend, since ev_async > is fully supported by libev. But if worker thread must run another event loop > there is no good option for how to run, for ex pure select or libuv on top of > running libev loop It's still not quite clear to me what you are trying to achieve - so you have a worker thread that uses libuv, and you somehow want to integrate this into your main libev loop. How does the communication work though? Do you want to wake up an uv loop that the worker is blocking in, or the main ev loop from a uv thread? Or, likely, something else? Do you want to write to the evpipe, or read from it? The equivalent of writing to it can already be done by using ev_async_send. Reading from it makes no sense, as whether the pipe has something to read or not depends on the state of libev, it does not signify pending asyncs. > I know, there are a lot of options for mutual integrations of event loops with > AnyEvent, EV and other event loops in the world of Perl. I hope to find > something like that for libev in the C world ;) Not really, AnyEvent just solves the problem of your loop-agnostic code running with any event loop, i.e. it allows you to write e.g. perl libraries that do not depend on a specific event loop. While AnyEvent+Coro can be used for a lot of tricks in running multiple event loops, they don't do this by themselves, and there is no generic recipe to do so. > >#include "ev.c" > > > >... code can now read evpipe, so you can define a function here. > > > > But for this to work reliably you probably have to patch libev anyway, to > > make evpipe signal async sends reliably. > > We'll try to test such an approach. (In my small test snippet it seems to > work) I assume you mean accessing evpipe - but accessing evpipe is only one part, you have to find out what semantics you want from evpipe, and likely patch libev to provide these semantics, for example, making sure libev actually allocates a pipe and write for every ev_async_send, neither of which libev does at the moment. -- The choice of a Deliantra, the free code+content MORPG -==- _GNU_ http://www.deliantra.net ==-- _ generation ---==---(_)__ __ __ Marc Lehmann --==---/ / _ \/ // /\ \/ / schm...@schmorp.de -=/_/_//_/\_,_/ /_/\_\ ___ libev mailing list libev@lists.schmorp.de http://lists.schmorp.de/mailman/listinfo/libev
Re: libev/libuv interoperation with ev_async
On Mon, Nov 30, 2020 at 09:29:48PM +0300, Mons Anderson wrote: > I've found that all I need to integrate them is to have access to > evpipe (to put a watcher on it from another event loop), but libev > interface doesn't provide any way to access it. What exactly do you mean with evpipe? The internal evpipe member? The problem is that it is not clear what you want to (or can) do with it - its value and use is essentialy undocumented, undefined, and can change at any time. I'd be hard presed to see what potential use its value could be. Specifically, evpipe can NOT reliably tell you when an ev_async_send is pending. Note that libev does not protect its internal members, so you could embed libev in a version where you understand and/or can guarantee that the evpipe member works as you want it to, e.g. in my_ev.c: #include "ev.c" ... code can now read evpipe, so you can define a function here. But for this to work reliably you probably have to patch libev anyway, to make evpipe signal async sends reliably. -- The choice of a Deliantra, the free code+content MORPG -==- _GNU_ http://www.deliantra.net ==-- _ generation ---==---(_)__ __ __ Marc Lehmann --==---/ / _ \/ // /\ \/ / schm...@schmorp.de -=/_/_//_/\_,_/ /_/\_\ ___ libev mailing list libev@lists.schmorp.de http://lists.schmorp.de/mailman/listinfo/libev
Re: Iteration over registered handlers
On Wed, Nov 25, 2020 at 02:43:21PM +0100, Rick van Rein wrote: > I love libev. The complex to-do list in my mind quickly clears up when > I add it to a project. It's lovely to have this global database under a > single pointer that pops out relevant data alongside interesting events. > > For cleanup purposes, I'd like to iterate over registered event handlers > after I've broken out of a loop. I cannot find a macro or function, is > that not part of the API? It's not part of the API because it would require libev to add overhead to track watchers. If you need this functionality, you can add it easily on top of libev. Doing it yourself also ensures that you are only doing it to your own watchers, and not watchers by others (e.g. libev internal ones). Even if you could iterate over all watchers in libev, how would you decide which watchers are your own without outside knowledge? > [I'm cautious about hacking it in, because of the obvious fine balance > of platform knowledge built into the code.] It's probably just as simple and easier to maintain wrappers around the start/stop functions you use. -- The choice of a Deliantra, the free code+content MORPG -==- _GNU_ http://www.deliantra.net ==-- _ generation ---==---(_)__ __ __ Marc Lehmann --==---/ / _ \/ // /\ \/ / schm...@schmorp.de -=/_/_//_/\_,_/ /_/\_\ ___ libev mailing list libev@lists.schmorp.de http://lists.schmorp.de/mailman/listinfo/libev
Re: libev tight loop
On Tue, Oct 27, 2020 at 01:54:38AM -0400, Felipe Gasper wrote: > strace perl -MAnyEvent -e'AnyEvent->condvar()->recv()' > > > … causes tight-loop 0-second “epoll_wait”s. > > Obviously this isn’t very useful logic, so maybe you don’t consider this a > bug, but just in case I thought I’d point it out. I know - it's suboptimal behaviour, but since cases like this are always applciation bugs, I decided that adding overhead to work around it is not worth the effort, especially as the behaviour is actually correct. It's also not an issue in libev or EV, but is due to how AnyEvent implements its condvar interface - EV effectively detects that there is a deadlock, but AnyEvent does not define deadlocks. -- The choice of a Deliantra, the free code+content MORPG -==- _GNU_ http://www.deliantra.net ==-- _ generation ---==---(_)__ __ __ Marc Lehmann --==---/ / _ \/ // /\ \/ / schm...@schmorp.de -=/_/_//_/\_,_/ /_/\_\ ___ libev mailing list libev@lists.schmorp.de http://lists.schmorp.de/mailman/listinfo/libev
Re: callback never invoked
On Sun, Oct 18, 2020 at 02:46:44PM +0500, rustahm wrote: > libev++ to watch a directory of files and call the callback function as > soon as inotify gives at least one relevant event. And I don't want libev++ > to block my main program, that's why I tried to use it with NOWAIT. All event libraries block at least one thread. You coulöd move that into something other than your main thread, but that complicates things so I would not reocmmend that. > Am I doing it totally wrong? Is libev++ a right tool for the task? I am not sure what you are trying to achieve, but libev (not libev++) does kind of impose event-based programming on the user. You could use it differently, but that would be very advanced, and possibly also painful. So if you can't use an event-based approach and do you work in callbacks, libev (or any other event library) would be the wrong tool. > If I add ev_run(EV_A_ 0); to engage_watcher() I get use of undeclared > identifier 'loop'. You cna either use "ev_run (ev_default_loop (), 0)" or, if you have more than one loop, pass the "struct ev_loop *" around. There is also a macro you can use "ev_run (EV_DEFAULT_UC_ 0)" that is faster than calling ev_default_loop if the loop is guartanteed to be initialiased already. > and run the program but it gets blocked until I change a file in the > directory being watched. Yes, that's how it works. You need to cal ev_run in your main program and do your work in callbacks, always returning to the main loop. -- The choice of a Deliantra, the free code+content MORPG -==- _GNU_ http://www.deliantra.net ==-- _ generation ---==---(_)__ __ __ Marc Lehmann --==---/ / _ \/ // /\ \/ / schm...@schmorp.de -=/_/_//_/\_,_/ /_/\_\ ___ libev mailing list libev@lists.schmorp.de http://lists.schmorp.de/mailman/listinfo/libev
Re: callback never invoked
On Sun, Oct 11, 2020 at 04:51:09PM +0500, rustahm wrote: > Please find attached the source code. Could anyone explain why the callback > is never invoked when I do change the files? You probably never call ev's mainloop: you only call ev::run once with NOWAIT, so unless any change happens before that call, libev code is never invoked and therefore does not have a chance of calling your callback. You would need to put something like ev_run (EV_A_ 0) into your main program (which you haven't shown). Also, ev::get_default_loop() (and .run) is internal undocumented functionality which you should not call if you want your program to continue to work with future versions. -- The choice of a Deliantra, the free code+content MORPG -==- _GNU_ http://www.deliantra.net ==-- _ generation ---==---(_)__ __ __ Marc Lehmann --==---/ / _ \/ // /\ \/ / schm...@schmorp.de -=/_/_//_/\_,_/ /_/\_\ ___ libev mailing list libev@lists.schmorp.de http://lists.schmorp.de/mailman/listinfo/libev
Re: Linux GPIO support
On Sat, Aug 15, 2020 at 04:45:19AM +, Anandu Suri wrote: > Hello Marc, Hi, and sorry for the delay. > Not sure if this is a design decision or buggy design. But given that Linux is It's a design bug. In any case, libev is not a linux-only library, and doe snot attempt to implement linux-only behaviour that is at odds with portable functionality. You can poll for POLLPRI events using libev by using epoll and watching the epoll file descriptor - while epoll is effectively a linux-only function, that is fine, as it seems only to be required on linux. There is little point in bloating libev with functionality that is a) already supported with a bit of extra work and b) is only useful for linux. -- The choice of a Deliantra, the free code+content MORPG -==- _GNU_ http://www.deliantra.net ==-- _ generation ---==---(_)__ __ __ Marc Lehmann --==---/ / _ \/ // /\ \/ / schm...@schmorp.de -=/_/_//_/\_,_/ /_/\_\ ___ libev mailing list libev@lists.schmorp.de http://lists.schmorp.de/mailman/listinfo/libev
io_uring news/status update
Hi! I have updated the io_uring backend in libev to the new features introduced in post-5.4 kernels (which are quite nice). If people would give it a try to test and see if it works for them (in CVS) that would be great - linux 5.6 is required, and the backend has to be manually requested (e.g. with LIBEV_FLAGS=128). I couldn't give it a thorough testing myself because, unfortunately, none of the blocking bugs in the kernel have been fixed, so I am unable to use it in most of my own programs. For obvious reasons, I would wish bugfixes would receive priority over features, but it's obviously not what's happening with io_uring. io_uring is currently like kqueue on OS X - it works with "important" file types, but it fails as generic mechanism, and there is no way to detect what will work or not, so workarounds are essentially impossible. In case anybody wonders about my policy in making a backend the default backend in libev: I use libev in a number of free software and proprietary packages, and until I can run most of them correctly for a while, it won't become default. At the moment, at least rxvt-unicode more or less instantly crashes due to kernel bugs in io_uring, and urxvt is one of the first things I run with new changes, so that's why it receives limited testing. -- The choice of a Deliantra, the free code+content MORPG -==- _GNU_ http://www.deliantra.net ==-- _ generation ---==---(_)__ __ __ Marc Lehmann --==---/ / _ \/ // /\ \/ / schm...@schmorp.de -=/_/_//_/\_,_/ /_/\_\ ___ libev mailing list libev@lists.schmorp.de http://lists.schmorp.de/mailman/listinfo/libev
Re: Critical sections with ev++.h
On Fri, Jul 17, 2020 at 05:10:32PM +0200, Michele Santullo wrote: > notifications in separate threads and just print something to stdout when > they > occur. It's mostly working but I get occasionas crashes, which makes me think > it might be a concurrency issue. Right, it also means you are not protecting calls to the same loop with a mutex, which would get rid of the crashes. > incredis/src/branch/master/src/async_connection.cpp#L122 that I was calling > ev_set_loop_release_cb(). If memory serves me right that was needed precisely ev_set_loop_release_cb won't get rid of the crashes for you, the purpose of that call is so that you can release your mutex for longer times, i.e. if another thread wants to acquire the mutex while libev is waiting in the kernel it normally can't, because ev_run/ev_loop calls need to be also protected via that mutex, and without ev_set_loop_release_cb this mutex will be held while libev is polling the kernel. > to solve some weird crash that I was getting, but in the C++ API I can't find > anything similar. Is it done is some other way in C++? ev_set_loop_release_cb works fine in C++. Note that, while ev++.h has some classes representing the main loop, only the watcher classes are a) documented b) maintained and c) guaranteed to some degree to stay. Everything else is undocumented, might be broken and might go away or change incompatibly at any time. > Also, small side question, is there an IRC channel for questions and > discussions about libev? You can go to irc.schmorp.de #schmorpforge - I'll be there as user schmorp, and some other users might also be able to help with libev. You might have to /msg me to get my attention, though. -- The choice of a Deliantra, the free code+content MORPG -==- _GNU_ http://www.deliantra.net ==-- _ generation ---==---(_)__ __ __ Marc Lehmann --==---/ / _ \/ // /\ \/ / schm...@schmorp.de -=/_/_//_/\_,_/ /_/\_\ ___ libev mailing list libev@lists.schmorp.de http://lists.schmorp.de/mailman/listinfo/libev
Re: [PATCH] Fix linking errors when building with MinGW.
On Sat, May 02, 2020 at 05:27:10PM +0800, DDoSolitary wrote: > One will get errors like "undefined reference to __imp_htonl" when > trying to build libev with MinGW on Windows. This patch fixes this by > linking to ws2_32, which provides winsock functions. Hmm, the patch doesn't seem to be for libev - Are you sure you got the right project and not something else that embeds libev? -- The choice of a Deliantra, the free code+content MORPG -==- _GNU_ http://www.deliantra.net ==-- _ generation ---==---(_)__ __ __ Marc Lehmann --==---/ / _ \/ // /\ \/ / schm...@schmorp.de -=/_/_//_/\_,_/ /_/\_\ ___ libev mailing list libev@lists.schmorp.de http://lists.schmorp.de/mailman/listinfo/libev
Re: About describe doubt for libev release ..
plication to design aorund it) and a timer implementation that just works for every application (as in libev), i.e. your goal of being generic and easy to use directly contradicts using a hardwired algorithm such as a timer wheel. > Have you considered optimizing by changing the data structure, or do > you still have other "killer" ways to optimize? Will this affect the > stability? I don't think there are much better data structures - while there or asymptotically better data structures, they are often much more complicated and dynamic, and are not actually faster in practise. Or, when they are faster, they are less generic (such as timer wheels), so require workarounds on the application side. And here a design parameter of libev comes into play again: If your application is fine with a timer wheel and can gain an advantage from it, you can put it on top of libev timers with practically no overhead. The reverse wouldn't be true. It is usually trivial to find a data structure that performs better for a specific problem, but libev needs to perform well for anything reasonable you throw at it (and preferably performs ok even in unreasonable cases). The 4-heap in libev is pretty close to be generically optimal for most use cases. -- The choice of a Deliantra, the free code+content MORPG -==- _GNU_ http://www.deliantra.net ==-- _ generation ---==---(_)__ __ __ Marc Lehmann --==---/ / _ \/ // /\ \/ / schm...@schmorp.de -=/_/_//_/\_,_/ /_/\_\ ___ libev mailing list libev@lists.schmorp.de http://lists.schmorp.de/mailman/listinfo/libev
Re: libev-4.31 has just been released
On Sun, Mar 22, 2020 at 05:57:02PM -0400, Benjamin Mahler wrote: > Thanks Marc, do you have any broader comments on the implications of > iouring for libev? It looks like iouring is finally bringing async system > calls (not just async io) to Linux. As far as I have been told, you will even be able to upload (ebpf) programs into the kernel, and you can already do chained requests of sorts (open/read/close for example). I haven't seen direct signs of bringing async system calls into iouring though, but that might be because I am simply uninformed, and concentrate on what libev needs first. It's also not really a concern for either libeio or libev. > Will libeio have an iouring backend that doesn't use a thread pool and > instead hands the io off to the kernel with io_uring? I don't think the threadpool can be avoided, as some things will have to be done in userspace (e.g. eio_readdir), but yes, as a very heavy libeio user I'd be very keen on getting many requests into iouring, stat(x), open/close, read/write are pretty high in my personal priority list, and it's certainly possible to do this in smaller steps. This will be useful even if io_uring turns out to be slower than epoll for I/O notifications - which is an ongoing concern, due to the less streamlined API (e.g. io_uring requires a hash lookup to find watchers whereas epoll can just use an array lookup, and this is worsened by io_uring requiring kernel interactions per event notification and not just per watcher, as epoll does). But even if slightly slower, it might still become the default backend simply because it avoids so many design issues that epoll suffers from. I currently can't see a reasonably easy way of sharing the io_uring between libev and libeio, so they will likely use a separate io_uring instance anyway. But while I am confident on the plans for libeio, I am less confident on the timeframe - it might be very quick, or it might take a long time. Important will be my experiences with io_uring in libev w.r.t. speed and queuing behaviour - libev already had some major design changes internally to accomodate io_uring (as documented in fd_reify), which can bring some instability to the codebase. -- The choice of a Deliantra, the free code+content MORPG -==- _GNU_ http://www.deliantra.net ==-- _ generation ---==---(_)__ __ ____ __ Marc Lehmann --==---/ / _ \/ // /\ \/ / schm...@schmorp.de -=/_/_//_/\_,_/ /_/\_\ ___ libev mailing list libev@lists.schmorp.de http://lists.schmorp.de/mailman/listinfo/libev
Re: About describe doubt for libev release ..
On Sun, Mar 22, 2020 at 12:56:16PM +0800, CandyMi <869646...@qq.com> wrote:
> I have the same opinion about Linux aio and io_uring. The performance of aio
> is not as good and problematic as described, but the use of io_uring may be
> limited by the version of the Linux kernel and it makes me hesitant.
>
> There may be very few people who follow up on Linux 4.x / 5.x aggressively,
> that is to say, there will be very few people who actually use it! And I also
> have to maintain the old 2.6.32 kernel project).
Actually, I would assume most people are now on a 4.x kernel, but of
course, select and poll keep being supported and lubev will fall back to
these if required.
> The following code is my implementation of the IO watcher wrapper. The only
> difference is: "the IO watcher passed to the core IO init method may have
> called the core IO stop multiple times (object reuse). Such behavior is to
> reduce the frequent creation or destruction of IO watcher."
This got rather garbled, but I have one comment:
> core_io_stop(core_loop *loop, core_io *io){
> if (io-events || io-fd){
If you want to check whether a watche ris active (strated), you could use
ev_is_active(w).
Your current if has the disadvantage of not stopping all acive watchers,
as 0 is a valid value for both fd and events.
> io-fd = io-events = 0x0;
Also, after stopping, you can reuse the watche rmemory in any way you like,
but you then cannot start it again without calling ev_io_init.
> It is worth mentioning the use of the ev_timer_again method: "The consequence
> of modifying timer- repeat is that it will cause the min-heap to be
> adjusted every time. Will frequent use have a certain impact or even worse
> performance?", Because The implementation of ev_timer_again is this (v4.25):
While other data structures are possible, and I vaguely plan some minor
optimisations, when you use ev_timer_again, libev of course has to adjust the
heap to reflect the new reality. ev_timer_again is typically faster than a
stop/start though, which is why it exists.
Depending on your needs, you may be able to further optimize timers by not
updating them at all on every change (e.g. for network timeouts, letting
them expire and reschedule to the real timeout in the callback), or by
creating your own data structure, e.g. if you have a lot of timers with
the same timeout, you can put them into a linked list and only create a
real timer for the next timer in that list, adding new timers to th end
and so on.
> Recently, I was watching the code of libeio. When will libeio be ready to
> release the official version?
I can call it an official version right now and create a CVS tag if that
helps you. The main blocking issue is probably the lack of documentation,
and some doubts on whether this is really the right API.
And of course, libeio might profit most from io_uring.
In terms of stability, libeio is pretty much stable and in maintenance for
a decade now.
--
The choice of a Deliantra, the free code+content MORPG
-==- _GNU_ http://www.deliantra.net
==-- _ generation
---==---(_)__ __ __ Marc Lehmann
--==---/ / _ \/ // /\ \/ / schm...@schmorp.de
-=/_/_//_/\_,_/ /_/\_\
___
libev mailing list
libev@lists.schmorp.de
http://lists.schmorp.de/mailman/listinfo/libev
Re: About describe doubt for libev release ..
> 1. "the documentation wrongly claimed that user may modify fd > and eventsmembers in io watchers when the watcher was stopped." means: > Can't modify internal members even if I/O watcher has been stopped? Yes - you have to use ev_io_set or the new ev_io_modify, direct modification does not work properly as the io watcher caches some kernel state info. If you modified these members directly, it might or might not work correctly, even in old versions. To be specific, in most versions of libev, I/O watchers assume that if the fd wasn't changed via ev_io_set, it will refer to the same underlying file description, unless you use ev_io_set. If you modify the fd directly, you might not receive events for it, and you might even receive events for fds that are no longer open (mostly due to the braindamage that is epoll). > 2. What is the scenario where timerfd checks "Time Jump" cannot > wake up? why a minute? I don't know of any scenarion where timerfds fail - I assume you refer to something in the Changes file, but what exactly? The only thing that changed is that libev does not wake up roughly every minute when it is sure it can keep the time without it. > 3. What are the special reasons why Linux AIO is no longer the > default backend? It never was the default backend, it merely was available for use by default. This is no longer the case - compiled libev versions no longer support the aio backend by default. The reasons are not that special: linux aio is buggy, extremely slow and suffers from a multitude of arbitrary limitations, which is why it couldn't be a default backend. It is expected that the io_uring interface will eventually replace the epoll backend, although at the moment it's not there yet. So basically, it is useless dead code that is of little to no use to anybody, and would only bloat the binary. -- The choice of a Deliantra, the free code+content MORPG -==- _GNU_ http://www.deliantra.net ==-- _ generation ---==---(_)__ __ __ Marc Lehmann --==---/ / _ \/ // /\ \/ / schm...@schmorp.de -=/_/_//_/\_,_/ /_/\_\ ___ libev mailing list libev@lists.schmorp.de http://lists.schmorp.de/mailman/listinfo/libev
Re: libev-4.31 has just been released
> Currently, the io_uring interface evelopment in libev is on hold, awaiting I might add, the iouring backend can be enabled in libev-4.33, and is expected to work. It has not really received testing, and it doesn't seem to have speed benefits yet. Anybody is invited to experiment with it - just don't use it in production. -- The choice of a Deliantra, the free code+content MORPG -==- _GNU_ http://www.deliantra.net ==-- _ generation ---==---(_)__ __ __ Marc Lehmann --==---/ / _ \/ // /\ \/ / schm...@schmorp.de -=/_/_//_/\_,_/ /_/\_\ ___ libev mailing list libev@lists.schmorp.de http://lists.schmorp.de/mailman/listinfo/libev
Re: libev-4.31 has just been released
On Wed, Mar 18, 2020 at 01:19:37PM -0400, Benjamin Mahler wrote: > Just to follow up on this, if there have been any findings to share I'm > sure many of us in the mailing list would be interested! Currently, the io_uring interface evelopment in libev is on hold, awaiting bugfixes and new features that are expected to have landed in linux 5.5 and 5.6. After 5.6 has somewhat stabilised, I will likely continue to work on the backend, taking advantage of the improvements made to it. I don't expect it to beat epoll in terms of speed immediately (both in terms of kernel speed and the way libev uses it), but hopefully this can be achieved eventually. Top priority is to get it working so it could become the default backend. I don't expect it to be usable on kernels before 5.6 - and since it is brand-new, maybe not for a bit after. -- The choice of a Deliantra, the free code+content MORPG -==- _GNU_ http://www.deliantra.net ==-- _ generation ---==---(_)__ __ __ Marc Lehmann --==---/ / _ \/ // /\ \/ / schm...@schmorp.de -=/_/_//_/\_,_/ /_/\_\ ___ libev mailing list libev@lists.schmorp.de http://lists.schmorp.de/mailman/listinfo/libev
libev-4.33 has just been released
I am pleased to announce libev 4.33! Distribution: http://dist.schmorp.de/libev/libev-4.33.tar.gz Documentation: http://pod.tst.eu/http://cvs.schmorp.de/libev/ev.pod?pathrev=rel-4_33 Changes: http://cvs.schmorp.de/libev/Changes?pathrev=rel-4_33 This release is mainly a bugfix/documentation/quality improvement release. The "major" new feature is ev_io_modify, which can modify the event watch mask in I/O watchers without incurring overhead caused by normally having to check whether the fd changed (as ev_io_modify doesn't change the fd), which can be substantial with some backends (i.e. epoll). The complete Changes since 4.31 are: 4.33 Wed Mar 18 13:22:29 CET 2020 - the 4.31 timerfd code wrongly changed the priority of the signal fd watcher, which is usually harmless unless signal fds are also used (found via cpan tester service). - the documentation wrongly claimed that user may modify fd and events members in io watchers when the watcher was stopped (found by b_jonas). - new ev_io_modify mutator which changes only the events member, which can be faster. also added ev::io::set (int events) method to ev++.h. - officially allow a zero events mask for io watchers. this should work with older libev versions as well but was not officially allowed before. - do not wake up every minute when timerfd is used to detect timejumps. - do not wake up every minute when periodics are disabled and we have a monotonic clock. - support a lot more "uncommon" compile time configurations, such as ev_embed enabled but ev_timer disabled. - use a start/stop wrapper class to reduce code duplication in ev++.h and make it needlessly more c++-y. - the linux aio backend is no longer compiled in by default. - update to libecb version 0x00010008. -- The choice of a Deliantra, the free code+content MORPG -==- _GNU_ http://www.deliantra.net ==-- _ generation ---==---(_)__ __ __ Marc Lehmann --==---/ / _ \/ // /\ \/ / schm...@schmorp.de -=/_/_//_/\_,_/ /_/\_\ ___ libev mailing list libev@lists.schmorp.de http://lists.schmorp.de/mailman/listinfo/libev
Re: accept socket occasional read error
On Tue, Feb 18, 2020 at 04:05:53PM +0100, Bo Lorentsen wrote: > >> that I can't read from, at all. recv returns -1 and EAGAIN, but it never > >> gets any data. > > You should not get readyness events from libev for sockets and then have > > recv > > return EAGAIN. > > No, and this is fortunately not what I experience :-) But I get a socket > from accept, and I try to read from it ( a single time ) right away, > just to see if it already contains data, and sometimes it does, at > leased a bit of it, and sometime it just gives me EAGAIN++, as expected. That is confusing - if you read from the socket _without_ getting a readyness notification from libev, then of course you might get EAGAIN, but that wouldn't have anything to do with libev, as it isn't involved, right? It would be more asurprising if libev told you you cna read and THEN you got EAGAIN - this is possible, but shouldn't happen when you only read after libev told you so. > The problem is that sometimes I get a socket that never give me any data > (or so it seems), so I ended up writing a small ev timer that "poked" > this socket if no data was received for more then 10 sec (this is purely > for benchmark purpose, so not for real life usage), and it did indeed > still return EAGAIN++ as expected, and no data at all. Yeah, that would either be a network problem, or what you think is the socket actually isn't the socket (i.e. somehow the program confuses fd's for example). > and is blistering fast. There may be some kind of strange shadow effect, > but to me at least it seems to boil down to me getting a socket from > accept that is valid, but no data ever arrive here. If that is the case,then it is because your kernel never received any data for it - or your kernel is buggy, which is possible, but much less likely than a bug somewhere else. > You are absolutely right, this is very likely my code that have a bug, > and this may be a cascading effect i don't understand. I just wrote > here, to see if anyone have some experience with something like this, or > maybe could spot the error in the test code. It is hard to find people > in the C/C++ world that do things like this :-) Well, I don't know if I am one of the poeple you want to reach, but of course, I had buggy programs, too, and my first steps would usually be to identify, with certainty, an fd that ahs the problem and see its kernel state - if no data is there, then no data was received (or it eas read earler). With TCP, iot is relatively easy to runa tcpdump and later identify the connection, and then you cna see _exqactly_ what packets were exchanged. That way you can find out with certainty whether the bednchmark tool sent something, whether your receive socket received somethign,a nd thus rule out your kernel, the benchmark tool, or your program. If you don't know how to read tcp exchanges, learning it will go a long way towards enlightenment (i.e. I think its worth learning in any case :) -- The choice of a Deliantra, the free code+content MORPG -==- _GNU_ http://www.deliantra.net ==-- _ generation ---==---(_)__ __ __ Marc Lehmann --==---/ / _ \/ // /\ \/ / schm...@schmorp.de -=/_/_//_/\_,_/ /_/\_\ ___ libev mailing list libev@lists.schmorp.de http://lists.schmorp.de/mailman/listinfo/libev
Re: accept socket occasional read error
On Sun, Feb 16, 2020 at 08:43:20AM +0100, Bo Lorentsen wrote: > I have a callback that gets called on every accept (where I loop on EAGAIN > to empty the backlog), and for each new socket I make an protocol structure You mean loop while accept is successful? > up to about 20k connections, and concurrency of 500 i gets a few sockets > that I can't read from, at all. recv returns -1 and EAGAIN, but it never > gets any data. You should not get readyness events from libev for sockets and then have recv return EAGAIN. > Are there something I am not aware of, or have others seen something like > this ? I really appreciate any comments, I have tried google and found > nothing useful on this specific subject. Well, this is not a very specific problem - for example, your code could simply be buggy and confuse fds (i.e. read from the wrong fd, move fds around and so on). Or it could corrupt ata structures (e.g. by modifying or freeing in-use watchers). There could also be some network problem, caused by an external setup - tcpdump'ing everything and later looking at the actual traffic for the socket that failed to work can help find out what is going on. netstat/ss can display socket buffers, which can tell you where the missing data is, or if there actually is any data waiting (for example, you could have wrongly read the data already). You could try recompiling libev with e.g. -DEV_VERIFY=2 or even 3 and see if that maybe catches an issue (although if what you describe is correct, it will probably not catch anything). Personally, I'd start with whichever is easier. There should be at least some insight to be gained from tcpdump/netstat/EV_VERIFY. -- The choice of a Deliantra, the free code+content MORPG -==- _GNU_ http://www.deliantra.net ==-- _ generation ---==---(_)__ __ ____ __ Marc Lehmann --==---/ / _ \/ // /\ \/ / schm...@schmorp.de -=/_/_//_/\_,_/ /_/\_\ ___ libev mailing list libev@lists.schmorp.de http://lists.schmorp.de/mailman/listinfo/libev
Re: libev-4.31 has just been released
So, after a few more mails from Jens, things do get clearer. He never got my mail, and was concerned that my explanation made him look careless, when he obviously is the opposite and wants io_uring to succeed (not his words, of course - I want it to succeed :). And, ahm, I guess, this is all great news - things are now a lot more amiable behind the scenes, and I think io_uring, if it isn't there yet, will in the not so far future, BE a drop-in replacement for select/poll,. but some work is still needed on both sides. So that means that I have the biggest expectation that the io_uring backend will indeed become the default backend in libev on new enough kernel in one of the next releases. Boy, I so want the epoll backend to die :) I already reflected some news in CVS - it seems that io_uring (and already did in some cases) take care of my concerns, or at least makes them a non-issue, and I am trying to coax Jens into a clear statement that _all_ fds supported by either select (or at least the subset supported by epoll) will be supported, so it will become a full replacement for either. just nicer, and faster. But I did get the impression that this is indeed the goal. Thanks, Jens, for giving me back my excitement that I felt about io_uring originally :) -- The choice of a Deliantra, the free code+content MORPG -==- _GNU_ http://www.deliantra.net ==-- _ generation ---==---(_)__ __ __ Marc Lehmann --==---/ / _ \/ // /\ \/ / schm...@schmorp.de -=/_/_//_/\_,_/ /_/\_\ ___ libev mailing list libev@lists.schmorp.de http://lists.schmorp.de/mailman/listinfo/libev
Re: libev-4.31 has just been released
oll, the > > issues with io_uring under high load, the abysmal state of documentation > > and so on, I didn't really bother to play with it much more in its current > > state. > > I'll try that too. 5.1 is ancient, and the first release with io_uring. > I would not expect too much, it's a lot more mature now. I think you mean 5.2, sure, and I accept your interesting definition of ancient :) > Jens Axboe So, Jens. When I got your first mail I was very excited because I thought there is a real way out of this dilemma. As I said, I really would like io_uring to become the default backend for libev (and thus a great deal of other software). You cannot fathom how excited I was when I first read about io_uring. Well, actually, it was more like "shit, now I absolutely _have_ to implement this, even though I have no time". I really want this to work. But this mail just attacks me for things I haven't said or done, and now I have the feeling of a maintainer who just wants to bash people who criticise their work, rather than making as good software solution. I am quite sure I do not deserve your attacks on me, as I was quite factual. For example, I told you that I mailed you *personally*. I do not expect, and I certainly do not wish you to then chastise me for not cc'ing you on a bug report or not telling you, or unfairly mixing linux aio with io_uring just because I *correctly* point out that io_uring and linux aio suffer from the same bug. So, we can either stop here, and leave me (and probably you) disappointed with the state of affairs, or you get down from your needlessly defensive posture. On the other side, I have zero desire to throw around shit and see what sticks - you can criticise me to your hearts content, I am used to usenet-style blunt words, but at least try to attack me for things I actually said or did. Or actually didn't. The choice is yours, I very much wish for an amiable and productive exchange, and I sincerely wish you nice holidays. And, we can continue this in private, or in public, but I would like to stick to either, and not both. -- The choice of a Deliantra, the free code+content MORPG -==- _GNU_ http://www.deliantra.net ==-- _ generation ---==---(_)__ __ __ Marc Lehmann --==---/ / _ \/ // /\ \/ / schm...@schmorp.de -=/_/_//_/\_,_/ /_/\_\ ___ libev mailing list libev@lists.schmorp.de http://lists.schmorp.de/mailman/listinfo/libev
Re: libev-4.31 has just been released
On Sat, Dec 21, 2019 at 06:45:20PM +, Benjamin Mahler
wrote:
> Sounds like some of the iouring findings are surprising to Jens (the
> author).
Well, I mailed him personally (no response), opened bug reports on
bugzilla.kernel.org (no response), and even found a discussion on the most
pressing problem on the linux-kernel list that I found after debugging
this in the kernel, from where I paraphrased linus's reaction.
https://bugzilla.kernel.org/show_bug.cgi?id=204081 that bug
https://bugzilla.kernel.org/show_bug.cgi?id=204065 oops bug
https://bugzilla.kernel.org/show_bug.cgi?id=204085 linux aio potential security
bug (not iouring)
https://lore.kernel.org/patchwork/patch/1047453/ the problem in linux aio, same
in io_uring
"I guess nobody really expects to use aio_poll on anything but a socket or
something?"
I understand we are all busy, and bugzilla.kernel.org sems to work a lot
like the usenet wizard groups (saome areas are just blackholes, such as
btrfs and aio/iouring, others are quite active, such as ext4 :), but on
the other hand, Jens really should not be that _surprised_ :)
Now, if this is what gets his attention, just the better, a working and
performant io_uring could finally get us out of epoll hell, although
kqueue still clearly leads the design, despite its cumbersome API.
In any case, I put most of my findings into the comments at the top of
ther source files - in my panic in trying to rescue what is clearly the
best event api in linux so far, I even started to embed epoll to *maybe*
get it into a working state, which kind of gets us the worst of all
worlds:
http://cvs.schmorp.de/libev/ev_iouring.c
http://cvs.schmorp.de/libev/ev_linuxaio.c
> Is there a benchmark he can run to look into this?
Haven't tried it, but the libevent/libev behcmark should show the bad
performance compared to epoll (and even poll/select):
http://data.plan9.de/bench.c
Can be compiled in a libev directory:
gcc -O3 -DNATIVE=1 -DEV_MULTIPLICITY=0 -o bench bench.c -I. ev.c event.c -lm
And then, for e.g. 1 fds:
LIBEV_FLAGS=1 ./bench -n 1 -a 100 # select
LIBEV_FLAGS=4 ./bench -n 1 -a 100 # epoll
LIBEV_FLAGS=128 ./bench -n 1 -a 100 # iouring
I don't actually remember what exactly I benchmarked (it was a number of
throwaway oneliners), but between kernel oopses, having to use epoll, the
issues with io_uring under high load, the abysmal state of documentation
and so on, I didn't really bother to play with it much more in its current
state.
> Do you have more explanation about "silently ignore parts of the requested
> events on an undocumented subset of file description types"?
I hope all required info is in the bugzilla bug report.
If anybody wants more information, I will happily provide more - it's just
that I already made a very large investment just to be disappointed, so I'd
like to have a perspective on whether its really worth it to invest more time
in this backend.
--
The choice of a Deliantra, the free code+content MORPG
-==- _GNU_ http://www.deliantra.net
==-- _ generation
---==---(_)__ __ __ Marc Lehmann
--==---/ / _ \/ // /\ \/ / schm...@schmorp.de
-=/_/_//_/\_,_/ /_/\_\
___
libev mailing list
libev@lists.schmorp.de
http://lists.schmorp.de/mailman/listinfo/libev
Re: Wallclock jumps with/in/and libev
On Thu, Dec 19, 2019 at 01:59:23AM +0100, Bernd Petrovitsch wrote: > Can this be handled with libev (and - if yes - how) somewhat > clean? Since you inadvertently forced me to give this some though, I have implemented the method using timerfd that I outlined in my last reply. The latest CVS version at http://software.schmorp.de/pkg/libev.html has support for this, but I plan to release libev 4.31 using this in a few hours. Basically, when the first ev_periodic watcher is started, libev tries to create a timerfd and then rearms it on every timejump, rescheduling periodics instantly. If you want to give it a try, feel free, but also feel free to not to if you are busy :) Feedback, of course, welcome. Thanbks for nudging me, I planned to do this for a long time. -- The choice of a Deliantra, the free code+content MORPG -==- _GNU_ http://www.deliantra.net ==-- _ generation ---==---(_)__ __ __ Marc Lehmann --==---/ / _ \/ // /\ \/ / schm...@schmorp.de -=/_/_//_/\_,_/ /_/\_\ ___ libev mailing list libev@lists.schmorp.de http://lists.schmorp.de/mailman/listinfo/libev
Re: Wallclock jumps with/in/and libev
On Thu, Dec 19, 2019 at 01:59:23AM +0100, Bernd Petrovitsch wrote: > Can this be handled with libev (and - if yes - how) somewhat > clean? Well, define "somewhat clean"? :) libev should handle all forms of timejumps correctly, although a delay can be introduced. You can study the gory details in ev.c:time_update () if you have very technical questions :) (You can assume have_monotonic to be true in your case). > I'm aware of timerfd_create(2) and TFD_TIMER_CANCEL_ON_SET on > somewhat recent Linux kernels. Me too, but libev doesn't use this at the moment (and would have to work without it in either case). Note that you can relatively easily make use of this yourself by adding such a timerfd as a wakeup source to your libev loop. Simply having an io watcher will do - libev currently checks for time jumps every second (see MIN_TIMEJUMP in the code, but reducing it to much lower than 1s probably won't improve things much). You should even be able to improve things by trying to call ev_suspend()/ev_resume() in your timerfd callback, as that reschedules periodics unconditionally. The drawback is that the time between ev_suspend and ev_resume is lost to any ev_timers. BTW., now that I think about how easy it might be, I will consider including TFD_TIMER_CANCEL_ON_SET more strongly - I hadn't given it much though till now because I thought integrating it would be a headache w.r.t. to the exiting code, but mybe it is as easy as rescheduling the periodics on each timejump... Thanks for forcing me to think about it :) > I want to avoid a workarounds like "use a monotonic timer > for the wallclock timeout and check the wallclock afterwards" > to not wait too long if the wallclock jumps back or to check > for wallclock changes in some very short timer. That's effectively what libev does for you, for some value of "short timer" that is probably around one second or one minute, depending on how you would implement it. Lastly, there is very little operational experience with all this - if you try some of these things out and have some feedback, I'd be happy to hear it. -- The choice of a Deliantra, the free code+content MORPG -==- _GNU_ http://www.deliantra.net ==-- _ generation ---==---(_)__ __ __ Marc Lehmann --==---/ / _ \/ // /\ \/ / schm...@schmorp.de -=/_/_//_/\_,_/ /_/\_\ ___ libev mailing list libev@lists.schmorp.de http://lists.schmorp.de/mailman/listinfo/libev
Re: Segmentation fault in pipecb
On Wed, Oct 30, 2019 at 01:22:22AM +, Calum McPherson wrote: > As noted this was 1st usage after a restart among otherwise stable, heavy > production use.I have scanned the mailing list from release 4.25 through > 4.27 and didn't see this issue raised therein. Likely because the issue is not in libev but in the rest of the program somehow causing memory corruption - for example, freeing memory that is still in use after forgetting to stop an active watcher, or make thread-unsafe calls, or... libev has "verification" mode that *might* be able to diagnose the problem a bit earlier by having a lot more and more frequent checks - look for EV_VERIFY in the documentation - you need to recompile/relink libev/your program for this. I'd try that first if the effort isn't prohibitive. valgrind would also be a very good thing to use, with luck, it can immediatelly pinpoint the bug. Otherwise, when running out of magic tools, you need to debug your program and find the issue, just like in the old days. -- The choice of a Deliantra, the free code+content MORPG -==- _GNU_ http://www.deliantra.net ==-- _ generation ---==---(_)__ __ __ Marc Lehmann --==---/ / _ \/ // /\ \/ / schm...@schmorp.de -=/_/_//_/\_,_/ /_/\_\ ___ libev mailing list libev@lists.schmorp.de http://lists.schmorp.de/mailman/listinfo/libev
Re: libev-4.27 timeout not being respected?
On Wed, Aug 28, 2019 at 10:30:36AM +1200, Samuel Williams wrote: > We noticed some regressions in timeout handling in libev-4.27 vs 4.25 > > I am just starting to investigate, but I thought I'd start a discussion > here too. There shouldn't be any intentional changes to timer handling, but bugs can always creep in, of course. Without knowing what regressions you see, it is a bit moot to discuss. > In addition, some changes to the source code are generating a lot of > warnings. You can see the test failures and warnings here: Well, which changes specifically? I only get one warning with current versions of gcc, so you probably use another compiler with its own warnings or enable extra warnings tailored to your specific coding style or needs. You can read a bit more about this in the documentation: http://pod.tst.eu/http://cvs.schmorp.de/libev/ev.pod#COMPILER_WARNINGS -- The choice of a Deliantra, the free code+content MORPG -==- _GNU_ http://www.deliantra.net ==-- _ generation ---==---(_)__ __ ____ __ Marc Lehmann --==---/ / _ \/ // /\ \/ / schm...@schmorp.de -=/_/_//_/\_,_/ /_/\_\ ___ libev mailing list libev@lists.schmorp.de http://lists.schmorp.de/mailman/listinfo/libev
libev-4.27 has just been released
I am pleased to announce libev 4.27! Distribution: http://dist.schmorp.de/libev/libev-4.27.tar.gz Documentation: http://pod.tst.eu/http://cvs.schmorp.de/libev/ev.pod?pathrev=rel-4_27 Changes: http://cvs.schmorp.de/libev/Changes?pathrev=rel-4_27 This release mainly adds a new backend, "linuxaio", which uses the new event polling mechanism introduced with Linux 4.18 (but libev requires at least 4.19). The new backend should be ready for general use, but is not in the "recommended" backend set, i.e., it will not be chosen by default unless explicitly requested either by adding EVBACKEND_LINUXAIO to your flags when creating a loop, or by setting the LIBEV_FLAGS environment variable. The linuxaio backend will likely be enabled by default, and take precedence over epoll, in a future release (likely 4.28). For example, to run a program using linuxaio if available, but keep fallbacks to epoll and poll/select, you can run it like this: LIBEV_FLAGS=127 ./myprog Or to try forcing linuxaio: LIBEV_FLAGS=64 ./myprog Feedback appreciated. The complete Changes since 4.25 are: 4.27 Thu Jun 27 22:43:44 CEST 2019 - linux aio backend almost complete rewritten to work around its limitations. - epoll backend now mandatory for linux aio backend. - fail assertions more aggressively on invalid fd's detected in the event loop, do not just silently fd_kill in case of user error. - ev_io_start/ev_io_stop now verify the watcher fd using a syscall when EV_VERIFY is 2 or higher. 4.26 (EV only) - update to libecb 0x00010006. - new experimental linux aio backend (linux 4.18+). - removed redundant 0-ptr check in ev_once. - updated/extended ev_set_allocator documentation. - replaced EMPTY2 macro by array_needsize_noinit. - minor code cleanups. - epoll backend now uses epoll_create1 also after fork. -- The choice of a Deliantra, the free code+content MORPG -==- _GNU_ http://www.deliantra.net ==-- _ generation ---==---(_)__ __ __ Marc Lehmann --==---/ / _ \/ // /\ \/ / schm...@schmorp.de -=/_/_//_/\_,_/ /_/\_\ ___ libev mailing list libev@lists.schmorp.de http://lists.schmorp.de/mailman/listinfo/libev
Re: About "./configure" Checking the build environment
On Wed, Jun 26, 2019 at 07:13:52AM +0800, CandyMI <869646...@qq.com> wrote: > I recently tried to install libev painlessly on Win32 to run my poor Lua Web > Web framework on it. > > > However, Win32 requires embedded compilation to take effect. Can I get better > libev build suggestions here? Other than using cygwin or windows linux emulation, or their services for unix, the only way would be to provide your own config.h file in advance. But since windows doesn't have as standard ABI or even API, and most compilers being incomaptible to each other, it's close to impssible to do anything else, as libev can't know the compilation environment in advance - that's what configure would be for. Even then, configure couldn't guess how you want to map handles to fds and vice versa. -- The choice of a Deliantra, the free code+content MORPG -==- _GNU_ http://www.deliantra.net ==-- _ generation ---==---(_)__ __ ____ __ Marc Lehmann --==---/ / _ \/ // /\ \/ / schm...@schmorp.de -=/_/_//_/\_,_/ /_/\_\ ___ libev mailing list libev@lists.schmorp.de http://lists.schmorp.de/mailman/listinfo/libev
Re: About "./configure" Checking the build environment
On Wed, Jun 26, 2019 at 03:43:33AM +0800, CandyMI <869646...@qq.com> wrote: > I checked and found that the "./configure" file was taken from the package > file of libev-4.25.tar.gz. Good to hear :) > By the way, I saw the macro definition of IOCP in "ev.c". Does it mean that > libev will have a plan supported by the Windows platform? libev has pretty much always (at least since 2007) supported windows using its select backend. the reference to iocp is a reference to windows i/o control ports, for which I tried to make a more performant interface, but it turned out that iocps (at least at the time) only supported very few handle types, many tcp socket types not among them, so it would be even less useful than select which at least seems to support all socket types, and the code was dropped. -- The choice of a Deliantra, the free code+content MORPG -==- _GNU_ http://www.deliantra.net ==-- _ generation ---==---(_)__ __ __ Marc Lehmann --==---/ / _ \/ // /\ \/ / schm...@schmorp.de -=/_/_//_/\_,_/ /_/\_\ ___ libev mailing list libev@lists.schmorp.de http://lists.schmorp.de/mailman/listinfo/libev
Re: About "./configure" Checking the build environment
On Wed, Jun 26, 2019 at 01:15:25AM +0800, CandyMI <869646...@qq.com> wrote: > When I tried to build libev based on "configure", it didn't try to detect if > the system contains the following "linux/aio_abi.h" file Unless I build with > "sh autogen.sh". > > > Does this mean that using "./configure" alone is wrong? Hi! linux/aio_abi.h is not checked or used in any released version of libev, so you must be using a CVS checkout, is this correct? If not, where did you get your copy? CVS does not contain a ./configure file, so where is your configure file from? When building for CVS, you indeed first need to generate a configure script by running autogen.sh. The release tarballs would contain all the necessary files pregenarated, and its configure scirpt should work. So in short, in a release tarball, ./configure should work. In a CVS checkout, there shouldn't be a configure script. -- The choice of a Deliantra, the free code+content MORPG -==- _GNU_ http://www.deliantra.net ----==-- _ generation ---==---(_)__ __ __ Marc Lehmann --==---/ / _ \/ // /\ \/ / schm...@schmorp.de -=/_/_//_/\_,_/ /_/\_\ ___ libev mailing list libev@lists.schmorp.de http://lists.schmorp.de/mailman/listinfo/libev
Re: How to know connection disconnected while receive paused
On Fri, Jun 21, 2019 at 08:25:22PM +, Tan Xiaofan wrote:
> > general, there is no way to do it, neither for you nor for libev - you
> > have to read the data that was sent to you until you get EOF or an error.
>
> If user space receive buffer is full, program can't call recv() because there
> is no buffer to use.
Then you can't detect EOF, really.
> > Well, if thew other side closed the socket, then eventually you will get an
> > error on send as well (most likely ECONNRESET).
>
> I call send() before protocol stack receive FIN packet from client,
A FIN from the rmeote end does not close the connection. This means it is
perfectly valid to call send, and the data will be delivered to the remote
host.
All a FIN does is signal that the remote host stopped sending dats ("end
of file").
> It should have waited a EV_WRITE event and continue to send remain data, but
> in this time protocol stack received client FIN packet,
> so transmit will be stopped and EV_WRITE never to come again, I have no
> opportunity call send() again to get that ECONNRESET code.
> Is there some way to handle this case?
Libev, your kernel and not even TCP/IP can know that the remote end
closing the connection means you also want to close the connection. Thats
policy you have to implement yourself, and the way to do so is generally
to consume the data you received until you hit EOF.
The key to understand here is that the connection isn't closed at that
point, so it wouldn't help you if you could detect a closed conenction
(which probably could be done e.g. using getpeername). But even after the
remote end sends a FIN, the connection is still open.
It's just how TCP works.
--
The choice of a Deliantra, the free code+content MORPG
-==- _GNU_ http://www.deliantra.net
----==-- _ generation
---==---(_)__ __ __ Marc Lehmann
--==---/ / _ \/ // /\ \/ / schm...@schmorp.de
-=/_/_//_/\_,_/ /_/\_\
___
libev mailing list
libev@lists.schmorp.de
http://lists.schmorp.de/mailman/listinfo/libev
Re: How to know connection disconnected while receive paused
On Fri, Jun 21, 2019 at 06:51:03PM +, Tan Xiaofan wrote: > I am learning how to write a server program using libev, but I encounter > some confusion. > > I write a very simple server, the server receive data and send back > intact. In order to handle the case of receive speed fast than send speed, > I pause data receive when send back not complete, send remain data while > EV_WRITE event coming, restart receive when remain data send finished. In general, with e.g. TCP sockets, you are the only one who can close the connection, and only your side. And you can't detect a connection close by the other end without reading all data. For reading, that means you you need to call read/recv/etc., and on EOF (remote connection close) you get a zero length read. For write, you will get an error, but usually you will have to initiate the shutdown from your side anyway, and handle errora on both read/write as well. If you don't read, you can't detect a remote connection close, and in general, there is no way to do it, neither for you nor for libev - you have to read the data that was sent to you until you get EOF or an error. > When receive paused, peer close socket after I call send(), I can not > receive any event to indicate the connection is disconnected. > How to fix it? Well, if thew other side closed the socket, then eventually you will get an error on send as well (most likely ECONNRESET). So, you just write yozur server normally, you read whenever oyu want, you write whenever you want, you use libev whenever you want to be notified of being able to read or write, and you handle errors as normal. There is nothing else you need to do, and nothing else you can do, and probably nothing else you should do :) -- The choice of a Deliantra, the free code+content MORPG -==- _GNU_ http://www.deliantra.net ==-- _ generation ---==---(_)__ __ __ Marc Lehmann --==---/ / _ \/ // /\ \/ / schm...@schmorp.de -=/_/_//_/\_,_/ /_/\_\ ___ libev mailing list libev@lists.schmorp.de http://lists.schmorp.de/mailman/listinfo/libev
Re: ocserv-main segfault in libev.so.4.0.0
On Thu, Jun 20, 2019 at 10:37:25PM +0800, §汤圆§/ty <20207...@qq.com> wrote: > I am a small development team here, about 20 users connect to the ocserv > server through cisco anyconnect. I don't know how to manually reproduce this > problem, but in my scenario, the ocserv-main process will exit with > segment-fault. All users will drop from the vpn at the same time. Every day, > it will fail 1-2 times. It is not a fault at startup. Failure after running > for a while. Hi! > #0 0x in ?? () > (gdb) where > #0 0x in ?? () > #1 0x7f305fc553d5 in ev_invoke_pending (loop=0x7f305fe5ea40 > ) at ev.c:3322 > #2 0x7f305fc585b5 in ev_run (loop=0x7f305fe5ea40 , > flags=flags@entry=0) at ev.c:3726 > #3 0x55d4d269d7da in main (argc=, argv=) > at main.c:1440 Good that you provided a backtrace. This doesn't really look like a crash/bug in libev, but more like an outside bug (e.g. specifying an invalid event callback or something like that). If you want to debug this further, you should compile the whole program with debugging info and (if possible) without optimisation, e.g. -ggdb -O0 or at least -g. That way, you could see in what file and what line it crashes, and this might shed a lot of info on what is causing the problem. If the last frame is still ev_invoke_pending, you can enter it and "print *p" and "print *p->w", which should tell you what watcher is involved, and what callback is specified. > Program terminated with signal 11, Segmentation fault. > #0 child_reap (status=0, pid=1619, chain=1619, loop=0x7f49ca995a40 > ) at ev.c:2658 > 2658if ((w->pid == pid || !w->pid) > (gdb) where > #0 child_reap (status=0, pid=1619, chain=1619, loop=0x7f49ca995a40 > ) at ev.c:2658 > #1 childcb (loop=0x7f49ca995a40 , sw=, > revents=) at ev.c:2690 > #2 0x7f49ca78c3d5 in ev_invoke_pending (loop=0x7f49ca995a40 > ) at ev.c:3322 > #3 0x7f49ca78f5b5 in ev_run (loop=0x7f49ca995a40 , > flags=flags@entry=0) at ev.c:3726 > #4 0x559f444867da in main (argc=, argv=) > at main.c:1440 > p w->pid is a wild pointer. Thats a very different crash. Maybe you have some random memory corruption going on - it's hard to explain why a registered watcher pointer would suiddenly become invalid - most likely, this is again a bug in ocserv-main somewhere. The most common problem is that people reuse/overwrite/zero/corrupt/free a watcher structure while it is still in use by libev. You can recompile libev with -DEV_VERIFY=3 - this will check all available data structures at almost every call. This can slow down event handling a lot, but it can also crash much earlier - more likely at the place where the problem actually happens. -- The choice of a Deliantra, the free code+content MORPG -----==- _GNU_ http://www.deliantra.net ==-- _ generation ---==---(_)__ __ __ Marc Lehmann --==---/ / _ \/ // /\ \/ / schm...@schmorp.de -=/_/_//_/\_,_/ /_/\_\ ___ libev mailing list libev@lists.schmorp.de http://lists.schmorp.de/mailman/listinfo/libev
libev: new experimental Linux AIO backend available for testing
Hi! I am pleased to announce that the current CVS version of libev has an experimental linux aio backend that can take advantage of the new functionality added to linux 4.18 (https://lwn.net/Articles/743714/). It is exciting because it can batch both requests and events (and mightr be much faster than epoll in common use cases), and does not have most of the downsides and design issues of epoll. Of course, this wouldn't be linux if it would be usable out of the box: as of 4.19, only very few file types are supported, namely, from my testing, pipes, sockets, files (surprise! beats epoll!) and a few devices (/dev/null), but not e.g. ttys (dangit!), so it is not yet working as a generic epoll replacement (basically, it's like a worse kqueue). And it's not embeddable either of course, so you can't even combine it with another more generic mechanism. D'oh. Currently, the code is not very optimized and probably buggy, but ready for experimentation (being a few hours old by now :). Feedback would be very much appreciated, even if you don't want to use it, as the backend should currently just be silently compiled in by default without being used, so even reports about compilation errors would be helpful. If you want to try it out, you need a CVS checkout of libev: cvs -z3 -d :pserver:anonym...@cvs.schmorp.de/schmorpforge co libev If you use the libev.m4 autoconf macros then support for EVBACKEND_LINUXAIO aio should be automatically enabled when your headers allow it (e.g. SYS_io_setup and so on are defined and linux/aio_abi.h is available). If not, you try to force compilation by specifying -DEV_USE_LINUXAIO when compiling, e.g. for the EV perl module: EV_EXTRA_DEFS=-DEV_USE_LINUXAIO perl Makefile.PL; make; make install It is also currently always disabled at runtime by default, so if it compiles, nothing should change until you explicitgly request it, e.g. using the LIBEV_FLAGS environment variable: LIBEV_FLAGS=64 ./my_libev_program again, feedback at this stage is very appreciated, although I wouldn't expect it to work flawlessly or conduct serious performance tests. Thanks in advamce and have fun! -- The choice of a Deliantra, the free code+content MORPG -==- _GNU_ http://www.deliantra.net ==-- _ generation ---==---(_)__ __ __ Marc Lehmann --==---/ / _ \/ // /\ \/ / schm...@schmorp.de -=/_/_//_/\_,_/ /_/\_\ ___ libev mailing list libev@lists.schmorp.de http://lists.schmorp.de/mailman/listinfo/libev
Re: migrating libev to github.
On Mon, May 06, 2019 at 04:53:57PM +0800, jianhui zhao wrote: > It is recommend migrating libev to github. Not true. -- The choice of a Deliantra, the free code+content MORPG -==- _GNU_ http://www.deliantra.net ==-- _ generation ---==---(_)__ __ __ Marc Lehmann --==---/ / _ \/ // /\ \/ / schm...@schmorp.de -=/_/_//_/\_,_/ /_/\_\ ___ libev mailing list libev@lists.schmorp.de http://lists.schmorp.de/mailman/listinfo/libev
Re: ev_set_allocator takes signed size
On Mon, Mar 18, 2019 at 11:28:11AM +1300, Samuel Williams wrote: > Is there some reason why ev_set_allocator is defined as: If I remember correctly, this was because size_t and ssize_t were not very portable. > Because this produces warnings when trying to use `realloc` style > functions, defined as: Well, "relloc style functions" is a rather fuzzy set - if you limit yourself to realloc functions only, then this is probably a good thing, as the semantic of a large number of realloc functions is not of the required type, so you probably need a wrapper in any case to work around incompatibilities (for example, libev itself uses such a wrapper by default, called ev_realloc_emul). -- The choice of a Deliantra, the free code+content MORPG -==- _GNU_ http://www.deliantra.net ==-- _ generation ---==---(_)__ __ __ Marc Lehmann --==---/ / _ \/ // /\ \/ / schm...@schmorp.de -=/_/_//_/\_,_/ /_/\_\ ___ libev mailing list libev@lists.schmorp.de http://lists.schmorp.de/mailman/listinfo/libev
libev 4.25 has just been released
libev 4.25 has just been released, you can find it at the usual place: http://dist.schmorp.de/libev/ This is predominentaly a bugfix/portability release, with some modernisations for automake and C++ 11 and later. If libev works for you currently, there is no urgent need to upgrade. The full changes are: 4.25 Fri Dec 21 07:49:20 CET 2018 - INCOMPATIBLE CHANGE: EV_THROW was renamed to EV_NOEXCEPT (EV_THROW still provided) and now uses noexcept on C++11 or newer. - move the darwin select workaround highe rin ev.c, as newer versions of darwin managed to break their broken select even more. - ANDROID => __ANDROID__ (reported by e...@google.com). - disable epoll_create1 on android because it has broken header files and google is unwilling to fix them (reported by e...@google.com). - avoid a minor compilation warning on win32. - c++: remove deprecated dynamic throw() specifications. - c++: improve the (unsupported) bad_loop exception class. - backport perl ev_periodic example to C, untested. - update libecb, biggets change is to include a memory fence in ECB_MEMORY_FENCE_RELEASE on x86/amd64. - minor autoconf/automake modernisation. -- The choice of a Deliantra, the free code+content MORPG -==- _GNU_ http://www.deliantra.net ==-- _ generation ---==---(_)__ __ __ Marc Lehmann --==---/ / _ \/ // /\ \/ / schm...@schmorp.de -=/_/_//_/\_,_/ /_/\_\ ___ libev mailing list libev@lists.schmorp.de http://lists.schmorp.de/mailman/listinfo/libev
Re: question on using INADDR_LOOPBACK in ev_pipe
On Thu, Nov 29, 2018 at 02:57:29PM -0800, Tim Na wrote: > Sorry about that silly mistake (or threats) - I just learned not to use > company's email system for such request now. Sincere apologies for such > annoyance to you and libev community. > > However, I question still stands and I am looking at libuv as alternative > choice as it seems to have no such limitation as libevent or libev. There is no limitation in libevent or libev, what you describe is a bug or misconfiguration of other components on your system (basically, you are saying that using TCP/IP is a limitation in libev, this is silly). If you disable your broken virus scanner of whatever causes the problems things start to work (this also affects firefox and many other programs which are forced to poll if they can't get a local socket to work, i.e. they might superficially work, but only because they run a 100Hz timer in the background and poll for events). Another option would be to switch to an OS such as GNU/Linux, which doesn't have the limitations of the windows platform (you haven't mentioned what platform you are using, but it sure sounds like windows) - both in terms of broken virus scanners/firewalls breaking the TCP/IP stack and in lacking efficient local sockets. You could also try libuv, although my personal totally unbiased opinion is that libuv suffers from many design bugs that more mature libraries such as libev (or libevent) do not suffer from because libuv was designed by people unknowledgable in event systems and therefore were bound to repeat many mistakes of earlier ad-hoc event libraries. Of course, if you don't care about correctness and just want to make things superficially working, then you could call it a limitation in libevent/libev and use libuv, if that works for you (and unknowingly suffer later because you run into design issues you wouldn't notice with better event libraries). A real reason to switch to libuv is performance - libev doesn't do I/O for you, and this tends to make it slow on windows, which doesn't support I/O readyness notifications efficiently. libuv _does_ do I/O for you and might be more performant if all you can use is windows. OTOH, if all you can use is windows, you clearly don't care about performance much, so why bother :) Using libuv in this way will also require to rewrite any existing event-based code to be I/O-based instead which might or might not be an issue. -- The choice of a Deliantra, the free code+content MORPG -==- _GNU_ http://www.deliantra.net ==-- _ generation ---==---(_)__ __ ____ __ Marc Lehmann --==---/ / _ \/ // /\ \/ / schm...@schmorp.de -=/_/_//_/\_,_/ /_/\_\ ___ libev mailing list libev@lists.schmorp.de http://lists.schmorp.de/mailman/listinfo/libev
Re: question on using INADDR_LOOPBACK in ev_pipe
> *Confidentiality Notice: The information contained in this e-mail and any > attachments may be confidential. If you are not an intended recipient, you > are hereby notified that any dissemination, distribution or copying of this > e-mail is strictly prohibited. If you have received this e-mail in error, > please notify the sender and permanently delete the e-mail and any > attachments immediately. You should not retain, copy or use this e-mail or > any attachment for any purpose, nor disclose all or any part of the > contents to any other person. Thank you.* Please don't send such silly threats to this mailinglist - there is no way for us to know what the intended recipient is and no way to stop redistribution, so the only way to comply with your wishes is to block you, which creates extra needless work for us. -- The choice of a Deliantra, the free code+content MORPG -==- _GNU_ http://www.deliantra.net ==-- _ generation ---==---(_)__ __ ____ __ Marc Lehmann --==---/ / _ \/ // /\ \/ / schm...@schmorp.de -=/_/_//_/\_,_/ /_/\_\ ___ libev mailing list libev@lists.schmorp.de http://lists.schmorp.de/mailman/listinfo/libev
Re: Switch to github
On Tue, Sep 04, 2018 at 07:54:37PM +0200, Carlo Wood wrote: > He probably meant meant git (although then using github.com is a > logical choice). Well, he seems to have meant github, after all. > One of the reasons that switching to git makes sense is that > CVS is simply outdated. You mean outdated as in, say, bicycles and desktop PCs are outdated? That doesn't seem to be a compelling reason to change... > of technical differences that boil down to the fact that git > allows easy cooperation between (many) developers. Git also makes cooperation between few developers harder, especially in a tightly-knit team, which is the case for almost all the projects I am working on. > The workflow is well fleshed out and supported. The last few times I tried to adopt git I kept running into "functionality not yet implemented" messages or similar problems - git is far from beiong as stable or fleshed out as CVS. Maybe it's better supported, though, but then, I didn't need much of support for CVS in recent years. Git surely has more (and different) features, none of which are compelling or useful enough for us to outweigh the downsides so far. > CVS still does the job, but it is for example way harder to > work on a project with more than one maintainer; I think this is simply untrue, by personal experience. > Even if you think "I don't WANT other people to mess with > my project; I want to be in full control as the only God-developer" Don't be silly, if I wanted that I wouldn't give my projects such liberal licenses in the first place. > then still that is not really reason to fear git: it will STILL Are you fearing git? I know nobody else who talks about fear w.r.t. git. > be easier for you to maintain libev and review patch proposals, Well, it won't, as I already know, which makes your assertion a bit puzzling. > and you can still reject everything. At most the chance that > someone forks you, write a few awesome improves that you refuse > to merge and people start using their close instead of yours, > is slightly larger; Interesting, why would that be so? Honestly, some of your statements sound as if you made them up on the spot, without any backing evidence. > because through github it is easier to find people who cloned projects Some people (that includes me) find it unethical to force others to use non-free software, even if, in some rare cases, it might make my life easier (although overall, tghe opposite has been the case for me). That's why github would be about the worst choice. Also, personal experience with other such hosting services in the past has shown that I can provide a more stable and longer lasting hosting service than those, and, more importantly, a hosting service that I can control and fix. Keep in mind that I am around providing and hosting software for far longer than github, sourceforge, gitlab and so on. I would encourage people to make mirrors, of course, as it is unlikely that I can provide this forever (I don't know what happens when I die, for example), but from what I can see, github or similar services are more likely to go down than me, so mirrors would do some good, moving to github almost certainly wouldn't. > mother project didn't accept their pull requests. That would maybe > be bad for you, but it certainly isn't bad for the community ;) Not sure if it were bad for me (why?), but I am not sure why you think it is a good thing for me to adopt something that you claim is bad for me? > A little bit of competition is healthy (ie, in the above scenario > you'd probably decide to take the time to look at those awesome > new improvements and merge them into your repository, instead of > losing your client base). I'm not sure at all how git or github would somehow increase "competition" (between whom?). In any case, I find your mail be very thin on facts, and rather full of assertions, many of which I already know to be false. I think if I wouldn't be informed about git and githib already, your mail would turn me away as it seems to be more of a religious thing and less something rooted in facts (again, if I judged only from your mail). -- The choice of a Deliantra, the free code+content MORPG -==- _GNU_ http://www.deliantra.net ==-- _ generation ---==---(_)__ __ __ Marc Lehmann --==---/ / _ \/ // /\ \/ / schm...@schmorp.de -=/_/_//_/\_,_/ /_/\_\ ___ libev mailing list libev@lists.schmorp.de http://lists.schmorp.de/mailman/listinfo/libev
Re: Switch to github
On Wed, Aug 29, 2018 at 02:30:21PM +0800, "jianhuizhao...@gmail.com" wrote: > It is recommended to switch from cvs to github. By whom? The evaluation of the FSF for example gave github the worst rating, an F. https://www.gnu.org/software/repo-criteria-evaluation.html -- The choice of a Deliantra, the free code+content MORPG -==- _GNU_ http://www.deliantra.net ==-- _ generation ---==---(_)__ __ ____ __ Marc Lehmann --==---/ / _ \/ // /\ \/ / schm...@schmorp.de -=/_/_//_/\_,_/ /_/\_\ ___ libev mailing list libev@lists.schmorp.de http://lists.schmorp.de/mailman/listinfo/libev
Re: I suggest you to add information about "FUNCTIONS CONTROLLING EVENT LOOPS" to several libev document sections.
On Mon, Jul 30, 2018 at 08:10:59PM +0900, 裕士辻 wrote: > > About two weeks ago, I asked you to "stop API of ev_once event?". > Because I couldn't find userdata yet at that time. I only found ev_once to > set user data in libev. Can you explain the connection between the userdata api and your ev_once question? It's not at all clear to me what one has to do with another. In any case, it's trivial to associate userdata with an event loop: struct ev_loop *myloop; mydata myuserdata; i.e., just store it wherever you store the loop. The userdata API is there for cases where you only have a loop pointer, such as in the callbacks mentioned in the documentation. Mentioning them in the watcher specific section cna only be confusing, as they are not related and will not be helpful. > It was difficult for me to find "userdata" area in WATCHER. Because there isn't any - the userdata api is for the loop, not for watchers, and certainly not for ev_once. To associate custom data with ev_once, you pass the custom data to ev_once (I have tried to clarify this in the documentation for ev_once). > After that, I could find it by your suggestion and some information. > And I thought 'Oh, I'm too fool, why didn't I read "FUNCTIONS CONTROLLING > EVENT LOOPS"'. If you are using userdata somehow because you use ev_once, then you are probably doing it wrong, but it's hard to tell. -- The choice of a Deliantra, the free code+content MORPG -==- _GNU_ http://www.deliantra.net ==-- _ generation ---==---(_)__ __ __ Marc Lehmann --==---/ / _ \/ // /\ \/ / schm...@schmorp.de -=/_/_//_/\_,_/ /_/\_\ ___ libev mailing list libev@lists.schmorp.de http://lists.schmorp.de/mailman/listinfo/libev
Re: How to watch and read a file that another process is appending to?
On Wed, Jul 25, 2018 at 08:06:39PM +0200, Carlo Wood wrote: > > can you please answer this question that I posted on > Stack Overflow: Sure. I have tried to give abit of background to help in udnerstanding these rather difficult problems, hope it helps! -- The choice of a Deliantra, the free code+content MORPG -==- _GNU_ http://www.deliantra.net ==-- _ generation ---==---(_)__ __ __ Marc Lehmann --==---/ / _ \/ // /\ \/ / schm...@schmorp.de -=/_/_//_/\_,_/ /_/\_\ ___ libev mailing list libev@lists.schmorp.de http://lists.schmorp.de/mailman/listinfo/libev
Re: Does libev provide stop API of ev_once event?
On Sat, Jul 14, 2018 at 01:43:07AM +0900, 裕士辻 wrote: > I use ev_once, it's very convenience. But I have a problem about it. > I seem there is no stop API for ev_once, isn't it? No, there isn't. If you need the ability to stop watchers, you need to create one with the other API functions. I suggest you look at the implementation of ev_once in ev.c and roll your own, as ev_once is close to trivial - it basically malloc's a struct with an ev_timer and an ev_io watcher and starts them, freeing the struct when the event fires. Adding a stop API seemingly makes little sense, as ev_once is provided for just those cases where you don't need this functionality, for everything else you can use a normal watcher. -- The choice of a Deliantra, the free code+content MORPG -==- _GNU_ http://www.deliantra.net ==-- _ generation ---==---(_)__ __ __ Marc Lehmann --==---/ / _ \/ // /\ \/ / schm...@schmorp.de -=/_/_//_/\_,_/ /_/\_\ ___ libev mailing list libev@lists.schmorp.de http://lists.schmorp.de/mailman/listinfo/libev
Re: Patch proposal.
On Wed, Jun 13, 2018 at 01:32:05AM +0200, Carlo Wood wrote: > I had to fix a few minor things, so I thought I'd drop a line here. > See > https://github.com/CarloWood/evio/commit/b46d1fd72166262fc81309d5ab8b267658ca28da I can't really see anything that would qualify as fixing something - the first change breaks things badly for people who want libev to work with fork but disable fork watchers (and "why not" is not a good enough rationale for this :), and the rest are obviously not fixing anything, as the correctness of the code didn't change. I can look into the warnings, but I am not a fan of complicating things and potentially breaking things to silence random compiler warnings, see http://pod.tst.eu/http://cvs.schmorp.de/libev/ev.pod#COMPILER_WARNINGS -- The choice of a Deliantra, the free code+content MORPG -==- _GNU_ http://www.deliantra.net ==-- _ generation ---==---(_)__ __ __ Marc Lehmann --==---/ / _ \/ // /\ \/ / schm...@schmorp.de -=/_/_//_/\_,_/ /_/\_\ ___ libev mailing list libev@lists.schmorp.de http://lists.schmorp.de/mailman/listinfo/libev
Re: Better Windows backend using wepoll?
On Thu, May 31, 2018 at 07:55:13PM +0800, Linus Yang wrote: > Libev has relatively weaker support on Windows than other Unix-like > platforms (Linux, BSD and macOS) due to the complexity of IOCP event > model. Well, it's not weaker when it supports the same interface on windows. The problem libev ran into was that IOCPs only work with some sockets, not all sockets, and do not portably support event notifications. If this has somehow been solved (or was only a problem on obsolete windows platforms) this could be changed. Or, rather, if the wepoll API is reasonably similar to epoll, one could indeed recompile libev with epoll as backend, probably with miniscule changes. > I tried to port shadowsocks-libev [2] to MinGW using wepoll. The libev > library only needs very few changes [3] to use wepoll as backend. We > currently maintain a local fork of libev and prefer if such change > could go upstream. > > Do you have any interest or opinions on integrating the wepoll library > inside libev for better Windows support? Since the wepoll licenses is more restrictive, it cannot be integrated, but it should not be an issue to support wepoll as another backend, especially since it seems to closely emulate the epoll API. Another option would be to find out how wepoll works around the existing problems with IOCPs only supporting a subset of sockets (if it does) and change the windows backend in libev to use the same method, especially if it seems simple. -- The choice of a Deliantra, the free code+content MORPG -==- _GNU_ http://www.deliantra.net ==-- _ generation ---==---(_)__ __ __ Marc Lehmann --==---/ / _ \/ // /\ \/ / schm...@schmorp.de -=/_/_//_/\_,_/ /_/\_\ ___ libev mailing list libev@lists.schmorp.de http://lists.schmorp.de/mailman/listinfo/libev
Re: Event notice between processes, not threads?
On Tue, May 29, 2018 at 09:07:26AM -0700, Daniel Austin wrote: > Hi, Marc. Thank you for responding. I did a little investigation on your > comment about fds being useless to other processes and did not know they > were process-unique. Thanks for stopping me early. Your suggestion of a > pipe looks like it will be the fastest (to implement) approach. Would I > simply have each watcher process ev_io watch the fifo for readable event, A pipe is not a (linux-) fifo, but you can use an ev_io watcher in every process on fifo or pipe fd's alike. > the writer process puts a byte into the fifo, all the watcher processes get > the readable event and wake up and non-block read to empty the pipe (one > succeeds), do their respective tasks, and then repeat? Thanks very much. That's one way to do it - make sure you handle the case where the pipe is full gracefully, though (e.g. by not blocking your writer, or ensuring that this a non-issue). -- The choice of a Deliantra, the free code+content MORPG -==- _GNU_ http://www.deliantra.net ==-- _ generation ---==---(_)__ __ ____ __ Marc Lehmann --==---/ / _ \/ // /\ \/ / schm...@schmorp.de -=/_/_//_/\_,_/ /_/\_\ ___ libev mailing list libev@lists.schmorp.de http://lists.schmorp.de/mailman/listinfo/libev
Re: Event notice between processes, not threads?
On Sun, May 27, 2018 at 10:03:32PM -0700, Daniel Austin wrote: > Hi. What is the proper mechanism to trigger an event in a separate > process, not a separate thread, assuming a modern Linux kernel? > > 1) Use ev_io on an eventfd file descriptor. Use normal eventfd file That, or a pipe, are the most reasonable options. Which one is faster depends very much on the application. > "register" with the triggering process, but they do need to get access to > the file description number somehow (e.g. shared memory or other means). The fd number is useless in another process - you cna pass file descriptions to other pürocesses with file descriptor passing via a unix domain socket, though, or inheriting using fork for example. > 2) Use ev_async. I think the proper usage here would be to put the ev_async cannot be used for this purpose. > 3) Something else? Use inotify to watch for file changes. There are some other mechanisms, but for general event handling, you want a pipe or something like eventfd. -- The choice of a Deliantra, the free code+content MORPG -==- _GNU_ http://www.deliantra.net ==-- _ generation ---==---(_)__ __ __ Marc Lehmann --==---/ / _ \/ // /\ \/ / schm...@schmorp.de -=/_/_//_/\_,_/ /_/\_\ ___ libev mailing list libev@lists.schmorp.de http://lists.schmorp.de/mailman/listinfo/libev
Re: periodics_reschedule crash
On Tue, May 01, 2018 at 11:33:06AM +0800, 醉清风 <525331...@qq.com> wrote: > Hi， > > app crash when call periodics_reschedule，how can I do？ Debug it and make it not crash, I guess? There is very little additional info you gave us, starting with the obvious "how and why did it crash?" > libsystem_kernel.dylib__pthread_kill + 8 > 1 libsystem_pthread.dylib_pthread_kill$VARIANT$mp + 376 > 2 libsystem_c.dylibabort + 140 > 3 libsystem_c.dylibbasename_r + 0 > 4 Teleboxperiodics_reschedule (ev.c:2846) > 5 Teleboxlocal_main (local.m:615) > 6 Telebox+[ShadowsocksRunner runProxy] (ShadowsocksRunner.m:27) If this is a backtrace of an abort call, maybe you get an assertion failure? If yes, which one? -- The choice of a Deliantra, the free code+content MORPG -==- _GNU_ http://www.deliantra.net ==-- _ generation ---==---(_)__ __ __ Marc Lehmann --==---/ / _ \/ // /\ \/ / schm...@schmorp.de -=/_/_//_/\_,_/ /_/\_\ ___ libev mailing list libev@lists.schmorp.de http://lists.schmorp.de/mailman/listinfo/libev
Re: libev and libevent
. giving up when your skills are not up tp the job is probably the best solution, also for your users. It's also fine to suggest people to avoid using libev - libev was always provided "as is", and certainly is a tool for experts and not beginners, and anybody making thier event loop selection based on misunderstanding of basic programming techniques is probably better of with other event libraries, although I am not sure what those would be. In any case, you are clearly only intersted in trolling and not interested in solving your problem, so I regret having tried to help you. Good luck for your futgure endeavours and good bye. -- The choice of a Deliantra, the free code+content MORPG -==- _GNU_ http://www.deliantra.net ==-- _ generation ---==---(_)__ __ __ Marc Lehmann --==---/ / _ \/ // /\ \/ / schm...@schmorp.de -=/_/_//_/\_,_/ /_/\_\ ___ libev mailing list libev@lists.schmorp.de http://lists.schmorp.de/mailman/listinfo/libev
Re: libev and libevent
On Tue, Feb 20, 2018 at 10:35:01AM +0800, Andy Green <a...@warmcat.com> wrote:
> > Not really, even C supports opaque types.
>
> No, there is no "not really" about it.
>
> C supports opaque type pointers because the size of a pointer is known
> regardless of the details of what it points to.
To be pedantic, C supports opaque pointers because it is defined to do so,
not because pointers naturally all have the same size (they don't).
Anyhow, I don't think this is the right place for introductory programming
in C, but the solution, as already mentioned, is to move libev code into
e.g. lws-libev.c, libevent code into lws-libevent.c and so on, and then
define an abstraction to those, e.g.
struct lws_ev_watcher;
lws_ev_io_watcher *
lws_ev_io_new (...);
void
lws_ev_io_start (lws_ev_watcher *, ...);
...
lws_ev_watcher is your opaque data type, and lws_ev_new etc. are the
methods used to access them.
Then you can use those in your lws_io_watcher struct:
struct lws_io_watcher {
#ifdef LWS_WITH_LIBEV
lws_ev_io_watcher *ev_watcher;
#endif
Quite possibly, lws_ev_io_watcher could even be identical to
struct ev_io, and with some work, it doesn't even have to be a pointer, and
lws_ev_watcher might not need to be a separate data type either.
Since you are not really telling us why you think you need this, it's hard to
tell you which solution might be best.
> However you cannot compose an opaque / forward-referenced struct into
> another struct with type safety, because the size of the undefined thing is
> unknown.
Yes, you can, multiple ways.
> I must have your struct definition and that means I must have your headers.
Yes, but not in the same source file as the headers for libevent. This is a
pretty standard problem with pretty standard solutions.
You can do away with pointers with some tricks, as the libev ABI has a
fixed watcher size that you could determine at compile time and reserve
enough space in your lws_io_watcher struct. This would result in virtually
no performance loss due to the indirection.
It's not trivial, but certainly possible with commonly used programming
techniques. It probbaly wouldn't buy you anything that you couldn't beat
by embedding libev, though.
The other problem, the ABI clash, can either be fixed with a custom libev
version not compiling in the event code (easy to do), or by using e.g.
shared libraries and dlopen, which allows you to open both libevent and
libev despite symbol clashes.
Again, not trivial, but what you are trying to do is hardly typical.
Embedding libev, would enable a lot of other possible solutions as well.
A last comment on your style here - I have looked hard, but I still
couldn't find a good explanation (or any explanation) of why you need to
do it this way as opposed to separate builds or even backends for each
event library, which might be faster, more efficient, and easier to extend
than your current approach (compare e.g. how gtk+ selects backends, how
opengl/opencl work and so on).
Nothing you said explains why you need to be able to switch event
libraries at runtime, as opposed to e.g. select it at runtime by loading
the correct backend library.
To say it bluntly, you come about as very tight-lipped about your needs
and a bit arrogant, when considering that your programming skills are
comparatively limited, given you are quite troubled by even simple
abstraction problems in C...
Both Harald and I have tried to help you, and it would suit you well if you
would explain your problem a bit more and would use rational arguments
instead of shouting at people...
--
The choice of a Deliantra, the free code+content MORPG
-==- _GNU_ http://www.deliantra.net
==-- _ generation
---==---(_)__ __ __ Marc Lehmann
--==---/ / _ \/ // /\ \/ / schm...@schmorp.de
-=/_/_//_/\_,_/ /_/\_\
___
libev mailing list
libev@lists.schmorp.de
http://lists.schmorp.de/mailman/listinfo/libev
Re: libev and libevent
On Tue, Feb 20, 2018 at 12:40:38AM +0800, Andy Green <a...@warmcat.com> wrote: > You sound confident about that, but actually having implemented it, you have > not considered that the structures composing event lib state (handles, loop > objects etc) must be defined in one place with types available for all event > libs simultaneously. Not really, even C supports opaque types. -- The choice of a Deliantra, the free code+content MORPG -==- _GNU_ http://www.deliantra.net ==-- _ generation ---==---(_)__ __ ____ __ Marc Lehmann --==---/ / _ \/ // /\ \/ / schm...@schmorp.de -=/_/_//_/\_,_/ /_/\_\ ___ libev mailing list libev@lists.schmorp.de http://lists.schmorp.de/mailman/listinfo/libev
Re: libev and libevent
On Mon, Feb 19, 2018 at 08:10:49PM +0800, Andy Green <a...@warmcat.com> wrote:
> Since libevent uses the preprocessor, it gets the final say; in other words
> since it defines EV_READ etc to a different value, building some software
> that can operate with both libevent and libev results in nothing working or
> not even build completing if you include the libevent bits first.
Most software choses the event model at compile time, where this causes
no issue. If you want to support both at runtime, just put the libev code
in one source file and the libevent code in another, and this problem is
solved.
> This is basically an interoperability problem between libevent and libev, or
> libev and libevent, it would be great if there is something that can be done
> to allow the happy coexistence merry-go-round to spin once more for these
> two libraries.
It's not really a problem in practise, and certainly not an obstacle for
users of these libraries.
> for libevent, but I learned there the situation is more complex than I
> expected, with libev providing some libevent compatibility by design.
Yes, libev offers the libevent (v1) api as well.
> For downstream maintainers like me whose users wish to be able to use both,
What you described doesn't seem to be an actual problem - can you give a
more detailed explanation, maybe with an example, of what you want to do
and can't?
--
The choice of a Deliantra, the free code+content MORPG
-==- _GNU_ http://www.deliantra.net
==-- _ generation
---==---(_)__ __ __ Marc Lehmann
--==---/ / _ \/ // /\ \/ / schm...@schmorp.de
-=/_/_//_/\_,_/ /_/\_\
___
libev mailing list
libev@lists.schmorp.de
http://lists.schmorp.de/mailman/listinfo/libev
Re: ev_run tracing
On Thu, Nov 23, 2017 at 03:34:11PM +0700, SlavAg <h...@inbox.ru> wrote: > Suddenly working ev_run is ending with return 0 - like no pending watcher > exist. But this I am not waiting for it at this moment. I am sorry, but I don't understand what you mean with that at all. > Sometimes application works good and not problem with watchers. How can I > trace ev working? Want spot problem. I don't understand what you mean with tracing, either. I am afraid you need to explain your issue in much more detail. -- The choice of a Deliantra, the free code+content MORPG -==- _GNU_ http://www.deliantra.net ==-- _ generation ---==---(_)__ __ ____ __ Marc Lehmann --==---/ / _ \/ // /\ \/ / schm...@schmorp.de -=/_/_//_/\_,_/ /_/\_\ ___ libev mailing list libev@lists.schmorp.de http://lists.schmorp.de/mailman/listinfo/libev
Re: Using libev with /sys/class/gpios
On Wed, Nov 22, 2017 at 01:39:13AM +, Deri James <d...@chuzzlewit.myzen.co.uk> wrote: > The file /sys/class/gpio/gpio??/value is a pseudo file which reflect the > current I assume you are talking about linux here. In general, on linux, if things work with poll or select, they also work with libev. > The kernel > actually raises an interrupt I severely doubt the kernel raises an interrupt - either the hardware raises an interrupt or the kernel somehow notifies the application with a signal or an I/O readyness notification. > when the state changes and using POLLPRI | > POLLERR with the standard poll() routine seems to work Ok. You can't specify poll flags with libev, because libev does not normally use poll. You also cannot specify POLLERR at all, as this is an output flag and specifying it would be a bug. Libev will report POLLERR as a treadyness event. Have you tried polling with EV_READ and/or EV_WRITE? It would be a kernel bug if POLLPRI is signalled but POLLIN isn't, as urgent read data is also read data. > Is it possible to do this with the libev library? libev might be able to poll for the gpio events, but it cannot specify custom poll flags simply because libev supports (and usually uses) other mechanisms than poll. If EV_READ/WRITE doesn't work, you likely can't make it work directly with libev. You might be able to use epoll and poll the epoll fd with libev though, if you have to use libev. epoll also doesn't support POLLPRI, but it does support the (hopefully equivalent) EPOLLPRI. If the fd really just signals POLLPRI but not POLLIN with poll, then you might consider reporting this as a kernel bug. -- -==- | ==-- _ | ---==---(_)__ __ __ Marc Lehmann +-- --==---/ / _ \/ // /\ \/ / p...@goof.com |e| -=/_/_//_/\_,_/ /_/\_\ XX11-RIPE --+ The choice of a GNU generation | | ___ libev mailing list libev@lists.schmorp.de http://lists.schmorp.de/mailman/listinfo/libev
Re: Cannot build packages needing libevent with those needing libev for the same system
On Mon, Nov 13, 2017 at 11:21:48PM -0500, "J. Hart" <jfhart...@gmail.com> wrote:
> This function is defined in libevent.so, and declared for the compiler in
> event2/event.h. Both of these objects are part of the libevent package (ex
> libevent-2.0.22). The /usr/include/event.h header of the libevent package
> conflicts with that of the libev package. The libev package does not define
> the "event_base_loopbreak" function required, in this case, by Qt-5.9.2, and
> the libev package cannot be installed as is along with the libevent package.
> If the libevent/libev header conflict is not resolved, then the packages
> requiring one of these must be patched. This can be rather complicated,
> particularly for large packages like Qt. This would also have to be redone
> for each new release.
The libevent 1 emulation is optional, you could embed libev without the
event.[ch] files.
It might be possible to supply an event_base_loopbreak function, although I
don't know if this is part of the libevent 1 API (the one libev emulates) or
libevent 2.
Maybe sth. like this (for event.c) works (this is not correct, so some
testing might be required):
int event_base_loopbreal (struct event_base *base)
{
dLOOPbase;
ev_break (EV_A_ EVBREAK_ONE);
return 0;
}
However, using libevent2 for libevent functions and libev for libev
functions (by embedding only the liubev core parts) would probably be the
best solution.
--
The choice of a Deliantra, the free code+content MORPG
-==- _GNU_ http://www.deliantra.net
==-- _ generation
---==---(_)__ __ __ Marc Lehmann
--==---/ / _ \/ // /\ \/ / schm...@schmorp.de
-=/_/_//_/\_,_/ /_/\_\
___
libev mailing list
libev@lists.schmorp.de
http://lists.schmorp.de/mailman/listinfo/libev
Re: A question about Macro ev_set_cb
On Mon, Nov 06, 2017 at 11:10:45AM +0800, 元哲伟 <yuenzhe...@gmail.com> wrote: > #define ev_set_cb(ev,cb_) (ev_cb_ (ev) = (cb_), memmove(&((ev_watcher > *)(ev))->cb, _cb_ (ev), sizeof (ev_cb_ (ev. I' m curious about the > reason why a redundant memmove is needed. Hi, a (very) strict reading of the aliasing rules in ISO C makes the move non-redundant, as src and dst refer to different types. We don't know of any compiler who would miscompile it under optimisations, but the compilers we tested all optimised the memmove away, so it's better to have it. The memove also ensures that EV_CB_INVOKE works, as that might refer to either ev_watcher.cb, or the oriignal ev_subtype.cb member. -- The choice of a Deliantra, the free code+content MORPG -==- _GNU_ http://www.deliantra.net ==-- _ generation ---==---(_)__ __ __ Marc Lehmann --==---/ / _ \/ // /\ \/ / schm...@schmorp.de -=/_/_//_/\_,_/ /_/\_\ ___ libev mailing list libev@lists.schmorp.de http://lists.schmorp.de/mailman/listinfo/libev
Re: [PATCH] Autoconf updates
On Tue, Sep 05, 2017 at 08:13:27PM -0400, Libbitc <libb...@protonmail.com> wrote: > When running autogen.sh with a recent version of autoconf installed the > following warnings are given: Hi! libev was written for automake-1.9, does it work for you when you use the correct version? > The followig patch fixes these warnings: I don't get any warnings when using the right version, and the patch inexplicably seems to move files around as well, so it wouldn't be a good idea to apply it. -- The choice of a Deliantra, the free code+content MORPG -==- _GNU_ http://www.deliantra.net ==-- _ generation ---==---(_)__ __ ____ __ Marc Lehmann --==---/ / _ \/ // /\ \/ / schm...@schmorp.de -=/_/_//_/\_,_/ /_/\_\ ___ libev mailing list libev@lists.schmorp.de http://lists.schmorp.de/mailman/listinfo/libev
Re: How to use libev in C Plus Plus
Hi! On Mon, Jun 26, 2017 at 03:22:04PM +0800, "liu...@shinezone.com" <liu...@shinezone.com> wrote: > ev::dynamic_loop loop; > I got these error when I compiled this program. If you have free time please > answer me why I got these error. Thank you... While ev++.h has some loop structs, they are undocumented - you have to limit yourself to the documented API, i.e. mostly the ev::watcher types, and the normal C API (which is available after including ev++.h). -- The choice of a Deliantra, the free code+content MORPG -==- _GNU_ http://www.deliantra.net ==-- _ generation ---==---(_)__ __ ____ __ Marc Lehmann --==---/ / _ \/ // /\ \/ / schm...@schmorp.de -=/_/_//_/\_,_/ /_/\_\ ___ libev mailing list libev@lists.schmorp.de http://lists.schmorp.de/mailman/listinfo/libev
Re: Handling ptr1/ptr2 free (minor nitpicks)
On Fri, Jun 16, 2017 at 01:46:17AM -0400, Jeffrey Johnson <n3...@mac.com> wrote: > There; 2 relatively minor issues that are fundamental libeio API design > choices. > > 1) req->pri being remapped from 0 -> 4 (by subtracting EIO_PRI_MIN) and hence > changed from what was sent. That's more like a result of the implementation and not a fundamental design choice. > But (at least) indicate that the memory has been de-allocated already by > setting ptr1/ptr2 to NULL > and perhaps by toggling the flags bits, if you see bio_req handling as > “stateful”. What purpose would that have? Whether you reuse the memory for the request or not, you have to initialise the structure, no? > I’d suggest that the better low-level API would be to leave the eio_req > object inputs unchanged > to the greatest extent possible. JMHO ... Well, it's probably too late for that. -- The choice of a Deliantra, the free code+content MORPG -==- _GNU_ http://www.deliantra.net ==-- _ generation ---==---(_)__ __ __ Marc Lehmann --==---/ / _ \/ // /\ \/ / schm...@schmorp.de -=/_/_//_/\_,_/ /_/\_\ ___ libev mailing list libev@lists.schmorp.de http://lists.schmorp.de/mailman/listinfo/libev
Re: [PATCH] Fixed Android support
On Wed, May 31, 2017 at 10:18:13PM -0700, enh <e...@google.com> wrote: > >> presence of EPOLL_CLOEXEC doesn't necessarily imply the presence of > >> epoll_create1 on Android. This is because (as of NDK r15) we're using > > > > If android uses header files that don't match it's libc *on purpose* then > > android is way beyond broken. > > no, these are the Android libc header files. I don't quite understand what you are trying to say - "no" to what? The header files match (but don't match?), or it's not on purpose (then it should be fixed), or something else? From what you said the header files don't match the libc *on purpose*. If it's not on purpose then open a bug against your header files and fix them. Or fix the libc. > > configure is optional for libev, so a "proper" test wouldn't work. > > i'm not sure what you're saying here: you already use configure and > have correctly used HAVE_ tests for other functions. I am not sure what you mean with "correctly", but the point is that configure is *optional*, a user doesn't have to run it and still can expect it to work. > > This is clearly a bug in the android headers, and since it apparently only > > affects old versions, I don't think it makes sense to make things worse > > for users of higher quality platforms that have correct header files. > > no, it's the opposite way round. older versions of Android didn't have > epoll_create1. newer versions do. That's exactly what I said: the problem only affects old versions, because the headers don't match the old versions. They do match the new version(s) apparently, don't they? > constant. (the constants come from the upstream kernel uapi header But libev does not include those headers - it's the responsibility of the header files to be consistent and correct. If you don't know how, you can look at how normal GNU/Linux distributions solve this problem - they either deliver corretc header files or consider this a bug. > > I would suggest opening a bug against android so these obvious bugs get > > fixed instead of trying to push patches to the rest of the world to work > > around it. > > i manage the Android libc/NDK team :-) That's great - you need to take responsibility then and fix your project, no? > this change is a result the NDK catching up with the platform headers > after being years behind. Then this would be the perfect opportunity to fix your header files, no? In a more general note, since the reason for all these issues is google's anti-freedom stance (there is no *technical* need to provide a low-quality and incompatible libc on android), google is responsible for making it work without pushing their maintenance work on others just because others work for free. Again, it's a matter of taking responsibility - you created the mess, you need fix it. Greetings, -- The choice of a Deliantra, the free code+content MORPG -==- _GNU_ http://www.deliantra.net ==-- _ generation ---==---(_)__ __ __ Marc Lehmann --==---/ / _ \/ // /\ \/ / schm...@schmorp.de -=/_/_//_/\_,_/ /_/\_\ ___ libev mailing list libev@lists.schmorp.de http://lists.schmorp.de/mailman/listinfo/libev
Re: [PATCH] Fixed Android support
On Tue, May 30, 2017 at 05:28:07PM -0700, enh <e...@google.com> wrote: > The attached patch fixes Android NDK bug > https://github.com/android-ndk/ndk/issues/396 --- basically the > presence of EPOLL_CLOEXEC doesn't necessarily imply the presence of > epoll_create1 on Android. This is because (as of NDK r15) we're using If android uses header files that don't match it's libc *on purpose* then android is way beyond broken. > Gingerbread, you'll have all the enums and macros and structs from > Marshmallow and O. so you actually need a proper configure test for > epoll_create1. configure is optional for libev, so a "proper" test wouldn't work. This is clearly a bug in the android headers, and since it apparently only affects old versions, I don't think it makes sense to make things worse for users of higher quality platforms that have correct header files. I would suggest opening a bug against android so these obvious bugs get fixed instead of trying to push patches to the rest of the world to work around it. Unrelated, the __ANDROID__ change will be in the next release. Greetings, -- The choice of a Deliantra, the free code+content MORPG -==- _GNU_ http://www.deliantra.net ==-- _ generation ---==---(_)__ __ __ Marc Lehmann --==---/ / _ \/ // /\ \/ / schm...@schmorp.de -=/_/_//_/\_,_/ /_/\_\ ___ libev mailing list libev@lists.schmorp.de http://lists.schmorp.de/mailman/listinfo/libev
Re: Failure to extract
On Thu, May 25, 2017 at 07:30:43AM +, Vytautas Rakeviius <vytautas1...@yahoo.com> wrote: > bash-4.3$ wget http://dist.schmorp.de/libev/Attic/libev-4.24.tar.gz > --2017-05-25 10:29:35-- http://dist.schmorp.de/libev/Attic/libev-4.24.tar.gz > Resolving dist.schmorp.de (dist.schmorp.de)... 5.9.56.12 > Connecting to dist.schmorp.de (dist.schmorp.de)|5.9.56.12|:80... connected. > HTTP request sent, awaiting response... 200 OK > Length: 524264 (512K) [application/gzip] > Saving to: 'libev-4.24.tar.gz.1' > > libev-4.24.tar.gz.1 100%[==>] 511.98K --.-KB/s in 0.06s > > 2017-05-25 10:29:35 (8.01 MB/s) - 'libev-4.24.tar.gz.1' saved [524264/524264] > > bash-4.3$ tar -xvf libev-4.24.tar.gz > tar: This does not look like a tar archive > tar: Skipping to next header > tar: Exiting with failure status due to previous errors Since you didn't really write anything in your mail, it's not clear what it is about, but note that you didn't call "tar" on the file that "wget" has downloaded. -- The choice of a Deliantra, the free code+content MORPG -==- _GNU_ http://www.deliantra.net ----==-- _ generation ---==---(_)__ __ __ Marc Lehmann --==---/ / _ \/ // /\ \/ / schm...@schmorp.de -=/_/_//_/\_,_/ /_/\_\ ___ libev mailing list libev@lists.schmorp.de http://lists.schmorp.de/mailman/listinfo/libev
Re: Clashing symbol name on MacOS EV_ERROR
On Tue, Mar 28, 2017 at 06:17:15PM +0800, Alex <a...@stty.io> wrote: > It seems like EV_ERROR is predefined in /usr/include/sys/event.h as well on > MacOS. > > I did some searching and it seems like it was previously reported[1] on this > mailing list with no reply so I'd like to bump this issue up. You are actually linking to the reply, so clearly there was a reply? It also seems that that issue is not quite the same, but it's hard to tell. In any case there are two headers that clash, which is a problem that is created by, well, including those two headers. The solution is to not include both headers at the same time, or employ some workaround such as using #undef, changing the order of includes or sth. like that. All of this has to be implemented in whatever includes those files, which in this case seems not to be libev, so there is nothing that libev can do and if you want sth. to be done about it you need to report it to the right place (which might or might not be hiredis). -- The choice of a Deliantra, the free code+content MORPG -==- _GNU_ http://www.deliantra.net ==-- _ generation ---==---(_)__ __ __ Marc Lehmann --==---/ / _ \/ // /\ \/ / schm...@schmorp.de -=/_/_//_/\_,_/ /_/\_\ ___ libev mailing list libev@lists.schmorp.de http://lists.schmorp.de/mailman/listinfo/libev
Re: i386 memory fence incompatible with valgrind
On Sat, Jan 14, 2017 at 07:06:44PM +0100, Matthias Urlichs <matth...@urlichs.de> wrote: > On 14.01.2017 17:52, Marc Lehmann wrote: > > That is, sorry to say, utter bullshit. If the kernel unmaps that page > > and returns to the userland, it would crash on the next stack allocation > > as well. > No it would not, because the kernel would notice that the access in > question, unlike yours is *not* below the stack pointer, because the CPU > first decrements the stack pointer and then tries to store something > there. The kernel would therefore just extend the stack downwards. A > memory checker would do the exact same thing. Matthias - the only thing you are demonstrating here is your ability to make up things and your total lack of technical understanding. The kernel allocates pages depending on the address thats faulting, it will not disassemble the instruction and see how the address was calculated. Evidence to the contrary is welcome. > In any case, I'd assume that the main author of a tool like valgrind > knows a whole damn lot about this stuff. After what you wrote, I'd assume so as well, yes. So do I, btw. In any case, agruments of the form "my dick is longer than yours" are utterly boring and, I hate to say it again, not at all convincing. > Bullshit? not so much, IMHO. Well, opinions are great, but what counts is facts - the instruction is valid and by yll appearances works. The problem is not that the instruction somehow fails, the problem is that valgrind doesn't understand it, and the problem in valgrind has nothing to do with the fact that the access is below the stack pointer at the time. > I wanted to call bullshit, which I'm not, I'd start with your assertion > that the -1 is to be found in the kernel. Everybody is entitled to mistakes, even me. The difference is that I instantly acknowledged that my memory was faulty and dropped it, while you continue to beat a dead horse with ever more bullshit. Even more so, I didn't use this as an argument on why the code is correct (that would be a logical fallacy), as you should be able to understand - even if I am totally wrong about where the code is from, this has no bearing on its validity, again a difference to the arguments you mentioned, which directly try and fail to attack the validity of the code. I hope you can see the difference. > No, it's not your job to improve valgrind, but it's both your and the > valgrind author's job to get the two to play nicely with each other. And why would that be the case? It would help if you stopped pulling more and more things out of your ass and stayed with facts. > Given that each of you thinks that the other is mistaken (NB: for what > it's worth, Julian didn't call "utter bullshit"), it's still a whole lot > easier to replace a single -1 with zero than to teach valgrind about > possibly-or-not legal accesses below the stack. Libev is not about going the easy way and adding layers of hacks upon hacks to make things "work". None of my code is. If you want code that adds wahtever hack necessary to make things superficially work, you are at the wrong place and need to look elsewhere. In this case it's clearly valgrind that is buggy. If the maintaienr of valgrind doesn't see that or doesn't care for whatever reason, it's not my duty, as you wrongly claim, to make my code potentially worse to work around a bug in valgrind. Nor is it my duty to communicate with the valgrind mkaintaienr and point out where he is wrong. I bellieve you when you say you notified him and he doesn't want to fix it. In any case, this conversation has long lost any purpose, I said what I wanted to say, and nothing you came up with is in any way or form convincing, mostly because it's not based on facts. There is really nothing to add. Good luck, -- The choice of a Deliantra, the free code+content MORPG -==- _GNU_ http://www.deliantra.net ==-- _ generation ---==---(_)__ __ __ Marc Lehmann --==---/ / _ \/ // /\ \/ / schm...@schmorp.de -=/_/_//_/\_,_/ /_/\_\ ___ libev mailing list libev@lists.schmorp.de http://lists.schmorp.de/mailman/listinfo/libev
Re: i386 memory fence incompatible with valgrind
On Thu, Jan 12, 2017 at 10:33:50AM +0100, Matthias Urlichs
<matth...@urlichs.de> wrote:
> > Personally, I would do as I suggesxted in my first reply, namly asking
> > them to correctly recognize the memory fence instruction.
> As I thought, the author of valgrind states that what you're doing is
> not OK.
He clearly is wrong, and shockignly so. So shicking that I wonder if he
really looked at the same instruction, because he is so obviously wrong.
> The libev code is incorrect and should be fixed. It it violates the ABI.
Nothing in the ABI is violated, of course. He probably doesn't understand
what atomicity means, or doesn't understand the x86 lock prefix.
> The problem isn't that the memory is uninitialised. It is that the
> program isn't allowed to access below %esp at any time, for at least
> the following reasons:
>
> * a signal may get delivered at any time, in which case the signal
> handler's frame will overwrite the value at -1(%esp).
Yes, a signal handler could do that and everything would just work of
course.
> * since the kernel "knows" that programs may not access below %esp, it
> would be within its rights to unmap the page containing -1(%esp).
> If %esp pointed exactly to the bottom of a page then the access
> at -1(%esp) would cause an unexpected page (segmentation) fault.
That is, sorry to say, utter bullshit. If the kernel unmaps that page
and returns to the userland, it would crash on the next stack allocation
as well. While the kernel cna always crash programs (and validly so for
example on stack overflow), this case is in no way different than other
non-contested accesses.
> If you think Julian is wrong,
It's clearly that julian has no clue what he is talking about.
I'm shocked that somebody like him has any say with valgrind, he clearlky
should have more competence if he speaks for valgrind.
> please talk to him directly.
Why would I? It's not my job to educate people, and not my job to improve
valgrind. If the valgrind maintainers don't want to fix their bugs, that
is their choice, not mine.
In any case, I guess everythiong has been clared up by now - the code
works fine and is correct on actual x86, and valgrind doesn't want to
emulate x86 correctly. Bad choice, imho, but not mine.
--
The choice of a Deliantra, the free code+content MORPG
-==- _GNU_ http://www.deliantra.net
==-- _ generation
---==---(_)__ __ __ Marc Lehmann
--==---/ / _ \/ // /\ \/ / schm...@schmorp.de
-=/_/_//_/\_,_/ /_/\_\
___
libev mailing list
libev@lists.schmorp.de
http://lists.schmorp.de/mailman/listinfo/libev
Re: i386 memory fence incompatible with valgrind
On Wed, Jan 11, 2017 at 06:01:10PM +0100, Matthias Urlichs <matth...@urlichs.de> wrote: > On 11.01.2017 09:10, Marc Lehmann wrote: > > The fence doesn't use an uninitialised value. > > ? It accesses -1(%sp), which is not initialized / undefined / whatever; > it then processes that value (by ORing it into a register). The fact > that the result is then thrown away is immaterial It doesn't throw away the result, but it also doesn't use the uninitialised value for anything. > valgrind's rule is that anything that's not a copy triggers the report. Well, the documentation I quoted disagrees with that, so either this rule is mentioned elsewhere, and therefore at least the docs are wrong, or it isn't valgrinds rule, in which case its a code bug. Either way, it's up to valgrind to improve their program. Since the documentation I quoted is in line with the intent of valgrind (finding real problems rather than being a useless tool with spurious false positives), I lean towards fixing valgrinds code to make it a better tool. > So even if I can convince the valgrind people to ignore the Personally, I would do as I suggesxted in my first reply, namly asking them to correctly recognize the memory fence instruction. I am not sure why you would want them to ignore the memory access, that seems weird. > below-the-stack access itself (which is an error in every other case you > might think of) I can think of other cases where it isn't an error, so this statement is not true. > the checker would immediately raise another error > because that value is used. That's just something you made up, and obviously not true. If valgrind recognized the memory fence correctly, it would of course not raise any error (and the value still isn't used :). > I seriously doubt that they'll implement another special case for > handling this – when an inconsequential change from -1 to zero in your > code would work just as well. Do you have *any* data to substantiate that claim? I think not, so this is another unsubstantiated claim. Making up stuff and throwing it around as if it were true will not convince me of changing things to the worse. Try it with less bullshitting and more fact, and we have can have a reasonable argument. > > The code is taken from linux, so if you see 0, you are probably looking > > at a diferent version. > > You may remember that you (or somebody) took that code from Linux, but > that memory is faulty. I checked. I trust your check then - I am seriously puzzled, though, and will have to question the validity of this memory in detail. -- The choice of a Deliantra, the free code+content MORPG -==- _GNU_ http://www.deliantra.net ----==-- _ generation ---==---(_)__ __ __ Marc Lehmann --==---/ / _ \/ // /\ \/ / schm...@schmorp.de -=/_/_//_/\_,_/ /_/\_\ ___ libev mailing list libev@lists.schmorp.de http://lists.schmorp.de/mailman/listinfo/libev
Re: i386 memory fence incompatible with valgrind
On Wed, Jan 11, 2017 at 08:35:38AM +0100, Matthias Urlichs <matth...@urlichs.de> wrote: > On 11.01.2017 08:14, Marc Lehmann wrote: > > The area *is* a legitimate memory location, and I am sure valgrind agrees. > Umm, while it's legitimate to access it, the problem is that it cannot > have any defined content – an interrupt could alter the data stored > there at any time. That's totally fine, the fence doesn't use or modify the contents. > Valgrind knows that, thus it complains. Thats a non-sequitur. I know that, but I don't complain. The cpu knows it, but neither does it complain (it will just work). This is clearly a problem in valgrind - assuming the message you get is about use of uninitialised values, then its a bug per documentation: An uninitialised-value use error is reported when your program uses a value which hasn't been initialised The fence doesn't use an uninitialised value. The problem here is that valgrind doesn't properly understand the instruction, and the fix can only be done by teaching valgrind about the instruction, just as with any other instruction not understood by valgrind. In general, valgrind as a tool loses usefulness if it generates spurious errors or warnings, and there realyl isn't an excuse for valgrind to fix this. > > The instruction is a standard memory fence pattern also used by linux > > for example, > Linux uses 0(%esp). Why are you using an offset of -1? The code is taken from linux, so if you see 0, you are probably looking at a diferent version. In fact, at least linux 4.9 doesn't even use or, but add, so it seems to change over time. > An offset of zero would fix the problem. That's wrong, the problem cannot go away just because libev doesn't emit the instrzuction anymore. Since the problem is valgrind not understanding the memory fence, it can only be fixed in valgrind. Changing libev will cause valgrind's bug to not trigger, but the bug will still remain in valgrind. It clearly would be better if valgrind were fixed to recognize this instruction and not generate spurious errors. -- The choice of a Deliantra, the free code+content MORPG -==- _GNU_ http://www.deliantra.net ==-- _ generation ---==---(_)__ __ __ Marc Lehmann --==---/ / _ \/ // /\ \/ / schm...@schmorp.de -=/_/_//_/\_,_/ /_/\_\ ___ libev mailing list libev@lists.schmorp.de http://lists.schmorp.de/mailman/listinfo/libev
Re: libev version numbers are not updated in the recent release 4.23
On Tue, Dec 27, 2016 at 09:39:04PM +, Ales Teska <ales.te...@teskalabs.com> wrote: > I see that the version numbers remains 4.22 in the recent release 4.23 Scary, but true, even though the cvs-tagged version has it (and it has no other changes). In any case, thanks for pointing this out, I have issued a 4.24 release that only bumps the version, which is hopefully the least-confusing fix. -- The choice of a Deliantra, the free code+content MORPG -==- _GNU_ http://www.deliantra.net ==-- _ generation ---==---(_)__ __ __ Marc Lehmann --==---/ / _ \/ // /\ \/ / schm...@schmorp.de -=/_/_//_/\_,_/ /_/\_\ ___ libev mailing list libev@lists.schmorp.de http://lists.schmorp.de/mailman/listinfo/libev
Re: ./confiugre options
On Tue, May 24, 2016 at 07:09:56AM +0200, Knut Olav Bøhmer <boh...@gmail.com> wrote: > Hi, > > I'm debugging some code that works on mac but not on linux > so I would like to compile libev without different features, like eventfd, > signalfd, epoll etc. to see if any of this makes a difference. I assume you mean OS X when you say macs? Since OSX doesn't support epoll, signalfd or eventfd, configure will automatically disable them. > Are there any options to the configure script to do that? Not options, but configure respects the usual environment variables (CPPFLAGS, CFLAGS and so on) that can be used to enable/disable options, as described in the docs, as long as there are no conflicts, of course. -- The choice of a Deliantra, the free code+content MORPG -==- _GNU_ http://www.deliantra.net ==-- _ generation ---==---(_)__ __ ____ __ Marc Lehmann --==---/ / _ \/ // /\ \/ / schm...@schmorp.de -=/_/_//_/\_,_/ /_/\_\ ___ libev mailing list libev@lists.schmorp.de http://lists.schmorp.de/mailman/listinfo/libev
Re: ev_async race condition??
On Fri, May 20, 2016 at 02:01:00PM -0500, Jose Madrigal <mac.yo...@gmail.com> wrote: > Hi!! I'm using libev in a multithreading project and I'm debugging with > ThreadSanitizer from clang. The ThreadSanitizer says there is a data race > in libev, but i'm not sure if it is a false positive. There aren't any known data races in libev. Maybe there is one, but clang tends to be comparatively buggy (clang historically diagnosed all kinds of bugs in libev when there weren't any), so at the moment I tend to assume that it's a false positive. > I'm wake up a event loop using ev_async because belongs to another thread. > This is not supposed cause races. Correct. > And then when I take a look on those files the variable that apparently > races is "pipe_write_skipped" but in the code looks like is a > "sig_atomic_t" type that should not be causing races. sig_atomic_t unfortunately doesn't make any guarantees w.r.t. threads, we use it only because there's not better portable type available and it happens to work correctly everywhere we know. > Is there any chances that this race condition is real or is just a false > positive?? Unless you compiled out synchronisation primitives with ECB_NO_THREADS or EV_NO_SMP it's probably a false positive till proven otherwise :) > Sorry if I was not clear enough or I misspoke, english is not my strong The interesting question is: did you run into any trouble, or is this just about some clang warning? If the latter, then most likely you can ignore it. Greetings, -- The choice of a Deliantra, the free code+content MORPG -==- _GNU_ http://www.deliantra.net ==-- _ generation ---==---(_)__ __ __ Marc Lehmann --==---/ / _ \/ // /\ \/ / schm...@schmorp.de -=/_/_//_/\_,_/ /_/\_\ ___ libev mailing list libev@lists.schmorp.de http://lists.schmorp.de/mailman/listinfo/libev
Re: libev featured fifo
On Wed, May 18, 2016 at 02:29:14AM +0200, Robert Eisele <rob...@xarg.org> wrote:
> directly accepting 0 byte reads after the first byte was read. Is this a
> problem with libev or am I missing something?
That's likely a problem with your code forgetting to close the fifo?
> epoll_wait(3, {{EPOLLHUP, {u32=5, u64=4294967301}}}, 64, 59743) = 1
> epoll_ctl(3, EPOLL_CTL_MOD, 5, {EPOLLIN, {u32=5, u64=4294967301}}) = 0
> clock_gettime(CLOCK_MONOTONIC, {43720, 29238758}) = 0
> read(5, "", 130)= 0
>
> I'd like to put the process asleep again when everything was read. Is
> there a way to accomplish that?
Unregister the I/O watcher, close the socket.
--
The choice of a Deliantra, the free code+content MORPG
-==- _GNU_ http://www.deliantra.net
==-- _ generation
---==---(_)__ __ __ Marc Lehmann
--==---/ / _ \/ // /\ \/ / schm...@schmorp.de
-=/_/_//_/\_,_/ /_/\_\
___
libev mailing list
libev@lists.schmorp.de
http://lists.schmorp.de/mailman/listinfo/libev
Re: Using libev with Windows sockets
On Wed, May 11, 2016 at 10:22:10PM +, Eric Fowler <eric.fow...@centritechnology.com> wrote: > I have inherited a pretty large Linux-based server project and am tasked with > porting it to Windows. It is heavily dependent upon libev and libeio. Both libeio and libev work fine in a win32 environment (well, in some envgironments, there are so many incompatible ones - I would try mingw, where it is known to work), as well as in cygwin, which would be a an eaiser choice to port uinix software to windows. (And Microsoft is officially workign on linux emulation already, might want to go for that once it's stable, who knows :). > (1)- I have noticed that libev uses socket descriptors as array indexes > in several places. This is fine on Linux because the socket descriptors are > always small (typical values range from 4 to 8 for small numbers of sockets). > However, Windows socket handles have integer values starting at nearly 200, > with the high value undefined. This would seem to make them useless as array > indices. Am I justified in my concern or is there something about Winsock or > libev that I am not seeing? You have to convert the socket handles to fd's. as described in the manual (have you read the WIN32 PLATFORM... chapter?). How you do that is up to you, most commonly (and supported out of the box) is using the Microsoft libc. > (2)- Our code is getting a Windows socket descriptor and passing it down > into libev, where it comes a-cropper on an EV_FD_TO_WIN32_HANDLE() macro in > ev.c!fd_reify() [line 2084]. I am not surprised that the macro choked; it is > trying to convert a windows socket descriptor to a windows socket descriptor, > which should not be expected to work. The bigger question is, 'What am I > doing wrong?' Should I not expect the libev code to *ever* see a windows > socket handle? There must be some design strategy WRT windows handles here, > but I am not seeing it. The code is within an #ifdef block that expects > Windows ... does this mea You need to use a file descriptor, just as on a POSIX system. > it does not expect to see windows handles? Is there a simple way to fish a > BSD style socket descriptor out of a windows socket handle? That depends on your environment - if you use the microsoft c library, the easy way would be open_osfhandle. You really need to read the ev manual on win32 though, to understand how to work around the usual windows limitations. -- The choice of a Deliantra, the free code+content MORPG -==- _GNU_ http://www.deliantra.net ==-- _ generation ---==---(_)__ __ __ Marc Lehmann --==---/ / _ \/ // /\ \/ / schm...@schmorp.de -=/_/_//_/\_,_/ /_/\_\ ___ libev mailing list libev@lists.schmorp.de http://lists.schmorp.de/mailman/listinfo/libev
Re: Error handling / ev_set_syserr_cb
On Thu, Feb 25, 2016 at 06:44:19PM +0300, Nick Zavaritsky <mej...@gmail.com> wrote: > >> 1) UB from the libev point of view, or > >> 2) works with certain versions of libev on certain OSes, but may break > >> without warning, or > >> 3) is fully supported and is a part of the public API contract. > > > > It's certainly 1 or 2. > > I am glad it was stated in clear now. After reading the rest of your mail, I frankly am not sure I appreciate your tone. > Remember a discussion a while ago when a patch was submitted to work around a > situation when libev runs out of file descriptors? Yup. > > Back then you suggested long jump, but today you admit it is unreliable. Not then and not now do I "admit" it is unreliable. Why do make this up? Please don't put words in my mouth that I never said, I don't appreciate that. At all. > This is a reasonable programming model though not universal. You surely won’t > assume that an embedded system has plenty of RAM. It is also possible to turn > overcommit off. It's clearly not libev's programming model, so I don't quite see why you bring it up. > The case with the descriptors running out falls somewhere in-between. Yes. To quote yourself, with careful programming it is possible to avoid this kind of errors. However, while it might be possible, it's certainly very hard, and IMnsHO (as backed up by ample real world evidence), relying on this is doomed. > If I was to consider adding resiliency to fd shortage in a library like libev > I would have asked the following questions: libev is already resilient to fd shortage - you probably work on very specific definitions of resilient. > B) Not quite. Controlling fd usage manually is hard due to many existing > libraries consuming descriptors internally. Besides, it is complex and feels > awkward. As is controlling memory and stack usage. I'm not seeing what your pointis? > C) It depends. But generally, yes. Especially, in a network server, that is > already prepared to handle connection errors. Sure, that's why libev allows applications to deal with this. > > Just saying... all this sounds like some inane customer feature request > > form a customer who doesn't know what he is doing and wants to go headlong > > through the nearest wall. > > May be I am missing something and a peace of advice will be greatly > appreciated. Well, the above is just my feeling of how this sounds like - it appears you want to go through a wall by forcing a solution that really isn't easier than much better solutions that are already available. Adding some kind of error returns to basically every function in libev, and wrapping every call done at any time does not sound like a reasonable plan for libev users,a nd doesn't seem like a reasonable plan to improve anything. It very much sounds like "we decided to do it this way, no matter what, the wall in front of me be damned". Wrapping ev_async isn't enough - libev can run out of fd's or memory and potentially other rersources at basically any time. If you want a good plan to deal with these issues, then dealing with these issues by e.g. avoiding the condition (by not using too many fds) or reporting the misconfiguration and relying on a service manager sounds a lot more reasonable. Even if libev had some interface thast would suit you, you have the same issues with a host of kernel interfaces, such as memory allocation or linux's non-posix accept behaviour, that you have to deal with and that are similar. I am doubtful that this would be a real-world requirement, or a real-world improvement to libev or its users. > We are spawning a new thread with a dedicated event loop. Sometimes it > fails due to fd shortage when we setup ev_async used for communication > with the thread. We would like to shutdown the thread in a clean way > and to deliver an error into the procedure waiting for the thread’s > completion. The procedure is fully prepared to handle the error (maybe > it attempts to restart processing sans the dedicated thread, or it just > throws the hands into the air.) If it is fully prepared to handle the error, then why not rely on that? I am also not sure how "throwing hand sin the air" is considered "fully prepared to handle the error"? -- The choice of a Deliantra, the free code+content MORPG -==- _GNU_ http://www.deliantra.net ==-- _ generation ---==---(_)__ __ __ Marc Lehmann --==---/ / _ \/ // /\ \/ / schm...@schmorp.de -=/_/_//_/\_,_/ /_/\_\ ___ libev mailing list libev@lists.schmorp.de http://lists.schmorp.de/mailman/listinfo/libev
Re: ABI report
On Fri, Feb 26, 2016 at 03:27:24PM +0300, Ponomarenko Andrey <andrewponomare...@yandex.ru> wrote: > Hello, > > I've prepared ABI changes/backward binary compatibility report for the recent > versions of the libev: http://abi-laboratory.pro/tracker/timeline/libev/ Cool thing, and makes me quite happy to see so many 100%s. Some things look fishy, though: http://abi-laboratory.pro/tracker/compat_report/libev/3.9/3.31/1b3e7/abi_compat_report.html#Removed That lists 12 removed symbols, but I a m not aware of them ever being removed. And especially not in 3.31, which only had a tiny patch compared to 3.3. I think the title shows the problem: Binary compatibility report for the libev.so object between 3.9 and 3.31 versions on x86_64 Maybe the report compares a newer version (3.9) with an older version (3.31), and thus concludes that the old version is not ABI compatible with the newer one? That would explain this. The libev version numbers are actual _numbers_, so 3.9 > 3.31, not version strings, which might explain the issue? -- The choice of a Deliantra, the free code+content MORPG -==- _GNU_ http://www.deliantra.net ==-- _ generation ---==---(_)__ __ __ Marc Lehmann --==---/ / _ \/ // /\ \/ / schm...@schmorp.de -=/_/_//_/\_,_/ /_/\_\ ___ libev mailing list libev@lists.schmorp.de http://lists.schmorp.de/mailman/listinfo/libev
Re: Error handling / ev_set_syserr_cb
On Thu, Feb 25, 2016 at 12:47:15PM +0300, Nick Zavaritsky <mej...@gmail.com> wrote: > 1) UB from the libev point of view, or > 2) works with certain versions of libev on certain OSes, but may break > without warning, or > 3) is fully supported and is a part of the public API contract. It's certainly 1 or 2. Even if it were supported under some circumstances, I'd say whatever thing you are trying to do is illdesigned - you'd have to wrap every call, and libev is not the only source of errors - if libev runs out of memory for example, then the program might crash at any time (e.g. because of kernel OOM, or because it needs more stack space or...). If you want to catch errors and do something sensible, the libev errorhandler is not your solution. In fact, no in-process solution exists - you should have a watchdog that does the right thing in fatal situations such as this. Just saying... all this sounds like some inane customer feature request form a customer who doesn't know what he is doing and wants to go headlong through the nearest wall. > > You won't get leaks, but you will likely have a non-working loop > > afterwards - nothing legal can be done with it anymore. > > Can we legally kill it with ev_loop_destroy()? Not legally, and it probably will not work in practise either. -- The choice of a Deliantra, the free code+content MORPG -==- _GNU_ http://www.deliantra.net ==-- _ generation ---==---(_)__ __ __ Marc Lehmann --==---/ / _ \/ // /\ \/ / schm...@schmorp.de -=/_/_//_/\_,_/ /_/\_\ ___ libev mailing list libev@lists.schmorp.de http://lists.schmorp.de/mailman/listinfo/libev
Re: unreliable timer with RUN_ONCE since 4.10
On Mon, Feb 22, 2016 at 04:13:07PM +0200, Alex Efros <power...@powerman.name> wrote: > Okay, that makes sense, thanks. > > If it works this way (because of internal libev watchers), then I see > possible use cases for EV::run(0) and EV::run(RUN_NOWAIT), but can't > imagine use cases for RUN_ONCE. Are there some examples for RUN_ONCE uses? The typical RUN_ONCE use case is to implement your own ev_run-like function with extra functionality. For example, ev_break-like functionality could be implemented via RUN_ONCE. -- The choice of a Deliantra, the free code+content MORPG -==- _GNU_ http://www.deliantra.net ==-- _ generation ---==---(_)__ __ __ Marc Lehmann --==---/ / _ \/ // /\ \/ / schm...@schmorp.de -=/_/_//_/\_,_/ /_/\_\ ___ libev mailing list libev@lists.schmorp.de http://lists.schmorp.de/mailman/listinfo/libev
Re: question on ev_loop_fork
On Thu, Feb 25, 2016 at 09:50:20AM +0800, adream <adream...@gmail.com> wrote: > It seems that ev_loop_fork didn't work as expected. > Am I misunderstand the document? Possibly - how _are_ you understanding it? :) > So, can any guys shows me an example which would have different > behavior if I call the ev_loop_fork or not? If you want to use a loop before and after a fork, you need to tell libev about it (or make it check for it). Not doing so is a bug, and finding out how things then break (or not) is academic. -- The choice of a Deliantra, the free code+content MORPG -==- _GNU_ http://www.deliantra.net ==-- _ generation ---==---(_)__ __ ____ __ Marc Lehmann --==---/ / _ \/ // /\ \/ / schm...@schmorp.de -=/_/_//_/\_,_/ /_/\_\ ___ libev mailing list libev@lists.schmorp.de http://lists.schmorp.de/mailman/listinfo/libev
Re: question on ev_loop_fork
On Wed, Feb 24, 2016 at 09:22:15PM +0800, adream <adream...@gmail.com> wrote:
> hi
> I thought, if I didn't call the ev_loop_fork in the child, then the
> watcher in child wouldn't be triggered.
ev_loop_fork (or EVFLAG_FORKCHECK) are required for the event loop to
survive a fork. they don't do anything else.
not using them and forking *might* work in some circumstances, but *will*
fail in others.
so the choice between calling ev_loop_fork and not is "make the loop work"
vs. "undefined behaviour". the latter is a libev usage bug.
--
The choice of a Deliantra, the free code+content MORPG
-==- _GNU_ http://www.deliantra.net
==-- _ generation
---==---(_)__ __ __ Marc Lehmann
--==---/ / _ \/ // /\ \/ / schm...@schmorp.de
-=/_/_//_/\_,_/ /_/\_\
___
libev mailing list
libev@lists.schmorp.de
http://lists.schmorp.de/mailman/listinfo/libev
Re: unreliable timer with RUN_ONCE since 4.10
On Mon, Feb 15, 2016 at 02:02:34PM +0200, Alex Efros <power...@powerman.name> wrote: > On Mon, Feb 15, 2016 at 12:25:19PM +0100, Marc Lehmann wrote: > > If I run your script, then the reason the timer isn't invoked instantly > > is simply because it hasn't elapsed yet - try with delay -1 for example, > > then it will be invoked instantly. For positive delays, it will only be > > triggered when it's time has elapsed. > > But there are no other event watchers, so if this timer hasn't elapsed yet > when event loop returns then isn't that means event loop returns too early? How do you know there are no other (for example internal) event watchers? > Isn't event loop must wait for at least one event if there are active > watchers when using EV::run(0) and EV::run(RUN_ONCE)? And as far as I can see, it does that. > If current behaviour is correct - is it possible to somehow find out is > RUN_ONCE did something (invoked callback for at least one watcher) or not? RUN_ONCE always does something, it would block otherwise. If you want to find out whether specific events (such as your timer) have been invoked, that's what the callback for your event is for. So, if you are interested in timer events, when your timer callback is invoked, then it elapsed, otherwise, something else happened. > My use case is testing event loop based modules/applications by using > RUN_ONCE to process and test events one-by-one… and when RUN_ONCE is > (sometimes) doing nothing RUN_ONCE always does something - if you wait for specific events to happen when you test them, you can set a flag in the callback you are waiting for for example, or don't use RUN_ONCE and use ev_break instead. > repeat RUN_ONCE, or don't use RUN_ONCE at all and add EV::break to all > event handlers to emulate it (which isn't ease or even possible for event > handlers defined outside of test's code). Well, libev cannot really know which events you are waiting for and instead returns after any single event. Note that using RUN_ONCE to wait for specific events likely means you are doing it wrong, so it's best to use an event-based approach, using callbacks. You shouldn't test for undocumented behaviour or internals in libev, that will break sooner or later. -- The choice of a Deliantra, the free code+content MORPG -==- _GNU_ http://www.deliantra.net ==-- _ generation ---==---(_)__ __ __ Marc Lehmann --==---/ / _ \/ // /\ \/ / schm...@schmorp.de -=/_/_//_/\_,_/ /_/\_\ ___ libev mailing list libev@lists.schmorp.de http://lists.schmorp.de/mailman/listinfo/libev
Re: unreliable timer with RUN_ONCE since 4.10
On Sun, Feb 14, 2016 at 06:15:39PM +0200, Alex Efros <power...@powerman.name> wrote: > The example is in Perl, but I suppose issue is with libev, not EV. > EV-4.03 doesn't have this issue, EV-4.10 to 4.22 have this issue. > It may be related to improvement for low resolution clock mentioned in > this list for 4.10 (May 2011). > > Problem is: timer for < 1sec doesn't invoked by first RUN_ONCE. I'm not quite sure what the problem is - RUN_ONCE doesn't wait for specific events, it simply runs the event loop once, and it seems to work as it should in your example. If I run your script, then the reason the timer isn't invoked instantly is simply because it hasn't elapsed yet - try with delay -1 for example, then it will be invoked instantly. For positive delays, it will only be triggered when it's time has elapsed. > With 0.9 sec versions 4.10-4.22 works unreliably: sometimes timer invoked > by first RUN_ONCE, sometimes not - so I've used 0.1 and 1 sec in this > example. I am not sure why you think it is unreliable - 0.9 means "after 0.9 seconds", not before. -- The choice of a Deliantra, the free code+content MORPG -==- _GNU_ http://www.deliantra.net ==-- _ generation ---==---(_)__ __ __ Marc Lehmann --==---/ / _ \/ // /\ \/ / schm...@schmorp.de -=/_/_//_/\_,_/ /_/\_\ ___ libev mailing list libev@lists.schmorp.de http://lists.schmorp.de/mailman/listinfo/libev
Re: libev error handling
On Thu, Feb 04, 2016 at 12:52:36PM +0300, Georgy Kirichenko <geo...@tarantool.org> wrote: > > That's clearly not true, the most obvious thing (but not the only thing) > > the callback could do is create some free fds, either by increasing the fd > > limit, or by freeing some fds. > I think this can't fix problem, The point I made was that your original claim was untrue, which I have shown by giving counterexamples. It's not an exhaustive list. Other options would be setjmp/longjmp or using another thread and canceling it - the latter has the advantage of actually catching all such cases, not just a single case. > because unprivileged user can't increase fdlimit as he want, That's an administrative issue. > and in case of "emergency descriptors" it isn't guaranteed, that freed > descriptors won't be intercepted by another threads. In my case solution > shouldn't have any impact on another threads. "Guaranteed" is a red herring - nothing is guaranteed when you run out of vital resources (e.g. memory). You have to draw a line somewhere, and libev draws it where it currently draws it. It's basically the same mechanism that libx11 uses, which has withstood the test of time. I'm not convinced that singling out _one_ case of fd exhaustion (or in general, resource exhaustion) is worth a special case (it doesn't handle epoll fd creation or event pipe re-creation), given plenty of alternatives you can implement already. > I need just return an error, that can be handled by application, in case if What when there is nothing to return to? The reason for the callback is to be able to handle exactly these cases. > something went wrong by libev initialization, avoiding any impact on another > parts of my application. libev currently doesn't allocate its pipe on init, but on various other events. catching one of those is not fixing anything, it's merely pretending. > > file descriptors are a limited resource, like memory - you can't fix out > > of memory conditions by checking at initialisation time and hoping that > > dynamic allocations later will always succeed. > Main problem is that ev_async_start tries to allocate limited resource, but > can't return any error to app. Right, and it's not the only part of libev that does that, so your patch doesn't fix the problem at all (and I don't see how a similar approach could). -- The choice of a Deliantra, the free code+content MORPG -==- _GNU_ http://www.deliantra.net ==-- _ generation ---==---(_)__ __ __ Marc Lehmann --==---/ / _ \/ // /\ \/ / schm...@schmorp.de -=/_/_//_/\_,_/ /_/\_\ ___ libev mailing list libev@lists.schmorp.de http://lists.schmorp.de/mailman/listinfo/libev
Re: sharing udp socket between several processes
On Tue, Dec 29, 2015 at 12:20:43PM -0800, Kirill Timofeev <kirill.timof...@hulu.com> wrote: > but I see that some incoming packets are lost with 2 childs: All I can see is that 8476 packets apparently have been received, not hat they have been lost - you don't seem to do any error checking on send, and depending on how fast ruby does it's thing, it could easily make the kernel run out of packet buffers. so first make sure you don't get ENOBUFS or another error. I haven't looked at your progrma in detail, but since running two processes is much less efficient than one, I would first check that you really did send all those packets. I also don't know of anything that could go wrong specifically with multiple processes, apart from the myriad issues around udp receiving in general. -- The choice of a Deliantra, the free code+content MORPG -==- _GNU_ http://www.deliantra.net ==-- _ generation ---==---(_)__ __ __ Marc Lehmann --==---/ / _ \/ // /\ \/ / schm...@schmorp.de -=/_/_//_/\_,_/ /_/\_\ ___ libev mailing list libev@lists.schmorp.de http://lists.schmorp.de/mailman/listinfo/libev
Re: Several questions concerning libeio internals(+)
On Wed, Dec 16, 2015 at 07:06:18PM +0300, Nick Zavaritsky <mej...@gmail.com> wrote: > Consider eio_open (const char *path, int flags, mode_t mode, int pri, eio_cb > cb, void *data). > > Assume thread A calls eio_open, I want the completion callback to be invoked > in the same thread. You need some communications mechanism for your threads - that's outside the scope of libeio, really. > However since there’s the single result queue, it’s impossible to route the > completed request to the particular thread. If it's impossible to route completed requests to the particular thread that wants the result, then nothing in libeio can fix that, since it is impossible. Since it is, in general, possible to route requests (or any data structure) to specific threads, I think that statement is wrong. What that mechanism is varies widely between implementations, so I am not sure libeio should force a specific one. -- The choice of a Deliantra, the free code+content MORPG -==- _GNU_ http://www.deliantra.net ==-- _ generation ---==---(_)__ __ __ Marc Lehmann --==---/ / _ \/ // /\ \/ / schm...@schmorp.de -=/_/_//_/\_,_/ /_/\_\ ___ libev mailing list libev@lists.schmorp.de http://lists.schmorp.de/mailman/listinfo/libev
