There was a small discussion after I voiced my opinion about the ecore event system. As can be expected, some one suggested I submit a patch. This is not a patch, but a statement of the problem and two proposed solutions. I'll write either solution.
My comments where prompted by someone suggesting that ecore should handle events "very well". My objection to the current state of event handling is that it is not done "very well", it is merely done "well". I have been dealing with event systems, at all levels, since the early 1980's. I have seen the entire spectrum, from really very good, to downright evil. I think I have a good handle on how to do event systems "very well". I should state right now that while I have dealt with the ecore event system during my work in ecore_exe, I have not had much more exposure than that. On the other hand, the executive summary of this email is that ecore events are probably fine for most of what they are used for, but will not scale well for fork'n'pipe. Raster suggested at the beginning that I use ecore events for fork'n'pipe. He knows the entire system much better than me, so I bowed to his wisdom. Solution two is to just not use ecore events for fork'n'pipe. I think I can do better than that with solution one though. One good metric for event systems is how many useless, ignored events do they send. A good event system will also have as little code between the event generator and the event consumer as possible. These two can be a bit of a trade off if things are not done well. Sending less useless events might involve having more code to decide what events to send where. It doesn't have to though. Typical event systems are part of systems that have some object oriented aspects. Not surprising as events are usually generated by one object to be consumed by one or more other objects. The ideal is if each generating object already knows exactly which consuming objects to send to, and just does straight away. Nothing useless is sent, and the sending object just iterates through a list of objects to send to, meaning minimal code gets in the way. We don't live in an ideal world however. The sending object does not know any internal state of the consuming objects, and thus cannot always make ignore/consume decisions on their behalf. So some useless events are always gonna happen. Useless events waste CPU time, and are to be minimised even if we can't avoid them altogether. The trick is to come up with the minimal code that will not waste too much CPU time. The registered callbacks system used in ecore is a good system that can make a decent trade off if the granularity is done right. My basic problem with ecore is that I think the granularity could be better. It's fine, although not ideal for user input level events, but won't scale well for heavy IPC level events. For user input events, we are talking low bandwidth, small amounts of data stuff. Keystrokes at 100 words a minute, mouse clicks, even mouse moves are not events that happen quickly, or require much data to describe. While quick responses are usually required for UI stuff, we are talking human quick, not computer quick. So these sort of events CAN take their time getting sent. On the other hand, it is often hard for user input event generating objects to know exactly which objects to send to. A keystroke often has to filter through many layers of objects, with contextual decisions being made in each until some object consumes it. User input events are the low end of the spectrum, but usually quite important, and event systems are often designed around them. My limited exposure to ecore events gives we the impression that they are up to the task. At the other end of the spectrum is fork'n'pipe, with possibly large amounts of data being sent as quickly as possible from one object to another. These sorts of events often CAN be dealt with using minimal code and minimal useless events, because when you start an exe to read from it, there is no one else interested in the read data. The sending object only needs to send to one consuming object, and that consuming object is known ahead of time, it's the object that started the exe. This is not a good match to the current ecore event system, as the ecore event system will send read data events to all objects that requested read data events, from all exes. It is up to the consuming objects to decide to ignore any particular read event. With lots of exes being read from, some of them sending possibly lots of data, this will not scale well. To make matters worse, there is line buffering, which will send lots of read data events one per line, rather than in one big blob. This could mean lots of small read data events being sent, sometimes lots of them very quickly. This will scale very poorly. Especially as they all have to be ignored by all but one of the read data consuming objects. So it only takes one line buffered exe with lots of short lines sent quickly to bog down the entire system. Solution one is for the ecore event system to do some checking of generating objects on behalf of the consuming object. Add a pointer to the callback structure that defaults to NULL. When an object registers a callback, it can optionally supply that pointer. When an event occurs, the system that creates the event can optionally provide a pointer as well, which also defaults to NULL. When the event system gets around to dispatching that event to that callback, it compares the two pointers, and only calls the callback if they match. Since both pointers default to NULL, we don't have to change any existing code to cope since NULL == NULL and the callback gets called anyway. Any callback that has to decide whether or not to ignore or consume an event has to do some sort of test. In the case of the exe read data event, most likely this is a comparison of the pointer to the exe returned when the code created the exe against the exe pointer in the read data event structure passed to the callback. This also means that the code has to conspire to pass the exe pointer to the callback somehow. This will be as the callbacks data pointer, as part of a structure that is passed via that data pointer, or in a global pointer/structure. In this instance we are exchanging a function call and return plus a pointer comparison with just the pointer comparison. This is a speed gain during the processing of multiple callbacks at a time when speed might be very important. At the other end of the spectrum, for user input events that don't make use of this extra pointer, we are adding a pointer comparison. This is a speed loss at a time when speed is probably not important at all. Each event type must document what it uses as a pointer so that code registering callbacks knows what to use if anything. As stated earlier, solution two is to not use the ecore event system for exe read data. As I am writing the exe data reading code (fork'n'pipe) I will want what I just described as the solution, I'll probably just copy ecore events and modify it as stated. Someone is likely to propose solution zero, ignore all this waffle and just use unmodified ecore events for fork'n'pipe. If we go with this solution, the time will come when some idiot will pass all the GNU source code through several line buffered fork'n'pipes and complain when his system freezes. At that point, I'll point out that I had predicted this exact problem, and point to solution one.
pgpJ8lnW6V4W8.pgp
Description: PGP signature
