On 05/08/16 20:11, Gustavo Sverzut Barbieri wrote:
> On Fri, Aug 5, 2016 at 11:30 AM, Tom Hacohen <t...@osg.samsung.com> wrote:
>> On 04/08/16 19:45, Gustavo Sverzut Barbieri wrote:
> [...]
>>> The blob/binbuf are important optimizations if we care about memory
>>> consumption and reducing copies, but they are not mandatory. I'll
>>> insist a bit more on them, but if it's the overall agreement to go
>>> with buffer copies, then Okay.
>>
>> I don't see why/how, but maybe I'm missing something, so let me tell you
>> what I have in mind, and please correct me if I'm missing anything.
>>
>> In any networking code you will have:
>> char buf[MAX_BUF];
>> read(..., buf, ...);
>> eo_callback_call(obj, ..., buf, size_of_buf);
>>
>> Or something similar. Binbuf just means you'll also add it to a binbuf
>> and then pass the binbuf instead of buf. So in the most basic case, that
>> is one reader of the data (is there any other case??) binbuf is at most
>> as good as the example above, and if someone really wants to keep it, he
>> just adds it to the binbuf on his own. I don't see how binbuf helps.
>
> - most of time you don read to a stack buffer.
As I said, not enough experience here, if you say that's the case, OK.
I'd assume some sort of recycle buffer, be it a stack or a recycled
allocated one.
>
> - if there is a protocol where you need a full message to process
> (ie: full line up to "\n"), or N bytes specified at the first 4 bytes,
> with raw buffer (no storage on the socket side), you can't say "I'm
> not taking it, I need more data", you'd need to implement that logic
> at your side, or provide something like Qt's "peek()", but that just
> work for them because there you need to call "read()" yourself (or
> peek()), while in ecore-con like API, we read and deliver to users, so
> fd-handler stops triggering read events.
The question here is who is our target audience with this API. I don't
think we are going to implement a high performance HTTP server with this
API. I'd rather optimise for convenience rather than performance.
Yes, that's what I meant. I think that provides better flexibility. You
just get a callback with "data ready" and then you can decide what to do
with it. Similar to "select". You just have loop integration. This is
all I expect out of ecore-con. Maybe with additional helper functions
that automatically read into a binbuf for example.
>
> - binbuf provides "remove()" function, then you can easily look into
> the buffer up to length (ie: memchr()), if there is no "\n" you simply
> keep the binbuf unaltered, that means data was not consumed. If you do
> use data, you'd remove it from the binbuf.
> ** NOTE: ** this is a problem for multiple event handlers, just
> one would consume the data, others would not! (Just realized this now)
>
Yes, that's what I said before about multiple callbacks... We need to
define, maybe that's not an allowed behaviour. That is also fine. I'm OK
with adding restrictions like this (one callback), but they need to be
defined.
> Alternatively we can change the behavior to not read() ->
> callback_call(), just callback_call() and have the user to call a
> read() or peek() method. Just be aware that when we read(), the other
> event handlers would also have no further data to read, similar
> problem as the "NOTE" above.
>
> What do you think?
That's what I suggested, so yes, I'm happy with this. This is exactly
what I had in mind. It provides better flexibility for high performance,
while also keeping it simple. Only thing is, you need to decide to limit
to one callback (easy to enforce in code).
>
>
>>> How to do that? Just use constructor { .property_name; } ?
>>
>> Look at win_type_set in elm.win. The constructor {} directive means a
>> function should be used during construction (not necessarily only in
>> construction though), and there is another way to say a property is only
>> allowed in the constructor. This is done in the C implementation, not
>> the .eo declaration though.
>
> by checking the eo_finalized_get(obj)? I found that pattern in
> efl_network stuff, using that in my code.
Correct. Thanks.
>
>
>>>> A comment I forgot to make, which is to something cedric mentioned, and
>>>> is something that I've said a million times already and I'll say it
>>>> again because it's so damn important and people should stop doing it.
>>>> Please do not delete objects implicitly. :( Connection is closed? Don't
>>>> delete the object, I may steal be dealing with it. Let me eo_unref/del
>>>> it if I'm still the rightful owner of the object, it's my
>>>> responsibility. Maybe I still want to introspect it before it's gone?
>>>> It's my decision to make... If you really think this pattern of
>>>> autodeletion is useful, add a helper function people can add as a
>>>> callback that just eo_dels an object. It's that simple.
>>>
>>> Not really, I'm trying to use what is the standard and likely i got it
>>> wrongly. I'm okay to add explicit events and methods to close the
>>> socket, if that's what I should be doing :-)
>>>
>>> I just wanted to use socket.close() if people were expecting to do
>>> socket.del().
>>>
>>> Note that this shouldn't mess with refcounts, it's more like
>>> evas_object_del() where you want the object to disappear immediately.
>>> In this case the socket is not available anymore, so people should
>>> stop using that after receiving the event.
>>
>> Yeah, don't mess with refcounts. Don't delete the object when the
>> connection dies.
>
> Added a close() and closed event. Thanks for the clarification.
>
>
>>>> As for the clarification, I was asked to explain what I meant by
>>>> "disagree with the concept". What I meant by that, is a "fatter" wrapper
>>>> than thin. I just want a very thin wrapper, and I disagree with anything
>>>> that's fatter. :)
>>>
>>> As per above, I'm trying to have a very thin layer. The blob/binbuf
>>> are ways to efficiently use memory in C.
>>
>> Please explain then, because I don't get it.
>
> The read/consumer part was listed above. As for the producer part,
> it's easier to understand:
>
> Most languages, except C, you get reference counted objects that can
> provide a buffer interface so you can read the bytes. Then you trade
> to keep a list of these objects, use them and then unreference, with
> duplicate all their memory in order to wait to dispatch, to then
> realloc() as the buffer is consumed until everything is gone and you
> free().
>
> Then with bindings, JS, Python and all we could play very nice with
> memory by having a way to keep their reference for a while.
>
> If the resource is a backing file we could avoid malloc() for the
> actual data by using mmap().
>
> Does it clarify?
>
OK, that's what I understood, I just didn't understand (and still don't)
how binbuf helps there. Yes, higher level languages have a small
advantage there because they have native types for COW and etc. Binbuf
doesn't solve it though, and it really only solves a problem we are not
interested in, multiple consumers on the same con. If that's ever the
case, users will just need to implement it on their own. That's fine!
--
Tom
------------------------------------------------------------------------------
_______________________________________________
enlightenment-devel mailing list
enlightenment-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/enlightenment-devel
