On Wed, 3 Aug 2016 01:29:39 -0300 Gustavo Sverzut Barbieri <barbi...@gmail.com>
said:
> On Tue, Aug 2, 2016 at 5:00 AM, Carsten Haitzler <ras...@rasterman.com> wrote:
> > On Mon, 1 Aug 2016 00:50:31 -0300 Gustavo Sverzut Barbieri
> > <barbi...@gmail.com>
> [...]
> >> struct Efl.Net.Address {
> >> family: Efl.Net.Family;
> >> address: union { in, in6, bt... };
> >> }
> >
> > are you sure this shouldn't just be a string above for address? and as
> > onefang said - the string can tell us what we need. it can just be a domain
> > name of an ipv4 addr or an ipv6 addr and so on... ? the actual
> > representation like this should be internal and not at any API level. at an
> > API return a stringified thing. set the destination address, get it back
> > as-is with a string name. request a result and once resolve is done
> > (callback or promise) you can get the string resolution of the address. eg
> > ipv4 addr string or ipv6 addr string for example. this just de-complicates
> > API across bindings to not need a union (a bit of an ugly issue) and who
> > really deals with addresses at this format/level?
>
> No hard opinion on string versus struct. The good part of strings is
> that it's easy to extend without going into unions and the likes. But
> it doesn't allow compile-time checks and simple switch/if on the
> family (which can be solved by using some functions to parse or check)
>
> If you think a string is good to go, I'll change to that.
i totally get the point about structs/compile time checks etc. but in this case
i think the flexibility and simplicity of a string will far outweigh the
downsides. reality is that 99% of usage will be like connecting to
"myserver.com" Or a simple ip address on a local network which is kind of hard
to get wrong. :)
> >> struct Eina.Blob {
> [...]
> > this is a big problem for bindings. a big fat "no - find another way". no
> > function pointers in structs or parameters to methods etc. no void *'s
> > (reserved for base eo classes only). use eina_binbuf - a list/array or
> > chain of binbufs or perhaps something else.
>
> see my other emails, the idea is to allow different "free" for
> different kinds of memory (ie: mmap) and windowing/slicing (ie: line
> of a file).
>
> It's not a hard requirement either, just my efficiency advice after
> invested lots of time to test this use case with Soletta.
i think this would be then best solved at the binbuf level - allow a binbuf to
be a region of a mmaped file (eina_file?) and handle referencing etc. etc.
there.
> >> @property local_address {
> >> [[the local IP or unix-local address]]
> >> get @virtual_pure {
> >> }
> >> values {
> >> address: Efl.Net.Address;
> >> }
> >> }
> >
> > naming: address_local
>
> ouch! keep an eye on that for me, in Soletta we changed the order and
> it will bite me for sure ;-) Fixed.
cool.
> > what local address is this exactly? if tis a server i'ts the address being
> > listened on? the ip address (ipv4/6/whatever) of the outgoing interface or
> > something?
>
> local: getsockname()
> remote: getpeername()
ok.
> >> @property remote_address {
> >> [[the remote IP or unix-local address]]
> >> get @virtual_pure {
> >> }
> >> values {
> >> address: Efl.Net.Address;
> >> }
> >> }
> >
> > naming: address_remote
> >
> > and remote - this is the address you probably want to SET right so its set
> > {} AND get {}. the get gets what you set. now i mention - what about a
> > address_resolved_remote property you can just get {} only that tells you the
> > resolving of the name or whatever specified by setting the address_remote
> > property.
>
> As per above, it would be the already resolved and actually connected
> name, it's in the base class (Efl.Net.BaseSocket), so also means a
> socket from a connecting/incoming client. Thus the "address_remote"
> like that is consistent.
>
> However I see that for connect-to-server role
> (ecore_con_server_connect() and current Efl.Base.Socket, the given
> parameter (ie: address_destination? address_connect?) could be
> remembered for reference purposes. Example
>
> @property address_connected {
> [[The address used to create and connect this socket. It
> is the unresolved address, see the property address_remote
> for the actual resolved address]]
> get @virtual_pure {
> }
> values {
> address: Efl.Net.Address;
> }
> }
yes. so once resolved you could re-use the existing resolution if you chose to
(possibly more efficient than relying on caching by libc or a local dns
proxy/server).
> > how do we deal with resolving? a promise? an event callback? we have
> > multiple stages here. first a resolve (async) then a connect (async) then
> > finally we are established.
>
> In the current proposal it's not there, it would be another class that
> resolves the string to an Efl.Net.Address and that could be with a
> promise.
>
> That could be more cumbersome to use, but would remember people to
> cache the resolved address.
>
> Node.js and Qt both use an event to notify they were resolved.
i think you should do that too. use a resolved event, and always automatically
resolve for the user as anything else is painful to use to have to break up
resolving and then connection into separate things. let people know what
events are/have happened along the way and *IF* they provide the right info -
eg a previously stored and resolved ip address then they can skip stages in the
connection (still provide a resolved event but call it immediately on connect
as the result is known already).
> >> @property internal_fd {
> >> [[the internal file descriptor (use with caution!)]]
> >> /* it seems the usage is to steal the fd to do something
> >> * else or set some flags. If so we could expose some
> >> * primitives such as fcntl_add(type, flags) and
> >> * fcntl_del(type, flags) as well as a destructor that
> >> * will return the stolen fd (destroys the object and
> >> * keeps fd alive).
> >> */
> >> get @virtual_pure {
> >> }
> >> values {
> >> fd: int;
> >> }
> >> }
> >
> > this has issues possibly on windows. fd's don't quote behave the same way
> > here. can we avoid this? if we need to do deep down ugly things with the
> > fd... can we just support them with the API? like fd passing on local unix
> > sockets. have our api deal with sending and getting fd's as a specific send
> > method and an input event. i get why you have this - as a "way out" for
> > when we don't support something. this is kind of bad for bindings though
> > and portability. :(
>
> don't get me wrong, I tried to justify to not have it there!
>
> I just put it here because Ecore_Con.h provides that and from my
> investigation, empc is the only user (gets the connected fd and
> handles the socket to mpd library:
that's kind of silly :( then you cant do proper buffering etc. and portability
is hurt.
> https://git.enlightenment.org/apps/empc.git/tree/src/bin/empdd.c#n828
>
> Then in this case we could provide a way to steal the fd and delete
> the client, keeping the fd alive.
i dislike this. i am ok with a "legacy api only" function that can steal out
the fd.. but in eo? i'd rather this be as portable as possible. mpd etc. libs
then probably should provide a virtual i/o interface instead.... that would make
it portable. :)
> >> @property timeout { /* this usually doesn't change,
> >> should be constructor-only */
> >> [[timeout to close the connection.]]
> >> values {
> >> timeout: double;
> >> }
> >> }
> >
> > i see no reason that this can't change. a timeout for a connect or a timeout
> > for buffers being able to send ANYTHING (if buffers can't drain for writes
> > for more than N sec - consider this connection dead? should they be the
> > same timer?)
>
> I still have to see when these would be dynamically changed in real
> usages. Usually they are a constant, eventually user configurable but
> you could apply to new sockets only.
>
> The benefit of never changing is that your code will be reduced to a
> simple ecore_timer_reset() instead of the slightly more complex
> _ecore_con_server_timer_update():
>
> https://git.enlightenment.org/core/efl.git/tree/src/lib/ecore_con/ecore_con.c#n1456
well just add another reset whenever you set the timeout... pretty simple :) 1
more line... :) (and document the timeout_set as resetting the current timeout
so a pending one will be restarted when set).
> >> class Efl.Net.Socket (Efl.Net.BaseSocket) {
> >> [[This is a client socket that is created and connected to a
> >> specific server given to the constructor]]
> >>
> >> methods {
> >> connect {
> >> [[The constructor that connects the socket to a given server]]
> >> params {
> >> @in type: Efl.Net.Socket.Type;
> >> @in address: Efl.Net.SocketAddress;
> >> @in timeout: double;
> >> }
> >> }
> >> }
> >
> > as people mentioned. we could have type implicit in socketaddress with
> > strings. but let's talk constructors. shouldn't we just have connect() take
> > no params and use the currently set up remote address, timeout and other
> > parameters at the time? this works well with eo's finalize constructing
> > mechanisms. these constructors atm are only really useful for c++ even then
> > we'r elooking at using lambdas for construction calling methods.
>
> could be, but is there a way to limit the "set" method of a property
> to be only usable in construction time (ie: before finalize)?
>
> like resetting the address... once the connection is done, this
> operation is not supported.
i agree that THIS would be silly (to change address once connected). i was
thinking that you may want to respond to a resolution event though. e.h. set a
name, on resolved event go "oh dear. that's wrong! i know the ip address
should not be this" and then try some fallback. with the same connection. It's
simple enough to have the functions return a bool and return a fail if it's not
possible to change this aspect. document them as such. i would actually just
document them as having no effect until a connect() method is called. a NEW
connect() will fail until a disconnect() is called. you can only change these
parameters when the connection is not connecting or connected (you must
disconnect first). ...
> >> class Efl.Net.Server (Eo.Base) {
> >> events {
> >> connection @hot: Efl.Net.BaseSocket; [[new connection was
> >> accepted]] error: int; [[errno]]
> >> /* no disconnect, listen on each individual connection */
> >> }
> >
> > i really don't like error as an int. really don't. a proper enum and/or
> > struct with fields like enum for error type and maybe a string for error
> > extra info (human readable from perror() style?)
>
> is Eina_Error ok?
errr... thats horrible., it's a global TLS. ugh. :( make it an enum :
> > what about disconnects? you expect peolpe to listen on del events on the
> > connection objects?
>
> yes, IMO makes more sense unless there is a common pattern to not do so.
well above i kind of imagined a disconnect() but a whole new connection obj can
do the job. you create a new one, set up params (address, timeout anything
else), then connect when you're happy with setup. if anything along the way you
don't like - delete the object, create new one, set up and then connect again.
repeat. :) that's fine by me too. so once a connect() is done several
properties can not be change anymore (like address/port target)
> >> @property connections {
> >> [[active connections]]
> >> /* Needed?
> >> *
> >> * actual members or just a counter? It seems all others
> >> * do not keep a list and we should be careful if
> >> * iterating over the list happens to modify it.
> >> *
> >> * count would be useful to gracefully quit if no
> >> * connections, but is easily done elswhere.
> >> *
> >> * the only user in our repo is eeze_scanner.c that uses
> >> * to send a single eet to all connections. Also easily
> >> * done at its own side (actually it already stores the
> >> * clients in a hash, could foreach() that one).
> >> */
> >> }
> >
> > how about return an eina iterator for connections that will list children.
> > that would help and not define the storage type. i think its good to have
> > this in some form, but best might be an iterator.
>
> my worry is that it can be tricky if people may change the storage
> during the iteration, such as loop in a list of connections and you
> can delete the next element.
iterators SHOULD handle this. e.g. either disallow modifications to the list
while there is an open iterator for it, or only allow non-destructive changes
that wouldn't affect the iterator badly.
> It's easy to create your own list if you wish, by listening to
> "connection" event and append to eina_list.
>
> The only user I found (eeze_scanner) already have a hash, so could do
> a foreach() there.
it just saves the clients from doing their own client store since the server
obj is already doing this anyway... :)
> > i'm going to skip the analysis below... and focus on...
> >
> > 1. missing anything to do with SSL. events for certificates, validity, being
> > able to register certs, verify etc. etc. etc. you touch on this in analysis
> > of ecore_con server/client.
>
> it's written on the first part of the email :-) I'll come to this, but
> let's first get the basics right and the SSL must follow the pattern.
yeah.. but it's not there. so needs doing. :) it adds quite a lot of things. :(
> > 2. no way to request a resolve FIRST separately to a connect. this would be
> > good i think (a connect implies a resolve then connect if not already
> > resolved, but allow a stand-alone resolve to then verify results before
> > doing a connect).
>
> indeed I forgot to write that, but as stated above it would be another
> function that does string -> Efl.Net.Address.
ok.
> If the methods and properties should move to strings, then a way to
> set the exact resolved address could be used, as well as an event to
> notify string was resolved. An explicit resolver (like the one in
> Golang) could always be of use.
>
>
>
> > 3. addresses i think should be just strings. we could then overload this to
> > do things like websockets just by a special address format (ie url) and the
> > details are hidden
>
> ok, I can change if others want that as well.
a quick read shows most do. :)
> > 4. how do you disconnect a client? del the client object you see in the
> > event info?
>
> yes. I see that other tools offer some "graceful shutdown" and
> "immediate abort". The graceful shutdown will stop reading and will do
> the actual close once all pending data is sent. The immediate abort
> will ignore pending stuff.
hmm. very good question. how do you handle this? hmmm.
> we can do 2 distinct methods if you wish, or we can let the user deal
> with graceful shutdown on his own by using the events (or promise as
> cedric wants).
or... have a "shutdown mode" for an object. the default would be graceful (stop
reading and flush all buffers already queued for writing). this would be the
nicest in that it preserves logical behaviour. if your wrote something it WILL
get sent even if you delete the object. just the delete is a task/event to do
after the writes you called are done. it gives the appearance of
order-of-operation and no nasty surprises.
the del should not necessarily BLOCK. it should just queue the deletion in this
case...
then have a "no do it right now" mode which abandons any unwritten/flushed
writes and kills it off right now. this mode you'd change prior to a del if you
really wanted this. this guarantees that other than some other accounting needs
which may hold refs to the obj, it'll be gone after the del along with any
buffered data not written yet.
> > 5. i think controls on max number of clients would be nice so it auto drops
> > clients without you bothering to listen for the connect cb's then disconnect
> > them by deleting the objects.
>
> You did skip the constructor method, it does contain:
>
> @in client_limit: int;
> @in reject_excess_clients: bool;
>
> :-)
oh yeah. make them properties :) then it also can be adjusted dynamically as
needed. :)
> > 6. have you considered freezing or thawing clients so write buffers are
> > frozen (and input buffers are ignored) while frozen? this would be like
> > pause/resume in node.js
>
> could add, but currently we do not provide that, thus no users on our
> codebase.
ok. just note this on a possible todo list for the future and leave it at
that. :)
> > 7. missing things like tcp keepalive and nodelay.
>
> also no users in our code base. We could add these to some "flags" to
> be given to the constructor. Actually NODELAY is currently implemented
> like that, it's a Ecore_Con_Type.
ok - add later as properties. :)
> > 8. what about udp?
>
> You mean sendmsg()/sendto()?
yes. where sends now are not part of a continual stream of bytes but are
individual udp packet sends that may or may not arrive or get re-ordered. here
a connect isnt really a connect. it's just a resolve and a "virtual connect" in
that it knows now where to send stuff to.
> For other tools they all offer different classes with specific
> methods, thus we could create:
>
> Efl.Net.UDPSocket interface that implements
> sendto/sendmsg/recvfrom/recvmsg. We could map to regular events by
> taking and sending the whole datagram in/from data.
i think the above. make it the same just with a property type of udp. this
simply changes the nature of data sent and received - no longer a "reliable
stream" but a series of packets sent/received up to a limited size. this makes
its so sos so easy to copy & paste gode from a udp to a tcp connection or
vice-versa. it's a property change and nothing more as long as you are prepared
to deal with packets being re-ordered or disappearing on the incoming end.
> > 9. no way to GET the current buffer status of a connection you made to a
> > server. e.g. how much data is buffers and not sent yet (not sent to kernel).
>
> indeed, you'd have to keep that information elsewhere. But it's a
> simple getter, so we can add if wanted.
yeah. a simple getter to get # of bytes or # of packets not set yet etc.
> what name to use?
>
> - pending_send_bytes
> - send_buffer_usage
buffered_data <- returns total # of bytes still waiting
buffered_sends <- returns number of sends/packets submitted not yet sent
both properties with only get {} methods.
so every send() is let's say a "blob of data" thats conceptually added to a
list of data to send. for udp this equates to a packet. for tcp its just a
series of data to append. the above would
> > 10. perhaps add a drain event to know when all write buffers are finished
> > being written to kernel? or maybe sent out of interface to other end?
>
> added
cool.
> > 11. should we consider things like socks proxies - how they work with this?
>
> Qt sets a proxy object to socket. I did not check their
> implementation. Others do not deal with that in their API.
>
> Node.js does for http/https using an "agent" option:
> https://github.com/TooTallNate/node-socks-proxy-agent
>
> Go does http/https using a "transport" option and for regular SOCKS
> they use another class https://godoc.org/golang.org/x/net/proxy#Dialer
> (analogous to our ecore_con_server_connect()).
ok maybe the "attach a proxy object" might work best if you wish to customise
proxy behaviour, otherwise we can just do whtever is most sensible - eg use
global env vars that may force socks proxies on everyone etc. - ok. that's am
implementation detail. worry about proxy object a bit later then.
> > 12. what about api's so do things like fd pass (and receive), do things like
> > get the uid/pid and other process unfo or the other end of the connection
> > of a unix socket?
>
> These should be an extra interface that only the UnixSocket would
> provide, they would do things like SCM_CREDENTIALS.
ok. so an added interface for a subclassed unixsocket class. got it.
> > what should we do about things like mqtt, mq/zeromq, ocf/oic - they belong
> > in another interface layer? http is a bit different too due to its nature.
> > websockets i can see fitting underneath the above, but not real http itself
>
> mqtt and the others are all PITA to get all the quirks right... pretty
> much like http they look simple to implement the basics, but then the
> other peers rely on bug-compatibility with mainstream projects such as
> mosquitto, iotivity... So implementing on top of pure socket is a
> major work, would be simpler to just do bindings for "de facto
> standard" libraries.
oh i wouldn't want to re-implement the protocols on top of raw sockets. hell no.
i was thinking that efl net work provide a layer that would offer these
(mq/mqtt/oic/ocf/allseen?) as a transport protocol layer... the problem is this
level of api is very dumb - it doesn't understand much but a blob of bytes going
in or out either as a stream or as a series of packets. the above i would
absolutely expect would be implement on top of a library layer that dealt with
the details and we're just providing a single "nice abstract class/interface"
for doing not just tcp/udp and local unix sockets but also these others...
since we're doing http - how would these others fit into this design? would
they? should they? there is value in these being wrapped by eo objects so they
can nicely sit in an object tree and be managed like everything else... :) i
don't say we do this NOW... but i say we should consider these in our design
and not make something that might preclude them unless it really is just insane
to support them.
> --
> Gustavo Sverzut Barbieri
> --------------------------------------
> Mobile: +55 (16) 99354-9890
>
> ------------------------------------------------------------------------------
> _______________________________________________
> enlightenment-devel mailing list
> enlightenment-devel@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/enlightenment-devel
>
--
------------- Codito, ergo sum - "I code, therefore I am" --------------
The Rasterman (Carsten Haitzler) ras...@rasterman.com
------------------------------------------------------------------------------
What NetFlow Analyzer can do for you? Monitors network bandwidth and traffic
patterns at an interface-level. Reveals which users, apps, and protocols are
consuming the most bandwidth. Provides multi-vendor support for NetFlow,
J-Flow, sFlow and other flows. Make informed decisions using capacity
planning reports. http://sdm.link/zohodev2dev
_______________________________________________
enlightenment-devel mailing list
enlightenment-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/enlightenment-devel
