On Tue, 20 Sep 2016 01:08:48 -0300 Gustavo Sverzut Barbieri
> On Mon, Sep 19, 2016 at 8:44 PM, Carsten Haitzler <ras...@rasterman.com>
> > On Mon, 19 Sep 2016 11:31:58 -0300 Gustavo Sverzut Barbieri
> > <barbi...@gmail.com> said:
> >> Hi all,
> >> Today I did commit support for libproxy in ecore_con's new API,
> >> Efl.Net.Dialer.Tcp, Efl.Net.Dialer.Http and Efl.Net.Dialer.Websocket
> >> (implicit from HTTP).
> > ummm i just am looking now (i have a lot of catching up to do). can you undo
> > the changes to configure.ac and make this a dlopen/dlsym approach? well use
> > eina_module. look a the old curl eina_module approach. look at what i did in
> > ecore_audio too. in fact i need to look over all the efl net code, but you
> > want to use emile for ssl stuff and eina_module for curl too there too (i
> > haven't looked yet). there are very good reasons to do all of this.
> I've checked what you did for curl, since I use that and need more symbols.
> my question and concern are just:
> - not being able to compile without libproxy. If we always detect,
> should we remove --enable-libproxy from configure.ac?
yes. it moves to runtime not compile-time. several reasons:
1. makes compiling simpler. people dont have to find dependencies and those
-dev/devel pkgs too. if they add libproxy later after building efl magically
the feature works without a rebuild.
2. speed up startup time with less linking going on for apps especially for
features they may never use
3. saves memory - symbol tables and dirtying private pages to do the symbol
fixups is expensive. i nuked like 320k or so before 1.18 release of actual
real memory usage that was dirty pages from rarely used libraries.features.
it's expensive and so only load in if/when needed to save this cost. this cost
is private pages PER PROCESS using efl so it multiples and is not a one-off
4. the idea of isolating such costs into a proxy daemon/process probably is a
good thing and that saves adding the above cost for any process then needing to
do proxy pac file parsing etc. and isolates the cost into a single daemon.
> - I ask the above because libproxy is a monster, it will pull in a JS
> loader, glib, C++... all of that, per process. If using pacrunner
> (from connman guys) you can just link with libdbus, less bad.
in GENERAL go for a dlopen(eina_module) style approach, BUT if a daemon makes
sense, then do that. it makes more sense for us to do our own daemon for efl
like efreet. dbus is not a good solution for this. supporting pacrunner later
of course is doable, but i would not do this out of the box because pacrunner
is not everywhere, dbus session buses do not exist when you do things like use
the console (terminology using fbcon is a major problem when it comes to ethumb
and was for efreet because no session bus is active thus everything bound to a
session bus is broken). i will eventually rewrite ethumb features likely in elm
and drop ethumb because of such issues. also ethumb design is broken for SMACK
systems so we really do need a "fork off a slave process that is custom for
your app only" method of working.
for proxies i see the value of a single efl.net.proxy daemon service to serve
as a proxy handler - send relevant info to this daemon, let it decide and then
answer back with what connection to make. so basically take the libproxy stuff
you just committed, move it to a daemon (like efreet does now), and connect to
it (execute then connect if connect fails). if you don't, then i'll eventually
need to do this.
keep this in mind as a design point. for data that is "global" (efreet style
stuff, proxy info etc.) then a single daemon doing the work will be the best
solution. maybe in future we might/can merge these daemons into a single "efl
> > emile too needs to switch to using this approach internally as well.
> >> Libproxy (https://libproxy.github.io) tries to dynamically find a
> >> proxy for a given URL, unlike plain $http_proxy, $no_proxy... which
> >> needs a mess to get right in complex environments such as big
> >> corporations. It will also use PAC - ProxyAutoConfiguration, those
> >> local network, where it tries to find the JS file.
> >> More than that, libproxy uses the configuration from Desktop
> >> Environments. There are none for E (some volunteer?), but we can use
> >> the one from Gnome3/Gnome or KDE. It will also handle envvars and some
> >> /etc/sysconf configuration.
> > what kind of config? you mean like a dbus interface exposed from e to
> > provide proxy info? i haven't looked...
> whatever you want, like
> they have config_envvars that check http_proxy, no_proxy... but they
> also have one for gnome that uses gsettings.
err so you mean write code into libproxy to get proxy settings from e? a quick
look at the above implies that... is there a way of hooking into this e.g. from
the above "efl proxy daemon"? actually i'm sure there is...
either way a single daemon would solve a lot of the above issues of libproxy
pulling in half the universe into every app needing to do network stuff.
> >> Just be aware that libproxy is nasty and will load many crap in your
> >> application binary, including mozjs to interpret JS mentioned above.
> > ugh. i know. pac files. js in there. ugh. it's times like this that all the
> > "web tech crowd" makes me want to cry. it's like they never cared about
> > performance... ever. just stuff some js anywhere because we have a browser
> > using it already and hey whats an extra few mb to something already needing
> > 1gb to run... :( oh wait1 we already have js spinning so it's no cost to
> > us! argh!
> indeed. At least pacrunner isolates this in their daemon, not every
> process. And the file is shared... not to say they picked a light JS
> engine (duktape) as the wpad.dat is super-simple.
yeah. for now i think the best soln for us is a single efreet-like daemon that
is spawned if you cant connect to it and THIS daemon deals with proxies.
efl/net would auto spawn it like efreet does to efreetd and connect and ask it
to process a proxy connection (ie here is a target ip/name/url - tell me what
> >> A better solution is to use libproxy replacement from pacrunner
> >> (http://git.kernel.org/cgit/network/connman/pacrunner.git/), that one
> >> will only pull in libdbus and the JS is done at the daemon, which also
> >> keeps shared cache not replicating it in all processes.
> > much better, OR we even spin off a process - let it execute as a child, read
> > back results in parent (and cache them until proxy data changes).
> less bad, but still one per process doing networking.
yeah... or just 1 process for all efl apps. ie like efreetd.
> >> Be traditional libproxy or pacrunner's alternative, the current code
> >> should be more dynamic, however needs more testing. If you work inside
> >> a proxy, give it a try and report problems, you can use the
> >> src/examples/ecore/efl_io_copier or efl_net_dialer_http_example to
> >> test.
> > we do indeed live inside one at work. and there is the PAC file from hell...
> >> It's important to get more testing as in near future I'm converting
> >> parts of E/EFL that used ecore_con to use this and I don't want to
> >> break things :-)
> > please please... don't race! you bring up very important things above. also
> > dlopen the sucker. don't make it a configure option + link option. runtime
> > link as needed. duplicate whatever elements of the libproxy headers needed
> > to talk to it.
> I can do this very easily, 3 functions, no public struct, just returns
> a char **.
that's even better! the simpler the headers/structs/api's needed to interface
to it the easier it is to make it runtime only. even if it is ONLY accessed by
a single daemon it simplified compilation. :)
> > we need to do much more of this for "rarely used dependencies". unless it's
> > needed by like 80%+ of apps, then it should be dlopen'd instead of linked.
> > linking is EXPENSIVE.
> > also the issue running the PAC files... it makes sense to ecore_exe() a
> > slave and have it figure it out once and store the result (ok ok PAC files
> > technically have to be re-executed per connect because they can switch per
> > target... urgh). we COULd do it like efreet with a daemon approach for now.
> > we can support pacrunner as a daemon later too and avoid running our own.
> > in fact this is likely the best approach - do it the efreetd way. no no. do
> > not use dbus. yes i know it does activation for us. no. this is far more
> > trouble, but i won't get into that here. just connect to a local user ipc
> > service of our own, if not there, spawn the daemon and try connecting again
> > a bit later. repeat until connect succeeded. do it in background async.
> I'd say the sanest route is:
> - dlopen trick on libproxy (3 functions)
> - hint packagers on optional dependency on libproxy
> - don't bother with libproxy.so (traditional) impact, hint users to
> go pacrunner. Note my code already uses libproxy 'get proxies for url'
> in a thread, as they recommend in their project.
> - use pacrunner where performance matters.
> - push distros to package pacrunner (glib + dbus + curl, very simple)
we pushed connman years ago. not again. distros just don't want to. we need to
DIY and provide our own "pacrunner". that's easy enough for us as we do it with
efreetd already. we have lots of infra to do this easily. if we just have an
eflnet "proxy handler" and efl.net spawns and talks to it we have a solution
that doesn't require pacrunner to be packaged and so on (going by history of
connman i just dont want to go here again), and it's TRIVIAL for us to support.
you have there all the relevant code already. just needs to be put into its own
binary, with an ipc/whatever conn for the user and some code to connect and/or
> AFAIU (still trying to get more details on the working), pacrunner
> will interact with connman's proxy configuration... I need to stop and
> read/ask which one is the controller. But that's the ideal situation,
> we already do connman, set property there and it spreads to the whole
the problem is 99% of users dont want or use connman. after years i've seen
that and i don't see that binding ourselves even deeper is a good idea. connman
works fine. i use it personally. but it's just not palatable for distros or
users. providing our own little proxy daemon process like efreetd would be a
self-sustained solution that is GUARANTEED to be installed and work in every
------------- Codito, ergo sum - "I code, therefore I am" --------------
The Rasterman (Carsten Haitzler) ras...@rasterman.com
enlightenment-devel mailing list