Hi Chris,
yes, but I hoped to find somebody hanging around here who could give some input.
As I'm currently really thinking basically 24/7 about our future "embedded"
stack I am going again through edgent to really understand what USPs it
currently has and what it already offers in comparison to our stack or
alternatives.
So if we would decide to use edgent this would also mean for us to go into the
project hands-on and fix or implement all things we like to have (like we
currently try to do with plc4x) so I try to get a good picture of the internals.
And the most confusing things for me currnelty are the naming (IotDevice,
IotpDevice, Oplet, ...) and the inner workings of the runtime and how
everything is organized.
So if there is no one here active except you that’s a pitty but then I can (for
now) try to support you on the inventory as best as I can until we can decide
how we will proceed. But for us its kind of the same situation as for you...
our thoughts are big but our productivity is limited and we have to be careful
not to overpace and start too many construction sites at once.
Julian
Am 22.09.18, 14:48 schrieb "Christofer Dutz" <christofer.d...@c-ware.de>:
Hi Julian,
well as I told you, I got on board of the project shortly before IBM
shifted their priorities. Unfortunately I didn't have the chance to dig into
all the details before that happened and other dropped out.
I wouldn't be expecting much input from the rest. So I guess we would have
to dig through stuff like this on our own.
But I am really willing to do this as I don't know of a similar project and
I really want to help revive Edgent again.
Chris
Am 22.09.18, 08:13 schrieb "Julian Feinauer" <j.feina...@pragmaticminds.de>:
Hey all,
just a short appendum.
I profiled through the code some more and found out the following:
* Edgent (ETIAO Runtime) creates (and deletes) one Thread for each
scheduled invocation of a Polling Source
* This thread is not only used to poll but also to do all subsequent
processing of the next pipeline steps (Oplets)
* This leads to creations of hundres of Threads / min which imposes
significant pressure on GC (I looked at the SensorAnalyticsApplication where GC
was triggered on my laptop each 30s and about 300MB in Thread Objets were
collected (about 1000 instances)
Furthermore this has the following consequences:
* No backpressure can be used with the current setup slow sinks have to
throw away messages(?)
* No guarantee of order as sheduleAtFixedRate is used and the runnable
that’s scheduled processes the complete pipeline for the fetched element, so
if, e.g., some intense operations is only performed on some objects (or simple
filtering) this could lead to races where elements that were pulled later enter
the sink earlier
I will try to provide a concrete code example where these situations
are demonstrated.
It would be interesting for me to get an idea why this implementation
was chosen or what you think are the advantages of this approach. I think
Edgent has some very nice features and can solve some of the common problems
for "edge" analytics in an elegant way but I'm not that sure with the ETIAO
Runtime for the reasons stated above.
But perhaps I am also wrong with some of my conclusions as they are
solely from profiling and debugging through the code so please correct me if I
am.
Best
Julian
Am 22.09.18, 10:07 schrieb "Julian Feinauer"
<j.feina...@pragmaticminds.de>:
Hi Edgenteers,
after chris does not stop to convince me that Edgent is THE THING
to use for IoT I started once again a deeper look into its internals.
I do not yet fully understand the core of Edgent and its runtime or
how it physically runs the users operation at the end.
But what I observed is that it uses a pretty high number of threads
and what’s way worse it consumes many threads, i.e., it starts and stops
threads all the time (about 1 per second in several Demo Applications (Water /
Sensor Analytics Application).
And this is also not dependent on the Provider (Direct vs
Development).
Can anyone here give me some explanations on how the Threading
Model of Edgents runtime is and why it is done that way as for me, it seems a
bit wasteful with regards to resources and we want to be as low footprint in
our applications as possible.
Thank you!
Julian
