I agree, I don't think it's possible to accurately tell base64 from some
random string without input from the user.
Am Sa., 18. Apr. 2020 um 23:46 Uhr schrieb Rodric Rabbah :
> Actually my suggested decoding step doesn't work either. I don't think
> there's a sound way to record the binary
Hi,
this does mean, that the prewarm pool's capacity cannot be reclaimed by
other actions should the need arise, right? I think that's problematic if
that notion isn't carried through into loadbalancing itself as AFAIK the
controller currently does not take the prewarm pool into account at all.
FWIW, the UUID entropy workaround where not JVM specific IIRC but OS
dependent, as we switched from random to urandom as a source of entropy.
I don't feel strongly for either OpenJDK vs. OpenJ9, but it'd be nice to
understand where the degradation comes from and thus make a deliberate
decision to
Thanks for taking this on you Rodric. For new contributors it can really
make all the difference to be welcomed into a community like this. Kudos to
you!
Happy holidays!
Am Mi., 18. Dez. 2019 um 17:16 Uhr schrieb Rodric Rabbah :
> As a project and community we have a lot to celebrate this year
I to build the controller cluster.
Am Do., 21. Nov. 2019 um 15:10 Uhr schrieb Carlos Santana <
csantan...@gmail.com>:
> Woot +1
>
> - Carlos Santana
> @csantanapr
>
> > On Nov 21, 2019, at 5:58 AM, David P Grove wrote:
> >
> >
> >
> >
&
The fix you shared seems to be only logging related so it seems that these
logs are harmless? If we don't see any detrimental effects to actual usage
I'd vote for waiting for the fix to be released to just shut down the noisy
logging. If we see actual errors though, we of course need to look at
Am Mi., 20. Nov. 2019 um 19:52 Uhr schrieb David P Grove :
>
> "Markus Thömmes" wrote on 11/20/2019 10:44:21
> AM:
> >
> > Hey fellow Openwhiskers,
> >
> > I somewhat fancy a little bit of Scala action and wanted to give updating
> > Openwhisk to
Hi Steffen,
the interface is already Future[Boolean], why is it not sufficient to fail
the future if the network calls fail for any reason in this case? After
all, that's what the Future's failure mode is all about. Or asked
differently: What states does a failed Future represent today?
Cheers,
ly for /run, will be handled by way of rescheduling back to
> ContainerPool (/init should already be handled by retry for a time period).
>
> Thanks!
> Tyson
>
> On 10/30/19, 7:03 AM, "Markus Thömmes" wrote:
>
> Increasing latency would be my biggest conce
Increasing latency would be my biggest concern here as well. With a health
ping, we can't even be sure that a container is still healthy for the "real
request". To guarantee that, I'd still propose to have a look at the
possible failure modes and implement a retry mechanism on them. If you get
a
Heya,
thanks for the elaborate proposal.
Do you have any more information on why these containers are dying off in
the first place? In the case of Kubernetes/Mesos I could imagine we might
want to keep the Invoker's state consistent by checking it against the
respective API repeatedly. On
Hi,
this sounds a lot like the "OpenWhisk 2.0" discussion we had in an older
thread where I proposed an architectural overhaul. See this thread for
context:
https://lists.apache.org/thread.html/d19306dd976138a153f48d32c5a55f2853e4b8ff405fc46f7260e905@%3Cdev.openwhisk.apache.org%3E
There have
Great stuff Chetan! The quick impl. also shows our abstractions kinda work.
That makes me happy too :).
Am Di., 16. Juli 2019 um 00:56 Uhr schrieb Carlos Santana <
csantan...@gmail.com>:
> Great work Chetan
>
> This must be a new record how fast to get a POC integrated in a few days
> after
er provides enough isolation where having a jvm in a
> container in a virtual machine is total nonsense... So there is a
> comeback to "native" code
>
> Yes of course there is a library to make it dynamic, there are code
> generator, there are tons of tools to overcome the l
e or
> > modules needed. I haven't kept up to date with C++ since pre-'10, but
> > I'd imagine that it's easier to learn Rust than the latest version of
> > C++ at this point.
> >
> > On Wed, 17 Jul 2019 at 08:06, Michele Sciabarra
> > wrote:
> >
+1 to all arguments. Rust's barrier of entry is considerably higher than
that of Scala even. As much as I like the language's design, from a
community attraction point-of-view we should absolutely opt for go,
especially for things that are built around Kubernetes.
That's of course to be taken
Thanks as always for your very detailed writeup and analysis of the issue
at hand! Great job Sven!
It seems to me that we might want to look at analyzing our runtimes for the
number of file-descriptors they open and make up for that for the user. The
user should always get at least the amount of
Continuous integration is a crucial aspect of a system used in production
like OpenWhisk.
As such, let's bite the bullet by merging things incrementally into master.
That will also force you to think about migration on each step you take,
which in the long run will pay off big time. Accumulating
+1 Apache OpenWhisk should graduate.
Thanks all for the great community and opportunity to work with such smart
people!
Am Mo., 10. Juni 2019 um 17:55 Uhr schrieb Justin Halsall <
jus...@juice10.com>:
> +1 OpenWhisk is ready to graduate!
>
> > On Jun 6, 2019, at 5:11 AM, Bertrand Delacretaz
>
ild API
> documented in the website. Is Tekton "compatible" or it has a different API?
> >
> >
> > --
> > Michele Sciabarra
> > mich...@sciabarra.com
> >
> > - Original message -
> > From: "Markus Thömmes"
> > To:
Hi Michele,
thank you for the detailed writeup. A few thoughts inline:
Am So., 19. Mai 2019 um 19:00 Uhr schrieb Michele Sciabarra <
mich...@sciabarra.com>:
> I have an idea for implementing a prototype of OpenWhisk on top of Knative.
>
> My basic ideas are: do not use any proxy, forwarding or
+1.
Make it happen!
Am Di., 19. März 2019 um 07:02 Uhr schrieb Michele Sciabarra <
mich...@sciabarra.com>:
> Actually it looks to me pretty unusual that a project of this size has not
> yet graduated.
>
> --
> Michele Sciabarra
> mich...@sciabarra.com
>
>
>
> - Original message -
>
Hi,
leaving a few remarks. Thanks as always for your hard work Michele, you're
cranking out a lot of stuff! Great job!
1. On the initial problem statement (and for your book): Have you
considered using an action with a very high concurrency setting for your
problem? The usual issue with these
Hi,
with Tyson's concurrency > 1 support we should be able to rehash the
requirements here. A single action that supports a high concurrency on one
shared connection should fulfill what's needed by the Kafka backend teams
here.
Outscaling a backend through a lot of functions is a common problem
Hi Jiang,
this is a fair question. The trade-off of using groupedWithin vs. batch is,
that groupedWithin **always** adds some latency to the database commands,
where batch only adds that latency on backpressure, as you've noticed.
Cheers,
Markus
Am Fr., 9. Nov. 2018 um 04:39 Uhr schrieb 蒋鹏程 :
Hi Christian,
given the recent work towards getting logs to a separate store to relieve
CouchDB and to make it possible to move it into a store that's more
appropriate for logging load and some other bits that are already
implemented (removing the DB based polling, being able to disable log
Hi Michele,
commenting only on Golang: While there are packages available that provide
a similar API surface, they have very little adoption throughout the Golang
community.
In general, the mechanism to achieve concurrent processing in Golang is
goroutines
Thanks Matt for putting this all together in lightning speed again! Very
much appreciated!
Am Mi., 12. Sep. 2018 um 20:07 Uhr schrieb Matt Rutkowski <
mrutk...@us.ibm.com>:
> Thanks Markus for moderating and special thanks to Tyson for leading us on
> the interesting topic of concurrent actions
Heya,
I tend to agree with Felix. Buffers are usually just used as a transport
mechanism and they shouldn't need to fit the whole response at once (which
doesn't imply we don't load the response in memory, but at least that input
buffer doesn't need to be == content-length). Could you give a bit
Huge +1 from my end!
I always thought of the "system.*" suite to cover exactly what you're
looking for, but we've been lax with regards to splitting test-suites and
the properties needed as you're experiencing now. It'd be great to clean
that up and put tests in appropriate packages to be able to
t; issues I've found in previous threads and may be solved in mind between
> folks however.)
>
> [1] https://tz70s.github.io/posts/openwhisk-performance-improvement/
>
> Thanks,
> Tzu-Chiao
>
> On Wed, Aug 29, 2018 at 11:31 AM Markus Thömmes >
> wrote:
>
>
gt; > > deprecated, the _possibility_ of no clean upgrade path, the immediate
> > split
> > > of the community between *-now and *-future and of course carries the
> > risk
> > > of the version 2 syndrome.
> > >
> > > I would propose to implement the fu
Hi Tyson,
shall we discuss the mechanics on how to start the prototype for the
future-arch discussion?
Cheers,
Markus
Am Di., 28. Aug. 2018 um 16:59 Uhr schrieb Tyson Norris
:
> Hi Whiskers!
>
> Please send any agenda items you would like to discuss at the Tech
> Interchange call tomorrow.
>
Hi all,
Am Mo., 27. Aug. 2018 um 20:04 Uhr schrieb David P Grove :
>
>
>
> "Markus Thömmes" wrote on 08/23/2018 04:19:33
> PM:
>
> >
> > Key point I want to make is: At some point we'll have to start to
> prototype
> > things out and see if our a
Thanks for bringing that up Rodric,
it seems like we could have a one stone two birds thing going on here? Do
you see an immediate reason why the proposal I made would not work? We'd
certainly need to check if any client hardcodes a fixed length for our
activation ids.
Am Di., 28. Aug. 2018 um
enerate a child id for that invocation. If we can
make that readable, we can easily correlate the parent with the child,
giving us even more of a gain from this proposal.
So from my PoV: Go for it!
>
> Thanks
> Best regards
> Dominic.
>
>
> 2018년 8월 23일 (목) 오후 5:34, Markus
Hi,
Am Fr., 24. Aug. 2018 um 00:07 Uhr schrieb Tyson Norris
:
> > Router is not pulling at queue for "specific actions", just for any
> action
> > that might replace idle containers - right? This is complicated with
> > concurrency though since while a container is not idle (paused +
Hi OpenWhiskers,
We've been discussing a new direction for our architecture based on a
proposal I made here:
https://cwiki.apache.org/confluence/display/OPENWHISK/OpenWhisk+future+architecture
Discussion threads:
-
Hi Dave,
I agree! I'll start another thread on a discussion of how/where we
prototype things to hash out some of the unknowns.
Cheers,
Markus
Am Do., 23. Aug. 2018 um 22:05 Uhr schrieb David P Grove :
>
> Related to the random vs. smart routing discussion.
>
> A key unknown that influences the
Hi Tyson,
Am Do., 23. Aug. 2018 um 21:28 Uhr schrieb Tyson Norris
:
> >
> > And each ContainerRouter has a queue consumer that presumably pulls
> from
> > the queue constantly? Or is consumption based on something else? If
> all
> > ContainerRouters are consuming at the same
igure-pod-configmap/#populate-a-volume-with-data-stored-in-a-configmap
>
> On Thu, Aug 23, 2018 at 1:32 PM Markus Thömmes
> wrote:
>
> > Hi Chetan,
> >
> > good idea!
> >
> > A bit of background on why it is how it is: When I implemented the
> approach
Hi Dominic,
this is certainly a good thought and would simplify debugging production
systems as well.
A rough and dirty idea for the uniqueness issue you mentioned:
What if we just append a suffix for each of the child ids (essentially they
are in a parent/child relationship). A simple example:
Hi Tyson,
Am Do., 23. Aug. 2018 um 00:33 Uhr schrieb Tyson Norris
:
> Hi - thanks for the discussion! More inline...
>
> On 8/22/18, 2:55 PM, "Markus Thömmes" wrote:
>
> Hi Tyson,
>
> Am Mi., 22. Aug. 2018 um 23:37 Uhr schrieb Ty
Hi Chetan,
good idea!
A bit of background on why it is how it is: When I implemented the approach
we're having today, the basic thought was to be able to detect on a quite
granular level what changes are needed to which components, even when they
share values. For example: In the Kubernetes
Hi Tyson,
Am Mi., 22. Aug. 2018 um 23:37 Uhr schrieb Tyson Norris
:
> Hi -
> >
> > When exactly is the case that a ContainerRouter should put a blocking
> > activation to a queue for stealing? Since a) it is not spawning
> containers
> > and b) it is not parsing request/response
nces to the Routers it gave Containers
to the Routers that have none. (This is the edge-case described in the
proposal).
The work-stealing queue though is used to rebalance work in case one of the
Routers get overloaded.
>
> Thanks
> Tyson
>
> On 8/21/18, 1:16 AM, "Markus Th
Hi Dominic,
it seems like you haven't published your writeup? The page seems empty.
Cheers,
Markus
Am Di., 21. Aug. 2018 um 12:44 Uhr schrieb Dominic Kim :
> Dear whiskers.
>
> I just wrote the guide to perform a throughput test against Openwhisk in
> cwiki.
>
ontainerRouters that receive all async activations, and others that
> receive all sync activations, but the end result is the same, I think.
>
>
> > On Aug 19, 2018, at 4:29 AM, Markus Thömmes
> wrote:
> >
> > Hi Tyson, Carlos,
> >
> > FWIW I should change th
Hi Tyson,
Am Di., 21. Aug. 2018 um 00:20 Uhr schrieb Tyson Norris
:
>
>
> On Aug 19, 2018, at 3:59 AM, Markus Thömmes <mailto:markusthoem...@apache.org>> wrote:
>
> Hi Tyson,
>
> Am Fr., 17. Aug. 2018 um 23:45 Uhr schrieb Tyson Norris
> mailto:tnor...@adobe.com
Am So., 19. Aug. 2018 um 18:59 Uhr schrieb TzuChiao Yeh <
su3g4284zo...@gmail.com>:
> On Sun, Aug 19, 2018 at 7:13 PM Markus Thömmes
> wrote:
>
> > Hi Tzu-Chiao,
> >
> > Am Sa., 18. Aug. 2018 um 06:56 Uhr schrieb TzuChiao Yeh <
> > su3g4284zo...@gmail.c
becoming optional, while opening the door for
> other connectors like AWS Kinesis, Azure Event Hub, and others (see the
> link at [1] for a more complete list of connectors )
>
> [1] - https://developer.lightbend.com/docs/alpakka/current/
> On Sun, Aug 19, 2018 at 7:30 AM Markus T
Hi Tyson, thanks for pushing forward on this! I'll try to get a review in
on it soon.
Am Fr., 17. Aug. 2018 um 19:04 Uhr schrieb Tyson Norris
:
> Hi -
> I have been noodling with a few tests and the akka http client and gotten
> the concurrency PR [1] to a good place, I think, so if anyone can
Hi Tyson, Carlos,
FWIW I should change that to no longer say "Kafka" but "buffer" or "message
queue".
I see two use-cases for a queue here:
1. What you two are alluding to: Buffering asynchronous requests because of
a different notion of "latency sensitivity" if the system is in an overload
Hi Tzu-Chiao,
Am Sa., 18. Aug. 2018 um 06:56 Uhr schrieb TzuChiao Yeh <
su3g4284zo...@gmail.com>:
> Hi Markus,
>
> Nice thoughts on separating logics in this revision! I'm not sure this
> question has already been clarified, sorry if duplicate.
>
> Same question on cluster singleton:
>
> I think
Hi Tyson,
Am Fr., 17. Aug. 2018 um 23:45 Uhr schrieb Tyson Norris
:
>
> If the failover of the singleton is too long (I think it will be based on
> cluster size, oldest node becomes the singleton host iirc), I think we need
> to consider how containers can launch in the meantime. A first step
Hi Tyson,
thanks for the great input!
Am Do., 16. Aug. 2018 um 23:14 Uhr schrieb Tyson Norris
:
> Thinking more about the singleton aspect, I guess this is mostly an issue
> for blackbox containers, where manifest/managed containers will mitigate at
> least some of the singleton failure delays
also for
> operational concerns).
> What are your thoughts about losing a shard (planned or crashed) or adding
> a shard?
>
> Michael
>
>
> On 15.08.18, 09:58, "Markus Thömmes" wrote:
>
> Hi Dragos,
>
> thanks for your questions, good discussi
is
encapsulated in the ContainerManager.
Cheers,
Markus
Am Mi., 15. Aug. 2018 um 10:35 Uhr schrieb Bertrand Delacretaz <
bdelacre...@apache.org>:
> Hi,
>
> On Tue, Aug 14, 2018 at 4:07 PM Markus Thömmes
> wrote:
> ...
> >
> https://cwiki.apache.org/confluence/disp
ocs/akka/2.5/distributed-data.html
>
>
> ____
> From: David P Grove
> Sent: Tuesday, August 14, 2018 2:15:13 PM
> To: dev@openwhisk.apache.org
> Subject: Re: Proposal on a future architecture of OpenWhisk
>
>
>
>
> "Markus Thöm
Hi Dave,
thanks a lot for your input! Greatly appreciated.
Am Di., 14. Aug. 2018 um 23:15 Uhr schrieb David P Grove :
>
>
>
> "Markus Thömmes" wrote on 08/14/2018 10:06:49
> AM:
> >
> > I just published a revision on the initial proposal I made. I sti
Hey OpenWhiskers,
I just published a revision on the initial proposal I made. I still owe a
lot of sequence diagrams for the container distribution, sorry for taking
so long on that, I'm working on it.
I did include a clear seperation of concerns into the proposal, where
user-facing abstractions
Hi Rahul,
thanks a lot for the contribution and the work you've put into this!
Very much appreciated.
I'm not familiar with Karate or BDD at all. Could you go into a little
bit more detail on what the benefit of this approach are and why we
should consider doubling our test-coverage (and thus
I guess this is the famous "context" object, that some other platforms
use in their function signature? @Chetan is that what you're referring
to in option 2?
Am Mo., 6. Aug. 2018 um 10:58 Uhr schrieb Chetan Mehrotra
:
>
> > What are you thinking as an alternarive?
>
> There can be 2 ways
>
> 1.
Ah yeah!
Dividing /init and /run time provided parameters makes a lot of sense
to minimize potential performance impact! +1 on that.
Am Mo., 6. Aug. 2018 um 10:37 Uhr schrieb Rodric Rabbah :
>
> +1 for backward compatibility.
>
> I would also provide the env vars at init time. (Currently they’re
Hi Vadim,
thanks for bringing this up!
If I were to design a new system from the ground up, I'd definitely
introduce a new structure to have a good distinction between
environment and parameters.
Taking into consideration that we need to be backwards compatible, I'd
definitely choose option 1.
ou know if the Kafka client
> library is able to handle both versions ?
>
> Thanks,
> Dragos
> On Mon, Jul 30, 2018 at 12:57 PM Markus Thömmes
> wrote:
>
> > Dear OpenWhiskers,
> >
> > James Dubee, Sven Lange-Last and I have been on a Kafka-related bug
> &g
Thanks Justin,
Yes you can put me on the agenda on knative. Consider it a shared "Ben
Browning/Markus Thömmes - Update on Knative" please!
Cheers,
Markus
Am Mo., 30. Juli 2018 um 23:39 Uhr schrieb Justin Halsall
:
>
> Hi everyone,
>
>
> Reminder that the bi-weekly ca
Dear OpenWhiskers,
James Dubee, Sven Lange-Last and I have been on a Kafka-related bug
hunt throughout the past days and found several issues in our current
KafkaConsumerConnector. Several PRs have been opened since and several
are yet to come.
Currently, I'm trying to make the Kafka 2.0.0
Hi David,
the problem is, that if you have N controllers in the system and M
containers but N > M and all controllers manage their containers
exclusively, you'll end up with controllers not having a container to
manage at all.
There's valid, very slow workload that needs to create only 1
Hi Ben,
thanks for the write-up! For full disclosure to the rest of the
dev-list: I've been
involved with Knative and have contributed some bits especially to
their autoscaling
strategy.
The usual word of caution: Knative is in its very early stages (which
is actually
a good thing, because we
The overflow queue is not intended to be drained by a large scale of invokers
but by a relatively low scale of loadbalancers! I don‘t think rebalancing is a
problem then.
That loadbalancer basically has 2 states:
1. capacity is there: schedule everything you get and put it on immediate
Hi,
I think OpenTracing will be a great way to implement this, so we don't need to
write SPIs for every tracing backend there is but OpenTracing already handles
that for us.
+1
Cheers,
Markus
Am 26. September 2017 um 08:20 schrieb "sandy...@gmail.com"
:
Hi,
sometime
Hi Tyson,
first of all: Thanks for the great effort put into all of these SPI
implementations. Here are some general comments from myself on SPIs
The ContainerFactory SPI feels natural and makes a lot of sense to support
different deployment models. There is no doubt about that one from my
Sounds all sensible to me.
To 1: Sounds like the right thing to do, if versions are common enough? Like
major bumps could probably use different repos but I guess we can just try and
find out what works. As a starter, the list you posted sounds sensible to me!
2. By Integration Tests I meant:
Hey all,
again we're hitting the Travis build time limit of 50 minutes and we need to
disable more tests again to bring us in good shape there. This is not really
acceptable and it's really painful for me to turn down testcoverage.
A thought has been lingering for some time now and with the
which would
involve a request payload passing through a sequence of actions. So I have
a mixed bag of heterogeneous micro services I want to mimic as actions.
I had already gone through your blog earlier.
On Wed, Sep 6, 2017 at 11:27 AM, Markus Thömmes <markusthoem...@me.com>
wrote:
Hi Mandeep,
OpenWhisk relies on container reuse (i.e. using warm containers) heavily to
reach its performance goals. Currently, containers are kept around for a
maximum of 10 minutes if they are not replaced by the need of some other
container.
You can refer to this article written by me
. Introduction/New faces on the call?
2. What's new? Short status updates depending on who's joining the call.
3. Topics:
3.1: Nick Mitchell - OpenWhisk Shell
3.2: Markus Thömmes - Invoker Reactive: What's the fuzz and how does it work?
3.3: Anything else you are interested in!
4. Confirm
Hi all,
development of the reactive containerpool is finished and it's been tested thoroughly in
our (IBM's) pipeline. It is used in the Bluemix hosted OpenWhisk deployment for a while
now which makes me believe we should start the deprecation process for the
"old" containerpool since it's
Right, I think the UI workflows are just an example of apps that are latency
sensitive in general.
I had a discussion with Stephen Fink on the matter of detecting ourselves that
an action is latency sensitive by using the blocking parameter or as mentioned
the user's configuration in terms of
Thanks for the vry detailed writeup! Great job!
One thing I'd note: As Rodric pointed out we should break the issues apart and
address them one by one.
For instance the proposed loadbalancer changes (Controllers knows of Containers
downstream) is desirable for each workload and not
Hey folks,
it's me again with the latest news on performance :).
As some of you probably now: Our current loadbalancer strategy is quite
"simple" and doesn't take load in the system into account at all. It hops to
the next available invoker after you've invoked an action X times (where X is a
Hi Ben,
That sounds awesome :). Welcome to the family and looking forward to working
with all of you.
Cheers,
Markus
Von meinem iPhone gesendet
> Am 31.05.2017 um 22:59 schrieb Ben Browning :
>
> My name is Ben Browning and I lead the Serverless and
>
Hi Tyson,
awesome, we had the same train of thought :). I looked into Gatling as well,
mainly because it's in Scala so it'd be easy for us to read and build new
scenarios. Gatling also has a Jenkins plugin (we use Jenkins) which makes it
very easy to integrate test results.
I also liked how
Hi Tyson,
Sounds like you did a lot of investigation here, thanks a lot for that :)
Seeing the numbers, 4 RPS in the "off" case seem very odd. The Travis build
that runs the current system as is also reaches 40+ RPS. So we'd need to look
at a mismatch here.
Other than that I'd indeed suspect
Hi Michael,
yeah that sounds pretty much spot on. I'd like to have at least 2 VMs with 4+
cores and 8GB memory. One VM would host the management stack while one would be
dedicated to an Invoker only. That way we could assert single-invoker
performance the easiest.
Thanks for helping!
Hi all,
Since many have asked I today started an effort to bring a comprehensive
performance test suite for OpenWhisk out in the open.
Without much talking, here's the repository:
https://github.com/markusthoemmes/openwhisk-performance
My initial idea was to have the tests run on Travis to
Good one!
Another vector where this will improve the system in general is the ability to
run a minimized set of tests for our core components and thus reducing feedback
time.
- mt
Von meinem iPhone gesendet
> Am 05.03.2017 um 17:39 schrieb Rodric Rabbah :
>
> I've been
ts use may be somewhat constrained (until we have
streaming?) by any payload limitations a whisk installation might have in
place.
On Thu, Jan 12, 2017 at 5:51 PM, Markus Thömmes <markusthoem...@me.com>
wrote:
Haven't looked at the implementation yet but I really dig the idea!
Are query pa
89 matches
Mail list logo