ecooperman edited a comment on issue #5170:
URL: https://github.com/apache/openwhisk/issues/5170#issuecomment-954793598
@rabbah thanks for responding! I worked with the Adobe I/O support team and
got the root cause of the issue, which appears to be a few different things as
follows:
dcfcd0681d804dafbcd0681d80edafb6 : (Pod eviction) [Pod ephemeral local
storage usage exceeds the total limit of containers 500Mi.]
4163948263e84268a3948263e8b2682e : (the node process may have crashed, or
the network was slow for some time)
50f29217fa9a4d6db29217fa9a7d6d54 : (Concurrent activation failed and
container was terminated) [Activation f27f18a7347146f0bf18a7347116f003 -
{"error":"An error has occurred: Cannot load module '<path to our main.js>':
ReferenceError: config is not defined"} ]
To give a little context, all our application does it to make JS calls
(using the Axios library), to a commerce system. The commerce system returns
data, and we then transform it into what our CMS expects, and return it to the
CMS. So the application effectively works as a middleware/ESL layer between a
CMS and Commerce system in order to integrate them in a way that makes them
feel like a single website. That's all the code does in every possible instance.
So I think this is probably not related to anything that's core to
OpenWhisk, but more that we're not handling some specific cases that I'm not
really sure how to handle. To answer your question, we're not intentional
reusing any containers. I'm new to this, so maybe I'm doing that by accident?
Is there some way I can force each activation to use a new container or
something?
Since we saw this error, I've at least added a timeout to our Axios calls
(30s), with the hopes that at least we won't have any long running calls after
that. I'm just not sure exactly what might be happening and what I can do to
prevent it. My understanding here is that each activation is independent of
all other activations, and so I'm a little confused as to where to even start.
That being said, I'm trying to escalate the issue with Adobe to try and get
some support from them at least. If you have any suggestions though, I'd love
to hear them! For the pod storage, I'm really confused. My code is
transient, and after each activation, the container should be destroyed and
space reclaimed. Is there anything specific I need to do in order to get
storage to clean up?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
