jthomas opened a new issue #2542: actionProxy runtime input parameters limited to 128k URL: https://github.com/apache/incubator-openwhisk/issues/2542 ## Environment details: IBM Bluemix (US-South). ## Steps to reproduce the issue: Create a new `openwhisk/dockerskeleton` action and use a JSON string with input parameters larger than 128KB. ``` $ printf 'a%.0s' {1..131072} | echo "\"$(cat)\"" | jq '{input: .}' > test_too_large.json $ wsk action create argmax --docker openwhisk/dockerskeleton $ wsk action invoke argmax -P test_too_large.json -b ``` This invocation fails with the following error logs... ``` 2017-07-28T10:42:39.378372326Z stdout: [Errno 7] Argument list too long ``` This was discovered by an external user (https://github.com/apache/incubator-openwhisk/issues/2465). ## Additional information you deem important: This is happening because the `actionProxy.py` passes the input parameters as a JSON string through a command-line argument. https://github.com/apache/incubator-openwhisk/blob/master/core/actionProxy/actionproxy.py#L131 Linux [limits the maximum argument length](https://stackoverflow.com/questions/29801975/why-is-the-subprocess-popen-argument-length-limit-smaller-than-what-the-os-repor) to `MAX_ARG_STRLEN` which is 128KB on Linux. ## Fixing this issue Changing the `actionProxy.py` to pass input parameters using `stdin` rather than a command-line argument would remove this limit. Here's an example of the change needed: ``` try: input = json.dumps(args) p = subprocess.Popen( [self.binary], stdin=subprocess.PIPE, stdout=subprocess.PIPE, stderr=subprocess.PIPE, env=env) except Exception as e: return error(e) # run the process and wait until it completes. # stdout/stderr will always be set because we passed PIPEs to Popen (o, e) = p.communicate(input=input.encode()) ``` https://gist.github.ibm.com/thomas6/99558d3c64bb8f571fe55c3cdd1b3817 I've built this change into an external image @ `jamesthomas/custom_skeleton` and been using it to resolve this issue in another project. **Do you want me to submit a PR? This would be a breaking change to existing users.** ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected]
With regards, Apache Git Services
