i like the proposed solution. i think this is an important problem to address, from usability perspective. as an example: recently, a client application started to experience bimodal performance: either very fast, or 30.x seconds. the logs gave no indication as to what was going on, and so a bit of a wild goose chase ensued.
side note: are we truly parsing the json results in memory? can't json validation and decoration be done in a streaming fashion? On Fri, Jan 27, 2017 at 10:35 AM, Rodric Rabbah <[email protected]> wrote: > Currently, the size of an action result is not checked by the system and > can exceed the assumed limit of 1MB. This will manifest itself in blocking > invokes will have at least a 30s delay on getting their results. Similarly > larger than allowed invoke time parameters are not rejected as early as > they should be in the pipeline. This is a standing open defect you can read > more about here [1] along with my proposed solution [2] which I include > below for convenience. > > > 1. Response is sized by the invoker. > 1. If it exceeds the system limit, the invoker will reject it. > 2. If it is within the limits, it will be processed as usual. > 2. When the invoker rejects an action response, it will do the > following: > 1. Construct an error response object with a message explaining the > failure > 2. In addition, it will provide as a debugging aid a partial result > that contains as many of the top level fields from the > dictionary response > into the error object. > 3. The error response for large responses shall include: > 1. "Action result of X MB exceeds system limit of Y MB". > 2. The number of top level fields contained in the response and how > many are actually included in the partial response. > > As part of doing this work, I will also apply the same technique to reject > large incoming payloads which currently get rejected a bit later in the > pipeline than they should. > > Comments and feedback welcomed and appreciated. > > -r > > [1] https://github.com/openwhisk/openwhisk/issues/566 > [2] https://github.com/openwhisk/openwhisk/issues/566# > issuecomment-275687757 >
