apucher commented on pull request #6354: URL: https://github.com/apache/incubator-pinot/pull/6354#issuecomment-747828188
some thoughts re API design. I understand that this one is mainly for testing purposes so it probably doesn't matter too much. If this is to ever become a production feature for "quickly uploading some data" it may be worthwhile to avoid the "json blob" as a URL fragment in the upload methods. This could be achieved by either (a) using a JSON payload with the batch configs, as well as a base64-encoded representation of the data, or (b) split out the multipart-upload and launching the ingestion job with "batch configs" as two separate invocations. Both have merit: (a) the JSON one is straight forward and makes this a clean API at the cost of expanding the size of the uploaded data via encoding. If the data is tiny anyways, this would be my suggested approach. (b) if we want to operate the controller like a small temp file server, then might as well have the ability to upload a multipart file. Then, separately, call an API to invoke the ingestion "locally" with a JSON payload that has the configs and a pointer to the uploaded file ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
