etr2460 edited a comment on issue #10139: URL: https://github.com/apache/incubator-superset/issues/10139#issuecomment-649068628
Echoing @villebro and @ktmud, some sort of backoff sounds good, so that a 0.1 second query comes back in 0.2 or 0.4 seconds, but the 30 second query comes back in 32. I will note that performance around the 1 to 5 second long queries does appear especially bad on Superset's side, especially because of all the steps required to run the query: 1. receive the request 2. schedule a task for a worker 3. pick up the worker task 4. Run the query (polling ever 2 seconds) 5. Write the result to the results backend (s3) 6. Wait for the next poll from the client side (also every 2 seconds I think) 7. Download the results from s3 This means that a query that takes a second or 2 on cli, can take 2-3 times as long in sql lab. Any improvements we can make around this without overloading services seems worth considering ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
