hyperbolix opened a new issue, #2896: URL: https://github.com/apache/drill/issues/2896
**Describe the bug** When Apache Drill is configured with the HTTP storage plugin, and INDEX pagination mode is enabled, and the HTTP service configured with that plugin includes a field in all records from page 1, but that field is absent in page 2, a query that spans into page 2 results in an OversizedAllocationException. **To Reproduce** Steps to reproduce the behavior: 1. Start Drill. 2. Start the Python3 HTTP server that demonstrates the issue: https://gist.github.com/hyperbolix/24b1696dbb03c6032fb7af1a06b75145 3. Enable the http storage plugin and configure it using this configuration: https://gist.github.com/hyperbolix/7df108c8169cf54bcc72fdc499be1dce 4. Open the Drill query interface and issue this query: `select * from http.a_bug_test where total_results_val=300 and page_size_val=10 limit 15` 5. See OversizedAllocationException error. Here is an example error: https://gist.github.com/hyperbolix/5e2d1e4e10eb92d5a6fa9604acd5cac8 6. Modify the Python3 HTTP server, commenting out line 46 and fixing indentation on line 47. 7. Restart the Python3 HTTP server and issue the query again from step 4. 8. Observe that now the error does not occur and result span into page 2. **Expected behavior** Drill is supposed to be resilient to flexible schemas with certain limitations. It should be reasonable for a field to be absent on some pages and present on others, or documentation should indicate the contrary. This would result in the response including the data form page 2 and not producing an exception. If this will not be supported the exception should indicate the issue with the result schema, not indicate that a buffer cannot be expanded. **Error detail, log output or screenshots** https://gist.github.com/hyperbolix/5e2d1e4e10eb92d5a6fa9604acd5cac8 `org.apache.drill.common.exceptions.UserException: SYSTEM ERROR: OversizedAllocationException: Unable to expand the buffer. Max allowed buffer size is reached.` **Drill version** Observed in 1.21.1 and also in the latest commit as of 2024-03-26: 749772cb0bd83c1a8fe455410ec80b1e5a9bf239 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@drill.apache.org.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
