The memory (heap) would climb as it tries to flatten the JSON data. Have you tried looking at Drill's LateralJoin-Unnest feature? It was meant to address memory issues for some use cases of the FLATTEN operator.
On 2/8/2019 5:17:01 AM, PRAVEEN DEVERACHETTY <[email protected]> wrote: I am running a query with UNION ALL. as below select from ( select FLATTEN(t.jdata) as record from ((select convert_from(json string, json) union all (select conver_from(json_string,json) union all ... ) as jdata) ) as t) ems Reason for giving union all is because we are invoking a call using rest app, there is limitation of 20,000 when we use convert_from function. Our heap size is 8GB, server is 8core. From profiling, it shows this perticula query spikes from 100MB to 8GB continuously. is there anything i am doing wrong?. Thanks, Prveen
