Github user HanumathRao commented on a diff in the pull request:
    --- Diff: 
    @@ -59,6 +59,7 @@ public static void setup() throws Exception {
         ClusterFixtureBuilder builder = ClusterFixture.builder(dirTestWatcher)
             .configProperty(ExecConstants.EXTERNAL_SORT_SPILL_THRESHOLD, 1) // 
             .configProperty(ExecConstants.EXTERNAL_SORT_SPILL_GROUP_SIZE, 1) 
// Unmanaged
    +        .configProperty(ExecConstants.EXTERNAL_SORT_MAX_MEMORY, 10 * 1024 
* 1024) //use less memory for sorting.
    --- End diff --
    @paul-rogers  I am sorry for not being clear in my previous reply. This 
query has only one sort and also one minor fragment. If we set 60MB as 
MAX_QUERY_MEMORY_PER_NODE_KEY, then the result of the computeOperatorMemory() 
is 60MB.
    However, as I said this memory is sufficient for the query to run 
successfully without spilling.
    Then I tried out setting MAX_QUERY_MEMORY_PER_NODE_KEY to as low as 10MB so 
that this query should spill and fail with an exception (for this testcase). 
But even after setting so low the output of the computeOperatorMemory is 40MB. 
This is because of the default value of MIN_MEMORY_PER_BUFFERED_OP is 40MB. 
Hence, 40MB is used by the sort to sort the rows. This memory is enough for the 
sort to successfully run without an exception.
    Thats when I used the setting EXTERNAL_SORT_MAX_MEMORY to set low values so 
that Sort for sure can spill and throw the exception which is being tested by 
this testcase.


Reply via email to