[GitHub] [spark] siknezevic commented on a change in pull request #27246: [SPARK-30536][CORE][SQL] Sort-merge join operator spilling performance improvements

siknezevic commented on a change in pull request #27246: URL: https://github.com/apache/spark/pull/27246#discussion_r445307223 ## File path: sql/core/benchmarks/ExternalAppendOnlyUnsafeRowArrayBenchmark-results.txt ## @@ -42,4 +42,8 @@ Spilling with 1 rows:

[GitHub] [spark] siknezevic commented on a change in pull request #27246: [SPARK-30536][CORE][SQL] Sort-merge join operator spilling performance improvements

siknezevic commented on a change in pull request #27246: URL: https://github.com/apache/spark/pull/27246#discussion_r445306120 ## File path: sql/core/benchmarks/ExternalAppendOnlyUnsafeRowArrayBenchmark-results.txt ## @@ -42,4 +42,8 @@ Spilling with 1 rows:

[GitHub] [spark] siknezevic commented on a change in pull request #27246: [SPARK-30536][CORE][SQL] Sort-merge join operator spilling performance improvements

siknezevic commented on a change in pull request #27246: URL: https://github.com/apache/spark/pull/27246#discussion_r445304792 ## File path: sql/core/benchmarks/ExternalAppendOnlyUnsafeRowArrayBenchmark-results.txt ## @@ -42,4 +42,8 @@ Spilling with 1 rows:

[GitHub] [spark] siknezevic commented on a change in pull request #27246: [SPARK-30536][CORE][SQL] Sort-merge join operator spilling performance improvements

siknezevic commented on a change in pull request #27246: URL: https://github.com/apache/spark/pull/27246#discussion_r445299048 ## File path: sql/core/src/test/scala/org/apache/spark/sql/execution/ExternalAppendOnlyUnsafeRowArrayBenchmark.scala ## @@ -182,6 +182,47 @@ object Ex

[GitHub] [spark] siknezevic commented on a change in pull request #27246: [SPARK-30536][CORE][SQL] Sort-merge join operator spilling performance improvements

siknezevic commented on a change in pull request #27246: URL: https://github.com/apache/spark/pull/27246#discussion_r444591210 ## File path: sql/core/benchmarks/ExternalAppendOnlyUnsafeRowArrayBenchmark-results.txt ## @@ -42,4 +42,8 @@ Spilling with 1 rows:

[GitHub] [spark] siknezevic commented on a change in pull request #27246: [SPARK-30536][CORE][SQL] Sort-merge join operator spilling performance improvements

siknezevic commented on a change in pull request #27246: URL: https://github.com/apache/spark/pull/27246#discussion_r444587737 ## File path: sql/core/benchmarks/ExternalAppendOnlyUnsafeRowArrayBenchmark-results.txt ## @@ -42,4 +42,8 @@ Spilling with 1 rows:

[GitHub] [spark] siknezevic commented on a change in pull request #27246: [SPARK-30536][CORE][SQL] Sort-merge join operator spilling performance improvements

siknezevic commented on a change in pull request #27246: URL: https://github.com/apache/spark/pull/27246#discussion_r444585857 ## File path: sql/core/src/test/scala/org/apache/spark/sql/execution/ExternalAppendOnlyUnsafeRowArrayBenchmark.scala ## @@ -182,6 +182,47 @@ object Ex

[GitHub] [spark] siknezevic commented on a change in pull request #27246: [SPARK-30536][CORE][SQL] Sort-merge join operator spilling performance improvements

siknezevic commented on a change in pull request #27246: URL: https://github.com/apache/spark/pull/27246#discussion_r443690502 ## File path: core/src/main/java/org/apache/spark/util/collection/unsafe/sort/UnsafeExternalSorter.java ## @@ -703,6 +702,7 @@ public boolean hasNext(

[GitHub] [spark] siknezevic commented on a change in pull request #27246: [SPARK-30536][CORE][SQL] Sort-merge join operator spilling performance improvements

siknezevic commented on a change in pull request #27246: URL: https://github.com/apache/spark/pull/27246#discussion_r443690725 ## File path: core/src/main/java/org/apache/spark/util/collection/unsafe/sort/UnsafeExternalSorter.java ## @@ -681,20 +681,19 @@ private void moveOver

[GitHub] [spark] siknezevic commented on a change in pull request #27246: [SPARK-30536][CORE][SQL] Sort-merge join operator spilling performance improvements

siknezevic commented on a change in pull request #27246: URL: https://github.com/apache/spark/pull/27246#discussion_r441919575 ## File path: core/src/main/scala/org/apache/spark/internal/config/package.scala ## @@ -1238,6 +1239,18 @@ package object config { s"The value

[GitHub] [spark] siknezevic commented on a change in pull request #27246: [SPARK-30536][CORE][SQL] Sort-merge join operator spilling performance improvements

siknezevic commented on a change in pull request #27246: URL: https://github.com/apache/spark/pull/27246#discussion_r441907490 ## File path: core/src/main/scala/org/apache/spark/internal/config/package.scala ## @@ -1238,6 +1239,18 @@ package object config { s"The value

[GitHub] [spark] siknezevic commented on a change in pull request #27246: [SPARK-30536][CORE][SQL] Sort-merge join operator spilling performance improvements

siknezevic commented on a change in pull request #27246: URL: https://github.com/apache/spark/pull/27246#discussion_r441907490 ## File path: core/src/main/scala/org/apache/spark/internal/config/package.scala ## @@ -1238,6 +1239,18 @@ package object config { s"The value

[GitHub] [spark] siknezevic commented on a change in pull request #27246: [SPARK-30536][CORE][SQL] Sort-merge join operator spilling performance improvements

siknezevic commented on a change in pull request #27246: URL: https://github.com/apache/spark/pull/27246#discussion_r441273689 ## File path: core/src/main/scala/org/apache/spark/internal/config/package.scala ## @@ -1238,6 +1239,18 @@ package object config { s"The value

[GitHub] [spark] siknezevic commented on a change in pull request #27246: [SPARK-30536][CORE][SQL] Sort-merge join operator spilling performance improvements

siknezevic commented on a change in pull request #27246: URL: https://github.com/apache/spark/pull/27246#discussion_r441273689 ## File path: core/src/main/scala/org/apache/spark/internal/config/package.scala ## @@ -1238,6 +1239,18 @@ package object config { s"The value

[GitHub] [spark] siknezevic commented on a change in pull request #27246: [SPARK-30536][CORE][SQL] Sort-merge join operator spilling performance improvements

siknezevic commented on a change in pull request #27246: URL: https://github.com/apache/spark/pull/27246#discussion_r437129696 ## File path: core/src/main/java/org/apache/spark/util/collection/unsafe/sort/UnsafeSorterSpillReader.java ## @@ -47,55 +47,48 @@ private int numRe

[GitHub] [spark] siknezevic commented on a change in pull request #27246: [SPARK-30536][CORE][SQL] Sort-merge join operator spilling performance improvements

siknezevic commented on a change in pull request #27246: URL: https://github.com/apache/spark/pull/27246#discussion_r436385980 ## File path: core/src/main/java/org/apache/spark/util/collection/unsafe/sort/UnsafeSorterSpillReader.java ## @@ -47,55 +47,48 @@ private int numRe

[GitHub] [spark] siknezevic commented on a change in pull request #27246: [SPARK-30536][CORE][SQL] Sort-merge join operator spilling performance improvements

siknezevic commented on a change in pull request #27246: URL: https://github.com/apache/spark/pull/27246#discussion_r436311708 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/ExternalAppendOnlyUnsafeRowArray.scala ## @@ -82,6 +82,8 @@ private[sql] class Ex

[GitHub] [spark] siknezevic commented on a change in pull request #27246: [SPARK-30536][CORE][SQL] Sort-merge join operator spilling performance improvements

siknezevic commented on a change in pull request #27246: URL: https://github.com/apache/spark/pull/27246#discussion_r436227550 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/ExternalAppendOnlyUnsafeRowArray.scala ## @@ -204,20 +199,37 @@ private[sql] clas

[GitHub] [spark] siknezevic commented on a change in pull request #27246: [SPARK-30536][CORE][SQL] Sort-merge join operator spilling performance improvements

siknezevic commented on a change in pull request #27246: URL: https://github.com/apache/spark/pull/27246#discussion_r436226844 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/ExternalAppendOnlyUnsafeRowArray.scala ## @@ -168,7 +159,11 @@ private[sql] class

[GitHub] [spark] siknezevic commented on a change in pull request #27246: [SPARK-30536][CORE][SQL] Sort-merge join operator spilling performance improvements

siknezevic commented on a change in pull request #27246: URL: https://github.com/apache/spark/pull/27246#discussion_r436226766 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/ExternalAppendOnlyUnsafeRowArray.scala ## @@ -124,29 +129,15 @@ private[sql] clas

[GitHub] [spark] siknezevic commented on a change in pull request #27246: [SPARK-30536][CORE][SQL] Sort-merge join operator spilling performance improvements

siknezevic commented on a change in pull request #27246: URL: https://github.com/apache/spark/pull/27246#discussion_r436226636 ## File path: core/src/main/java/org/apache/spark/util/collection/unsafe/sort/UnsafeSorterSpillReader.java ## @@ -148,4 +141,34 @@ public void close()

[GitHub] [spark] siknezevic commented on a change in pull request #27246: [SPARK-30536][CORE][SQL] Sort-merge join operator spilling performance improvements

siknezevic commented on a change in pull request #27246: URL: https://github.com/apache/spark/pull/27246#discussion_r436226544 ## File path: core/src/main/java/org/apache/spark/util/collection/unsafe/sort/UnsafeSorterSpillReader.java ## @@ -148,4 +141,34 @@ public void close()

[GitHub] [spark] siknezevic commented on a change in pull request #27246: [SPARK-30536][CORE][SQL] Sort-merge join operator spilling performance improvements

siknezevic commented on a change in pull request #27246: URL: https://github.com/apache/spark/pull/27246#discussion_r436226351 ## File path: core/src/main/java/org/apache/spark/util/collection/unsafe/sort/UnsafeSorterSpillReader.java ## @@ -47,55 +47,48 @@ private int numRe

[GitHub] [spark] siknezevic commented on a change in pull request #27246: [SPARK-30536][CORE][SQL] Sort-merge join operator spilling performance improvements

siknezevic commented on a change in pull request #27246: URL: https://github.com/apache/spark/pull/27246#discussion_r436226277 ## File path: core/src/main/java/org/apache/spark/util/collection/unsafe/sort/UnsafeExternalSorter.java ## @@ -720,5 +720,14 @@ public void loadNext()

[GitHub] [spark] siknezevic commented on a change in pull request #27246: [SPARK-30536][CORE][SQL] Sort-merge join operator spilling performance improvements

siknezevic commented on a change in pull request #27246: URL: https://github.com/apache/spark/pull/27246#discussion_r436226152 ## File path: core/src/main/java/org/apache/spark/util/collection/unsafe/sort/UnsafeExternalSorter.java ## @@ -703,6 +702,7 @@ public boolean hasNext(

[GitHub] [spark] siknezevic commented on a change in pull request #27246: [SPARK-30536][CORE][SQL] Sort-merge join operator spilling performance improvements

siknezevic commented on a change in pull request #27246: [SPARK-30536][CORE][SQL] Sort-merge join operator spilling performance improvements URL: https://github.com/apache/spark/pull/27246#discussion_r369937789 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/E

[GitHub] [spark] siknezevic commented on a change in pull request #27246: [SPARK-30536][CORE][SQL] Sort-merge join operator spilling performance improvements

siknezevic commented on a change in pull request #27246: [SPARK-30536][CORE][SQL] Sort-merge join operator spilling performance improvements URL: https://github.com/apache/spark/pull/27246#discussion_r369932384 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/E