[GitHub] [spark] dongjoon-hyun commented on a change in pull request #25972: [SPARK-29300][TESTS] Compare `catalyst` and `avro` module benchmark in JDK8/11
dongjoon-hyun commented on a change in pull request #25972: [SPARK-29300][TESTS] Compare `catalyst` and `avro` module benchmark in JDK8/11 URL: https://github.com/apache/spark/pull/25972#discussion_r329412672 ## File path: external/avro/benchmarks/AvroReadBenchmark-jdk11-results.txt ## @@ -0,0 +1,122 @@ + +SQL Single Numeric Column Scan + + +OpenJDK 64-Bit Server VM 11.0.4+11-LTS on Linux 3.10.0-862.3.2.el7.x86_64 +Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz +SQL Single TINYINT Column Scan: Best Time(ms) Avg Time(ms) Stdev(ms)Rate(M/s) Per Row(ns) Relative + +Sum3039 3047 12 5.2 193.2 1.0X + +OpenJDK 64-Bit Server VM 11.0.4+11-LTS on Linux 3.10.0-862.3.2.el7.x86_64 +Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz +SQL Single SMALLINT Column Scan: Best Time(ms) Avg Time(ms) Stdev(ms)Rate(M/s) Per Row(ns) Relative + +Sum2792 2845 75 5.6 177.5 1.0X + +OpenJDK 64-Bit Server VM 11.0.4+11-LTS on Linux 3.10.0-862.3.2.el7.x86_64 +Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz +SQL Single INT Column Scan: Best Time(ms) Avg Time(ms) Stdev(ms)Rate(M/s) Per Row(ns) Relative + +Sum2780 2788 11 5.7 176.7 1.0X + +OpenJDK 64-Bit Server VM 11.0.4+11-LTS on Linux 3.10.0-862.3.2.el7.x86_64 +Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz +SQL Single BIGINT Column Scan:Best Time(ms) Avg Time(ms) Stdev(ms)Rate(M/s) Per Row(ns) Relative + +Sum3068 3141 103 5.1 195.1 1.0X + +OpenJDK 64-Bit Server VM 11.0.4+11-LTS on Linux 3.10.0-862.3.2.el7.x86_64 +Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz +SQL Single FLOAT Column Scan: Best Time(ms) Avg Time(ms) Stdev(ms)Rate(M/s) Per Row(ns) Relative + +Sum2650 2664 20 5.9 168.5 1.0X + +OpenJDK 64-Bit Server VM 11.0.4+11-LTS on Linux 3.10.0-862.3.2.el7.x86_64 +Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz +SQL Single DOUBLE Column Scan:Best Time(ms) Avg Time(ms) Stdev(ms)Rate(M/s) Per Row(ns) Relative + +Sum2797 2799 3 5.6 177.8 1.0X + + + +Int and String Scan + + +OpenJDK 64-Bit Server VM 11.0.4+11-LTS on Linux 3.10.0-862.3.2.el7.x86_64 +Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz +Int and String Scan: Best Time(ms) Avg Time(ms) Stdev(ms)Rate(M/s) Per Row(ns) Relative + +Sum of columns 4446 4446 0 2.4 424.0 1.0X + + + +Partitioned Table Scan + + +OpenJDK 64-Bit Server VM 11.0.4+11-LTS on Linux 3.10.0-862.3.2.el7.x86_64 +Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz +Partitioned Table:Best Time(ms) Avg Time(ms) Stdev(ms)Rate(M/s) Per Row(ns) Relative + +Data column3071 3075 5 5.1 195.2 1.0X +Partition column 3155 3161 7
[GitHub] [spark] dongjoon-hyun commented on a change in pull request #25972: [SPARK-29300][TESTS] Compare `catalyst` and `avro` module benchmark in JDK8/11
dongjoon-hyun commented on a change in pull request #25972: [SPARK-29300][TESTS] Compare `catalyst` and `avro` module benchmark in JDK8/11 URL: https://github.com/apache/spark/pull/25972#discussion_r329415098 ## File path: sql/catalyst/benchmarks/UnsafeProjectionBenchmark-jdk11-results.txt ## @@ -0,0 +1,14 @@ + +unsafe projection Review comment: Although the ratio is the same, JDK11 seems to run very slightly slower. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on a change in pull request #25972: [SPARK-29300][TESTS] Compare `catalyst` and `avro` module benchmark in JDK8/11
dongjoon-hyun commented on a change in pull request #25972: [SPARK-29300][TESTS] Compare `catalyst` and `avro` module benchmark in JDK8/11 URL: https://github.com/apache/spark/pull/25972#discussion_r329415098 ## File path: sql/catalyst/benchmarks/UnsafeProjectionBenchmark-jdk11-results.txt ## @@ -0,0 +1,14 @@ + +unsafe projection Review comment: Although the ratio is the same, JDK11 seems to run slightly slower. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on a change in pull request #25972: [SPARK-29300][TESTS] Compare `catalyst` and `avro` module benchmark in JDK8/11
dongjoon-hyun commented on a change in pull request #25972: [SPARK-29300][TESTS] Compare `catalyst` and `avro` module benchmark in JDK8/11 URL: https://github.com/apache/spark/pull/25972#discussion_r329413882 ## File path: sql/catalyst/benchmarks/UnsafeProjectionBenchmark-jdk11-results.txt ## @@ -0,0 +1,14 @@ + +unsafe projection + + +OpenJDK 64-Bit Server VM 11.0.4+11-LTS on Linux 3.10.0-862.3.2.el7.x86_64 +Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz +unsafe projection:Best Time(ms) Avg Time(ms) Stdev(ms)Rate(M/s) Per Row(ns) Relative + +single long2218 2219 1121.0 8.3 1.0X +single nullable long 3200 3201 1 83.9 11.9 0.7X +7 primitive types 7809 7813 5 34.4 29.1 0.3X Review comment: The ratio is the same although this is slightly slower. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on a change in pull request #25972: [SPARK-29300][TESTS] Compare `catalyst` and `avro` module benchmark in JDK8/11
dongjoon-hyun commented on a change in pull request #25972: [SPARK-29300][TESTS] Compare `catalyst` and `avro` module benchmark in JDK8/11 URL: https://github.com/apache/spark/pull/25972#discussion_r329413882 ## File path: sql/catalyst/benchmarks/UnsafeProjectionBenchmark-jdk11-results.txt ## @@ -0,0 +1,14 @@ + +unsafe projection + + +OpenJDK 64-Bit Server VM 11.0.4+11-LTS on Linux 3.10.0-862.3.2.el7.x86_64 +Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz +unsafe projection:Best Time(ms) Avg Time(ms) Stdev(ms)Rate(M/s) Per Row(ns) Relative + +single long2218 2219 1121.0 8.3 1.0X +single nullable long 3200 3201 1 83.9 11.9 0.7X +7 primitive types 7809 7813 5 34.4 29.1 0.3X Review comment: The ratio is the same although this is slightly slower. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on a change in pull request #25972: [SPARK-29300][TESTS] Compare `catalyst` and `avro` module benchmark in JDK8/11
dongjoon-hyun commented on a change in pull request #25972: [SPARK-29300][TESTS] Compare `catalyst` and `avro` module benchmark in JDK8/11 URL: https://github.com/apache/spark/pull/25972#discussion_r329412672 ## File path: external/avro/benchmarks/AvroReadBenchmark-jdk11-results.txt ## @@ -0,0 +1,122 @@ + +SQL Single Numeric Column Scan + + +OpenJDK 64-Bit Server VM 11.0.4+11-LTS on Linux 3.10.0-862.3.2.el7.x86_64 +Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz +SQL Single TINYINT Column Scan: Best Time(ms) Avg Time(ms) Stdev(ms)Rate(M/s) Per Row(ns) Relative + +Sum3039 3047 12 5.2 193.2 1.0X + +OpenJDK 64-Bit Server VM 11.0.4+11-LTS on Linux 3.10.0-862.3.2.el7.x86_64 +Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz +SQL Single SMALLINT Column Scan: Best Time(ms) Avg Time(ms) Stdev(ms)Rate(M/s) Per Row(ns) Relative + +Sum2792 2845 75 5.6 177.5 1.0X + +OpenJDK 64-Bit Server VM 11.0.4+11-LTS on Linux 3.10.0-862.3.2.el7.x86_64 +Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz +SQL Single INT Column Scan: Best Time(ms) Avg Time(ms) Stdev(ms)Rate(M/s) Per Row(ns) Relative + +Sum2780 2788 11 5.7 176.7 1.0X + +OpenJDK 64-Bit Server VM 11.0.4+11-LTS on Linux 3.10.0-862.3.2.el7.x86_64 +Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz +SQL Single BIGINT Column Scan:Best Time(ms) Avg Time(ms) Stdev(ms)Rate(M/s) Per Row(ns) Relative + +Sum3068 3141 103 5.1 195.1 1.0X + +OpenJDK 64-Bit Server VM 11.0.4+11-LTS on Linux 3.10.0-862.3.2.el7.x86_64 +Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz +SQL Single FLOAT Column Scan: Best Time(ms) Avg Time(ms) Stdev(ms)Rate(M/s) Per Row(ns) Relative + +Sum2650 2664 20 5.9 168.5 1.0X + +OpenJDK 64-Bit Server VM 11.0.4+11-LTS on Linux 3.10.0-862.3.2.el7.x86_64 +Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz +SQL Single DOUBLE Column Scan:Best Time(ms) Avg Time(ms) Stdev(ms)Rate(M/s) Per Row(ns) Relative + +Sum2797 2799 3 5.6 177.8 1.0X + + + +Int and String Scan + + +OpenJDK 64-Bit Server VM 11.0.4+11-LTS on Linux 3.10.0-862.3.2.el7.x86_64 +Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz +Int and String Scan: Best Time(ms) Avg Time(ms) Stdev(ms)Rate(M/s) Per Row(ns) Relative + +Sum of columns 4446 4446 0 2.4 424.0 1.0X + + + +Partitioned Table Scan + + +OpenJDK 64-Bit Server VM 11.0.4+11-LTS on Linux 3.10.0-862.3.2.el7.x86_64 +Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz +Partitioned Table:Best Time(ms) Avg Time(ms) Stdev(ms)Rate(M/s) Per Row(ns) Relative + +Data column3071 3075 5 5.1 195.2 1.0X +Partition column 3155 3161 7
[GitHub] [spark] dongjoon-hyun commented on a change in pull request #25972: [SPARK-29300][TESTS] Compare `catalyst` and `avro` module benchmark in JDK8/11
dongjoon-hyun commented on a change in pull request #25972: [SPARK-29300][TESTS] Compare `catalyst` and `avro` module benchmark in JDK8/11 URL: https://github.com/apache/spark/pull/25972#discussion_r329412672 ## File path: external/avro/benchmarks/AvroReadBenchmark-jdk11-results.txt ## @@ -0,0 +1,122 @@ + +SQL Single Numeric Column Scan + + +OpenJDK 64-Bit Server VM 11.0.4+11-LTS on Linux 3.10.0-862.3.2.el7.x86_64 +Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz +SQL Single TINYINT Column Scan: Best Time(ms) Avg Time(ms) Stdev(ms)Rate(M/s) Per Row(ns) Relative + +Sum3039 3047 12 5.2 193.2 1.0X + +OpenJDK 64-Bit Server VM 11.0.4+11-LTS on Linux 3.10.0-862.3.2.el7.x86_64 +Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz +SQL Single SMALLINT Column Scan: Best Time(ms) Avg Time(ms) Stdev(ms)Rate(M/s) Per Row(ns) Relative + +Sum2792 2845 75 5.6 177.5 1.0X + +OpenJDK 64-Bit Server VM 11.0.4+11-LTS on Linux 3.10.0-862.3.2.el7.x86_64 +Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz +SQL Single INT Column Scan: Best Time(ms) Avg Time(ms) Stdev(ms)Rate(M/s) Per Row(ns) Relative + +Sum2780 2788 11 5.7 176.7 1.0X + +OpenJDK 64-Bit Server VM 11.0.4+11-LTS on Linux 3.10.0-862.3.2.el7.x86_64 +Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz +SQL Single BIGINT Column Scan:Best Time(ms) Avg Time(ms) Stdev(ms)Rate(M/s) Per Row(ns) Relative + +Sum3068 3141 103 5.1 195.1 1.0X + +OpenJDK 64-Bit Server VM 11.0.4+11-LTS on Linux 3.10.0-862.3.2.el7.x86_64 +Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz +SQL Single FLOAT Column Scan: Best Time(ms) Avg Time(ms) Stdev(ms)Rate(M/s) Per Row(ns) Relative + +Sum2650 2664 20 5.9 168.5 1.0X + +OpenJDK 64-Bit Server VM 11.0.4+11-LTS on Linux 3.10.0-862.3.2.el7.x86_64 +Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz +SQL Single DOUBLE Column Scan:Best Time(ms) Avg Time(ms) Stdev(ms)Rate(M/s) Per Row(ns) Relative + +Sum2797 2799 3 5.6 177.8 1.0X + + + +Int and String Scan + + +OpenJDK 64-Bit Server VM 11.0.4+11-LTS on Linux 3.10.0-862.3.2.el7.x86_64 +Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz +Int and String Scan: Best Time(ms) Avg Time(ms) Stdev(ms)Rate(M/s) Per Row(ns) Relative + +Sum of columns 4446 4446 0 2.4 424.0 1.0X + + + +Partitioned Table Scan + + +OpenJDK 64-Bit Server VM 11.0.4+11-LTS on Linux 3.10.0-862.3.2.el7.x86_64 +Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz +Partitioned Table:Best Time(ms) Avg Time(ms) Stdev(ms)Rate(M/s) Per Row(ns) Relative + +Data column3071 3075 5 5.1 195.2 1.0X +Partition column 3155 3161 7
[GitHub] [spark] dongjoon-hyun commented on a change in pull request #25972: [SPARK-29300][TESTS] Compare `catalyst` and `avro` module benchmark in JDK8/11
dongjoon-hyun commented on a change in pull request #25972: [SPARK-29300][TESTS] Compare `catalyst` and `avro` module benchmark in JDK8/11 URL: https://github.com/apache/spark/pull/25972#discussion_r329413882 ## File path: sql/catalyst/benchmarks/UnsafeProjectionBenchmark-jdk11-results.txt ## @@ -0,0 +1,14 @@ + +unsafe projection + + +OpenJDK 64-Bit Server VM 11.0.4+11-LTS on Linux 3.10.0-862.3.2.el7.x86_64 +Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz +unsafe projection:Best Time(ms) Avg Time(ms) Stdev(ms)Rate(M/s) Per Row(ns) Relative + +single long2218 2219 1121.0 8.3 1.0X +single nullable long 3200 3201 1 83.9 11.9 0.7X +7 primitive types 7809 7813 5 34.4 29.1 0.3X Review comment: The ratio is the same, but this is slightly slower. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on a change in pull request #25972: [SPARK-29300][TESTS] Compare `catalyst` and `avro` module benchmark in JDK8/11
dongjoon-hyun commented on a change in pull request #25972: [SPARK-29300][TESTS] Compare `catalyst` and `avro` module benchmark in JDK8/11 URL: https://github.com/apache/spark/pull/25972#discussion_r329413253 ## File path: external/avro/benchmarks/AvroWriteBenchmark-jdk11-results.txt ## @@ -0,0 +1,10 @@ +OpenJDK 64-Bit Server VM 11.0.4+11-LTS on Linux 3.10.0-862.3.2.el7.x86_64 Review comment: JDK11 is faster. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on a change in pull request #25972: [SPARK-29300][TESTS] Compare `catalyst` and `avro` module benchmark in JDK8/11
dongjoon-hyun commented on a change in pull request #25972: [SPARK-29300][TESTS] Compare `catalyst` and `avro` module benchmark in JDK8/11 URL: https://github.com/apache/spark/pull/25972#discussion_r329412981 ## File path: external/avro/benchmarks/AvroReadBenchmark-jdk11-results.txt ## @@ -0,0 +1,122 @@ + +SQL Single Numeric Column Scan + + +OpenJDK 64-Bit Server VM 11.0.4+11-LTS on Linux 3.10.0-862.3.2.el7.x86_64 +Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz +SQL Single TINYINT Column Scan: Best Time(ms) Avg Time(ms) Stdev(ms)Rate(M/s) Per Row(ns) Relative + +Sum3039 3047 12 5.2 193.2 1.0X + +OpenJDK 64-Bit Server VM 11.0.4+11-LTS on Linux 3.10.0-862.3.2.el7.x86_64 +Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz +SQL Single SMALLINT Column Scan: Best Time(ms) Avg Time(ms) Stdev(ms)Rate(M/s) Per Row(ns) Relative + +Sum2792 2845 75 5.6 177.5 1.0X + +OpenJDK 64-Bit Server VM 11.0.4+11-LTS on Linux 3.10.0-862.3.2.el7.x86_64 +Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz +SQL Single INT Column Scan: Best Time(ms) Avg Time(ms) Stdev(ms)Rate(M/s) Per Row(ns) Relative + +Sum2780 2788 11 5.7 176.7 1.0X + +OpenJDK 64-Bit Server VM 11.0.4+11-LTS on Linux 3.10.0-862.3.2.el7.x86_64 +Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz +SQL Single BIGINT Column Scan:Best Time(ms) Avg Time(ms) Stdev(ms)Rate(M/s) Per Row(ns) Relative + +Sum3068 3141 103 5.1 195.1 1.0X + +OpenJDK 64-Bit Server VM 11.0.4+11-LTS on Linux 3.10.0-862.3.2.el7.x86_64 +Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz +SQL Single FLOAT Column Scan: Best Time(ms) Avg Time(ms) Stdev(ms)Rate(M/s) Per Row(ns) Relative + +Sum2650 2664 20 5.9 168.5 1.0X + +OpenJDK 64-Bit Server VM 11.0.4+11-LTS on Linux 3.10.0-862.3.2.el7.x86_64 +Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz +SQL Single DOUBLE Column Scan:Best Time(ms) Avg Time(ms) Stdev(ms)Rate(M/s) Per Row(ns) Relative + +Sum2797 2799 3 5.6 177.8 1.0X + + + +Int and String Scan + + +OpenJDK 64-Bit Server VM 11.0.4+11-LTS on Linux 3.10.0-862.3.2.el7.x86_64 +Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz +Int and String Scan: Best Time(ms) Avg Time(ms) Stdev(ms)Rate(M/s) Per Row(ns) Relative + +Sum of columns 4446 4446 0 2.4 424.0 1.0X + + + +Partitioned Table Scan + + +OpenJDK 64-Bit Server VM 11.0.4+11-LTS on Linux 3.10.0-862.3.2.el7.x86_64 +Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz +Partitioned Table:Best Time(ms) Avg Time(ms) Stdev(ms)Rate(M/s) Per Row(ns) Relative + +Data column3071 3075 5 5.1 195.2 1.0X +Partition column 3155 3161 7
[GitHub] [spark] dongjoon-hyun commented on a change in pull request #25972: [SPARK-29300][TESTS] Compare `catalyst` and `avro` module benchmark in JDK8/11
dongjoon-hyun commented on a change in pull request #25972: [SPARK-29300][TESTS] Compare `catalyst` and `avro` module benchmark in JDK8/11 URL: https://github.com/apache/spark/pull/25972#discussion_r329412981 ## File path: external/avro/benchmarks/AvroReadBenchmark-jdk11-results.txt ## @@ -0,0 +1,122 @@ + +SQL Single Numeric Column Scan + + +OpenJDK 64-Bit Server VM 11.0.4+11-LTS on Linux 3.10.0-862.3.2.el7.x86_64 +Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz +SQL Single TINYINT Column Scan: Best Time(ms) Avg Time(ms) Stdev(ms)Rate(M/s) Per Row(ns) Relative + +Sum3039 3047 12 5.2 193.2 1.0X + +OpenJDK 64-Bit Server VM 11.0.4+11-LTS on Linux 3.10.0-862.3.2.el7.x86_64 +Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz +SQL Single SMALLINT Column Scan: Best Time(ms) Avg Time(ms) Stdev(ms)Rate(M/s) Per Row(ns) Relative + +Sum2792 2845 75 5.6 177.5 1.0X + +OpenJDK 64-Bit Server VM 11.0.4+11-LTS on Linux 3.10.0-862.3.2.el7.x86_64 +Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz +SQL Single INT Column Scan: Best Time(ms) Avg Time(ms) Stdev(ms)Rate(M/s) Per Row(ns) Relative + +Sum2780 2788 11 5.7 176.7 1.0X + +OpenJDK 64-Bit Server VM 11.0.4+11-LTS on Linux 3.10.0-862.3.2.el7.x86_64 +Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz +SQL Single BIGINT Column Scan:Best Time(ms) Avg Time(ms) Stdev(ms)Rate(M/s) Per Row(ns) Relative + +Sum3068 3141 103 5.1 195.1 1.0X + +OpenJDK 64-Bit Server VM 11.0.4+11-LTS on Linux 3.10.0-862.3.2.el7.x86_64 +Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz +SQL Single FLOAT Column Scan: Best Time(ms) Avg Time(ms) Stdev(ms)Rate(M/s) Per Row(ns) Relative + +Sum2650 2664 20 5.9 168.5 1.0X + +OpenJDK 64-Bit Server VM 11.0.4+11-LTS on Linux 3.10.0-862.3.2.el7.x86_64 +Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz +SQL Single DOUBLE Column Scan:Best Time(ms) Avg Time(ms) Stdev(ms)Rate(M/s) Per Row(ns) Relative + +Sum2797 2799 3 5.6 177.8 1.0X + + + +Int and String Scan + + +OpenJDK 64-Bit Server VM 11.0.4+11-LTS on Linux 3.10.0-862.3.2.el7.x86_64 +Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz +Int and String Scan: Best Time(ms) Avg Time(ms) Stdev(ms)Rate(M/s) Per Row(ns) Relative + +Sum of columns 4446 4446 0 2.4 424.0 1.0X + + + +Partitioned Table Scan + + +OpenJDK 64-Bit Server VM 11.0.4+11-LTS on Linux 3.10.0-862.3.2.el7.x86_64 +Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz +Partitioned Table:Best Time(ms) Avg Time(ms) Stdev(ms)Rate(M/s) Per Row(ns) Relative + +Data column3071 3075 5 5.1 195.2 1.0X +Partition column 3155 3161 7
[GitHub] [spark] dongjoon-hyun commented on a change in pull request #25972: [SPARK-29300][TESTS] Compare `catalyst` and `avro` module benchmark in JDK8/11
dongjoon-hyun commented on a change in pull request #25972: [SPARK-29300][TESTS] Compare `catalyst` and `avro` module benchmark in JDK8/11 URL: https://github.com/apache/spark/pull/25972#discussion_r329412672 ## File path: external/avro/benchmarks/AvroReadBenchmark-jdk11-results.txt ## @@ -0,0 +1,122 @@ + +SQL Single Numeric Column Scan + + +OpenJDK 64-Bit Server VM 11.0.4+11-LTS on Linux 3.10.0-862.3.2.el7.x86_64 +Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz +SQL Single TINYINT Column Scan: Best Time(ms) Avg Time(ms) Stdev(ms)Rate(M/s) Per Row(ns) Relative + +Sum3039 3047 12 5.2 193.2 1.0X + +OpenJDK 64-Bit Server VM 11.0.4+11-LTS on Linux 3.10.0-862.3.2.el7.x86_64 +Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz +SQL Single SMALLINT Column Scan: Best Time(ms) Avg Time(ms) Stdev(ms)Rate(M/s) Per Row(ns) Relative + +Sum2792 2845 75 5.6 177.5 1.0X + +OpenJDK 64-Bit Server VM 11.0.4+11-LTS on Linux 3.10.0-862.3.2.el7.x86_64 +Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz +SQL Single INT Column Scan: Best Time(ms) Avg Time(ms) Stdev(ms)Rate(M/s) Per Row(ns) Relative + +Sum2780 2788 11 5.7 176.7 1.0X + +OpenJDK 64-Bit Server VM 11.0.4+11-LTS on Linux 3.10.0-862.3.2.el7.x86_64 +Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz +SQL Single BIGINT Column Scan:Best Time(ms) Avg Time(ms) Stdev(ms)Rate(M/s) Per Row(ns) Relative + +Sum3068 3141 103 5.1 195.1 1.0X + +OpenJDK 64-Bit Server VM 11.0.4+11-LTS on Linux 3.10.0-862.3.2.el7.x86_64 +Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz +SQL Single FLOAT Column Scan: Best Time(ms) Avg Time(ms) Stdev(ms)Rate(M/s) Per Row(ns) Relative + +Sum2650 2664 20 5.9 168.5 1.0X + +OpenJDK 64-Bit Server VM 11.0.4+11-LTS on Linux 3.10.0-862.3.2.el7.x86_64 +Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz +SQL Single DOUBLE Column Scan:Best Time(ms) Avg Time(ms) Stdev(ms)Rate(M/s) Per Row(ns) Relative + +Sum2797 2799 3 5.6 177.8 1.0X + + + +Int and String Scan + + +OpenJDK 64-Bit Server VM 11.0.4+11-LTS on Linux 3.10.0-862.3.2.el7.x86_64 +Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz +Int and String Scan: Best Time(ms) Avg Time(ms) Stdev(ms)Rate(M/s) Per Row(ns) Relative + +Sum of columns 4446 4446 0 2.4 424.0 1.0X + + + +Partitioned Table Scan + + +OpenJDK 64-Bit Server VM 11.0.4+11-LTS on Linux 3.10.0-862.3.2.el7.x86_64 +Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz +Partitioned Table:Best Time(ms) Avg Time(ms) Stdev(ms)Rate(M/s) Per Row(ns) Relative + +Data column3071 3075 5 5.1 195.2 1.0X +Partition column 3155 3161 7
[GitHub] [spark] dongjoon-hyun commented on a change in pull request #25972: [SPARK-29300][TESTS] Compare `catalyst` and `avro` module benchmark in JDK8/11
dongjoon-hyun commented on a change in pull request #25972: [SPARK-29300][TESTS] Compare `catalyst` and `avro` module benchmark in JDK8/11 URL: https://github.com/apache/spark/pull/25972#discussion_r329412672 ## File path: external/avro/benchmarks/AvroReadBenchmark-jdk11-results.txt ## @@ -0,0 +1,122 @@ + +SQL Single Numeric Column Scan + + +OpenJDK 64-Bit Server VM 11.0.4+11-LTS on Linux 3.10.0-862.3.2.el7.x86_64 +Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz +SQL Single TINYINT Column Scan: Best Time(ms) Avg Time(ms) Stdev(ms)Rate(M/s) Per Row(ns) Relative + +Sum3039 3047 12 5.2 193.2 1.0X + +OpenJDK 64-Bit Server VM 11.0.4+11-LTS on Linux 3.10.0-862.3.2.el7.x86_64 +Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz +SQL Single SMALLINT Column Scan: Best Time(ms) Avg Time(ms) Stdev(ms)Rate(M/s) Per Row(ns) Relative + +Sum2792 2845 75 5.6 177.5 1.0X + +OpenJDK 64-Bit Server VM 11.0.4+11-LTS on Linux 3.10.0-862.3.2.el7.x86_64 +Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz +SQL Single INT Column Scan: Best Time(ms) Avg Time(ms) Stdev(ms)Rate(M/s) Per Row(ns) Relative + +Sum2780 2788 11 5.7 176.7 1.0X + +OpenJDK 64-Bit Server VM 11.0.4+11-LTS on Linux 3.10.0-862.3.2.el7.x86_64 +Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz +SQL Single BIGINT Column Scan:Best Time(ms) Avg Time(ms) Stdev(ms)Rate(M/s) Per Row(ns) Relative + +Sum3068 3141 103 5.1 195.1 1.0X + +OpenJDK 64-Bit Server VM 11.0.4+11-LTS on Linux 3.10.0-862.3.2.el7.x86_64 +Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz +SQL Single FLOAT Column Scan: Best Time(ms) Avg Time(ms) Stdev(ms)Rate(M/s) Per Row(ns) Relative + +Sum2650 2664 20 5.9 168.5 1.0X + +OpenJDK 64-Bit Server VM 11.0.4+11-LTS on Linux 3.10.0-862.3.2.el7.x86_64 +Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz +SQL Single DOUBLE Column Scan:Best Time(ms) Avg Time(ms) Stdev(ms)Rate(M/s) Per Row(ns) Relative + +Sum2797 2799 3 5.6 177.8 1.0X + + + +Int and String Scan + + +OpenJDK 64-Bit Server VM 11.0.4+11-LTS on Linux 3.10.0-862.3.2.el7.x86_64 +Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz +Int and String Scan: Best Time(ms) Avg Time(ms) Stdev(ms)Rate(M/s) Per Row(ns) Relative + +Sum of columns 4446 4446 0 2.4 424.0 1.0X + + + +Partitioned Table Scan + + +OpenJDK 64-Bit Server VM 11.0.4+11-LTS on Linux 3.10.0-862.3.2.el7.x86_64 +Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz +Partitioned Table:Best Time(ms) Avg Time(ms) Stdev(ms)Rate(M/s) Per Row(ns) Relative + +Data column3071 3075 5 5.1 195.2 1.0X +Partition column 3155 3161 7
[GitHub] [spark] AmplabJenkins removed a comment on issue #25972: [SPARK-29300][TESTS] Compare `catalyst` and `avro` module benchmark in JDK8/11
AmplabJenkins removed a comment on issue #25972: [SPARK-29300][TESTS] Compare `catalyst` and `avro` module benchmark in JDK8/11 URL: https://github.com/apache/spark/pull/25972#issuecomment-536405803 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] yaooqinn commented on issue #25962: [SPARK-29285][Shuffle] Temporary shuffle files should be able to handle disk failures
yaooqinn commented on issue #25962: [SPARK-29285][Shuffle] Temporary shuffle files should be able to handle disk failures URL: https://github.com/apache/spark/pull/25962#issuecomment-536406064 cc @cloud-fan This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #25972: [SPARK-29300][TESTS] Compare `catalyst` and `avro` module benchmark in JDK8/11
AmplabJenkins removed a comment on issue #25972: [SPARK-29300][TESTS] Compare `catalyst` and `avro` module benchmark in JDK8/11 URL: https://github.com/apache/spark/pull/25972#issuecomment-536405809 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/16600/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #25972: [SPARK-29300][TESTS] Compare `catalyst` and `avro` module benchmark in JDK8/11
AmplabJenkins commented on issue #25972: [SPARK-29300][TESTS] Compare `catalyst` and `avro` module benchmark in JDK8/11 URL: https://github.com/apache/spark/pull/25972#issuecomment-536405809 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/16600/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #25972: [SPARK-29300][TESTS] Compare `catalyst` and `avro` module benchmark in JDK8/11
AmplabJenkins commented on issue #25972: [SPARK-29300][TESTS] Compare `catalyst` and `avro` module benchmark in JDK8/11 URL: https://github.com/apache/spark/pull/25972#issuecomment-536405803 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #25972: [SPARK-29300][TESTS] Compare `catalyst` and `avro` module benchmark in JDK8/11
SparkQA commented on issue #25972: [SPARK-29300][TESTS] Compare `catalyst` and `avro` module benchmark in JDK8/11 URL: https://github.com/apache/spark/pull/25972#issuecomment-536405575 **[Test build #111597 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/111597/testReport)** for PR 25972 at commit [`8627495`](https://github.com/apache/spark/commit/8627495093569898edc78192e2cdae6cc246a212). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] dongjoon-hyun opened a new pull request #25972: [SPARK-29300][TESTS] Compare `catalyst` and `avro` module benchmark in JDK8/11
dongjoon-hyun opened a new pull request #25972: [SPARK-29300][TESTS] Compare `catalyst` and `avro` module benchmark in JDK8/11 URL: https://github.com/apache/spark/pull/25972 ### What changes were proposed in this pull request? ### Why are the changes needed? ### Does this PR introduce any user-facing change? ### How was this patch tested? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on a change in pull request #25969: [SPARK-29297][TESTS] Compare `core`/`mllib` module benchmarks in JDK8/11
dongjoon-hyun commented on a change in pull request #25969: [SPARK-29297][TESTS] Compare `core`/`mllib` module benchmarks in JDK8/11 URL: https://github.com/apache/spark/pull/25969#discussion_r329410259 ## File path: core/benchmarks/PropertiesCloneBenchmark-jdk11-results.txt ## @@ -6,35 +6,35 @@ OpenJDK 64-Bit Server VM 11.0.4+11-LTS on Linux 3.10.0-862.3.2.el7.x86_64 Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz Empty Properties: Best Time(ms) Avg Time(ms) Stdev(ms)Rate(M/s) Per Row(ns) Relative -SerializationUtils.clone 0 0 0 0.1 13755.0 1.0X -Utils.cloneProperties 0 0 0 3.5 285.0 48.3X +SerializationUtils.clone 0 0 0 0.1 11539.0 1.0X +Utils.cloneProperties 0 0 0 1.7 572.0 20.2X Review comment: Yes. The ratio is different due to the underlying machine change. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on a change in pull request #25969: [SPARK-29297][TESTS] Compare `core`/`mllib` module benchmarks in JDK8/11
dongjoon-hyun commented on a change in pull request #25969: [SPARK-29297][TESTS] Compare `core`/`mllib` module benchmarks in JDK8/11 URL: https://github.com/apache/spark/pull/25969#discussion_r329410154 ## File path: core/benchmarks/PropertiesCloneBenchmark-results.txt ## @@ -2,39 +2,39 @@ Properties Cloning -Java HotSpot(TM) 64-Bit Server VM 1.8.0_131-b11 on Mac OS X 10.14.6 -Intel(R) Core(TM) i9-8950HK CPU @ 2.90GHz +OpenJDK 64-Bit Server VM 1.8.0_222-b10 on Linux 3.10.0-862.3.2.el7.x86_64 +Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz Empty Properties: Best Time(ms) Avg Time(ms) Stdev(ms)Rate(M/s) Per Row(ns) Relative -SerializationUtils.clone 0 0 0 0.24184.0 1.0X -Utils.cloneProperties 0 0 0 55.6 18.0 232.4X +SerializationUtils.clone 0 0 0 0.1 13640.0 1.0X +Utils.cloneProperties 0 0 0 1.6 608.0 22.4X Review comment: Yep. Thanks~ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #25933: [SPARK-29252][BUILD] Upgrade zookeeper to 3.4.14 and fix vulnerabilities.
AmplabJenkins commented on issue #25933: [SPARK-29252][BUILD] Upgrade zookeeper to 3.4.14 and fix vulnerabilities. URL: https://github.com/apache/spark/pull/25933#issuecomment-536402944 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #25933: [SPARK-29252][BUILD] Upgrade zookeeper to 3.4.14 and fix vulnerabilities.
AmplabJenkins removed a comment on issue #25933: [SPARK-29252][BUILD] Upgrade zookeeper to 3.4.14 and fix vulnerabilities. URL: https://github.com/apache/spark/pull/25933#issuecomment-536402944 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #25933: [SPARK-29252][BUILD] Upgrade zookeeper to 3.4.14 and fix vulnerabilities.
AmplabJenkins removed a comment on issue #25933: [SPARK-29252][BUILD] Upgrade zookeeper to 3.4.14 and fix vulnerabilities. URL: https://github.com/apache/spark/pull/25933#issuecomment-536402950 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/111586/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #25933: [SPARK-29252][BUILD] Upgrade zookeeper to 3.4.14 and fix vulnerabilities.
AmplabJenkins commented on issue #25933: [SPARK-29252][BUILD] Upgrade zookeeper to 3.4.14 and fix vulnerabilities. URL: https://github.com/apache/spark/pull/25933#issuecomment-536402950 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/111586/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #25933: [SPARK-29252][BUILD] Upgrade zookeeper to 3.4.14 and fix vulnerabilities.
SparkQA removed a comment on issue #25933: [SPARK-29252][BUILD] Upgrade zookeeper to 3.4.14 and fix vulnerabilities. URL: https://github.com/apache/spark/pull/25933#issuecomment-536374403 **[Test build #111586 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/111586/testReport)** for PR 25933 at commit [`b72b588`](https://github.com/apache/spark/commit/b72b588ed33e8f30c67e5ceb0ddd3486b35a3adc). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #25933: [SPARK-29252][BUILD] Upgrade zookeeper to 3.4.14 and fix vulnerabilities.
SparkQA commented on issue #25933: [SPARK-29252][BUILD] Upgrade zookeeper to 3.4.14 and fix vulnerabilities. URL: https://github.com/apache/spark/pull/25933#issuecomment-536402576 **[Test build #111586 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/111586/testReport)** for PR 25933 at commit [`b72b588`](https://github.com/apache/spark/commit/b72b588ed33e8f30c67e5ceb0ddd3486b35a3adc). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #25666: [SPARK-28962][SQL] Provide index argument to filter lambda functions
SparkQA commented on issue #25666: [SPARK-28962][SQL] Provide index argument to filter lambda functions URL: https://github.com/apache/spark/pull/25666#issuecomment-536400200 **[Test build #111596 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/111596/testReport)** for PR 25666 at commit [`9304867`](https://github.com/apache/spark/commit/93048670b427bd8c1b54f4be3459738c90806f0b). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #25962: [SPARK-29285][Shuffle] Temporary shuffle files should be able to handle disk failures
AmplabJenkins removed a comment on issue #25962: [SPARK-29285][Shuffle] Temporary shuffle files should be able to handle disk failures URL: https://github.com/apache/spark/pull/25962#issuecomment-536399097 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #25962: [SPARK-29285][Shuffle] Temporary shuffle files should be able to handle disk failures
AmplabJenkins removed a comment on issue #25962: [SPARK-29285][Shuffle] Temporary shuffle files should be able to handle disk failures URL: https://github.com/apache/spark/pull/25962#issuecomment-536399103 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/111584/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #25666: [SPARK-28962][SQL] Provide index argument to filter lambda functions
AmplabJenkins removed a comment on issue #25666: [SPARK-28962][SQL] Provide index argument to filter lambda functions URL: https://github.com/apache/spark/pull/25666#issuecomment-536399072 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #25666: [SPARK-28962][SQL] Provide index argument to filter lambda functions
AmplabJenkins removed a comment on issue #25666: [SPARK-28962][SQL] Provide index argument to filter lambda functions URL: https://github.com/apache/spark/pull/25666#issuecomment-536399075 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/16599/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #25962: [SPARK-29285][Shuffle] Temporary shuffle files should be able to handle disk failures
AmplabJenkins commented on issue #25962: [SPARK-29285][Shuffle] Temporary shuffle files should be able to handle disk failures URL: https://github.com/apache/spark/pull/25962#issuecomment-536399097 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #25666: [SPARK-28962][SQL] Provide index argument to filter lambda functions
AmplabJenkins commented on issue #25666: [SPARK-28962][SQL] Provide index argument to filter lambda functions URL: https://github.com/apache/spark/pull/25666#issuecomment-536399075 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/16599/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #25962: [SPARK-29285][Shuffle] Temporary shuffle files should be able to handle disk failures
AmplabJenkins commented on issue #25962: [SPARK-29285][Shuffle] Temporary shuffle files should be able to handle disk failures URL: https://github.com/apache/spark/pull/25962#issuecomment-536399103 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/111584/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] henrydavidge commented on issue #25666: [SPARK-28962][SQL] Provide index argument to filter lambda functions
henrydavidge commented on issue #25666: [SPARK-28962][SQL] Provide index argument to filter lambda functions URL: https://github.com/apache/spark/pull/25666#issuecomment-536398999 @ueshin comments addressed. I added a test to `DataFrameFunctionsSuite` as well. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #25666: [SPARK-28962][SQL] Provide index argument to filter lambda functions
AmplabJenkins commented on issue #25666: [SPARK-28962][SQL] Provide index argument to filter lambda functions URL: https://github.com/apache/spark/pull/25666#issuecomment-536399072 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] henrydavidge commented on a change in pull request #25666: [SPARK-28962][SQL] Provide index argument to filter lambda functions
henrydavidge commented on a change in pull request #25666: [SPARK-28962][SQL]
Provide index argument to filter lambda functions
URL: https://github.com/apache/spark/pull/25666#discussion_r329406929
##
File path:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/higherOrderFunctions.scala
##
@@ -357,10 +359,23 @@ case class ArrayFilter(
override def bind(f: (Expression, Seq[(DataType, Boolean)]) =>
LambdaFunction): ArrayFilter = {
val ArrayType(elementType, containsNull) = argument.dataType
-copy(function = f(function, (elementType, containsNull) :: Nil))
+function match {
+ case LambdaFunction(_, arguments, _) if arguments.size == 2 =>
+copy(function = f(function, (elementType, containsNull) ::
(IntegerType, false) :: Nil))
+ case _ =>
+copy(function = f(function, (elementType, containsNull) :: Nil))
Review comment:
Yes, it does. See the test here:
https://github.com/apache/spark/pull/25666/files#diff-8e1a34391fdefa4a3a0349d7d454d86fR2204.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #25962: [SPARK-29285][Shuffle] Temporary shuffle files should be able to handle disk failures
SparkQA removed a comment on issue #25962: [SPARK-29285][Shuffle] Temporary shuffle files should be able to handle disk failures URL: https://github.com/apache/spark/pull/25962#issuecomment-536369327 **[Test build #111584 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/111584/testReport)** for PR 25962 at commit [`7b080fb`](https://github.com/apache/spark/commit/7b080fb4244b254ed45045e7563f9866562e9fd3). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] dongjoon-hyun closed pull request #25969: [SPARK-29297][TESTS] Compare `core`/`mllib` module benchmarks in JDK8/11
dongjoon-hyun closed pull request #25969: [SPARK-29297][TESTS] Compare `core`/`mllib` module benchmarks in JDK8/11 URL: https://github.com/apache/spark/pull/25969 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #25962: [SPARK-29285][Shuffle] Temporary shuffle files should be able to handle disk failures
SparkQA commented on issue #25962: [SPARK-29285][Shuffle] Temporary shuffle files should be able to handle disk failures URL: https://github.com/apache/spark/pull/25962#issuecomment-536398734 **[Test build #111584 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/111584/testReport)** for PR 25962 at commit [`7b080fb`](https://github.com/apache/spark/commit/7b080fb4244b254ed45045e7563f9866562e9fd3). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on issue #25969: [SPARK-29297][TESTS] Compare `core`/`mllib` module benchmarks in JDK8/11
dongjoon-hyun commented on issue #25969: [SPARK-29297][TESTS] Compare `core`/`mllib` module benchmarks in JDK8/11 URL: https://github.com/apache/spark/pull/25969#issuecomment-536398547 Thank you for review. Merged to master. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] viirya commented on a change in pull request #25969: [SPARK-29297][TESTS] Compare `core`/`mllib` module benchmarks in JDK8/11
viirya commented on a change in pull request #25969: [SPARK-29297][TESTS] Compare `core`/`mllib` module benchmarks in JDK8/11 URL: https://github.com/apache/spark/pull/25969#discussion_r329406755 ## File path: core/benchmarks/PropertiesCloneBenchmark-results.txt ## @@ -2,39 +2,39 @@ Properties Cloning -Java HotSpot(TM) 64-Bit Server VM 1.8.0_131-b11 on Mac OS X 10.14.6 -Intel(R) Core(TM) i9-8950HK CPU @ 2.90GHz +OpenJDK 64-Bit Server VM 1.8.0_222-b10 on Linux 3.10.0-862.3.2.el7.x86_64 +Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz Empty Properties: Best Time(ms) Avg Time(ms) Stdev(ms)Rate(M/s) Per Row(ns) Relative -SerializationUtils.clone 0 0 0 0.24184.0 1.0X -Utils.cloneProperties 0 0 0 55.6 18.0 232.4X +SerializationUtils.clone 0 0 0 0.1 13640.0 1.0X +Utils.cloneProperties 0 0 0 1.6 608.0 22.4X Review comment: I see. This just updates benchmark result. So I think it is fine. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] viirya commented on a change in pull request #25969: [SPARK-29297][TESTS] Compare `core`/`mllib` module benchmarks in JDK8/11
viirya commented on a change in pull request #25969: [SPARK-29297][TESTS] Compare `core`/`mllib` module benchmarks in JDK8/11 URL: https://github.com/apache/spark/pull/25969#discussion_r329406585 ## File path: core/benchmarks/PropertiesCloneBenchmark-jdk11-results.txt ## @@ -6,35 +6,35 @@ OpenJDK 64-Bit Server VM 11.0.4+11-LTS on Linux 3.10.0-862.3.2.el7.x86_64 Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz Empty Properties: Best Time(ms) Avg Time(ms) Stdev(ms)Rate(M/s) Per Row(ns) Relative -SerializationUtils.clone 0 0 0 0.1 13755.0 1.0X -Utils.cloneProperties 0 0 0 3.5 285.0 48.3X +SerializationUtils.clone 0 0 0 0.1 11539.0 1.0X +Utils.cloneProperties 0 0 0 1.7 572.0 20.2X Review comment: oh, I meant previously 48.3X -> now 20.2X, so seems the speed up of Utils.cloneProperties seems less. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on a change in pull request #25969: [SPARK-29297][TESTS] Compare `core`/`mllib` module benchmarks in JDK8/11
dongjoon-hyun commented on a change in pull request #25969: [SPARK-29297][TESTS] Compare `core`/`mllib` module benchmarks in JDK8/11 URL: https://github.com/apache/spark/pull/25969#discussion_r329406224 ## File path: core/benchmarks/PropertiesCloneBenchmark-results.txt ## @@ -2,39 +2,39 @@ Properties Cloning -Java HotSpot(TM) 64-Bit Server VM 1.8.0_131-b11 on Mac OS X 10.14.6 -Intel(R) Core(TM) i9-8950HK CPU @ 2.90GHz +OpenJDK 64-Bit Server VM 1.8.0_222-b10 on Linux 3.10.0-862.3.2.el7.x86_64 +Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz Empty Properties: Best Time(ms) Avg Time(ms) Stdev(ms)Rate(M/s) Per Row(ns) Relative -SerializationUtils.clone 0 0 0 0.24184.0 1.0X -Utils.cloneProperties 0 0 0 55.6 18.0 232.4X +SerializationUtils.clone 0 0 0 0.1 13640.0 1.0X +Utils.cloneProperties 0 0 0 1.6 608.0 22.4X Review comment: This is due to the change from `Mac OS X` to `Linux Server`. We can ignore the previous `Mac OS X` because Spark is running on server in production. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on a change in pull request #25969: [SPARK-29297][TESTS] Compare `core`/`mllib` module benchmarks in JDK8/11
dongjoon-hyun commented on a change in pull request #25969: [SPARK-29297][TESTS] Compare `core`/`mllib` module benchmarks in JDK8/11 URL: https://github.com/apache/spark/pull/25969#discussion_r329406224 ## File path: core/benchmarks/PropertiesCloneBenchmark-results.txt ## @@ -2,39 +2,39 @@ Properties Cloning -Java HotSpot(TM) 64-Bit Server VM 1.8.0_131-b11 on Mac OS X 10.14.6 -Intel(R) Core(TM) i9-8950HK CPU @ 2.90GHz +OpenJDK 64-Bit Server VM 1.8.0_222-b10 on Linux 3.10.0-862.3.2.el7.x86_64 +Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz Empty Properties: Best Time(ms) Avg Time(ms) Stdev(ms)Rate(M/s) Per Row(ns) Relative -SerializationUtils.clone 0 0 0 0.24184.0 1.0X -Utils.cloneProperties 0 0 0 55.6 18.0 232.4X +SerializationUtils.clone 0 0 0 0.1 13640.0 1.0X +Utils.cloneProperties 0 0 0 1.6 608.0 22.4X Review comment: This is a change from `Mac OS X` to `Linux Server`. We can ignore `Mac OS X` because Spark is running on server in production. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on a change in pull request #25969: [SPARK-29297][TESTS] Compare `core`/`mllib` module benchmarks in JDK8/11
dongjoon-hyun commented on a change in pull request #25969: [SPARK-29297][TESTS] Compare `core`/`mllib` module benchmarks in JDK8/11 URL: https://github.com/apache/spark/pull/25969#discussion_r329406224 ## File path: core/benchmarks/PropertiesCloneBenchmark-results.txt ## @@ -2,39 +2,39 @@ Properties Cloning -Java HotSpot(TM) 64-Bit Server VM 1.8.0_131-b11 on Mac OS X 10.14.6 -Intel(R) Core(TM) i9-8950HK CPU @ 2.90GHz +OpenJDK 64-Bit Server VM 1.8.0_222-b10 on Linux 3.10.0-862.3.2.el7.x86_64 +Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz Empty Properties: Best Time(ms) Avg Time(ms) Stdev(ms)Rate(M/s) Per Row(ns) Relative -SerializationUtils.clone 0 0 0 0.24184.0 1.0X -Utils.cloneProperties 0 0 0 55.6 18.0 232.4X +SerializationUtils.clone 0 0 0 0.1 13640.0 1.0X +Utils.cloneProperties 0 0 0 1.6 608.0 22.4X Review comment: This is a change from `Mac OS X` to `Linux Server`. We can ignore the previous `Mac OS X` because Spark is running on server in production. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on a change in pull request #25969: [SPARK-29297][TESTS] Compare `core`/`mllib` module benchmarks in JDK8/11
dongjoon-hyun commented on a change in pull request #25969: [SPARK-29297][TESTS] Compare `core`/`mllib` module benchmarks in JDK8/11 URL: https://github.com/apache/spark/pull/25969#discussion_r329406071 ## File path: core/benchmarks/PropertiesCloneBenchmark-jdk11-results.txt ## @@ -6,35 +6,35 @@ OpenJDK 64-Bit Server VM 11.0.4+11-LTS on Linux 3.10.0-862.3.2.el7.x86_64 Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz Empty Properties: Best Time(ms) Avg Time(ms) Stdev(ms)Rate(M/s) Per Row(ns) Relative -SerializationUtils.clone 0 0 0 0.1 13755.0 1.0X -Utils.cloneProperties 0 0 0 3.5 285.0 48.3X +SerializationUtils.clone 0 0 0 0.1 11539.0 1.0X +Utils.cloneProperties 0 0 0 1.7 572.0 20.2X Review comment: Thank you for review. Do you mean JDK11 is slower? Here, JDK8 took more time. The following is the JDK8 result in this PR. JDK11 took `572.0ns` per row while JDK8 did `608.0ns` per row. ``` SerializationUtils.clone 0 0 0 0.1 13640.0 1.0X Utils.cloneProperties 0 0 0 1.6 608.0 22.4X ``` This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #25969: [SPARK-29297][TESTS] Compare `core`/`mllib` module benchmarks in JDK8/11
AmplabJenkins removed a comment on issue #25969: [SPARK-29297][TESTS] Compare `core`/`mllib` module benchmarks in JDK8/11 URL: https://github.com/apache/spark/pull/25969#issuecomment-536397255 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/111585/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #25969: [SPARK-29297][TESTS] Compare `core`/`mllib` module benchmarks in JDK8/11
AmplabJenkins commented on issue #25969: [SPARK-29297][TESTS] Compare `core`/`mllib` module benchmarks in JDK8/11 URL: https://github.com/apache/spark/pull/25969#issuecomment-536397255 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/111585/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #25969: [SPARK-29297][TESTS] Compare `core`/`mllib` module benchmarks in JDK8/11
AmplabJenkins commented on issue #25969: [SPARK-29297][TESTS] Compare `core`/`mllib` module benchmarks in JDK8/11 URL: https://github.com/apache/spark/pull/25969#issuecomment-536397251 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #25969: [SPARK-29297][TESTS] Compare `core`/`mllib` module benchmarks in JDK8/11
AmplabJenkins removed a comment on issue #25969: [SPARK-29297][TESTS] Compare `core`/`mllib` module benchmarks in JDK8/11 URL: https://github.com/apache/spark/pull/25969#issuecomment-536397251 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #25969: [SPARK-29297][TESTS] Compare `core`/`mllib` module benchmarks in JDK8/11
SparkQA removed a comment on issue #25969: [SPARK-29297][TESTS] Compare `core`/`mllib` module benchmarks in JDK8/11 URL: https://github.com/apache/spark/pull/25969#issuecomment-536370492 **[Test build #111585 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/111585/testReport)** for PR 25969 at commit [`7b5dcd7`](https://github.com/apache/spark/commit/7b5dcd7a98d6b1957649eda3de1641faa367a798). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #25969: [SPARK-29297][TESTS] Compare `core`/`mllib` module benchmarks in JDK8/11
SparkQA commented on issue #25969: [SPARK-29297][TESTS] Compare `core`/`mllib` module benchmarks in JDK8/11 URL: https://github.com/apache/spark/pull/25969#issuecomment-536396907 **[Test build #111585 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/111585/testReport)** for PR 25969 at commit [`7b5dcd7`](https://github.com/apache/spark/commit/7b5dcd7a98d6b1957649eda3de1641faa367a798). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #25198: [SPARK-28443][SQL] Spark sql add exception when create field type NullType
AmplabJenkins removed a comment on issue #25198: [SPARK-28443][SQL] Spark sql add exception when create field type NullType URL: https://github.com/apache/spark/pull/25198#issuecomment-536393607 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/111595/ Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #25198: [SPARK-28443][SQL] Spark sql add exception when create field type NullType
AmplabJenkins removed a comment on issue #25198: [SPARK-28443][SQL] Spark sql add exception when create field type NullType URL: https://github.com/apache/spark/pull/25198#issuecomment-536393605 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #25198: [SPARK-28443][SQL] Spark sql add exception when create field type NullType
AmplabJenkins commented on issue #25198: [SPARK-28443][SQL] Spark sql add exception when create field type NullType URL: https://github.com/apache/spark/pull/25198#issuecomment-536393607 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/111595/ Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #25198: [SPARK-28443][SQL] Spark sql add exception when create field type NullType
AmplabJenkins commented on issue #25198: [SPARK-28443][SQL] Spark sql add exception when create field type NullType URL: https://github.com/apache/spark/pull/25198#issuecomment-536393605 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #25198: [SPARK-28443][SQL] Spark sql add exception when create field type NullType
SparkQA commented on issue #25198: [SPARK-28443][SQL] Spark sql add exception when create field type NullType URL: https://github.com/apache/spark/pull/25198#issuecomment-536393592 **[Test build #111595 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/111595/testReport)** for PR 25198 at commit [`6a1b4f5`](https://github.com/apache/spark/commit/6a1b4f5b4a14e8b4bb5a97b3020ce9a65add1baf). * This patch **fails to build**. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #25198: [SPARK-28443][SQL] Spark sql add exception when create field type NullType
SparkQA removed a comment on issue #25198: [SPARK-28443][SQL] Spark sql add exception when create field type NullType URL: https://github.com/apache/spark/pull/25198#issuecomment-536392548 **[Test build #111595 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/111595/testReport)** for PR 25198 at commit [`6a1b4f5`](https://github.com/apache/spark/commit/6a1b4f5b4a14e8b4bb5a97b3020ce9a65add1baf). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] viirya commented on a change in pull request #25969: [SPARK-29297][TESTS] Compare `core`/`mllib` module benchmarks in JDK8/11
viirya commented on a change in pull request #25969: [SPARK-29297][TESTS] Compare `core`/`mllib` module benchmarks in JDK8/11 URL: https://github.com/apache/spark/pull/25969#discussion_r329402650 ## File path: core/benchmarks/PropertiesCloneBenchmark-results.txt ## @@ -2,39 +2,39 @@ Properties Cloning -Java HotSpot(TM) 64-Bit Server VM 1.8.0_131-b11 on Mac OS X 10.14.6 -Intel(R) Core(TM) i9-8950HK CPU @ 2.90GHz +OpenJDK 64-Bit Server VM 1.8.0_222-b10 on Linux 3.10.0-862.3.2.el7.x86_64 +Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz Empty Properties: Best Time(ms) Avg Time(ms) Stdev(ms)Rate(M/s) Per Row(ns) Relative -SerializationUtils.clone 0 0 0 0.24184.0 1.0X -Utils.cloneProperties 0 0 0 55.6 18.0 232.4X +SerializationUtils.clone 0 0 0 0.1 13640.0 1.0X +Utils.cloneProperties 0 0 0 1.6 608.0 22.4X Review comment: 232 -> 22? Looks slower significantly. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] viirya commented on a change in pull request #25969: [SPARK-29297][TESTS] Compare `core`/`mllib` module benchmarks in JDK8/11
viirya commented on a change in pull request #25969: [SPARK-29297][TESTS] Compare `core`/`mllib` module benchmarks in JDK8/11 URL: https://github.com/apache/spark/pull/25969#discussion_r329402018 ## File path: core/benchmarks/PropertiesCloneBenchmark-jdk11-results.txt ## @@ -6,35 +6,35 @@ OpenJDK 64-Bit Server VM 11.0.4+11-LTS on Linux 3.10.0-862.3.2.el7.x86_64 Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz Empty Properties: Best Time(ms) Avg Time(ms) Stdev(ms)Rate(M/s) Per Row(ns) Relative -SerializationUtils.clone 0 0 0 0.1 13755.0 1.0X -Utils.cloneProperties 0 0 0 3.5 285.0 48.3X +SerializationUtils.clone 0 0 0 0.1 11539.0 1.0X +Utils.cloneProperties 0 0 0 1.7 572.0 20.2X Review comment: Seems only this is slower? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #25198: [SPARK-28443][SQL] Spark sql add exception when create field type NullType
AmplabJenkins removed a comment on issue #25198: [SPARK-28443][SQL] Spark sql add exception when create field type NullType URL: https://github.com/apache/spark/pull/25198#issuecomment-531892284 Can one of the admins verify this patch? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #25333: [SPARK-28597][SS] Add config to retry spark streaming's meta log when it met error
AmplabJenkins removed a comment on issue #25333: [SPARK-28597][SS] Add config to retry spark streaming's meta log when it met error URL: https://github.com/apache/spark/pull/25333#issuecomment-531891975 Can one of the admins verify this patch? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #25333: [SPARK-28597][SS] Add config to retry spark streaming's meta log when it met error
SparkQA commented on issue #25333: [SPARK-28597][SS] Add config to retry spark streaming's meta log when it met error URL: https://github.com/apache/spark/pull/25333#issuecomment-536392543 **[Test build #111594 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/111594/testReport)** for PR 25333 at commit [`98273ad`](https://github.com/apache/spark/commit/98273add89fef04662625997f8fcceb1f1278f92). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #25198: [SPARK-28443][SQL] Spark sql add exception when create field type NullType
SparkQA commented on issue #25198: [SPARK-28443][SQL] Spark sql add exception when create field type NullType URL: https://github.com/apache/spark/pull/25198#issuecomment-536392548 **[Test build #111595 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/111595/testReport)** for PR 25198 at commit [`6a1b4f5`](https://github.com/apache/spark/commit/6a1b4f5b4a14e8b4bb5a97b3020ce9a65add1baf). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #25963: [SPARK-28137][SQL] Add Postgresql function to_number.
AmplabJenkins removed a comment on issue #25963: [SPARK-28137][SQL] Add Postgresql function to_number. URL: https://github.com/apache/spark/pull/25963#issuecomment-536391528 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/16597/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #25963: [SPARK-28137][SQL] Add Postgresql function to_number.
AmplabJenkins removed a comment on issue #25963: [SPARK-28137][SQL] Add Postgresql function to_number. URL: https://github.com/apache/spark/pull/25963#issuecomment-536391525 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #25464: [SPARK-28746][SQL] Add partitionby hint for sql queries
AmplabJenkins removed a comment on issue #25464: [SPARK-28746][SQL] Add partitionby hint for sql queries URL: https://github.com/apache/spark/pull/25464#issuecomment-536391522 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/16598/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #25464: [SPARK-28746][SQL] Add partitionby hint for sql queries
AmplabJenkins removed a comment on issue #25464: [SPARK-28746][SQL] Add partitionby hint for sql queries URL: https://github.com/apache/spark/pull/25464#issuecomment-536391520 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #25464: [SPARK-28746][SQL] Add partitionby hint for sql queries
AmplabJenkins commented on issue #25464: [SPARK-28746][SQL] Add partitionby hint for sql queries URL: https://github.com/apache/spark/pull/25464#issuecomment-536391522 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/16598/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #25963: [SPARK-28137][SQL] Add Postgresql function to_number.
AmplabJenkins commented on issue #25963: [SPARK-28137][SQL] Add Postgresql function to_number. URL: https://github.com/apache/spark/pull/25963#issuecomment-536391525 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #25963: [SPARK-28137][SQL] Add Postgresql function to_number.
AmplabJenkins commented on issue #25963: [SPARK-28137][SQL] Add Postgresql function to_number. URL: https://github.com/apache/spark/pull/25963#issuecomment-536391528 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/16597/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #25464: [SPARK-28746][SQL] Add partitionby hint for sql queries
AmplabJenkins commented on issue #25464: [SPARK-28746][SQL] Add partitionby hint for sql queries URL: https://github.com/apache/spark/pull/25464#issuecomment-536391520 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #25963: [SPARK-28137][SQL] Add Postgresql function to_number.
SparkQA commented on issue #25963: [SPARK-28137][SQL] Add Postgresql function to_number. URL: https://github.com/apache/spark/pull/25963#issuecomment-536391328 **[Test build #111592 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/111592/testReport)** for PR 25963 at commit [`c4a170c`](https://github.com/apache/spark/commit/c4a170c69d4b0e34eb9927d7f8838345e95e83a8). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #25464: [SPARK-28746][SQL] Add partitionby hint for sql queries
SparkQA commented on issue #25464: [SPARK-28746][SQL] Add partitionby hint for sql queries URL: https://github.com/apache/spark/pull/25464#issuecomment-536391324 **[Test build #111593 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/111593/testReport)** for PR 25464 at commit [`493a193`](https://github.com/apache/spark/commit/493a19329dd8725f736c7188366f560144375d75). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on issue #25970: [SPARK-29186][SQL][2.4][Followup] AliasIdentifier should be converted to Json in prettyJson
dongjoon-hyun commented on issue #25970: [SPARK-29186][SQL][2.4][Followup] AliasIdentifier should be converted to Json in prettyJson URL: https://github.com/apache/spark/pull/25970#issuecomment-536390690 No problem~ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on a change in pull request #25969: [SPARK-29297][TESTS] Compare `core`/`mllib` module benchmarks in JDK8/11
dongjoon-hyun commented on a change in pull request #25969: [SPARK-29297][TESTS] Compare `core`/`mllib` module benchmarks in JDK8/11 URL: https://github.com/apache/spark/pull/25969#discussion_r329401344 ## File path: core/benchmarks/XORShiftRandomBenchmark-results.txt ## @@ -2,43 +2,43 @@ Pseudo random -OpenJDK 64-Bit Server VM 1.8.0_191-b12 on Linux 3.10.0-862.3.2.el7.x86_64 +OpenJDK 64-Bit Server VM 1.8.0_222-b10 on Linux 3.10.0-862.3.2.el7.x86_64 Review comment: This is regenerate to match the format. There is no effective change according to the result. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on a change in pull request #25969: [SPARK-29297][TESTS] Compare `core`/`mllib` module benchmarks in JDK8/11
dongjoon-hyun commented on a change in pull request #25969: [SPARK-29297][TESTS] Compare `core`/`mllib` module benchmarks in JDK8/11 URL: https://github.com/apache/spark/pull/25969#discussion_r329401188 ## File path: core/benchmarks/CoalescedRDDBenchmark-results.txt ## @@ -2,39 +2,39 @@ Coalesced RDD , large scale -Java HotSpot(TM) 64-Bit Server VM 1.8.0_201-b09 on Windows 10 10.0 -Intel64 Family 6 Model 63 Stepping 2, GenuineIntel +OpenJDK 64-Bit Server VM 1.8.0_222-b10 on Linux 3.10.0-862.3.2.el7.x86_64 +Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz Review comment: This is regenerated to compare with JDK11. There is no change in JDK8. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on a change in pull request #25969: [SPARK-29297][TESTS] Compare `core`/`mllib` module benchmarks in JDK8/11
dongjoon-hyun commented on a change in pull request #25969: [SPARK-29297][TESTS] Compare `core`/`mllib` module benchmarks in JDK8/11 URL: https://github.com/apache/spark/pull/25969#discussion_r329401132 ## File path: core/benchmarks/CoalescedRDDBenchmark-jdk11-results.txt ## @@ -6,35 +6,35 @@ OpenJDK 64-Bit Server VM 11.0.4+11-LTS on Linux 3.10.0-862.3.2.el7.x86_64 Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz Coalesced RDD:Best Time(ms) Avg Time(ms) Stdev(ms)Rate(M/s) Per Row(ns) Relative -Coalesce Num Partitions: 100 Num Hosts: 1455623 151 0.24547.4 1.0X -Coalesce Num Partitions: 100 Num Hosts: 5340409 65 0.33397.1 1.3X -Coalesce Num Partitions: 100 Num Hosts: 10292363 95 0.32923.3 1.6X -Coalesce Num Partitions: 100 Num Hosts: 20307320 11 0.33069.8 1.5X -Coalesce Num Partitions: 100 Num Hosts: 40333368 55 0.33329.1 1.4X -Coalesce Num Partitions: 100 Num Hosts: 80286338 63 0.32862.5 1.6X -Coalesce Num Partitions: 500 Num Hosts: 1769837 59 0.17693.5 0.6X -Coalesce Num Partitions: 500 Num Hosts: 5427461 31 0.24268.5 1.1X -Coalesce Num Partitions: 500 Num Hosts: 10372389 27 0.33722.2 1.2X -Coalesce Num Partitions: 500 Num Hosts: 20347365 31 0.33468.5 1.3X -Coalesce Num Partitions: 500 Num Hosts: 40335336 1 0.33347.3 1.4X -Coalesce Num Partitions: 500 Num Hosts: 80329360 49 0.33294.5 1.4X -Coalesce Num Partitions: 1000 Num Hosts: 1 1254 1292 47 0.1 12538.6 0.4X -Coalesce Num Partitions: 1000 Num Hosts: 5518553 47 0.25177.0 0.9X -Coalesce Num Partitions: 1000 Num Hosts: 10394432 42 0.33937.3 1.2X -Coalesce Num Partitions: 1000 Num Hosts: 20341381 44 0.33414.4 1.3X -Coalesce Num Partitions: 1000 Num Hosts: 40313358 48 0.33134.9 1.5X -Coalesce Num Partitions: 1000 Num Hosts: 80335360 38 0.33347.0 1.4X -Coalesce Num Partitions: 5000 Num Hosts: 1 3937 4066 156 0.0 39375.0 0.1X -Coalesce Num Partitions: 5000 Num Hosts: 5 1413 1453 40 0.1 14133.4 0.3X -Coalesce Num Partitions: 5000 Num Hosts: 10826861 49 0.18255.2 0.6X -Coalesce Num Partitions: 5000 Num Hosts: 20542609 58 0.25423.3 0.8X -Coalesce Num Partitions: 5000 Num Hosts: 40410470 64 0.24101.0 1.1X -Coalesce Num Partitions: 5000 Num Hosts: 80352427 69 0.33515.3 1.3X -Coalesce Num Partitions: 1 Num Hosts: 1 7101 7151 54 0.0 71007.4 0.1X -Coalesce Num Partitions: 1 Num Hosts: 5 2540 2582 59 0.0 25396.2 0.2X -Coalesce Num Partitions: 1 Num Hosts: 10 1378 1432 48 0.1 13781.4 0.3X -Coalesce Num Partitions: 1 Num Hosts: 20829867 66 0.18286.8 0.5X -Coalesce Num Partitions: 1 Num Hosts: 40573630 49 0.25730.2 0.8X -Coalesce Num Partitions: 1 Num Hosts: 80438449 9 0.24382.5 1.0X +Coalesce Num Partitions: 100 Num Hosts: 1344360 14 0.33441.4 1.0X Review comment: This is improved via https://github.com/apache/spark/pull/25966 . This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards,
[GitHub] [spark] dongjoon-hyun commented on issue #25969: [SPARK-29297][TESTS] Compare `core`/`mllib` module benchmarks in JDK8/11
dongjoon-hyun commented on issue #25969: [SPARK-29297][TESTS] Compare `core`/`mllib` module benchmarks in JDK8/11 URL: https://github.com/apache/spark/pull/25969#issuecomment-536390043 Could you review this please, @viirya ? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] viirya commented on issue #25970: [SPARK-29186][SQL][2.4][Followup] AliasIdentifier should be converted to Json in prettyJson
viirya commented on issue #25970: [SPARK-29186][SQL][2.4][Followup] AliasIdentifier should be converted to Json in prettyJson URL: https://github.com/apache/spark/pull/25970#issuecomment-536389929 Thanks @dongjoon-hyun This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] dongjoon-hyun closed pull request #25970: [SPARK-29186][SQL][2.4][Followup] AliasIdentifier should be converted to Json in prettyJson
dongjoon-hyun closed pull request #25970: [SPARK-29186][SQL][2.4][Followup] AliasIdentifier should be converted to Json in prettyJson URL: https://github.com/apache/spark/pull/25970 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on issue #25970: [SPARK-29186][SQL][2.4][Followup] AliasIdentifier should be converted to Json in prettyJson
dongjoon-hyun commented on issue #25970: [SPARK-29186][SQL][2.4][Followup] AliasIdentifier should be converted to Json in prettyJson URL: https://github.com/apache/spark/pull/25970#issuecomment-536389532 It was a compilation error, and now GitHub Action looks good. I'll merge this to branch-2.4. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #25963: [SPARK-28137][SQL] Add Postgresql function to_number.
AmplabJenkins removed a comment on issue #25963: [SPARK-28137][SQL] Add Postgresql function to_number. URL: https://github.com/apache/spark/pull/25963#issuecomment-536389328 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/111591/ Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #25971: [SPARK-29298][CORE] Separate block manager heartbeat endpoint from driver endpoint
AmplabJenkins removed a comment on issue #25971: [SPARK-29298][CORE] Separate block manager heartbeat endpoint from driver endpoint URL: https://github.com/apache/spark/pull/25971#issuecomment-536389211 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #25963: [SPARK-28137][SQL] Add Postgresql function to_number.
AmplabJenkins removed a comment on issue #25963: [SPARK-28137][SQL] Add Postgresql function to_number. URL: https://github.com/apache/spark/pull/25963#issuecomment-536389197 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #25963: [SPARK-28137][SQL] Add Postgresql function to_number.
SparkQA commented on issue #25963: [SPARK-28137][SQL] Add Postgresql function to_number. URL: https://github.com/apache/spark/pull/25963#issuecomment-536389319 **[Test build #111591 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/111591/testReport)** for PR 25963 at commit [`7b4ed77`](https://github.com/apache/spark/commit/7b4ed77aaf0308d495deb740b4e147b7b530faf3). * This patch **fails Scala style tests**. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #25963: [SPARK-28137][SQL] Add Postgresql function to_number.
AmplabJenkins commented on issue #25963: [SPARK-28137][SQL] Add Postgresql function to_number. URL: https://github.com/apache/spark/pull/25963#issuecomment-536389328 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/111591/ Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #25963: [SPARK-28137][SQL] Add Postgresql function to_number.
AmplabJenkins removed a comment on issue #25963: [SPARK-28137][SQL] Add Postgresql function to_number. URL: https://github.com/apache/spark/pull/25963#issuecomment-536389201 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/16596/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #25963: [SPARK-28137][SQL] Add Postgresql function to_number.
AmplabJenkins removed a comment on issue #25963: [SPARK-28137][SQL] Add Postgresql function to_number. URL: https://github.com/apache/spark/pull/25963#issuecomment-536389323 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #25963: [SPARK-28137][SQL] Add Postgresql function to_number.
SparkQA removed a comment on issue #25963: [SPARK-28137][SQL] Add Postgresql function to_number. URL: https://github.com/apache/spark/pull/25963#issuecomment-536389004 **[Test build #111591 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/111591/testReport)** for PR 25963 at commit [`7b4ed77`](https://github.com/apache/spark/commit/7b4ed77aaf0308d495deb740b4e147b7b530faf3). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #25971: [SPARK-29298][CORE] Separate block manager heartbeat endpoint from driver endpoint
AmplabJenkins removed a comment on issue #25971: [SPARK-29298][CORE] Separate block manager heartbeat endpoint from driver endpoint URL: https://github.com/apache/spark/pull/25971#issuecomment-536389215 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/16595/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #25963: [SPARK-28137][SQL] Add Postgresql function to_number.
AmplabJenkins commented on issue #25963: [SPARK-28137][SQL] Add Postgresql function to_number. URL: https://github.com/apache/spark/pull/25963#issuecomment-536389323 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #25971: [SPARK-29298][CORE] Separate block manager heartbeat endpoint from driver endpoint
AmplabJenkins commented on issue #25971: [SPARK-29298][CORE] Separate block manager heartbeat endpoint from driver endpoint URL: https://github.com/apache/spark/pull/25971#issuecomment-536389211 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #25963: [SPARK-28137][SQL] Add Postgresql function to_number.
AmplabJenkins commented on issue #25963: [SPARK-28137][SQL] Add Postgresql function to_number. URL: https://github.com/apache/spark/pull/25963#issuecomment-536389201 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/16596/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #25963: [SPARK-28137][SQL] Add Postgresql function to_number.
AmplabJenkins commented on issue #25963: [SPARK-28137][SQL] Add Postgresql function to_number. URL: https://github.com/apache/spark/pull/25963#issuecomment-536389197 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #25971: [SPARK-29298][CORE] Separate block manager heartbeat endpoint from driver endpoint
AmplabJenkins commented on issue #25971: [SPARK-29298][CORE] Separate block manager heartbeat endpoint from driver endpoint URL: https://github.com/apache/spark/pull/25971#issuecomment-536389215 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/16595/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #25971: [SPARK-29298][CORE] Separate block manager heartbeat endpoint from driver endpoint
SparkQA commented on issue #25971: [SPARK-29298][CORE] Separate block manager heartbeat endpoint from driver endpoint URL: https://github.com/apache/spark/pull/25971#issuecomment-536388988 **[Test build #111590 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/111590/testReport)** for PR 25971 at commit [`c348176`](https://github.com/apache/spark/commit/c3481768b80e74b2edbd2f75e4bb5ee25f7535fd). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
