[jira] [Commented] (SPARK-26337) Add benchmark for LongToUnsafeRowMap
[
https://issues.apache.org/jira/browse/SPARK-26337?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16720862#comment-16720862
]
ASF GitHub Bot commented on SPARK-26337:
asfgit closed pull request #23284: [SPARK-26337][SQL][TEST] Add benchmark for
LongToUnsafeRowMap
URL: https://github.com/apache/spark/pull/23284
This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:
As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):
diff --git a/sql/core/benchmarks/HashedRelationMetricsBenchmark-results.txt
b/sql/core/benchmarks/HashedRelationMetricsBenchmark-results.txt
new file mode 100644
index 0..338244ad542f4
--- /dev/null
+++ b/sql/core/benchmarks/HashedRelationMetricsBenchmark-results.txt
@@ -0,0 +1,11 @@
+
+LongToUnsafeRowMap metrics
+
+
+Java HotSpot(TM) 64-Bit Server VM 1.8.0_181-b13 on Mac OS X 10.13.6
+Intel(R) Core(TM) i7-7700HQ CPU @ 2.80GHz
+LongToUnsafeRowMap metrics: Best/Avg Time(ms)Rate(M/s) Per
Row(ns) Relative
+
+LongToUnsafeRowMap 234 / 315 2.1
467.3 1.0X
+
+
diff --git
a/sql/core/src/test/scala/org/apache/spark/sql/execution/benchmark/HashedRelationMetricsBenchmark.scala
b/sql/core/src/test/scala/org/apache/spark/sql/execution/benchmark/HashedRelationMetricsBenchmark.scala
new file mode 100644
index 0..bdf753debe62a
--- /dev/null
+++
b/sql/core/src/test/scala/org/apache/spark/sql/execution/benchmark/HashedRelationMetricsBenchmark.scala
@@ -0,0 +1,84 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ *http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.spark.sql.execution.benchmark
+
+import org.apache.spark.SparkConf
+import org.apache.spark.benchmark.Benchmark
+import org.apache.spark.internal.config.MEMORY_OFFHEAP_ENABLED
+import org.apache.spark.memory.{StaticMemoryManager, TaskMemoryManager}
+import org.apache.spark.sql.catalyst.InternalRow
+import org.apache.spark.sql.catalyst.expressions.{BoundReference,
UnsafeProjection}
+import org.apache.spark.sql.execution.joins.LongToUnsafeRowMap
+import org.apache.spark.sql.types.LongType
+
+/**
+ * Benchmark to measure metrics performance at HashedRelation.
+ * To run this benchmark:
+ * {{{
+ * 1. without sbt: bin/spark-submit --class
+ * 2. build/sbt "sql/test:runMain "
+ * 3. generate result: SPARK_GENERATE_BENCHMARK_FILES=1 build/sbt
"sql/test:runMain "
+ * Results will be written to
"benchmarks/HashedRelationMetricsBenchmark-results.txt".
+ * }}}
+ */
+object HashedRelationMetricsBenchmark extends SqlBasedBenchmark {
+
+ def benchmarkLongToUnsafeRowMapMetrics(numRows: Int): Unit = {
+runBenchmark("LongToUnsafeRowMap metrics") {
+ val benchmark = new Benchmark("LongToUnsafeRowMap metrics", numRows,
output = output)
+ benchmark.addCase("LongToUnsafeRowMap") { iter =>
+val taskMemoryManager = new TaskMemoryManager(
+ new StaticMemoryManager(
+new SparkConf().set(MEMORY_OFFHEAP_ENABLED.key, "false"),
+Long.MaxValue,
+Long.MaxValue,
+1),
+ 0)
+val unsafeProj = UnsafeProjection.create(Seq(BoundReference(0,
LongType, false)))
+
+val keys = Range.Long(0, numRows, 1)
+val map = new LongToUnsafeRowMap(taskMemoryManager, 1)
+keys.foreach { k =>
+ map.append(k, unsafeProj(InternalRow(k)))
+}
+map.optimize()
+
+val threads = (0 to 100).map { _ =>
+ val thread = new Thread {
+override def run: Unit = {
+ val row = unsafeProj(InternalRow(0L)).copy()
+ keys.foreach { k =>
+assert(map.getValue(k, row) eq row)
+assert(row.getLong(0) == k)
+ }
+}
[jira] [Commented] (SPARK-26337) Add benchmark for LongToUnsafeRowMap
[
https://issues.apache.org/jira/browse/SPARK-26337?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16718050#comment-16718050
]
ASF GitHub Bot commented on SPARK-26337:
dongjoon-hyun commented on a change in pull request #23284:
[SPARK-26337][SQL][TEST] Add benchmark for LongToUnsafeRowMap
URL: https://github.com/apache/spark/pull/23284#discussion_r240802915
##
File path:
sql/core/src/test/scala/org/apache/spark/sql/execution/benchmark/HashedRelationMetricsBenchmark.scala
##
@@ -0,0 +1,84 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ *http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.spark.sql.execution.benchmark
+
+import org.apache.spark.SparkConf
+import org.apache.spark.benchmark.Benchmark
+import org.apache.spark.internal.config.MEMORY_OFFHEAP_ENABLED
+import org.apache.spark.memory.{StaticMemoryManager, TaskMemoryManager}
+import org.apache.spark.sql.catalyst.InternalRow
+import org.apache.spark.sql.catalyst.expressions.{BoundReference,
UnsafeProjection}
+import org.apache.spark.sql.execution.joins.LongToUnsafeRowMap
+import org.apache.spark.sql.types.LongType
+
+/**
+ * Benchmark to measure metrics performance at HashedRelation.
+ * To run this benchmark:
+ * {{{
+ * 1. without sbt: bin/spark-submit --class
+ * 2. build/sbt "sql/test:runMain "
+ * 3. generate result: SPARK_GENERATE_BENCHMARK_FILES=1 build/sbt
"sql/test:runMain "
+ * Results will be written to
"benchmarks/HashedRelationMetricsBenchmark-results.txt".
+ * }}}
+ */
+object HashedRelationMetricsBenchmark extends SqlBasedBenchmark {
+
+ def benchmarkLongToUnsafeRowMapMetrics(numRows: Int): Unit = {
+runBenchmark("LongToUnsafeRowMap metrics") {
+ val benchmark = new Benchmark("LongToUnsafeRowMap metrics", numRows,
output = output)
+ benchmark.addCase("LongToUnsafeRowMap") { iter =>
+val taskMemoryManager = new TaskMemoryManager(
+ new StaticMemoryManager(
+new SparkConf().set(MEMORY_OFFHEAP_ENABLED.key, "false"),
+Long.MaxValue,
+Long.MaxValue,
+1),
+ 0)
+val unsafeProj = UnsafeProjection.create(Seq(BoundReference(0,
LongType, false)))
+
+val keys = Range.Long(0, numRows, 1)
+val map = new LongToUnsafeRowMap(taskMemoryManager, 1)
+keys.foreach { k =>
+ map.append(k, unsafeProj(InternalRow(k)))
+}
+map.optimize()
+
+val threads = (0 to 100).map { _ =>
+ val thread = new Thread {
Review comment:
So, is this the real difference from
[AggregateBenchmark.LongToUnsafeRowMap](https://github.com/apache/spark/blob/master/sql/core/src/test/scala/org/apache/spark/sql/execution/benchmark/AggregateBenchmark.scala#L468-L502)
benchmark case?
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Add benchmark for LongToUnsafeRowMap
>
>
> Key: SPARK-26337
> URL: https://issues.apache.org/jira/browse/SPARK-26337
> Project: Spark
> Issue Type: Test
> Components: SQL
>Affects Versions: 3.0.0
>Reporter: Liang-Chi Hsieh
>Priority: Major
>
> Regarding the performance issue of SPARK-26155, I think it is better to add a
> benchmark for LongToUnsafeRowMap which is the root cause of performance
> regression. It can be easier to show performance difference between different
> metric implementation in LongToUnsafeRowMap.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-26337) Add benchmark for LongToUnsafeRowMap
[ https://issues.apache.org/jira/browse/SPARK-26337?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16718037#comment-16718037 ] ASF GitHub Bot commented on SPARK-26337: dongjoon-hyun commented on a change in pull request #23284: [SPARK-26337][SQL][TEST] Add benchmark for LongToUnsafeRowMap URL: https://github.com/apache/spark/pull/23284#discussion_r240799221 ## File path: sql/core/benchmarks/HashedRelationMetricsBenchmark-results.txt ## @@ -6,6 +6,6 @@ Java HotSpot(TM) 64-Bit Server VM 1.8.0_181-b13 on Mac OS X 10.13.6 Intel(R) Core(TM) i7-7700HQ CPU @ 2.80GHz LongToUnsafeRowMap metrics: Best/Avg Time(ms)Rate(M/s) Per Row(ns) Relative -LongToUnsafeRowMap1265 / 1336 0.4 2530.5 1.0X +LongToUnsafeRowMap 234 / 315 2.1 467.3 1.0X Review comment: Yep. It's a clear reason to have this benchmark (https://github.com/apache/spark/pull/23284/commits/723b27ce55770f5846d4244377b37f66f2f9ad60) This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Add benchmark for LongToUnsafeRowMap > > > Key: SPARK-26337 > URL: https://issues.apache.org/jira/browse/SPARK-26337 > Project: Spark > Issue Type: Test > Components: SQL >Affects Versions: 3.0.0 >Reporter: Liang-Chi Hsieh >Priority: Major > > Regarding the performance issue of SPARK-26155, I think it is better to add a > benchmark for LongToUnsafeRowMap which is the root cause of performance > regression. It can be easier to show performance difference between different > metric implementation in LongToUnsafeRowMap. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-26337) Add benchmark for LongToUnsafeRowMap
[ https://issues.apache.org/jira/browse/SPARK-26337?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16718033#comment-16718033 ] ASF GitHub Bot commented on SPARK-26337: dongjoon-hyun commented on a change in pull request #23284: [SPARK-26337][SQL][TEST] Add benchmark for LongToUnsafeRowMap URL: https://github.com/apache/spark/pull/23284#discussion_r240799221 ## File path: sql/core/benchmarks/HashedRelationMetricsBenchmark-results.txt ## @@ -6,6 +6,6 @@ Java HotSpot(TM) 64-Bit Server VM 1.8.0_181-b13 on Mac OS X 10.13.6 Intel(R) Core(TM) i7-7700HQ CPU @ 2.80GHz LongToUnsafeRowMap metrics: Best/Avg Time(ms)Rate(M/s) Per Row(ns) Relative -LongToUnsafeRowMap1265 / 1336 0.4 2530.5 1.0X +LongToUnsafeRowMap 234 / 315 2.1 467.3 1.0X Review comment: Yep. It's a clear reason to have this benchmark. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Add benchmark for LongToUnsafeRowMap > > > Key: SPARK-26337 > URL: https://issues.apache.org/jira/browse/SPARK-26337 > Project: Spark > Issue Type: Test > Components: SQL >Affects Versions: 3.0.0 >Reporter: Liang-Chi Hsieh >Priority: Major > > Regarding the performance issue of SPARK-26155, I think it is better to add a > benchmark for LongToUnsafeRowMap which is the root cause of performance > regression. It can be easier to show performance difference between different > metric implementation in LongToUnsafeRowMap. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-26337) Add benchmark for LongToUnsafeRowMap
[ https://issues.apache.org/jira/browse/SPARK-26337?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16718014#comment-16718014 ] ASF GitHub Bot commented on SPARK-26337: AmplabJenkins removed a comment on issue #23284: [SPARK-26337][SQL][TEST] Add benchmark for LongToUnsafeRowMap URL: https://github.com/apache/spark/pull/23284#issuecomment-446365591 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Add benchmark for LongToUnsafeRowMap > > > Key: SPARK-26337 > URL: https://issues.apache.org/jira/browse/SPARK-26337 > Project: Spark > Issue Type: Test > Components: SQL >Affects Versions: 3.0.0 >Reporter: Liang-Chi Hsieh >Priority: Major > > Regarding the performance issue of SPARK-26155, I think it is better to add a > benchmark for LongToUnsafeRowMap which is the root cause of performance > regression. It can be easier to show performance difference between different > metric implementation in LongToUnsafeRowMap. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-26337) Add benchmark for LongToUnsafeRowMap
[ https://issues.apache.org/jira/browse/SPARK-26337?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16718015#comment-16718015 ] ASF GitHub Bot commented on SPARK-26337: AmplabJenkins removed a comment on issue #23284: [SPARK-26337][SQL][TEST] Add benchmark for LongToUnsafeRowMap URL: https://github.com/apache/spark/pull/23284#issuecomment-446365598 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/99985/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Add benchmark for LongToUnsafeRowMap > > > Key: SPARK-26337 > URL: https://issues.apache.org/jira/browse/SPARK-26337 > Project: Spark > Issue Type: Test > Components: SQL >Affects Versions: 3.0.0 >Reporter: Liang-Chi Hsieh >Priority: Major > > Regarding the performance issue of SPARK-26155, I think it is better to add a > benchmark for LongToUnsafeRowMap which is the root cause of performance > regression. It can be easier to show performance difference between different > metric implementation in LongToUnsafeRowMap. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-26337) Add benchmark for LongToUnsafeRowMap
[ https://issues.apache.org/jira/browse/SPARK-26337?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16718013#comment-16718013 ] ASF GitHub Bot commented on SPARK-26337: AmplabJenkins commented on issue #23284: [SPARK-26337][SQL][TEST] Add benchmark for LongToUnsafeRowMap URL: https://github.com/apache/spark/pull/23284#issuecomment-446365598 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/99985/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Add benchmark for LongToUnsafeRowMap > > > Key: SPARK-26337 > URL: https://issues.apache.org/jira/browse/SPARK-26337 > Project: Spark > Issue Type: Test > Components: SQL >Affects Versions: 3.0.0 >Reporter: Liang-Chi Hsieh >Priority: Major > > Regarding the performance issue of SPARK-26155, I think it is better to add a > benchmark for LongToUnsafeRowMap which is the root cause of performance > regression. It can be easier to show performance difference between different > metric implementation in LongToUnsafeRowMap. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-26337) Add benchmark for LongToUnsafeRowMap
[ https://issues.apache.org/jira/browse/SPARK-26337?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16718012#comment-16718012 ] ASF GitHub Bot commented on SPARK-26337: AmplabJenkins commented on issue #23284: [SPARK-26337][SQL][TEST] Add benchmark for LongToUnsafeRowMap URL: https://github.com/apache/spark/pull/23284#issuecomment-446365591 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Add benchmark for LongToUnsafeRowMap > > > Key: SPARK-26337 > URL: https://issues.apache.org/jira/browse/SPARK-26337 > Project: Spark > Issue Type: Test > Components: SQL >Affects Versions: 3.0.0 >Reporter: Liang-Chi Hsieh >Priority: Major > > Regarding the performance issue of SPARK-26155, I think it is better to add a > benchmark for LongToUnsafeRowMap which is the root cause of performance > regression. It can be easier to show performance difference between different > metric implementation in LongToUnsafeRowMap. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-26337) Add benchmark for LongToUnsafeRowMap
[ https://issues.apache.org/jira/browse/SPARK-26337?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16718009#comment-16718009 ] ASF GitHub Bot commented on SPARK-26337: SparkQA removed a comment on issue #23284: [SPARK-26337][SQL][TEST] Add benchmark for LongToUnsafeRowMap URL: https://github.com/apache/spark/pull/23284#issuecomment-446291896 **[Test build #99985 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99985/testReport)** for PR 23284 at commit [`723b27c`](https://github.com/apache/spark/commit/723b27ce55770f5846d4244377b37f66f2f9ad60). This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Add benchmark for LongToUnsafeRowMap > > > Key: SPARK-26337 > URL: https://issues.apache.org/jira/browse/SPARK-26337 > Project: Spark > Issue Type: Test > Components: SQL >Affects Versions: 3.0.0 >Reporter: Liang-Chi Hsieh >Priority: Major > > Regarding the performance issue of SPARK-26155, I think it is better to add a > benchmark for LongToUnsafeRowMap which is the root cause of performance > regression. It can be easier to show performance difference between different > metric implementation in LongToUnsafeRowMap. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-26337) Add benchmark for LongToUnsafeRowMap
[ https://issues.apache.org/jira/browse/SPARK-26337?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16718008#comment-16718008 ] ASF GitHub Bot commented on SPARK-26337: SparkQA commented on issue #23284: [SPARK-26337][SQL][TEST] Add benchmark for LongToUnsafeRowMap URL: https://github.com/apache/spark/pull/23284#issuecomment-446365066 **[Test build #99985 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99985/testReport)** for PR 23284 at commit [`723b27c`](https://github.com/apache/spark/commit/723b27ce55770f5846d4244377b37f66f2f9ad60). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Add benchmark for LongToUnsafeRowMap > > > Key: SPARK-26337 > URL: https://issues.apache.org/jira/browse/SPARK-26337 > Project: Spark > Issue Type: Test > Components: SQL >Affects Versions: 3.0.0 >Reporter: Liang-Chi Hsieh >Priority: Major > > Regarding the performance issue of SPARK-26155, I think it is better to add a > benchmark for LongToUnsafeRowMap which is the root cause of performance > regression. It can be easier to show performance difference between different > metric implementation in LongToUnsafeRowMap. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-26337) Add benchmark for LongToUnsafeRowMap
[ https://issues.apache.org/jira/browse/SPARK-26337?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16717933#comment-16717933 ] ASF GitHub Bot commented on SPARK-26337: srowen commented on issue #23284: [SPARK-26337][SQL][TEST] Add benchmark for LongToUnsafeRowMap URL: https://github.com/apache/spark/pull/23284#issuecomment-446347640 There's no good reason why 64-bit reads/writes shouldn't be atomic on a 64-bit machine, and I assume everything we're testing on is 64-bit these days. It was an issue in the past, and yes as you note, the JLS seems to allow for it to be implementation-specific. No idea... This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Add benchmark for LongToUnsafeRowMap > > > Key: SPARK-26337 > URL: https://issues.apache.org/jira/browse/SPARK-26337 > Project: Spark > Issue Type: Test > Components: SQL >Affects Versions: 3.0.0 >Reporter: Liang-Chi Hsieh >Priority: Major > > Regarding the performance issue of SPARK-26155, I think it is better to add a > benchmark for LongToUnsafeRowMap which is the root cause of performance > regression. It can be easier to show performance difference between different > metric implementation in LongToUnsafeRowMap. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-26337) Add benchmark for LongToUnsafeRowMap
[ https://issues.apache.org/jira/browse/SPARK-26337?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16717665#comment-16717665 ] ASF GitHub Bot commented on SPARK-26337: kiszk commented on issue #23284: [SPARK-26337][SQL][TEST] Add benchmark for LongToUnsafeRowMap URL: https://github.com/apache/spark/pull/23284#issuecomment-446299651 To be honest, I cannot understand why the original performance degradation occurred. I think that read/write of `long` value does not require any sychronization or memory barrier without declaring `volatile`. At this [PR](https://github.com/apache/spark/pull/23204/files), `numKeyLookups` and `numProbes` are non-volatile long variables. I confirmed it by decompling a class file. However, I have no time to disassemble the generated code today and tomorrow. [Here](https://stackoverflow.com/questions/25173208/value-integrity-guarantee-for-concurrent-long-writes-in-64-bit-openjdk-7-8) is an article that addresses the similar topic regarding `static long`. cc @rednaxelafx This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Add benchmark for LongToUnsafeRowMap > > > Key: SPARK-26337 > URL: https://issues.apache.org/jira/browse/SPARK-26337 > Project: Spark > Issue Type: Test > Components: SQL >Affects Versions: 3.0.0 >Reporter: Liang-Chi Hsieh >Priority: Major > > Regarding the performance issue of SPARK-26155, I think it is better to add a > benchmark for LongToUnsafeRowMap which is the root cause of performance > regression. It can be easier to show performance difference between different > metric implementation in LongToUnsafeRowMap. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-26337) Add benchmark for LongToUnsafeRowMap
[ https://issues.apache.org/jira/browse/SPARK-26337?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16717633#comment-16717633 ] ASF GitHub Bot commented on SPARK-26337: AmplabJenkins removed a comment on issue #23284: [SPARK-26337][SQL][TEST] Add benchmark for LongToUnsafeRowMap URL: https://github.com/apache/spark/pull/23284#issuecomment-446291590 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/5984/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Add benchmark for LongToUnsafeRowMap > > > Key: SPARK-26337 > URL: https://issues.apache.org/jira/browse/SPARK-26337 > Project: Spark > Issue Type: Test > Components: SQL >Affects Versions: 3.0.0 >Reporter: Liang-Chi Hsieh >Priority: Major > > Regarding the performance issue of SPARK-26155, I think it is better to add a > benchmark for LongToUnsafeRowMap which is the root cause of performance > regression. It can be easier to show performance difference between different > metric implementation in LongToUnsafeRowMap. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-26337) Add benchmark for LongToUnsafeRowMap
[ https://issues.apache.org/jira/browse/SPARK-26337?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16717631#comment-16717631 ] ASF GitHub Bot commented on SPARK-26337: SparkQA commented on issue #23284: [SPARK-26337][SQL][TEST] Add benchmark for LongToUnsafeRowMap URL: https://github.com/apache/spark/pull/23284#issuecomment-446291896 **[Test build #99985 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99985/testReport)** for PR 23284 at commit [`723b27c`](https://github.com/apache/spark/commit/723b27ce55770f5846d4244377b37f66f2f9ad60). This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Add benchmark for LongToUnsafeRowMap > > > Key: SPARK-26337 > URL: https://issues.apache.org/jira/browse/SPARK-26337 > Project: Spark > Issue Type: Test > Components: SQL >Affects Versions: 3.0.0 >Reporter: Liang-Chi Hsieh >Priority: Major > > Regarding the performance issue of SPARK-26155, I think it is better to add a > benchmark for LongToUnsafeRowMap which is the root cause of performance > regression. It can be easier to show performance difference between different > metric implementation in LongToUnsafeRowMap. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-26337) Add benchmark for LongToUnsafeRowMap
[ https://issues.apache.org/jira/browse/SPARK-26337?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16717632#comment-16717632 ] ASF GitHub Bot commented on SPARK-26337: AmplabJenkins removed a comment on issue #23284: [SPARK-26337][SQL][TEST] Add benchmark for LongToUnsafeRowMap URL: https://github.com/apache/spark/pull/23284#issuecomment-446291579 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Add benchmark for LongToUnsafeRowMap > > > Key: SPARK-26337 > URL: https://issues.apache.org/jira/browse/SPARK-26337 > Project: Spark > Issue Type: Test > Components: SQL >Affects Versions: 3.0.0 >Reporter: Liang-Chi Hsieh >Priority: Major > > Regarding the performance issue of SPARK-26155, I think it is better to add a > benchmark for LongToUnsafeRowMap which is the root cause of performance > regression. It can be easier to show performance difference between different > metric implementation in LongToUnsafeRowMap. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-26337) Add benchmark for LongToUnsafeRowMap
[ https://issues.apache.org/jira/browse/SPARK-26337?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16717630#comment-16717630 ] ASF GitHub Bot commented on SPARK-26337: AmplabJenkins commented on issue #23284: [SPARK-26337][SQL][TEST] Add benchmark for LongToUnsafeRowMap URL: https://github.com/apache/spark/pull/23284#issuecomment-446291590 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/5984/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Add benchmark for LongToUnsafeRowMap > > > Key: SPARK-26337 > URL: https://issues.apache.org/jira/browse/SPARK-26337 > Project: Spark > Issue Type: Test > Components: SQL >Affects Versions: 3.0.0 >Reporter: Liang-Chi Hsieh >Priority: Major > > Regarding the performance issue of SPARK-26155, I think it is better to add a > benchmark for LongToUnsafeRowMap which is the root cause of performance > regression. It can be easier to show performance difference between different > metric implementation in LongToUnsafeRowMap. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-26337) Add benchmark for LongToUnsafeRowMap
[ https://issues.apache.org/jira/browse/SPARK-26337?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16717629#comment-16717629 ] ASF GitHub Bot commented on SPARK-26337: AmplabJenkins commented on issue #23284: [SPARK-26337][SQL][TEST] Add benchmark for LongToUnsafeRowMap URL: https://github.com/apache/spark/pull/23284#issuecomment-446291579 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Add benchmark for LongToUnsafeRowMap > > > Key: SPARK-26337 > URL: https://issues.apache.org/jira/browse/SPARK-26337 > Project: Spark > Issue Type: Test > Components: SQL >Affects Versions: 3.0.0 >Reporter: Liang-Chi Hsieh >Priority: Major > > Regarding the performance issue of SPARK-26155, I think it is better to add a > benchmark for LongToUnsafeRowMap which is the root cause of performance > regression. It can be easier to show performance difference between different > metric implementation in LongToUnsafeRowMap. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-26337) Add benchmark for LongToUnsafeRowMap
[ https://issues.apache.org/jira/browse/SPARK-26337?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16717620#comment-16717620 ] ASF GitHub Bot commented on SPARK-26337: kiszk commented on issue #23284: [SPARK-26337][SQL][TEST] Add benchmark for LongToUnsafeRowMap URL: https://github.com/apache/spark/pull/23284#issuecomment-446290221 retest this please This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Add benchmark for LongToUnsafeRowMap > > > Key: SPARK-26337 > URL: https://issues.apache.org/jira/browse/SPARK-26337 > Project: Spark > Issue Type: Test > Components: SQL >Affects Versions: 3.0.0 >Reporter: Liang-Chi Hsieh >Priority: Major > > Regarding the performance issue of SPARK-26155, I think it is better to add a > benchmark for LongToUnsafeRowMap which is the root cause of performance > regression. It can be easier to show performance difference between different > metric implementation in LongToUnsafeRowMap. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-26337) Add benchmark for LongToUnsafeRowMap
[ https://issues.apache.org/jira/browse/SPARK-26337?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16717574#comment-16717574 ] ASF GitHub Bot commented on SPARK-26337: AmplabJenkins removed a comment on issue #23284: [SPARK-26337][SQL][TEST] Add benchmark for LongToUnsafeRowMap URL: https://github.com/apache/spark/pull/23284#issuecomment-446284001 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/99979/ Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Add benchmark for LongToUnsafeRowMap > > > Key: SPARK-26337 > URL: https://issues.apache.org/jira/browse/SPARK-26337 > Project: Spark > Issue Type: Test > Components: SQL >Affects Versions: 3.0.0 >Reporter: Liang-Chi Hsieh >Priority: Major > > Regarding the performance issue of SPARK-26155, I think it is better to add a > benchmark for LongToUnsafeRowMap which is the root cause of performance > regression. It can be easier to show performance difference between different > metric implementation in LongToUnsafeRowMap. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-26337) Add benchmark for LongToUnsafeRowMap
[ https://issues.apache.org/jira/browse/SPARK-26337?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16717569#comment-16717569 ] ASF GitHub Bot commented on SPARK-26337: AmplabJenkins removed a comment on issue #23284: [SPARK-26337][SQL][TEST] Add benchmark for LongToUnsafeRowMap URL: https://github.com/apache/spark/pull/23284#issuecomment-446283989 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Add benchmark for LongToUnsafeRowMap > > > Key: SPARK-26337 > URL: https://issues.apache.org/jira/browse/SPARK-26337 > Project: Spark > Issue Type: Test > Components: SQL >Affects Versions: 3.0.0 >Reporter: Liang-Chi Hsieh >Priority: Major > > Regarding the performance issue of SPARK-26155, I think it is better to add a > benchmark for LongToUnsafeRowMap which is the root cause of performance > regression. It can be easier to show performance difference between different > metric implementation in LongToUnsafeRowMap. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-26337) Add benchmark for LongToUnsafeRowMap
[ https://issues.apache.org/jira/browse/SPARK-26337?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16717568#comment-16717568 ] ASF GitHub Bot commented on SPARK-26337: SparkQA removed a comment on issue #23284: [SPARK-26337][SQL][TEST] Add benchmark for LongToUnsafeRowMap URL: https://github.com/apache/spark/pull/23284#issuecomment-446218759 **[Test build #99979 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99979/testReport)** for PR 23284 at commit [`723b27c`](https://github.com/apache/spark/commit/723b27ce55770f5846d4244377b37f66f2f9ad60). This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Add benchmark for LongToUnsafeRowMap > > > Key: SPARK-26337 > URL: https://issues.apache.org/jira/browse/SPARK-26337 > Project: Spark > Issue Type: Test > Components: SQL >Affects Versions: 3.0.0 >Reporter: Liang-Chi Hsieh >Priority: Major > > Regarding the performance issue of SPARK-26155, I think it is better to add a > benchmark for LongToUnsafeRowMap which is the root cause of performance > regression. It can be easier to show performance difference between different > metric implementation in LongToUnsafeRowMap. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-26337) Add benchmark for LongToUnsafeRowMap
[ https://issues.apache.org/jira/browse/SPARK-26337?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16717565#comment-16717565 ] ASF GitHub Bot commented on SPARK-26337: AmplabJenkins commented on issue #23284: [SPARK-26337][SQL][TEST] Add benchmark for LongToUnsafeRowMap URL: https://github.com/apache/spark/pull/23284#issuecomment-446284001 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/99979/ Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Add benchmark for LongToUnsafeRowMap > > > Key: SPARK-26337 > URL: https://issues.apache.org/jira/browse/SPARK-26337 > Project: Spark > Issue Type: Test > Components: SQL >Affects Versions: 3.0.0 >Reporter: Liang-Chi Hsieh >Priority: Major > > Regarding the performance issue of SPARK-26155, I think it is better to add a > benchmark for LongToUnsafeRowMap which is the root cause of performance > regression. It can be easier to show performance difference between different > metric implementation in LongToUnsafeRowMap. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-26337) Add benchmark for LongToUnsafeRowMap
[ https://issues.apache.org/jira/browse/SPARK-26337?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16717564#comment-16717564 ] ASF GitHub Bot commented on SPARK-26337: AmplabJenkins commented on issue #23284: [SPARK-26337][SQL][TEST] Add benchmark for LongToUnsafeRowMap URL: https://github.com/apache/spark/pull/23284#issuecomment-446283989 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Add benchmark for LongToUnsafeRowMap > > > Key: SPARK-26337 > URL: https://issues.apache.org/jira/browse/SPARK-26337 > Project: Spark > Issue Type: Test > Components: SQL >Affects Versions: 3.0.0 >Reporter: Liang-Chi Hsieh >Priority: Major > > Regarding the performance issue of SPARK-26155, I think it is better to add a > benchmark for LongToUnsafeRowMap which is the root cause of performance > regression. It can be easier to show performance difference between different > metric implementation in LongToUnsafeRowMap. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-26337) Add benchmark for LongToUnsafeRowMap
[ https://issues.apache.org/jira/browse/SPARK-26337?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16717562#comment-16717562 ] ASF GitHub Bot commented on SPARK-26337: SparkQA commented on issue #23284: [SPARK-26337][SQL][TEST] Add benchmark for LongToUnsafeRowMap URL: https://github.com/apache/spark/pull/23284#issuecomment-446283643 **[Test build #99979 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99979/testReport)** for PR 23284 at commit [`723b27c`](https://github.com/apache/spark/commit/723b27ce55770f5846d4244377b37f66f2f9ad60). * This patch **fails Spark unit tests**. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Add benchmark for LongToUnsafeRowMap > > > Key: SPARK-26337 > URL: https://issues.apache.org/jira/browse/SPARK-26337 > Project: Spark > Issue Type: Test > Components: SQL >Affects Versions: 3.0.0 >Reporter: Liang-Chi Hsieh >Priority: Major > > Regarding the performance issue of SPARK-26155, I think it is better to add a > benchmark for LongToUnsafeRowMap which is the root cause of performance > regression. It can be easier to show performance difference between different > metric implementation in LongToUnsafeRowMap. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-26337) Add benchmark for LongToUnsafeRowMap
[ https://issues.apache.org/jira/browse/SPARK-26337?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16717526#comment-16717526 ] ASF GitHub Bot commented on SPARK-26337: AmplabJenkins removed a comment on issue #23284: [SPARK-26337][SQL][TEST] Add benchmark for LongToUnsafeRowMap URL: https://github.com/apache/spark/pull/23284#issuecomment-446273530 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Add benchmark for LongToUnsafeRowMap > > > Key: SPARK-26337 > URL: https://issues.apache.org/jira/browse/SPARK-26337 > Project: Spark > Issue Type: Test > Components: SQL >Affects Versions: 3.0.0 >Reporter: Liang-Chi Hsieh >Priority: Major > > Regarding the performance issue of SPARK-26155, I think it is better to add a > benchmark for LongToUnsafeRowMap which is the root cause of performance > regression. It can be easier to show performance difference between different > metric implementation in LongToUnsafeRowMap. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-26337) Add benchmark for LongToUnsafeRowMap
[ https://issues.apache.org/jira/browse/SPARK-26337?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16717545#comment-16717545 ] ASF GitHub Bot commented on SPARK-26337: dongjoon-hyun commented on issue #23284: [SPARK-26337][SQL][TEST] Add benchmark for LongToUnsafeRowMap URL: https://github.com/apache/spark/pull/23284#issuecomment-446280368 Thank you for ping me, @viirya and @cloud-fan . This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Add benchmark for LongToUnsafeRowMap > > > Key: SPARK-26337 > URL: https://issues.apache.org/jira/browse/SPARK-26337 > Project: Spark > Issue Type: Test > Components: SQL >Affects Versions: 3.0.0 >Reporter: Liang-Chi Hsieh >Priority: Major > > Regarding the performance issue of SPARK-26155, I think it is better to add a > benchmark for LongToUnsafeRowMap which is the root cause of performance > regression. It can be easier to show performance difference between different > metric implementation in LongToUnsafeRowMap. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-26337) Add benchmark for LongToUnsafeRowMap
[ https://issues.apache.org/jira/browse/SPARK-26337?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16717521#comment-16717521 ] ASF GitHub Bot commented on SPARK-26337: AmplabJenkins commented on issue #23284: [SPARK-26337][SQL][TEST] Add benchmark for LongToUnsafeRowMap URL: https://github.com/apache/spark/pull/23284#issuecomment-446273530 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Add benchmark for LongToUnsafeRowMap > > > Key: SPARK-26337 > URL: https://issues.apache.org/jira/browse/SPARK-26337 > Project: Spark > Issue Type: Test > Components: SQL >Affects Versions: 3.0.0 >Reporter: Liang-Chi Hsieh >Priority: Major > > Regarding the performance issue of SPARK-26155, I think it is better to add a > benchmark for LongToUnsafeRowMap which is the root cause of performance > regression. It can be easier to show performance difference between different > metric implementation in LongToUnsafeRowMap. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-26337) Add benchmark for LongToUnsafeRowMap
[ https://issues.apache.org/jira/browse/SPARK-26337?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16717516#comment-16717516 ] ASF GitHub Bot commented on SPARK-26337: SparkQA commented on issue #23284: [SPARK-26337][SQL][TEST] Add benchmark for LongToUnsafeRowMap URL: https://github.com/apache/spark/pull/23284#issuecomment-446272649 **[Test build #99973 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99973/testReport)** for PR 23284 at commit [`ccb6fed`](https://github.com/apache/spark/commit/ccb6fedb0fcccb49b1ae64bd4f88de65dae7697e). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Add benchmark for LongToUnsafeRowMap > > > Key: SPARK-26337 > URL: https://issues.apache.org/jira/browse/SPARK-26337 > Project: Spark > Issue Type: Test > Components: SQL >Affects Versions: 3.0.0 >Reporter: Liang-Chi Hsieh >Priority: Major > > Regarding the performance issue of SPARK-26155, I think it is better to add a > benchmark for LongToUnsafeRowMap which is the root cause of performance > regression. It can be easier to show performance difference between different > metric implementation in LongToUnsafeRowMap. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-26337) Add benchmark for LongToUnsafeRowMap
[ https://issues.apache.org/jira/browse/SPARK-26337?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16717527#comment-16717527 ] ASF GitHub Bot commented on SPARK-26337: AmplabJenkins removed a comment on issue #23284: [SPARK-26337][SQL][TEST] Add benchmark for LongToUnsafeRowMap URL: https://github.com/apache/spark/pull/23284#issuecomment-446273536 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/99973/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Add benchmark for LongToUnsafeRowMap > > > Key: SPARK-26337 > URL: https://issues.apache.org/jira/browse/SPARK-26337 > Project: Spark > Issue Type: Test > Components: SQL >Affects Versions: 3.0.0 >Reporter: Liang-Chi Hsieh >Priority: Major > > Regarding the performance issue of SPARK-26155, I think it is better to add a > benchmark for LongToUnsafeRowMap which is the root cause of performance > regression. It can be easier to show performance difference between different > metric implementation in LongToUnsafeRowMap. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-26337) Add benchmark for LongToUnsafeRowMap
[ https://issues.apache.org/jira/browse/SPARK-26337?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16717522#comment-16717522 ] ASF GitHub Bot commented on SPARK-26337: AmplabJenkins commented on issue #23284: [SPARK-26337][SQL][TEST] Add benchmark for LongToUnsafeRowMap URL: https://github.com/apache/spark/pull/23284#issuecomment-446273536 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/99973/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Add benchmark for LongToUnsafeRowMap > > > Key: SPARK-26337 > URL: https://issues.apache.org/jira/browse/SPARK-26337 > Project: Spark > Issue Type: Test > Components: SQL >Affects Versions: 3.0.0 >Reporter: Liang-Chi Hsieh >Priority: Major > > Regarding the performance issue of SPARK-26155, I think it is better to add a > benchmark for LongToUnsafeRowMap which is the root cause of performance > regression. It can be easier to show performance difference between different > metric implementation in LongToUnsafeRowMap. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-26337) Add benchmark for LongToUnsafeRowMap
[ https://issues.apache.org/jira/browse/SPARK-26337?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16717518#comment-16717518 ] ASF GitHub Bot commented on SPARK-26337: SparkQA removed a comment on issue #23284: [SPARK-26337][SQL][TEST] Add benchmark for LongToUnsafeRowMap URL: https://github.com/apache/spark/pull/23284#issuecomment-446195917 **[Test build #99973 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99973/testReport)** for PR 23284 at commit [`ccb6fed`](https://github.com/apache/spark/commit/ccb6fedb0fcccb49b1ae64bd4f88de65dae7697e). This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Add benchmark for LongToUnsafeRowMap > > > Key: SPARK-26337 > URL: https://issues.apache.org/jira/browse/SPARK-26337 > Project: Spark > Issue Type: Test > Components: SQL >Affects Versions: 3.0.0 >Reporter: Liang-Chi Hsieh >Priority: Major > > Regarding the performance issue of SPARK-26155, I think it is better to add a > benchmark for LongToUnsafeRowMap which is the root cause of performance > regression. It can be easier to show performance difference between different > metric implementation in LongToUnsafeRowMap. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-26337) Add benchmark for LongToUnsafeRowMap
[ https://issues.apache.org/jira/browse/SPARK-26337?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16717413#comment-16717413 ] ASF GitHub Bot commented on SPARK-26337: cloud-fan commented on issue #23284: [SPARK-26337][SQL][TEST] Add benchmark for LongToUnsafeRowMap URL: https://github.com/apache/spark/pull/23284#issuecomment-446250046 BTW I think this proves that, in Java if `long` is accessed by multiple threads, it will cause perf problems even without lock. Maybe it's related to memory barrier. cc @kiszk @dongjoon-hyun @JkSelf This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Add benchmark for LongToUnsafeRowMap > > > Key: SPARK-26337 > URL: https://issues.apache.org/jira/browse/SPARK-26337 > Project: Spark > Issue Type: Test > Components: SQL >Affects Versions: 3.0.0 >Reporter: Liang-Chi Hsieh >Priority: Major > > Regarding the performance issue of SPARK-26155, I think it is better to add a > benchmark for LongToUnsafeRowMap which is the root cause of performance > regression. It can be easier to show performance difference between different > metric implementation in LongToUnsafeRowMap. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-26337) Add benchmark for LongToUnsafeRowMap
[ https://issues.apache.org/jira/browse/SPARK-26337?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16717412#comment-16717412 ] ASF GitHub Bot commented on SPARK-26337: cloud-fan commented on issue #23284: [SPARK-26337][SQL][TEST] Add benchmark for LongToUnsafeRowMap URL: https://github.com/apache/spark/pull/23284#issuecomment-446249386 can you update https://github.com/apache/spark/pull/23284#issuecomment-446143380 ? The revert PR is already merged, so we should revert the revert PR and run the benchmark again, and post the results in the comment. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Add benchmark for LongToUnsafeRowMap > > > Key: SPARK-26337 > URL: https://issues.apache.org/jira/browse/SPARK-26337 > Project: Spark > Issue Type: Test > Components: SQL >Affects Versions: 3.0.0 >Reporter: Liang-Chi Hsieh >Priority: Major > > Regarding the performance issue of SPARK-26155, I think it is better to add a > benchmark for LongToUnsafeRowMap which is the root cause of performance > regression. It can be easier to show performance difference between different > metric implementation in LongToUnsafeRowMap. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-26337) Add benchmark for LongToUnsafeRowMap
[ https://issues.apache.org/jira/browse/SPARK-26337?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16717282#comment-16717282 ] ASF GitHub Bot commented on SPARK-26337: AmplabJenkins removed a comment on issue #23284: [SPARK-26337][SQL][TEST] Add benchmark for LongToUnsafeRowMap URL: https://github.com/apache/spark/pull/23284#issuecomment-446226292 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Add benchmark for LongToUnsafeRowMap > > > Key: SPARK-26337 > URL: https://issues.apache.org/jira/browse/SPARK-26337 > Project: Spark > Issue Type: Test > Components: SQL >Affects Versions: 3.0.0 >Reporter: Liang-Chi Hsieh >Priority: Major > > Regarding the performance issue of SPARK-26155, I think it is better to add a > benchmark for LongToUnsafeRowMap which is the root cause of performance > regression. It can be easier to show performance difference between different > metric implementation in LongToUnsafeRowMap. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-26337) Add benchmark for LongToUnsafeRowMap
[ https://issues.apache.org/jira/browse/SPARK-26337?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16717285#comment-16717285 ] ASF GitHub Bot commented on SPARK-26337: AmplabJenkins removed a comment on issue #23284: [SPARK-26337][SQL][TEST] Add benchmark for LongToUnsafeRowMap URL: https://github.com/apache/spark/pull/23284#issuecomment-446226299 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/99974/ Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Add benchmark for LongToUnsafeRowMap > > > Key: SPARK-26337 > URL: https://issues.apache.org/jira/browse/SPARK-26337 > Project: Spark > Issue Type: Test > Components: SQL >Affects Versions: 3.0.0 >Reporter: Liang-Chi Hsieh >Priority: Major > > Regarding the performance issue of SPARK-26155, I think it is better to add a > benchmark for LongToUnsafeRowMap which is the root cause of performance > regression. It can be easier to show performance difference between different > metric implementation in LongToUnsafeRowMap. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-26337) Add benchmark for LongToUnsafeRowMap
[ https://issues.apache.org/jira/browse/SPARK-26337?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16717280#comment-16717280 ] ASF GitHub Bot commented on SPARK-26337: AmplabJenkins commented on issue #23284: [SPARK-26337][SQL][TEST] Add benchmark for LongToUnsafeRowMap URL: https://github.com/apache/spark/pull/23284#issuecomment-446226299 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/99974/ Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Add benchmark for LongToUnsafeRowMap > > > Key: SPARK-26337 > URL: https://issues.apache.org/jira/browse/SPARK-26337 > Project: Spark > Issue Type: Test > Components: SQL >Affects Versions: 3.0.0 >Reporter: Liang-Chi Hsieh >Priority: Major > > Regarding the performance issue of SPARK-26155, I think it is better to add a > benchmark for LongToUnsafeRowMap which is the root cause of performance > regression. It can be easier to show performance difference between different > metric implementation in LongToUnsafeRowMap. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-26337) Add benchmark for LongToUnsafeRowMap
[ https://issues.apache.org/jira/browse/SPARK-26337?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16717281#comment-16717281 ] ASF GitHub Bot commented on SPARK-26337: SparkQA removed a comment on issue #23284: [SPARK-26337][SQL][TEST] Add benchmark for LongToUnsafeRowMap URL: https://github.com/apache/spark/pull/23284#issuecomment-446197580 **[Test build #99974 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99974/testReport)** for PR 23284 at commit [`cdbae0a`](https://github.com/apache/spark/commit/cdbae0a43d4ae01889feb66f5e0e7d55799e6dda). This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Add benchmark for LongToUnsafeRowMap > > > Key: SPARK-26337 > URL: https://issues.apache.org/jira/browse/SPARK-26337 > Project: Spark > Issue Type: Test > Components: SQL >Affects Versions: 3.0.0 >Reporter: Liang-Chi Hsieh >Priority: Major > > Regarding the performance issue of SPARK-26155, I think it is better to add a > benchmark for LongToUnsafeRowMap which is the root cause of performance > regression. It can be easier to show performance difference between different > metric implementation in LongToUnsafeRowMap. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-26337) Add benchmark for LongToUnsafeRowMap
[ https://issues.apache.org/jira/browse/SPARK-26337?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16717277#comment-16717277 ] ASF GitHub Bot commented on SPARK-26337: SparkQA commented on issue #23284: [SPARK-26337][SQL][TEST] Add benchmark for LongToUnsafeRowMap URL: https://github.com/apache/spark/pull/23284#issuecomment-446226047 **[Test build #99974 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99974/testReport)** for PR 23284 at commit [`cdbae0a`](https://github.com/apache/spark/commit/cdbae0a43d4ae01889feb66f5e0e7d55799e6dda). * This patch **fails Spark unit tests**. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Add benchmark for LongToUnsafeRowMap > > > Key: SPARK-26337 > URL: https://issues.apache.org/jira/browse/SPARK-26337 > Project: Spark > Issue Type: Test > Components: SQL >Affects Versions: 3.0.0 >Reporter: Liang-Chi Hsieh >Priority: Major > > Regarding the performance issue of SPARK-26155, I think it is better to add a > benchmark for LongToUnsafeRowMap which is the root cause of performance > regression. It can be easier to show performance difference between different > metric implementation in LongToUnsafeRowMap. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-26337) Add benchmark for LongToUnsafeRowMap
[ https://issues.apache.org/jira/browse/SPARK-26337?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16717279#comment-16717279 ] ASF GitHub Bot commented on SPARK-26337: AmplabJenkins commented on issue #23284: [SPARK-26337][SQL][TEST] Add benchmark for LongToUnsafeRowMap URL: https://github.com/apache/spark/pull/23284#issuecomment-446226292 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Add benchmark for LongToUnsafeRowMap > > > Key: SPARK-26337 > URL: https://issues.apache.org/jira/browse/SPARK-26337 > Project: Spark > Issue Type: Test > Components: SQL >Affects Versions: 3.0.0 >Reporter: Liang-Chi Hsieh >Priority: Major > > Regarding the performance issue of SPARK-26155, I think it is better to add a > benchmark for LongToUnsafeRowMap which is the root cause of performance > regression. It can be easier to show performance difference between different > metric implementation in LongToUnsafeRowMap. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-26337) Add benchmark for LongToUnsafeRowMap
[ https://issues.apache.org/jira/browse/SPARK-26337?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16717245#comment-16717245 ] ASF GitHub Bot commented on SPARK-26337: SparkQA commented on issue #23284: [SPARK-26337][SQL][TEST] Add benchmark for LongToUnsafeRowMap URL: https://github.com/apache/spark/pull/23284#issuecomment-446218759 **[Test build #99979 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99979/testReport)** for PR 23284 at commit [`723b27c`](https://github.com/apache/spark/commit/723b27ce55770f5846d4244377b37f66f2f9ad60). This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Add benchmark for LongToUnsafeRowMap > > > Key: SPARK-26337 > URL: https://issues.apache.org/jira/browse/SPARK-26337 > Project: Spark > Issue Type: Test > Components: SQL >Affects Versions: 3.0.0 >Reporter: Liang-Chi Hsieh >Priority: Major > > Regarding the performance issue of SPARK-26155, I think it is better to add a > benchmark for LongToUnsafeRowMap which is the root cause of performance > regression. It can be easier to show performance difference between different > metric implementation in LongToUnsafeRowMap. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-26337) Add benchmark for LongToUnsafeRowMap
[ https://issues.apache.org/jira/browse/SPARK-26337?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16717250#comment-16717250 ] ASF GitHub Bot commented on SPARK-26337: AmplabJenkins removed a comment on issue #23284: [SPARK-26337][SQL][TEST] Add benchmark for LongToUnsafeRowMap URL: https://github.com/apache/spark/pull/23284#issuecomment-446218765 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/5978/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Add benchmark for LongToUnsafeRowMap > > > Key: SPARK-26337 > URL: https://issues.apache.org/jira/browse/SPARK-26337 > Project: Spark > Issue Type: Test > Components: SQL >Affects Versions: 3.0.0 >Reporter: Liang-Chi Hsieh >Priority: Major > > Regarding the performance issue of SPARK-26155, I think it is better to add a > benchmark for LongToUnsafeRowMap which is the root cause of performance > regression. It can be easier to show performance difference between different > metric implementation in LongToUnsafeRowMap. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-26337) Add benchmark for LongToUnsafeRowMap
[ https://issues.apache.org/jira/browse/SPARK-26337?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16717249#comment-16717249 ] ASF GitHub Bot commented on SPARK-26337: AmplabJenkins removed a comment on issue #23284: [SPARK-26337][SQL][TEST] Add benchmark for LongToUnsafeRowMap URL: https://github.com/apache/spark/pull/23284#issuecomment-446218757 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Add benchmark for LongToUnsafeRowMap > > > Key: SPARK-26337 > URL: https://issues.apache.org/jira/browse/SPARK-26337 > Project: Spark > Issue Type: Test > Components: SQL >Affects Versions: 3.0.0 >Reporter: Liang-Chi Hsieh >Priority: Major > > Regarding the performance issue of SPARK-26155, I think it is better to add a > benchmark for LongToUnsafeRowMap which is the root cause of performance > regression. It can be easier to show performance difference between different > metric implementation in LongToUnsafeRowMap. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-26337) Add benchmark for LongToUnsafeRowMap
[ https://issues.apache.org/jira/browse/SPARK-26337?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16717246#comment-16717246 ] ASF GitHub Bot commented on SPARK-26337: AmplabJenkins commented on issue #23284: [SPARK-26337][SQL][TEST] Add benchmark for LongToUnsafeRowMap URL: https://github.com/apache/spark/pull/23284#issuecomment-446218765 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/5978/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Add benchmark for LongToUnsafeRowMap > > > Key: SPARK-26337 > URL: https://issues.apache.org/jira/browse/SPARK-26337 > Project: Spark > Issue Type: Test > Components: SQL >Affects Versions: 3.0.0 >Reporter: Liang-Chi Hsieh >Priority: Major > > Regarding the performance issue of SPARK-26155, I think it is better to add a > benchmark for LongToUnsafeRowMap which is the root cause of performance > regression. It can be easier to show performance difference between different > metric implementation in LongToUnsafeRowMap. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-26337) Add benchmark for LongToUnsafeRowMap
[ https://issues.apache.org/jira/browse/SPARK-26337?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16717244#comment-16717244 ] ASF GitHub Bot commented on SPARK-26337: AmplabJenkins commented on issue #23284: [SPARK-26337][SQL][TEST] Add benchmark for LongToUnsafeRowMap URL: https://github.com/apache/spark/pull/23284#issuecomment-446218757 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Add benchmark for LongToUnsafeRowMap > > > Key: SPARK-26337 > URL: https://issues.apache.org/jira/browse/SPARK-26337 > Project: Spark > Issue Type: Test > Components: SQL >Affects Versions: 3.0.0 >Reporter: Liang-Chi Hsieh >Priority: Major > > Regarding the performance issue of SPARK-26155, I think it is better to add a > benchmark for LongToUnsafeRowMap which is the root cause of performance > regression. It can be easier to show performance difference between different > metric implementation in LongToUnsafeRowMap. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-26337) Add benchmark for LongToUnsafeRowMap
[
https://issues.apache.org/jira/browse/SPARK-26337?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16717228#comment-16717228
]
ASF GitHub Bot commented on SPARK-26337:
viirya commented on a change in pull request #23284: [SPARK-26337][SQL][TEST]
Add benchmark for LongToUnsafeRowMap
URL: https://github.com/apache/spark/pull/23284#discussion_r240623884
##
File path:
sql/core/src/test/scala/org/apache/spark/sql/execution/benchmark/HashedRelationMetricsBenchmark.scala
##
@@ -0,0 +1,84 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ *http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.spark.sql.execution.benchmark
+
+import org.apache.spark.SparkConf
+import org.apache.spark.benchmark.Benchmark
+import org.apache.spark.internal.config.MEMORY_OFFHEAP_ENABLED
+import org.apache.spark.memory.{StaticMemoryManager, TaskMemoryManager}
+import org.apache.spark.sql.catalyst.InternalRow
+import org.apache.spark.sql.catalyst.expressions.{BoundReference,
UnsafeProjection}
+import org.apache.spark.sql.execution.joins.LongToUnsafeRowMap
+import org.apache.spark.sql.types.LongType
+
+/**
+ * Benchmark to measure metrics performance at HashedRelation.
+ * To run this benchmark:
+ * {{{
+ * 1. without sbt: bin/spark-submit --class
+ * 2. build/sbt "sql/test:runMain "
+ * 3. generate result: SPARK_GENERATE_BENCHMARK_FILES=1 build/sbt
"sql/test:runMain "
+ * Results will be written to
"benchmarks/HashedRelationMetricsBenchmark-results.txt".
Review comment:
The "revert" PR gets merged. Let me update the benchmark result.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Add benchmark for LongToUnsafeRowMap
>
>
> Key: SPARK-26337
> URL: https://issues.apache.org/jira/browse/SPARK-26337
> Project: Spark
> Issue Type: Test
> Components: SQL
>Affects Versions: 3.0.0
>Reporter: Liang-Chi Hsieh
>Priority: Major
>
> Regarding the performance issue of SPARK-26155, I think it is better to add a
> benchmark for LongToUnsafeRowMap which is the root cause of performance
> regression. It can be easier to show performance difference between different
> metric implementation in LongToUnsafeRowMap.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-26337) Add benchmark for LongToUnsafeRowMap
[ https://issues.apache.org/jira/browse/SPARK-26337?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16717189#comment-16717189 ] ASF GitHub Bot commented on SPARK-26337: AmplabJenkins commented on issue #23284: [SPARK-26337][SQL][TEST] Add benchmark for LongToUnsafeRowMap URL: https://github.com/apache/spark/pull/23284#issuecomment-446208310 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Add benchmark for LongToUnsafeRowMap > > > Key: SPARK-26337 > URL: https://issues.apache.org/jira/browse/SPARK-26337 > Project: Spark > Issue Type: Test > Components: SQL >Affects Versions: 3.0.0 >Reporter: Liang-Chi Hsieh >Priority: Major > > Regarding the performance issue of SPARK-26155, I think it is better to add a > benchmark for LongToUnsafeRowMap which is the root cause of performance > regression. It can be easier to show performance difference between different > metric implementation in LongToUnsafeRowMap. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-26337) Add benchmark for LongToUnsafeRowMap
[ https://issues.apache.org/jira/browse/SPARK-26337?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16717193#comment-16717193 ] ASF GitHub Bot commented on SPARK-26337: AmplabJenkins removed a comment on issue #23284: [SPARK-26337][SQL][TEST] Add benchmark for LongToUnsafeRowMap URL: https://github.com/apache/spark/pull/23284#issuecomment-446208316 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/99964/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Add benchmark for LongToUnsafeRowMap > > > Key: SPARK-26337 > URL: https://issues.apache.org/jira/browse/SPARK-26337 > Project: Spark > Issue Type: Test > Components: SQL >Affects Versions: 3.0.0 >Reporter: Liang-Chi Hsieh >Priority: Major > > Regarding the performance issue of SPARK-26155, I think it is better to add a > benchmark for LongToUnsafeRowMap which is the root cause of performance > regression. It can be easier to show performance difference between different > metric implementation in LongToUnsafeRowMap. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-26337) Add benchmark for LongToUnsafeRowMap
[ https://issues.apache.org/jira/browse/SPARK-26337?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16717190#comment-16717190 ] ASF GitHub Bot commented on SPARK-26337: AmplabJenkins commented on issue #23284: [SPARK-26337][SQL][TEST] Add benchmark for LongToUnsafeRowMap URL: https://github.com/apache/spark/pull/23284#issuecomment-446208316 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/99964/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Add benchmark for LongToUnsafeRowMap > > > Key: SPARK-26337 > URL: https://issues.apache.org/jira/browse/SPARK-26337 > Project: Spark > Issue Type: Test > Components: SQL >Affects Versions: 3.0.0 >Reporter: Liang-Chi Hsieh >Priority: Major > > Regarding the performance issue of SPARK-26155, I think it is better to add a > benchmark for LongToUnsafeRowMap which is the root cause of performance > regression. It can be easier to show performance difference between different > metric implementation in LongToUnsafeRowMap. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-26337) Add benchmark for LongToUnsafeRowMap
[ https://issues.apache.org/jira/browse/SPARK-26337?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16717192#comment-16717192 ] ASF GitHub Bot commented on SPARK-26337: AmplabJenkins removed a comment on issue #23284: [SPARK-26337][SQL][TEST] Add benchmark for LongToUnsafeRowMap URL: https://github.com/apache/spark/pull/23284#issuecomment-446208310 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Add benchmark for LongToUnsafeRowMap > > > Key: SPARK-26337 > URL: https://issues.apache.org/jira/browse/SPARK-26337 > Project: Spark > Issue Type: Test > Components: SQL >Affects Versions: 3.0.0 >Reporter: Liang-Chi Hsieh >Priority: Major > > Regarding the performance issue of SPARK-26155, I think it is better to add a > benchmark for LongToUnsafeRowMap which is the root cause of performance > regression. It can be easier to show performance difference between different > metric implementation in LongToUnsafeRowMap. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-26337) Add benchmark for LongToUnsafeRowMap
[ https://issues.apache.org/jira/browse/SPARK-26337?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16717182#comment-16717182 ] ASF GitHub Bot commented on SPARK-26337: SparkQA removed a comment on issue #23284: [SPARK-26337][SQL][TEST] Add benchmark for LongToUnsafeRowMap URL: https://github.com/apache/spark/pull/23284#issuecomment-446143804 **[Test build #99964 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99964/testReport)** for PR 23284 at commit [`fa70205`](https://github.com/apache/spark/commit/fa702056efb1cd1a8a635c05908964d8ad96bb3a). This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Add benchmark for LongToUnsafeRowMap > > > Key: SPARK-26337 > URL: https://issues.apache.org/jira/browse/SPARK-26337 > Project: Spark > Issue Type: Test > Components: SQL >Affects Versions: 3.0.0 >Reporter: Liang-Chi Hsieh >Priority: Major > > Regarding the performance issue of SPARK-26155, I think it is better to add a > benchmark for LongToUnsafeRowMap which is the root cause of performance > regression. It can be easier to show performance difference between different > metric implementation in LongToUnsafeRowMap. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-26337) Add benchmark for LongToUnsafeRowMap
[ https://issues.apache.org/jira/browse/SPARK-26337?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16717178#comment-16717178 ] ASF GitHub Bot commented on SPARK-26337: SparkQA commented on issue #23284: [SPARK-26337][SQL][TEST] Add benchmark for LongToUnsafeRowMap URL: https://github.com/apache/spark/pull/23284#issuecomment-446207762 **[Test build #99964 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99964/testReport)** for PR 23284 at commit [`fa70205`](https://github.com/apache/spark/commit/fa702056efb1cd1a8a635c05908964d8ad96bb3a). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Add benchmark for LongToUnsafeRowMap > > > Key: SPARK-26337 > URL: https://issues.apache.org/jira/browse/SPARK-26337 > Project: Spark > Issue Type: Test > Components: SQL >Affects Versions: 3.0.0 >Reporter: Liang-Chi Hsieh >Priority: Major > > Regarding the performance issue of SPARK-26155, I think it is better to add a > benchmark for LongToUnsafeRowMap which is the root cause of performance > regression. It can be easier to show performance difference between different > metric implementation in LongToUnsafeRowMap. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-26337) Add benchmark for LongToUnsafeRowMap
[ https://issues.apache.org/jira/browse/SPARK-26337?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16717109#comment-16717109 ] ASF GitHub Bot commented on SPARK-26337: AmplabJenkins removed a comment on issue #23284: [SPARK-26337][SQL][TEST] Add benchmark for LongToUnsafeRowMap URL: https://github.com/apache/spark/pull/23284#issuecomment-446197534 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/5973/ Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Add benchmark for LongToUnsafeRowMap > > > Key: SPARK-26337 > URL: https://issues.apache.org/jira/browse/SPARK-26337 > Project: Spark > Issue Type: Test > Components: SQL >Affects Versions: 3.0.0 >Reporter: Liang-Chi Hsieh >Priority: Major > > Regarding the performance issue of SPARK-26155, I think it is better to add a > benchmark for LongToUnsafeRowMap which is the root cause of performance > regression. It can be easier to show performance difference between different > metric implementation in LongToUnsafeRowMap. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-26337) Add benchmark for LongToUnsafeRowMap
[ https://issues.apache.org/jira/browse/SPARK-26337?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16717108#comment-16717108 ] ASF GitHub Bot commented on SPARK-26337: AmplabJenkins removed a comment on issue #23284: [SPARK-26337][SQL][TEST] Add benchmark for LongToUnsafeRowMap URL: https://github.com/apache/spark/pull/23284#issuecomment-446197528 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Add benchmark for LongToUnsafeRowMap > > > Key: SPARK-26337 > URL: https://issues.apache.org/jira/browse/SPARK-26337 > Project: Spark > Issue Type: Test > Components: SQL >Affects Versions: 3.0.0 >Reporter: Liang-Chi Hsieh >Priority: Major > > Regarding the performance issue of SPARK-26155, I think it is better to add a > benchmark for LongToUnsafeRowMap which is the root cause of performance > regression. It can be easier to show performance difference between different > metric implementation in LongToUnsafeRowMap. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-26337) Add benchmark for LongToUnsafeRowMap
[ https://issues.apache.org/jira/browse/SPARK-26337?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16717104#comment-16717104 ] ASF GitHub Bot commented on SPARK-26337: AmplabJenkins commented on issue #23284: [SPARK-26337][SQL][TEST] Add benchmark for LongToUnsafeRowMap URL: https://github.com/apache/spark/pull/23284#issuecomment-446197528 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Add benchmark for LongToUnsafeRowMap > > > Key: SPARK-26337 > URL: https://issues.apache.org/jira/browse/SPARK-26337 > Project: Spark > Issue Type: Test > Components: SQL >Affects Versions: 3.0.0 >Reporter: Liang-Chi Hsieh >Priority: Major > > Regarding the performance issue of SPARK-26155, I think it is better to add a > benchmark for LongToUnsafeRowMap which is the root cause of performance > regression. It can be easier to show performance difference between different > metric implementation in LongToUnsafeRowMap. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-26337) Add benchmark for LongToUnsafeRowMap
[ https://issues.apache.org/jira/browse/SPARK-26337?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16717106#comment-16717106 ] ASF GitHub Bot commented on SPARK-26337: AmplabJenkins commented on issue #23284: [SPARK-26337][SQL][TEST] Add benchmark for LongToUnsafeRowMap URL: https://github.com/apache/spark/pull/23284#issuecomment-446197534 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/5973/ Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Add benchmark for LongToUnsafeRowMap > > > Key: SPARK-26337 > URL: https://issues.apache.org/jira/browse/SPARK-26337 > Project: Spark > Issue Type: Test > Components: SQL >Affects Versions: 3.0.0 >Reporter: Liang-Chi Hsieh >Priority: Major > > Regarding the performance issue of SPARK-26155, I think it is better to add a > benchmark for LongToUnsafeRowMap which is the root cause of performance > regression. It can be easier to show performance difference between different > metric implementation in LongToUnsafeRowMap. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-26337) Add benchmark for LongToUnsafeRowMap
[ https://issues.apache.org/jira/browse/SPARK-26337?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16717107#comment-16717107 ] ASF GitHub Bot commented on SPARK-26337: SparkQA commented on issue #23284: [SPARK-26337][SQL][TEST] Add benchmark for LongToUnsafeRowMap URL: https://github.com/apache/spark/pull/23284#issuecomment-446197580 **[Test build #99974 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99974/testReport)** for PR 23284 at commit [`cdbae0a`](https://github.com/apache/spark/commit/cdbae0a43d4ae01889feb66f5e0e7d55799e6dda). This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Add benchmark for LongToUnsafeRowMap > > > Key: SPARK-26337 > URL: https://issues.apache.org/jira/browse/SPARK-26337 > Project: Spark > Issue Type: Test > Components: SQL >Affects Versions: 3.0.0 >Reporter: Liang-Chi Hsieh >Priority: Major > > Regarding the performance issue of SPARK-26155, I think it is better to add a > benchmark for LongToUnsafeRowMap which is the root cause of performance > regression. It can be easier to show performance difference between different > metric implementation in LongToUnsafeRowMap. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-26337) Add benchmark for LongToUnsafeRowMap
[
https://issues.apache.org/jira/browse/SPARK-26337?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16717097#comment-16717097
]
ASF GitHub Bot commented on SPARK-26337:
viirya commented on a change in pull request #23284: [SPARK-26337][SQL][TEST]
Add benchmark for LongToUnsafeRowMap
URL: https://github.com/apache/spark/pull/23284#discussion_r240598831
##
File path:
sql/core/src/test/scala/org/apache/spark/sql/execution/benchmark/HashedRelationMetricsBenchmark.scala
##
@@ -0,0 +1,84 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ *http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.spark.sql.execution.benchmark
+
+import org.apache.spark.SparkConf
+import org.apache.spark.benchmark.Benchmark
+import org.apache.spark.internal.config.MEMORY_OFFHEAP_ENABLED
+import org.apache.spark.memory.{StaticMemoryManager, TaskMemoryManager}
+import org.apache.spark.sql.catalyst.InternalRow
+import org.apache.spark.sql.catalyst.expressions.{BoundReference,
UnsafeProjection}
+import org.apache.spark.sql.execution.joins.LongToUnsafeRowMap
+import org.apache.spark.sql.types.LongType
+
+/**
+ * Benchmark to measure metrics performance at HashedRelation.
+ * To run this benchmark:
+ * {{{
+ * 1. without sbt: bin/spark-submit --class
+ * 2. build/sbt "sql/test:runMain "
+ * 3. generate result: SPARK_GENERATE_BENCHMARK_FILES=1 build/sbt
"sql/test:runMain "
+ * Results will be written to
"benchmarks/HashedRelationMetricsBenchmark-results.txt".
Review comment:
oh, actually I can do it by removing the metrics myself.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Add benchmark for LongToUnsafeRowMap
>
>
> Key: SPARK-26337
> URL: https://issues.apache.org/jira/browse/SPARK-26337
> Project: Spark
> Issue Type: Test
> Components: SQL
>Affects Versions: 3.0.0
>Reporter: Liang-Chi Hsieh
>Priority: Major
>
> Regarding the performance issue of SPARK-26155, I think it is better to add a
> benchmark for LongToUnsafeRowMap which is the root cause of performance
> regression. It can be easier to show performance difference between different
> metric implementation in LongToUnsafeRowMap.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-26337) Add benchmark for LongToUnsafeRowMap
[ https://issues.apache.org/jira/browse/SPARK-26337?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16717083#comment-16717083 ] ASF GitHub Bot commented on SPARK-26337: SparkQA commented on issue #23284: [SPARK-26337][SQL][TEST] Add benchmark for LongToUnsafeRowMap URL: https://github.com/apache/spark/pull/23284#issuecomment-446195917 **[Test build #99973 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99973/testReport)** for PR 23284 at commit [`ccb6fed`](https://github.com/apache/spark/commit/ccb6fedb0fcccb49b1ae64bd4f88de65dae7697e). This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Add benchmark for LongToUnsafeRowMap > > > Key: SPARK-26337 > URL: https://issues.apache.org/jira/browse/SPARK-26337 > Project: Spark > Issue Type: Test > Components: SQL >Affects Versions: 3.0.0 >Reporter: Liang-Chi Hsieh >Priority: Major > > Regarding the performance issue of SPARK-26155, I think it is better to add a > benchmark for LongToUnsafeRowMap which is the root cause of performance > regression. It can be easier to show performance difference between different > metric implementation in LongToUnsafeRowMap. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-26337) Add benchmark for LongToUnsafeRowMap
[
https://issues.apache.org/jira/browse/SPARK-26337?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16717080#comment-16717080
]
ASF GitHub Bot commented on SPARK-26337:
viirya commented on a change in pull request #23284: [SPARK-26337][SQL][TEST]
Add benchmark for LongToUnsafeRowMap
URL: https://github.com/apache/spark/pull/23284#discussion_r240601636
##
File path:
sql/core/src/test/scala/org/apache/spark/sql/execution/benchmark/HashedRelationMetricsBenchmark.scala
##
@@ -0,0 +1,84 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ *http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.spark.sql.execution.benchmark
+
+import org.apache.spark.SparkConf
+import org.apache.spark.benchmark.Benchmark
+import org.apache.spark.internal.config.MEMORY_OFFHEAP_ENABLED
+import org.apache.spark.memory.{StaticMemoryManager, TaskMemoryManager}
+import org.apache.spark.sql.catalyst.InternalRow
+import org.apache.spark.sql.catalyst.expressions.{BoundReference,
UnsafeProjection}
+import org.apache.spark.sql.execution.joins.LongToUnsafeRowMap
+import org.apache.spark.sql.types.LongType
+
+/**
+ * Benchmark to measure metrics performance at HashedRelation.
+ * To run this benchmark:
+ * {{{
+ * 1. without sbt: bin/spark-submit --class
+ * 2. build/sbt "sql/test:runMain "
+ * 3. generate result: SPARK_GENERATE_BENCHMARK_FILES=1 build/sbt
"sql/test:runMain "
+ * Results will be written to
"benchmarks/HashedRelationMetricsBenchmark-results.txt".
Review comment:
The result was checked in.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Add benchmark for LongToUnsafeRowMap
>
>
> Key: SPARK-26337
> URL: https://issues.apache.org/jira/browse/SPARK-26337
> Project: Spark
> Issue Type: Test
> Components: SQL
>Affects Versions: 3.0.0
>Reporter: Liang-Chi Hsieh
>Priority: Major
>
> Regarding the performance issue of SPARK-26155, I think it is better to add a
> benchmark for LongToUnsafeRowMap which is the root cause of performance
> regression. It can be easier to show performance difference between different
> metric implementation in LongToUnsafeRowMap.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-26337) Add benchmark for LongToUnsafeRowMap
[
https://issues.apache.org/jira/browse/SPARK-26337?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16717078#comment-16717078
]
ASF GitHub Bot commented on SPARK-26337:
cloud-fan commented on a change in pull request #23284:
[SPARK-26337][SQL][TEST] Add benchmark for LongToUnsafeRowMap
URL: https://github.com/apache/spark/pull/23284#discussion_r240600768
##
File path:
sql/core/src/test/scala/org/apache/spark/sql/execution/benchmark/HashedRelationMetricsBenchmark.scala
##
@@ -0,0 +1,84 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ *http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.spark.sql.execution.benchmark
+
+import org.apache.spark.SparkConf
+import org.apache.spark.benchmark.Benchmark
+import org.apache.spark.internal.config.MEMORY_OFFHEAP_ENABLED
+import org.apache.spark.memory.{StaticMemoryManager, TaskMemoryManager}
+import org.apache.spark.sql.catalyst.InternalRow
+import org.apache.spark.sql.catalyst.expressions.{BoundReference,
UnsafeProjection}
+import org.apache.spark.sql.execution.joins.LongToUnsafeRowMap
+import org.apache.spark.sql.types.LongType
+
+/**
+ * Benchmark to measure metrics performance at HashedRelation.
+ * To run this benchmark:
+ * {{{
+ * 1. without sbt: bin/spark-submit --class
+ * 2. build/sbt "sql/test:runMain "
+ * 3. generate result: SPARK_GENERATE_BENCHMARK_FILES=1 build/sbt
"sql/test:runMain "
+ * Results will be written to
"benchmarks/HashedRelationMetricsBenchmark-results.txt".
Review comment:
the "revert" PR need to update the result, if your PR gets merged first.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Add benchmark for LongToUnsafeRowMap
>
>
> Key: SPARK-26337
> URL: https://issues.apache.org/jira/browse/SPARK-26337
> Project: Spark
> Issue Type: Test
> Components: SQL
>Affects Versions: 3.0.0
>Reporter: Liang-Chi Hsieh
>Priority: Major
>
> Regarding the performance issue of SPARK-26155, I think it is better to add a
> benchmark for LongToUnsafeRowMap which is the root cause of performance
> regression. It can be easier to show performance difference between different
> metric implementation in LongToUnsafeRowMap.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-26337) Add benchmark for LongToUnsafeRowMap
[
https://issues.apache.org/jira/browse/SPARK-26337?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16717069#comment-16717069
]
ASF GitHub Bot commented on SPARK-26337:
viirya commented on a change in pull request #23284: [SPARK-26337][SQL][TEST]
Add benchmark for LongToUnsafeRowMap
URL: https://github.com/apache/spark/pull/23284#discussion_r240598831
##
File path:
sql/core/src/test/scala/org/apache/spark/sql/execution/benchmark/HashedRelationMetricsBenchmark.scala
##
@@ -0,0 +1,84 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ *http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.spark.sql.execution.benchmark
+
+import org.apache.spark.SparkConf
+import org.apache.spark.benchmark.Benchmark
+import org.apache.spark.internal.config.MEMORY_OFFHEAP_ENABLED
+import org.apache.spark.memory.{StaticMemoryManager, TaskMemoryManager}
+import org.apache.spark.sql.catalyst.InternalRow
+import org.apache.spark.sql.catalyst.expressions.{BoundReference,
UnsafeProjection}
+import org.apache.spark.sql.execution.joins.LongToUnsafeRowMap
+import org.apache.spark.sql.types.LongType
+
+/**
+ * Benchmark to measure metrics performance at HashedRelation.
+ * To run this benchmark:
+ * {{{
+ * 1. without sbt: bin/spark-submit --class
+ * 2. build/sbt "sql/test:runMain "
+ * 3. generate result: SPARK_GENERATE_BENCHMARK_FILES=1 build/sbt
"sql/test:runMain "
+ * Results will be written to
"benchmarks/HashedRelationMetricsBenchmark-results.txt".
Review comment:
oh, actually I can do it by removing the metrics myself.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Add benchmark for LongToUnsafeRowMap
>
>
> Key: SPARK-26337
> URL: https://issues.apache.org/jira/browse/SPARK-26337
> Project: Spark
> Issue Type: Test
> Components: SQL
>Affects Versions: 3.0.0
>Reporter: Liang-Chi Hsieh
>Priority: Major
>
> Regarding the performance issue of SPARK-26155, I think it is better to add a
> benchmark for LongToUnsafeRowMap which is the root cause of performance
> regression. It can be easier to show performance difference between different
> metric implementation in LongToUnsafeRowMap.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-26337) Add benchmark for LongToUnsafeRowMap
[
https://issues.apache.org/jira/browse/SPARK-26337?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16716934#comment-16716934
]
ASF GitHub Bot commented on SPARK-26337:
viirya commented on a change in pull request #23284: [SPARK-26337][SQL][TEST]
Add benchmark for LongToUnsafeRowMap
URL: https://github.com/apache/spark/pull/23284#discussion_r240581421
##
File path:
sql/core/src/test/scala/org/apache/spark/sql/execution/benchmark/HashedRelationMetricsBenchmark.scala
##
@@ -0,0 +1,84 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ *http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.spark.sql.execution.benchmark
+
+import org.apache.spark.SparkConf
+import org.apache.spark.benchmark.Benchmark
+import org.apache.spark.internal.config.MEMORY_OFFHEAP_ENABLED
+import org.apache.spark.memory.{StaticMemoryManager, TaskMemoryManager}
+import org.apache.spark.sql.catalyst.InternalRow
+import org.apache.spark.sql.catalyst.expressions.{BoundReference,
UnsafeProjection}
+import org.apache.spark.sql.execution.joins.LongToUnsafeRowMap
+import org.apache.spark.sql.types.LongType
+
+/**
+ * Benchmark to measure metrics performance at HashedRelation.
+ * To run this benchmark:
+ * {{{
+ * 1. without sbt: bin/spark-submit --class
+ * 2. build/sbt "sql/test:runMain "
+ * 3. generate result: SPARK_GENERATE_BENCHMARK_FILES=1 build/sbt
"sql/test:runMain "
+ * Results will be written to
"benchmarks/HashedRelationMetricsBenchmark-results.txt".
Review comment:
I see. I think we should check in the result after the metrics is removed.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Add benchmark for LongToUnsafeRowMap
>
>
> Key: SPARK-26337
> URL: https://issues.apache.org/jira/browse/SPARK-26337
> Project: Spark
> Issue Type: Test
> Components: SQL
>Affects Versions: 3.0.0
>Reporter: Liang-Chi Hsieh
>Priority: Major
>
> Regarding the performance issue of SPARK-26155, I think it is better to add a
> benchmark for LongToUnsafeRowMap which is the root cause of performance
> regression. It can be easier to show performance difference between different
> metric implementation in LongToUnsafeRowMap.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-26337) Add benchmark for LongToUnsafeRowMap
[
https://issues.apache.org/jira/browse/SPARK-26337?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16716814#comment-16716814
]
ASF GitHub Bot commented on SPARK-26337:
cloud-fan commented on a change in pull request #23284:
[SPARK-26337][SQL][TEST] Add benchmark for LongToUnsafeRowMap
URL: https://github.com/apache/spark/pull/23284#discussion_r240556557
##
File path:
sql/core/src/test/scala/org/apache/spark/sql/execution/benchmark/HashedRelationMetricsBenchmark.scala
##
@@ -0,0 +1,84 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ *http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.spark.sql.execution.benchmark
+
+import org.apache.spark.SparkConf
+import org.apache.spark.benchmark.Benchmark
+import org.apache.spark.internal.config.MEMORY_OFFHEAP_ENABLED
+import org.apache.spark.memory.{StaticMemoryManager, TaskMemoryManager}
+import org.apache.spark.sql.catalyst.InternalRow
+import org.apache.spark.sql.catalyst.expressions.{BoundReference,
UnsafeProjection}
+import org.apache.spark.sql.execution.joins.LongToUnsafeRowMap
+import org.apache.spark.sql.types.LongType
+
+/**
+ * Benchmark to measure metrics performance at HashedRelation.
+ * To run this benchmark:
+ * {{{
+ * 1. without sbt: bin/spark-submit --class
+ * 2. build/sbt "sql/test:runMain "
+ * 3. generate result: SPARK_GENERATE_BENCHMARK_FILES=1 build/sbt
"sql/test:runMain "
+ * Results will be written to
"benchmarks/HashedRelationMetricsBenchmark-results.txt".
Review comment:
we should also check in the result
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Add benchmark for LongToUnsafeRowMap
>
>
> Key: SPARK-26337
> URL: https://issues.apache.org/jira/browse/SPARK-26337
> Project: Spark
> Issue Type: Test
> Components: SQL
>Affects Versions: 3.0.0
>Reporter: Liang-Chi Hsieh
>Priority: Major
>
> Regarding the performance issue of SPARK-26155, I think it is better to add a
> benchmark for LongToUnsafeRowMap which is the root cause of performance
> regression. It can be easier to show performance difference between different
> metric implementation in LongToUnsafeRowMap.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-26337) Add benchmark for LongToUnsafeRowMap
[ https://issues.apache.org/jira/browse/SPARK-26337?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16716745#comment-16716745 ] ASF GitHub Bot commented on SPARK-26337: SparkQA commented on issue #23284: [SPARK-26337][SQL][TEST] Add benchmark for LongToUnsafeRowMap URL: https://github.com/apache/spark/pull/23284#issuecomment-446143804 **[Test build #99964 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99964/testReport)** for PR 23284 at commit [`fa70205`](https://github.com/apache/spark/commit/fa702056efb1cd1a8a635c05908964d8ad96bb3a). This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Add benchmark for LongToUnsafeRowMap > > > Key: SPARK-26337 > URL: https://issues.apache.org/jira/browse/SPARK-26337 > Project: Spark > Issue Type: Test > Components: SQL >Affects Versions: 3.0.0 >Reporter: Liang-Chi Hsieh >Priority: Major > > Regarding the performance issue of SPARK-26155, I think it is better to add a > benchmark for LongToUnsafeRowMap which is the root cause of performance > regression. It can be easier to show performance difference between different > metric implementation in LongToUnsafeRowMap. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-26337) Add benchmark for LongToUnsafeRowMap
[ https://issues.apache.org/jira/browse/SPARK-26337?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16716748#comment-16716748 ] ASF GitHub Bot commented on SPARK-26337: AmplabJenkins commented on issue #23284: [SPARK-26337][SQL][TEST] Add benchmark for LongToUnsafeRowMap URL: https://github.com/apache/spark/pull/23284#issuecomment-446143869 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Add benchmark for LongToUnsafeRowMap > > > Key: SPARK-26337 > URL: https://issues.apache.org/jira/browse/SPARK-26337 > Project: Spark > Issue Type: Test > Components: SQL >Affects Versions: 3.0.0 >Reporter: Liang-Chi Hsieh >Priority: Major > > Regarding the performance issue of SPARK-26155, I think it is better to add a > benchmark for LongToUnsafeRowMap which is the root cause of performance > regression. It can be easier to show performance difference between different > metric implementation in LongToUnsafeRowMap. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-26337) Add benchmark for LongToUnsafeRowMap
[ https://issues.apache.org/jira/browse/SPARK-26337?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16716749#comment-16716749 ] ASF GitHub Bot commented on SPARK-26337: AmplabJenkins commented on issue #23284: [SPARK-26337][SQL][TEST] Add benchmark for LongToUnsafeRowMap URL: https://github.com/apache/spark/pull/23284#issuecomment-446143875 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/5965/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Add benchmark for LongToUnsafeRowMap > > > Key: SPARK-26337 > URL: https://issues.apache.org/jira/browse/SPARK-26337 > Project: Spark > Issue Type: Test > Components: SQL >Affects Versions: 3.0.0 >Reporter: Liang-Chi Hsieh >Priority: Major > > Regarding the performance issue of SPARK-26155, I think it is better to add a > benchmark for LongToUnsafeRowMap which is the root cause of performance > regression. It can be easier to show performance difference between different > metric implementation in LongToUnsafeRowMap. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-26337) Add benchmark for LongToUnsafeRowMap
[ https://issues.apache.org/jira/browse/SPARK-26337?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16716753#comment-16716753 ] ASF GitHub Bot commented on SPARK-26337: AmplabJenkins removed a comment on issue #23284: [SPARK-26337][SQL][TEST] Add benchmark for LongToUnsafeRowMap URL: https://github.com/apache/spark/pull/23284#issuecomment-446143869 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Add benchmark for LongToUnsafeRowMap > > > Key: SPARK-26337 > URL: https://issues.apache.org/jira/browse/SPARK-26337 > Project: Spark > Issue Type: Test > Components: SQL >Affects Versions: 3.0.0 >Reporter: Liang-Chi Hsieh >Priority: Major > > Regarding the performance issue of SPARK-26155, I think it is better to add a > benchmark for LongToUnsafeRowMap which is the root cause of performance > regression. It can be easier to show performance difference between different > metric implementation in LongToUnsafeRowMap. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-26337) Add benchmark for LongToUnsafeRowMap
[ https://issues.apache.org/jira/browse/SPARK-26337?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16716754#comment-16716754 ] ASF GitHub Bot commented on SPARK-26337: AmplabJenkins removed a comment on issue #23284: [SPARK-26337][SQL][TEST] Add benchmark for LongToUnsafeRowMap URL: https://github.com/apache/spark/pull/23284#issuecomment-446143875 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/5965/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Add benchmark for LongToUnsafeRowMap > > > Key: SPARK-26337 > URL: https://issues.apache.org/jira/browse/SPARK-26337 > Project: Spark > Issue Type: Test > Components: SQL >Affects Versions: 3.0.0 >Reporter: Liang-Chi Hsieh >Priority: Major > > Regarding the performance issue of SPARK-26155, I think it is better to add a > benchmark for LongToUnsafeRowMap which is the root cause of performance > regression. It can be easier to show performance difference between different > metric implementation in LongToUnsafeRowMap. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-26337) Add benchmark for LongToUnsafeRowMap
[ https://issues.apache.org/jira/browse/SPARK-26337?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16716743#comment-16716743 ] ASF GitHub Bot commented on SPARK-26337: viirya commented on issue #23284: [SPARK-26337][SQL][TEST] Add benchmark for LongToUnsafeRowMap URL: https://github.com/apache/spark/pull/23284#issuecomment-446143692 cc @cloud-fan @dongjoon-hyun This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Add benchmark for LongToUnsafeRowMap > > > Key: SPARK-26337 > URL: https://issues.apache.org/jira/browse/SPARK-26337 > Project: Spark > Issue Type: Test > Components: SQL >Affects Versions: 3.0.0 >Reporter: Liang-Chi Hsieh >Priority: Major > > Regarding the performance issue of SPARK-26155, I think it is better to add a > benchmark for LongToUnsafeRowMap which is the root cause of performance > regression. It can be easier to show performance difference between different > metric implementation in LongToUnsafeRowMap. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-26337) Add benchmark for LongToUnsafeRowMap
[ https://issues.apache.org/jira/browse/SPARK-26337?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16716733#comment-16716733 ] ASF GitHub Bot commented on SPARK-26337: viirya commented on issue #23284: [SPARK-26337][SQL][TEST] Add benchmark for LongToUnsafeRowMap URL: https://github.com/apache/spark/pull/23284#issuecomment-446143380 ``` Using Long variables (master): [info] Java HotSpot(TM) 64-Bit Server VM 1.8.0_181-b13 on Mac OS X 10.13.6 [info] Intel(R) Core(TM) i7-7700HQ CPU @ 2.80GHz [info] LongToUnsafeRowMap metrics: Best/Avg Time(ms) Rate(M/s) Per Row(ns) Relative [info] [info] LongToUnsafeRowMap1220 / 1233 0.42439.1 1.0X LongAdder (PR 23214): [info] Java HotSpot(TM) 64-Bit Server VM 1.8.0_181-b13 on Mac OS X 10.13.6 [info] Intel(R) Core(TM) i7-7700HQ CPU @ 2.80GHz [info] LongToUnsafeRowMap metrics: Best/Avg Time(ms) Rate(M/s) Per Row(ns) Relative [info] [info] LongToUnsafeRowMap 401 / 460 1.2 802.2 1.0X No metrics: [info] Java HotSpot(TM) 64-Bit Server VM 1.8.0_181-b13 on Mac OS X 10.13.6 [info] Intel(R) Core(TM) i7-7700HQ CPU @ 2.80GHz [info] LongToUnsafeRowMap metrics: Best/Avg Time(ms) Rate(M/s) Per Row(ns) Relative [info] [info] LongToUnsafeRowMap 243 / 347 2.1 485.0 1.0X ``` This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Add benchmark for LongToUnsafeRowMap > > > Key: SPARK-26337 > URL: https://issues.apache.org/jira/browse/SPARK-26337 > Project: Spark > Issue Type: Test > Components: SQL >Affects Versions: 3.0.0 >Reporter: Liang-Chi Hsieh >Priority: Major > > Regarding the performance issue of SPARK-26155, I think it is better to add a > benchmark for LongToUnsafeRowMap which is the root cause of performance > regression. It can be easier to show performance difference between different > metric implementation in LongToUnsafeRowMap. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-26337) Add benchmark for LongToUnsafeRowMap
[ https://issues.apache.org/jira/browse/SPARK-26337?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16716727#comment-16716727 ] ASF GitHub Bot commented on SPARK-26337: viirya opened a new pull request #23284: [SPARK-26337][SQL][TEST] Add benchmark for LongToUnsafeRowMap URL: https://github.com/apache/spark/pull/23284 ## What changes were proposed in this pull request? Regarding the performance issue of SPARK-26155, it reports the issue on TPC-DS. I think it is better to add a benchmark for `LongToUnsafeRowMap` which is the root cause of performance regression. It can be easier to show performance difference between different metric implementations in `LongToUnsafeRowMap`. ## How was this patch tested? Added test. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Add benchmark for LongToUnsafeRowMap > > > Key: SPARK-26337 > URL: https://issues.apache.org/jira/browse/SPARK-26337 > Project: Spark > Issue Type: Test > Components: SQL >Affects Versions: 3.0.0 >Reporter: Liang-Chi Hsieh >Priority: Major > > Regarding the performance issue of SPARK-26155, I think it is better to add a > benchmark for LongToUnsafeRowMap which is the root cause of performance > regression. It can be easier to show performance difference between different > metric implementation in LongToUnsafeRowMap. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
