Github user rxin commented on the pull request:
https://github.com/apache/spark/pull/7418#issuecomment-122186634
Thanks for understanding. Do you mind closing this one first? If we need to
merge this one, let's reopen it.
---
If your project is set up for it, you can reply to this
Github user viirya commented on the pull request:
https://github.com/apache/spark/pull/7418#issuecomment-122189888
ok. no problem.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this
Github user chenghao-intel commented on the pull request:
https://github.com/apache/spark/pull/7418#issuecomment-122191060
@viirya, I mean you at least have a single group (less than 50 fields), it
probably cause performance issues for case like:
`select a+b from src`, as the
Github user viirya closed the pull request at:
https://github.com/apache/spark/pull/7418
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is
Github user viirya commented on the pull request:
https://github.com/apache/spark/pull/7418#issuecomment-122197117
@chenghao-intel no, if there are less 50 columns, the generated java codes
are as the same as the before.
---
If your project is set up for it, you can reply to this
Github user chenghao-intel commented on a diff in the pull request:
https://github.com/apache/spark/pull/7418#discussion_r34866421
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/GenerateMutableProjection.scala
---
@@ -45,10 +45,31 @@
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/7418#issuecomment-121843409
[Test build #37461 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/37461/console)
for PR 7418 at commit
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/7418#issuecomment-121843429
Merged build finished. Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/7418#issuecomment-121846610
[Test build #37464 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/37464/consoleFull)
for PR 7418 at commit
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/7418#issuecomment-121846213
Merged build triggered.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/7418#issuecomment-121846231
Merged build started.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/7418#issuecomment-121872112
[Test build #37464 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/37464/console)
for PR 7418 at commit
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/7418#issuecomment-121872267
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your
Github user viirya commented on the pull request:
https://github.com/apache/spark/pull/7418#issuecomment-122153575
@chenghao-intel This pr already groups a certain number of projection
columns (now it is 50) into a function and call it. It is due to the problem
caused by a very long
Github user rxin commented on the pull request:
https://github.com/apache/spark/pull/7418#issuecomment-122177398
@viirya let's use #7076 since it was first submitted and had a test case.
If @saurfang doesn't respond, let's merge this one. Thanks.
---
If your project is set up for
Github user viirya commented on the pull request:
https://github.com/apache/spark/pull/7418#issuecomment-122183508
@rxin sure, no problem.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
Github user chenghao-intel commented on the pull request:
https://github.com/apache/spark/pull/7418#issuecomment-122137758
Or at least, we can group several project fields in a codegened function,
and give a configuration to enable the codegen grouping.
---
If your project is set
Github user chenghao-intel commented on the pull request:
https://github.com/apache/spark/pull/7418#issuecomment-122137477
My concern on this fix is, that probably cause performance issue, if we add
a function call for each of the projection field, as codegen aim to inline the
logic
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/7418#issuecomment-121827685
[Test build #37461 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/37461/consoleFull)
for PR 7418 at commit
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/7418#issuecomment-121827606
Merged build triggered.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/7418#issuecomment-121827618
Merged build started.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not
Github user chenghao-intel commented on a diff in the pull request:
https://github.com/apache/spark/pull/7418#discussion_r34752387
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/GenerateMutableProjection.scala
---
@@ -45,10 +45,32 @@
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/7418#issuecomment-121810983
[Test build #37444 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/37444/consoleFull)
for PR 7418 at commit
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/7418#issuecomment-121826150
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/7418#issuecomment-121826077
[Test build #37444 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/37444/console)
for PR 7418 at commit
Github user chenghao-intel commented on a diff in the pull request:
https://github.com/apache/spark/pull/7418#discussion_r34752283
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/GenerateMutableProjection.scala
---
@@ -45,10 +45,32 @@
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/7418#issuecomment-121810218
Merged build triggered.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/7418#issuecomment-121810230
Merged build started.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not
GitHub user viirya opened a pull request:
https://github.com/apache/spark/pull/7418
[SPARK-9058][SQL] Split projectionCode if it is too large for JVM
JIRA: https://issues.apache.org/jira/browse/SPARK-9058
There is a limit for code size in a function in JVM. If the generated
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/7418#issuecomment-121547828
[Test build #37349 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/37349/consoleFull)
for PR 7418 at commit
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/7418#issuecomment-121547266
Merged build triggered.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/7418#issuecomment-121547333
Merged build started.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/7418#issuecomment-121574042
Merged build finished. Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/7418#issuecomment-121574016
[Test build #37349 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/37349/console)
for PR 7418 at commit
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/7418#issuecomment-121632113
Merged build started.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/7418#issuecomment-121632087
Merged build triggered.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/7418#issuecomment-121632903
[Test build #37363 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/37363/consoleFull)
for PR 7418 at commit
Github user viirya commented on the pull request:
https://github.com/apache/spark/pull/7418#issuecomment-121659150
This is created for the new JIRA ticket. I didn't notice there is already a
related one.
---
If your project is set up for it, you can reply to this email and have your
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/7418#issuecomment-121660626
[Test build #37363 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/37363/console)
for PR 7418 at commit
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/7418#issuecomment-121660744
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your
Github user maropu commented on the pull request:
https://github.com/apache/spark/pull/7418#issuecomment-121654917
This patch seems to be duplicated with #7076.
Why you make a new PR?
---
If your project is set up for it, you can reply to this email and have your
reply appear on
Github user maropu commented on a diff in the pull request:
https://github.com/apache/spark/pull/7418#discussion_r34692458
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/GenerateMutableProjection.scala
---
@@ -45,7 +45,28 @@ object
42 matches
Mail list logo
