Github user zzcclp commented on the pull request:
https://github.com/apache/spark/pull/5154#issuecomment-85877991
@chenghao-intel , can you review again, thanks.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/5154#issuecomment-85879026
[Test build #29152 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/29152/consoleFull)
for PR 5154 at commit
Github user zzcclp commented on the pull request:
https://github.com/apache/spark/pull/5154#issuecomment-85882008
@AmplabJenkins , please re test
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not
Github user chenghao-intel commented on a diff in the pull request:
https://github.com/apache/spark/pull/5154#discussion_r27100123
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/ScalaUdf.scala
---
@@ -39,363 +39,668 @@ case class
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/5154#issuecomment-85909601
[Test build #29152 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/29152/consoleFull)
for PR 5154 at commit
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/5154#issuecomment-85909608
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user zzcclp commented on the pull request:
https://github.com/apache/spark/pull/5154#issuecomment-85911436
Add a newline at the end of source file, does it need to re test?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub
Github user zzcclp commented on the pull request:
https://github.com/apache/spark/pull/5154#issuecomment-85925500
@AmplabJenkins , please re test, thanks.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project
Github user zzcclp commented on the pull request:
https://github.com/apache/spark/pull/5154#issuecomment-85936678
OK, I am new sparker, :smile:
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/5154#issuecomment-85912545
[Test build #29157 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/29157/consoleFull)
for PR 5154 at commit
Github user chenghao-intel commented on the pull request:
https://github.com/apache/spark/pull/5154#issuecomment-85935997
The unit test will be auto-triggered, once the code changed, you needn't
say anything to @AmplabJenkins .
---
If your project is set up for it, you can reply to
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/5154#issuecomment-85957300
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/5154#issuecomment-85957281
[Test build #29157 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/29157/consoleFull)
for PR 5154 at commit
Github user zzcclp commented on the pull request:
https://github.com/apache/spark/pull/5154#issuecomment-86305650
@liancheng , I already set my real name on my GitHub *Name* and JIRA *Full
Name*. :smiley:
---
If your project is set up for it, you can reply to this email and have
Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/5154
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is
Github user liancheng commented on the pull request:
https://github.com/apache/spark/pull/5154#issuecomment-85986863
@zzcclp Would you please set your real name on both GitHub and JIRA so that
our script can put your name on the credit list of the next release? Also, it
would be good
Github user liancheng commented on the pull request:
https://github.com/apache/spark/pull/5154#issuecomment-85985315
@zzcclp @chenghao-intel Thanks for working on this and the review comments!
Merged to master.
---
If your project is set up for it, you can reply to this email and
Github user chenghao-intel commented on the pull request:
https://github.com/apache/spark/pull/5154#issuecomment-85569221
Verified the code change by the following micro-benchmark
```scala
import org.apache.spark.sql.catalyst.expressions._
import
Github user chenghao-intel commented on a diff in the pull request:
https://github.com/apache/spark/pull/5154#discussion_r27089272
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/ScalaUdf.scala
---
@@ -30,372 +30,423 @@ case class
Github user zzcclp commented on the pull request:
https://github.com/apache/spark/pull/5154#issuecomment-85805763
@chenghao-intel , I have update the code. can you take a look again. thanks.
@AmplabJenkins , re-test
---
If your project is set up for it, you can reply to
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/5154#issuecomment-85806931
[Test build #29142 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/29142/consoleFull)
for PR 5154 at commit
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/5154#issuecomment-85806948
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/5154#issuecomment-85768748
[Test build #29134 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/29134/consoleFull)
for PR 5154 at commit
Github user chenghao-intel commented on the pull request:
https://github.com/apache/spark/pull/5154#issuecomment-85775183
@zzcclp LGTM, except some small issues.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/5154#issuecomment-85791483
[Test build #29134 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/29134/consoleFull)
for PR 5154 at commit
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/5154#issuecomment-85805945
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/5154#issuecomment-85805918
[Test build #29141 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/29141/consoleFull)
for PR 5154 at commit
Github user zzcclp commented on the pull request:
https://github.com/apache/spark/pull/5154#issuecomment-85768950
@chenghao-intel ,I have update the code. can you take a look again. thanks.
---
If your project is set up for it, you can reply to this email and have your
reply appear
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/5154#issuecomment-85791509
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
Github user zzcclp commented on the pull request:
https://github.com/apache/spark/pull/5154#issuecomment-85797911
@SparkQA , merge again.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/5154#issuecomment-85797757
[Test build #29141 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/29141/consoleFull)
for PR 5154 at commit
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/5154#issuecomment-85800269
[Test build #29142 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/29142/consoleFull)
for PR 5154 at commit
Github user chenghao-intel commented on the pull request:
https://github.com/apache/spark/pull/5154#issuecomment-85772375
You need to fetch the latest code and resolve the conflicts.
---
If your project is set up for it, you can reply to this email and have your
reply appear on
Github user chenghao-intel commented on a diff in the pull request:
https://github.com/apache/spark/pull/5154#discussion_r27089149
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/ScalaUdf.scala
---
@@ -30,372 +30,423 @@ case class
Github user chenghao-intel commented on the pull request:
https://github.com/apache/spark/pull/5154#issuecomment-85829370
@zzcclp
children:Seq[Expression] essentially can be either `List[Expression]` or
`ArraySeq[Expression]`, the later doesn't support the pattern matching.
Github user liancheng commented on the pull request:
https://github.com/apache/spark/pull/5154#issuecomment-85592245
ok to test
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/5154#issuecomment-85592883
[Test build #29093 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/29093/consoleFull)
for PR 5154 at commit
Github user chenghao-intel commented on the pull request:
https://github.com/apache/spark/pull/5154#issuecomment-85354705
OK, probably we can also move the `children.size match {..}` out of the
`eval`.
---
If your project is set up for it, you can reply to this email and have your
Github user chenghao-intel commented on the pull request:
https://github.com/apache/spark/pull/5154#issuecomment-85361005
I mean we can do something like
```scala
val f = children.size match {
case 1 =
val func = function.asInstanceOf[(Any) = Any]
val child0
Github user chenghao-intel commented on a diff in the pull request:
https://github.com/apache/spark/pull/5154#discussion_r27009702
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/ScalaUdf.scala
---
@@ -49,353 +49,378 @@ case class
Github user chenghao-intel commented on a diff in the pull request:
https://github.com/apache/spark/pull/5154#discussion_r27009796
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/ScalaUdf.scala
---
@@ -49,353 +49,378 @@ case class
Github user chenghao-intel commented on the pull request:
https://github.com/apache/spark/pull/5154#issuecomment-85401197
@zzcclp I will run the benchmark in my local machine, will get back soon.
@liancheng , can you trigger the unit test?
---
If your project is set up
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/5154#issuecomment-85624889
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/5154#issuecomment-85624857
[Test build #29093 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/29093/consoleFull)
for PR 5154 at commit
Github user zzcclp commented on the pull request:
https://github.com/apache/spark/pull/5154#issuecomment-85374620
OK, I will modify code and test again.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project
Github user zzcclp commented on the pull request:
https://github.com/apache/spark/pull/5154#issuecomment-85389879
@chenghao-intel , I change code and test it, the result is the same as last
commit , is 5 minutes.
Please help me for checking code.
---
If your project is set up
Github user zzcclp commented on the pull request:
https://github.com/apache/spark/pull/5154#issuecomment-85353974
Before this change, it takes 17 minutes, and now takes 5 minutes, which is
the same as *HiveContext + udf floor* and *non-udf*
---
If your project is set up for it, you
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/5154#issuecomment-85351687
Can one of the admins verify this patch?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your
Github user chenghao-intel commented on the pull request:
https://github.com/apache/spark/pull/5154#issuecomment-85352825
Hmm, have you try what the performance gain by this change? From my
understanding the bottleneck is in the function call
`ScalaReflection.convertToScala`
---
If
49 matches
Mail list logo
