Github user DoingDone9 closed the pull request at:
https://github.com/apache/spark/pull/5538
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/5538#issuecomment-96767138
Can one of the admins verify this patch?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your
Github user DoingDone9 commented on a diff in the pull request:
https://github.com/apache/spark/pull/5538#discussion_r28933363
--- Diff:
sql/hive/v0.13.1/src/main/scala/org/apache/spark/sql/hive/Shim13.scala ---
@@ -218,7 +218,13 @@ private[hive] object HiveShim {
Github user DoingDone9 commented on a diff in the pull request:
https://github.com/apache/spark/pull/5538#discussion_r28933263
--- Diff:
sql/hive/src/main/scala/org/apache/spark/sql/hive/sqlUDFCurrentDB.scala ---
@@ -0,0 +1,42 @@
+/*
+ * Licensed to the Apache Software
Github user DoingDone9 commented on a diff in the pull request:
https://github.com/apache/spark/pull/5538#discussion_r28751859
--- Diff:
sql/hive/src/main/scala/org/apache/spark/sql/hive/sqlUDFCurrentDB.scala ---
@@ -0,0 +1,43 @@
+/*
+ * Licensed to the Apache Software
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/5538#issuecomment-94807716
[Test build #30671 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/30671/consoleFull)
for PR 5538 at commit
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/5538#issuecomment-94807750
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/5538#issuecomment-94798862
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/5538#issuecomment-94798841
[Test build #30668 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/30668/consoleFull)
for PR 5538 at commit
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/5538#issuecomment-94760655
[Test build #30668 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/30668/consoleFull)
for PR 5538 at commit
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/5538#issuecomment-94775676
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/5538#issuecomment-94772903
[Test build #30671 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/30671/consoleFull)
for PR 5538 at commit
Github user marmbrus commented on a diff in the pull request:
https://github.com/apache/spark/pull/5538#discussion_r28830966
--- Diff:
sql/hive/v0.12.0/src/main/scala/org/apache/spark/sql/hive/Shim12.scala ---
@@ -135,7 +135,13 @@ private[hive] object HiveShim {
Github user DoingDone9 commented on a diff in the pull request:
https://github.com/apache/spark/pull/5538#discussion_r28838875
--- Diff:
sql/hive/v0.12.0/src/main/scala/org/apache/spark/sql/hive/Shim12.scala ---
@@ -135,7 +135,13 @@ private[hive] object HiveShim {
Github user chenghao-intel commented on a diff in the pull request:
https://github.com/apache/spark/pull/5538#discussion_r28839484
--- Diff:
sql/hive/src/main/scala/org/apache/spark/sql/hive/sqlUDFCurrentDB.scala ---
@@ -0,0 +1,42 @@
+/*
+ * Licensed to the Apache Software
Github user chenghao-intel commented on a diff in the pull request:
https://github.com/apache/spark/pull/5538#discussion_r28839705
--- Diff:
sql/hive/src/main/scala/org/apache/spark/sql/hive/sqlUDFCurrentDB.scala ---
@@ -0,0 +1,42 @@
+/*
+ * Licensed to the Apache Software
Github user chenghao-intel commented on a diff in the pull request:
https://github.com/apache/spark/pull/5538#discussion_r28841761
--- Diff:
sql/hive/src/main/scala/org/apache/spark/sql/hive/sqlUDFCurrentDB.scala ---
@@ -0,0 +1,42 @@
+/*
+ * Licensed to the Apache Software
Github user chenghao-intel commented on a diff in the pull request:
https://github.com/apache/spark/pull/5538#discussion_r28839390
--- Diff:
sql/hive/v0.12.0/src/main/scala/org/apache/spark/sql/hive/Shim12.scala ---
@@ -135,7 +135,13 @@ private[hive] object HiveShim {
Github user DoingDone9 commented on a diff in the pull request:
https://github.com/apache/spark/pull/5538#discussion_r28840547
--- Diff:
sql/hive/v0.12.0/src/main/scala/org/apache/spark/sql/hive/Shim12.scala ---
@@ -135,7 +135,13 @@ private[hive] object HiveShim {
Github user chenghao-intel commented on a diff in the pull request:
https://github.com/apache/spark/pull/5538#discussion_r28839787
--- Diff:
sql/hive/v0.13.1/src/main/scala/org/apache/spark/sql/hive/Shim13.scala ---
@@ -218,7 +218,13 @@ private[hive] object HiveShim {
Github user DoingDone9 commented on a diff in the pull request:
https://github.com/apache/spark/pull/5538#discussion_r28839749
--- Diff:
sql/hive/v0.12.0/src/main/scala/org/apache/spark/sql/hive/Shim12.scala ---
@@ -135,7 +135,13 @@ private[hive] object HiveShim {
Github user DoingDone9 commented on a diff in the pull request:
https://github.com/apache/spark/pull/5538#discussion_r28840911
--- Diff:
sql/hive/src/main/scala/org/apache/spark/sql/hive/sqlUDFCurrentDB.scala ---
@@ -0,0 +1,42 @@
+/*
+ * Licensed to the Apache Software
Github user chenghao-intel commented on a diff in the pull request:
https://github.com/apache/spark/pull/5538#discussion_r28841751
--- Diff:
sql/hive/src/main/scala/org/apache/spark/sql/hive/sqlUDFCurrentDB.scala ---
@@ -0,0 +1,42 @@
+/*
+ * Licensed to the Apache Software
Github user DoingDone9 commented on a diff in the pull request:
https://github.com/apache/spark/pull/5538#discussion_r28746146
--- Diff:
sql/hive/src/main/scala/org/apache/spark/sql/hive/sqlUDFCurrentDB.scala ---
@@ -0,0 +1,43 @@
+/*
+ * Licensed to the Apache Software
Github user DoingDone9 commented on the pull request:
https://github.com/apache/spark/pull/5538#issuecomment-94635160
@chenghao-intel your idea is good but âselect current_databaseâ is
syntax of hive. and i want to implemente it . And this UDF do not run within
Github user chenghao-intel commented on the pull request:
https://github.com/apache/spark/pull/5538#issuecomment-94640080
@DoingDone9 thanks for the explanation. In thericially, applying the
`Optimizer` rule is optional, and probably not everyone understand how the
constant folding
Github user marmbrus commented on a diff in the pull request:
https://github.com/apache/spark/pull/5538#discussion_r28744774
--- Diff:
sql/hive/src/main/scala/org/apache/spark/sql/hive/sqlUDFCurrentDB.scala ---
@@ -0,0 +1,43 @@
+/*
+ * Licensed to the Apache Software
Github user DoingDone9 commented on the pull request:
https://github.com/apache/spark/pull/5538#issuecomment-94629707
@chenghao-intel i konw this method that can get dbName, but it can only be
used with CLI. It is necessary to get dbName without cli. And i have explained
that this
Github user chenghao-intel commented on a diff in the pull request:
https://github.com/apache/spark/pull/5538#discussion_r28749336
--- Diff:
sql/hive/src/main/scala/org/apache/spark/sql/hive/sqlUDFCurrentDB.scala ---
@@ -0,0 +1,43 @@
+/*
+ * Licensed to the Apache Software
Github user marmbrus commented on the pull request:
https://github.com/apache/spark/pull/5538#issuecomment-94609816
Is there a reason you have not added a comment about the lifecycle of this
UDF?
---
If your project is set up for it, you can reply to this email and have your
reply
Github user chenghao-intel commented on the pull request:
https://github.com/apache/spark/pull/5538#issuecomment-94633954
@DoingDone9 Not like Hive, I don't think Spark SQL supports the `local
mode`, the UDF definitely will run within executor(s), or, we can transform it
into an
Github user DoingDone9 commented on the pull request:
https://github.com/apache/spark/pull/5538#issuecomment-94617955
sorry, i changed the code then the comment disappear, i will add it again.
---
If your project is set up for it, you can reply to this email and have your
reply
Github user DoingDone9 commented on a diff in the pull request:
https://github.com/apache/spark/pull/5538#discussion_r28746101
--- Diff:
sql/hive/src/main/scala/org/apache/spark/sql/hive/sqlUDFCurrentDB.scala ---
@@ -0,0 +1,43 @@
+/*
+ * Licensed to the Apache Software
Github user chenghao-intel commented on the pull request:
https://github.com/apache/spark/pull/5538#issuecomment-94626030
I am not so sure if this is the correct implementation, as we probably not
able to get the correct `SessionState` object in executors.
Hive seems doesn't
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/5538#issuecomment-93899836
[Test build #30459 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/30459/consoleFull)
for PR 5538 at commit
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/5538#issuecomment-93937013
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user marmbrus commented on a diff in the pull request:
https://github.com/apache/spark/pull/5538#discussion_r28488519
--- Diff:
sql/hive/src/main/scala/org/apache/spark/sql/hive/sqlUDFCurrentDB.scala ---
@@ -0,0 +1,43 @@
+/*
+ * Licensed to the Apache Software
Github user DoingDone9 commented on a diff in the pull request:
https://github.com/apache/spark/pull/5538#discussion_r28498708
--- Diff:
sql/hive/src/main/scala/org/apache/spark/sql/hive/sqlUDFCurrentDB.scala ---
@@ -0,0 +1,43 @@
+/*
+ * Licensed to the Apache Software
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/5538#issuecomment-93878012
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
Github user marmbrus commented on the pull request:
https://github.com/apache/spark/pull/5538#issuecomment-93870998
ok to test
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/5538#issuecomment-93878007
[Test build #30446 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/30446/consoleFull)
for PR 5538 at commit
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/5538#issuecomment-93871493
[Test build #30446 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/30446/consoleFull)
for PR 5538 at commit
Github user marmbrus commented on a diff in the pull request:
https://github.com/apache/spark/pull/5538#discussion_r28535110
--- Diff:
sql/hive/src/main/scala/org/apache/spark/sql/hive/sqlUDFCurrentDB.scala ---
@@ -0,0 +1,43 @@
+/*
+ * Licensed to the Apache Software
Github user marmbrus commented on the pull request:
https://github.com/apache/spark/pull/4995#issuecomment-93587797
Ah, I see. Can you add comments that explain that and reopen this PR?
---
If your project is set up for it, you can reply to this email and have your
reply appear on
GitHub user DoingDone9 opened a pull request:
https://github.com/apache/spark/pull/5538
[SPARK-6198][SQL] Support select current_database()
to support select current_database()
```
The method(evaluate) has changed in UDFCurrentDB, it just throws a
exception.But hiveUdfs
Github user DoingDone9 commented on the pull request:
https://github.com/apache/spark/pull/5538#issuecomment-93625012
@marmbrus
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
Github user DoingDone9 commented on the pull request:
https://github.com/apache/spark/pull/4995#issuecomment-93624977
I have opened a new pr https://github.com/apache/spark/pull/5538
---
If your project is set up for it, you can reply to this email and have your
reply appear on
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/5538#issuecomment-93625269
Can one of the admins verify this patch?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your
Github user marmbrus commented on the pull request:
https://github.com/apache/spark/pull/4995#issuecomment-92464041
Here is the command I ran:
```
sc.parallelize(1 to 10).map(_ =
org.apache.hadoop.hive.ql.session.SessionState.get().getCurrentDatabase()).collect()
```
Github user DoingDone9 commented on the pull request:
https://github.com/apache/spark/pull/4995#issuecomment-92599430
I do not agree that. Because this expression is foldable, then it will be
computed in ConstantFolding of Optimizer. So I will get the name of currentDB
after
Github user DoingDone9 commented on the pull request:
https://github.com/apache/spark/pull/4995#issuecomment-92159014
my previous test was successful and i will test it again. Thank you
@marmbrus
---
If your project is set up for it, you can reply to this email and have your
reply
Github user DoingDone9 commented on the pull request:
https://github.com/apache/spark/pull/4995#issuecomment-92189086
Could you tell me how you got this exception? I test with three nodes , and
it works again. Thank you @marmbrus
---
If your project is set up for it, you can reply
Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/4995
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is
Github user marmbrus commented on the pull request:
https://github.com/apache/spark/pull/4995#issuecomment-91955538
@DoingDone9, really? In my tests it null pointers when you try to get the
session state on an executor. It seems like they made this change in Hive 13
on purpose
Github user rxin commented on a diff in the pull request:
https://github.com/apache/spark/pull/4995#discussion_r27191997
--- Diff:
sql/hive/src/main/scala/org/apache/spark/sql/hive/sqlUDFCurrentDB.scala ---
@@ -0,0 +1,18 @@
+package org.apache.spark.sql.hive
+
+import
Github user DoingDone9 commented on the pull request:
https://github.com/apache/spark/pull/4995#issuecomment-86384179
yes, it works. I have tested it in the distributed mode with two nodes.
@rxin
---
If your project is set up for it, you can reply to this email and have your
reply
Github user DoingDone9 commented on the pull request:
https://github.com/apache/spark/pull/4995#issuecomment-86329564
anyone will test it ? @marmbrus @srowen
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your
Github user DoingDone9 closed the pull request at:
https://github.com/apache/spark/pull/4926
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is
Github user DoingDone9 commented on the pull request:
https://github.com/apache/spark/pull/4926#issuecomment-78447224
I have opened a new pr for this .I create a new UDF and register it
instead of intercepting code.
https://github.com/apache/spark/pull/4995 @chenghao-intel
GitHub user DoingDone9 opened a pull request:
https://github.com/apache/spark/pull/4995
[SPARK-6198][SQL] Support select current_database()
The method(evaluate) has changed in UDFCurrentDB, it just throws a
exception.But hiveUdfs call this method and failed.
@Override
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/4995#issuecomment-78447220
Can one of the admins verify this patch?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your
Github user DoingDone9 commented on the pull request:
https://github.com/apache/spark/pull/4926#issuecomment-78182986
could you test it @marmbrus
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not
Github user DoingDone9 commented on a diff in the pull request:
https://github.com/apache/spark/pull/4926#discussion_r26103224
--- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/hiveUdfs.scala
---
@@ -179,7 +179,12 @@ private[hive] case class HiveGenericUdf(funcWrapper:
Github user chenghao-intel commented on a diff in the pull request:
https://github.com/apache/spark/pull/4926#discussion_r26041674
--- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/hiveUdfs.scala
---
@@ -179,7 +179,12 @@ private[hive] case class
Github user chenghao-intel commented on the pull request:
https://github.com/apache/spark/pull/4926#issuecomment-77866817
`SELECT 1` Seems doesn't work in Hive 0.12, probably introduced since Hive
0.13. See:https://issues.apache.org/jira/browse/HIVE-4144
---
If your project is set
Github user DoingDone9 commented on the pull request:
https://github.com/apache/spark/pull/4926#issuecomment-7793
yes, my version is 0.13.1. @chenghao-intel
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your
Github user DoingDone9 commented on the pull request:
https://github.com/apache/spark/pull/4926#issuecomment-77808256
HiveQL supports the SELECT clause without FROM. I have test it for several
times. And you can try run SQL like select 1, it works. @chenghao-intel
---
If your
Github user DoingDone9 commented on a diff in the pull request:
https://github.com/apache/spark/pull/4926#discussion_r26017842
--- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/hiveUdfs.scala
---
@@ -179,7 +179,12 @@ private[hive] case class HiveGenericUdf(funcWrapper:
Github user chenghao-intel commented on a diff in the pull request:
https://github.com/apache/spark/pull/4926#discussion_r26013109
--- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/hiveUdfs.scala
---
@@ -179,7 +179,12 @@ private[hive] case class
Github user chenghao-intel commented on a diff in the pull request:
https://github.com/apache/spark/pull/4926#discussion_r26013144
--- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/UDFSuite.scala
---
@@ -32,5 +32,6 @@ class UDFSuite extends QueryTest {
Github user marmbrus commented on the pull request:
https://github.com/apache/spark/pull/4926#issuecomment-77654187
please add a test
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this
Github user DoingDone9 commented on the pull request:
https://github.com/apache/spark/pull/4926#issuecomment-77678092
I have add a test, pleat test it. @marmbrus
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your
GitHub user DoingDone9 opened a pull request:
https://github.com/apache/spark/pull/4926
[SPARK-6198][SQL] Support select current_database()
The method(evaluate) has changed in UDFCurrentDB, it just throws a
exception.But hiveUdfs call this method and failed.
@Override
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/4926#issuecomment-77503622
Can one of the admins verify this patch?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your
74 matches
Mail list logo
