Github user chenghao-intel commented on a diff in the pull request:
https://github.com/apache/spark/pull/4289#discussion_r24290001
--- Diff:
sql/hive/src/main/scala/org/apache/spark/sql/hive/TableReader.scala ---
@@ -315,9 +335,17 @@ private[hive] object HadoopTableReader extends
Github user chenghao-intel commented on the pull request:
https://github.com/apache/spark/pull/4289#issuecomment-73368110
Thank you @jeanlyn for the explanation offline, I've added some comments
also. And I also have some idea to simplify the code, just take it if you feel
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/4289#issuecomment-73377819
[Test build #27008 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/27008/consoleFull)
for PR 4289 at commit
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/4289#issuecomment-73377821
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
Github user chenghao-intel commented on a diff in the pull request:
https://github.com/apache/spark/pull/4289#discussion_r24290012
--- Diff:
sql/hive/v0.12.0/src/main/scala/org/apache/spark/sql/hive/Shim12.scala ---
@@ -242,6 +242,14 @@ private[hive] object HiveShim {
}
Github user chenghao-intel commented on a diff in the pull request:
https://github.com/apache/spark/pull/4289#discussion_r24290013
--- Diff:
sql/hive/v0.13.1/src/main/scala/org/apache/spark/sql/hive/Shim13.scala ---
@@ -397,7 +398,12 @@ private[hive] object HiveShim {
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/4289#issuecomment-73380175
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/4289#issuecomment-73380174
[Test build #27010 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/27010/consoleFull)
for PR 4289 at commit
Github user chenghao-intel commented on a diff in the pull request:
https://github.com/apache/spark/pull/4289#discussion_r24290009
--- Diff:
sql/hive/v0.12.0/src/main/scala/org/apache/spark/sql/hive/Shim12.scala ---
@@ -242,6 +242,14 @@ private[hive] object HiveShim {
}
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/4289#issuecomment-73377784
[Test build #27008 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/27008/consoleFull)
for PR 4289 at commit
Github user chenghao-intel commented on a diff in the pull request:
https://github.com/apache/spark/pull/4289#discussion_r24290017
--- Diff:
sql/hive/src/main/scala/org/apache/spark/sql/hive/TableReader.scala ---
@@ -264,15 +268,31 @@ private[hive] object HadoopTableReader extends
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/4289#issuecomment-73378452
[Test build #27010 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/27010/consoleFull)
for PR 4289 at commit
Github user jeanlyn commented on the pull request:
https://github.com/apache/spark/pull/4289#issuecomment-73459718
Thanks @chenghao-intel for review and suggestions!I take some of your
advises to simplify the code.
---
If your project is set up for it, you can reply to this email
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/4289#issuecomment-73453424
[Test build #27066 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/27066/consoleFull)
for PR 4289 at commit
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/4289#issuecomment-73453428
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/4289#issuecomment-73449265
[Test build #27066 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/27066/consoleFull)
for PR 4289 at commit
Github user chenghao-intel commented on a diff in the pull request:
https://github.com/apache/spark/pull/4289#discussion_r24222130
--- Diff:
sql/hive/src/main/scala/org/apache/spark/sql/hive/TableReader.scala ---
@@ -315,9 +335,17 @@ private[hive] object HadoopTableReader extends
Github user chenghao-intel commented on a diff in the pull request:
https://github.com/apache/spark/pull/4289#discussion_r24222165
--- Diff:
sql/hive/src/main/scala/org/apache/spark/sql/hive/TableReader.scala ---
@@ -315,9 +335,17 @@ private[hive] object HadoopTableReader extends
Github user chenghao-intel commented on a diff in the pull request:
https://github.com/apache/spark/pull/4289#discussion_r24221959
--- Diff:
sql/hive/src/main/scala/org/apache/spark/sql/hive/TableReader.scala ---
@@ -264,15 +268,31 @@ private[hive] object HadoopTableReader extends
Github user jeanlyn commented on the pull request:
https://github.com/apache/spark/pull/4289#issuecomment-73176093
hi,@chenghao-intel @marmbrus any suggestions?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/4289#issuecomment-72988937
[Test build #26820 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/26820/consoleFull)
for PR 4289 at commit
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/4289#issuecomment-72993874
[Test build #26820 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/26820/consoleFull)
for PR 4289 at commit
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/4289#issuecomment-72993877
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user jeanlyn commented on a diff in the pull request:
https://github.com/apache/spark/pull/4289#discussion_r24138028
--- Diff:
sql/hive/src/main/scala/org/apache/spark/sql/hive/TableReader.scala ---
@@ -315,9 +335,23 @@ private[hive] object HadoopTableReader extends
Github user jeanlyn commented on a diff in the pull request:
https://github.com/apache/spark/pull/4289#discussion_r24139046
--- Diff:
sql/hive/src/main/scala/org/apache/spark/sql/hive/TableReader.scala ---
@@ -264,15 +268,31 @@ private[hive] object HadoopTableReader extends
Github user chenghao-intel commented on a diff in the pull request:
https://github.com/apache/spark/pull/4289#discussion_r24139524
--- Diff:
sql/hive/src/main/scala/org/apache/spark/sql/hive/TableReader.scala ---
@@ -264,15 +268,31 @@ private[hive] object HadoopTableReader extends
Github user chenghao-intel commented on the pull request:
https://github.com/apache/spark/pull/4289#issuecomment-7280
In general I think the change looks reasonable to me, and we'd better use
the Hive `ObjectConverter` directly, and some of the code can be cleaner.
---
If your
Github user chenghao-intel commented on a diff in the pull request:
https://github.com/apache/spark/pull/4289#discussion_r24057600
--- Diff:
sql/hive/src/main/scala/org/apache/spark/sql/hive/TableReader.scala ---
@@ -315,9 +335,23 @@ private[hive] object HadoopTableReader extends
Github user chenghao-intel commented on a diff in the pull request:
https://github.com/apache/spark/pull/4289#discussion_r24057968
--- Diff:
sql/hive/v0.12.0/src/main/scala/org/apache/spark/sql/hive/Shim12.scala ---
@@ -242,6 +242,11 @@ private[hive] object HiveShim {
}
Github user chenghao-intel commented on a diff in the pull request:
https://github.com/apache/spark/pull/4289#discussion_r24058262
--- Diff:
sql/hive/v0.12.0/src/main/scala/org/apache/spark/sql/hive/Shim12.scala ---
@@ -242,6 +242,11 @@ private[hive] object HiveShim {
}
Github user chenghao-intel commented on the pull request:
https://github.com/apache/spark/pull/4289#issuecomment-72671869
Sorry for the late reply @jeanlyn !
I think it's a bug of Hive DDL, which probably was resolved in Hive 0.14 /
0.15, and I am not sure if we really want to fix
Github user chenghao-intel commented on a diff in the pull request:
https://github.com/apache/spark/pull/4289#discussion_r24010127
--- Diff:
sql/hive/src/test/scala/org/apache/spark/sql/hive/InsertIntoHiveTableSuite.scala
---
@@ -172,4 +172,19 @@ class InsertIntoHiveTableSuite
Github user jeanlyn commented on a diff in the pull request:
https://github.com/apache/spark/pull/4289#discussion_r24013566
--- Diff:
sql/hive/src/test/scala/org/apache/spark/sql/hive/InsertIntoHiveTableSuite.scala
---
@@ -172,4 +172,19 @@ class InsertIntoHiveTableSuite extends
Github user jeanlyn commented on the pull request:
https://github.com/apache/spark/pull/4289#issuecomment-72680801
Thanks @chenghao-intel for review and suggestions! We want to replace some
hive sql to spark-sql in our production environment,so I use some sql in our
production
Github user chenghao-intel commented on the pull request:
https://github.com/apache/spark/pull/4289#issuecomment-72773457
Oh, @jeanlyn, I've also tested that in Hive 0.13, seems it works.
Hive will do the data type converting if it realizes the partition schema
is not the same as
Github user chenghao-intel commented on a diff in the pull request:
https://github.com/apache/spark/pull/4289#discussion_r24057467
--- Diff:
sql/hive/src/main/scala/org/apache/spark/sql/hive/TableReader.scala ---
@@ -264,15 +268,31 @@ private[hive] object HadoopTableReader extends
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/4289#issuecomment-72587496
[Test build #26592 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/26592/consoleFull)
for PR 4289 at commit
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/4289#issuecomment-72587499
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/4289#issuecomment-72596283
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/4289#issuecomment-72596740
[Test build #26611 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/26611/consoleFull)
for PR 4289 at commit
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/4289#issuecomment-72587243
[Test build #26592 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/26592/consoleFull)
for PR 4289 at commit
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/4289#issuecomment-72591570
[Test build #26602 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/26602/consoleFull)
for PR 4289 at commit
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/4289#issuecomment-72604052
[Test build #26611 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/26611/consoleFull)
for PR 4289 at commit
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/4289#issuecomment-72604056
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/4289#issuecomment-72596278
[Test build #26602 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/26602/consoleFull)
for PR 4289 at commit
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/4289#issuecomment-72400702
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/4289#issuecomment-72400699
[Test build #26484 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/26484/consoleFull)
for PR 4289 at commit
Github user marmbrus commented on the pull request:
https://github.com/apache/spark/pull/4289#issuecomment-72397218
ok to test
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
Github user marmbrus commented on the pull request:
https://github.com/apache/spark/pull/4289#issuecomment-72397279
/cc @chenghao-intel
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/4289#issuecomment-72397455
[Test build #26479 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/26479/consoleFull)
for PR 4289 at commit
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/4289#issuecomment-72397543
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/4289#issuecomment-72397542
[Test build #26479 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/26479/consoleFull)
for PR 4289 at commit
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/4289#issuecomment-72400374
[Test build #26484 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/26484/consoleFull)
for PR 4289 at commit
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/4289#issuecomment-72398464
[Test build #26481 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/26481/consoleFull)
for PR 4289 at commit
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/4289#issuecomment-72398796
[Test build #26481 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/26481/consoleFull)
for PR 4289 at commit
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/4289#issuecomment-72398798
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/4289#issuecomment-72406797
[Test build #26489 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/26489/consoleFull)
for PR 4289 at commit
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/4289#issuecomment-72410648
[Test build #26489 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/26489/consoleFull)
for PR 4289 at commit
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/4289#issuecomment-72410655
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
59 matches
Mail list logo
