Github user moustaki commented on the pull request:
https://github.com/apache/spark/pull/5059#issuecomment-141522701
Perfect, thanks!
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this
Github user marmbrus commented on the pull request:
https://github.com/apache/spark/pull/5059#issuecomment-141521839
We turned this off by default in Spark 1.5 as it was causing problems
similar to what you saw.
---
If your project is set up for it, you can reply to this email and
Github user moustaki commented on the pull request:
https://github.com/apache/spark/pull/5059#issuecomment-141298174
@marmbrus On your earlier documentation point, I was thinking that there
might be genuine cases where the actual location of a partition might not be
matching the
Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/5059
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/5059#issuecomment-90336832
[Test build #29776 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/29776/consoleFull)
for PR 5059 at commit
Github user lazyman500 commented on the pull request:
https://github.com/apache/spark/pull/5059#issuecomment-90372569
Thanks for suggestion ! @chenghao-intel @marmbrus
I had optimized them.
---
If your project is set up for it, you can reply to this email and have your
reply
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/5059#issuecomment-90357377
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/5059#issuecomment-90357369
[Test build #29776 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/29776/consoleFull)
for PR 5059 at commit
Github user chenghao-intel commented on a diff in the pull request:
https://github.com/apache/spark/pull/5059#discussion_r27782363
--- Diff:
sql/hive/src/main/scala/org/apache/spark/sql/hive/TableReader.scala ---
@@ -142,7 +142,46 @@ class HadoopTableReader(
Github user chenghao-intel commented on a diff in the pull request:
https://github.com/apache/spark/pull/5059#discussion_r27782294
--- Diff:
sql/hive/src/main/scala/org/apache/spark/sql/hive/TableReader.scala ---
@@ -142,7 +142,46 @@ class HadoopTableReader(
Github user marmbrus commented on a diff in the pull request:
https://github.com/apache/spark/pull/5059#discussion_r27710294
--- Diff:
sql/hive/src/test/scala/org/apache/spark/sql/hive/QueryPartitionSuite.scala ---
@@ -0,0 +1,64 @@
+/*
+ * Licensed to the Apache Software
Github user marmbrus commented on a diff in the pull request:
https://github.com/apache/spark/pull/5059#discussion_r27710285
--- Diff:
sql/hive/src/main/scala/org/apache/spark/sql/hive/TableReader.scala ---
@@ -142,7 +142,46 @@ class HadoopTableReader(
Github user marmbrus commented on the pull request:
https://github.com/apache/spark/pull/5059#issuecomment-89100086
Thanks for working on this! Only two minor comments, then I think we can
probably commit this. We can probably leave the flag undocumented for now and
only add it to
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/5059#issuecomment-82887863
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/5059#issuecomment-82887857
[Test build #28791 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/28791/consoleFull)
for PR 5059 at commit
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/5059#issuecomment-82887730
[Test build #28791 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/28791/consoleFull)
for PR 5059 at commit
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/5059#issuecomment-82931839
[Test build #28796 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/28796/consoleFull)
for PR 5059 at commit
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/5059#issuecomment-82924647
[Test build #28795 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/28795/consoleFull)
for PR 5059 at commit
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/5059#issuecomment-82924201
[Test build #28795 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/28795/consoleFull)
for PR 5059 at commit
Github user lazyman500 commented on the pull request:
https://github.com/apache/spark/pull/5059#issuecomment-82946111
Thanks for your review. I feel so sorry to make so much style mistakes.
I have added config flag. But how does user know this config? Do I need add
some document
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/5059#issuecomment-82965974
[Test build #28796 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/28796/consoleFull)
for PR 5059 at commit
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/5059#issuecomment-82965990
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/5059#issuecomment-82924654
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
Github user marmbrus commented on the pull request:
https://github.com/apache/spark/pull/5059#issuecomment-82725022
Lots of style comments. Please checkout:
https://cwiki.apache.org/confluence/display/SPARK/Spark+Code+Style+Guide
Also, while this seems better it does seem
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/5059#issuecomment-82608365
[Test build #28738 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/28738/consoleFull)
for PR 5059 at commit
Github user marmbrus commented on the pull request:
https://github.com/apache/spark/pull/5059#issuecomment-82607319
ok to test
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
Github user marmbrus commented on a diff in the pull request:
https://github.com/apache/spark/pull/5059#discussion_r26635569
--- Diff:
sql/hive/src/main/scala/org/apache/spark/sql/hive/TableReader.scala ---
@@ -142,7 +142,39 @@ class HadoopTableReader(
Github user marmbrus commented on a diff in the pull request:
https://github.com/apache/spark/pull/5059#discussion_r26635579
--- Diff:
sql/hive/src/main/scala/org/apache/spark/sql/hive/TableReader.scala ---
@@ -142,7 +142,39 @@ class HadoopTableReader(
Github user marmbrus commented on a diff in the pull request:
https://github.com/apache/spark/pull/5059#discussion_r26635598
--- Diff:
sql/hive/src/main/scala/org/apache/spark/sql/hive/TableReader.scala ---
@@ -142,7 +142,39 @@ class HadoopTableReader(
Github user marmbrus commented on a diff in the pull request:
https://github.com/apache/spark/pull/5059#discussion_r26635618
--- Diff:
sql/hive/src/main/scala/org/apache/spark/sql/hive/TableReader.scala ---
@@ -142,7 +142,39 @@ class HadoopTableReader(
Github user marmbrus commented on a diff in the pull request:
https://github.com/apache/spark/pull/5059#discussion_r26635610
--- Diff:
sql/hive/src/main/scala/org/apache/spark/sql/hive/TableReader.scala ---
@@ -142,7 +142,39 @@ class HadoopTableReader(
Github user marmbrus commented on a diff in the pull request:
https://github.com/apache/spark/pull/5059#discussion_r26635635
--- Diff:
sql/hive/src/main/scala/org/apache/spark/sql/hive/TableReader.scala ---
@@ -142,7 +142,39 @@ class HadoopTableReader(
Github user marmbrus commented on a diff in the pull request:
https://github.com/apache/spark/pull/5059#discussion_r26635640
--- Diff:
sql/hive/src/main/scala/org/apache/spark/sql/hive/TableReader.scala ---
@@ -142,7 +142,39 @@ class HadoopTableReader(
Github user marmbrus commented on a diff in the pull request:
https://github.com/apache/spark/pull/5059#discussion_r26635634
--- Diff:
sql/hive/src/main/scala/org/apache/spark/sql/hive/TableReader.scala ---
@@ -142,7 +142,39 @@ class HadoopTableReader(
Github user marmbrus commented on a diff in the pull request:
https://github.com/apache/spark/pull/5059#discussion_r26635531
--- Diff:
sql/hive/src/main/scala/org/apache/spark/sql/hive/TableReader.scala ---
@@ -142,7 +142,39 @@ class HadoopTableReader(
Github user marmbrus commented on a diff in the pull request:
https://github.com/apache/spark/pull/5059#discussion_r26635540
--- Diff:
sql/hive/src/main/scala/org/apache/spark/sql/hive/TableReader.scala ---
@@ -142,7 +142,39 @@ class HadoopTableReader(
Github user marmbrus commented on a diff in the pull request:
https://github.com/apache/spark/pull/5059#discussion_r26635550
--- Diff:
sql/hive/src/main/scala/org/apache/spark/sql/hive/TableReader.scala ---
@@ -142,7 +142,39 @@ class HadoopTableReader(
Github user marmbrus commented on a diff in the pull request:
https://github.com/apache/spark/pull/5059#discussion_r26635933
--- Diff:
sql/hive/src/main/scala/org/apache/spark/sql/hive/TableReader.scala ---
@@ -142,7 +142,39 @@ class HadoopTableReader(
Github user marmbrus commented on a diff in the pull request:
https://github.com/apache/spark/pull/5059#discussion_r26635919
--- Diff:
sql/hive/src/main/scala/org/apache/spark/sql/hive/TableReader.scala ---
@@ -142,7 +142,39 @@ class HadoopTableReader(
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/5059#issuecomment-82632721
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/5059#issuecomment-82632708
[Test build #28738 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/28738/consoleFull)
for PR 5059 at commit
GitHub user lazyman500 opened a pull request:
https://github.com/apache/spark/pull/5059
[Spark-5068][SQL]Fix bug query data when path doesn't exist for HiveContext
This RP follow up PR #3907 #3891 #4356.
According to @marmbrus @liancheng 's comment,I try to use fs.globStatus
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/5059#issuecomment-82072538
Can one of the admins verify this patch?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your
Github user jeanlyn commented on the pull request:
https://github.com/apache/spark/pull/3907#issuecomment-75260761
OK.I close this one
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this
Github user jeanlyn commented on the pull request:
https://github.com/apache/spark/pull/3891#issuecomment-75260856
OK.I close this one
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this
Github user jeanlyn closed the pull request at:
https://github.com/apache/spark/pull/3891
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is
Github user jeanlyn closed the pull request at:
https://github.com/apache/spark/pull/3907
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is
Github user srowen commented on the pull request:
https://github.com/apache/spark/pull/3907#issuecomment-74755965
Is this superseded by https://github.com/apache/spark/pull/4356 ? if so can
this be closed?
---
If your project is set up for it, you can reply to this email and have
Github user srowen commented on the pull request:
https://github.com/apache/spark/pull/3891#issuecomment-74755930
Is this superseded by https://github.com/apache/spark/pull/3907 or
https://github.com/apache/spark/pull/4356 ? if so can this be closed?
---
If your project is set up
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/3891#issuecomment-72453373
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/3891#issuecomment-72453364
[Test build #26511 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/26511/consoleFull)
for PR 3891 at commit
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/3891#issuecomment-72456006
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/3891#issuecomment-72456000
[Test build #26513 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/26513/consoleFull)
for PR 3891 at commit
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/3891#issuecomment-72446914
[Test build #26513 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/26513/consoleFull)
for PR 3891 at commit
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/3891#issuecomment-72445310
[Test build #26511 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/26511/consoleFull)
for PR 3891 at commit
Github user jeanlyn commented on a diff in the pull request:
https://github.com/apache/spark/pull/3907#discussion_r22655119
--- Diff: core/src/main/scala/org/apache/spark/rdd/HadoopRDD.scala ---
@@ -198,12 +190,19 @@ class HadoopRDD[K, V](
if
Github user jeanlyn commented on a diff in the pull request:
https://github.com/apache/spark/pull/3907#discussion_r22633042
--- Diff: core/src/main/scala/org/apache/spark/rdd/HadoopRDD.scala ---
@@ -198,12 +190,19 @@ class HadoopRDD[K, V](
if
Github user jeanlyn commented on a diff in the pull request:
https://github.com/apache/spark/pull/3907#discussion_r22634166
--- Diff: core/src/main/scala/org/apache/spark/rdd/HadoopRDD.scala ---
@@ -198,12 +190,19 @@ class HadoopRDD[K, V](
if
Github user chenghao-intel commented on a diff in the pull request:
https://github.com/apache/spark/pull/3907#discussion_r22635547
--- Diff: core/src/main/scala/org/apache/spark/rdd/HadoopRDD.scala ---
@@ -198,12 +190,19 @@ class HadoopRDD[K, V](
if
Github user chenghao-intel commented on a diff in the pull request:
https://github.com/apache/spark/pull/3907#discussion_r22635649
--- Diff: core/src/main/scala/org/apache/spark/rdd/HadoopRDD.scala ---
@@ -198,12 +190,19 @@ class HadoopRDD[K, V](
if
Github user scwf commented on a diff in the pull request:
https://github.com/apache/spark/pull/3907#discussion_r22635771
--- Diff: core/src/main/scala/org/apache/spark/rdd/HadoopRDD.scala ---
@@ -198,12 +190,19 @@ class HadoopRDD[K, V](
if
Github user chenghao-intel commented on a diff in the pull request:
https://github.com/apache/spark/pull/3907#discussion_r22633589
--- Diff: core/src/main/scala/org/apache/spark/rdd/HadoopRDD.scala ---
@@ -198,12 +190,19 @@ class HadoopRDD[K, V](
if
Github user marmbrus commented on the pull request:
https://github.com/apache/spark/pull/3907#issuecomment-69111291
ok to test
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/3907#issuecomment-69111516
[Test build #25181 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/25181/consoleFull)
for PR 3907 at commit
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/3907#issuecomment-69120687
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/3907#issuecomment-69120679
[Test build #25181 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/25181/consoleFull)
for PR 3907 at commit
Github user chenghao-intel commented on a diff in the pull request:
https://github.com/apache/spark/pull/3907#discussion_r22630492
--- Diff: core/src/main/scala/org/apache/spark/rdd/HadoopRDD.scala ---
@@ -198,12 +190,19 @@ class HadoopRDD[K, V](
if
Github user srowen commented on a diff in the pull request:
https://github.com/apache/spark/pull/3907#discussion_r22515156
--- Diff: core/src/main/scala/org/apache/spark/rdd/HadoopRDD.scala ---
@@ -26,15 +26,7 @@ import scala.reflect.ClassTag
import
Github user srowen commented on a diff in the pull request:
https://github.com/apache/spark/pull/3907#discussion_r22515175
--- Diff: core/src/main/scala/org/apache/spark/rdd/HadoopRDD.scala ---
@@ -198,12 +190,19 @@ class HadoopRDD[K, V](
if
Github user jeanlyn commented on the pull request:
https://github.com/apache/spark/pull/3907#issuecomment-68839176
Hi @marmbrus. Any suggestions?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not
GitHub user jeanlyn reopened a pull request:
https://github.com/apache/spark/pull/3907
[spark 5068][SQL]fix bug query data when path doesn't exists
the issue is descript on [SPARK-5068]
(https://issues.apache.org/jira/browse/SPARK-5068) and this PR is fix the same
problem as
Github user jeanlyn closed the pull request at:
https://github.com/apache/spark/pull/3907
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is
Github user jeanlyn commented on a diff in the pull request:
https://github.com/apache/spark/pull/3907#discussion_r22518647
--- Diff: core/src/main/scala/org/apache/spark/rdd/HadoopRDD.scala ---
@@ -198,12 +190,19 @@ class HadoopRDD[K, V](
if
Github user jeanlyn commented on a diff in the pull request:
https://github.com/apache/spark/pull/3907#discussion_r22518690
--- Diff: core/src/main/scala/org/apache/spark/rdd/HadoopRDD.scala ---
@@ -26,15 +26,7 @@ import scala.reflect.ClassTag
import
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/3891#issuecomment-68833142
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
Github user marmbrus commented on the pull request:
https://github.com/apache/spark/pull/3891#issuecomment-68832961
ok to test
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/3891#issuecomment-68833139
[Test build #25090 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/25090/consoleFull)
for PR 3891 at commit
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/3891#issuecomment-68833133
[Test build #25090 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/25090/consoleFull)
for PR 3891 at commit
Github user marmbrus commented on the pull request:
https://github.com/apache/spark/pull/3891#issuecomment-68803151
What is the rational behind this change? It seems like the table is
corrupted and you should know about it. Does hive work in this case?
---
If your project is set
Github user jeanlyn commented on the pull request:
https://github.com/apache/spark/pull/3891#issuecomment-68813094
Yes,hive is work in this situation.I found this issue from our production
environment when i try to use spark-sql to test some sql which run in hive
original. I am not
Github user marmbrus commented on a diff in the pull request:
https://github.com/apache/spark/pull/3891#discussion_r22498932
--- Diff:
sql/hive/src/main/scala/org/apache/spark/sql/hive/TableReader.scala ---
@@ -141,7 +141,11 @@ class HadoopTableReader(
Github user marmbrus commented on a diff in the pull request:
https://github.com/apache/spark/pull/3891#discussion_r22498912
--- Diff:
sql/hive/src/main/scala/org/apache/spark/sql/hive/TableReader.scala ---
@@ -141,7 +141,11 @@ class HadoopTableReader(
Github user marmbrus commented on the pull request:
https://github.com/apache/spark/pull/3891#issuecomment-68813712
Okay, that is reasonable and we should probably support this. So then the
question is can we do this check on the executor in parallel (or just catch the
exception if
Github user jeanlyn commented on the pull request:
https://github.com/apache/spark/pull/3891#issuecomment-68814331
Thanks for suggestion! I would optimize this and commit later.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as
Github user jeanlyn commented on the pull request:
https://github.com/apache/spark/pull/3891#issuecomment-68676770
hi, @marmbrus ,can you please take a look and give some suggestions?thx.
---
If your project is set up for it, you can reply to this email and have your
reply appear on
GitHub user jeanlyn opened a pull request:
https://github.com/apache/spark/pull/3891
[SPARK-5068][SQL]fix bug query data when path doesn't exists
the issue is descript on [SPARK-5068]
(https://issues.apache.org/jira/browse/SPARK-5068)
the purpose of this pull request is to
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/3891#issuecomment-68639582
Can one of the admins verify this patch?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your
87 matches
Mail list logo
