Github user liancheng commented on a diff in the pull request:
https://github.com/apache/spark/pull/12781#discussion_r61845886
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/command/tables.scala ---
@@ -389,3 +392,238 @@ case class ShowTablePropertiesCommand
GitHub user liancheng opened a pull request:
https://github.com/apache/spark/pull/12844
[SPARK-14127][SQL] Native "DESC [EXTENDED | FORMATTED] " DDL command
## What changes were proposed in this pull request?
This PR implements native `DESC [EXTENDED | FORMAT
Github user liancheng commented on a diff in the pull request:
https://github.com/apache/spark/pull/12781#discussion_r61619036
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/command/commands.scala
---
@@ -117,101 +112,3 @@ case class ExplainCommand
GitHub user liancheng opened a pull request:
https://github.com/apache/spark/pull/12781
[SPARK-14346] SHOW CREATE TABLE for data source tables
## What changes were proposed in this pull request?
This PR adds native `SHOW CREATE TABLE` DDL command for data source tables
Github user liancheng commented on the pull request:
https://github.com/apache/spark/pull/12759#issuecomment-215628056
retest this please
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
GitHub user liancheng opened a pull request:
https://github.com/apache/spark/pull/12759
[SPARK-14981][SQL] Throws exception if DESC is specified for sorting columns
## What changes were proposed in this pull request?
Currently Spark SQL doesn't support sorting columns
Github user liancheng commented on the pull request:
https://github.com/apache/spark/pull/12734#issuecomment-215310529
@jodersky Oh sorry, pasted the JIRA ticket summary to the PR title but
forgot to add the tags. Updated!
---
If your project is set up for it, you can reply
Github user liancheng commented on a diff in the pull request:
https://github.com/apache/spark/pull/12081#discussion_r61283810
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/catalog/SessionCatalog.scala
---
@@ -693,7 +697,6 @@ class SessionCatalog
Github user liancheng commented on a diff in the pull request:
https://github.com/apache/spark/pull/12081#discussion_r61282206
--- Diff:
sql/core/src/test/scala/org/apache/spark/sql/execution/command/DDLSuite.scala
---
@@ -118,7 +148,7 @@ class DDLSuite extends QueryTest
Github user liancheng commented on a diff in the pull request:
https://github.com/apache/spark/pull/12081#discussion_r61278961
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/catalog/SessionCatalog.scala
---
@@ -693,7 +697,6 @@ class SessionCatalog
Github user liancheng commented on the pull request:
https://github.com/apache/spark/pull/12460#issuecomment-215115938
@dilipbiswal One purpose of re-implementing all DDL as native Spark SQL
command is to minimize dependency to Hive so that we can move Hive into a
separate data
GitHub user liancheng opened a pull request:
https://github.com/apache/spark/pull/12734
Add PARTITION BY and BUCKET BY clause for "CREATE TABLE ... USING ..."
syntax
## What changes were proposed in this pull request?
Currently, we can only create persisted p
Github user liancheng closed the pull request at:
https://github.com/apache/spark/pull/12703
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature
Github user liancheng commented on the pull request:
https://github.com/apache/spark/pull/12703#issuecomment-214938531
@dilipbiswal also updated #1 and passed Jenkins. So I merged it.
Closing this one.
---
If your project is set up for it, you can reply to this email and have
Github user liancheng commented on the pull request:
https://github.com/apache/spark/pull/1#issuecomment-214938170
@dilipbiswal Great! LGTM now, I'm merging this one to master. Thanks!
---
If your project is set up for it, you can reply to this email and have your
reply appear
Github user liancheng commented on a diff in the pull request:
https://github.com/apache/spark/pull/12703#discussion_r61185701
--- Diff:
sql/hive/src/test/scala/org/apache/spark/sql/hive/execution/HiveCommandSuite.scala
---
@@ -247,4 +259,105 @@ class HiveCommandSuite extends
Github user liancheng commented on the pull request:
https://github.com/apache/spark/pull/1#issuecomment-214803024
Hi @dilipbiswal, we did a bunch of major refactoring on the master branch,
and it's pretty close to 2.0 code freeze, so I took it over based on your
version
GitHub user liancheng opened a pull request:
https://github.com/apache/spark/pull/12703
[SPARK-14445][SQL] Supports native execution of SHOW COLUMNS and SHOW
PARTITIONS
(This PR is mostly a rebased version of PR #1 by @dilipbiswal with all
pending PR review comments addressed
Github user liancheng commented on a diff in the pull request:
https://github.com/apache/spark/pull/1#discussion_r61011875
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/PartitioningUtils.scala
---
@@ -424,7 +424,7 @@ private[sql] object
Github user liancheng commented on the pull request:
https://github.com/apache/spark/pull/12579#issuecomment-214419087
test this please
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
Github user liancheng commented on a diff in the pull request:
https://github.com/apache/spark/pull/1#discussion_r60937575
--- Diff:
sql/hive/src/test/scala/org/apache/spark/sql/hive/execution/HiveCommandSuite.scala
---
@@ -36,12 +38,22 @@ class HiveCommandSuite extends
Github user liancheng commented on a diff in the pull request:
https://github.com/apache/spark/pull/1#discussion_r60937498
--- Diff:
sql/hive/src/test/scala/org/apache/spark/sql/hive/execution/HiveCommandSuite.scala
---
@@ -17,11 +17,13 @@
package
Github user liancheng commented on a diff in the pull request:
https://github.com/apache/spark/pull/1#discussion_r60934317
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/PartitioningUtils.scala
---
@@ -424,7 +424,7 @@ private[sql] object
Github user liancheng commented on a diff in the pull request:
https://github.com/apache/spark/pull/1#discussion_r60934120
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/command/commands.scala
---
@@ -423,6 +426,102 @@ case class ShowTablePropertiesCommand
Github user liancheng commented on a diff in the pull request:
https://github.com/apache/spark/pull/1#discussion_r60931135
--- Diff:
sql/hive/src/test/scala/org/apache/spark/sql/hive/execution/HiveCommandSuite.scala
---
@@ -122,4 +134,105 @@ class HiveCommandSuite extends
Github user liancheng commented on a diff in the pull request:
https://github.com/apache/spark/pull/1#discussion_r60930580
--- Diff:
sql/hive/src/test/scala/org/apache/spark/sql/hive/execution/HiveCommandSuite.scala
---
@@ -122,4 +134,105 @@ class HiveCommandSuite extends
Github user liancheng commented on a diff in the pull request:
https://github.com/apache/spark/pull/1#discussion_r60928725
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/command/commands.scala
---
@@ -423,6 +426,102 @@ case class ShowTablePropertiesCommand
Github user liancheng commented on a diff in the pull request:
https://github.com/apache/spark/pull/1#discussion_r60927623
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/command/commands.scala
---
@@ -423,6 +426,102 @@ case class ShowTablePropertiesCommand
Github user liancheng commented on a diff in the pull request:
https://github.com/apache/spark/pull/1#discussion_r60927757
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/command/commands.scala
---
@@ -423,6 +426,102 @@ case class ShowTablePropertiesCommand
Github user liancheng commented on a diff in the pull request:
https://github.com/apache/spark/pull/1#discussion_r60927505
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/command/commands.scala
---
@@ -423,6 +426,102 @@ case class ShowTablePropertiesCommand
Github user liancheng commented on a diff in the pull request:
https://github.com/apache/spark/pull/1#discussion_r60927399
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/command/commands.scala
---
@@ -423,6 +426,102 @@ case class ShowTablePropertiesCommand
Github user liancheng commented on a diff in the pull request:
https://github.com/apache/spark/pull/1#discussion_r60926964
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/command/commands.scala
---
@@ -423,6 +426,102 @@ case class ShowTablePropertiesCommand
Github user liancheng commented on a diff in the pull request:
https://github.com/apache/spark/pull/1#discussion_r60926196
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/command/commands.scala
---
@@ -423,6 +426,102 @@ case class ShowTablePropertiesCommand
Github user liancheng commented on a diff in the pull request:
https://github.com/apache/spark/pull/1#discussion_r60925953
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/command/commands.scala
---
@@ -423,6 +426,102 @@ case class ShowTablePropertiesCommand
Github user liancheng commented on a diff in the pull request:
https://github.com/apache/spark/pull/1#discussion_r60923441
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/catalog/InMemoryCatalog.scala
---
@@ -277,9 +277,18 @@ class InMemoryCatalog extends
Github user liancheng commented on a diff in the pull request:
https://github.com/apache/spark/pull/1#discussion_r60923122
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/catalog/SessionCatalog.scala
---
@@ -427,13 +427,18 @@ class SessionCatalog
Github user liancheng commented on a diff in the pull request:
https://github.com/apache/spark/pull/1#discussion_r60923039
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/catalog/SessionCatalog.scala
---
@@ -427,13 +427,18 @@ class SessionCatalog
Github user liancheng commented on a diff in the pull request:
https://github.com/apache/spark/pull/1#discussion_r60922965
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/catalog/InMemoryCatalog.scala
---
@@ -277,9 +277,18 @@ class InMemoryCatalog extends
Github user liancheng commented on the pull request:
https://github.com/apache/spark/pull/12652#issuecomment-214290243
Merging to master.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
GitHub user liancheng opened a pull request:
https://github.com/apache/spark/pull/12652
[SPARK-14875][SQL] Makes OutputWriterFactory.newInstance public
## What changes were proposed in this pull request?
This method was accidentally made `private[sql]` in Spark 2.0. This PR
Github user liancheng commented on the pull request:
https://github.com/apache/spark/pull/12319#issuecomment-213659267
LGTM pending Jenkins.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
Github user liancheng commented on the pull request:
https://github.com/apache/spark/pull/12319#issuecomment-213658964
add to whitelist
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
Github user liancheng commented on the pull request:
https://github.com/apache/spark/pull/12319#issuecomment-213658842
test this please
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
Github user liancheng commented on the pull request:
https://github.com/apache/spark/pull/12611#issuecomment-213514108
LGTM. Merging to master, thanks!
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does
Github user liancheng commented on a diff in the pull request:
https://github.com/apache/spark/pull/12527#discussion_r60695719
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/FileScanRDD.scala
---
@@ -131,4 +134,23 @@ class FileScanRDD
Github user liancheng commented on the pull request:
https://github.com/apache/spark/pull/12527#issuecomment-213167317
retest this please
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
Github user liancheng commented on the pull request:
https://github.com/apache/spark/pull/12527#issuecomment-213034425
The root cause of the deadlock has been found. Essentially, we should
prevent "localhost" to be returned as `FileScanRDD` preferred locations. Here's
Github user liancheng commented on the pull request:
https://github.com/apache/spark/pull/12527#issuecomment-212696111
Yea, agree. Should either prove this is an existing bug (so that it can be
fixed in another PR), or fix it if it's a bug introduced by this change.
---
If your
Github user liancheng commented on the pull request:
https://github.com/apache/spark/pull/12527#issuecomment-212425913
@yhuai This one can probably avoid the deadlock since I worked around it
[here][1]. Also check [this comment] for more information.
Not sure whether we
Github user liancheng commented on the pull request:
https://github.com/apache/spark/pull/12153#issuecomment-212424940
I'm closing this one for #12527, which is a rebased version of this one.
---
If your project is set up for it, you can reply to this email and have your
reply appear
Github user liancheng closed the pull request at:
https://github.com/apache/spark/pull/12153
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature
Github user liancheng commented on the pull request:
https://github.com/apache/spark/pull/12153#issuecomment-212424785
All the timeout in the Jenkins builds were due to a deadlock in
`DAGScheduler`, and can be steadily reproduced locally by running the following
test case
GitHub user liancheng opened a pull request:
https://github.com/apache/spark/pull/12527
[SPARK-14369][SQL] Locality support for FileScanRDD
(This PR is a rebased version of PR #12153.)
## What changes were proposed in this pull request?
This PR adds preliminary
Github user liancheng commented on the pull request:
https://github.com/apache/spark/pull/12319#issuecomment-212323419
@rajeshbalamohan Sorry for the late review and thanks for working on this!
My major concern is that the newly added `OrcRecordReader` should be live in
spark-hive
Github user liancheng commented on a diff in the pull request:
https://github.com/apache/spark/pull/12319#discussion_r60368033
--- Diff:
sql/hive/src/main/scala/org/apache/spark/sql/hive/orc/OrcRelation.scala ---
@@ -337,10 +340,12 @@ private[orc] case class OrcTableScan
Github user liancheng commented on a diff in the pull request:
https://github.com/apache/spark/pull/12319#discussion_r60367904
--- Diff:
sql/hive/src/main/scala/org/apache/spark/sql/hive/orc/OrcRelation.scala ---
@@ -160,19 +160,22 @@ private[sql] class DefaultSource
Github user liancheng commented on a diff in the pull request:
https://github.com/apache/spark/pull/12319#discussion_r60365888
--- Diff: sql/core/pom.xml ---
@@ -128,6 +128,10 @@
xbean-asm5-shaded
test
+
+ org.spark-project.hive
Github user liancheng commented on a diff in the pull request:
https://github.com/apache/spark/pull/12319#discussion_r60365709
--- Diff:
sql/core/src/main/java/org/apache/hadoop/hive/ql/io/orc/OrcRecordReader.java ---
@@ -0,0 +1,94 @@
+/*
+ * Licensed to the Apache
Github user liancheng commented on a diff in the pull request:
https://github.com/apache/spark/pull/12319#discussion_r60365527
--- Diff:
sql/hive/src/main/scala/org/apache/spark/sql/hive/orc/OrcRelation.scala ---
@@ -160,19 +160,22 @@ private[sql] class DefaultSource
Github user liancheng commented on a diff in the pull request:
https://github.com/apache/spark/pull/12319#discussion_r60365406
--- Diff:
sql/hive/src/main/scala/org/apache/spark/sql/hive/orc/OrcRelation.scala ---
@@ -160,19 +160,22 @@ private[sql] class DefaultSource
Github user liancheng commented on a diff in the pull request:
https://github.com/apache/spark/pull/12399#discussion_r60267607
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/objects.scala
---
@@ -130,60 +126,52 @@ case class Invoke(
case
Github user liancheng commented on a diff in the pull request:
https://github.com/apache/spark/pull/12179#discussion_r60094164
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/DataSource.scala
---
@@ -288,15 +288,25 @@ case class DataSource
Github user liancheng commented on the pull request:
https://github.com/apache/spark/pull/12153#issuecomment-211446073
Seems that Jenkins is down...
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does
Github user liancheng commented on the pull request:
https://github.com/apache/spark/pull/12153#issuecomment-211423901
retest this please
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
Github user liancheng commented on the pull request:
https://github.com/apache/spark/pull/12153#issuecomment-210092607
retest this please
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
Github user liancheng commented on a diff in the pull request:
https://github.com/apache/spark/pull/12153#discussion_r59766576
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/sources/interfaces.scala ---
@@ -624,20 +624,40 @@ class HDFSFileCatalog(
def
Github user liancheng commented on a diff in the pull request:
https://github.com/apache/spark/pull/12153#discussion_r59707908
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/FileSourceStrategy.scala
---
@@ -197,4 +204,26 @@ private[sql] object
Github user liancheng commented on a diff in the pull request:
https://github.com/apache/spark/pull/12153#discussion_r59707891
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/FileScanRDD.scala
---
@@ -86,4 +87,9 @@ class FileScanRDD
Github user liancheng commented on a diff in the pull request:
https://github.com/apache/spark/pull/12179#discussion_r59596010
--- Diff:
sql/hive/src/test/scala/org/apache/spark/sql/sources/CommitFailureTestRelationSuite.scala
---
@@ -0,0 +1,45 @@
+/*
+ * Licensed
Github user liancheng commented on a diff in the pull request:
https://github.com/apache/spark/pull/12179#discussion_r59595837
--- Diff:
sql/hive/src/test/scala/org/apache/spark/sql/sources/CommitFailureTestRelationSuite.scala
---
@@ -0,0 +1,45 @@
+/*
+ * Licensed
Github user liancheng commented on a diff in the pull request:
https://github.com/apache/spark/pull/12153#discussion_r59595284
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/FileSourceStrategy.scala
---
@@ -197,4 +204,26 @@ private[sql] object
Github user liancheng commented on a diff in the pull request:
https://github.com/apache/spark/pull/12153#discussion_r59595123
--- Diff:
sql/hive/src/test/scala/org/apache/spark/sql/sources/hadoopFsRelationSuites.scala
---
@@ -668,6 +671,42 @@ abstract class HadoopFsRelationTest
Github user liancheng commented on a diff in the pull request:
https://github.com/apache/spark/pull/12153#discussion_r59594996
--- Diff:
sql/hive/src/test/scala/org/apache/spark/sql/sources/hadoopFsRelationSuites.scala
---
@@ -668,6 +671,42 @@ abstract class HadoopFsRelationTest
Github user liancheng commented on the pull request:
https://github.com/apache/spark/pull/12178#issuecomment-209510234
Closing this in favor of #12361 since master went through some major
changes and it would be hard to resolve conflicts of this one.
---
If your project is set up
Github user liancheng closed the pull request at:
https://github.com/apache/spark/pull/12178
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature
GitHub user liancheng opened a pull request:
https://github.com/apache/spark/pull/12361
[SPARK-14407][SQL] Hides HadoopFsRelation related data source API into
execution/datasources package #12178
## What changes were proposed in this pull request?
This PR moves
Github user liancheng commented on the pull request:
https://github.com/apache/spark/pull/12179#issuecomment-209462358
@cloud-fan Actually not completely copy-pasted since we did major changes
to the data sources API...
---
If your project is set up for it, you can reply
Github user liancheng commented on the pull request:
https://github.com/apache/spark/pull/12179#issuecomment-209047957
Not sure why MiMA complained about `InputMetrics` stuff...
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub
Github user liancheng commented on the pull request:
https://github.com/apache/spark/pull/12179#issuecomment-209047839
retest this please
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
Github user liancheng commented on the pull request:
https://github.com/apache/spark/pull/12179#issuecomment-209047622
@yhuai This should be ready for review.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your
Github user liancheng commented on a diff in the pull request:
https://github.com/apache/spark/pull/12179#discussion_r59430563
--- Diff:
sql/hive/src/main/scala/org/apache/spark/sql/hive/execution/commands.scala ---
@@ -222,7 +222,7 @@ case class CreateMetastoreDataSourceAsSelect
Github user liancheng commented on a diff in the pull request:
https://github.com/apache/spark/pull/12179#discussion_r59430279
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/DataSource.scala
---
@@ -288,15 +288,25 @@ case class DataSource
Github user liancheng commented on the pull request:
https://github.com/apache/spark/pull/12327#issuecomment-208939061
LGTM
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
Github user liancheng commented on the pull request:
https://github.com/apache/spark/pull/12327#issuecomment-208939090
retest this please
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
Github user liancheng commented on the pull request:
https://github.com/apache/spark/pull/12303#issuecomment-208930853
Thanks for the review! Merging to master.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your
Github user liancheng commented on a diff in the pull request:
https://github.com/apache/spark/pull/12303#discussion_r59306581
--- Diff:
sql/hive/src/test/scala/org/apache/spark/sql/hive/execution/SQLQuerySuite.scala
---
@@ -1852,4 +1852,31 @@ class SQLQuerySuite extends
GitHub user liancheng opened a pull request:
https://github.com/apache/spark/pull/12303
[SPARK-14488][SQL] "CREATE TEMPORARY TABLE ... USING ... AS SELECT"
shouldn't create persisted table
## What changes were proposed in this pull request?
When planning logical
Github user liancheng commented on the pull request:
https://github.com/apache/spark/pull/12300#issuecomment-208411048
And the ORC data source, which lives in the `sql/hive`, hasn't been cleaned
yet.
---
If your project is set up for it, you can reply to this email and have your
Github user liancheng commented on the pull request:
https://github.com/apache/spark/pull/12300#issuecomment-208410251
We can also remove [these pattern match guard conditions][1] and the
`spark.sql.sources.fileScan` SQL option.
[1]:
https://github.com/apache/spark/blob
Github user liancheng commented on the pull request:
https://github.com/apache/spark/pull/12184#issuecomment-208244759
Thanks! Merged to master.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does
Github user liancheng commented on the pull request:
https://github.com/apache/spark/pull/12184#issuecomment-208243291
LGTM
@marmbrus Returning `Seq` seems to be not so Java-friendly?
---
If your project is set up for it, you can reply to this email and have your
reply
Github user liancheng commented on a diff in the pull request:
https://github.com/apache/spark/pull/12153#discussion_r59057227
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/FileSourceStrategy.scala
---
@@ -137,10 +140,14 @@ private[sql] object
Github user liancheng commented on a diff in the pull request:
https://github.com/apache/spark/pull/12153#discussion_r58909434
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/FileSourceStrategy.scala
---
@@ -137,10 +140,14 @@ private[sql] object
Github user liancheng commented on a diff in the pull request:
https://github.com/apache/spark/pull/12153#discussion_r58720687
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/FileScanRDD.scala
---
@@ -86,4 +87,8 @@ class FileScanRDD
Github user liancheng commented on a diff in the pull request:
https://github.com/apache/spark/pull/12153#discussion_r58720699
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/FileSourceStrategy.scala
---
@@ -205,4 +210,20 @@ private[sql] object
Github user liancheng commented on a diff in the pull request:
https://github.com/apache/spark/pull/12153#discussion_r58720668
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/sources/interfaces.scala ---
@@ -620,17 +621,28 @@ class HDFSFileCatalog(
if (paths.length
Github user liancheng commented on a diff in the pull request:
https://github.com/apache/spark/pull/12153#discussion_r58720664
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/sources/interfaces.scala ---
@@ -620,17 +621,28 @@ class HDFSFileCatalog(
if (paths.length
Github user liancheng commented on a diff in the pull request:
https://github.com/apache/spark/pull/12153#discussion_r58720692
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/FileSourceStrategy.scala
---
@@ -137,10 +140,12 @@ private[sql] object
GitHub user liancheng opened a pull request:
https://github.com/apache/spark/pull/12179
[SPARK-13681][SQL][WIP] Add back once removed CommitFailureTestRelationSuite
## What changes were proposed in this pull request?
This test suite was removed while refactoring
GitHub user liancheng opened a pull request:
https://github.com/apache/spark/pull/12178
[SPARK-14407][SQL] Hides HadoopFsRelation related data source API into
execution package
## What changes were proposed in this pull request?
This PR moves `HadoopFsRelation` related
901 - 1000 of 5176 matches
Mail list logo
