[GitHub] spark pull request #21711: [SPARK-24681][SQL] Verify nested column names in ...

2018-07-03 Thread maropu
GitHub user maropu opened a pull request: https://github.com/apache/spark/pull/21711 [SPARK-24681][SQL] Verify nested column names in Hive metastore ## What changes were proposed in this pull request? This pr added code to check if nested column names do not include

[GitHub] spark pull request #21704: [SPARK-24734][SQL] Fix containsNull of Concat for...

2018-07-03 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/21704#discussion_r14021 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala --- @@ -2007,7 +2007,14 @@ case class Concat

[GitHub] spark issue #21706: [SPARK-24702] Fix Unable to cast to calendar interval in...

2018-07-03 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/21706 Adds tests in `sql/core/src/test/resources/sql-tests/inputs/cast.sql` then run `SPARK_GENERATE_GOLDEN_FILES=1 ./build/mvn -Dtest=none -DwildcardSuites=org.apache.spark.sql.SQLQueryTestSuite test

[GitHub] spark issue #21657: [SPARK-24676][SQL] Project required data from CSV parsed...

2018-07-03 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/21657 ok, will revive the code here --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail

[GitHub] spark issue #21706: [SPARK-24702] Fix Unable to cast to calendar interval in...

2018-07-03 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/21706 Can you add tests in `sql/core/src/test/resources/sql-tests/inputs/cast.sql`? --- - To unsubscribe, e-mail: reviews-unsubscr

[GitHub] spark pull request #21705: [SPARK-24727][SQL] Add a static config to control...

2018-07-03 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/21705#discussion_r199820857 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/internal/ExecutorSideSQLConfSuite.scala --- @@ -40,16 +40,24 @@ class ExecutorSideSQLConfSuite

[GitHub] spark pull request #21705: [SPARK-24727][SQL] Add a static config to control...

2018-07-03 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/21705#discussion_r199817849 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/internal/ExecutorSideSQLConfSuite.scala --- @@ -40,16 +40,24 @@ class ExecutorSideSQLConfSuite

[GitHub] spark issue #21705: [SPARK-24727][SQL] Add a static config to control cache ...

2018-07-03 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/21705 ok, will do. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark issue #21657: [SPARK-24676][SQL] Project required data from CSV parsed...

2018-07-03 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/21657 @HyukjinKwon kindly ping --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail

[GitHub] spark pull request #21705: [SPARK-24727][SQL] Add a static config to control...

2018-07-03 Thread maropu
GitHub user maropu opened a pull request: https://github.com/apache/spark/pull/21705 [SPARK-24727][SQL] Add a static config to control cache size for generated classes ## What changes were proposed in this pull request? Since SPARK-24250 has been resolved, executors correctly

[GitHub] spark issue #21705: [SPARK-24727][SQL] Add a static config to control cache ...

2018-07-03 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/21705 cc: @cloud-fan --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark issue #21693: [SPARK-24673][SQL] scala sql function from_utc_timestamp...

2018-07-03 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/21693 cc: @ueshin @gatorsmile --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail

[GitHub] spark pull request #21693: [SPARK-24673][SQL] scala sql function from_utc_ti...

2018-07-03 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/21693#discussion_r199744569 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/functions.scala --- @@ -2934,6 +2934,17 @@ object functions { FromUTCTimestamp(ts.expr

[GitHub] spark pull request #21693: [SPARK-24673][SQL] scala sql function from_utc_ti...

2018-07-03 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/21693#discussion_r199744502 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/functions.scala --- @@ -2945,6 +2956,17 @@ object functions { ToUTCTimestamp(ts.expr, Literal

[GitHub] spark issue #21699: [SPARK-24722][SQL] pivot() with Column type argument

2018-07-03 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/21699 `def pivot(pivotColumn: String)`, too? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21699: [SPARK-24722][SQL] pivot() with Column type argument

2018-07-03 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/21699 cc: @rxin @gatorsmile --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail

[GitHub] spark pull request #21699: [SPARK-24722][SQL] pivot() with Column type argum...

2018-07-03 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/21699#discussion_r199742580 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/RelationalGroupedDataset.scala --- @@ -340,36 +340,52 @@ class RelationalGroupedDataset protected[sql

[GitHub] spark issue #21703: [SPARK-24732][SQL] Type coercion between MapTypes.

2018-07-03 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/21703 LGTM --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark pull request #21668: [SPARK-24690][SQL] Add a new config to control pl...

2018-07-02 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/21668#discussion_r199682495 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/catalog/interface.scala --- @@ -375,16 +375,16 @@ case class CatalogStatistics

[GitHub] spark issue #21668: [SPARK-24690][SQL] Add a new config to control plan stat...

2018-07-02 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/21668 yea, ok. I'll reconsider this again. Thanks! --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21657: [SPARK-24676][SQL] Project required data from CSV parsed...

2018-07-02 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/21657 @HyukjinKwon WDYT? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark issue #21679: [SPARK-24695] [SQL]: To add support to return Calendar i...

2018-07-02 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/21679 Actually, the unsafe package does not include user-facing classes. --- - To unsubscribe, e-mail: reviews-unsubscr

[GitHub] spark issue #21657: [SPARK-24676][SQL] Project required data from CSV parsed...

2018-07-02 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/21657 Not sure though, the `tokenIndexArr` implementation is always faster than the unsafe projection? --- - To unsubscribe, e-mail

[GitHub] spark issue #21692: [SPARK-24715][Build] Override jline version in SBT

2018-07-02 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/21692 Thanks, I feel it'd be better to put concrete version numbers in the description. --- - To unsubscribe, e-mail: reviews-unsubscr

[GitHub] spark issue #21687: [SPARK-24165][SQL] Fixing the output data type of CaseWh...

2018-07-02 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/21687 IMHO we need the same fix for both cases. Also, I like simpler implementations in expressions, so IMO it'd be better to fix in analysis phases

[GitHub] spark issue #21692: [SPARK-24715][Build] Override jline version in SBT

2018-07-02 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/21692 If we don't override this, which jline version does `sbt` use? `2.12.1`? --- - To unsubscribe, e-mail: reviews-unsubscr

[GitHub] spark pull request #21667: [SPARK-24691][SQL]Add new API `supportDataType` i...

2018-07-02 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/21667#discussion_r199418986 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/json/JsonFileFormat.scala --- @@ -30,7 +30,7 @@ import

[GitHub] spark pull request #21667: [SPARK-24691][SQL]Add new API `supportDataType` i...

2018-07-02 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/21667#discussion_r199418036 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/FileFormat.scala --- @@ -152,6 +152,16 @@ trait FileFormat

[GitHub] spark pull request #21667: [SPARK-24691][SQL]Add new API `supportDataType` i...

2018-07-02 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/21667#discussion_r199416587 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/DataSourceScanExec.scala --- @@ -306,6 +306,7 @@ case class FileSourceScanExec

[GitHub] spark issue #21694: SPARK-24714 AnalysisSuite should use ClassTag to check t...

2018-07-02 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/21694 I'm not sure this issue should be fixed for test code qualities though, I have one question; is this only a place having this issue

[GitHub] spark issue #21631: [SPARK-24645][SQL] Skip parsing when csvColumnPruning en...

2018-07-02 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/21631 oh, super quick fix ;) Thanks, @MaxGekk In the master, do we still hit the bug when parsing csv data? --- - To unsubscribe

[GitHub] spark issue #21556: [SPARK-24549][SQL] Support Decimal type push down to the...

2018-07-02 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/21556 @wangyum Thanks for the benchmarks! @dongjoon-hyun In the benchmarks above, the results of ORC except for the case `decimal(9, 2)` have worse performance values as compared to the Parquet ones

[GitHub] spark issue #20345: [SPARK-23172][SQL] Expand the ReorderJoin rule to handle...

2018-07-02 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/20345 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark issue #21657: [SPARK-24676][SQL] Project required data from CSV parsed...

2018-07-02 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/21657 @MaxGekk Do you mean we remove the option for column pruning? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org

[GitHub] spark issue #21682: [SPARK-24706][SQL] ByteType and ShortType support pushdo...

2018-07-02 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/21682 @gatorsmile aha, thanks. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail

[GitHub] spark issue #21679: SPARK-24695: To add support to return Calendar interval ...

2018-07-02 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/21679 btw, can you update the title like `[SPARK-24695][SQL]...` --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org

[GitHub] spark issue #21679: SPARK-24695: To add support to return Calendar interval ...

2018-07-02 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/21679 Since `CalendarInterval` is an internal class, I think users are not intended to use the class directly... --- - To unsubscribe

[GitHub] spark issue #21682: [SPARK-24706][SQL] ByteType and ShortType support pushdo...

2018-07-01 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/21682 @HyukjinKwon btw, why we don't support pushdown for these types? Any historical reason? --- - To unsubscribe, e-mail: reviews

[GitHub] spark pull request #21687: [SPARK-24165][SQL] Fixing the output data type of...

2018-07-01 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/21687#discussion_r199386837 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/conditionalExpressions.scala --- @@ -129,7 +129,7 @@ case class CaseWhen

[GitHub] spark pull request #21687: [SPARK-24165][SQL] Fixing the output data type of...

2018-07-01 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/21687#discussion_r199385955 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/ConditionalExpressionSuite.scala --- @@ -113,6 +113,35 @@ class

[GitHub] spark issue #21674: [SPARK-24696][SQL] ColumnPruning rule fails to remove ex...

2018-06-29 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/21674 LGTM, too. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark issue #21667: [SPARK-24691][SQL]Add new API `supportDataType` in FileF...

2018-06-29 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/21667 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark issue #20345: [SPARK-23172][SQL] Expand the ReorderJoin rule to handle...

2018-06-29 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/20345 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark pull request #21668: [SPARK-24690][SQL] Add a new config to control pl...

2018-06-29 Thread maropu
GitHub user maropu opened a pull request: https://github.com/apache/spark/pull/21668 [SPARK-24690][SQL] Add a new config to control plan stats computation in LogicalRelation ## What changes were proposed in this pull request? This pr proposes to propose a new separate config so

[GitHub] spark issue #21668: [SPARK-24690][SQL] Add a new config to control plan stat...

2018-06-29 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/21668 This comes from #20345. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail

[GitHub] spark issue #20345: [SPARK-23172][SQL] Expand the ReorderJoin rule to handle...

2018-06-29 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/20345 I made a pr separately for adding the new config described above: https://github.com/apache/spark/compare/master...maropu:PlanStatsConf I'll file a jira and make a pr for separating discussions

[GitHub] spark pull request #21657: [SPARK-24676][SQL] Project required data from CSV...

2018-06-28 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/21657#discussion_r199034387 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/csv/CSVSuite.scala --- @@ -1579,4 +1579,16 @@ class CSVSuite extends QueryTest

[GitHub] spark pull request #21657: [SPARK-24676][SQL] Project required data from CSV...

2018-06-28 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/21657#discussion_r199033486 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/csv/CSVSuite.scala --- @@ -1579,4 +1579,16 @@ class CSVSuite extends QueryTest

[GitHub] spark pull request #21657: [SPARK-24676][SQL] Project required data from CSV...

2018-06-28 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/21657#discussion_r199033421 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/csv/CSVFileFormat.scala --- @@ -131,20 +132,30 @@ class CSVFileFormat extends

[GitHub] spark pull request #21657: [SPARK-24676][SQL] Project required data from CSV...

2018-06-28 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/21657#discussion_r199032884 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/csv/CSVSuite.scala --- @@ -1579,4 +1579,16 @@ class CSVSuite extends QueryTest

[GitHub] spark pull request #21657: [SPARK-24676][SQL] Project required data from CSV...

2018-06-28 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/21657#discussion_r199021518 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/csv/CSVSuite.scala --- @@ -1579,4 +1579,16 @@ class CSVSuite extends QueryTest

[GitHub] spark issue #21556: [SPARK-24549][SQL] Support Decimal type push down to the...

2018-06-28 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/21556 Can you benchmark code and results (on your env) in `FilterPushdownBenchmark` for this type? --- - To unsubscribe, e-mail

[GitHub] spark pull request #21655: [SPARK-24675][SQL]Rename table: validate existenc...

2018-06-28 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/21655#discussion_r198752208 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/command/DDLSuite.scala --- @@ -441,6 +441,24 @@ abstract class DDLSuite extends QueryTest

[GitHub] spark pull request #21655: [SPARK-24675][SQL]Rename table: validate existenc...

2018-06-28 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/21655#discussion_r198750322 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/catalog/SessionCatalog.scala --- @@ -1366,4 +1367,18 @@ class SessionCatalog

[GitHub] spark pull request #21655: [SPARK-24675][SQL]Rename table: validate existenc...

2018-06-28 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/21655#discussion_r198750034 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/catalog/SessionCatalog.scala --- @@ -1366,4 +1367,18 @@ class SessionCatalog

[GitHub] spark pull request #21655: [SPARK-24675][SQL]Rename table: validate existenc...

2018-06-28 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/21655#discussion_r198748602 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/command/DDLSuite.scala --- @@ -441,6 +441,24 @@ abstract class DDLSuite extends QueryTest

[GitHub] spark pull request #21655: [SPARK-24675][SQL]Rename table: validate existenc...

2018-06-28 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/21655#discussion_r198748442 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/catalog/SessionCatalog.scala --- @@ -1366,4 +1367,18 @@ class SessionCatalog

[GitHub] spark issue #21656: [SPARK-24677][Core]MedianHeap is empty when speculation ...

2018-06-28 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/21656 Can you add a test to check if no exception thrown in that condition with this patch? --- - To unsubscribe, e-mail: reviews

[GitHub] spark issue #21657: [SPARK-24676][SQL] Project required data from CSV parsed...

2018-06-28 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/21657 @HyukjinKwon @MaxGekk --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail

[GitHub] spark pull request #21657: [SPARK-24676][SQL] Project required data from CSV...

2018-06-28 Thread maropu
GitHub user maropu opened a pull request: https://github.com/apache/spark/pull/21657 [SPARK-24676][SQL] Project required data from CSV parsed data when column pruning disabled ## What changes were proposed in this pull request? This pr modified code to project required data

[GitHub] spark issue #20345: [SPARK-23172][SQL] Expand the ReorderJoin rule to handle...

2018-06-28 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/20345 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark issue #20345: [SPARK-23172][SQL] Expand the ReorderJoin rule to handle...

2018-06-28 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/20345 When trying to add the end-to-end test, I got stuck in a trouble; IIUC we cannot enable `StarSchemaDetection.reorderStarJoins` now? That's because we [need to turn off CBO to enable it](https

[GitHub] spark issue #21631: [SPARK-24645][SQL] Skip parsing when csvColumnPruning en...

2018-06-27 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/21631 @HyukjinKwon BTW, can you check this? @MaxGekk Probably, I feel you'd be better to file a new jira for the point you're looking

[GitHub] spark issue #21650: [SPARK-24624] Support mixture of Python UDF and Scalar P...

2018-06-27 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/21650 nit: Also, can you put `[SQL][PYTHON]` in the title? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org

[GitHub] spark issue #20345: [SPARK-23172][SQL] Expand the ReorderJoin rule to handle...

2018-06-27 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/20345 yea, sounds good to me. I'll add end-to-end tests. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org

[GitHub] spark issue #21389: [SPARK-24204][SQL] Verify a schema in Json/Orc/ParquetFi...

2018-06-27 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/21389 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark issue #21598: [SPARK-24605][SQL] size(null) returns null instead of -1

2018-06-26 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/21598 @cloud-fan This merge breaks the build? https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92364/consoleFull

[GitHub] spark pull request #21389: [SPARK-24204][SQL] Verify a schema in Json/Orc/Pa...

2018-06-26 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/21389#discussion_r198355457 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/FileBasedDataSourceSuite.scala --- @@ -202,4 +204,230 @@ class FileBasedDataSourceSuite extends

[GitHub] spark pull request #21389: [SPARK-24204][SQL] Verify a schema in Json/Orc/Pa...

2018-06-26 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/21389#discussion_r198355370 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/DataSourceUtils.scala --- @@ -0,0 +1,101 @@ +/* + * Licensed

[GitHub] spark pull request #21389: [SPARK-24204][SQL] Verify a schema in Json/Orc/Pa...

2018-06-26 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/21389#discussion_r198350214 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/FileBasedDataSourceSuite.scala --- @@ -202,4 +204,230 @@ class FileBasedDataSourceSuite extends

[GitHub] spark pull request #21389: [SPARK-24204][SQL] Verify a schema in Json/Orc/Pa...

2018-06-26 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/21389#discussion_r198350225 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/FileBasedDataSourceSuite.scala --- @@ -202,4 +204,230 @@ class FileBasedDataSourceSuite extends

[GitHub] spark pull request #21389: [SPARK-24204][SQL] Verify a schema in Json/Orc/Pa...

2018-06-26 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/21389#discussion_r198348474 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/FileBasedDataSourceSuite.scala --- @@ -202,4 +204,230 @@ class FileBasedDataSourceSuite extends

[GitHub] spark pull request #20345: [SPARK-23172][SQL] Expand the ReorderJoin rule to...

2018-06-26 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/20345#discussion_r198347568 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/joins.scala --- @@ -84,19 +84,51 @@ object ReorderJoin extends Rule

[GitHub] spark pull request #20345: [SPARK-23172][SQL] Expand the ReorderJoin rule to...

2018-06-26 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/20345#discussion_r198345839 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/joins.scala --- @@ -84,19 +84,50 @@ object ReorderJoin extends Rule

[GitHub] spark pull request #20345: [SPARK-23172][SQL] Expand the ReorderJoin rule to...

2018-06-26 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/20345#discussion_r198341155 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/joins.scala --- @@ -84,19 +84,51 @@ object ReorderJoin extends Rule

[GitHub] spark pull request #21590: [SPARK-24423][SQL] Add a new option for JDBC sour...

2018-06-26 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/21590#discussion_r198340472 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/jdbc/JDBCOptions.scala --- @@ -109,6 +134,20 @@ class JDBCOptions( s

[GitHub] spark issue #21643: [SPARK-24659][SQL] GenericArrayData.equals should respec...

2018-06-26 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/21643 LGTM except for one minor comment. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e

[GitHub] spark pull request #21643: [SPARK-24659][SQL] GenericArrayData.equals should...

2018-06-26 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/21643#discussion_r198339961 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/util/ComplexDataSuite.scala --- @@ -104,4 +104,40 @@ class ComplexDataSuite extends

[GitHub] spark issue #21631: [SPARK-24645][SQL] Skip parsing when csvColumnPruning en...

2018-06-26 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/21631 v2.5.9 also have the same behaviour? Anyway, it'd be better to ask the author ;) I asked before and I got quick response

[GitHub] spark pull request #21643: [SPARK-24659][SQL] GenericArrayData.equals should...

2018-06-26 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/21643#discussion_r198118072 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/util/ComplexDataSuite.scala --- @@ -104,4 +104,13 @@ class ComplexDataSuite extends

[GitHub] spark issue #20345: [SPARK-23172][SQL] Expand the ReorderJoin rule to handle...

2018-06-26 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/20345 I'm looking into the failure... --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e

[GitHub] spark pull request #20345: [SPARK-23172][SQL] Expand the ReorderJoin rule to...

2018-06-26 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/20345#discussion_r198059157 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/planning/patterns.scala --- @@ -141,14 +141,16 @@ object ExtractEquiJoinKeys extends

[GitHub] spark pull request #20345: [SPARK-23172][SQL] Expand the ReorderJoin rule to...

2018-06-26 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/20345#discussion_r198052416 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/planning/patterns.scala --- @@ -141,14 +141,16 @@ object ExtractEquiJoinKeys extends

[GitHub] spark pull request #20345: [SPARK-23172][SQL] Expand the ReorderJoin rule to...

2018-06-26 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/20345#discussion_r198043685 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/joins.scala --- @@ -84,19 +84,50 @@ object ReorderJoin extends Rule

[GitHub] spark pull request #20345: [SPARK-23172][SQL] Expand the ReorderJoin rule to...

2018-06-26 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/20345#discussion_r198043470 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/optimizer/JoinOptimizationSuite.scala --- @@ -81,14 +92,14 @@ class

[GitHub] spark pull request #20345: [SPARK-23172][SQL] Expand the ReorderJoin rule to...

2018-06-26 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/20345#discussion_r198043374 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/joins.scala --- @@ -84,19 +84,50 @@ object ReorderJoin extends Rule

[GitHub] spark issue #21590: [SPARK-24423][SQL] Add a new option for JDBC sources

2018-06-26 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/21590 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark issue #21641: [SPARK-24658][SQL] Remove workaround for ANTLR bug

2018-06-26 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/21641 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark issue #21641: [SPARK-24658][SQL] Remove workaround for ANTLR bug

2018-06-26 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/21641 I'm not 100% that we could remove this workaround because we hold some workarounds that are not essentially needed anymore, e.g., `SnappyOutputStreamWrapper`. Anyway, in the description

[GitHub] spark pull request #21590: [SPARK-24423][SQL] Add a new option for JDBC sour...

2018-06-26 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/21590#discussion_r198027023 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/jdbc/JDBCOptions.scala --- @@ -174,3 +209,25 @@ object JDBCOptions

[GitHub] spark pull request #21590: [SPARK-24423][SQL] Add a new option for JDBC sour...

2018-06-25 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/21590#discussion_r198021171 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/jdbc/JDBCOptions.scala --- @@ -65,13 +65,32 @@ class JDBCOptions

[GitHub] spark pull request #21590: [SPARK-24423][SQL] Add a new option for JDBC sour...

2018-06-25 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/21590#discussion_r198021225 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/jdbc/JDBCOptions.scala --- @@ -65,13 +65,32 @@ class JDBCOptions

[GitHub] spark pull request #21590: [SPARK-24423][SQL] Add a new option for JDBC sour...

2018-06-25 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/21590#discussion_r198021157 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/jdbc/JDBCOptions.scala --- @@ -65,13 +65,32 @@ class JDBCOptions

[GitHub] spark pull request #21590: [SPARK-24423][SQL] Add a new option for JDBC sour...

2018-06-25 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/21590#discussion_r198019979 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/jdbc/JDBCOptions.scala --- @@ -174,3 +209,25 @@ object JDBCOptions

[GitHub] spark pull request #21590: [SPARK-24423][SQL] Add a new option for JDBC sour...

2018-06-25 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/21590#discussion_r198019710 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/jdbc/JDBCOptions.scala --- @@ -150,6 +183,7 @@ class JDBCOptions

[GitHub] spark pull request #21590: [SPARK-24423][SQL] Add a new option for JDBC sour...

2018-06-25 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/21590#discussion_r198018682 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/jdbc/JDBCOptions.scala --- @@ -65,13 +65,32 @@ class JDBCOptions

[GitHub] spark pull request #21590: [SPARK-24423][SQL] Add a new option for JDBC sour...

2018-06-25 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/21590#discussion_r198018360 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/jdbc/JDBCOptions.scala --- @@ -174,3 +209,25 @@ object JDBCOptions

[GitHub] spark pull request #21631: [SPARK-24645][SQL] Skip parsing when csvColumnPru...

2018-06-25 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/21631#discussion_r197973962 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/csv/UnivocityParser.scala --- @@ -183,11 +183,19 @@ class UnivocityParser

[GitHub] spark issue #21389: [SPARK-24204][SQL] Verify a schema in Json/Orc/ParquetFi...

2018-06-25 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/21389 ping --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark issue #20345: [SPARK-23172][SQL] Expand the ReorderJoin rule to handle...

2018-06-25 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/20345 ping --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

<    4   5   6   7   8   9   10   11   12   13   >