[jira] [Commented] (SPARK-14959) ​Problem Reading partitioned ORC or Parquet files

[ https://issues.apache.org/jira/browse/SPARK-14959?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15267410#comment-15267410 ] Gang Wu commented on SPARK-14959: - [~syepes] I faced the same exception when I try to query partitioned

[jira] [Commented] (SPARK-17243) Spark 2.0 history server summary page gets stuck at "loading history summary" with 10K+ application history

[ https://issues.apache.org/jira/browse/SPARK-17243?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15447172#comment-15447172 ] Gang Wu commented on SPARK-17243: - I imported the last change. I can get all application list from rest

[jira] [Commented] (SPARK-17243) Spark 2.0 history server summary page gets stuck at "loading history summary" with 10K+ application history

[ https://issues.apache.org/jira/browse/SPARK-17243?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15447099#comment-15447099 ] Gang Wu commented on SPARK-17243: - I've test this PR. It indeed reduces the number of application

[jira] [Commented] (SPARK-17243) Spark 2.0 history server summary page gets stuck at "loading history summary" with 10K+ application history

[ https://issues.apache.org/jira/browse/SPARK-17243?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15447285#comment-15447285 ] Gang Wu commented on SPARK-17243: - Yup you're right. I finally got some app_ids that were not in the

[jira] [Created] (SPARK-17477) SparkSQL cannot handle schema evolution from Int -> Long when parquet files have Int as its type while hive metastore has Long as its type

Gang Wu created SPARK-17477: --- Summary: SparkSQL cannot handle schema evolution from Int -> Long when parquet files have Int as its type while hive metastore has Long as its type Key: SPARK-17477 URL:

[jira] [Commented] (SPARK-17477) SparkSQL cannot handle schema evolution from Int -> Long when parquet files have Int as its type while hive metastore has Long as its type

[ https://issues.apache.org/jira/browse/SPARK-17477?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15477945#comment-15477945 ] Gang Wu commented on SPARK-17477: - I'm working on a fix for this issue. Will send pull request soon. >

[jira] [Updated] (SPARK-17477) SparkSQL cannot handle schema evolution from Int -> Long when parquet files have Int as its type while hive metastore has Long as its type

[ https://issues.apache.org/jira/browse/SPARK-17477?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gang Wu updated SPARK-17477: Shepherd: Gang Wu > SparkSQL cannot handle schema evolution from Int -> Long when parquet files > have

[jira] [Commented] (SPARK-17477) SparkSQL cannot handle schema evolution from Int -> Long when parquet files have Int as its type while hive metastore has Long as its type

[ https://issues.apache.org/jira/browse/SPARK-17477?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15484629#comment-15484629 ] Gang Wu commented on SPARK-17477: - [~hyukjin.kwon] I agree with you. But both issues are targeting at

[jira] [Updated] (SPARK-17477) SparkSQL cannot handle schema evolution from Int -> Long when parquet files have Int as its type while hive metastore has Long as its type

[ https://issues.apache.org/jira/browse/SPARK-17477?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gang Wu updated SPARK-17477: Target Version/s: (was: 2.1.0) > SparkSQL cannot handle schema evolution from Int -> Long when parquet

[jira] [Commented] (SPARK-17477) SparkSQL cannot handle schema evolution from Int -> Long when parquet files have Int as its type while hive metastore has Long as its type

[ https://issues.apache.org/jira/browse/SPARK-17477?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15494681#comment-15494681 ] Gang Wu commented on SPARK-17477: - Just confirmed that this also doesn't work with vectorized reader.

[jira] [Updated] (SPARK-17477) SparkSQL cannot handle schema evolution from Int -> Long when parquet files have Int as its type while hive metastore has Long as its type

[ https://issues.apache.org/jira/browse/SPARK-17477?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gang Wu updated SPARK-17477: Shepherd: (was: Gang Wu) > SparkSQL cannot handle schema evolution from Int -> Long when parquet files

[jira] [Created] (SPARK-17671) Spark 2.0 history server summary page is slow even set spark.history.ui.maxApplications

Gang Wu created SPARK-17671: --- Summary: Spark 2.0 history server summary page is slow even set spark.history.ui.maxApplications Key: SPARK-17671 URL: https://issues.apache.org/jira/browse/SPARK-17671

[jira] [Commented] (SPARK-17671) Spark 2.0 history server summary page is slow even set spark.history.ui.maxApplications

[ https://issues.apache.org/jira/browse/SPARK-17671?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15524173#comment-15524173 ] Gang Wu commented on SPARK-17671: - I'm working on this and will send a pull request soon. > Spark 2.0

[jira] [Commented] (SPARK-17601) SparkSQL vectorization cannot handle schema evolution for parquet tables when parquet files use Int whereas DataFrame uses Long

[ https://issues.apache.org/jira/browse/SPARK-17601?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15514019#comment-15514019 ] Gang Wu commented on SPARK-17601: - [~hyukjin.kwon] Yes I agree. I just created these JIRAs for issues we

[jira] [Created] (SPARK-17601) SparkSQL vectorization cannot handle schema evolution for parquet tables when parquet files use Int whereas DataFrame uses Long

Gang Wu created SPARK-17601: --- Summary: SparkSQL vectorization cannot handle schema evolution for parquet tables when parquet files use Int whereas DataFrame uses Long Key: SPARK-17601 URL:

[jira] [Commented] (SPARK-17243) Spark 2.0 history server summary page gets stuck at "loading history summary" with 10K+ application history

[ https://issues.apache.org/jira/browse/SPARK-17243?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15439517#comment-15439517 ] Gang Wu commented on SPARK-17243: - Thanks [~ajbozarth]! Let me know when it is done. > Spark 2.0 history

[jira] [Commented] (SPARK-17243) Spark 2.0 history server summary page gets stuck at "loading history summary" with 10K+ application history

[ https://issues.apache.org/jira/browse/SPARK-17243?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15437688#comment-15437688 ] Gang Wu commented on SPARK-17243: - Hi Alex, I think in Spark 1.5 history server obtains all application

[jira] [Updated] (SPARK-17243) Spark 2.0 history server summary page gets stuck at "loading history summary" with 10K+ application history

[ https://issues.apache.org/jira/browse/SPARK-17243?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gang Wu updated SPARK-17243: Summary: Spark 2.0 history server summary page gets stuck at "loading history summary" with 10K+

[jira] [Created] (SPARK-17243) Spark history server summary page gets stuck at "loading history summary" with 10K+ application history

Gang Wu created SPARK-17243: --- Summary: Spark history server summary page gets stuck at "loading history summary" with 10K+ application history Key: SPARK-17243 URL: https://issues.apache.org/jira/browse/SPARK-17243

[jira] [Updated] (SPARK-17243) Spark history server summary page gets stuck at "loading history summary" with 10K+ application history

[ https://issues.apache.org/jira/browse/SPARK-17243?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gang Wu updated SPARK-17243: Description: The summary page of Spark 2.0 history server web UI keep displaying "Loading history

[jira] [Commented] (SPARK-17243) Spark 2.0 history server summary page gets stuck at "loading history summary" with 10K+ application history

[ https://issues.apache.org/jira/browse/SPARK-17243?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15437691#comment-15437691 ] Gang Wu commented on SPARK-17243: - This doesn't work. This is for the cache of WEB UIs not for the

[jira] [Commented] (SPARK-17672) Spark 2.0 history server web Ui takes too long for a single application

[ https://issues.apache.org/jira/browse/SPARK-17672?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15525144#comment-15525144 ] Gang Wu commented on SPARK-17672: - They are similar but different. This JIRA deals with the approach to

[jira] [Created] (SPARK-17672) Spark 2.0 history server web Ui takes too long for a single application

Gang Wu created SPARK-17672: --- Summary: Spark 2.0 history server web Ui takes too long for a single application Key: SPARK-17672 URL: https://issues.apache.org/jira/browse/SPARK-17672 Project: Spark

[jira] [Updated] (SPARK-17672) Spark 2.0 history server web Ui takes too long for a single application

[ https://issues.apache.org/jira/browse/SPARK-17672?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gang Wu updated SPARK-17672: Description: When there are 10K application history in the history server back end, it can take a very

[jira] [Commented] (SPARK-17671) Spark 2.0 history server summary page is slow even set spark.history.ui.maxApplications

[ https://issues.apache.org/jira/browse/SPARK-17671?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15524595#comment-15524595 ] Gang Wu commented on SPARK-17671: - Hi [~ajbozarth], can you take a look at the PR? Thanks! > Spark 2.0

[jira] [Commented] (SPARK-17672) Spark 2.0 history server web Ui takes too long for a single application

[ https://issues.apache.org/jira/browse/SPARK-17672?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15524593#comment-15524593 ] Gang Wu commented on SPARK-17672: - Hi [~ajbozarth], can you take a look at the PR? Thanks! > Spark 2.0

[jira] [Commented] (SPARK-17672) Spark 2.0 history server web Ui takes too long for a single application

[ https://issues.apache.org/jira/browse/SPARK-17672?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15524424#comment-15524424 ] Gang Wu commented on SPARK-17672: - I'm working on a fix and will send a PR soon. > Spark 2.0 history