[jira] [Commented] (SPARK-14959) ​Problem Reading partitioned ORC or Parquet files

[ https://issues.apache.org/jira/browse/SPARK-14959?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15267410#comment-15267410 ] Gang Wu commented on SPARK-14959: - [~syepes] I faced the same exception when I try to que

[jira] [Created] (SPARK-17477) SparkSQL cannot handle schema evolution from Int -> Long when parquet files have Int as its type while hive metastore has Long as its type

Gang Wu created SPARK-17477: --- Summary: SparkSQL cannot handle schema evolution from Int -> Long when parquet files have Int as its type while hive metastore has Long as its type Key: SPARK-17477 URL: https://issues.apa

[jira] [Updated] (SPARK-17477) SparkSQL cannot handle schema evolution from Int -> Long when parquet files have Int as its type while hive metastore has Long as its type

[ https://issues.apache.org/jira/browse/SPARK-17477?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gang Wu updated SPARK-17477: Shepherd: Gang Wu > SparkSQL cannot handle schema evolution from Int -> Long when parquet files > have Int

[jira] [Commented] (SPARK-17477) SparkSQL cannot handle schema evolution from Int -> Long when parquet files have Int as its type while hive metastore has Long as its type

[ https://issues.apache.org/jira/browse/SPARK-17477?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15477945#comment-15477945 ] Gang Wu commented on SPARK-17477: - I'm working on a fix for this issue. Will send pull re

[jira] [Updated] (SPARK-17477) SparkSQL cannot handle schema evolution from Int -> Long when parquet files have Int as its type while hive metastore has Long as its type

[ https://issues.apache.org/jira/browse/SPARK-17477?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gang Wu updated SPARK-17477: Shepherd: (was: Gang Wu) > SparkSQL cannot handle schema evolution from Int -> Long when parquet files >

[jira] [Updated] (SPARK-17477) SparkSQL cannot handle schema evolution from Int -> Long when parquet files have Int as its type while hive metastore has Long as its type

[ https://issues.apache.org/jira/browse/SPARK-17477?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gang Wu updated SPARK-17477: Target Version/s: (was: 2.1.0) > SparkSQL cannot handle schema evolution from Int -> Long when parquet fi

[jira] [Commented] (SPARK-17477) SparkSQL cannot handle schema evolution from Int -> Long when parquet files have Int as its type while hive metastore has Long as its type

[ https://issues.apache.org/jira/browse/SPARK-17477?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15484629#comment-15484629 ] Gang Wu commented on SPARK-17477: - [~hyukjin.kwon] I agree with you. But both issues are

[jira] [Commented] (SPARK-17477) SparkSQL cannot handle schema evolution from Int -> Long when parquet files have Int as its type while hive metastore has Long as its type

[ https://issues.apache.org/jira/browse/SPARK-17477?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15494681#comment-15494681 ] Gang Wu commented on SPARK-17477: - Just confirmed that this also doesn't work with vector

[jira] [Created] (SPARK-17601) SparkSQL vectorization cannot handle schema evolution for parquet tables when parquet files use Int whereas DataFrame uses Long

Gang Wu created SPARK-17601: --- Summary: SparkSQL vectorization cannot handle schema evolution for parquet tables when parquet files use Int whereas DataFrame uses Long Key: SPARK-17601 URL: https://issues.apache.org/jira

[jira] [Commented] (SPARK-17601) SparkSQL vectorization cannot handle schema evolution for parquet tables when parquet files use Int whereas DataFrame uses Long

[ https://issues.apache.org/jira/browse/SPARK-17601?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15514019#comment-15514019 ] Gang Wu commented on SPARK-17601: - [~hyukjin.kwon] Yes I agree. I just created these JIRA

[jira] [Created] (SPARK-17671) Spark 2.0 history server summary page is slow even set spark.history.ui.maxApplications

Gang Wu created SPARK-17671: --- Summary: Spark 2.0 history server summary page is slow even set spark.history.ui.maxApplications Key: SPARK-17671 URL: https://issues.apache.org/jira/browse/SPARK-17671 Project

[jira] [Commented] (SPARK-17671) Spark 2.0 history server summary page is slow even set spark.history.ui.maxApplications

[ https://issues.apache.org/jira/browse/SPARK-17671?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15524173#comment-15524173 ] Gang Wu commented on SPARK-17671: - I'm working on this and will send a pull request soon.

[jira] [Created] (SPARK-17672) Spark 2.0 history server web Ui takes too long for a single application

Gang Wu created SPARK-17672: --- Summary: Spark 2.0 history server web Ui takes too long for a single application Key: SPARK-17672 URL: https://issues.apache.org/jira/browse/SPARK-17672 Project: Spark

[jira] [Updated] (SPARK-17672) Spark 2.0 history server web Ui takes too long for a single application

[ https://issues.apache.org/jira/browse/SPARK-17672?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gang Wu updated SPARK-17672: Description: When there are 10K application history in the history server back end, it can take a very lon

[jira] [Commented] (SPARK-17672) Spark 2.0 history server web Ui takes too long for a single application

[ https://issues.apache.org/jira/browse/SPARK-17672?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15524424#comment-15524424 ] Gang Wu commented on SPARK-17672: - I'm working on a fix and will send a PR soon. > Spark

[jira] [Commented] (SPARK-17671) Spark 2.0 history server summary page is slow even set spark.history.ui.maxApplications

[ https://issues.apache.org/jira/browse/SPARK-17671?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15524595#comment-15524595 ] Gang Wu commented on SPARK-17671: - Hi [~ajbozarth], can you take a look at the PR? Thanks

[jira] [Commented] (SPARK-17672) Spark 2.0 history server web Ui takes too long for a single application

[ https://issues.apache.org/jira/browse/SPARK-17672?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15524593#comment-15524593 ] Gang Wu commented on SPARK-17672: - Hi [~ajbozarth], can you take a look at the PR? Thanks

[jira] [Commented] (SPARK-17672) Spark 2.0 history server web Ui takes too long for a single application

[ https://issues.apache.org/jira/browse/SPARK-17672?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15525144#comment-15525144 ] Gang Wu commented on SPARK-17672: - They are similar but different. This JIRA deals with t

[jira] [Updated] (SPARK-17243) Spark history server summary page gets stuck at "loading history summary" with 10K+ application history

[ https://issues.apache.org/jira/browse/SPARK-17243?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gang Wu updated SPARK-17243: Description: The summary page of Spark 2.0 history server web UI keep displaying "Loading history summary.

[jira] [Created] (SPARK-17243) Spark history server summary page gets stuck at "loading history summary" with 10K+ application history

Gang Wu created SPARK-17243: --- Summary: Spark history server summary page gets stuck at "loading history summary" with 10K+ application history Key: SPARK-17243 URL: https://issues.apache.org/jira/browse/SPARK-17243

[jira] [Updated] (SPARK-17243) Spark 2.0 history server summary page gets stuck at "loading history summary" with 10K+ application history

[ https://issues.apache.org/jira/browse/SPARK-17243?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gang Wu updated SPARK-17243: Summary: Spark 2.0 history server summary page gets stuck at "loading history summary" with 10K+ applicatio

[jira] [Commented] (SPARK-17243) Spark 2.0 history server summary page gets stuck at "loading history summary" with 10K+ application history

[ https://issues.apache.org/jira/browse/SPARK-17243?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15437688#comment-15437688 ] Gang Wu commented on SPARK-17243: - Hi Alex, I think in Spark 1.5 history server obtains a

[jira] [Commented] (SPARK-17243) Spark 2.0 history server summary page gets stuck at "loading history summary" with 10K+ application history

[ https://issues.apache.org/jira/browse/SPARK-17243?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15437691#comment-15437691 ] Gang Wu commented on SPARK-17243: - This doesn't work. This is for the cache of WEB UIs no

[jira] [Commented] (SPARK-17243) Spark 2.0 history server summary page gets stuck at "loading history summary" with 10K+ application history

[ https://issues.apache.org/jira/browse/SPARK-17243?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15439517#comment-15439517 ] Gang Wu commented on SPARK-17243: - Thanks [~ajbozarth]! Let me know when it is done. > S

[jira] [Commented] (SPARK-17243) Spark 2.0 history server summary page gets stuck at "loading history summary" with 10K+ application history

[ https://issues.apache.org/jira/browse/SPARK-17243?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15447099#comment-15447099 ] Gang Wu commented on SPARK-17243: - I've test this PR. It indeed reduces the number of app

[jira] [Commented] (SPARK-17243) Spark 2.0 history server summary page gets stuck at "loading history summary" with 10K+ application history

[ https://issues.apache.org/jira/browse/SPARK-17243?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15447172#comment-15447172 ] Gang Wu commented on SPARK-17243: - I imported the last change. I can get all application

[jira] [Commented] (SPARK-17243) Spark 2.0 history server summary page gets stuck at "loading history summary" with 10K+ application history

[ https://issues.apache.org/jira/browse/SPARK-17243?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15447285#comment-15447285 ] Gang Wu commented on SPARK-17243: - Yup you're right. I finally got some app_ids that were