[jira] [Created] (SPARK-25693) Fix the multiple "manager" classes in org.apache.spark.deploy.security

2018-10-09 Thread Marcelo Vanzin (JIRA)
Marcelo Vanzin created SPARK-25693: -- Summary: Fix the multiple "manager" classes in org.apache.spark.deploy.security Key: SPARK-25693 URL: https://issues.apache.org/jira/browse/SPARK-25693 Project:

[jira] [Updated] (SPARK-25692) Flaky test: ChunkFetchIntegrationSuite

2018-10-09 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25692?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-25692: Priority: Blocker (was: Major) > Flaky test: ChunkFetchIntegrationSuite >

[jira] [Updated] (SPARK-25692) Flaky test: ChunkFetchIntegrationSuite

2018-10-09 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25692?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-25692: - Description: Looks like the whole test suite is pretty flaky. See:

[jira] [Updated] (SPARK-25694) URL.setURLStreamHandlerFactory causing incompatible HttpURLConnection issue

2018-10-09 Thread Bo Yang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25694?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bo Yang updated SPARK-25694: Summary: URL.setURLStreamHandlerFactory causing incompatible HttpURLConnection issue (was:

[jira] [Commented] (SPARK-25682) Docker images generated from dev build and from dist tarball are different

2018-10-09 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25682?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16644270#comment-16644270 ] Apache Spark commented on SPARK-25682: -- User 'vanzin' has created a pull request for this issue:

[jira] [Updated] (SPARK-25640) Clarify/Improve EvalType for grouped aggregate and window aggregate

2018-10-09 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25640?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-25640: Target Version/s: 3.0.0 > Clarify/Improve EvalType for grouped aggregate and window aggregate >

[jira] [Created] (SPARK-25695) Spark history server event log store problem

2018-10-09 Thread Si Chen (JIRA)
Si Chen created SPARK-25695: --- Summary: Spark history server event log store problem Key: SPARK-25695 URL: https://issues.apache.org/jira/browse/SPARK-25695 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-25695) Spark history server event log store problem

2018-10-09 Thread Si Chen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25695?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Si Chen updated SPARK-25695: Description: Envirment:spark 2.0.0 hadoop 2.7.3 spark-default.conf {code:java}

[jira] [Commented] (SPARK-24291) Data source table is not displaying records when files are uploaded to table location

2018-10-09 Thread sandeep katta (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24291?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16644435#comment-16644435 ] sandeep katta commented on SPARK-24291: --- [~cloud_fan] [~srowen] is this valid use case ? . Because

[jira] [Commented] (SPARK-24291) Data source table is not displaying records when files are uploaded to table location

2018-10-09 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24291?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16644459#comment-16644459 ] Sean Owen commented on SPARK-24291: --- I don't know. I would not expect this to necessarily update. What

[jira] [Commented] (SPARK-21569) Internal Spark class needs to be kryo-registered

2018-10-09 Thread Lijun Cao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21569?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16644394#comment-16644394 ] Lijun Cao commented on SPARK-21569: --- Register

[jira] [Assigned] (SPARK-25682) Docker images generated from dev build and from dist tarball are different

2018-10-09 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25682?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25682: Assignee: Apache Spark > Docker images generated from dev build and from dist tarball

[jira] [Assigned] (SPARK-25682) Docker images generated from dev build and from dist tarball are different

2018-10-09 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25682?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25682: Assignee: (was: Apache Spark) > Docker images generated from dev build and from dist

[jira] [Commented] (SPARK-25682) Docker images generated from dev build and from dist tarball are different

2018-10-09 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25682?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16644269#comment-16644269 ] Apache Spark commented on SPARK-25682: -- User 'vanzin' has created a pull request for this issue:

[jira] [Assigned] (SPARK-25685) Allow regression testing in enterprise Jenkins

2018-10-09 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25685?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25685: Assignee: Apache Spark > Allow regression testing in enterprise Jenkins >

[jira] [Commented] (SPARK-25684) Organize header related codes in CSV datasource

2018-10-09 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25684?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16642922#comment-16642922 ] Apache Spark commented on SPARK-25684: -- User 'HyukjinKwon' has created a pull request for this

[jira] [Commented] (SPARK-25683) Make AsyncEventQueue.lastReportTimestamp inital value as the currentTime instead of 0

2018-10-09 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25683?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16642936#comment-16642936 ] Apache Spark commented on SPARK-25683: -- User 'shivusondur' has created a pull request for this

[jira] [Assigned] (SPARK-25683) Make AsyncEventQueue.lastReportTimestamp inital value as the currentTime instead of 0

2018-10-09 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25683?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25683: Assignee: Apache Spark > Make AsyncEventQueue.lastReportTimestamp inital value as the

[jira] [Assigned] (SPARK-25683) Make AsyncEventQueue.lastReportTimestamp inital value as the currentTime instead of 0

2018-10-09 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25683?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25683: Assignee: (was: Apache Spark) > Make AsyncEventQueue.lastReportTimestamp inital

[jira] [Commented] (SPARK-25683) Make AsyncEventQueue.lastReportTimestamp inital value as the currentTime instead of 0

2018-10-09 Thread shivusondur (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25683?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16642874#comment-16642874 ] shivusondur commented on SPARK-25683: - i am working on it > Make

[jira] [Assigned] (SPARK-25497) limit operation within whole stage codegen should not consume all the inputs

2018-10-09 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25497?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kazuaki Ishizaki reassigned SPARK-25497: Assignee: Wenchen Fan > limit operation within whole stage codegen should not

[jira] [Assigned] (SPARK-25684) Organize header related codes in CSV datasource

2018-10-09 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25684?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25684: Assignee: Hyukjin Kwon (was: Apache Spark) > Organize header related codes in CSV

[jira] [Assigned] (SPARK-25684) Organize header related codes in CSV datasource

2018-10-09 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25684?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25684: Assignee: Apache Spark (was: Hyukjin Kwon) > Organize header related codes in CSV

[jira] [Resolved] (SPARK-25669) Check CSV header only when it exists

2018-10-09 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25669?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-25669. -- Resolution: Fixed Fix Version/s: 2.4.0 Issue resolved by pull request 22656

[jira] [Updated] (SPARK-25669) Check CSV header only when it exists

2018-10-09 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25669?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-25669: - Fix Version/s: 3.0.0 > Check CSV header only when it exists >

[jira] [Assigned] (SPARK-25669) Check CSV header only when it exists

2018-10-09 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25669?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-25669: Assignee: Maxim Gekk > Check CSV header only when it exists >

[jira] [Created] (SPARK-25684) Organize header related codes in CSV datasource

2018-10-09 Thread Hyukjin Kwon (JIRA)
Hyukjin Kwon created SPARK-25684: Summary: Organize header related codes in CSV datasource Key: SPARK-25684 URL: https://issues.apache.org/jira/browse/SPARK-25684 Project: Spark Issue Type:

[jira] [Resolved] (SPARK-25497) limit operation within whole stage codegen should not consume all the inputs

2018-10-09 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25497?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kazuaki Ishizaki resolved SPARK-25497. -- Resolution: Fixed Fix Version/s: 3.0.0 > limit operation within whole stage

[jira] [Assigned] (SPARK-25685) Allow regression testing in enterprise Jenkins

2018-10-09 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25685?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25685: Assignee: (was: Apache Spark) > Allow regression testing in enterprise Jenkins >

[jira] [Commented] (SPARK-25685) Allow regression testing in enterprise Jenkins

2018-10-09 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25685?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16642963#comment-16642963 ] Apache Spark commented on SPARK-25685: -- User 'LantaoJin' has created a pull request for this issue:

[jira] [Commented] (SPARK-25685) Allow regression testing in enterprise Jenkins

2018-10-09 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25685?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16642966#comment-16642966 ] Apache Spark commented on SPARK-25685: -- User 'LantaoJin' has created a pull request for this issue:

[jira] [Created] (SPARK-25685) Allow regression testing in enterprise Jenkins

2018-10-09 Thread Lantao Jin (JIRA)
Lantao Jin created SPARK-25685: -- Summary: Allow regression testing in enterprise Jenkins Key: SPARK-25685 URL: https://issues.apache.org/jira/browse/SPARK-25685 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-25685) Allow running tests in Jenkins in enterprise Git repository

2018-10-09 Thread Lantao Jin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25685?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lantao Jin updated SPARK-25685: --- Summary: Allow running tests in Jenkins in enterprise Git repository (was: Allow regression

[jira] [Commented] (SPARK-20780) Spark Kafka10 Consumer Hangs

2018-10-09 Thread Mary Scott (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20780?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16643235#comment-16643235 ] Mary Scott commented on SPARK-20780: Hello [~jayadeepj]. I am struggling with the identical issue.

[jira] [Commented] (SPARK-25344) Break large tests.py files into smaller files

2018-10-09 Thread Imran Rashid (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25344?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16643663#comment-16643663 ] Imran Rashid commented on SPARK-25344: -- [~hyukjin.kwon] go for it! I dunno if [~bryanc] has

[jira] [Commented] (SPARK-18660) Parquet complains "Can not initialize counter due to context is not a instance of TaskInputOutputContext, but is org.apache.hadoop.mapreduce.task.TaskAttemptContextImp

2018-10-09 Thread Paul Praet (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18660?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16643418#comment-16643418 ] Paul Praet commented on SPARK-18660: It's really polluting our logs. Any workaround ? > Parquet

[jira] [Assigned] (SPARK-24851) Map a Stage ID to it's Associated Job ID in UI

2018-10-09 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24851?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves reassigned SPARK-24851: - Assignee: Parth Gandhi > Map a Stage ID to it's Associated Job ID in UI >

[jira] [Created] (SPARK-25686) date_trunc Spark SQL function silently returns null if parameters are swapped

2018-10-09 Thread Zack Behringer (JIRA)
Zack Behringer created SPARK-25686: -- Summary: date_trunc Spark SQL function silently returns null if parameters are swapped Key: SPARK-25686 URL: https://issues.apache.org/jira/browse/SPARK-25686

[jira] [Assigned] (SPARK-25535) Work around bad error checking in commons-crypto

2018-10-09 Thread Imran Rashid (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25535?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Imran Rashid reassigned SPARK-25535: Assignee: Marcelo Vanzin > Work around bad error checking in commons-crypto >

[jira] [Resolved] (SPARK-24851) Map a Stage ID to it's Associated Job ID in UI

2018-10-09 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24851?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-24851. --- Resolution: Fixed Fix Version/s: 3.0.0 > Map a Stage ID to it's Associated Job ID in

[jira] [Resolved] (SPARK-25535) Work around bad error checking in commons-crypto

2018-10-09 Thread Imran Rashid (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25535?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Imran Rashid resolved SPARK-25535. -- Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 22557

[jira] [Commented] (SPARK-22388) Limit push down

2018-10-09 Thread Aleksey Zinoviev (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22388?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16643478#comment-16643478 ] Aleksey Zinoviev commented on SPARK-22388: -- [~cloud_fan] Could you please add any description

[jira] [Created] (SPARK-25687) A dataset can store a column as sequence of Vectors but not directly vectors

2018-10-09 Thread Francisco Orchard (JIRA)
Francisco Orchard created SPARK-25687: - Summary: A dataset can store a column as sequence of Vectors but not directly vectors Key: SPARK-25687 URL: https://issues.apache.org/jira/browse/SPARK-25687

[jira] [Commented] (SPARK-25344) Break large tests.py files into smaller files

2018-10-09 Thread Bryan Cutler (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25344?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16643762#comment-16643762 ] Bryan Cutler commented on SPARK-25344: -- No I don't have strong feelings, my only preference was to

[jira] [Commented] (SPARK-25559) Just remove the unsupported predicates in Parquet

2018-10-09 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25559?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16643765#comment-16643765 ] Apache Spark commented on SPARK-25559: -- User 'gatorsmile' has created a pull request for this

[jira] [Commented] (SPARK-25559) Just remove the unsupported predicates in Parquet

2018-10-09 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25559?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16643761#comment-16643761 ] Apache Spark commented on SPARK-25559: -- User 'gatorsmile' has created a pull request for this

[jira] [Commented] (SPARK-25692) Flaky test: ChunkFetchIntegrationSuite

2018-10-09 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25692?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16644162#comment-16644162 ] Shixiong Zhu commented on SPARK-25692: -- It may be caused by

[jira] [Created] (SPARK-25694) URL.setURLStreamHandlerFactory causing incompatible HttpURLConnection

2018-10-09 Thread Bo Yang (JIRA)
Bo Yang created SPARK-25694: --- Summary: URL.setURLStreamHandlerFactory causing incompatible HttpURLConnection Key: SPARK-25694 URL: https://issues.apache.org/jira/browse/SPARK-25694 Project: Spark

[jira] [Updated] (SPARK-25694) URL.setURLStreamHandlerFactory causing incompatible HttpURLConnection

2018-10-09 Thread Bo Yang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25694?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bo Yang updated SPARK-25694: Description: URL.setURLStreamHandlerFactory() in SharedState causes URL.openConnection() returns 

[jira] [Updated] (SPARK-25694) URL.setURLStreamHandlerFactory causing incompatible HttpURLConnection issue

2018-10-09 Thread Bo Yang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25694?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bo Yang updated SPARK-25694: Description: URL.setURLStreamHandlerFactory() in SharedState causes URL.openConnection() returns 

[jira] [Commented] (SPARK-25682) Docker images generated from dev build and from dist tarball are different

2018-10-09 Thread Yinan Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25682?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16644198#comment-16644198 ] Yinan Li commented on SPARK-25682: -- Cool, thanks! > Docker images generated from dev build and from

[jira] [Commented] (SPARK-25682) Docker images generated from dev build and from dist tarball are different

2018-10-09 Thread Yinan Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25682?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16644157#comment-16644157 ] Yinan Li commented on SPARK-25682: -- That looks like to me the only difference. {{bin}}, {{sbin}}, and

[jira] [Updated] (SPARK-25694) URL.setURLStreamHandlerFactory causing incompatible HttpURLConnection

2018-10-09 Thread Bo Yang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25694?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bo Yang updated SPARK-25694: Description: URL.setURLStreamHandlerFactory() in SharedState causes URL.openConnection() returns 

[jira] [Commented] (SPARK-25682) Docker images generated from dev build and from dist tarball are different

2018-10-09 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25682?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16644194#comment-16644194 ] Marcelo Vanzin commented on SPARK-25682: I was busy with other things but I can send a patch

[jira] [Created] (SPARK-25692) Flaky test: ChunkFetchIntegrationSuite

2018-10-09 Thread Shixiong Zhu (JIRA)
Shixiong Zhu created SPARK-25692: Summary: Flaky test: ChunkFetchIntegrationSuite Key: SPARK-25692 URL: https://issues.apache.org/jira/browse/SPARK-25692 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-25692) Flaky test: ChunkFetchIntegrationSuite

2018-10-09 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25692?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-25692: Affects Version/s: (was: 2.4.0) 3.0.0 > Flaky test: ChunkFetchIntegrationSuite

[jira] [Updated] (SPARK-25694) URL.setURLStreamHandlerFactory causing incompatible HttpURLConnection issue

2018-10-09 Thread Bo Yang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25694?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bo Yang updated SPARK-25694: Priority: Minor (was: Major) > URL.setURLStreamHandlerFactory causing incompatible HttpURLConnection

[jira] [Updated] (SPARK-25694) URL.setURLStreamHandlerFactory causing incompatible HttpURLConnection

2018-10-09 Thread Bo Yang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25694?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bo Yang updated SPARK-25694: Description: URL.setURLStreamHandlerFactory() in SharedState causes URL.openConnection() returns 

[jira] [Updated] (SPARK-25694) URL.setURLStreamHandlerFactory causing incompatible HttpURLConnection issue

2018-10-09 Thread Bo Yang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25694?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bo Yang updated SPARK-25694: Description: URL.setURLStreamHandlerFactory() in SharedState causes URL.openConnection() returns 

[jira] [Commented] (SPARK-25690) Analyzer rule "HandleNullInputsForUDF" does not stabilize and can be applied infinitely

2018-10-09 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25690?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16644273#comment-16644273 ] Xiao Li commented on SPARK-25690: - The changes made in https://issues.apache.org/jira/browse/SPARK-25044

[jira] [Commented] (SPARK-19256) Hive bucketing support

2018-10-09 Thread Shay Elbaz (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19256?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16643828#comment-16643828 ] Shay Elbaz commented on SPARK-19256: +1 [~tejasp] is this still under progress? > Hive bucketing

[jira] [Updated] (SPARK-25688) org.apache.spark.sql.FileBasedDataSourceSuite never pass

2018-10-09 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25688?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-25688: Priority: Blocker (was: Major) > org.apache.spark.sql.FileBasedDataSourceSuite never pass >

[jira] [Updated] (SPARK-25688) Potential resource leak in ORC

2018-10-09 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25688?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-25688: Summary: Potential resource leak in ORC (was: org.apache.spark.sql.FileBasedDataSourceSuite never pass)

[jira] [Commented] (SPARK-25688) Potential resource leak in ORC

2018-10-09 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16643926#comment-16643926 ] Dongjoon Hyun commented on SPARK-25688: --- [~smilegator] . This is a duplication of SPARK-23390.

[jira] [Created] (SPARK-25689) Move token renewal logic to driver in yarn-client mode

2018-10-09 Thread Marcelo Vanzin (JIRA)
Marcelo Vanzin created SPARK-25689: -- Summary: Move token renewal logic to driver in yarn-client mode Key: SPARK-25689 URL: https://issues.apache.org/jira/browse/SPARK-25689 Project: Spark

[jira] [Resolved] (SPARK-25688) Potential resource leak in ORC

2018-10-09 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25688?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-25688. --- Resolution: Duplicate Sorry, but let's gether the same error report into one JIRA. >

[jira] [Updated] (SPARK-23390) Flaky test: FileBasedDataSourceSuite

2018-10-09 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23390?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-23390: -- Summary: Flaky test: FileBasedDataSourceSuite (was: Flaky test: FileBasedDataSourceSuite in

[jira] [Commented] (SPARK-25493) CRLF Line Separators don't work in multiline CSVs

2018-10-09 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25493?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16643805#comment-16643805 ] Apache Spark commented on SPARK-25493: -- User 'justinuang' has created a pull request for this

[jira] [Created] (SPARK-25688) org.apache.spark.sql.FileBasedDataSourceSuite never pass

2018-10-09 Thread Xiao Li (JIRA)
Xiao Li created SPARK-25688: --- Summary: org.apache.spark.sql.FileBasedDataSourceSuite never pass Key: SPARK-25688 URL: https://issues.apache.org/jira/browse/SPARK-25688 Project: Spark Issue Type:

[jira] [Updated] (SPARK-25688) org.apache.spark.sql.FileBasedDataSourceSuite never pass

2018-10-09 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25688?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-25688: Priority: Critical (was: Blocker) > org.apache.spark.sql.FileBasedDataSourceSuite never pass >

[jira] [Commented] (SPARK-25688) Potential resource leak in ORC

2018-10-09 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16643870#comment-16643870 ] Xiao Li commented on SPARK-25688: - It sounds like ORC still has a resource leak even after the latest

[jira] [Commented] (SPARK-24523) InterruptedException when closing SparkContext

2018-10-09 Thread Ankur Gupta (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24523?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16643936#comment-16643936 ] Ankur Gupta commented on SPARK-24523: - Hi [~umayr_nuna] - just checking if you had a chance to try

[jira] [Updated] (SPARK-25688) org.apache.spark.sql.FileBasedDataSourceSuite never pass

2018-10-09 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25688?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-25688: Description:

[jira] [Commented] (SPARK-25688) Potential resource leak in ORC

2018-10-09 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16643910#comment-16643910 ] Xiao Li commented on SPARK-25688: -

[jira] [Updated] (SPARK-23390) Flaky test: FileBasedDataSourceSuite

2018-10-09 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23390?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-23390: -- Priority: Critical (was: Major) > Flaky test: FileBasedDataSourceSuite >

[jira] [Commented] (SPARK-23390) Flaky test: FileBasedDataSourceSuite

2018-10-09 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23390?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16644036#comment-16644036 ] Shixiong Zhu commented on SPARK-23390: -- I didn't look at parquet. It may have a similar issue. >

[jira] [Commented] (SPARK-23390) Flaky test: FileBasedDataSourceSuite

2018-10-09 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23390?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16643960#comment-16643960 ] Dongjoon Hyun commented on SPARK-23390: --- As reported in SPARK-25688 by [~smilegator] , all recent

[jira] [Created] (SPARK-25690) Analyzer rule "HandleNullInputsForUDF" does not stabilize and can be applied infinitely

2018-10-09 Thread Maryann Xue (JIRA)
Maryann Xue created SPARK-25690: --- Summary: Analyzer rule "HandleNullInputsForUDF" does not stabilize and can be applied infinitely Key: SPARK-25690 URL: https://issues.apache.org/jira/browse/SPARK-25690

[jira] [Updated] (SPARK-25690) Analyzer rule "HandleNullInputsForUDF" does not stabilize and can be applied infinitely

2018-10-09 Thread Maryann Xue (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25690?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maryann Xue updated SPARK-25690: Description: This was fixed in SPARK-24891 and was then broken by SPARK-25044. The tests added

[jira] [Created] (SPARK-25691) Analyzer rule "AliasViewChild" does not stabilize

2018-10-09 Thread Maryann Xue (JIRA)
Maryann Xue created SPARK-25691: --- Summary: Analyzer rule "AliasViewChild" does not stabilize Key: SPARK-25691 URL: https://issues.apache.org/jira/browse/SPARK-25691 Project: Spark Issue Type:

[jira] [Updated] (SPARK-25650) Make analyzer rules used in once-policy idempotent

2018-10-09 Thread Maryann Xue (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25650?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maryann Xue updated SPARK-25650: Description: Rules like {{HandleNullInputsForUDF}} 

[jira] [Updated] (SPARK-23390) Flaky test: FileBasedDataSourceSuite

2018-10-09 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23390?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-23390: -- Description: *RECENT HISTORY*

[jira] [Updated] (SPARK-23390) Flaky test: FileBasedDataSourceSuite

2018-10-09 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23390?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-23390: -- Affects Version/s: 2.4.0 > Flaky test: FileBasedDataSourceSuite >

[jira] [Updated] (SPARK-25690) Analyzer rule "HandleNullInputsForUDF" does not stabilize and can be applied infinitely

2018-10-09 Thread Maryann Xue (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25690?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maryann Xue updated SPARK-25690: Issue Type: Bug (was: Sub-task) Parent: (was: SPARK-14220) > Analyzer rule

[jira] [Updated] (SPARK-25690) Analyzer rule "HandleNullInputsForUDF" does not stabilize and can be applied infinitely

2018-10-09 Thread Maryann Xue (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25690?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maryann Xue updated SPARK-25690: Issue Type: Sub-task (was: Bug) Parent: SPARK-25650 > Analyzer rule

[jira] [Commented] (SPARK-23390) Flaky test: FileBasedDataSourceSuite

2018-10-09 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23390?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16644034#comment-16644034 ] Shixiong Zhu commented on SPARK-23390: -- I think the issue is probably in orc. Any exception

[jira] [Assigned] (SPARK-25690) Analyzer rule "HandleNullInputsForUDF" does not stabilize and can be applied infinitely

2018-10-09 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25690?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen reassigned SPARK-25690: - Assignee: (was: Sean Owen) Fix Version/s: (was: 2.4.0) > Analyzer rule

[jira] [Commented] (SPARK-25690) Analyzer rule "HandleNullInputsForUDF" does not stabilize and can be applied infinitely

2018-10-09 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25690?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16644117#comment-16644117 ] Sean Owen commented on SPARK-25690: --- I'm unclear what the particular problem is here. Is there a test