[jira] [Resolved] (SPARK-24833) Allow specifying Kubernetes host name aliases in the pod specs

2018-10-31 Thread Rob Vesse (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24833?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rob Vesse resolved SPARK-24833. --- Resolution: Won't Fix As discussed in PR the pod template feature provides the ability to do this

[jira] [Commented] (SPARK-25861) Remove unused refreshInterval parameter from the headerSparkPage method.

2018-10-31 Thread shahid (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25861?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16670222#comment-16670222 ] shahid commented on SPARK-25861: Sorry [~srowen], next time I will take care. > Remove unused

[jira] [Created] (SPARK-25895) No test to compare Zstd and Lz4 Compression Algorithm

2018-10-31 Thread Udbhav Agrawal (JIRA)
Udbhav Agrawal created SPARK-25895: -- Summary: No test to compare Zstd and Lz4 Compression Algorithm Key: SPARK-25895 URL: https://issues.apache.org/jira/browse/SPARK-25895 Project: Spark

[jira] [Updated] (SPARK-25861) Remove unused refreshInterval parameter from the headerSparkPage method.

2018-10-31 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25861?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-25861: -- Priority: Trivial (was: Minor) Issue Type: Improvement (was: Bug) This isn't a bug, even if

[jira] [Created] (SPARK-25896) Accumulator should only be updated once for each successful task in shuffle map stage

2018-10-31 Thread Wenchen Fan (JIRA)
Wenchen Fan created SPARK-25896: --- Summary: Accumulator should only be updated once for each successful task in shuffle map stage Key: SPARK-25896 URL: https://issues.apache.org/jira/browse/SPARK-25896

[jira] [Assigned] (SPARK-25894) Include a count of the number of physical columns read for a columnar data source in the metadata of FileSourceScanExec

2018-10-31 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25894?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25894: Assignee: (was: Apache Spark) > Include a count of the number of physical columns

[jira] [Assigned] (SPARK-25895) No test to compare Zstd and Lz4 Compression Algorithm

2018-10-31 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25895?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25895: Assignee: (was: Apache Spark) > No test to compare Zstd and Lz4 Compression

[jira] [Assigned] (SPARK-25895) No test to compare Zstd and Lz4 Compression Algorithm

2018-10-31 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25895?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25895: Assignee: Apache Spark > No test to compare Zstd and Lz4 Compression Algorithm >

[jira] [Commented] (SPARK-25895) No test to compare Zstd and Lz4 Compression Algorithm

2018-10-31 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25895?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16670113#comment-16670113 ] Apache Spark commented on SPARK-25895: -- User 'Udbhav30' has created a pull request for this issue:

[jira] [Assigned] (SPARK-25894) Include a count of the number of physical columns read for a columnar data source in the metadata of FileSourceScanExec

2018-10-31 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25894?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25894: Assignee: Apache Spark > Include a count of the number of physical columns read for a

[jira] [Commented] (SPARK-25894) Include a count of the number of physical columns read for a columnar data source in the metadata of FileSourceScanExec

2018-10-31 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25894?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16670068#comment-16670068 ] Apache Spark commented on SPARK-25894: -- User 'mallman' has created a pull request for this issue:

[jira] [Commented] (SPARK-25896) Accumulator should only be updated once for each successful task in shuffle map stage

2018-10-31 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25896?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16670246#comment-16670246 ] Apache Spark commented on SPARK-25896: -- User 'cloud-fan' has created a pull request for this issue:

[jira] [Assigned] (SPARK-25896) Accumulator should only be updated once for each successful task in shuffle map stage

2018-10-31 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25896?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25896: Assignee: Apache Spark (was: Wenchen Fan) > Accumulator should only be updated once for

[jira] [Assigned] (SPARK-25896) Accumulator should only be updated once for each successful task in shuffle map stage

2018-10-31 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25896?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25896: Assignee: Wenchen Fan (was: Apache Spark) > Accumulator should only be updated once for

[jira] [Resolved] (SPARK-25893) Show a directional error message for unsupported Hive Metastore versions

2018-10-31 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25893?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-25893. --- Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 22902

[jira] [Assigned] (SPARK-25893) Show a directional error message for unsupported Hive Metastore versions

2018-10-31 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25893?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-25893: - Assignee: Dongjoon Hyun > Show a directional error message for unsupported Hive

[jira] [Assigned] (SPARK-25897) Cannot run k8s integration tests in sbt

2018-10-31 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25897?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25897: Assignee: Apache Spark > Cannot run k8s integration tests in sbt >

[jira] [Commented] (SPARK-25897) Cannot run k8s integration tests in sbt

2018-10-31 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25897?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16670429#comment-16670429 ] Apache Spark commented on SPARK-25897: -- User 'vanzin' has created a pull request for this issue:

[jira] [Assigned] (SPARK-25899) Flaky test: CoarseGrainedSchedulerBackendSuite.compute max number of concurrent tasks can be launched

2018-10-31 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25899?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25899: Assignee: Shixiong Zhu (was: Apache Spark) > Flaky test:

[jira] [Commented] (SPARK-25899) Flaky test: CoarseGrainedSchedulerBackendSuite.compute max number of concurrent tasks can be launched

2018-10-31 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25899?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16670443#comment-16670443 ] Apache Spark commented on SPARK-25899: -- User 'zsxwing' has created a pull request for this issue:

[jira] [Assigned] (SPARK-25899) Flaky test: CoarseGrainedSchedulerBackendSuite.compute max number of concurrent tasks can be launched

2018-10-31 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25899?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25899: Assignee: Apache Spark (was: Shixiong Zhu) > Flaky test:

[jira] [Updated] (SPARK-25899) Flaky test: CoarseGrainedSchedulerBackendSuite.compute max number of concurrent tasks can be launched

2018-10-31 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25899?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-25899: - Issue Type: Test (was: Documentation) > Flaky test: CoarseGrainedSchedulerBackendSuite.compute

[jira] [Created] (SPARK-25900) If the When the page number is more than the total page size, then fall back to the first page

2018-10-31 Thread ABHISHEK KUMAR GUPTA (JIRA)
ABHISHEK KUMAR GUPTA created SPARK-25900: Summary: If the When the page number is more than the total page size, then fall back to the first page Key: SPARK-25900 URL:

[jira] [Reopened] (SPARK-25206) wrong records are returned when Hive metastore schema and parquet schema are in different letter cases

2018-10-31 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25206?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reopened SPARK-25206: - > wrong records are returned when Hive metastore schema and parquet schema are > in different

[jira] [Created] (SPARK-25897) Cannot run k8s integration tests in sbt

2018-10-31 Thread Marcelo Vanzin (JIRA)
Marcelo Vanzin created SPARK-25897: -- Summary: Cannot run k8s integration tests in sbt Key: SPARK-25897 URL: https://issues.apache.org/jira/browse/SPARK-25897 Project: Spark Issue Type:

[jira] [Created] (SPARK-25899) Flaky test: CoarseGrainedSchedulerBackendSuite.compute max number of concurrent tasks can be launched

2018-10-31 Thread Shixiong Zhu (JIRA)
Shixiong Zhu created SPARK-25899: Summary: Flaky test: CoarseGrainedSchedulerBackendSuite.compute max number of concurrent tasks can be launched Key: SPARK-25899 URL:

[jira] [Resolved] (SPARK-24746) AWS S3 301 Moved Permanently error message even after setting fs.s3a.endpoint for bucket in Mumbai region.

2018-10-31 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24746?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-24746. Resolution: Information Provided In general, please use the mailing list for questions.

[jira] [Updated] (SPARK-25898) pandas_udf not catching errors early on

2018-10-31 Thread Kaushalya (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25898?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kaushalya updated SPARK-25898: -- Summary: pandas_udf not catching errors early on (was: pandas_udf ) > pandas_udf not catching errors

[jira] [Commented] (SPARK-25899) Flaky test: CoarseGrainedSchedulerBackendSuite.compute max number of concurrent tasks can be launched

2018-10-31 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25899?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16670445#comment-16670445 ] Apache Spark commented on SPARK-25899: -- User 'zsxwing' has created a pull request for this issue:

[jira] [Assigned] (SPARK-25815) Kerberos Support in Kubernetes resource manager (Client Mode)

2018-10-31 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25815?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25815: Assignee: (was: Apache Spark) > Kerberos Support in Kubernetes resource manager

[jira] [Updated] (SPARK-25901) Barrier mode spawns a bunch of threads that get collected on gc

2018-10-31 Thread yogesh garg (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25901?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] yogesh garg updated SPARK-25901: Attachment: Screen Shot 2018-10-31 at 11.57.25 AM.png Screen Shot 2018-10-31 at

[jira] [Created] (SPARK-25901) Barrier mode spawns a bunch of threads that get collected on gc

2018-10-31 Thread yogesh garg (JIRA)
yogesh garg created SPARK-25901: --- Summary: Barrier mode spawns a bunch of threads that get collected on gc Key: SPARK-25901 URL: https://issues.apache.org/jira/browse/SPARK-25901 Project: Spark

[jira] [Commented] (SPARK-25901) Barrier mode spawns a bunch of threads that get collected on gc

2018-10-31 Thread yogesh garg (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25901?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16670573#comment-16670573 ] yogesh garg commented on SPARK-25901: - I am working on this task. > Barrier mode spawns a bunch of

[jira] [Comment Edited] (SPARK-25901) Barrier mode spawns a bunch of threads that get collected on gc

2018-10-31 Thread yogesh garg (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25901?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16670573#comment-16670573 ] yogesh garg edited comment on SPARK-25901 at 10/31/18 7:06 PM: --- I am

[jira] [Resolved] (SPARK-25206) wrong records are returned when Hive metastore schema and parquet schema are in different letter cases

2018-10-31 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25206?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-25206. - Resolution: Fixed Fix Version/s: 2.4.0 > wrong records are returned when Hive metastore

[jira] [Commented] (SPARK-25206) wrong records are returned when Hive metastore schema and parquet schema are in different letter cases

2018-10-31 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16670339#comment-16670339 ] Wenchen Fan commented on SPARK-25206: - marked it as resolved in 2.4.0, since the test is already in

[jira] [Assigned] (SPARK-25897) Cannot run k8s integration tests in sbt

2018-10-31 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25897?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25897: Assignee: (was: Apache Spark) > Cannot run k8s integration tests in sbt >

[jira] [Created] (SPARK-25902) Support for dates with milliseconds in Arrow bindings

2018-10-31 Thread Javier Luraschi (JIRA)
Javier Luraschi created SPARK-25902: --- Summary: Support for dates with milliseconds in Arrow bindings Key: SPARK-25902 URL: https://issues.apache.org/jira/browse/SPARK-25902 Project: Spark

[jira] [Commented] (SPARK-25900) If the When the page number is more than the total page size, then fall back to the first page

2018-10-31 Thread shahid (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25900?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16670531#comment-16670531 ] shahid commented on SPARK-25900: Thanks. I am working on it > If the When the page number is more than

[jira] [Updated] (SPARK-25901) Barrier mode spawns a bunch of threads that get collected on gc

2018-10-31 Thread yogesh garg (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25901?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] yogesh garg updated SPARK-25901: Description: After a barrier job is terminated (successfully or interrupted), the accompanying

[jira] [Assigned] (SPARK-25901) Barrier mode spawns a bunch of threads that get collected on gc

2018-10-31 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25901?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25901: Assignee: (was: Apache Spark) > Barrier mode spawns a bunch of threads that get

[jira] [Created] (SPARK-25898) pandas_udf

2018-10-31 Thread Kaushalya (JIRA)
Kaushalya created SPARK-25898: - Summary: pandas_udf Key: SPARK-25898 URL: https://issues.apache.org/jira/browse/SPARK-25898 Project: Spark Issue Type: Bug Components: SQL Affects

[jira] [Resolved] (SPARK-23781) Merge YARN and Mesos token renewal code

2018-10-31 Thread Imran Rashid (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23781?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Imran Rashid resolved SPARK-23781. -- Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 22624

[jira] [Assigned] (SPARK-23781) Merge YARN and Mesos token renewal code

2018-10-31 Thread Imran Rashid (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23781?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Imran Rashid reassigned SPARK-23781: Assignee: Marcelo Vanzin > Merge YARN and Mesos token renewal code >

[jira] [Commented] (SPARK-25901) Barrier mode spawns a bunch of threads that get collected on gc

2018-10-31 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25901?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16670583#comment-16670583 ] Apache Spark commented on SPARK-25901: -- User 'yogeshg' has created a pull request for this issue:

[jira] [Assigned] (SPARK-25901) Barrier mode spawns a bunch of threads that get collected on gc

2018-10-31 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25901?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25901: Assignee: Apache Spark > Barrier mode spawns a bunch of threads that get collected on gc

[jira] [Assigned] (SPARK-25891) Upgrade to Py4J 0.10.8.1

2018-10-31 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25891?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-25891: - Assignee: Dongjoon Hyun > Upgrade to Py4J 0.10.8.1 > > >

[jira] [Resolved] (SPARK-25891) Upgrade to Py4J 0.10.8.1

2018-10-31 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25891?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-25891. --- Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 22901

[jira] [Resolved] (SPARK-25855) Don't use Erasure Coding for event log files

2018-10-31 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25855?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-25855. Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 22881

[jira] [Assigned] (SPARK-25855) Don't use Erasure Coding for event log files

2018-10-31 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25855?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin reassigned SPARK-25855: -- Assignee: Imran Rashid > Don't use Erasure Coding for event log files >

[jira] [Resolved] (SPARK-25860) Replace Literal(null, _) with FalseLiteral whenever possible

2018-10-31 Thread DB Tsai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25860?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] DB Tsai resolved SPARK-25860. - Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 22857

[jira] [Assigned] (SPARK-25815) Kerberos Support in Kubernetes resource manager (Client Mode)

2018-10-31 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25815?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25815: Assignee: Apache Spark > Kerberos Support in Kubernetes resource manager (Client Mode) >

[jira] [Commented] (SPARK-25815) Kerberos Support in Kubernetes resource manager (Client Mode)

2018-10-31 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25815?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16670565#comment-16670565 ] Apache Spark commented on SPARK-25815: -- User 'vanzin' has created a pull request for this issue:

[jira] [Commented] (SPARK-25900) If the When the page number is more than the total page size, then fall back to the first page

2018-10-31 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25900?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16670684#comment-16670684 ] Apache Spark commented on SPARK-25900: -- User 'shahidki31' has created a pull request for this

[jira] [Created] (SPARK-25905) BlockManager should expose getRemoteManagedBuffer to avoid creating bytebuffers

2018-10-31 Thread Imran Rashid (JIRA)
Imran Rashid created SPARK-25905: Summary: BlockManager should expose getRemoteManagedBuffer to avoid creating bytebuffers Key: SPARK-25905 URL: https://issues.apache.org/jira/browse/SPARK-25905

[jira] [Commented] (SPARK-25855) Don't use Erasure Coding for event log files

2018-10-31 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25855?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16670738#comment-16670738 ] Apache Spark commented on SPARK-25855: -- User 'squito' has created a pull request for this issue:

[jira] [Commented] (SPARK-24758) Create table wants to use /user/hive/warehouse in clean clone

2018-10-31 Thread Bruce Robbins (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24758?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16670665#comment-16670665 ] Bruce Robbins commented on SPARK-24758: --- This issue was introduced by commit 

[jira] [Created] (SPARK-25903) Flaky test: BarrierTaskContextSuite.throw exception on barrier() call timeout

2018-10-31 Thread Marcelo Vanzin (JIRA)
Marcelo Vanzin created SPARK-25903: -- Summary: Flaky test: BarrierTaskContextSuite.throw exception on barrier() call timeout Key: SPARK-25903 URL: https://issues.apache.org/jira/browse/SPARK-25903

[jira] [Commented] (SPARK-25825) Kerberos Support for Long Running Jobs in Kubernetes

2018-10-31 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16670715#comment-16670715 ] Apache Spark commented on SPARK-25825: -- User 'ifilonenko' has created a pull request for this

[jira] [Assigned] (SPARK-25825) Kerberos Support for Long Running Jobs in Kubernetes

2018-10-31 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25825?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25825: Assignee: (was: Apache Spark) > Kerberos Support for Long Running Jobs in Kubernetes

[jira] [Assigned] (SPARK-25825) Kerberos Support for Long Running Jobs in Kubernetes

2018-10-31 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25825?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25825: Assignee: Apache Spark > Kerberos Support for Long Running Jobs in Kubernetes >

[jira] [Commented] (SPARK-25825) Kerberos Support for Long Running Jobs in Kubernetes

2018-10-31 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16670719#comment-16670719 ] Apache Spark commented on SPARK-25825: -- User 'ifilonenko' has created a pull request for this

[jira] [Commented] (SPARK-25900) If the When the page number is more than the total page size, then fall back to the first page

2018-10-31 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25900?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16670682#comment-16670682 ] Apache Spark commented on SPARK-25900: -- User 'shahidki31' has created a pull request for this

[jira] [Assigned] (SPARK-25900) If the When the page number is more than the total page size, then fall back to the first page

2018-10-31 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25900?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25900: Assignee: Apache Spark > If the When the page number is more than the total page size,

[jira] [Assigned] (SPARK-25900) If the When the page number is more than the total page size, then fall back to the first page

2018-10-31 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25900?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25900: Assignee: (was: Apache Spark) > If the When the page number is more than the total

[jira] [Commented] (SPARK-25827) Replicating a block > 2gb with encryption fails

2018-10-31 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25827?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16670773#comment-16670773 ] Apache Spark commented on SPARK-25827: -- User 'squito' has created a pull request for this issue:

[jira] [Issue Comment Deleted] (SPARK-25855) Don't use Erasure Coding for event log files

2018-10-31 Thread Imran Rashid (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25855?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Imran Rashid updated SPARK-25855: - Comment: was deleted (was: User 'squito' has created a pull request for this issue:

[jira] [Updated] (SPARK-25827) Replicating a block > 2gb with encryption fails

2018-10-31 Thread Imran Rashid (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25827?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Imran Rashid updated SPARK-25827: - Description: There are a couple of issues with replicating & remote reads of large encrypted

[jira] [Commented] (SPARK-25827) Replicating a block > 2gb with encryption fails

2018-10-31 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25827?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16670777#comment-16670777 ] Apache Spark commented on SPARK-25827: -- User 'squito' has created a pull request for this issue:

[jira] [Commented] (SPARK-25827) Replicating a block > 2gb with encryption fails

2018-10-31 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25827?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16670776#comment-16670776 ] Apache Spark commented on SPARK-25827: -- User 'squito' has created a pull request for this issue:

[jira] [Commented] (SPARK-25904) Avoid allocating arrays too large for JVMs

2018-10-31 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16670695#comment-16670695 ] Apache Spark commented on SPARK-25904: -- User 'squito' has created a pull request for this issue:

[jira] [Assigned] (SPARK-25904) Avoid allocating arrays too large for JVMs

2018-10-31 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25904?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25904: Assignee: (was: Apache Spark) > Avoid allocating arrays too large for JVMs >

[jira] [Assigned] (SPARK-25904) Avoid allocating arrays too large for JVMs

2018-10-31 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25904?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25904: Assignee: Apache Spark > Avoid allocating arrays too large for JVMs >

[jira] [Created] (SPARK-25904) Avoid allocating arrays too large for JVMs

2018-10-31 Thread Imran Rashid (JIRA)
Imran Rashid created SPARK-25904: Summary: Avoid allocating arrays too large for JVMs Key: SPARK-25904 URL: https://issues.apache.org/jira/browse/SPARK-25904 Project: Spark Issue Type:

[jira] [Commented] (SPARK-25892) AttributeReference.withMetadata method should have return type AttributeReference

2018-10-31 Thread kevin yu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25892?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16670735#comment-16670735 ] kevin yu commented on SPARK-25892: -- I am looking into this now. Kevin >

[jira] [Resolved] (SPARK-25899) Flaky test: CoarseGrainedSchedulerBackendSuite.compute max number of concurrent tasks can be launched

2018-10-31 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25899?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-25899. - Resolution: Fixed Fix Version/s: 3.0.0 2.4.1 > Flaky test:

[jira] [Updated] (SPARK-25900) When the the page number is more than the total page size, then fall back to the first page

2018-10-31 Thread shahid (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25900?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] shahid updated SPARK-25900: --- Summary: When the the page number is more than the total page size, then fall back to the first page

[jira] [Updated] (SPARK-25900) If the the page number is more than the total page size, then fall back to the first page

2018-10-31 Thread shahid (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25900?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] shahid updated SPARK-25900: --- Summary: If the the page number is more than the total page size, then fall back to the first page (was:

[jira] [Assigned] (SPARK-25902) Support for dates with milliseconds in Arrow bindings

2018-10-31 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25902?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25902: Assignee: (was: Apache Spark) > Support for dates with milliseconds in Arrow

[jira] [Assigned] (SPARK-25902) Support for dates with milliseconds in Arrow bindings

2018-10-31 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25902?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25902: Assignee: Apache Spark > Support for dates with milliseconds in Arrow bindings >

[jira] [Commented] (SPARK-25902) Support for dates with milliseconds in Arrow bindings

2018-10-31 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25902?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16670600#comment-16670600 ] Apache Spark commented on SPARK-25902: -- User 'javierluraschi' has created a pull request for this

[jira] [Commented] (SPARK-20568) Delete files after processing in structured streaming

2018-10-31 Thread Jungtaek Lim (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20568?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16670904#comment-16670904 ] Jungtaek Lim commented on SPARK-20568: -- [~zsxwing] Yes I'll see the time slot and take a look at

[jira] [Commented] (SPARK-20568) Delete files after processing in structured streaming

2018-10-31 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20568?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16670897#comment-16670897 ] Shixiong Zhu commented on SPARK-20568: -- [~kabhwan] I think this is pretty useful. Do you have time

[jira] [Resolved] (SPARK-25672) Inferring schema from CSV string literal

2018-10-31 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25672?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-25672. -- Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 22666

[jira] [Commented] (SPARK-25902) Support for dates with milliseconds in Arrow bindings

2018-10-31 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25902?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16671000#comment-16671000 ] Apache Spark commented on SPARK-25902: -- User 'kevinyu98' has created a pull request for this issue:

[jira] [Commented] (SPARK-25902) Support for dates with milliseconds in Arrow bindings

2018-10-31 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25902?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16671001#comment-16671001 ] Apache Spark commented on SPARK-25902: -- User 'kevinyu98' has created a pull request for this issue:

[jira] [Commented] (SPARK-25902) Support for dates with milliseconds in Arrow bindings

2018-10-31 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25902?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16671002#comment-16671002 ] Apache Spark commented on SPARK-25902: -- User 'kevinyu98' has created a pull request for this issue:

[jira] [Assigned] (SPARK-25672) Inferring schema from CSV string literal

2018-10-31 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25672?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-25672: Assignee: Maxim Gekk > Inferring schema from CSV string literal >

[jira] [Resolved] (SPARK-25884) Add TBLPROPERTIES and COMMENT, and use LOCATION when SHOW CREATE TABLE.

2018-10-31 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25884?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-25884. - Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 22892

[jira] [Created] (SPARK-25894) Include a count of the number of physical columns read for a columnar data source in the metadata of FileSourceScanExec

2018-10-31 Thread Michael Allman (JIRA)
Michael Allman created SPARK-25894: -- Summary: Include a count of the number of physical columns read for a columnar data source in the metadata of FileSourceScanExec Key: SPARK-25894 URL:

[jira] [Assigned] (SPARK-25887) Allow specifying Kubernetes context to use

2018-10-31 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25887?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25887: Assignee: (was: Apache Spark) > Allow specifying Kubernetes context to use >

[jira] [Assigned] (SPARK-25887) Allow specifying Kubernetes context to use

2018-10-31 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25887?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25887: Assignee: Apache Spark > Allow specifying Kubernetes context to use >

[jira] [Commented] (SPARK-25887) Allow specifying Kubernetes context to use

2018-10-31 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25887?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16670005#comment-16670005 ] Apache Spark commented on SPARK-25887: -- User 'rvesse' has created a pull request for this issue:

[jira] [Resolved] (SPARK-25886) Improve error message of `FailureSafeParser` and `from_avro` in FAILFAST mode

2018-10-31 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25886?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-25886. -- Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 22895

[jira] [Assigned] (SPARK-25886) Improve error message of `FailureSafeParser` and `from_avro` in FAILFAST mode

2018-10-31 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25886?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-25886: Assignee: Gengliang Wang > Improve error message of `FailureSafeParser` and `from_avro`

[jira] [Assigned] (SPARK-25618) KafkaContinuousSourceStressForDontFailOnDataLossSuite: stress test for failOnDataLoss=false 1 min 1 sec

2018-10-31 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25618?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25618: Assignee: (was: Apache Spark) >

[jira] [Commented] (SPARK-25618) KafkaContinuousSourceStressForDontFailOnDataLossSuite: stress test for failOnDataLoss=false 1 min 1 sec

2018-10-31 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25618?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16669672#comment-16669672 ] Apache Spark commented on SPARK-25618: -- User 'dilipbiswal' has created a pull request for this

[jira] [Assigned] (SPARK-25618) KafkaContinuousSourceStressForDontFailOnDataLossSuite: stress test for failOnDataLoss=false 1 min 1 sec

2018-10-31 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25618?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25618: Assignee: Apache Spark > KafkaContinuousSourceStressForDontFailOnDataLossSuite: stress

[jira] [Resolved] (SPARK-25883) Override method `prettyName` in `from_avro`/`to_avro`

2018-10-31 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25883?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-25883. - Resolution: Fixed Assignee: Gengliang Wang Fix Version/s: 3.0.0 > Override method

[jira] [Created] (SPARK-25891) Upgrade to Py4J 0.10.8.1

2018-10-31 Thread Dongjoon Hyun (JIRA)
Dongjoon Hyun created SPARK-25891: - Summary: Upgrade to Py4J 0.10.8.1 Key: SPARK-25891 URL: https://issues.apache.org/jira/browse/SPARK-25891 Project: Spark Issue Type: Improvement

  1   2   >