[jira] [Commented] (SPARK-28895) Spark client process is unable to upload jars to hdfs while using ConfigMap not HADOOP_CONF_DIR

2019-09-03 Thread Stavros Kontopoulos (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28895?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16921394#comment-16921394 ] Stavros Kontopoulos commented on SPARK-28895: - I changed the version to Spark 3.0.0 as this

[jira] [Updated] (SPARK-28896) Spark client process is unable to upload jars to hdfs while using ConfigMap not HADOOP_CONF_DIR

2019-09-03 Thread Stavros Kontopoulos (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28896?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stavros Kontopoulos updated SPARK-28896: Affects Version/s: (was: 2.4.3) 3.0.0 > Spark client

[jira] [Comment Edited] (SPARK-28641) MicroBatchExecution committed offsets greater than available offsets

2019-09-03 Thread Gabor Somogyi (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28641?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16921253#comment-16921253 ] Gabor Somogyi edited comment on SPARK-28641 at 9/3/19 8:33 AM: ---

[jira] [Commented] (SPARK-28641) MicroBatchExecution committed offsets greater than available offsets

2019-09-03 Thread Gabor Somogyi (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28641?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16921254#comment-16921254 ] Gabor Somogyi commented on SPARK-28641: --- I'm intended to close this jira unless any objection. >

[jira] [Created] (SPARK-28957) Copy any "spark.hive.foo=bar" spark properties into hadoop conf as "hive.foo=bar"

2019-09-03 Thread Weichen Xu (Jira)
Weichen Xu created SPARK-28957: -- Summary: Copy any "spark.hive.foo=bar" spark properties into hadoop conf as "hive.foo=bar" Key: SPARK-28957 URL: https://issues.apache.org/jira/browse/SPARK-28957

[jira] [Commented] (SPARK-28896) Spark client process is unable to upload jars to hdfs while using ConfigMap not HADOOP_CONF_DIR

2019-09-03 Thread Stavros Kontopoulos (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28896?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16921397#comment-16921397 ] Stavros Kontopoulos commented on SPARK-28896: - Changed to Spark 3.0.0. I will review the PR.

[jira] [Updated] (SPARK-28958) pyspark.ml function parity

2019-09-03 Thread zhengruifeng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28958?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhengruifeng updated SPARK-28958: - Attachment: ML_SYNC.pdf > pyspark.ml function parity > -- > >

[jira] [Commented] (SPARK-28883) Fix a flaky test: ThriftServerQueryTestSuite

2019-09-03 Thread Shixiong Zhu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28883?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16921413#comment-16921413 ] Shixiong Zhu commented on SPARK-28883: -- Marked this is a 3.0.0 blocker. We should either fix the

[jira] [Commented] (SPARK-28921) Spark jobs failing on latest versions of Kubernetes (1.15.3, 1.14.6, 1,13.10, 1.12.10, 1.11.10)

2019-09-03 Thread Stavros Kontopoulos (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28921?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16921315#comment-16921315 ] Stavros Kontopoulos commented on SPARK-28921: - [~andygrove] could you please clarify what do

[jira] [Comment Edited] (SPARK-28921) Spark jobs failing on latest versions of Kubernetes (1.15.3, 1.14.6, 1,13.10, 1.12.10, 1.11.10)

2019-09-03 Thread Stavros Kontopoulos (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28921?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16921315#comment-16921315 ] Stavros Kontopoulos edited comment on SPARK-28921 at 9/3/19 10:25 AM:

[jira] [Updated] (SPARK-28895) Spark client process is unable to upload jars to hdfs while using ConfigMap not HADOOP_CONF_DIR

2019-09-03 Thread Stavros Kontopoulos (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28895?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stavros Kontopoulos updated SPARK-28895: Affects Version/s: (was: 2.4.3) 3.0.0 > Spark client

[jira] [Created] (SPARK-28958) pyspark.ml function parity

2019-09-03 Thread zhengruifeng (Jira)
zhengruifeng created SPARK-28958: Summary: pyspark.ml function parity Key: SPARK-28958 URL: https://issues.apache.org/jira/browse/SPARK-28958 Project: Spark Issue Type: Improvement

[jira] [Commented] (SPARK-28121) String Functions: decode can not accept 'escape' and 'hex' as charset

2019-09-03 Thread daile (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28121?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16921369#comment-16921369 ] daile commented on SPARK-28121: --- i will work on this > String Functions: decode can not accept 'escape'

[jira] [Updated] (SPARK-28883) Fix a flaky test: ThriftServerQueryTestSuite

2019-09-03 Thread Shixiong Zhu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28883?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-28883: - Priority: Blocker (was: Major) > Fix a flaky test: ThriftServerQueryTestSuite >

[jira] [Updated] (SPARK-28883) Fix a flaky test: ThriftServerQueryTestSuite

2019-09-03 Thread Shixiong Zhu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28883?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-28883: - Target Version/s: 3.0.0 > Fix a flaky test: ThriftServerQueryTestSuite >

[jira] [Updated] (SPARK-28930) Spark DESC FORMATTED TABLENAME information display issues

2019-09-03 Thread Sujith Chacko (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28930?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sujith Chacko updated SPARK-28930: -- Description: Spark DESC FORMATTED TABLENAME information display issues.Showing incorrect 

[jira] [Updated] (SPARK-28930) Spark DESC FORMATTED TABLENAME information display issues

2019-09-03 Thread jobit mathew (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28930?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jobit mathew updated SPARK-28930: - Description: Spark DESC FORMATTED TABLENAME information display issues.Showing incorrect  *Last

[jira] [Updated] (SPARK-28956) Make it tighter

2019-09-03 Thread daile (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28956?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] daile updated SPARK-28956: -- Priority: Minor (was: Major) > Make it tighter > --- > > Key: SPARK-28956 >

[jira] [Resolved] (SPARK-28946) Add some more information about building SparkR on Windows

2019-09-03 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28946?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-28946. -- Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull request 25647

[jira] [Commented] (SPARK-28444) Bump Kubernetes Client Version to 4.3.0

2019-09-03 Thread Jira
[ https://issues.apache.org/jira/browse/SPARK-28444?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16921202#comment-16921202 ] Björn commented on SPARK-28444: --- This is fixed in SPARK-28921 Workaround in 2.4.3 is to replace the three

[jira] [Commented] (SPARK-28641) MicroBatchExecution committed offsets greater than available offsets

2019-09-03 Thread Gabor Somogyi (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28641?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16921253#comment-16921253 ] Gabor Somogyi commented on SPARK-28641: --- [~MariaCarrie] this is a know issue in Kafka, please see 

[jira] [Comment Edited] (SPARK-28641) MicroBatchExecution committed offsets greater than available offsets

2019-09-03 Thread Gabor Somogyi (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28641?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16921253#comment-16921253 ] Gabor Somogyi edited comment on SPARK-28641 at 9/3/19 8:31 AM: ---

[jira] [Assigned] (SPARK-28946) Add some more information about building SparkR on Windows

2019-09-03 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28946?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-28946: Assignee: Hyukjin Kwon > Add some more information about building SparkR on Windows >

[jira] [Commented] (SPARK-28864) Add spark connector for Alibaba Log Service

2019-09-03 Thread Gabor Somogyi (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28864?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16921236#comment-16921236 ] Gabor Somogyi commented on SPARK-28864: --- Or just a separate project. > Add spark connector for

[jira] [Created] (SPARK-28956) Make it tighter

2019-09-03 Thread daile (Jira)
daile created SPARK-28956: - Summary: Make it tighter Key: SPARK-28956 URL: https://issues.apache.org/jira/browse/SPARK-28956 Project: Spark Issue Type: Improvement Components: Build

[jira] [Commented] (SPARK-28770) Flaky Tests: Test ReplayListenerSuite.End-to-end replay with compression failed

2019-09-03 Thread zhao bo (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28770?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16921224#comment-16921224 ] zhao bo commented on SPARK-28770: - Hi guys, Thank you very much for your concern. I had post an PR for

[jira] [Comment Edited] (SPARK-28895) Spark client process is unable to upload jars to hdfs while using ConfigMap not HADOOP_CONF_DIR

2019-09-03 Thread Stavros Kontopoulos (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28895?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16921394#comment-16921394 ] Stavros Kontopoulos edited comment on SPARK-28895 at 9/3/19 1:24 PM: -

[jira] [Commented] (SPARK-28883) Fix a flaky test: ThriftServerQueryTestSuite

2019-09-03 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28883?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16921440#comment-16921440 ] Hyukjin Kwon commented on SPARK-28883: -- Sounds good for targeting 3.0.0. I'll revert this if it

[jira] [Commented] (SPARK-28610) Support larger buffer for sum of long

2019-09-03 Thread Gengliang Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28610?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16921459#comment-16921459 ] Gengliang Wang commented on SPARK-28610: Hi [~mgaido] I tried the following SQL on PostgreSQL

[jira] [Commented] (SPARK-28610) Support larger buffer for sum of long

2019-09-03 Thread Marco Gaido (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28610?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16921471#comment-16921471 ] Marco Gaido commented on SPARK-28610: - Hi [~Gengliang.Wang]. That's a different thing. you are doing

[jira] [Created] (SPARK-28959) Support `EqualNullSafe` as join condition in Dynamic Partition Pruning

2019-09-03 Thread Maryann Xue (Jira)
Maryann Xue created SPARK-28959: --- Summary: Support `EqualNullSafe` as join condition in Dynamic Partition Pruning Key: SPARK-28959 URL: https://issues.apache.org/jira/browse/SPARK-28959 Project: Spark

[jira] [Commented] (SPARK-28917) Jobs can hang because of race of RDD.dependencies

2019-09-03 Thread Thomas Graves (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28917?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16921424#comment-16921424 ] Thomas Graves commented on SPARK-28917: --- So for this to happen, you essentially have to have that

[jira] [Comment Edited] (SPARK-28121) String Functions: decode can not accept 'escape' and 'hex' as charset

2019-09-03 Thread daile (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28121?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16921369#comment-16921369 ] daile edited comment on SPARK-28121 at 9/3/19 1:27 PM: --- you can use sql like this

[jira] [Commented] (SPARK-28610) Support larger buffer for sum of long

2019-09-03 Thread Gengliang Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28610?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16921509#comment-16921509 ] Gengliang Wang commented on SPARK-28610: [~mgaido] I see, thanks for the explanation! > Support

[jira] [Created] (SPARK-28960) maven 3.6.1 gone from apache maven repo

2019-09-03 Thread Thomas Graves (Jira)
Thomas Graves created SPARK-28960: - Summary: maven 3.6.1 gone from apache maven repo Key: SPARK-28960 URL: https://issues.apache.org/jira/browse/SPARK-28960 Project: Spark Issue Type: Bug

[jira] [Resolved] (SPARK-28805) Document DESCRIBE FUNCTION in SQL Reference.

2019-09-03 Thread Xiao Li (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28805?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-28805. - Fix Version/s: 3.0.0 Assignee: Dilip Biswal Resolution: Fixed > Document DESCRIBE

[jira] [Resolved] (SPARK-28806) Document SHOW COLUMNS in SQL Reference.

2019-09-03 Thread Xiao Li (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28806?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-28806. - Fix Version/s: 3.0.0 Assignee: Dilip Biswal Resolution: Fixed > Document SHOW COLUMNS

[jira] [Commented] (SPARK-28960) maven 3.6.1 gone from apache maven repo

2019-09-03 Thread Thomas Graves (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28960?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16921611#comment-16921611 ] Thomas Graves commented on SPARK-28960: --- its a bit odd I don't see 3.6.2 on their page as released

[jira] [Updated] (SPARK-22955) Error generating jobs when Stopping JobGenerator gracefully

2019-09-03 Thread Sean Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-22955?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-22955: -- Labels: (was: bulk-closed) > Error generating jobs when Stopping JobGenerator gracefully >

[jira] [Reopened] (SPARK-22955) Error generating jobs when Stopping JobGenerator gracefully

2019-09-03 Thread Sean Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-22955?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen reopened SPARK-22955: --- > Error generating jobs when Stopping JobGenerator gracefully >

[jira] [Resolved] (SPARK-22955) Error generating jobs when Stopping JobGenerator gracefully

2019-09-03 Thread Sean Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-22955?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-22955. --- Fix Version/s: 3.0.0 2.4.5 Resolution: Fixed Resolved by

[jira] [Commented] (SPARK-28953) Integration tests fail due to malformed URL

2019-09-03 Thread shane knapp (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28953?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16921691#comment-16921691 ] shane knapp commented on SPARK-28953: - could you link to a build that failed this way? >

[jira] [Resolved] (SPARK-28956) Make it tighter

2019-09-03 Thread Sean Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28956?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-28956. --- Resolution: Invalid Please see comments on PR and review https://spark.apache.org/contributing.html

[jira] [Updated] (SPARK-27495) SPIP: Support Stage level resource configuration and scheduling

2019-09-03 Thread Thomas Graves (Jira)
[ https://issues.apache.org/jira/browse/SPARK-27495?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves updated SPARK-27495: -- Description: *Q1.* What are you trying to do? Articulate your objectives using absolutely no

[jira] [Commented] (SPARK-27495) SPIP: Support Stage level resource configuration and scheduling

2019-09-03 Thread Thomas Graves (Jira)
[ https://issues.apache.org/jira/browse/SPARK-27495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16921608#comment-16921608 ] Thomas Graves commented on SPARK-27495: --- Thanks for the feedback [~felixcheung] 1.   yes this is

[jira] [Created] (SPARK-28961) Upgrade Maven to 3.6.1

2019-09-03 Thread Xiao Li (Jira)
Xiao Li created SPARK-28961: --- Summary: Upgrade Maven to 3.6.1 Key: SPARK-28961 URL: https://issues.apache.org/jira/browse/SPARK-28961 Project: Spark Issue Type: Improvement Components:

[jira] [Updated] (SPARK-28961) Upgrade Maven to 3.6.2

2019-09-03 Thread Xiao Li (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28961?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-28961: Summary: Upgrade Maven to 3.6.2 (was: Upgrade Maven to 3.6.1) > Upgrade Maven to 3.6.2 >

[jira] [Created] (SPARK-28962) High-order function: filter(array, function) → array

2019-09-03 Thread Henry Davidge (Jira)
Henry Davidge created SPARK-28962: - Summary: High-order function: filter(array, function) → array Key: SPARK-28962 URL: https://issues.apache.org/jira/browse/SPARK-28962 Project: Spark Issue

[jira] [Updated] (SPARK-28961) Upgrade Maven to 3.6.2

2019-09-03 Thread Thomas Graves (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28961?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves updated SPARK-28961: -- Description: It looks like maven 3.6.1 is missing from the apache maven repo:

[jira] [Commented] (SPARK-28943) NoSuchMethodError: shaded.parquet.org.apache.thrift.EncodingUtils.setBit(BIZ)B

2019-09-03 Thread Michael Heuer (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28943?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16921581#comment-16921581 ] Michael Heuer commented on SPARK-28943: --- To be honest, I am burned out trying to track down all of

[jira] [Closed] (SPARK-28444) Bump Kubernetes Client Version to 4.3.0

2019-09-03 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28444?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun closed SPARK-28444. - > Bump Kubernetes Client Version to 4.3.0 > --- > >

[jira] [Resolved] (SPARK-28444) Bump Kubernetes Client Version to 4.3.0

2019-09-03 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28444?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-28444. --- Resolution: Duplicate > Bump Kubernetes Client Version to 4.3.0 >

[jira] [Created] (SPARK-28963) Fall back to archive.apache.org to download Maven if mirrors don't have requested version

2019-09-03 Thread Sean Owen (Jira)
Sean Owen created SPARK-28963: - Summary: Fall back to archive.apache.org to download Maven if mirrors don't have requested version Key: SPARK-28963 URL: https://issues.apache.org/jira/browse/SPARK-28963

[jira] [Resolved] (SPARK-28960) maven 3.6.1 gone from apache maven repo

2019-09-03 Thread Thomas Graves (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28960?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-28960. --- Resolution: Duplicate > maven 3.6.1 gone from apache maven repo >

[jira] [Commented] (SPARK-27781) Tried to access method org.apache.avro.specific.SpecificData.()V

2019-09-03 Thread Michael Heuer (Jira)
[ https://issues.apache.org/jira/browse/SPARK-27781?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16921700#comment-16921700 ] Michael Heuer commented on SPARK-27781: --- This is still an issue with the Spark 2.4.4 binary

[jira] [Commented] (SPARK-28628) Support namespaces in V2SessionCatalog

2019-09-03 Thread Burak Yavuz (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28628?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16921705#comment-16921705 ] Burak Yavuz commented on SPARK-28628: - Resolved by [https://github.com/apache/spark/pull/25363] >

[jira] [Resolved] (SPARK-28628) Support namespaces in V2SessionCatalog

2019-09-03 Thread Burak Yavuz (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28628?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Burak Yavuz resolved SPARK-28628. - Fix Version/s: 3.0.0 Resolution: Fixed > Support namespaces in V2SessionCatalog >

[jira] [Created] (SPARK-28966) Cached DNS name resolution error

2019-09-03 Thread Roger Liu (Jira)
Roger Liu created SPARK-28966: - Summary: Cached DNS name resolution error Key: SPARK-28966 URL: https://issues.apache.org/jira/browse/SPARK-28966 Project: Spark Issue Type: Bug

[jira] [Resolved] (SPARK-28966) Cached DNS name resolution error

2019-09-03 Thread Roger Liu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28966?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Roger Liu resolved SPARK-28966. --- Resolution: Invalid > Cached DNS name resolution error > > >

[jira] [Updated] (SPARK-28709) Memory leaks after stopping of StreamingContext

2019-09-03 Thread Sean Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28709?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-28709: -- Fix Version/s: 2.4.5 > Memory leaks after stopping of StreamingContext >

[jira] [Updated] (SPARK-27781) Tried to access method org.apache.avro.specific.SpecificData.()V

2019-09-03 Thread Michael Heuer (Jira)
[ https://issues.apache.org/jira/browse/SPARK-27781?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Heuer updated SPARK-27781: -- Affects Version/s: 2.4.4 > Tried to access method org.apache.avro.specific.SpecificData.()V >

[jira] [Updated] (SPARK-28855) Remove outdated Experimental, Evolving annotations

2019-09-03 Thread Sean Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28855?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-28855: -- Issue Type: Task (was: Bug) > Remove outdated Experimental, Evolving annotations >

[jira] [Assigned] (SPARK-28628) Support namespaces in V2SessionCatalog

2019-09-03 Thread Burak Yavuz (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28628?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Burak Yavuz reassigned SPARK-28628: --- Assignee: Ryan Blue > Support namespaces in V2SessionCatalog >

[jira] [Resolved] (SPARK-27489) UI updates to show executor resource information

2019-09-03 Thread Gengliang Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-27489?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang resolved SPARK-27489. Fix Version/s: 3.0.0 Resolution: Fixed This issue is resolved in

[jira] [Created] (SPARK-28965) Document workings for CBO

2019-09-03 Thread Dilip Biswal (Jira)
Dilip Biswal created SPARK-28965: Summary: Document workings for CBO Key: SPARK-28965 URL: https://issues.apache.org/jira/browse/SPARK-28965 Project: Spark Issue Type: Sub-task

[jira] [Commented] (SPARK-28965) Document workings for CBO

2019-09-03 Thread Dilip Biswal (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28965?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16921745#comment-16921745 ] Dilip Biswal commented on SPARK-28965: -- cc [~smilegator] > Document workings for CBO >

[jira] [Updated] (SPARK-28965) Document workings of CBO

2019-09-03 Thread Dilip Biswal (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28965?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dilip Biswal updated SPARK-28965: - Summary: Document workings of CBO (was: Document workings for CBO) > Document workings of CBO

[jira] [Resolved] (SPARK-28788) Document ANALYZE TABLE statement in SQL Reference.

2019-09-03 Thread Xiao Li (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28788?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-28788. - Fix Version/s: 3.0.0 Assignee: Huaxin Gao Resolution: Fixed > Document ANALYZE TABLE

[jira] [Resolved] (SPARK-3137) Use finer grained locking in TorrentBroadcast.readObject

2019-09-03 Thread Shixiong Zhu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-3137?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu resolved SPARK-3137. - Fix Version/s: 3.0.0 Resolution: Fixed > Use finer grained locking in

[jira] [Created] (SPARK-28964) saveAsTable doesn't pass in the data source information for V2 nodes

2019-09-03 Thread Burak Yavuz (Jira)
Burak Yavuz created SPARK-28964: --- Summary: saveAsTable doesn't pass in the data source information for V2 nodes Key: SPARK-28964 URL: https://issues.apache.org/jira/browse/SPARK-28964 Project: Spark

[jira] [Created] (SPARK-28968) Add HasNumFeatures in the scala side

2019-09-03 Thread zhengruifeng (Jira)
zhengruifeng created SPARK-28968: Summary: Add HasNumFeatures in the scala side Key: SPARK-28968 URL: https://issues.apache.org/jira/browse/SPARK-28968 Project: Spark Issue Type: Improvement

[jira] [Resolved] (SPARK-28694) Add Java/Scala StructuredKerberizedKafkaWordCount examples

2019-09-03 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28694?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-28694. -- Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull request 25649

[jira] [Assigned] (SPARK-28694) Add Java/Scala StructuredKerberizedKafkaWordCount examples

2019-09-03 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28694?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-28694: Assignee: hong dongdong > Add Java/Scala StructuredKerberizedKafkaWordCount examples >

[jira] [Resolved] (SPARK-28963) Fall back to archive.apache.org to download Maven if mirrors don't have requested version

2019-09-03 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28963?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-28963. -- Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull request 25667

[jira] [Commented] (SPARK-28967) ConcurrentModificationException is thrown from EventLoggingListener

2019-09-03 Thread Jungtaek Lim (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28967?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16921943#comment-16921943 ] Jungtaek Lim commented on SPARK-28967: -- FYI, SPARK-26714 causes this bug. >

[jira] [Created] (SPARK-28967) ConcurrentModificationException is thrown from EventLoggingListener

2019-09-03 Thread Jungtaek Lim (Jira)
Jungtaek Lim created SPARK-28967: Summary: ConcurrentModificationException is thrown from EventLoggingListener Key: SPARK-28967 URL: https://issues.apache.org/jira/browse/SPARK-28967 Project: Spark

[jira] [Commented] (SPARK-28967) ConcurrentModificationException is thrown from EventLoggingListener

2019-09-03 Thread Jungtaek Lim (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28967?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16921888#comment-16921888 ] Jungtaek Lim commented on SPARK-28967: -- Working on the fix. It should be simple fix, though I'm not

[jira] [Created] (SPARK-28969) OneVsRestModel in the py side should not set WeightCol and Classifier

2019-09-03 Thread zhengruifeng (Jira)
zhengruifeng created SPARK-28969: Summary: OneVsRestModel in the py side should not set WeightCol and Classifier Key: SPARK-28969 URL: https://issues.apache.org/jira/browse/SPARK-28969 Project: Spark

[jira] [Commented] (SPARK-28921) Spark jobs failing on latest versions of Kubernetes (1.15.3, 1.14.6, 1,13.10, 1.12.10, 1.11.10)

2019-09-03 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28921?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16921934#comment-16921934 ] Dongjoon Hyun commented on SPARK-28921: --- Please launch a new EKS cluster and build/publish master

[jira] [Commented] (SPARK-28770) Flaky Tests: Test ReplayListenerSuite.End-to-end replay with compression failed

2019-09-03 Thread Wing Yew Poon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28770?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16921955#comment-16921955 ] Wing Yew Poon commented on SPARK-28770: --- My earlier analysis of why the test failure occurs was

[jira] [Created] (SPARK-28970) implement USE CATALOG/NAMESPACE for Data Source V2

2019-09-03 Thread Wenchen Fan (Jira)
Wenchen Fan created SPARK-28970: --- Summary: implement USE CATALOG/NAMESPACE for Data Source V2 Key: SPARK-28970 URL: https://issues.apache.org/jira/browse/SPARK-28970 Project: Spark Issue Type:

[jira] [Commented] (SPARK-28970) implement USE CATALOG/NAMESPACE for Data Source V2

2019-09-03 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16921959#comment-16921959 ] Wenchen Fan commented on SPARK-28970: - cc [~imback82] do you want to work on it? > implement USE

[jira] [Commented] (SPARK-28970) implement USE CATALOG/NAMESPACE for Data Source V2

2019-09-03 Thread Terry Kim (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16921960#comment-16921960 ] Terry Kim commented on SPARK-28970: --- Yes, sure. Thanks! > implement USE CATALOG/NAMESPACE for Data

[jira] [Updated] (SPARK-28885) Follow ANSI store assignment rules in table insertion by default

2019-09-03 Thread Gengliang Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28885?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang updated SPARK-28885: --- Description: When inserting a value into a column with the different data type, Spark