[jira] [Commented] (SPARK-26512) Spark 2.4.0 is not working with Hadoop 2.8.3 in windows 10?

2019-01-04 Thread Anubhav Jain (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26512?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16733917#comment-16733917 ] Anubhav Jain commented on SPARK-26512: -- Thanks [~jerryshao] , for the reply . I Hav

[jira] [Commented] (SPARK-26359) Spark checkpoint restore fails after query restart

2019-01-04 Thread Kaspar Tint (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26359?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16733935#comment-16733935 ] Kaspar Tint commented on SPARK-26359: - [~gsomogyi]: Yes, this ticket can be closed.

[jira] [Resolved] (SPARK-26359) Spark checkpoint restore fails after query restart

2019-01-04 Thread Gabor Somogyi (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26359?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gabor Somogyi resolved SPARK-26359. --- Resolution: Information Provided > Spark checkpoint restore fails after query restart >

[jira] [Closed] (SPARK-26359) Spark checkpoint restore fails after query restart

2019-01-04 Thread Gabor Somogyi (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26359?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gabor Somogyi closed SPARK-26359. - > Spark checkpoint restore fails after query restart > -

[jira] [Commented] (SPARK-26254) Move delegation token providers into a separate project

2019-01-04 Thread Gabor Somogyi (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26254?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16733956#comment-16733956 ] Gabor Somogyi commented on SPARK-26254: --- ping [~vanzin] > Move delegation token p

[jira] [Assigned] (SPARK-26462) Use ConfigEntry for hardcoded configs for execution categories.

2019-01-04 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26462?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26462: Assignee: Apache Spark > Use ConfigEntry for hardcoded configs for execution categories.

[jira] [Assigned] (SPARK-26462) Use ConfigEntry for hardcoded configs for execution categories.

2019-01-04 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26462?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26462: Assignee: (was: Apache Spark) > Use ConfigEntry for hardcoded configs for execution c

[jira] [Commented] (SPARK-26462) Use ConfigEntry for hardcoded configs for execution categories.

2019-01-04 Thread pralabhkumar (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26462?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16733988#comment-16733988 ] pralabhkumar commented on SPARK-26462: -- [~ueshin] Have created pull request with 

[jira] [Updated] (SPARK-26126) Put scala-library deps into root pom instead of spark-tags module

2019-01-04 Thread liupengcheng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26126?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liupengcheng updated SPARK-26126: - Summary: Put scala-library deps into root pom instead of spark-tags module (was: Should put sca

[jira] [Updated] (SPARK-26529) Add logs for IOException when preparing local resource

2019-01-04 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26529?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-26529: - Priority: Trivial (was: Major) > Add logs for IOException when preparing local resource >

[jira] [Updated] (SPARK-26512) Spark 2.4.0 is not working with Hadoop 2.8.3 in windows 10

2019-01-04 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26512?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-26512: - Summary: Spark 2.4.0 is not working with Hadoop 2.8.3 in windows 10 (was: Spark 2.4.0 is not wo

[jira] [Commented] (SPARK-26254) Move delegation token providers into a separate project

2019-01-04 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26254?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16734004#comment-16734004 ] Hyukjin Kwon commented on SPARK-26254: -- [~gsomogyi], can you point out which commen

[jira] [Updated] (SPARK-25919) Date value corrupts when tables are "ParquetHiveSerDe" formatted and target table is Partitioned

2019-01-04 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25919?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-25919: - Priority: Major (was: Blocker) > Date value corrupts when tables are "ParquetHiveSerDe" formatt

[jira] [Resolved] (SPARK-25873) Date corruption when Spark and Hive both are on different timezones

2019-01-04 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25873?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-25873. -- Resolution: Fixed > Date corruption when Spark and Hive both are on different timezones >

[jira] [Commented] (SPARK-26511) java.lang.ClassCastException error when loading Spark MLlib model from parquet file

2019-01-04 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26511?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16734009#comment-16734009 ] Hyukjin Kwon commented on SPARK-26511: -- BTW, please make sure such facts are descri

[jira] [Assigned] (SPARK-26126) Put scala-library deps into root pom instead of spark-tags module

2019-01-04 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26126?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26126: Assignee: (was: Apache Spark) > Put scala-library deps into root pom instead of spark

[jira] [Resolved] (SPARK-19217) Offer easy cast from vector to array

2019-01-04 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19217?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-19217. -- Resolution: Later Let's track this in another JIRA. It's been inactive for so long time. At le

[jira] [Assigned] (SPARK-26126) Put scala-library deps into root pom instead of spark-tags module

2019-01-04 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26126?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26126: Assignee: Apache Spark > Put scala-library deps into root pom instead of spark-tags modul

[jira] [Updated] (SPARK-26334) NullPointerException in CallMethodViaReflection when we apply reflect function for empty field

2019-01-04 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26334?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-26334: - Target Version/s: (was: 2.2.0, 2.3.0) > NullPointerException in CallMethodViaReflection when

[jira] [Updated] (SPARK-26529) Add debug logs for confArchive when preparing local resource

2019-01-04 Thread liupengcheng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26529?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liupengcheng updated SPARK-26529: - Summary: Add debug logs for confArchive when preparing local resource (was: Add logs for IOExc

[jira] [Resolved] (SPARK-26334) NullPointerException in CallMethodViaReflection when we apply reflect function for empty field

2019-01-04 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26334?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-26334. -- Resolution: Invalid The method itself already throws an NPE: {code} scala> java.net.URLDecode

[jira] [Updated] (SPARK-24661) Window API - using multiple fields for partitioning with WindowSpec API and dataset that is cached causes org.apache.spark.sql.catalyst.errors.package$TreeNodeException

2019-01-04 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24661?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-24661: - Component/s: (was: PySpark) > Window API - using multiple fields for partitioning with Windo

[jira] [Resolved] (SPARK-26454) While creating new UDF with JAR though UDF is created successfully, it throws IllegegalArgument Exception

2019-01-04 Thread Udbhav Agrawal (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26454?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Udbhav Agrawal resolved SPARK-26454. Resolution: Invalid Apparently i placed wrong jar in different version, this issue seems i

[jira] [Commented] (SPARK-26512) Spark 2.4.0 is not working with Hadoop 2.8.3 in windows 10

2019-01-04 Thread Saisai Shao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26512?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16734072#comment-16734072 ] Saisai Shao commented on SPARK-26512: - This seems like a Netty version problem, nett

[jira] [Updated] (SPARK-26334) NullPointerException in CallMethodViaReflection when we apply reflect function for empty field

2019-01-04 Thread chenliang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26334?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] chenliang updated SPARK-26334: -- Attachment: 1537537286575.jpg > NullPointerException in CallMethodViaReflection when we apply reflect

[jira] [Commented] (SPARK-26334) NullPointerException in CallMethodViaReflection when we apply reflect function for empty field

2019-01-04 Thread chenliang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26334?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16734138#comment-16734138 ] chenliang commented on SPARK-26334: --- [~hyukjin.kwon] Thank you ,I did know that the me

[jira] [Issue Comment Deleted] (SPARK-26334) NullPointerException in CallMethodViaReflection when we apply reflect function for empty field

2019-01-04 Thread chenliang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26334?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] chenliang updated SPARK-26334: -- Comment: was deleted (was: [~hyukjin.kwon] Thank you ,I did know that the method itself throws an NPE

[jira] [Updated] (SPARK-26334) NullPointerException in CallMethodViaReflection when we apply reflect function for empty field

2019-01-04 Thread chenliang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26334?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] chenliang updated SPARK-26334: -- Attachment: 21_47_01__01_04_2019.jpg > NullPointerException in CallMethodViaReflection when we apply

[jira] [Updated] (SPARK-26334) NullPointerException in CallMethodViaReflection when we apply reflect function for empty field

2019-01-04 Thread chenliang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26334?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] chenliang updated SPARK-26334: -- Attachment: (was: 1537537286575.jpg) > NullPointerException in CallMethodViaReflection when we ap

[jira] [Commented] (SPARK-26334) NullPointerException in CallMethodViaReflection when we apply reflect function for empty field

2019-01-04 Thread chenliang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26334?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16734142#comment-16734142 ] chenliang commented on SPARK-26334: --- [~hyukjin.kwon]Thanks for your attention,I did kn

[jira] [Commented] (SPARK-26334) NullPointerException in CallMethodViaReflection when we apply reflect function for empty field

2019-01-04 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26334?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16734150#comment-16734150 ] Hyukjin Kwon commented on SPARK-26334: -- I think either way makes sense. Spark doesn

[jira] [Resolved] (SPARK-26445) Use ConfigEntry for hardcoded configs for driver/executor categories.

2019-01-04 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26445?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-26445. -- Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 23415 [https://gi

[jira] [Comment Edited] (SPARK-22231) Support of map, filter, withColumn, dropColumn in nested list of structures

2019-01-04 Thread Timothy Pharo (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22231?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16734160#comment-16734160 ] Timothy Pharo edited comment on SPARK-22231 at 1/4/19 2:15 PM: ---

[jira] [Commented] (SPARK-22231) Support of map, filter, withColumn, dropColumn in nested list of structures

2019-01-04 Thread Timothy Pharo (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22231?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16734160#comment-16734160 ] Timothy Pharo commented on SPARK-22231: --- [~dbtsai] and [~jeremyrsmith], this all l

[jira] [Commented] (SPARK-26436) Dataframe resulting from a GroupByKey and flatMapGroups operation throws java.lang.UnsupportedException when groupByKey is applied on it.

2019-01-04 Thread Liang-Chi Hsieh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26436?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16734178#comment-16734178 ] Liang-Chi Hsieh commented on SPARK-26436: - This issue is caused by how you creat

[jira] [Assigned] (SPARK-26445) Use ConfigEntry for hardcoded configs for driver/executor categories.

2019-01-04 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26445?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-26445: Assignee: Takuya Ueshin > Use ConfigEntry for hardcoded configs for driver/executor categ

[jira] [Commented] (SPARK-26334) NullPointerException in CallMethodViaReflection when we apply reflect function for empty field

2019-01-04 Thread chenliang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26334?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16734182#comment-16734182 ] chenliang commented on SPARK-26334: --- Anyway,thanks. > NullPointerException in CallMe

[jira] [Commented] (SPARK-4476) Use MapType for dict in json which has unique keys in each row.

2019-01-04 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4476?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16734184#comment-16734184 ] Hyukjin Kwon commented on SPARK-4476: - I think we can resolve this now in favour of {

[jira] [Resolved] (SPARK-4476) Use MapType for dict in json which has unique keys in each row.

2019-01-04 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4476?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-4476. - Resolution: Done > Use MapType for dict in json which has unique keys in each row. >

[jira] [Resolved] (SPARK-6540) Spark SQL thrift server fails to pass settings to following query in the same session

2019-01-04 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6540?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-6540. - Resolution: Incomplete Leaving this resolved due to inactivity from the reporter > Spark SQL thr

[jira] [Resolved] (SPARK-7148) Configure Parquet block size (row group size) for ML model import/export

2019-01-04 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7148?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-7148. - Resolution: Not A Problem Then, I'm leaving resolved. Please reopen this if I am mistaken. > Con

[jira] [Resolved] (SPARK-4235) Add union data type support

2019-01-04 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4235?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-4235. - Resolution: Later I'm going to leave this resolved as {{Later}}. It's been inactive for few years

[jira] [Resolved] (SPARK-7428) DataFrame.join() could create a new df with duplicate column name

2019-01-04 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7428?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-7428. - Resolution: Won't Fix > DataFrame.join() could create a new df with duplicate column name > -

[jira] [Resolved] (SPARK-7754) [SQL] Use PartialFunction literals instead of objects in Catalyst

2019-01-04 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7754?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-7754. - Resolution: Won't Fix > [SQL] Use PartialFunction literals instead of objects in Catalyst > -

[jira] [Resolved] (SPARK-7755) MetadataCache.refresh does not take into account _SUCCESS

2019-01-04 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7755?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-7755. - Resolution: Not A Problem > MetadataCache.refresh does not take into account _SUCCESS > -

[jira] [Resolved] (SPARK-7789) sql on security hbase:Token generation only allowed for Kerberos authenticated clients

2019-01-04 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7789?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-7789. - Resolution: Incomplete Leaving this resolved due to inactivity from the reporter. > sql on secu

[jira] [Resolved] (SPARK-8102) Big performance difference when joining 3 tables in different order

2019-01-04 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8102?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-8102. - Resolution: Not A Problem The code path has been drastically changed. Now we're heading for Spark

[jira] [Commented] (SPARK-8324) Register Query as view through JDBC interface

2019-01-04 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8324?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16734203#comment-16734203 ] Hyukjin Kwon commented on SPARK-8324: - Fixed in SPARK-24423 > Register Query as view

[jira] [Resolved] (SPARK-8324) Register Query as view through JDBC interface

2019-01-04 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8324?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-8324. - Resolution: Duplicate > Register Query as view through JDBC interface > -

[jira] [Commented] (SPARK-7754) [SQL] Use PartialFunction literals instead of objects in Catalyst

2019-01-04 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7754?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16734197#comment-16734197 ] Hyukjin Kwon commented on SPARK-7754: - I don't think we're going to change if it targ

[jira] [Commented] (SPARK-8370) Add API for data sources to register databases

2019-01-04 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8370?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16734205#comment-16734205 ] Hyukjin Kwon commented on SPARK-8370: - I think we're working on multiple catalog supp

[jira] [Resolved] (SPARK-8403) Pruner partition won't effective when partition field and fieldSchema exist in sql predicate

2019-01-04 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8403?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-8403. - Resolution: Incomplete Leaving this resolved due to reporter's inactivity. > Pruner partition wo

[jira] [Resolved] (SPARK-8577) ScalaReflectionLock.synchronized can cause deadlock

2019-01-04 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8577?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-8577. - Resolution: Not A Problem I'm leaving resolved since we removed 2.10. > ScalaReflectionLock.sync

[jira] [Resolved] (SPARK-8602) Shared cached DataFrames

2019-01-04 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8602?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-8602. - Resolution: Not A Problem > Shared cached DataFrames > > >

[jira] [Resolved] (SPARK-8370) Add API for data sources to register databases

2019-01-04 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8370?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-8370. - Resolution: Not A Problem > Add API for data sources to register databases >

[jira] [Commented] (SPARK-9218) Falls back to getAllPartitions when getPartitionsByFilter fails

2019-01-04 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9218?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16734218#comment-16734218 ] Hyukjin Kwon commented on SPARK-9218: - ping [~lian cheng] > Falls back to getAllPart

[jira] [Resolved] (SPARK-8427) Incorrect ACL checking for partitioned table in Spark SQL-1.4

2019-01-04 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8427?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-8427. - Resolution: Not A Problem We're heading for Spark 3.0 and the issue was Spark 1.4. The codes have

[jira] [Commented] (SPARK-8602) Shared cached DataFrames

2019-01-04 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8602?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16734216#comment-16734216 ] Hyukjin Kwon commented on SPARK-8602: - I think we now have one Spark context and mult

[jira] [Created] (SPARK-26534) Closure Cleaner Bug

2019-01-04 Thread sam (JIRA)
sam created SPARK-26534: --- Summary: Closure Cleaner Bug Key: SPARK-26534 URL: https://issues.apache.org/jira/browse/SPARK-26534 Project: Spark Issue Type: Bug Components: Spark Core Affect

[jira] [Commented] (SPARK-26436) Dataframe resulting from a GroupByKey and flatMapGroups operation throws java.lang.UnsupportedException when groupByKey is applied on it.

2019-01-04 Thread Liang-Chi Hsieh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26436?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16734282#comment-16734282 ] Liang-Chi Hsieh commented on SPARK-26436: - Sorry I don't know what you mean "sho

[jira] [Commented] (SPARK-26436) Dataframe resulting from a GroupByKey and flatMapGroups operation throws java.lang.UnsupportedException when groupByKey is applied on it.

2019-01-04 Thread Manish (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26436?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16734267#comment-16734267 ] Manish commented on SPARK-26436: [~viirya]: Thanks for the explanation. But should group

[jira] [Created] (SPARK-26535) Parsing literals as DOUBLE instead of DECIMAL

2019-01-04 Thread Marco Gaido (JIRA)
Marco Gaido created SPARK-26535: --- Summary: Parsing literals as DOUBLE instead of DECIMAL Key: SPARK-26535 URL: https://issues.apache.org/jira/browse/SPARK-26535 Project: Spark Issue Type: Impro

[jira] [Created] (SPARK-26536) Upgrade Mockito to 2.23.4

2019-01-04 Thread Dongjoon Hyun (JIRA)
Dongjoon Hyun created SPARK-26536: - Summary: Upgrade Mockito to 2.23.4 Key: SPARK-26536 URL: https://issues.apache.org/jira/browse/SPARK-26536 Project: Spark Issue Type: Sub-task Co

[jira] [Updated] (SPARK-26536) Upgrade Mockito to 2.23.4

2019-01-04 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26536?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-26536: -- Component/s: Tests > Upgrade Mockito to 2.23.4 > - > >

[jira] [Assigned] (SPARK-26536) Upgrade Mockito to 2.23.4

2019-01-04 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26536?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26536: Assignee: Apache Spark > Upgrade Mockito to 2.23.4 > - > >

[jira] [Assigned] (SPARK-26536) Upgrade Mockito to 2.23.4

2019-01-04 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26536?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26536: Assignee: Apache Spark > Upgrade Mockito to 2.23.4 > - > >

[jira] [Assigned] (SPARK-26536) Upgrade Mockito to 2.23.4

2019-01-04 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26536?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26536: Assignee: (was: Apache Spark) > Upgrade Mockito to 2.23.4 > -

[jira] [Updated] (SPARK-26536) Upgrade Mockito to 2.23.4

2019-01-04 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26536?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-26536: -- Description: This issue upgrades Mockito from 1.10.19 to 2.23.4. The following changes are re

[jira] [Commented] (SPARK-26362) Remove 'spark.driver.allowMultipleContexts' to disallow multiple Spark contexts

2019-01-04 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26362?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16734355#comment-16734355 ] Reynold Xin commented on SPARK-26362: - [~hyukjin.kwon] please make sure we add relea

[jira] [Commented] (SPARK-26246) Infer timestamp types from JSON

2019-01-04 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26246?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16734361#comment-16734361 ] Reynold Xin commented on SPARK-26246: - |Is there an option flag for this? This is a

[jira] [Comment Edited] (SPARK-26246) Infer timestamp types from JSON

2019-01-04 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26246?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16734361#comment-16734361 ] Reynold Xin edited comment on SPARK-26246 at 1/4/19 5:22 PM: -

[jira] [Issue Comment Deleted] (SPARK-26246) Infer timestamp types from JSON

2019-01-04 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26246?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-26246: Comment: was deleted (was: asfgit closed pull request #23201: [SPARK-26246][SQL] Inferring Timest

[jira] [Issue Comment Deleted] (SPARK-26366) Except with transform regression

2019-01-04 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26366?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-26366: Comment: was deleted (was: gatorsmile closed pull request #23350: [SPARK-26366][SQL][BACKPORT-2.3]

[jira] [Commented] (SPARK-26366) Except with transform regression

2019-01-04 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26366?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16734363#comment-16734363 ] Reynold Xin commented on SPARK-26366: - Please make sure you guys tag these tickets w

[jira] [Updated] (SPARK-26366) Except with transform regression

2019-01-04 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26366?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-26366: Labels: correctness (was: ) > Except with transform regression >

[jira] [Issue Comment Deleted] (SPARK-26366) Except with transform regression

2019-01-04 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26366?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-26366: Comment: was deleted (was: asfgit closed pull request #23315: [SPARK-26366][SQL] ReplaceExceptWit

[jira] [Issue Comment Deleted] (SPARK-26366) Except with transform regression

2019-01-04 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26366?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-26366: Comment: was deleted (was: mgaido91 opened a new pull request #23350: [SPARK-26366][SQL][BACKPORT

[jira] [Issue Comment Deleted] (SPARK-26366) Except with transform regression

2019-01-04 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26366?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-26366: Comment: was deleted (was: mgaido91 opened a new pull request #23315: [SPARK-26366][SQL] ReplaceE

[jira] [Issue Comment Deleted] (SPARK-26362) Remove 'spark.driver.allowMultipleContexts' to disallow multiple Spark contexts

2019-01-04 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26362?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-26362: Comment: was deleted (was: asfgit closed pull request #23311: [SPARK-26362][CORE] Remove 'spark.d

[jira] [Updated] (SPARK-26362) Remove 'spark.driver.allowMultipleContexts' to disallow multiple Spark contexts

2019-01-04 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26362?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-26362: Labels: releasenotes (was: ) > Remove 'spark.driver.allowMultipleContexts' to disallow multiple S

[jira] [Updated] (SPARK-26362) Remove 'spark.driver.allowMultipleContexts' to disallow multiple Spark contexts

2019-01-04 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26362?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-26362: -- Priority: Minor (was: Major) > Remove 'spark.driver.allowMultipleContexts' to disallow multiple Spark

[jira] [Updated] (SPARK-26362) Remove 'spark.driver.allowMultipleContexts' to disallow multiple Spark contexts

2019-01-04 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26362?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-26362: -- Docs Text: Support for multiple Spark contexts in the same driver has been removed in Spark 3. > Remo

[jira] [Issue Comment Deleted] (SPARK-26366) Except with transform regression

2019-01-04 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26366?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-26366: Comment: was deleted (was: mgaido91 opened a new pull request #23372: [SPARK-26366][SQL][BACKPORT

[jira] [Commented] (SPARK-26362) Remove 'spark.driver.allowMultipleContexts' to disallow multiple Spark contexts

2019-01-04 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26362?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16734371#comment-16734371 ] Hyukjin Kwon commented on SPARK-26362: -- Sure will do. > Remove 'spark.driver.allow

[jira] [Assigned] (SPARK-26089) Handle large corrupt shuffle blocks

2019-01-04 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26089?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26089: Assignee: (was: Apache Spark) > Handle large corrupt shuffle blocks > ---

[jira] [Assigned] (SPARK-26089) Handle large corrupt shuffle blocks

2019-01-04 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26089?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26089: Assignee: Apache Spark > Handle large corrupt shuffle blocks > --

[jira] [Commented] (SPARK-26254) Move delegation token providers into a separate project

2019-01-04 Thread Gabor Somogyi (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26254?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16734393#comment-16734393 ] Gabor Somogyi commented on SPARK-26254: --- [~hyukjin.kwon] In my last comment right

[jira] [Commented] (SPARK-19809) NullPointerException on zero-size ORC file

2019-01-04 Thread Prashanth Sandela (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16734510#comment-16734510 ] Prashanth Sandela commented on SPARK-19809: --- [~dongjoon] I'm encountering same

[jira] [Commented] (SPARK-19809) NullPointerException on zero-size ORC file

2019-01-04 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16734521#comment-16734521 ] Dongjoon Hyun commented on SPARK-19809: --- Hi, did you use `spark.sql.orc.impl=nativ

[jira] [Commented] (SPARK-19809) NullPointerException on zero-size ORC file

2019-01-04 Thread Prashanth Sandela (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16734523#comment-16734523 ] Prashanth Sandela commented on SPARK-19809: --- Awesome! This config of `spark.sq

[jira] [Comment Edited] (SPARK-19809) NullPointerException on zero-size ORC file

2019-01-04 Thread Prashanth Sandela (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16734510#comment-16734510 ] Prashanth Sandela edited comment on SPARK-19809 at 1/4/19 7:40 PM: ---

[jira] [Created] (SPARK-26537) update the release scripts to point to gitbox

2019-01-04 Thread shane knapp (JIRA)
shane knapp created SPARK-26537: --- Summary: update the release scripts to point to gitbox Key: SPARK-26537 URL: https://issues.apache.org/jira/browse/SPARK-26537 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-26537) update the release scripts to point to gitbox

2019-01-04 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26537?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16734531#comment-16734531 ] Dongjoon Hyun commented on SPARK-26537: --- Thank you for investing this, [~shaneknap

[jira] [Comment Edited] (SPARK-26537) update the release scripts to point to gitbox

2019-01-04 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26537?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16734531#comment-16734531 ] Dongjoon Hyun edited comment on SPARK-26537 at 1/4/19 8:20 PM: ---

[jira] [Assigned] (SPARK-26537) update the release scripts to point to gitbox

2019-01-04 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26537?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26537: Assignee: Apache Spark (was: shane knapp) > update the release scripts to point to gitbo

[jira] [Assigned] (SPARK-26537) update the release scripts to point to gitbox

2019-01-04 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26537?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26537: Assignee: shane knapp (was: Apache Spark) > update the release scripts to point to gitbo

[jira] [Commented] (SPARK-26537) update the release scripts to point to gitbox

2019-01-04 Thread shane knapp (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26537?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16734540#comment-16734540 ] shane knapp commented on SPARK-26537: - [~srowen] i'm pretty sure that my change to t

[jira] [Created] (SPARK-26538) Postgres numeric array support

2019-01-04 Thread Oleksii (JIRA)
Oleksii created SPARK-26538: --- Summary: Postgres numeric array support Key: SPARK-26538 URL: https://issues.apache.org/jira/browse/SPARK-26538 Project: Spark Issue Type: Bug Components: SQ

[jira] [Assigned] (SPARK-26538) Postgres numeric array support

2019-01-04 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26538?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26538: Assignee: Apache Spark > Postgres numeric array support > --

[jira] [Assigned] (SPARK-26538) Postgres numeric array support

2019-01-04 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26538?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26538: Assignee: (was: Apache Spark) > Postgres numeric array support >

  1   2   >