[jira] [Assigned] (SPARK-19168) Improvement: filter late data using watermark for `Append` mode

2017-01-10 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19168?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19168: Assignee: Apache Spark > Improvement: filter late data using watermark for `Append` mode

[jira] [Assigned] (SPARK-19168) Improvement: filter late data using watermark for `Append` mode

2017-01-10 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19168?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19168: Assignee: (was: Apache Spark) > Improvement: filter late data using watermark for

[jira] [Commented] (SPARK-19168) Improvement: filter late data using watermark for `Append` mode

2017-01-10 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19168?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15817559#comment-15817559 ] Apache Spark commented on SPARK-19168: -- User 'lw-lin' has created a pull request for this issue:

[jira] [Updated] (SPARK-19168) Improvement: filter late data using watermark for `Append` mode

2017-01-10 Thread Liwei Lin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19168?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liwei Lin updated SPARK-19168: -- Summary: Improvement: filter late data using watermark for `Append` mode (was: Filter late data using

[jira] [Created] (SPARK-19168) Filter late data using watermark for `Append` mode

2017-01-10 Thread Liwei Lin (JIRA)
Liwei Lin created SPARK-19168: - Summary: Filter late data using watermark for `Append` mode Key: SPARK-19168 URL: https://issues.apache.org/jira/browse/SPARK-19168 Project: Spark Issue Type:

[jira] [Created] (SPARK-19167) .inprogress file can't be renamed if driver exit when OOM occured in AsynchronousListenerBus thread

2017-01-10 Thread xwebber (JIRA)
xwebber created SPARK-19167: --- Summary: .inprogress file can't be renamed if driver exit when OOM occured in AsynchronousListenerBus thread Key: SPARK-19167 URL: https://issues.apache.org/jira/browse/SPARK-19167

[jira] [Assigned] (SPARK-19166) change method name from InsertIntoHadoopFsRelationCommand.deleteMatchingPartitions to InsertIntoHadoopFsRelationCommand.deleteMatchingPrefix

2017-01-10 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19166?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19166: Assignee: (was: Apache Spark) > change method name from >

[jira] [Commented] (SPARK-19166) change method name from InsertIntoHadoopFsRelationCommand.deleteMatchingPartitions to InsertIntoHadoopFsRelationCommand.deleteMatchingPrefix

2017-01-10 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19166?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15817450#comment-15817450 ] Apache Spark commented on SPARK-19166: -- User 'windpiger' has created a pull request for this issue:

[jira] [Assigned] (SPARK-19166) change method name from InsertIntoHadoopFsRelationCommand.deleteMatchingPartitions to InsertIntoHadoopFsRelationCommand.deleteMatchingPrefix

2017-01-10 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19166?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19166: Assignee: Apache Spark > change method name from >

[jira] [Commented] (SPARK-19132) Add test cases for row size estimation

2017-01-10 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19132?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15817434#comment-15817434 ] Reynold Xin commented on SPARK-19132: - We should also fix the comment in

[jira] [Commented] (SPARK-19149) Unify two sets of statistics in LogicalPlan

2017-01-10 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15817425#comment-15817425 ] Apache Spark commented on SPARK-19149: -- User 'rxin' has created a pull request for this issue:

[jira] [Resolved] (SPARK-19149) Unify two sets of statistics in LogicalPlan

2017-01-10 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19149?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-19149. - Resolution: Fixed Assignee: Zhenhua Wang Fix Version/s: 2.2.0 > Unify two sets

[jira] [Created] (SPARK-19166) change method name from InsertIntoHadoopFsRelationCommand.deleteMatchingPartitions to InsertIntoHadoopFsRelationCommand.deleteMatchingPrefix

2017-01-10 Thread Song Jun (JIRA)
Song Jun created SPARK-19166: Summary: change method name from InsertIntoHadoopFsRelationCommand.deleteMatchingPartitions to InsertIntoHadoopFsRelationCommand.deleteMatchingPrefix Key: SPARK-19166 URL:

[jira] [Commented] (SPARK-19133) SparkR glm Gamma family results in error

2017-01-10 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19133?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15817313#comment-15817313 ] Apache Spark commented on SPARK-19133: -- User 'felixcheung' has created a pull request for this

[jira] [Updated] (SPARK-12196) Store/retrieve blocks in different speed storage devices by hierarchy way

2017-01-10 Thread yucai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12196?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] yucai updated SPARK-12196: -- Description: *Motivation* Our customers want to use SSD to speed up machine learning and SQL workload, but

[jira] [Updated] (SPARK-12196) Store/retrieve blocks in different speed storage devices by hierarchy way

2017-01-10 Thread yucai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12196?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] yucai updated SPARK-12196: -- Description: *Motivation* Our customers want to use SSD to speed up machine learning and SQL workload, but

[jira] [Resolved] (SPARK-19157) should be able to change spark.sql.runSQLOnFiles at runtime

2017-01-10 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19157?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-19157. - Resolution: Fixed Fix Version/s: 2.2.0 Issue resolved by pull request 16531

[jira] [Commented] (SPARK-19154) support read and overwrite a same table

2017-01-10 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19154?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15817213#comment-15817213 ] Wenchen Fan commented on SPARK-19154: - it turns out that we still remove all the files before

[jira] [Closed] (SPARK-19154) support read and overwrite a same table

2017-01-10 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19154?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan closed SPARK-19154. --- Resolution: Later > support read and overwrite a same table >

[jira] [Commented] (SPARK-18857) SparkSQL ThriftServer hangs while extracting huge data volumes in incremental collect mode

2017-01-10 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18857?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15817178#comment-15817178 ] Dongjoon Hyun commented on SPARK-18857: --- Or, could you cherry-pick that please? When I try to

[jira] [Commented] (SPARK-18857) SparkSQL ThriftServer hangs while extracting huge data volumes in incremental collect mode

2017-01-10 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18857?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15817170#comment-15817170 ] Dongjoon Hyun commented on SPARK-18857: --- Hi, [~srowen]. This is a bug existing 2.0.2 and 2.1.X.

[jira] [Commented] (SPARK-18113) Sending AskPermissionToCommitOutput failed, driver enter into task deadloop

2017-01-10 Thread jin xing (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18113?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15817067#comment-15817067 ] jin xing commented on SPARK-18113: -- Thanks a lot for your reply~ [~aash]. Your comment is very helpful;

[jira] [Commented] (SPARK-19154) support read and overwrite a same table

2017-01-10 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19154?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15816961#comment-15816961 ] Wenchen Fan commented on SPARK-19154: - Please investigate first, I haven't gone through all of the

[jira] [Commented] (SPARK-19154) support read and overwrite a same table

2017-01-10 Thread Song Jun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19154?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15816942#comment-15816942 ] Song Jun commented on SPARK-19154: -- I am working on this~ > support read and overwrite a same table >

[jira] [Issue Comment Deleted] (SPARK-19090) Dynamic Resource Allocation not respecting spark.executor.cores

2017-01-10 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19090?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-19090: Comment: was deleted (was: Try this, it works for me, my spark version is 2.1.0: {code} spark-sql

[jira] [Comment Edited] (SPARK-12076) countDistinct behaves inconsistently

2017-01-10 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12076?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15816938#comment-15816938 ] Hyukjin Kwon edited comment on SPARK-12076 at 1/11/17 2:44 AM: --- Would you

[jira] [Comment Edited] (SPARK-12076) countDistinct behaves inconsistently

2017-01-10 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12076?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15816938#comment-15816938 ] Hyukjin Kwon edited comment on SPARK-12076 at 1/11/17 2:43 AM: --- Would you

[jira] [Comment Edited] (SPARK-12076) countDistinct behaves inconsistently

2017-01-10 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12076?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15816938#comment-15816938 ] Hyukjin Kwon edited comment on SPARK-12076 at 1/11/17 2:43 AM: --- Would you

[jira] [Commented] (SPARK-12076) countDistinct behaves inconsistently

2017-01-10 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12076?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15816938#comment-15816938 ] Hyukjin Kwon commented on SPARK-12076: -- Would you be able to try this in Spark 2.1? It is painful

[jira] [Comment Edited] (SPARK-19090) Dynamic Resource Allocation not respecting spark.executor.cores

2017-01-10 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19090?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15816920#comment-15816920 ] Yuming Wang edited comment on SPARK-19090 at 1/11/17 2:42 AM: -- Try this, it

[jira] [Commented] (SPARK-19090) Dynamic Resource Allocation not respecting spark.executor.cores

2017-01-10 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19090?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15816920#comment-15816920 ] Yuming Wang commented on SPARK-19090: - Try this, it works for me: {code} sbin/start-thriftserver.sh

[jira] [Commented] (SPARK-19081) spark sql use HIVE UDF throw exception when return a Map value

2017-01-10 Thread Davy Song (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19081?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15816869#comment-15816869 ] Davy Song commented on SPARK-19081: --- The case I used is like this, Insert into a table's map value

[jira] [Commented] (SPARK-19081) spark sql use HIVE UDF throw exception when return a Map value

2017-01-10 Thread Davy Song (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19081?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15816862#comment-15816862 ] Davy Song commented on SPARK-19081: --- Thanks, I'll make a test in a higher version. > spark sql use

[jira] [Commented] (SPARK-19101) Spark Beeline catchs an exeception when run command " load data inpath '/data/test/test.csv' overwrite into table db.test partition(area='021')"

2017-01-10 Thread Xiaochen Ouyang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19101?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15816858#comment-15816858 ] Xiaochen Ouyang commented on SPARK-19101: - thanks,this error is accidental in beeline command. >

[jira] [Updated] (SPARK-19101) Spark Beeline catchs an exeception when run command " load data inpath '/data/test/test.csv' overwrite into table db.test partition(area='021')"

2017-01-10 Thread Xiaochen Ouyang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19101?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaochen Ouyang updated SPARK-19101: Summary: Spark Beeline catchs an exeception when run command " load data inpath

[jira] [Resolved] (SPARK-19140) Allow update mode for non-aggregation streaming queries

2017-01-10 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19140?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu resolved SPARK-19140. -- Resolution: Fixed Fix Version/s: 2.2.0 2.1.1 > Allow update mode for

[jira] [Commented] (SPARK-17993) Spark prints an avalanche of warning messages from Parquet when reading parquet files written by older versions of Parquet-mr

2017-01-10 Thread Michael Allman (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17993?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15816771#comment-15816771 ] Michael Allman commented on SPARK-17993: Cool. I'll work on a simple PR to silence those warnings

[jira] [Updated] (SPARK-19115) SparkSQL unsupports the command " create external table if not exist new_tbl like old_tbl"

2017-01-10 Thread Xiaochen Ouyang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19115?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaochen Ouyang updated SPARK-19115: Summary: SparkSQL unsupports the command " create external table if not exist new_tbl

[jira] [Commented] (SPARK-17993) Spark prints an avalanche of warning messages from Parquet when reading parquet files written by older versions of Parquet-mr

2017-01-10 Thread Emre Colak (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17993?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15816761#comment-15816761 ] Emre Colak commented on SPARK-17993: Editing the log4j.properties helped. Thanks Michael. > Spark

[jira] [Commented] (SPARK-16742) Kerberos support for Spark on Mesos

2017-01-10 Thread Mohammad Kamrul Islam (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15816742#comment-15816742 ] Mohammad Kamrul Islam commented on SPARK-16742: --- any update on this effort? > Kerberos

[jira] [Updated] (SPARK-19150) completely support using hive as data source to create tables

2017-01-10 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19150?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan updated SPARK-19150: Description: After SPARK-19107, we now can treat hive as a data source and create hive tables with

[jira] [Commented] (SPARK-19115) SparkSQL unsupported the command " create external table if not exist new_tbl like old_tbl"

2017-01-10 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19115?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15816679#comment-15816679 ] Hyukjin Kwon commented on SPARK-19115: -- Hi all, I just wonder if it should be resolved as {{Won't

[jira] [Commented] (SPARK-19081) spark sql use HIVE UDF throw exception when return a Map value

2017-01-10 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19081?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15816668#comment-15816668 ] Hyukjin Kwon commented on SPARK-19081: -- Would you be possible to try it in higher version if

[jira] [Commented] (SPARK-13331) AES support for over-the-wire encryption

2017-01-10 Thread Ferdinand Xu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13331?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15816649#comment-15816649 ] Ferdinand Xu commented on SPARK-13331: -- Hi [~jlewandowski], this feature provides an option to

[jira] [Assigned] (SPARK-18905) Potential Issue of Semantics of BatchCompleted

2017-01-10 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18905?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-18905: Assignee: (was: Apache Spark) > Potential Issue of Semantics of BatchCompleted >

[jira] [Commented] (SPARK-18905) Potential Issue of Semantics of BatchCompleted

2017-01-10 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18905?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15816646#comment-15816646 ] Apache Spark commented on SPARK-18905: -- User 'CodingCat' has created a pull request for this issue:

[jira] [Assigned] (SPARK-18905) Potential Issue of Semantics of BatchCompleted

2017-01-10 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18905?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-18905: Assignee: Apache Spark > Potential Issue of Semantics of BatchCompleted >

[jira] [Commented] (SPARK-17993) Spark prints an avalanche of warning messages from Parquet when reading parquet files written by older versions of Parquet-mr

2017-01-10 Thread Michael Allman (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17993?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15816641#comment-15816641 ] Michael Allman commented on SPARK-17993: Also, if that doesn't work, add the following line as

[jira] [Assigned] (SPARK-19088) Optimize sequence type deserialization codegen

2017-01-10 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19088?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19088: Assignee: (was: Apache Spark) > Optimize sequence type deserialization codegen >

[jira] [Assigned] (SPARK-19088) Optimize sequence type deserialization codegen

2017-01-10 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19088?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19088: Assignee: Apache Spark > Optimize sequence type deserialization codegen >

[jira] [Commented] (SPARK-19088) Optimize sequence type deserialization codegen

2017-01-10 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19088?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15816630#comment-15816630 ] Apache Spark commented on SPARK-19088: -- User 'michalsenkyr' has created a pull request for this

[jira] [Commented] (SPARK-17993) Spark prints an avalanche of warning messages from Parquet when reading parquet files written by older versions of Parquet-mr

2017-01-10 Thread Michael Allman (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17993?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15816629#comment-15816629 ] Michael Allman commented on SPARK-17993: Try adding the following line to

[jira] [Comment Edited] (SPARK-19159) PySpark UDF API improvements

2017-01-10 Thread Maciej Szymkiewicz (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19159?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15816557#comment-15816557 ] Maciej Szymkiewicz edited comment on SPARK-19159 at 1/10/17 11:45 PM:

[jira] [Commented] (SPARK-19159) PySpark UDF API improvements

2017-01-10 Thread Maciej Szymkiewicz (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19159?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15816557#comment-15816557 ] Maciej Szymkiewicz commented on SPARK-19159: [~rdblue] There are all independent. You can

[jira] [Commented] (SPARK-18727) Support schema evolution as new files are inserted into table

2017-01-10 Thread Simeon Simeonov (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18727?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15816538#comment-15816538 ] Simeon Simeonov commented on SPARK-18727: - [~xwu0226] A common use case is adding a new field in

[jira] [Commented] (SPARK-19159) PySpark UDF API improvements

2017-01-10 Thread Ryan Blue (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19159?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15816530#comment-15816530 ] Ryan Blue commented on SPARK-19159: --- [~zero323], is there an order to the pull requests? I'll start

[jira] [Updated] (SPARK-19102) Accuracy error of spark SQL results

2017-01-10 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19102?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-19102: - Component/s: (was: Spark Core) > Accuracy error of spark SQL results >

[jira] [Commented] (SPARK-8855) Python API for Association Rules

2017-01-10 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8855?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15816440#comment-15816440 ] Apache Spark commented on SPARK-8855: - User 'aray' has created a pull request for this issue:

[jira] [Assigned] (SPARK-8855) Python API for Association Rules

2017-01-10 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8855?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-8855: --- Assignee: Apache Spark > Python API for Association Rules >

[jira] [Assigned] (SPARK-8855) Python API for Association Rules

2017-01-10 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8855?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-8855: --- Assignee: (was: Apache Spark) > Python API for Association Rules >

[jira] [Updated] (SPARK-19136) Aggregator with case class as output type fails with ClassCastException

2017-01-10 Thread Mathieu D (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19136?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mathieu D updated SPARK-19136: -- Priority: Minor (was: Major) > Aggregator with case class as output type fails with

[jira] [Assigned] (SPARK-19164) Review of UserDefinedFunction._broadcast

2017-01-10 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19164?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19164: Assignee: Apache Spark > Review of UserDefinedFunction._broadcast >

[jira] [Commented] (SPARK-19164) Review of UserDefinedFunction._broadcast

2017-01-10 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15816421#comment-15816421 ] Apache Spark commented on SPARK-19164: -- User 'zero323' has created a pull request for this issue:

[jira] [Commented] (SPARK-19136) Aggregator with case class as output type fails with ClassCastException

2017-01-10 Thread Mathieu D (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19136?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15816422#comment-15816422 ] Mathieu D commented on SPARK-19136: --- And... a RDD version based on treeAggregate is even quicker :-/ At

[jira] [Assigned] (SPARK-19164) Review of UserDefinedFunction._broadcast

2017-01-10 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19164?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19164: Assignee: (was: Apache Spark) > Review of UserDefinedFunction._broadcast >

[jira] [Commented] (SPARK-14280) Update change-version.sh and pom.xml to add Scala 2.12 profiles

2017-01-10 Thread Jakob Odersky (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15816398#comment-15816398 ] Jakob Odersky commented on SPARK-14280: --- Twitter chill for scala 2.12 is finally out and I'm

[jira] [Commented] (SPARK-19136) Aggregator with case class as output type fails with ClassCastException

2017-01-10 Thread Mathieu D (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19136?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15816366#comment-15816366 ] Mathieu D commented on SPARK-19136: --- Both queries are not equivalent, the dummy group generate a

[jira] [Commented] (SPARK-19164) Review of UserDefinedFunction._broadcast

2017-01-10 Thread Maciej Szymkiewicz (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15816361#comment-15816361 ] Maciej Szymkiewicz commented on SPARK-19164: [~rxin] Could you review this? > Review of

[jira] [Updated] (SPARK-19164) Review of UserDefinedFunction._broadcast

2017-01-10 Thread Maciej Szymkiewicz (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19164?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maciej Szymkiewicz updated SPARK-19164: --- Description: It doesn't look like {{UserDefinedFunction._broadcast}} is used at all.

[jira] [Commented] (SPARK-13331) AES support for over-the-wire encryption

2017-01-10 Thread Jacek Lewandowski (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13331?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15816350#comment-15816350 ] Jacek Lewandowski commented on SPARK-13331: --- I wonder where those changes is going to end up.

[jira] [Assigned] (SPARK-19165) UserDefinedFunction should verify call arguments and provide readable exception in case of mismatch

2017-01-10 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19165?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19165: Assignee: (was: Apache Spark) > UserDefinedFunction should verify call arguments and

[jira] [Assigned] (SPARK-19165) UserDefinedFunction should verify call arguments and provide readable exception in case of mismatch

2017-01-10 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19165?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19165: Assignee: Apache Spark > UserDefinedFunction should verify call arguments and provide

[jira] [Commented] (SPARK-19165) UserDefinedFunction should verify call arguments and provide readable exception in case of mismatch

2017-01-10 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19165?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15816341#comment-15816341 ] Apache Spark commented on SPARK-19165: -- User 'zero323' has created a pull request for this issue:

[jira] [Assigned] (SPARK-19163) Lazy creation of the _judf

2017-01-10 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19163?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19163: Assignee: Apache Spark > Lazy creation of the _judf > -- > >

[jira] [Assigned] (SPARK-19163) Lazy creation of the _judf

2017-01-10 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19163?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19163: Assignee: (was: Apache Spark) > Lazy creation of the _judf >

[jira] [Commented] (SPARK-19163) Lazy creation of the _judf

2017-01-10 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19163?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15816318#comment-15816318 ] Apache Spark commented on SPARK-19163: -- User 'zero323' has created a pull request for this issue:

[jira] [Assigned] (SPARK-19162) UserDefinedFunction constructor should verify that func is callable

2017-01-10 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19162?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19162: Assignee: Apache Spark > UserDefinedFunction constructor should verify that func is

[jira] [Commented] (SPARK-19162) UserDefinedFunction constructor should verify that func is callable

2017-01-10 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15816286#comment-15816286 ] Apache Spark commented on SPARK-19162: -- User 'zero323' has created a pull request for this issue:

[jira] [Assigned] (SPARK-19162) UserDefinedFunction constructor should verify that func is callable

2017-01-10 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19162?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19162: Assignee: (was: Apache Spark) > UserDefinedFunction constructor should verify that

[jira] [Created] (SPARK-19165) UserDefinedFunction should verify call arguments and provide readable exception in case of mismatch

2017-01-10 Thread Maciej Szymkiewicz (JIRA)
Maciej Szymkiewicz created SPARK-19165: -- Summary: UserDefinedFunction should verify call arguments and provide readable exception in case of mismatch Key: SPARK-19165 URL:

[jira] [Updated] (SPARK-19162) UserDefinedFunction constructor should verify that func is callable

2017-01-10 Thread Maciej Szymkiewicz (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19162?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maciej Szymkiewicz updated SPARK-19162: --- Description: Current state Right now `UserDefinedFunctions` don't perform any input

[jira] [Issue Comment Deleted] (SPARK-19162) UserDefinedFunction constructor should verify that func is callable

2017-01-10 Thread Maciej Szymkiewicz (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19162?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maciej Szymkiewicz updated SPARK-19162: --- Comment: was deleted (was: This could be further extend to basic validation of the

[jira] [Updated] (SPARK-19162) UserDefinedFunction constructor should verify that func is callable

2017-01-10 Thread Maciej Szymkiewicz (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19162?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maciej Szymkiewicz updated SPARK-19162: --- Description: Current state Right now `UserDefinedFunctions` don't perform any input

[jira] [Updated] (SPARK-19162) UserDefinedFunction constructor should verify that func is callable

2017-01-10 Thread Maciej Szymkiewicz (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19162?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maciej Szymkiewicz updated SPARK-19162: --- Summary: UserDefinedFunction constructor should verify that func is callable (was:

[jira] [Updated] (SPARK-19162) UDF input types validation

2017-01-10 Thread Maciej Szymkiewicz (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19162?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maciej Szymkiewicz updated SPARK-19162: --- Summary: UDF input types validation (was: Input types validation) > UDF input types

[jira] [Assigned] (SPARK-19161) Improving UDF Docstrings

2017-01-10 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19161?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19161: Assignee: (was: Apache Spark) > Improving UDF Docstrings > >

[jira] [Assigned] (SPARK-19161) Improving UDF Docstrings

2017-01-10 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19161?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19161: Assignee: Apache Spark > Improving UDF Docstrings > > >

[jira] [Commented] (SPARK-19162) Input types validation

2017-01-10 Thread Maciej Szymkiewicz (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15816232#comment-15816232 ] Maciej Szymkiewicz commented on SPARK-19162: This could be further extend to basic validation

[jira] [Commented] (SPARK-19161) Improving UDF Docstrings

2017-01-10 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19161?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15816230#comment-15816230 ] Apache Spark commented on SPARK-19161: -- User 'zero323' has created a pull request for this issue:

[jira] [Issue Comment Deleted] (SPARK-18784) Managed memory leak - spark-2.0.2

2017-01-10 Thread Imran Rashid (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18784?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Imran Rashid updated SPARK-18784: - Comment: was deleted (was: That warning is a false alarm when you use {{rdd.take()}}.) >

[jira] [Commented] (SPARK-18784) Managed memory leak - spark-2.0.2

2017-01-10 Thread Imran Rashid (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18784?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15816180#comment-15816180 ] Imran Rashid commented on SPARK-18784: -- That warning is a false alarm when you use {{rdd.take()}}.

[jira] [Commented] (SPARK-19147) netty throw NPE

2017-01-10 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15816170#comment-15816170 ] Shixiong Zhu commented on SPARK-19147: -- I think there is a race condition when the executor is

[jira] [Assigned] (SPARK-19160) Decorator for UDF creation.

2017-01-10 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19160?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19160: Assignee: Apache Spark > Decorator for UDF creation. > --- > >

[jira] [Commented] (SPARK-19160) Decorator for UDF creation.

2017-01-10 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19160?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15816163#comment-15816163 ] Apache Spark commented on SPARK-19160: -- User 'zero323' has created a pull request for this issue:

[jira] [Updated] (SPARK-19161) Improving UDF Docstrings

2017-01-10 Thread Maciej Szymkiewicz (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19161?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maciej Szymkiewicz updated SPARK-19161: --- Affects Version/s: (was: 1.5.0) > Improving UDF Docstrings >

[jira] [Assigned] (SPARK-19160) Decorator for UDF creation.

2017-01-10 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19160?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19160: Assignee: (was: Apache Spark) > Decorator for UDF creation. >

[jira] [Commented] (SPARK-18181) Huge managed memory leak (2.7G) when running reduceByKey

2017-01-10 Thread Imran Rashid (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18181?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15816160#comment-15816160 ] Imran Rashid commented on SPARK-18181: -- [~barrybecker4] I'm pretty sure this is just a misleading

[jira] [Closed] (SPARK-18181) Huge managed memory leak (2.7G) when running reduceByKey

2017-01-10 Thread Imran Rashid (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18181?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Imran Rashid closed SPARK-18181. Resolution: Information Provided > Huge managed memory leak (2.7G) when running reduceByKey >

[jira] [Updated] (SPARK-19163) Lazy creation of the _judf

2017-01-10 Thread Maciej Szymkiewicz (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19163?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maciej Szymkiewicz updated SPARK-19163: --- Affects Version/s: (was: 1.5.0) > Lazy creation of the _judf >

[jira] [Updated] (SPARK-19162) Input types validation

2017-01-10 Thread Maciej Szymkiewicz (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19162?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maciej Szymkiewicz updated SPARK-19162: --- Affects Version/s: (was: 1.5.0) > Input types validation >

  1   2   3   >