[jira] [Updated] (SPARK-12922) Implement gapply() on DataFrame in SparkR

2016-01-19 Thread Sun Rui (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12922?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sun Rui updated SPARK-12922: Description: gapply() applies an R function on groups grouped by one or more columns of a DataFrame, and r

[jira] [Created] (SPARK-12922) Implement gapply() on DataFrame in SparkR

2016-01-19 Thread Sun Rui (JIRA)
Sun Rui created SPARK-12922: --- Summary: Implement gapply() on DataFrame in SparkR Key: SPARK-12922 URL: https://issues.apache.org/jira/browse/SPARK-12922 Project: Spark Issue Type: Sub-task

[jira] [Commented] (SPARK-12921) Use SparkHadoopUtil reflection to access TaskAttemptContext in SpecificParquetRecordReaderBase

2016-01-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12921?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15108184#comment-15108184 ] Apache Spark commented on SPARK-12921: -- User 'JoshRosen' has created a pull request

[jira] [Created] (SPARK-12921) Use SparkHadoopUtil reflection to access TaskAttemptContext in SpecificParquetRecordReaderBase

2016-01-19 Thread Josh Rosen (JIRA)
Josh Rosen created SPARK-12921: -- Summary: Use SparkHadoopUtil reflection to access TaskAttemptContext in SpecificParquetRecordReaderBase Key: SPARK-12921 URL: https://issues.apache.org/jira/browse/SPARK-12921

[jira] [Assigned] (SPARK-12921) Use SparkHadoopUtil reflection to access TaskAttemptContext in SpecificParquetRecordReaderBase

2016-01-19 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12921?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen reassigned SPARK-12921: -- Assignee: Josh Rosen > Use SparkHadoopUtil reflection to access TaskAttemptContext in > Speci

[jira] [Updated] (SPARK-12920) Spark thrift server can run at very high CPU with concurrent users

2016-01-19 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12920?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated SPARK-12920: - Attachment: SPARK-12920.profiler.png SPARK-12920.profiler_job_progress_lis

[jira] [Updated] (SPARK-12919) Implement dapply() on DataFrame in SparkR

2016-01-19 Thread Sun Rui (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12919?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sun Rui updated SPARK-12919: Description: dapply() applies an R function on each partition of a DataFrame and returns a new DataFrame.

[jira] [Created] (SPARK-12920) Spark thrift server can run at very high CPU with concurrent users

2016-01-19 Thread Rajesh Balamohan (JIRA)
Rajesh Balamohan created SPARK-12920: Summary: Spark thrift server can run at very high CPU with concurrent users Key: SPARK-12920 URL: https://issues.apache.org/jira/browse/SPARK-12920 Project: S

[jira] [Created] (SPARK-12919) Implement dapply() on DataFrame in SparkR

2016-01-19 Thread Sun Rui (JIRA)
Sun Rui created SPARK-12919: --- Summary: Implement dapply() on DataFrame in SparkR Key: SPARK-12919 URL: https://issues.apache.org/jira/browse/SPARK-12919 Project: Spark Issue Type: Sub-task

[jira] [Commented] (SPARK-12646) Support _HOST in kerberos principal for connecting to secure cluster

2016-01-19 Thread Hari Krishna Dara (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12646?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15108164#comment-15108164 ] Hari Krishna Dara commented on SPARK-12646: --- Apologies for the delay in getting

[jira] [Comment Edited] (SPARK-12845) During join Spark should pushdown predicates on joining column to both tables

2016-01-19 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12845?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15108158#comment-15108158 ] Xiao Li edited comment on SPARK-12845 at 1/20/16 7:27 AM: -- It su

[jira] [Commented] (SPARK-12845) During join Spark should pushdown predicates on joining column to both tables

2016-01-19 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12845?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15108158#comment-15108158 ] Xiao Li commented on SPARK-12845: - It supports both Inner and Outer Join > During join S

[jira] [Commented] (SPARK-12790) Remove HistoryServer old multiple files format

2016-01-19 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12790?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15108140#comment-15108140 ] Felix Cheung commented on SPARK-12790: -- FsHistoryProviderSuite, which I did change,

[jira] [Commented] (SPARK-12790) Remove HistoryServer old multiple files format

2016-01-19 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12790?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15108138#comment-15108138 ] Felix Cheung commented on SPARK-12790: -- I have made the changes but running tests, I

[jira] [Commented] (SPARK-12845) During join Spark should pushdown predicates on joining column to both tables

2016-01-19 Thread JIRA
[ https://issues.apache.org/jira/browse/SPARK-12845?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15108132#comment-15108132 ] Maciej Bryński commented on SPARK-12845: [~smilegator] I'll try. Is this patch wo

[jira] [Commented] (SPARK-12532) Join-key Pushdown via Predicate Transitivity

2016-01-19 Thread JIRA
[ https://issues.apache.org/jira/browse/SPARK-12532?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15108128#comment-15108128 ] Maciej Bryński commented on SPARK-12532: [~rxin] I don't agree with you. This opt

[jira] [Updated] (SPARK-12624) When schema is specified, we should treat undeclared fields as null (in Python)

2016-01-19 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12624?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-12624: Priority: Blocker (was: Critical) > When schema is specified, we should treat undeclared fields as

[jira] [Commented] (SPARK-12918) Support R SQL UDF in SparkR

2016-01-19 Thread Sun Rui (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12918?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15108110#comment-15108110 ] Sun Rui commented on SPARK-12918: - Spark is now supporting vectorized execution via Colum

[jira] [Closed] (SPARK-12794) Support Defining and Registration of R UDF

2016-01-19 Thread Sun Rui (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12794?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sun Rui closed SPARK-12794. --- Resolution: Later > Support Defining and Registration of R UDF > -- >

[jira] [Closed] (SPARK-12793) Support R UDF Evaluation

2016-01-19 Thread Sun Rui (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12793?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sun Rui closed SPARK-12793. --- Resolution: Later > Support R UDF Evaluation > > > Key: SPARK-12793

[jira] [Updated] (SPARK-12918) Support R SQL UDF in SparkR

2016-01-19 Thread Sun Rui (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12918?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sun Rui updated SPARK-12918: Attachment: SparkR SQL UDF Design Documentation v1.pdf > Support R SQL UDF in SparkR >

[jira] [Updated] (SPARK-12918) Support R SQL UDF in SparkR

2016-01-19 Thread Sun Rui (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12918?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sun Rui updated SPARK-12918: Attachment: (was: SparkR UDF Design Documentation v1.pdf) > Support R SQL UDF in SparkR > -

[jira] [Updated] (SPARK-12918) Support R SQL UDF in SparkR

2016-01-19 Thread Sun Rui (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12918?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sun Rui updated SPARK-12918: Attachment: SparkR UDF Design Documentation v1.pdf > Support R SQL UDF in SparkR >

[jira] [Commented] (SPARK-6817) DataFrame UDFs in R

2016-01-19 Thread Sun Rui (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6817?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15108098#comment-15108098 ] Sun Rui commented on SPARK-6817: I wrote an implementation document at https://docs.googl

[jira] [Commented] (SPARK-7848) Update SparkStreaming docs to incorporate FAQ and/or bullets w/ "knobs" information.

2016-01-19 Thread Nirman Narang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7848?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15108100#comment-15108100 ] Nirman Narang commented on SPARK-7848: -- Started working on this. > Update SparkStrea

[jira] [Commented] (SPARK-6817) DataFrame UDFs in R

2016-01-19 Thread Sun Rui (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6817?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15108097#comment-15108097 ] Sun Rui commented on SPARK-6817: Moved SQL UDF related stuff to SPARK-12918. > DataFrame

[jira] [Updated] (SPARK-6817) DataFrame UDFs in R

2016-01-19 Thread Sun Rui (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6817?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sun Rui updated SPARK-6817: --- Attachment: (was: SparkR UDF Design Documentation v1.pdf) > DataFrame UDFs in R > --- > >

[jira] [Created] (SPARK-12918) Support R SQL UDF in SparkR

2016-01-19 Thread Sun Rui (JIRA)
Sun Rui created SPARK-12918: --- Summary: Support R SQL UDF in SparkR Key: SPARK-12918 URL: https://issues.apache.org/jira/browse/SPARK-12918 Project: Spark Issue Type: New Feature Component

[jira] [Commented] (SPARK-12417) Orc bloom filter options are not propagated during file write in spark

2016-01-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15108085#comment-15108085 ] Apache Spark commented on SPARK-12417: -- User 'rajeshbalamohan' has created a pull re

[jira] [Commented] (SPARK-12904) Strength reduction for integer/decimal comparisons

2016-01-19 Thread Liang-Chi Hsieh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15108020#comment-15108020 ] Liang-Chi Hsieh commented on SPARK-12904: - And also: 5. decimal_literal > int_co

[jira] [Resolved] (SPARK-12871) Support to specify the option for compression codec.

2016-01-19 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12871?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-12871. - Resolution: Fixed Assignee: Hyukjin Kwon Fix Version/s: 2.0.0 > Support to specif

[jira] [Commented] (SPARK-12904) Strength reduction for integer/decimal comparisons

2016-01-19 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15108001#comment-15108001 ] Reynold Xin commented on SPARK-12904: - ah yes. > Strength reduction for integer/dec

[jira] [Commented] (SPARK-12904) Strength reduction for integer/decimal comparisons

2016-01-19 Thread Liang-Chi Hsieh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15107998#comment-15107998 ] Liang-Chi Hsieh commented on SPARK-12904: - The rules should be: 1. int_col > dec

[jira] [Created] (SPARK-12917) Add DML support to Spark SQL for HIVE

2016-01-19 Thread Hemang Nagar (JIRA)
Hemang Nagar created SPARK-12917: Summary: Add DML support to Spark SQL for HIVE Key: SPARK-12917 URL: https://issues.apache.org/jira/browse/SPARK-12917 Project: Spark Issue Type: New Feature

[jira] [Updated] (SPARK-12910) Support for specifying version of R to use while creating sparkR libraries

2016-01-19 Thread Shubhanshu Mishra (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12910?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shubhanshu Mishra updated SPARK-12910: -- Description: When we use `$SPARK_HOME/R/install-dev.sh` it uses the default system R.

[jira] [Created] (SPARK-12916) Support Row.fromSeq and Row.toSeq methods in pyspark

2016-01-19 Thread Shubhanshu Mishra (JIRA)
Shubhanshu Mishra created SPARK-12916: - Summary: Support Row.fromSeq and Row.toSeq methods in pyspark Key: SPARK-12916 URL: https://issues.apache.org/jira/browse/SPARK-12916 Project: Spark

[jira] [Created] (SPARK-12915) SQL metrics for generated operators

2016-01-19 Thread Davies Liu (JIRA)
Davies Liu created SPARK-12915: -- Summary: SQL metrics for generated operators Key: SPARK-12915 URL: https://issues.apache.org/jira/browse/SPARK-12915 Project: Spark Issue Type: Improvement

[jira] [Commented] (SPARK-12669) Organize options for default values

2016-01-19 Thread Hossein Falaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12669?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15107892#comment-15107892 ] Hossein Falaki commented on SPARK-12669: If/when Spark Data sources API supports

[jira] [Commented] (SPARK-12669) Organize options for default values

2016-01-19 Thread Mohit Jaggi (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12669?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15107887#comment-15107887 ] Mohit Jaggi commented on SPARK-12669: - hmm...wouldn't it be good to have a typesafe A

[jira] [Commented] (SPARK-12635) More efficient (column batch) serialization for Python/R

2016-01-19 Thread Shivaram Venkataraman (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12635?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15107860#comment-15107860 ] Shivaram Venkataraman commented on SPARK-12635: --- Just to clarify a couple o

[jira] [Commented] (SPARK-12864) initialize executorIdCounter after ApplicationMaster killed for max number of executor failures reached

2016-01-19 Thread iward (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12864?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15107859#comment-15107859 ] iward commented on SPARK-12864: --- The important point of the idea is to fix this conflict ex

[jira] [Updated] (SPARK-12232) Create new R API for read.table to avoid conflict

2016-01-19 Thread Shivaram Venkataraman (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12232?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shivaram Venkataraman updated SPARK-12232: -- Assignee: Felix Cheung > Create new R API for read.table to avoid conflict > --

[jira] [Resolved] (SPARK-12232) Create new R API for read.table to avoid conflict

2016-01-19 Thread Shivaram Venkataraman (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12232?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shivaram Venkataraman resolved SPARK-12232. --- Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request

[jira] [Commented] (SPARK-12864) initialize executorIdCounter after ApplicationMaster killed for max number of executor failures reached

2016-01-19 Thread Mao, Wei (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12864?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15107812#comment-15107812 ] Mao, Wei commented on SPARK-12864: -- I don't quite get your point here. With your new exe

[jira] [Updated] (SPARK-10620) Look into whether accumulator mechanism can replace TaskMetrics

2016-01-19 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10620?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-10620: -- Attachment: accums-and-task-metrics.pdf > Look into whether accumulator mechanism can replace TaskMetri

[jira] [Updated] (SPARK-10620) Look into whether accumulator mechanism can replace TaskMetrics

2016-01-19 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10620?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-10620: -- Attachment: (was: AccumulatorsandTaskMetricsinSpark2.0.pdf) > Look into whether accumulator mechani

[jira] [Updated] (SPARK-10620) Look into whether accumulator mechanism can replace TaskMetrics

2016-01-19 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10620?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-10620: -- Attachment: AccumulatorsandTaskMetricsinSpark2.0.pdf > Look into whether accumulator mechanism can repl

[jira] [Updated] (SPARK-12898) Consider having dummyCallSite for HiveTableScan

2016-01-19 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12898?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated SPARK-12898: - Attachment: callsiteProf.png > Consider having dummyCallSite for HiveTableScan >

[jira] [Updated] (SPARK-12898) Consider having dummyCallSite for HiveTableScan

2016-01-19 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12898?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated SPARK-12898: - Attachment: (was: callsiteProf) > Consider having dummyCallSite for HiveTableScan > -

[jira] [Updated] (SPARK-12898) Consider having dummyCallSite for HiveTableScan

2016-01-19 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12898?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated SPARK-12898: - Attachment: callsiteProf > Consider having dummyCallSite for HiveTableScan >

[jira] [Updated] (SPARK-12337) Implement dropDuplicates() method of DataFrame in SparkR

2016-01-19 Thread Shivaram Venkataraman (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12337?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shivaram Venkataraman updated SPARK-12337: -- Assignee: Sun Rui > Implement dropDuplicates() method of DataFrame in SparkR >

[jira] [Resolved] (SPARK-12337) Implement dropDuplicates() method of DataFrame in SparkR

2016-01-19 Thread Shivaram Venkataraman (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12337?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shivaram Venkataraman resolved SPARK-12337. --- Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request

[jira] [Updated] (SPARK-12168) Need test for conflicted function in R

2016-01-19 Thread Shivaram Venkataraman (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12168?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shivaram Venkataraman updated SPARK-12168: -- Assignee: Felix Cheung > Need test for conflicted function in R > -

[jira] [Resolved] (SPARK-12168) Need test for conflicted function in R

2016-01-19 Thread Shivaram Venkataraman (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12168?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shivaram Venkataraman resolved SPARK-12168. --- Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request

[jira] [Resolved] (SPARK-12770) Implement rules for branch elimination for CaseWhen in SimplifyConditionals

2016-01-19 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12770?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-12770. - Resolution: Fixed Fix Version/s: 2.0.0 > Implement rules for branch elimination for CaseWh

[jira] [Updated] (SPARK-8514) LU factorization on BlockMatrix

2016-01-19 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8514?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-8514: - Target Version/s: 2.0.0 (was: ) > LU factorization on BlockMatrix > -

[jira] [Updated] (SPARK-6519) Add spark.ml API for bisecting k-means

2016-01-19 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6519?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-6519: - Target Version/s: 2.0.0 (was: ) > Add spark.ml API for bisecting k-means > --

[jira] [Commented] (SPARK-12469) Consistent Accumulators for Spark

2016-01-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12469?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15107674#comment-15107674 ] Apache Spark commented on SPARK-12469: -- User 'holdenk' has created a pull request fo

[jira] [Assigned] (SPARK-12469) Consistent Accumulators for Spark

2016-01-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12469?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-12469: Assignee: (was: Apache Spark) > Consistent Accumulators for Spark > --

[jira] [Assigned] (SPARK-12469) Consistent Accumulators for Spark

2016-01-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12469?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-12469: Assignee: Apache Spark > Consistent Accumulators for Spark > -

[jira] [Commented] (SPARK-12177) Update KafkaDStreams to new Kafka 0.9 Consumer API

2016-01-19 Thread Mark Grover (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12177?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15107669#comment-15107669 ] Mark Grover commented on SPARK-12177: - Posting an update. Took out Broker.scala, the

[jira] [Updated] (SPARK-12913) Reimplement all builtin aggregate functions as declarative function

2016-01-19 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12913?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-12913: --- Description: As benchmarked and discussed here: https://github.com/apache/spark/pull/10786/files#r50

[jira] [Created] (SPARK-12914) Generate TungstenAggregate with grouping keys

2016-01-19 Thread Davies Liu (JIRA)
Davies Liu created SPARK-12914: -- Summary: Generate TungstenAggregate with grouping keys Key: SPARK-12914 URL: https://issues.apache.org/jira/browse/SPARK-12914 Project: Spark Issue Type: Improve

[jira] [Created] (SPARK-12913) Reimplement all builtin aggregate functions as declarative function

2016-01-19 Thread Davies Liu (JIRA)
Davies Liu created SPARK-12913: -- Summary: Reimplement all builtin aggregate functions as declarative function Key: SPARK-12913 URL: https://issues.apache.org/jira/browse/SPARK-12913 Project: Spark

[jira] [Assigned] (SPARK-12797) Aggregation without grouping keys

2016-01-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12797?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-12797: Assignee: Apache Spark > Aggregation without grouping keys > -

[jira] [Assigned] (SPARK-12797) Aggregation without grouping keys

2016-01-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12797?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-12797: Assignee: (was: Apache Spark) > Aggregation without grouping keys > --

[jira] [Commented] (SPARK-12797) Aggregation without grouping keys

2016-01-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12797?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15107645#comment-15107645 ] Apache Spark commented on SPARK-12797: -- User 'davies' has created a pull request for

[jira] [Updated] (SPARK-6166) Add config to limit number of concurrent outbound connections for shuffle fetch

2016-01-19 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6166?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves updated SPARK-6166: - Assignee: (was: Shixiong Zhu) > Add config to limit number of concurrent outbound connections

[jira] [Resolved] (SPARK-9716) BinaryClassificationEvaluator should accept Double prediction column

2016-01-19 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9716?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley resolved SPARK-9716. -- Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 10472 [http

[jira] [Resolved] (SPARK-2750) Add Https support for Web UI

2016-01-19 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2750?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-2750. --- Resolution: Fixed Assignee: Fei Wang Fix Version/s: 2.0.0 > Add Https support

[jira] [Commented] (SPARK-12912) Add test suite for EliminateSubQueries

2016-01-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12912?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15107606#comment-15107606 ] Apache Spark commented on SPARK-12912: -- User 'rxin' has created a pull request for t

[jira] [Created] (SPARK-12912) Add test suite for EliminateSubQueries

2016-01-19 Thread Reynold Xin (JIRA)
Reynold Xin created SPARK-12912: --- Summary: Add test suite for EliminateSubQueries Key: SPARK-12912 URL: https://issues.apache.org/jira/browse/SPARK-12912 Project: Spark Issue Type: Improvement

[jira] [Assigned] (SPARK-12912) Add test suite for EliminateSubQueries

2016-01-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12912?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-12912: Assignee: Apache Spark (was: Reynold Xin) > Add test suite for EliminateSubQueries >

[jira] [Assigned] (SPARK-12912) Add test suite for EliminateSubQueries

2016-01-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12912?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-12912: Assignee: Reynold Xin (was: Apache Spark) > Add test suite for EliminateSubQueries >

[jira] [Commented] (SPARK-12845) During join Spark should pushdown predicates on joining column to both tables

2016-01-19 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12845?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15107603#comment-15107603 ] Xiao Li commented on SPARK-12845: - Let me know if you hit any bug. Thanks! > During join

[jira] [Commented] (SPARK-12845) During join Spark should pushdown predicates on joining column to both tables

2016-01-19 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12845?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15107600#comment-15107600 ] Xiao Li commented on SPARK-12845: - I think the following PR resolves your issue: https:/

[jira] [Updated] (SPARK-11295) Add packages to JUnit output for Python tests

2016-01-19 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11295?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-11295: -- Component/s: PySpark > Add packages to JUnit output for Python tests >

[jira] [Updated] (SPARK-11295) Add packages to JUnit output for Python tests

2016-01-19 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11295?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-11295: -- Assignee: Gabor Liptak > Add packages to JUnit output for Python tests > --

[jira] [Updated] (SPARK-11295) Add packages to JUnit output for Python tests

2016-01-19 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11295?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-11295: -- Target Version/s: 2.0.0 > Add packages to JUnit output for Python tests > -

[jira] [Resolved] (SPARK-11295) Add packages to JUnit output for Python tests

2016-01-19 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11295?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng resolved SPARK-11295. --- Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 9263 [https://gi

[jira] [Assigned] (SPARK-12479) sparkR collect on GroupedData throws R error "missing value where TRUE/FALSE needed"

2016-01-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12479?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-12479: Assignee: (was: Apache Spark) > sparkR collect on GroupedData throws R error "missin

[jira] [Assigned] (SPARK-12479) sparkR collect on GroupedData throws R error "missing value where TRUE/FALSE needed"

2016-01-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12479?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-12479: Assignee: Apache Spark > sparkR collect on GroupedData throws R error "missing value whe

[jira] [Assigned] (SPARK-12869) Optimize conversion from BlockMatrix to IndexedRowMatrix

2016-01-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12869?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-12869: Assignee: Apache Spark > Optimize conversion from BlockMatrix to IndexedRowMatrix > --

[jira] [Assigned] (SPARK-12869) Optimize conversion from BlockMatrix to IndexedRowMatrix

2016-01-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12869?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-12869: Assignee: (was: Apache Spark) > Optimize conversion from BlockMatrix to IndexedRowMatr

[jira] [Commented] (SPARK-12869) Optimize conversion from BlockMatrix to IndexedRowMatrix

2016-01-19 Thread Fokko Driesprong (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12869?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15107533#comment-15107533 ] Fokko Driesprong commented on SPARK-12869: -- Hi guys, I've implemented an improv

[jira] [Commented] (SPARK-12869) Optimize conversion from BlockMatrix to IndexedRowMatrix

2016-01-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12869?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15107534#comment-15107534 ] Apache Spark commented on SPARK-12869: -- User 'Fokko' has created a pull request for

[jira] [Commented] (SPARK-6166) Add config to limit number of concurrent outbound connections for shuffle fetch

2016-01-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6166?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15107526#comment-15107526 ] Apache Spark commented on SPARK-6166: - User 'redsanket' has created a pull request for

[jira] [Created] (SPARK-12911) Cacheing a dataframe causes array comparisons to fail (in filter / where) after 1.6

2016-01-19 Thread Jesse English (JIRA)
Jesse English created SPARK-12911: - Summary: Cacheing a dataframe causes array comparisons to fail (in filter / where) after 1.6 Key: SPARK-12911 URL: https://issues.apache.org/jira/browse/SPARK-12911

[jira] [Assigned] (SPARK-10777) order by fails when column is aliased and projection includes windowed aggregate

2016-01-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10777?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-10777: Assignee: (was: Apache Spark) > order by fails when column is aliased and projection i

[jira] [Commented] (SPARK-10777) order by fails when column is aliased and projection includes windowed aggregate

2016-01-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10777?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15107434#comment-15107434 ] Apache Spark commented on SPARK-10777: -- User 'gatorsmile' has created a pull request

[jira] [Assigned] (SPARK-10777) order by fails when column is aliased and projection includes windowed aggregate

2016-01-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10777?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-10777: Assignee: Apache Spark > order by fails when column is aliased and projection includes win

[jira] [Resolved] (SPARK-12816) Schema generation for type aliases does not work

2016-01-19 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12816?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-12816. -- Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 10749 [htt

[jira] [Updated] (SPARK-12560) SqlTestUtils.stripSparkFilter needs to copy utf8strings

2016-01-19 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12560?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-12560: --- Fix Version/s: 2.0.0 > SqlTestUtils.stripSparkFilter needs to copy utf8strings >

[jira] [Commented] (SPARK-12910) Support for specifying version of R to use while creating sparkR libraries

2016-01-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12910?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15107369#comment-15107369 ] Apache Spark commented on SPARK-12910: -- User 'napsternxg' has created a pull request

[jira] [Resolved] (SPARK-12560) SqlTestUtils.stripSparkFilter needs to copy utf8strings

2016-01-19 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12560?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian resolved SPARK-12560. Resolution: Fixed Assignee: Imran Rashid Resolved by https://github.com/apache/spark/pull/105

[jira] [Assigned] (SPARK-12910) Support for specifying version of R to use while creating sparkR libraries

2016-01-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12910?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-12910: Assignee: (was: Apache Spark) > Support for specifying version of R to use while creat

[jira] [Assigned] (SPARK-12910) Support for specifying version of R to use while creating sparkR libraries

2016-01-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12910?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-12910: Assignee: Apache Spark > Support for specifying version of R to use while creating sparkR

[jira] [Commented] (SPARK-12910) Support for specifying version of R to use while creating sparkR libraries

2016-01-19 Thread Shubhanshu Mishra (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12910?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15107370#comment-15107370 ] Shubhanshu Mishra commented on SPARK-12910: --- I have created a pull request at h

[jira] [Created] (SPARK-12910) Support for specifying version of R to use while creating sparkR libraries

2016-01-19 Thread Shubhanshu Mishra (JIRA)
Shubhanshu Mishra created SPARK-12910: - Summary: Support for specifying version of R to use while creating sparkR libraries Key: SPARK-12910 URL: https://issues.apache.org/jira/browse/SPARK-12910

[jira] [Assigned] (SPARK-12895) Implement TaskMetrics using accumulators

2016-01-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12895?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-12895: Assignee: Apache Spark (was: Andrew Or) > Implement TaskMetrics using accumulators >

  1   2   >