[jira] [Closed] (SPARK-5232) CombineFileInputFormatShim#getDirIndices is expensive

[ https://issues.apache.org/jira/browse/SPARK-5232?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jimmy Xiang closed SPARK-5232. -- Resolution: Invalid Wrong project. CombineFileInputFormatShim#getDirIndices is expensive

[jira] [Resolved] (SPARK-5123) Stabilize Spark SQL data type API

[ https://issues.apache.org/jira/browse/SPARK-5123?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-5123. Resolution: Fixed Fix Version/s: 1.3.0 Stabilize Spark SQL data type API

[jira] [Commented] (SPARK-5220) keepPushingBlocks in BlockGenerator terminated when an exception occurs, which causes the block pushing thread to terminate and blocks receiver

[ https://issues.apache.org/jira/browse/SPARK-5220?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14276350#comment-14276350 ] Saisai Shao commented on SPARK-5220: Hi Max, as I said in the mail, this is an

[jira] [Commented] (SPARK-4894) Add Bernoulli-variant of Naive Bayes

[ https://issues.apache.org/jira/browse/SPARK-4894?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14276380#comment-14276380 ] RJ Nowling commented on SPARK-4894: --- Hi @lmcguire, Always happy to have more help! :)

[jira] [Updated] (SPARK-1805) Error launching cluster when master and slaves machines are of different visualization types

[ https://issues.apache.org/jira/browse/SPARK-1805?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicholas Chammas updated SPARK-1805: Issue Type: Bug (was: Improvement) Error launching cluster when master and slaves

[jira] [Updated] (SPARK-1805) Error launching cluster when master and slave machines are of different virtualization types

[ https://issues.apache.org/jira/browse/SPARK-1805?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicholas Chammas updated SPARK-1805: Description: In the current EC2 script, the AMI image object is loaded only once. This

[jira] [Created] (SPARK-5235) java.io.NotSerializableException: org.apache.spark.sql.SQLConf

Alex Baretta created SPARK-5235: --- Summary: java.io.NotSerializableException: org.apache.spark.sql.SQLConf Key: SPARK-5235 URL: https://issues.apache.org/jira/browse/SPARK-5235 Project: Spark

[jira] [Commented] (SPARK-3678) Yarn app name reported in RM is different between cluster and client mode

[ https://issues.apache.org/jira/browse/SPARK-3678?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14276429#comment-14276429 ] WangTaoTheTonic commented on SPARK-3678: In SparkHdfsLR there has {quote}val

[jira] [Created] (SPARK-5234) examples for ml don't have sparkContext.stop

yuhao yang created SPARK-5234: - Summary: examples for ml don't have sparkContext.stop Key: SPARK-5234 URL: https://issues.apache.org/jira/browse/SPARK-5234 Project: Spark Issue Type: Improvement

[jira] [Commented] (SPARK-3821) Develop an automated way of creating Spark images (AMI, Docker, and others)

[ https://issues.apache.org/jira/browse/SPARK-3821?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14276471#comment-14276471 ] Nicholas Chammas commented on SPARK-3821: - [~shivaram] Are we ready to open a PR

[jira] [Commented] (SPARK-3185) SPARK launch on Hadoop 2 in EC2 throws Tachyon exception when Formatting JOURNAL_FOLDER

[ https://issues.apache.org/jira/browse/SPARK-3185?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14276436#comment-14276436 ] Florian Verhein commented on SPARK-3185: I'm also getting this, though with Server

[jira] [Commented] (SPARK-3821) Develop an automated way of creating Spark images (AMI, Docker, and others)

[ https://issues.apache.org/jira/browse/SPARK-3821?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14276505#comment-14276505 ] Shivaram Venkataraman commented on SPARK-3821: -- [~nchammas] Yes -- That

[jira] [Commented] (SPARK-5235) java.io.NotSerializableException: org.apache.spark.sql.SQLConf

[ https://issues.apache.org/jira/browse/SPARK-5235?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14276507#comment-14276507 ] Apache Spark commented on SPARK-5235: - User 'alexbaretta' has created a pull request

[jira] [Updated] (SPARK-5233) Error replay of WAL when recovered from driver failue

[ https://issues.apache.org/jira/browse/SPARK-5233?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao updated SPARK-5233: --- Description: Spark Streaming will write all the event into WAL for driver recovery, the sequence in

[jira] [Created] (SPARK-5236) parquet.io.ParquetDecodingException: Can not read value at 0 in block 0

Alex Baretta created SPARK-5236: --- Summary: parquet.io.ParquetDecodingException: Can not read value at 0 in block 0 Key: SPARK-5236 URL: https://issues.apache.org/jira/browse/SPARK-5236 Project: Spark

[jira] [Commented] (SPARK-5233) Error replay of WAL when recovered from driver failue

[ https://issues.apache.org/jira/browse/SPARK-5233?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14276513#comment-14276513 ] Apache Spark commented on SPARK-5233: - User 'jerryshao' has created a pull request for

[jira] [Created] (SPARK-5237) UDTF don't work on SparK SQL

Yi Zhou created SPARK-5237: -- Summary: UDTF don't work on SparK SQL Key: SPARK-5237 URL: https://issues.apache.org/jira/browse/SPARK-5237 Project: Spark Issue Type: Bug Components: SQL

[jira] [Updated] (SPARK-5233) Error replay of WAL when recovered from driver failue

[ https://issues.apache.org/jira/browse/SPARK-5233?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao updated SPARK-5233: --- Issue Type: Sub-task (was: Bug) Parent: SPARK-5238 Error replay of WAL when recovered from

[jira] [Updated] (SPARK-5237) UDTF don't work on SparK SQL

[ https://issues.apache.org/jira/browse/SPARK-5237?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Zhou updated SPARK-5237: --- Description: Hive query with UDTF don't work on Spark SQL 15/01/14 13:23:50 INFO ParseDriver: Parse Completed

[jira] [Updated] (SPARK-5238) Improve the robustness of Spark Streaming WAL mechanism

[ https://issues.apache.org/jira/browse/SPARK-5238?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao updated SPARK-5238: --- Description: Several issues identified in Spark Streaming's WAL mechanism, this is a cap of all the

[jira] [Created] (SPARK-5238) Improve the robustness of Spark Streaming WAL mechanism

Saisai Shao created SPARK-5238: -- Summary: Improve the robustness of Spark Streaming WAL mechanism Key: SPARK-5238 URL: https://issues.apache.org/jira/browse/SPARK-5238 Project: Spark Issue

[jira] [Updated] (SPARK-5147) write ahead logs from streaming receiver are not purged because cleanupOldBlocks in WriteAheadLogBasedBlockHandler is never called

[ https://issues.apache.org/jira/browse/SPARK-5147?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao updated SPARK-5147: --- Issue Type: Sub-task (was: Bug) Parent: SPARK-5238 write ahead logs from streaming receiver

[jira] [Updated] (SPARK-5142) Possibly data may be ruined in Spark Streaming's WAL mechanism.

[ https://issues.apache.org/jira/browse/SPARK-5142?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao updated SPARK-5142: --- Issue Type: Sub-task (was: Bug) Parent: SPARK-5238 Possibly data may be ruined in Spark

[jira] [Created] (SPARK-5239) JdbcRDD throws java.lang.AbstractMethodError: oracle.jdbc.driver.xxxxxx.isClosed()Z

Gankun Luo created SPARK-5239: - Summary: JdbcRDD throws java.lang.AbstractMethodError: oracle.jdbc.driver.xx.isClosed()Z Key: SPARK-5239 URL: https://issues.apache.org/jira/browse/SPARK-5239 Project:

[jira] [Commented] (SPARK-5239) JdbcRDD throws java.lang.AbstractMethodError: oracle.jdbc.driver.xxxxxx.isClosed()Z

[ https://issues.apache.org/jira/browse/SPARK-5239?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14276540#comment-14276540 ] Apache Spark commented on SPARK-5239: - User 'luogankun' has created a pull request for

[jira] [Updated] (SPARK-5236) java.lang.ClassCastException: org.apache.spark.sql.catalyst.expressions.MutableAny cannot be cast to org.apache.spark.sql.catalyst.expressions.MutableInt

[ https://issues.apache.org/jira/browse/SPARK-5236?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alex Baretta updated SPARK-5236: Summary: java.lang.ClassCastException: org.apache.spark.sql.catalyst.expressions.MutableAny cannot

[jira] [Commented] (SPARK-4923) Maven build should keep publishing spark-repl

[ https://issues.apache.org/jira/browse/SPARK-4923?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14276558#comment-14276558 ] Apache Spark commented on SPARK-4923: - User 'rcsenkbeil' has created a pull request

[jira] [Created] (SPARK-5240) Adding `createDataSourceTable` interface to Catalog

wangfei created SPARK-5240: -- Summary: Adding `createDataSourceTable` interface to Catalog Key: SPARK-5240 URL: https://issues.apache.org/jira/browse/SPARK-5240 Project: Spark Issue Type:

[jira] [Commented] (SPARK-4923) Maven build should keep publishing spark-repl

[ https://issues.apache.org/jira/browse/SPARK-4923?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14276562#comment-14276562 ] Chip Senkbeil commented on SPARK-4923: -- As the nice bot has stated, I created a pull

[jira] [Closed] (SPARK-5006) spark.port.maxRetries doesn't work

[ https://issues.apache.org/jira/browse/SPARK-5006?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or closed SPARK-5006. Resolution: Fixed Fix Version/s: 1.3.0 Assignee: WangTaoTheTonic Target

[jira] [Commented] (SPARK-3288) All fields in TaskMetrics should be private and use getters/setters

[ https://issues.apache.org/jira/browse/SPARK-3288?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14275600#comment-14275600 ] Apache Spark commented on SPARK-3288: - User 'ilganeli' has created a pull request for

[jira] [Created] (SPARK-5223) Use pickle instead of MapConvert and ListConvert in MLlib Python API

Davies Liu created SPARK-5223: - Summary: Use pickle instead of MapConvert and ListConvert in MLlib Python API Key: SPARK-5223 URL: https://issues.apache.org/jira/browse/SPARK-5223 Project: Spark

[jira] [Updated] (SPARK-4697) System properties should override environment variables

[ https://issues.apache.org/jira/browse/SPARK-4697?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-4697: - Affects Version/s: 1.0.0 System properties should override environment variables

[jira] [Updated] (SPARK-5219) Race condition in TaskSchedulerImpl and TaskSetManager

[ https://issues.apache.org/jira/browse/SPARK-5219?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-5219: - Assignee: Shixiong Zhu Race condition in TaskSchedulerImpl and TaskSetManager

[jira] [Updated] (SPARK-5219) Race condition in TaskSchedulerImpl and TaskSetManager

[ https://issues.apache.org/jira/browse/SPARK-5219?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-5219: - Affects Version/s: 1.2.0 Race condition in TaskSchedulerImpl and TaskSetManager

[jira] [Commented] (SPARK-3885) Provide mechanism to remove accumulators once they are no longer used

[ https://issues.apache.org/jira/browse/SPARK-3885?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14275607#comment-14275607 ] Apache Spark commented on SPARK-3885: - User 'ilganeli' has created a pull request for

[jira] [Created] (SPARK-5222) YARN client and cluster modes have different app name behaviors

Andrew Or created SPARK-5222: Summary: YARN client and cluster modes have different app name behaviors Key: SPARK-5222 URL: https://issues.apache.org/jira/browse/SPARK-5222 Project: Spark Issue

[jira] [Updated] (SPARK-5222) YARN client and cluster modes have different app name behaviors

[ https://issues.apache.org/jira/browse/SPARK-5222?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-5222: - Description: The behavior is summarized in a table produced by [~WangTaoTheTonic] here:

[jira] [Commented] (SPARK-5223) Use pickle instead of MapConvert and ListConvert in MLlib Python API

[ https://issues.apache.org/jira/browse/SPARK-5223?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14275671#comment-14275671 ] Apache Spark commented on SPARK-5223: - User 'davies' has created a pull request for

[jira] [Updated] (SPARK-4697) System properties should override environment variables

[ https://issues.apache.org/jira/browse/SPARK-4697?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-4697: - Assignee: WangTaoTheTonic System properties should override environment variables

[jira] [Commented] (SPARK-733) Add documentation on use of accumulators in lazy transformation

[ https://issues.apache.org/jira/browse/SPARK-733?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14275608#comment-14275608 ] Apache Spark commented on SPARK-733: User 'ilganeli' has created a pull request for

[jira] [Updated] (SPARK-5222) YARN client and cluster modes have different app name behaviors

[ https://issues.apache.org/jira/browse/SPARK-5222?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-5222: - Affects Version/s: 1.0.0 YARN client and cluster modes have different app name behaviors

[jira] [Commented] (SPARK-5008) Persistent HDFS does not recognize EBS Volumes

[ https://issues.apache.org/jira/browse/SPARK-5008?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14275586#comment-14275586 ] Brad Willard commented on SPARK-5008: - [~nchammas] I went ahead and created a cluster

[jira] [Updated] (SPARK-5222) YARN client and cluster modes have different app name behaviors

[ https://issues.apache.org/jira/browse/SPARK-5222?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-5222: - Component/s: YARN YARN client and cluster modes have different app name behaviors

[jira] [Updated] (SPARK-4955) Dynamic allocation doesn't work in YARN cluster mode

[ https://issues.apache.org/jira/browse/SPARK-4955?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-4955: - Summary: Dynamic allocation doesn't work in YARN cluster mode (was: Executor does not get killed after

[jira] [Updated] (SPARK-4955) Dynamic allocation doesn't work in YARN cluster mode

[ https://issues.apache.org/jira/browse/SPARK-4955?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-4955: - Priority: Critical (was: Major) Dynamic allocation doesn't work in YARN cluster mode

[jira] [Updated] (SPARK-4955) Dynamic allocation doesn't work in YARN cluster mode

[ https://issues.apache.org/jira/browse/SPARK-4955?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-4955: - Assignee: Lianhui Wang Dynamic allocation doesn't work in YARN cluster mode

[jira] [Created] (SPARK-5242) ec2/spark_ec2.py lauch does not work with VPC if no public DNS or IP is available

Vladimir Grigor created SPARK-5242: -- Summary: ec2/spark_ec2.py lauch does not work with VPC if no public DNS or IP is available Key: SPARK-5242 URL: https://issues.apache.org/jira/browse/SPARK-5242

[jira] [Commented] (SPARK-3821) Develop an automated way of creating Spark images (AMI, Docker, and others)

[ https://issues.apache.org/jira/browse/SPARK-3821?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14276572#comment-14276572 ] Florian Verhein commented on SPARK-3821: Thanks [~nchammas], that makes sense.

[jira] [Commented] (SPARK-5147) write ahead logs from streaming receiver are not purged because cleanupOldBlocks in WriteAheadLogBasedBlockHandler is never called

[ https://issues.apache.org/jira/browse/SPARK-5147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14276581#comment-14276581 ] Apache Spark commented on SPARK-5147: - User 'jerryshao' has created a pull request for

[jira] [Created] (SPARK-5243) Spark will hang if (driver memory + executor memory) exceeds limit on a 1-worker cluster

yuhao yang created SPARK-5243: - Summary: Spark will hang if (driver memory + executor memory) exceeds limit on a 1-worker cluster Key: SPARK-5243 URL: https://issues.apache.org/jira/browse/SPARK-5243

[jira] [Commented] (SPARK-5242) ec2/spark_ec2.py lauch does not work with VPC if no public DNS or IP is available

[ https://issues.apache.org/jira/browse/SPARK-5242?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14276590#comment-14276590 ] Apache Spark commented on SPARK-5242: - User 'voukka' has created a pull request for

[jira] [Commented] (SPARK-5236) java.lang.ClassCastException: org.apache.spark.sql.catalyst.expressions.MutableAny cannot be cast to org.apache.spark.sql.catalyst.expressions.MutableInt

[ https://issues.apache.org/jira/browse/SPARK-5236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14276594#comment-14276594 ] Apache Spark commented on SPARK-5236: - User 'alexbaretta' has created a pull request

[jira] [Commented] (SPARK-5242) ec2/spark_ec2.py lauch does not work with VPC if no public DNS or IP is available

[ https://issues.apache.org/jira/browse/SPARK-5242?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14276591#comment-14276591 ] Vladimir Grigor commented on SPARK-5242: This bug is fixed in

[jira] [Created] (SPARK-5241) spark-ec2 spark init scripts do not handle all hadoop (or tachyon?) dependencies correctly

Florian Verhein created SPARK-5241: -- Summary: spark-ec2 spark init scripts do not handle all hadoop (or tachyon?) dependencies correctly Key: SPARK-5241 URL: https://issues.apache.org/jira/browse/SPARK-5241

[jira] [Commented] (SPARK-5240) Adding `createDataSourceTable` interface to Catalog

[ https://issues.apache.org/jira/browse/SPARK-5240?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14276566#comment-14276566 ] Apache Spark commented on SPARK-5240: - User 'scwf' has created a pull request for this

[jira] [Commented] (SPARK-5095) Support launching multiple mesos executors in coarse grained mesos mode

[ https://issues.apache.org/jira/browse/SPARK-5095?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14276607#comment-14276607 ] Josh Devins commented on SPARK-5095: Nice one, gonna try and test it this week.

[jira] [Created] (SPARK-5224) parallelize list/ndarray is really slow

Davies Liu created SPARK-5224: - Summary: parallelize list/ndarray is really slow Key: SPARK-5224 URL: https://issues.apache.org/jira/browse/SPARK-5224 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-5224) parallelize list/ndarray is really slow

[ https://issues.apache.org/jira/browse/SPARK-5224?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14275711#comment-14275711 ] Apache Spark commented on SPARK-5224: - User 'davies' has created a pull request for

[jira] [Created] (SPARK-5225) Support coalesed Input Metrics from different sources

Kostas Sakellis created SPARK-5225: -- Summary: Support coalesed Input Metrics from different sources Key: SPARK-5225 URL: https://issues.apache.org/jira/browse/SPARK-5225 Project: Spark

[jira] [Commented] (SPARK-2909) Indexing for SparseVector in pyspark

[ https://issues.apache.org/jira/browse/SPARK-2909?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14275741#comment-14275741 ] Apache Spark commented on SPARK-2909: - User 'MechCoder' has created a pull request for

[jira] [Comment Edited] (SPARK-4879) Missing output partitions after job completes with speculative execution

[ https://issues.apache.org/jira/browse/SPARK-4879?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14272291#comment-14272291 ] Zach Fry edited comment on SPARK-4879 at 1/13/15 7:53 PM: -- Hey

[jira] [Updated] (SPARK-5225) Support coalesed Input Metrics from different sources

[ https://issues.apache.org/jira/browse/SPARK-5225?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kostas Sakellis updated SPARK-5225: --- Description: Currently, If task reads data from more than one block and it is from different

[jira] [Commented] (SPARK-5211) Restore HiveMetastoreTypes.toDataType

[ https://issues.apache.org/jira/browse/SPARK-5211?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14275846#comment-14275846 ] Apache Spark commented on SPARK-5211: - User 'yhuai' has created a pull request for

[jira] [Updated] (SPARK-3185) SPARK launch on Hadoop 2 in EC2 throws Tachyon exception when Formatting JOURNAL_FOLDER

[ https://issues.apache.org/jira/browse/SPARK-3185?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicholas Chammas updated SPARK-3185: Description: {code} org.apache.hadoop.ipc.RemoteException: Server IPC version 7 cannot

[jira] [Commented] (SPARK-3821) Develop an automated way of creating Spark images (AMI, Docker, and others)

[ https://issues.apache.org/jira/browse/SPARK-3821?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14276263#comment-14276263 ] Florian Verhein commented on SPARK-3821: This is great stuff! It'll also help

[jira] [Commented] (SPARK-5167) Move Row into sql package and make it usable for Java

[ https://issues.apache.org/jira/browse/SPARK-5167?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14276327#comment-14276327 ] Apache Spark commented on SPARK-5167: - User 'rxin' has created a pull request for this

[jira] [Comment Edited] (SPARK-4894) Add Bernoulli-variant of Naive Bayes

[ https://issues.apache.org/jira/browse/SPARK-4894?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14276380#comment-14276380 ] RJ Nowling edited comment on SPARK-4894 at 1/14/15 2:06 AM: Hi

[jira] [Commented] (SPARK-4296) Throw Expression not in GROUP BY when using same expression in group by clause and select clause

[ https://issues.apache.org/jira/browse/SPARK-4296?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14276278#comment-14276278 ] Cheng Lian commented on SPARK-4296: --- Yeah, I think whenever we use expressions that are

[jira] [Created] (SPARK-5233) Error replay of WAL when recovered from driver failue

Saisai Shao created SPARK-5233: -- Summary: Error replay of WAL when recovered from driver failue Key: SPARK-5233 URL: https://issues.apache.org/jira/browse/SPARK-5233 Project: Spark Issue Type:

[jira] [Commented] (SPARK-3821) Develop an automated way of creating Spark images (AMI, Docker, and others)

[ https://issues.apache.org/jira/browse/SPARK-3821?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14276411#comment-14276411 ] Nicholas Chammas commented on SPARK-3821: - Hi [~florianverhein] and thanks for

[jira] [Commented] (SPARK-5213) Support the SQL Parser Registry

[ https://issues.apache.org/jira/browse/SPARK-5213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14274908#comment-14274908 ] Apache Spark commented on SPARK-5213: - User 'chenghao-intel' has created a pull

[jira] [Created] (SPARK-5213) Support the SQL Parser Registry

Cheng Hao created SPARK-5213: Summary: Support the SQL Parser Registry Key: SPARK-5213 URL: https://issues.apache.org/jira/browse/SPARK-5213 Project: Spark Issue Type: New Feature

[jira] [Commented] (SPARK-1507) Spark on Yarn: Add support for user to specify # cores for ApplicationMaster

[ https://issues.apache.org/jira/browse/SPARK-1507?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14275244#comment-14275244 ] Apache Spark commented on SPARK-1507: - User 'WangTaoTheTonic' has created a pull

[jira] [Created] (SPARK-5219) Race condition in TaskSchedulerImpl and TaskSetManager

Shixiong Zhu created SPARK-5219: --- Summary: Race condition in TaskSchedulerImpl and TaskSetManager Key: SPARK-5219 URL: https://issues.apache.org/jira/browse/SPARK-5219 Project: Spark Issue

[jira] [Commented] (SPARK-5219) Race condition in TaskSchedulerImpl and TaskSetManager

[ https://issues.apache.org/jira/browse/SPARK-5219?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14275284#comment-14275284 ] Apache Spark commented on SPARK-5219: - User 'zsxwing' has created a pull request for

[jira] [Created] (SPARK-5215) concat support in sqlcontext

Adrian Wang created SPARK-5215: -- Summary: concat support in sqlcontext Key: SPARK-5215 URL: https://issues.apache.org/jira/browse/SPARK-5215 Project: Spark Issue Type: New Feature

[jira] [Commented] (SPARK-5215) concat support in sqlcontext

[ https://issues.apache.org/jira/browse/SPARK-5215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14275011#comment-14275011 ] Apache Spark commented on SPARK-5215: - User 'adrian-wang' has created a pull request

[jira] [Created] (SPARK-5218) Report per stage remaining time estimate for each stage.

Prashant Sharma created SPARK-5218: -- Summary: Report per stage remaining time estimate for each stage. Key: SPARK-5218 URL: https://issues.apache.org/jira/browse/SPARK-5218 Project: Spark

[jira] [Updated] (SPARK-5217) Spark UI should report waiting stages during job execution.

[ https://issues.apache.org/jira/browse/SPARK-5217?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prashant Sharma updated SPARK-5217: --- Attachment: waiting_stages.png Spark UI should report waiting stages during job execution.

[jira] [Commented] (SPARK-5124) Standardize internal RPC interface

[ https://issues.apache.org/jira/browse/SPARK-5124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14275014#comment-14275014 ] Shixiong Zhu commented on SPARK-5124: - For 1) I prefer to finish it before this JIRA.

[jira] [Created] (SPARK-5214) Add EventLoop and change DAGScheduler to an EventLoop

Shixiong Zhu created SPARK-5214: --- Summary: Add EventLoop and change DAGScheduler to an EventLoop Key: SPARK-5214 URL: https://issues.apache.org/jira/browse/SPARK-5214 Project: Spark Issue

[jira] [Updated] (SPARK-5216) Spark Ui should report estimated time remaining for each stage.

[ https://issues.apache.org/jira/browse/SPARK-5216?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prashant Sharma updated SPARK-5216: --- Description: Per stage feedback on estimated remaining time can help user get a grasp on how

[jira] [Commented] (SPARK-5214) Add EventLoop and change DAGScheduler to an EventLoop

[ https://issues.apache.org/jira/browse/SPARK-5214?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14275007#comment-14275007 ] Apache Spark commented on SPARK-5214: - User 'zsxwing' has created a pull request for

[jira] [Created] (SPARK-5216) Spark Ui should report estimated time remaining for each stage.

Prashant Sharma created SPARK-5216: -- Summary: Spark Ui should report estimated time remaining for each stage. Key: SPARK-5216 URL: https://issues.apache.org/jira/browse/SPARK-5216 Project: Spark

[jira] [Created] (SPARK-5220) keepPushingBlocks in BlockGenerator terminated when an exception occurs, which causes the block pushing thread to terminate and blocks receiver

Max Xu created SPARK-5220: - Summary: keepPushingBlocks in BlockGenerator terminated when an exception occurs, which causes the block pushing thread to terminate and blocks receiver Key: SPARK-5220 URL:

[jira] [Commented] (SPARK-4794) Wrong parse of GROUP BY query

[ https://issues.apache.org/jira/browse/SPARK-4794?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14275464#comment-14275464 ] Damien Carol commented on SPARK-4794: - [~marmbrus] Sorry for the late answer. For the

[jira] [Created] (SPARK-5221) FileInputDStream remember window in certain situations causes files to be ignored

Jem Tucker created SPARK-5221: - Summary: FileInputDStream remember window in certain situations causes files to be ignored Key: SPARK-5221 URL: https://issues.apache.org/jira/browse/SPARK-5221 Project:

[jira] [Commented] (SPARK-4796) Spark does not remove temp files

[ https://issues.apache.org/jira/browse/SPARK-4796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14275494#comment-14275494 ] Fabian Gebert commented on SPARK-4796: -- suffering from this issue as well and can't

[jira] [Comment Edited] (SPARK-4879) Missing output partitions after job completes with speculative execution

[ https://issues.apache.org/jira/browse/SPARK-4879?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14272291#comment-14272291 ] Zach Fry edited comment on SPARK-4879 at 1/13/15 7:51 PM: -- Hey

[jira] [Commented] (SPARK-5095) Support launching multiple mesos executors in coarse grained mesos mode

[ https://issues.apache.org/jira/browse/SPARK-5095?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14276094#comment-14276094 ] Timothy Chen commented on SPARK-5095: - [~joshdevins][~maasg] I have a PR out now, I

[jira] [Commented] (SPARK-5095) Support launching multiple mesos executors in coarse grained mesos mode

[ https://issues.apache.org/jira/browse/SPARK-5095?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14276093#comment-14276093 ] Apache Spark commented on SPARK-5095: - User 'tnachen' has created a pull request for

[jira] [Commented] (SPARK-5228) Hide tables for Active Jobs/Completed Jobs/Failed Jobs when they are empty

[ https://issues.apache.org/jira/browse/SPARK-5228?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14276109#comment-14276109 ] Apache Spark commented on SPARK-5228: - User 'sarutak' has created a pull request for

[jira] [Updated] (SPARK-5223) Use pickle instead of MapConvert and ListConvert in MLlib Python API

[ https://issues.apache.org/jira/browse/SPARK-5223?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-5223: -- Description: It will introduce problems if the object in dict/list/tuple can not support by py4j, such

[jira] [Commented] (SPARK-5097) Adding data frame APIs to SchemaRDD

[ https://issues.apache.org/jira/browse/SPARK-5097?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14275960#comment-14275960 ] Mohit Jaggi commented on SPARK-5097: minor comment: mutate existing can do df(x) =

[jira] [Commented] (SPARK-5226) Add DBSCAN Clustering Algorithm to MLlib

[ https://issues.apache.org/jira/browse/SPARK-5226?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14275981#comment-14275981 ] Muhammad-Ali A'rabi commented on SPARK-5226: Although I can't assign this task

[jira] [Updated] (SPARK-5179) Spark UI history job duration is wrong

[ https://issues.apache.org/jira/browse/SPARK-5179?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Olivier Toupin updated SPARK-5179: -- Target Version/s: 1.2.1 Spark UI history job duration is wrong

[jira] [Resolved] (SPARK-5223) Use pickle instead of MapConvert and ListConvert in MLlib Python API

[ https://issues.apache.org/jira/browse/SPARK-5223?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng resolved SPARK-5223. -- Resolution: Fixed Fix Version/s: 1.2.1 1.3.0 Issue resolved by pull

[jira] [Updated] (SPARK-5223) Use pickle instead of MapConvert and ListConvert in MLlib Python API

[ https://issues.apache.org/jira/browse/SPARK-5223?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-5223: - Assignee: Davies Liu Use pickle instead of MapConvert and ListConvert in MLlib Python API

[jira] [Resolved] (SPARK-4912) Persistent data source tables

[ https://issues.apache.org/jira/browse/SPARK-4912?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-4912. - Resolution: Fixed Fix Version/s: 1.3.0 Issue resolved by pull request 3960

  1   2   >