[jira] [Commented] (SPARK-3226) Doc update for MLlib dependencies

[ https://issues.apache.org/jira/browse/SPARK-3226?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14110335#comment-14110335 ] Apache Spark commented on SPARK-3226: - User 'mengxr' has created a pull request for

[jira] [Commented] (SPARK-2344) Add Fuzzy C-Means algorithm to MLlib

[ https://issues.apache.org/jira/browse/SPARK-2344?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14110342#comment-14110342 ] Alex commented on SPARK-2344: - Hi, I'm currently working on the implementation of FCM myself.

[jira] [Updated] (SPARK-3223) runAsSparkUser cannot change HDFS write permission properly in mesos cluster mode

[ https://issues.apache.org/jira/browse/SPARK-3223?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-3223: --- Priority: Critical (was: Major) runAsSparkUser cannot change HDFS write permission

[jira] [Updated] (SPARK-3224) FetchFailed stages could show up multiple times in failed stages in web ui

[ https://issues.apache.org/jira/browse/SPARK-3224?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-3224: --- Priority: Blocker (was: Critical) FetchFailed stages could show up multiple times in

[jira] [Commented] (SPARK-3213) spark_ec2.py cannot find slave instances launched with Launch More Like This

[ https://issues.apache.org/jira/browse/SPARK-3213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14110353#comment-14110353 ] Patrick Wendell commented on SPARK-3213: Hey I don't think we previously supported

[jira] [Updated] (SPARK-3213) spark_ec2.py cannot find slave instances launched with Launch More Like This

[ https://issues.apache.org/jira/browse/SPARK-3213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-3213: --- Issue Type: Improvement (was: Bug) spark_ec2.py cannot find slave instances launched with

[jira] [Updated] (SPARK-3178) setting SPARK_WORKER_MEMORY to a value without a label (m or g) sets the worker memory limit to zero

[ https://issues.apache.org/jira/browse/SPARK-3178?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-3178: --- Labels: starter (was: ) setting SPARK_WORKER_MEMORY to a value without a label (m or g)

[jira] [Updated] (SPARK-3145) Hive on Spark umbrella

[ https://issues.apache.org/jira/browse/SPARK-3145?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-3145: --- Summary: Hive on Spark umbrella (was: Hive on Spark dependency umbrella) Hive on Spark

[jira] [Commented] (SPARK-3145) Hive on Spark umbrella

[ https://issues.apache.org/jira/browse/SPARK-3145?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14110362#comment-14110362 ] Patrick Wendell commented on SPARK-3145: [~bcwalrus] hey BC I made a minor change

[jira] [Assigned] (SPARK-2886) Use more specific actor system name than spark

[ https://issues.apache.org/jira/browse/SPARK-2886?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or reassigned SPARK-2886: Assignee: Andrew Or Use more specific actor system name than spark

[jira] [Resolved] (SPARK-2886) Use more specific actor system name than spark

[ https://issues.apache.org/jira/browse/SPARK-2886?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or resolved SPARK-2886. -- Resolution: Fixed Fixed by https://github.com/apache/spark/pull/1810 Use more specific actor system

[jira] [Commented] (SPARK-3167) Port recent spark-submit changes to windows

[ https://issues.apache.org/jira/browse/SPARK-3167?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14110382#comment-14110382 ] Apache Spark commented on SPARK-3167: - User 'andrewor14' has created a pull request

[jira] [Created] (SPARK-3228) When DStream save RDD to hdfs , don't create directory and empty file if there are no data received from source in the batch duration .

Leo created SPARK-3228: -- Summary: When DStream save RDD to hdfs , don't create directory and empty file if there are no data received from source in the batch duration . Key: SPARK-3228 URL:

[jira] [Commented] (SPARK-3170) Bug Fix in Storage UI

[ https://issues.apache.org/jira/browse/SPARK-3170?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14110421#comment-14110421 ] Apache Spark commented on SPARK-3170: - User 'uncleGen' has created a pull request for

[jira] [Commented] (SPARK-3228) When DStream save RDD to hdfs , don't create directory and empty file if there are no data received from source in the batch duration .

[ https://issues.apache.org/jira/browse/SPARK-3228?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14110442#comment-14110442 ] Apache Spark commented on SPARK-3228: - User 'Leolh' has created a pull request for

[jira] [Commented] (SPARK-3098) In some cases, operation zipWithIndex get a wrong results

[ https://issues.apache.org/jira/browse/SPARK-3098?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14110463#comment-14110463 ] Sean Owen commented on SPARK-3098: -- [~matei] The question isn't whether distinct returns

[jira] [Commented] (SPARK-2344) Add Fuzzy C-Means algorithm to MLlib

[ https://issues.apache.org/jira/browse/SPARK-2344?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14110518#comment-14110518 ] Alex commented on SPARK-2344: - this is my branch: https://github.com/salexln/spark i do not

[jira] [Assigned] (SPARK-3200) Class defined with reference to external variables crashes in REPL.

[ https://issues.apache.org/jira/browse/SPARK-3200?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prashant Sharma reassigned SPARK-3200: -- Assignee: Prashant Sharma Class defined with reference to external variables crashes

[jira] [Created] (SPARK-3229) spark.shuffle.safetyFraction and spark.storage.safetyFraction is not documented

Kousuke Saruta created SPARK-3229: - Summary: spark.shuffle.safetyFraction and spark.storage.safetyFraction is not documented Key: SPARK-3229 URL: https://issues.apache.org/jira/browse/SPARK-3229

[jira] [Created] (SPARK-3230) UDFs that return structs result in ClassCastException

Michael Armbrust created SPARK-3230: --- Summary: UDFs that return structs result in ClassCastException Key: SPARK-3230 URL: https://issues.apache.org/jira/browse/SPARK-3230 Project: Spark

[jira] [Commented] (SPARK-3230) UDFs that return structs result in ClassCastException

[ https://issues.apache.org/jira/browse/SPARK-3230?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14110562#comment-14110562 ] Apache Spark commented on SPARK-3230: - User 'marmbrus' has created a pull request for

[jira] [Created] (SPARK-3231) select on a table in parquet format containing smallest as a field type does not work

chirag aggarwal created SPARK-3231: -- Summary: select on a table in parquet format containing smallest as a field type does not work Key: SPARK-3231 URL: https://issues.apache.org/jira/browse/SPARK-3231

[jira] [Updated] (SPARK-3231) select on a table in parquet format containing smallint as a field type does not work

[ https://issues.apache.org/jira/browse/SPARK-3231?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] chirag aggarwal updated SPARK-3231: --- Summary: select on a table in parquet format containing smallint as a field type does not

[jira] [Commented] (SPARK-3000) drop old blocks to disk in parallel when memory is not large enough for caching new blocks

[ https://issues.apache.org/jira/browse/SPARK-3000?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14110586#comment-14110586 ] Apache Spark commented on SPARK-3000: - User 'liyezhang556520' has created a pull

[jira] [Commented] (SPARK-3229) spark.shuffle.safetyFraction and spark.storage.safetyFraction is not documented

[ https://issues.apache.org/jira/browse/SPARK-3229?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14110593#comment-14110593 ] Apache Spark commented on SPARK-3229: - User 'sarutak' has created a pull request for

[jira] [Commented] (SPARK-2693) Support for UDAF Hive Aggregates like PERCENTILE

[ https://issues.apache.org/jira/browse/SPARK-2693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14110707#comment-14110707 ] Ravindra Pesala commented on SPARK-2693: UDAF is deprecated in HIve, Though there

[jira] [Comment Edited] (SPARK-2693) Support for UDAF Hive Aggregates like PERCENTILE

[ https://issues.apache.org/jira/browse/SPARK-2693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14110707#comment-14110707 ] Ravindra Pesala edited comment on SPARK-2693 at 8/26/14 2:05 PM:

[jira] [Created] (SPARK-3232) Backport SPARK-3006 into branch-1.0

Kousuke Saruta created SPARK-3232: - Summary: Backport SPARK-3006 into branch-1.0 Key: SPARK-3232 URL: https://issues.apache.org/jira/browse/SPARK-3232 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-2541) Standalone mode can't access secure HDFS anymore

[ https://issues.apache.org/jira/browse/SPARK-2541?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14110730#comment-14110730 ] Thomas Graves commented on SPARK-2541: -- So I haven't had time to full investigate

[jira] [Commented] (SPARK-3006) Failed to execute spark-shell in Windows OS

[ https://issues.apache.org/jira/browse/SPARK-3006?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14110733#comment-14110733 ] Apache Spark commented on SPARK-3006: - User 'sarutak' has created a pull request for

[jira] [Updated] (SPARK-2541) Standalone mode can't access secure HDFS anymore

[ https://issues.apache.org/jira/browse/SPARK-2541?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves updated SPARK-2541: - Attachment: SPARK-2541-partial.patch Standalone mode can't access secure HDFS anymore

[jira] [Commented] (SPARK-3216) Spark-shell is broken for branch-1.0

[ https://issues.apache.org/jira/browse/SPARK-3216?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14110731#comment-14110731 ] Apache Spark commented on SPARK-3216: - User 'sarutak' has created a pull request for

[jira] [Commented] (SPARK-3232) Backport SPARK-3006 into branch-1.0

[ https://issues.apache.org/jira/browse/SPARK-3232?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14110732#comment-14110732 ] Apache Spark commented on SPARK-3232: - User 'sarutak' has created a pull request for

[jira] [Comment Edited] (SPARK-2541) Standalone mode can't access secure HDFS anymore

[ https://issues.apache.org/jira/browse/SPARK-2541?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14110730#comment-14110730 ] Thomas Graves edited comment on SPARK-2541 at 8/26/14 2:11 PM:

[jira] [Commented] (SPARK-2189) Method for removing temp tables created by registerAsTable

[ https://issues.apache.org/jira/browse/SPARK-2189?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14110740#comment-14110740 ] Venkata Ramana G commented on SPARK-2189: - unregisterTempTable(cachedTableName)

[jira] [Created] (SPARK-3233) Executor never stop its SparnEnv, BlockManager, ConnectionManager etc.

Kousuke Saruta created SPARK-3233: - Summary: Executor never stop its SparnEnv, BlockManager, ConnectionManager etc. Key: SPARK-3233 URL: https://issues.apache.org/jira/browse/SPARK-3233 Project:

[jira] [Updated] (SPARK-3233) Executor never stop its SparnEnv, BlockManager, ConnectionManager etc.

[ https://issues.apache.org/jira/browse/SPARK-3233?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta updated SPARK-3233: -- Description: Executor never stop its SparnEnv. Because of this, following component in

[jira] [Commented] (SPARK-3233) Executor never stop its SparnEnv, BlockManager, ConnectionManager etc.

[ https://issues.apache.org/jira/browse/SPARK-3233?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14110832#comment-14110832 ] Apache Spark commented on SPARK-3233: - User 'sarutak' has created a pull request for

[jira] [Commented] (SPARK-1647) Prevent data loss when Streaming driver goes down

[ https://issues.apache.org/jira/browse/SPARK-1647?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14110942#comment-14110942 ] Giulio De Vecchi commented on SPARK-1647: - Not sure if this make sense, but maybe

[jira] [Commented] (SPARK-2377) Create a Python API for Spark Streaming

[ https://issues.apache.org/jira/browse/SPARK-2377?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=1409#comment-1409 ] Matthew Farrellee commented on SPARK-2377: -- is this still in progress? is the

[jira] [Commented] (SPARK-1284) pyspark hangs after IOError on Executor

[ https://issues.apache.org/jira/browse/SPARK-1284?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=1412#comment-1412 ] Matthew Farrellee commented on SPARK-1284: -- [~jblomo] master should be buildable

[jira] [Commented] (SPARK-3217) Shaded Guava jar doesn't play well with Maven build

[ https://issues.apache.org/jira/browse/SPARK-3217?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=1430#comment-1430 ] Apache Spark commented on SPARK-3217: - User 'vanzin' has created a pull request for

[jira] [Commented] (SPARK-2189) Method for removing temp tables created by registerAsTable

[ https://issues.apache.org/jira/browse/SPARK-2189?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=1432#comment-1432 ] Michael Armbrust commented on SPARK-2189: - I think the complication here is there

[jira] [Updated] (SPARK-3217) Shaded Guava jar doesn't play well with Maven build when SPARK_PREPEND_CLASSES is set

[ https://issues.apache.org/jira/browse/SPARK-3217?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-3217: -- Summary: Shaded Guava jar doesn't play well with Maven build when SPARK_PREPEND_CLASSES is set (was:

[jira] [Updated] (SPARK-3217) Shaded Guava jar doesn't play well with Maven build when SPARK_PREPEND_CLASSES is set

[ https://issues.apache.org/jira/browse/SPARK-3217?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-3217: -- Description: PR [#1813|https://github.com/apache/spark/pull/1813] shaded Guava jar file and moved

[jira] [Updated] (SPARK-3217) Shaded Guava jar doesn't play well with Maven build when SPARK_PREPEND_CLASSES is set

[ https://issues.apache.org/jira/browse/SPARK-3217?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-3217: -- Priority: Major (was: Blocker) Shaded Guava jar doesn't play well with Maven build when

[jira] [Commented] (SPARK-2377) Create a Python API for Spark Streaming

[ https://issues.apache.org/jira/browse/SPARK-2377?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=1435#comment-1435 ] Tathagata Das commented on SPARK-2377: -- It is a in pre-alpha PR towards my

[jira] [Commented] (SPARK-3217) Shaded Guava jar doesn't play well with Maven build when SPARK_PREPEND_CLASSES is set

[ https://issues.apache.org/jira/browse/SPARK-3217?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=1438#comment-1438 ] Cheng Lian commented on SPARK-3217: --- [~vanzin] Verified locally, everything is OK after

[jira] [Resolved] (SPARK-3131) Allow user to set parquet compression codec for writing ParquetFile in SQLContext

[ https://issues.apache.org/jira/browse/SPARK-3131?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-3131. - Resolution: Fixed Fix Version/s: 1.1.0 Allow user to set parquet compression

[jira] [Commented] (SPARK-2871) Missing API in PySpark

[ https://issues.apache.org/jira/browse/SPARK-2871?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=1436#comment-1436 ] Apache Spark commented on SPARK-2871: - User 'davies' has created a pull request for

[jira] [Comment Edited] (SPARK-3139) Akka timeouts from ContextCleaner when cleaning shuffles

[ https://issues.apache.org/jira/browse/SPARK-3139?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=1437#comment-1437 ] Tathagata Das edited comment on SPARK-3139 at 8/26/14 6:54 PM:

[jira] [Commented] (SPARK-3139) Akka timeouts from ContextCleaner when cleaning shuffles

[ https://issues.apache.org/jira/browse/SPARK-3139?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=1442#comment-1442 ] Apache Spark commented on SPARK-3139: - User 'tdas' has created a pull request for this

[jira] [Created] (SPARK-3234) SPARK_HADOOP_VERSION doesn't have a valid value by default in make-distribution.sh

Cheng Lian created SPARK-3234: - Summary: SPARK_HADOOP_VERSION doesn't have a valid value by default in make-distribution.sh Key: SPARK-3234 URL: https://issues.apache.org/jira/browse/SPARK-3234 Project:

[jira] [Comment Edited] (SPARK-3044) Create RSS feed for Spark News

[ https://issues.apache.org/jira/browse/SPARK-3044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14109677#comment-14109677 ] Nicholas Chammas edited comment on SPARK-3044 at 8/26/14 7:30 PM:

[jira] [Commented] (SPARK-3094) Support run pyspark in PyPy

[ https://issues.apache.org/jira/browse/SPARK-3094?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=1483#comment-1483 ] Apache Spark commented on SPARK-3094: - User 'davies' has created a pull request for

[jira] [Commented] (SPARK-2308) Add KMeans MiniBatch clustering algorithm to MLlib

[ https://issues.apache.org/jira/browse/SPARK-2308?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14111224#comment-14111224 ] RJ Nowling commented on SPARK-2308: --- Xiangrui, I realized that sampling in Spark is

[jira] [Created] (SPARK-3235) In-Memory relation has a bad default size.

Michael Armbrust created SPARK-3235: --- Summary: In-Memory relation has a bad default size. Key: SPARK-3235 URL: https://issues.apache.org/jira/browse/SPARK-3235 Project: Spark Issue Type:

[jira] [Commented] (SPARK-3213) spark_ec2.py cannot find slave instances launched with Launch More Like This

[ https://issues.apache.org/jira/browse/SPARK-3213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14111249#comment-14111249 ] Vida Ha commented on SPARK-3213: Okay, I'm able to reproduce now. It's occurring when you

[jira] [Updated] (SPARK-3213) spark_ec2.py cannot find slave instances launched with Launch More Like This

[ https://issues.apache.org/jira/browse/SPARK-3213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vida Ha updated SPARK-3213: --- Attachment: Screen Shot 2014-08-26 at 1.22.32 PM.png spark_ec2.py cannot find slave instances launched with

[jira] [Commented] (SPARK-3213) spark_ec2.py cannot find slave instances launched with Launch More Like This

[ https://issues.apache.org/jira/browse/SPARK-3213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14111264#comment-14111264 ] Vida Ha commented on SPARK-3213: FYI - more info on the bug: Amazon seems to be copying

[jira] [Resolved] (SPARK-2969) Make ScalaReflection be able to handle ArrayType.containsNull and MapType.valueContainsNull.

[ https://issues.apache.org/jira/browse/SPARK-2969?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-2969. - Resolution: Fixed Fix Version/s: 1.1.0 Make ScalaReflection be able to handle

[jira] [Commented] (SPARK-2608) Mesos scheduler backend create executor launch command not correctly

[ https://issues.apache.org/jira/browse/SPARK-2608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14111282#comment-14111282 ] Apache Spark commented on SPARK-2608: - User 'liancheng' has created a pull request for

[jira] [Commented] (SPARK-3235) In-Memory relation has a bad default size.

[ https://issues.apache.org/jira/browse/SPARK-3235?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14111283#comment-14111283 ] Apache Spark commented on SPARK-3235: - User 'marmbrus' has created a pull request for

[jira] [Commented] (SPARK-2377) Create a Python API for Spark Streaming

[ https://issues.apache.org/jira/browse/SPARK-2377?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14111304#comment-14111304 ] Matthew Farrellee commented on SPARK-2377: -- thanks, i'll take a look Create a

[jira] [Commented] (SPARK-3155) Support DecisionTree pruning

[ https://issues.apache.org/jira/browse/SPARK-3155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14111396#comment-14111396 ] Manish Amde commented on SPARK-3155: Hi Qiping, Thanks for creating the JIRA.

[jira] [Resolved] (SPARK-3226) Doc update for MLlib dependencies

[ https://issues.apache.org/jira/browse/SPARK-3226?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng resolved SPARK-3226. -- Resolution: Fixed Fix Version/s: 1.1.0 Issue resolved by pull request 2128

[jira] [Commented] (SPARK-3146) Improve the flexibility of Spark Streaming Kafka API to offer user the ability to process message before storing into BM

[ https://issues.apache.org/jira/browse/SPARK-3146?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14111614#comment-14111614 ] Tathagata Das commented on SPARK-3146: -- I think this can be further generalized to

[jira] [Created] (SPARK-3236) Reading Parquet tables from Metastore mangles location

Aaron Davidson created SPARK-3236: - Summary: Reading Parquet tables from Metastore mangles location Key: SPARK-3236 URL: https://issues.apache.org/jira/browse/SPARK-3236 Project: Spark Issue

[jira] [Resolved] (SPARK-3073) improve large sort (external sort) for PySpark

[ https://issues.apache.org/jira/browse/SPARK-3073?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matei Zaharia resolved SPARK-3073. -- Resolution: Fixed Fix Version/s: 1.2.0 improve large sort (external sort) for PySpark

[jira] [Commented] (SPARK-2926) Add MR-style (merge-sort) SortShuffleReader for sort-based shuffle

[ https://issues.apache.org/jira/browse/SPARK-2926?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14111631#comment-14111631 ] Matei Zaharia commented on SPARK-2926: -- I see, thanks for posting the benchmarks.

[jira] [Updated] (SPARK-3225) Typo in script

[ https://issues.apache.org/jira/browse/SPARK-3225?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matei Zaharia updated SPARK-3225: - Priority: Trivial (was: Minor) Typo in script -- Key:

[jira] [Updated] (SPARK-3225) Typo in script

[ https://issues.apache.org/jira/browse/SPARK-3225?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matei Zaharia updated SPARK-3225: - Assignee: WangTaoTheTonic Typo in script -- Key: SPARK-3225

[jira] [Resolved] (SPARK-3225) Typo in script

[ https://issues.apache.org/jira/browse/SPARK-3225?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matei Zaharia resolved SPARK-3225. -- Resolution: Fixed Fix Version/s: 1.2.0 Typo in script --

[jira] [Resolved] (SPARK-2964) Remove duplicated code from spark-sql and start-thriftserver.sh

[ https://issues.apache.org/jira/browse/SPARK-2964?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-2964. - Resolution: Fixed Fix Version/s: 1.1.0 Remove duplicated code from spark-sql and

[jira] [Created] (SPARK-3237) Push down of predicates with UDFS into parquet scan can result in serialization errors

Michael Armbrust created SPARK-3237: --- Summary: Push down of predicates with UDFS into parquet scan can result in serialization errors Key: SPARK-3237 URL: https://issues.apache.org/jira/browse/SPARK-3237

[jira] [Commented] (SPARK-2830) MLlib v1.1 documentation

[ https://issues.apache.org/jira/browse/SPARK-2830?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14111644#comment-14111644 ] Apache Spark commented on SPARK-2830: - User 'mengxr' has created a pull request for

[jira] [Updated] (SPARK-3238) Commas/spaces/dashes are not escaped properly when transferring schema information to parquet readers

[ https://issues.apache.org/jira/browse/SPARK-3238?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-3238: Summary: Commas/spaces/dashes are not escaped properly when transferring schema

[jira] [Created] (SPARK-3238) Commas are not escaped properly when transferring schema information to parquet readers

Michael Armbrust created SPARK-3238: --- Summary: Commas are not escaped properly when transferring schema information to parquet readers Key: SPARK-3238 URL: https://issues.apache.org/jira/browse/SPARK-3238

[jira] [Created] (SPARK-3239) Choose disks for spilling randomly

Davies Liu created SPARK-3239: - Summary: Choose disks for spilling randomly Key: SPARK-3239 URL: https://issues.apache.org/jira/browse/SPARK-3239 Project: Spark Issue Type: Improvement

[jira] [Commented] (SPARK-3155) Support DecisionTree pruning

[ https://issues.apache.org/jira/browse/SPARK-3155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14111685#comment-14111685 ] Joseph K. Bradley commented on SPARK-3155: -- With respect to

[jira] [Commented] (SPARK-3239) Choose disks for spilling randomly

[ https://issues.apache.org/jira/browse/SPARK-3239?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14111693#comment-14111693 ] Apache Spark commented on SPARK-3239: - User 'davies' has created a pull request for

[jira] [Commented] (SPARK-3155) Support DecisionTree pruning

[ https://issues.apache.org/jira/browse/SPARK-3155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14111697#comment-14111697 ] Manish Amde commented on SPARK-3155: Agree. I was hoping that the code

[jira] [Updated] (SPARK-3240) Document workaround for MESOS-1688

[ https://issues.apache.org/jira/browse/SPARK-3240?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matei Zaharia updated SPARK-3240: - Assignee: Martin Weindel Document workaround for MESOS-1688 --

[jira] [Created] (SPARK-3240) Document workaround for MESOS-1688

Matei Zaharia created SPARK-3240: Summary: Document workaround for MESOS-1688 Key: SPARK-3240 URL: https://issues.apache.org/jira/browse/SPARK-3240 Project: Spark Issue Type: Documentation

[jira] [Resolved] (SPARK-3037) Add ArrayType containing null value support to Parquet.

[ https://issues.apache.org/jira/browse/SPARK-3037?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-3037. - Resolution: Fixed Fix Version/s: 1.1.0 Add ArrayType containing null value

[jira] [Resolved] (SPARK-3240) Document workaround for MESOS-1688

[ https://issues.apache.org/jira/browse/SPARK-3240?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matei Zaharia resolved SPARK-3240. -- Resolution: Fixed Document workaround for MESOS-1688 --

[jira] [Resolved] (SPARK-3036) Add MapType containing null value support to Parquet.

[ https://issues.apache.org/jira/browse/SPARK-3036?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-3036. - Resolution: Fixed Fix Version/s: 1.1.0 Add MapType containing null value support

[jira] [Assigned] (SPARK-3237) Push down of predicates with UDFS into parquet scan can result in serialization errors

[ https://issues.apache.org/jira/browse/SPARK-3237?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust reassigned SPARK-3237: --- Assignee: Michael Armbrust Push down of predicates with UDFS into parquet scan can

[jira] [Updated] (SPARK-3237) Push down of predicates with UDFS into parquet scan can result in serialization errors

[ https://issues.apache.org/jira/browse/SPARK-3237?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-3237: Component/s: SQL Push down of predicates with UDFS into parquet scan can result in

[jira] [Commented] (SPARK-3237) Push down of predicates with UDFS into parquet scan can result in serialization errors

[ https://issues.apache.org/jira/browse/SPARK-3237?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14111731#comment-14111731 ] Apache Spark commented on SPARK-3237: - User 'marmbrus' has created a pull request for

[jira] [Resolved] (SPARK-3194) Optimizer removes required attributes with capital letters when using case insensitive resolution.

[ https://issues.apache.org/jira/browse/SPARK-3194?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-3194. - Resolution: Fixed Fix Version/s: 1.1.0 Optimizer removes required attributes

[jira] [Created] (SPARK-3242) Spark 1.0.2 ec2 scripts creates clusters with Spark 1.0.1 installed by default

Tathagata Das created SPARK-3242: Summary: Spark 1.0.2 ec2 scripts creates clusters with Spark 1.0.1 installed by default Key: SPARK-3242 URL: https://issues.apache.org/jira/browse/SPARK-3242

[jira] [Commented] (SPARK-3242) Spark 1.0.2 ec2 scripts creates clusters with Spark 1.0.1 installed by default

[ https://issues.apache.org/jira/browse/SPARK-3242?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14111773#comment-14111773 ] Tathagata Das commented on SPARK-3242: -- Current workaround is to create a explicitly

[jira] [Created] (SPARK-3243) spark.driver.extra* system properties are outdated if --driver-* options exist

Andrew Or created SPARK-3243: Summary: spark.driver.extra* system properties are outdated if --driver-* options exist Key: SPARK-3243 URL: https://issues.apache.org/jira/browse/SPARK-3243 Project: Spark

[jira] [Created] (SPARK-3244) Add fate sharing across files in Jenkins

Andrew Or created SPARK-3244: Summary: Add fate sharing across files in Jenkins Key: SPARK-3244 URL: https://issues.apache.org/jira/browse/SPARK-3244 Project: Spark Issue Type: Improvement

[jira] [Updated] (SPARK-3244) Add fate sharing across related files in Jenkins

[ https://issues.apache.org/jira/browse/SPARK-3244?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-3244: - Summary: Add fate sharing across related files in Jenkins (was: Add fate sharing across files in

[jira] [Commented] (SPARK-3155) Support DecisionTree pruning

[ https://issues.apache.org/jira/browse/SPARK-3155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14111813#comment-14111813 ] Joseph K. Bradley commented on SPARK-3155: -- Qiping, I think it's up to you; both

[jira] [Updated] (SPARK-3243) spark.driver.* system properties are outdated if --driver-* options exist

[ https://issues.apache.org/jira/browse/SPARK-3243?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-3243: - Summary: spark.driver.* system properties are outdated if --driver-* options exist (was:

[jira] [Commented] (SPARK-3243) spark.driver.* system properties are outdated if --driver-* options exist

[ https://issues.apache.org/jira/browse/SPARK-3243?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14111823#comment-14111823 ] Apache Spark commented on SPARK-3243: - User 'andrewor14' has created a pull request

[jira] [Resolved] (SPARK-3224) FetchFailed stages could show up multiple times in failed stages in web ui

[ https://issues.apache.org/jira/browse/SPARK-3224?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell resolved SPARK-3224. Resolution: Fixed Fix Version/s: 1.1.0 Issue resolved by pull request 2127

  1   2   >