[jira] [Created] (SPARK-6426) User could also point the yarn cluster config directory via YARN_CONF_DIR

Tao Wang created SPARK-6426: --- Summary: User could also point the yarn cluster config directory via YARN_CONF_DIR Key: SPARK-6426 URL: https://issues.apache.org/jira/browse/SPARK-6426 Project: Spark

[jira] [Created] (SPARK-6429) Add to style checker hashCode and equals should be defined together

Reynold Xin created SPARK-6429: -- Summary: Add to style checker hashCode and equals should be defined together Key: SPARK-6429 URL: https://issues.apache.org/jira/browse/SPARK-6429 Project: Spark

[jira] [Updated] (SPARK-6424) Support user-defined aggregators in AggregateFunction

[ https://issues.apache.org/jira/browse/SPARK-6424?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takeshi Yamamuro updated SPARK-6424: Description: Add a new interface to implement user-defined aggregators in

[jira] [Closed] (SPARK-6427) spark-sql does not throw error if running in yarn-cluster mode

[ https://issues.apache.org/jira/browse/SPARK-6427?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liangliang Gu closed SPARK-6427. Resolution: Not a Problem spark-sql does not throw error if running in yarn-cluster mode

[jira] [Commented] (SPARK-6406) Launcher backward compatibility issues

[ https://issues.apache.org/jira/browse/SPARK-6406?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14370815#comment-14370815 ] Nishkam Ravi commented on SPARK-6406: - Also, we should not look for a separate

[jira] [Commented] (SPARK-6421) _regression_train_wrapper does not test initialWeights correctly

[ https://issues.apache.org/jira/browse/SPARK-6421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14370836#comment-14370836 ] Apache Spark commented on SPARK-6421: - User 'Lewuathe' has created a pull request for

[jira] [Created] (SPARK-6427) spark-sql does not throw error if running in yarn-cluster mode

Liangliang Gu created SPARK-6427: Summary: spark-sql does not throw error if running in yarn-cluster mode Key: SPARK-6427 URL: https://issues.apache.org/jira/browse/SPARK-6427 Project: Spark

[jira] [Commented] (SPARK-6426) User could also point the yarn cluster config directory via YARN_CONF_DIR

[ https://issues.apache.org/jira/browse/SPARK-6426?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14370868#comment-14370868 ] Apache Spark commented on SPARK-6426: - User 'WangTaoTheTonic' has created a pull

[jira] [Created] (SPARK-6428) Add to style checker public method must have explicit type defined

Reynold Xin created SPARK-6428: -- Summary: Add to style checker public method must have explicit type defined Key: SPARK-6428 URL: https://issues.apache.org/jira/browse/SPARK-6428 Project: Spark

[jira] [Commented] (SPARK-6428) Add to style checker public method must have explicit type defined

[ https://issues.apache.org/jira/browse/SPARK-6428?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14370920#comment-14370920 ] Apache Spark commented on SPARK-6428: - User 'rxin' has created a pull request for this

[jira] [Commented] (SPARK-6428) Add to style checker public method must have explicit type defined

[ https://issues.apache.org/jira/browse/SPARK-6428?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14370928#comment-14370928 ] Apache Spark commented on SPARK-6428: - User 'rxin' has created a pull request for this

[jira] [Created] (SPARK-6431) Couldn't find leader offsets exception when creating KafkaDirectStream

Alberto created SPARK-6431: -- Summary: Couldn't find leader offsets exception when creating KafkaDirectStream Key: SPARK-6431 URL: https://issues.apache.org/jira/browse/SPARK-6431 Project: Spark

[jira] [Commented] (SPARK-6432) Cannot load parquet data with partitions if not all partition columns match data columns

[ https://issues.apache.org/jira/browse/SPARK-6432?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14371053#comment-14371053 ] zzc commented on SPARK-6432: [~liancheng], My data schema has the following columns: root |--

[jira] [Commented] (SPARK-6432) Cannot load parquet data with partitions if not all partition columns match data columns

[ https://issues.apache.org/jira/browse/SPARK-6432?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14371075#comment-14371075 ] zzc commented on SPARK-6432: Thanks, [~liancheng], [~huangjs] Cannot load parquet data with

[jira] [Commented] (SPARK-6426) User could also point the yarn cluster config directory via YARN_CONF_DIR

[ https://issues.apache.org/jira/browse/SPARK-6426?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14371082#comment-14371082 ] Sean Owen commented on SPARK-6426: -- What is the motivation for this? I'm always slightly

[jira] [Commented] (SPARK-6432) Cannot load parquet data with partitions if not all partition columns match data columns

[ https://issues.apache.org/jira/browse/SPARK-6432?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14371116#comment-14371116 ] Cheng Lian commented on SPARK-6432: --- Yeah, sorry for my carelessness, it's OK if either

[jira] [Created] (SPARK-6430) Cannot resolve column correctlly when using left semi join

zzc created SPARK-6430: -- Summary: Cannot resolve column correctlly when using left semi join Key: SPARK-6430 URL: https://issues.apache.org/jira/browse/SPARK-6430 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-6137) G-Means clustering algorithm implementation

[ https://issues.apache.org/jira/browse/SPARK-6137?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14371149#comment-14371149 ] Vikas Veshishth commented on SPARK-6137: Is anyone working on this. I can start on

[jira] [Commented] (SPARK-6160) ChiSqSelector should keep test statistic info

[ https://issues.apache.org/jira/browse/SPARK-6160?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14371172#comment-14371172 ] Vikas Veshishth commented on SPARK-6160: Do you want the Array[ChiSqTestResult]

[jira] [Commented] (SPARK-6334) spark-local dir not getting cleared during ALS

[ https://issues.apache.org/jira/browse/SPARK-6334?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14371122#comment-14371122 ] Antony Mayi commented on SPARK-6334: bq. 2. Use less number of blocks, even you have

[jira] [Created] (SPARK-6432) Cannot load parquet data with partitions if not all partition columns match data columns

Jianshi Huang created SPARK-6432: Summary: Cannot load parquet data with partitions if not all partition columns match data columns Key: SPARK-6432 URL: https://issues.apache.org/jira/browse/SPARK-6432

[jira] [Commented] (SPARK-6432) Cannot load parquet data with partitions if not all partition columns match data columns

[ https://issues.apache.org/jira/browse/SPARK-6432?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14371035#comment-14371035 ] zzc commented on SPARK-6432: [~huangjs], I have some parquet files in partitions path, as

[jira] [Commented] (SPARK-6432) Cannot load parquet data with partitions if not all partition columns match data columns

[ https://issues.apache.org/jira/browse/SPARK-6432?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14371052#comment-14371052 ] zzc commented on SPARK-6432: [~liancheng], My data schema has the following columns: root |--

[jira] [Commented] (SPARK-6432) Cannot load parquet data with partitions if not all partition columns match data columns

[ https://issues.apache.org/jira/browse/SPARK-6432?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14371061#comment-14371061 ] Jianshi Huang commented on SPARK-6432: -- If no partition column appear in the data

[jira] [Updated] (SPARK-5707) Enabling spark.sql.codegen throws ClassNotFound exception

[ https://issues.apache.org/jira/browse/SPARK-5707?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-5707: Target Version/s: 1.4.0 Enabling spark.sql.codegen throws ClassNotFound exception

[jira] [Updated] (SPARK-5707) Enabling spark.sql.codegen throws ClassNotFound exception

[ https://issues.apache.org/jira/browse/SPARK-5707?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-5707: Priority: Blocker (was: Major) Enabling spark.sql.codegen throws ClassNotFound exception

[jira] [Commented] (SPARK-6229) Support SASL encryption in network/common module

[ https://issues.apache.org/jira/browse/SPARK-6229?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14372381#comment-14372381 ] Marcelo Vanzin commented on SPARK-6229: --- Just to illustrate: {noformat} $ git grep

[jira] [Created] (SPARK-6441) [MLLIB] Add Deflation/Schur Complement to Power Iteration Clustering for improved resilience to inter-class collisions

Stephen Boesch created SPARK-6441: - Summary: [MLLIB] Add Deflation/Schur Complement to Power Iteration Clustering for improved resilience to inter-class collisions Key: SPARK-6441 URL:

[jira] [Commented] (SPARK-6369) InsertIntoHiveTable should use logic from SparkHadoopWriter

[ https://issues.apache.org/jira/browse/SPARK-6369?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14372273#comment-14372273 ] Yin Huai commented on SPARK-6369: - Seems we need to change SparkHiveWriterContainer.commit

[jira] [Commented] (SPARK-6229) Support SASL encryption in network/common module

[ https://issues.apache.org/jira/browse/SPARK-6229?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14372352#comment-14372352 ] Marcelo Vanzin commented on SPARK-6229: --- I'm a little wary of exposing the pipeline

[jira] [Resolved] (SPARK-6025) Helper method for GradientBoostedTrees to compute validation error

[ https://issues.apache.org/jira/browse/SPARK-6025?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley resolved SPARK-6025. -- Resolution: Fixed Fix Version/s: 1.4.0 Issue resolved by pull request 4906

[jira] [Commented] (SPARK-6229) Support SASL encryption in network/common module

[ https://issues.apache.org/jira/browse/SPARK-6229?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14372372#comment-14372372 ] Reynold Xin commented on SPARK-6229: I didn't mean to have this as a public API (I was

[jira] [Comment Edited] (SPARK-6229) Support SASL encryption in network/common module

[ https://issues.apache.org/jira/browse/SPARK-6229?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14372372#comment-14372372 ] Reynold Xin edited comment on SPARK-6229 at 3/21/15 12:21 AM: --

[jira] [Resolved] (SPARK-6248) LocalRelation needs to implement statistics

[ https://issues.apache.org/jira/browse/SPARK-6248?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-6248. - Resolution: Fixed Fix Version/s: 1.3.1 Assignee: Michael Armbrust

[jira] [Commented] (SPARK-6229) Support SASL encryption in network/common module

[ https://issues.apache.org/jira/browse/SPARK-6229?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14372391#comment-14372391 ] Reynold Xin commented on SPARK-6229: In case it is still not clear -- I agree with you

[jira] [Comment Edited] (SPARK-6229) Support SASL encryption in network/common module

[ https://issues.apache.org/jira/browse/SPARK-6229?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14372391#comment-14372391 ] Reynold Xin edited comment on SPARK-6229 at 3/21/15 12:39 AM: --

[jira] [Created] (SPARK-6442) MLlib 1.4 Local Linear Algebra Package

Burak Yavuz created SPARK-6442: -- Summary: MLlib 1.4 Local Linear Algebra Package Key: SPARK-6442 URL: https://issues.apache.org/jira/browse/SPARK-6442 Project: Spark Issue Type: New Feature

[jira] [Updated] (SPARK-4925) Publish Spark SQL hive-thriftserver maven artifact

[ https://issues.apache.org/jira/browse/SPARK-4925?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-4925: --- Fix Version/s: (was: 1.2.1) (was: 1.3.0) Publish Spark SQL

[jira] [Updated] (SPARK-4925) Publish Spark SQL hive-thriftserver maven artifact

[ https://issues.apache.org/jira/browse/SPARK-4925?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-4925: --- Priority: Critical (was: Major) Publish Spark SQL hive-thriftserver maven artifact

[jira] [Updated] (SPARK-4925) Publish Spark SQL hive-thriftserver maven artifact

[ https://issues.apache.org/jira/browse/SPARK-4925?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-4925: --- Affects Version/s: (was: 1.2.0) 1.3.0 1.2.1

[jira] [Updated] (SPARK-2686) Add Length support to Spark SQL and HQL and Strlen support to SQL

[ https://issues.apache.org/jira/browse/SPARK-2686?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai updated SPARK-2686: Target Version/s: 1.4.0 (was: 1.3.0) Add Length support to Spark SQL and HQL and Strlen support to SQL

[jira] [Updated] (SPARK-6337) Spark 1.3 doc fixes

[ https://issues.apache.org/jira/browse/SPARK-6337?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-6337: - Description: I'll try to track doc issues to be fixed for the 1.3.1 release in this JIRA.

[jira] [Updated] (SPARK-4123) Show dependency changes in pull requests

[ https://issues.apache.org/jira/browse/SPARK-4123?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-4123: --- Summary: Show dependency changes in pull requests (was: Show new dependencies added in pull

[jira] [Updated] (SPARK-6025) Helper method for GradientBoostedTrees to compute validation error

[ https://issues.apache.org/jira/browse/SPARK-6025?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-6025: - Assignee: Manoj Kumar Helper method for GradientBoostedTrees to compute validation error

[jira] [Commented] (SPARK-6229) Support SASL encryption in network/common module

[ https://issues.apache.org/jira/browse/SPARK-6229?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14372379#comment-14372379 ] Marcelo Vanzin commented on SPARK-6229: --- But I do mean internal Spark users when I

[jira] [Commented] (SPARK-6229) Support SASL encryption in network/common module

[ https://issues.apache.org/jira/browse/SPARK-6229?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14372394#comment-14372394 ] Marcelo Vanzin commented on SPARK-6229: --- Yes, and that's actually what I'm

[jira] [Reopened] (SPARK-4925) Publish Spark SQL hive-thriftserver maven artifact

[ https://issues.apache.org/jira/browse/SPARK-4925?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell reopened SPARK-4925: Thanks for bringing this up. Actually - realized this wasn't fixed by some of the other work

[jira] [Updated] (SPARK-4925) Publish Spark SQL hive-thriftserver maven artifact

[ https://issues.apache.org/jira/browse/SPARK-4925?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-4925: --- Target Version/s: 1.3.1 Publish Spark SQL hive-thriftserver maven artifact

[jira] [Commented] (SPARK-6428) Add to style checker public method must have explicit type defined

[ https://issues.apache.org/jira/browse/SPARK-6428?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14372313#comment-14372313 ] Apache Spark commented on SPARK-6428: - User 'rxin' has created a pull request for this

[jira] [Commented] (SPARK-6425) Add parallel Q-learning algorithm to MLLib

[ https://issues.apache.org/jira/browse/SPARK-6425?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14372385#comment-14372385 ] Joseph K. Bradley commented on SPARK-6425: -- Reinforcement learning is a huge

[jira] [Commented] (SPARK-6229) Support SASL encryption in network/common module

[ https://issues.apache.org/jira/browse/SPARK-6229?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14372383#comment-14372383 ] Reynold Xin commented on SPARK-6229: Can't you just put those in TransportContext?

[jira] [Updated] (SPARK-6369) InsertIntoHiveTable should use logic from SparkHadoopWriter

[ https://issues.apache.org/jira/browse/SPARK-6369?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai updated SPARK-6369: Priority: Blocker (was: Critical) InsertIntoHiveTable should use logic from SparkHadoopWriter

[jira] [Commented] (SPARK-6369) InsertIntoHiveTable should use logic from SparkHadoopWriter

[ https://issues.apache.org/jira/browse/SPARK-6369?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14372397#comment-14372397 ] Yin Huai commented on SPARK-6369: - Three places that need to be fixed are

[jira] [Commented] (SPARK-6231) Join on two tables (generated from same one) is broken

[ https://issues.apache.org/jira/browse/SPARK-6231?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14372447#comment-14372447 ] Shivaram Venkataraman commented on SPARK-6231: -- [~marmbrus] I've sent the

[jira] [Commented] (SPARK-6433) hive tests to import spark-sql test JAR for QueryTest access

[ https://issues.apache.org/jira/browse/SPARK-6433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14371299#comment-14371299 ] Steve Loughran commented on SPARK-6433: --- Similarly, the original sql package's

[jira] [Updated] (SPARK-6435) spark-shell --jars option does not add all jars to classpath

[ https://issues.apache.org/jira/browse/SPARK-6435?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] vijay updated SPARK-6435: - Description: Not all jars supplied via the --jars option will be added to the driver (and presumably executor)

[jira] [Commented] (SPARK-6433) hive tests to import spark-sql test JAR for QueryTest access

[ https://issues.apache.org/jira/browse/SPARK-6433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14371283#comment-14371283 ] Steve Loughran commented on SPARK-6433: --- They have diverged, the original sql one

[jira] [Created] (SPARK-6436) io/netty missing from external shuffle service jars for yarn

Thomas Graves created SPARK-6436: Summary: io/netty missing from external shuffle service jars for yarn Key: SPARK-6436 URL: https://issues.apache.org/jira/browse/SPARK-6436 Project: Spark

[jira] [Commented] (SPARK-6433) hive tests to import spark-sql test JAR for QueryTest access

[ https://issues.apache.org/jira/browse/SPARK-6433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14371378#comment-14371378 ] Sean Owen commented on SPARK-6433: -- Tough one. My instinct is to not do anything special

[jira] [Resolved] (SPARK-6400) It would be great if you could share your test jars in Maven central repository for the Spark SQL module

[ https://issues.apache.org/jira/browse/SPARK-6400?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-6400. -- Resolution: Duplicate It would be great if you could share your test jars in Maven central

[jira] [Updated] (SPARK-5134) Bump default Hadoop version to 2+

[ https://issues.apache.org/jira/browse/SPARK-5134?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-5134: - Assignee: Ryan Williams Bump default Hadoop version to 2+ -

[jira] [Resolved] (SPARK-5134) Bump default Hadoop version to 2+

[ https://issues.apache.org/jira/browse/SPARK-5134?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-5134. -- Resolution: Fixed Fix Version/s: 1.4.0 Issue resolved by pull request 5027

[jira] [Commented] (SPARK-6425) Add parallel Q-learning algorithm to MLLib

[ https://issues.apache.org/jira/browse/SPARK-6425?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14371385#comment-14371385 ] Sean Owen commented on SPARK-6425: -- There are loads of algorithms that could be added;

[jira] [Commented] (SPARK-6152) Spark does not support Java 8 compiled Scala classes

[ https://issues.apache.org/jira/browse/SPARK-6152?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14371252#comment-14371252 ] Martin Grotzke commented on SPARK-6152: --- I just released reflectasm-1.10.1 (which

[jira] [Resolved] (SPARK-6434) [Streaming][Kafka] CreateDirectStream for empty topics

[ https://issues.apache.org/jira/browse/SPARK-6434?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-6434. -- Resolution: Duplicate Fix Version/s: (was: 1.3.1) I hear you, maybe there's no way to tell

[jira] [Commented] (SPARK-6433) hive tests to import spark-sql test JAR for QueryTest access

[ https://issues.apache.org/jira/browse/SPARK-6433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14371350#comment-14371350 ] Steve Loughran commented on SPARK-6433: --- There's one interesting question: whether

[jira] [Created] (SPARK-6435) spark-shell --jars option does not add all jars to classpath

vijay created SPARK-6435: Summary: spark-shell --jars option does not add all jars to classpath Key: SPARK-6435 URL: https://issues.apache.org/jira/browse/SPARK-6435 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-4227) Document external shuffle service

[ https://issues.apache.org/jira/browse/SPARK-4227?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves updated SPARK-4227: - Target Version/s: 1.3.0, 1.4.0 (was: 1.3.0) Document external shuffle service

[jira] [Commented] (SPARK-6435) spark-shell --jars option does not add all jars to classpath

[ https://issues.apache.org/jira/browse/SPARK-6435?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14371356#comment-14371356 ] Sean Owen commented on SPARK-6435: -- Hm, does it work without the dummy jars? I suspect a

[jira] [Commented] (SPARK-6433) hive tests to import spark-sql test JAR for QueryTest access

[ https://issues.apache.org/jira/browse/SPARK-6433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14371280#comment-14371280 ] Steve Loughran commented on SPARK-6433: --- ..sorry, I'd missed that previous report.

[jira] [Resolved] (SPARK-6338) Use standard temp dir mechanisms in tests to avoid orphaned temp files

[ https://issues.apache.org/jira/browse/SPARK-6338?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-6338. -- Resolution: Fixed Fix Version/s: 1.4.0 Issue resolved by pull request 5029

[jira] [Commented] (SPARK-6434) [Streaming][Kafka] CreateDirectStream for empty topics

[ https://issues.apache.org/jira/browse/SPARK-6434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14371330#comment-14371330 ] Cody Koeninger commented on SPARK-6434: --- Yeah, I didn't notice alberto had also

[jira] [Commented] (SPARK-6433) hive tests to import spark-sql test JAR for QueryTest access

[ https://issues.apache.org/jira/browse/SPARK-6433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14371220#comment-14371220 ] Sean Owen commented on SPARK-6433: -- Yeah I tend to support publishing test artifacts to

[jira] [Commented] (SPARK-6152) Spark does not support Java 8 compiled Scala classes

[ https://issues.apache.org/jira/browse/SPARK-6152?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14371254#comment-14371254 ] Sean Owen commented on SPARK-6152: -- Nice one. It looks like {{reflectasm}} comes in via

[jira] [Created] (SPARK-6434) [Streaming][Kafka] CreateDirectStream for empty topics

Cody Koeninger created SPARK-6434: - Summary: [Streaming][Kafka] CreateDirectStream for empty topics Key: SPARK-6434 URL: https://issues.apache.org/jira/browse/SPARK-6434 Project: Spark Issue

[jira] [Commented] (SPARK-6434) [Streaming][Kafka] CreateDirectStream for empty topics

[ https://issues.apache.org/jira/browse/SPARK-6434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14371320#comment-14371320 ] Sean Owen commented on SPARK-6434: -- This was already covered by SPARK-6431 right? the

[jira] [Commented] (SPARK-6096) Support model save/load in Python's naive Bayes

[ https://issues.apache.org/jira/browse/SPARK-6096?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14371382#comment-14371382 ] Apache Spark commented on SPARK-6096: - User 'yanboliang' has created a pull request

[jira] [Updated] (SPARK-6429) Add to style checker hashCode and equals should be defined together

[ https://issues.apache.org/jira/browse/SPARK-6429?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-6429: - Component/s: Build Add to style checker hashCode and equals should be defined together

[jira] [Updated] (SPARK-6428) Add to style checker public method must have explicit type defined

[ https://issues.apache.org/jira/browse/SPARK-6428?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-6428: - Component/s: Build Add to style checker public method must have explicit type defined

[jira] [Updated] (SPARK-6418) Add simple per-stage visualization to the UI

[ https://issues.apache.org/jira/browse/SPARK-6418?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-6418: - Component/s: Web UI Add simple per-stage visualization to the UI

[jira] [Commented] (SPARK-5782) Python Worker / Pyspark Daemon Memory Issue

[ https://issues.apache.org/jira/browse/SPARK-5782?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14371418#comment-14371418 ] Sean Owen commented on SPARK-5782: -- Doesn't this make an RDD tens of billions of elements

[jira] [Commented] (SPARK-6435) spark-shell --jars option does not add all jars to classpath

[ https://issues.apache.org/jira/browse/SPARK-6435?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14371429#comment-14371429 ] Sean Owen commented on SPARK-6435: -- The weird thing is the example shows it finds the

[jira] [Commented] (SPARK-6435) spark-shell --jars option does not add all jars to classpath

[ https://issues.apache.org/jira/browse/SPARK-6435?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14371420#comment-14371420 ] vijay commented on SPARK-6435: -- It works when guava is the 1st or 2nd jar. Not sure at what

[jira] [Commented] (SPARK-5821) JSONRelation should check if delete is successful for the overwrite operation.

[ https://issues.apache.org/jira/browse/SPARK-5821?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14371445#comment-14371445 ] Apache Spark commented on SPARK-5821: - User 'yanboliang' has created a pull request

[jira] [Updated] (SPARK-6443) Could not submit app in standalone cluster mode when HA is enabled

[ https://issues.apache.org/jira/browse/SPARK-6443?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tao Wang updated SPARK-6443: Description: After digging some codes, I found user could not submit app in standalone cluster mode when

[jira] [Commented] (SPARK-6337) Spark 1.3 doc fixes

[ https://issues.apache.org/jira/browse/SPARK-6337?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14372519#comment-14372519 ] Apache Spark commented on SPARK-6337: - User 'vinodkc' has created a pull request for

[jira] [Resolved] (SPARK-5821) JSONRelation should check if delete is successful for the overwrite operation.

[ https://issues.apache.org/jira/browse/SPARK-5821?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian resolved SPARK-5821. --- Resolution: Fixed Fix Version/s: 1.4.0 1.3.1 Issue resolved by pull request

[jira] [Updated] (SPARK-5821) JSONRelation and ParquetRelation2 should check if delete is successful for the overwrite operation.

[ https://issues.apache.org/jira/browse/SPARK-5821?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-5821: -- Target Version/s: 1.4.0, 1.3.1 (was: 1.3.1) JSONRelation and ParquetRelation2 should check if delete

[jira] [Updated] (SPARK-5821) JSONRelation and ParquetRelation2 should check if delete is successful for the overwrite operation.

[ https://issues.apache.org/jira/browse/SPARK-5821?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-5821: -- Assignee: Yanbo Liang JSONRelation and ParquetRelation2 should check if delete is successful for the

[jira] [Resolved] (SPARK-6315) SparkSQL 1.3.0 (RC3) fails to read parquet file generated by 1.1.1

[ https://issues.apache.org/jira/browse/SPARK-6315?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian resolved SPARK-6315. --- Resolution: Fixed Fix Version/s: 1.4.0 1.3.1 Issue resolved by pull request

[jira] [Updated] (SPARK-5821) JSONRelation and ParquetRelation2 should check if delete is successful for the overwrite operation.

[ https://issues.apache.org/jira/browse/SPARK-5821?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-5821: -- Summary: JSONRelation and ParquetRelation2 should check if delete is successful for the overwrite

[jira] [Created] (SPARK-6444) Sum expression should remain unresolved if the data type isn't a numeric type

Cheng Lian created SPARK-6444: - Summary: Sum expression should remain unresolved if the data type isn't a numeric type Key: SPARK-6444 URL: https://issues.apache.org/jira/browse/SPARK-6444 Project: Spark

[jira] [Created] (SPARK-6443) Could not submit app in standalone cluster mode when HA is enabled

Tao Wang created SPARK-6443: --- Summary: Could not submit app in standalone cluster mode when HA is enabled Key: SPARK-6443 URL: https://issues.apache.org/jira/browse/SPARK-6443 Project: Spark

[jira] [Created] (SPARK-6433) hive tests to import spark-sql test JAR for QueryTest access

Steve Loughran created SPARK-6433: - Summary: hive tests to import spark-sql test JAR for QueryTest access Key: SPARK-6433 URL: https://issues.apache.org/jira/browse/SPARK-6433 Project: Spark

[jira] [Created] (SPARK-6437) Spark SQL ExternalSort should use CompletionIterator to clean up temp files

Yin Huai created SPARK-6437: --- Summary: Spark SQL ExternalSort should use CompletionIterator to clean up temp files Key: SPARK-6437 URL: https://issues.apache.org/jira/browse/SPARK-6437 Project: Spark

[jira] [Updated] (SPARK-6437) SQL ExternalSort should use CompletionIterator to clean up temp files

[ https://issues.apache.org/jira/browse/SPARK-6437?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai updated SPARK-6437: Summary: SQL ExternalSort should use CompletionIterator to clean up temp files (was: Spark SQL

[jira] [Updated] (SPARK-6390) Add MatrixUDT in PySpark

[ https://issues.apache.org/jira/browse/SPARK-6390?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-6390: - Assignee: Manoj Kumar Add MatrixUDT in PySpark Key:

[jira] [Updated] (SPARK-6309) Add MatrixUDT to support dense/sparse matrices in DataFrames

[ https://issues.apache.org/jira/browse/SPARK-6309?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-6309: - Assignee: Manoj Kumar Add MatrixUDT to support dense/sparse matrices in DataFrames

[jira] [Commented] (SPARK-5782) Python Worker / Pyspark Daemon Memory Issue

[ https://issues.apache.org/jira/browse/SPARK-5782?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14371394#comment-14371394 ] Mark Khaitman commented on SPARK-5782: -- I didn't think this would be an extreme

[jira] [Commented] (SPARK-4227) Document external shuffle service

[ https://issues.apache.org/jira/browse/SPARK-4227?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14371417#comment-14371417 ] Thomas Graves commented on SPARK-4227: -- this would definitely be nice to get in the

  1   2   >