[jira] [Commented] (SPARK-15439) Failed to run unit test in SparkR

2016-05-26 Thread Shivaram Venkataraman (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15439?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15302525#comment-15302525 ] Shivaram Venkataraman commented on SPARK-15439: --- Hmm - I think the check ne

[jira] [Commented] (SPARK-7159) Support multiclass logistic regression in spark.ml

2016-05-26 Thread Seth Hendrickson (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7159?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15302521#comment-15302521 ] Seth Hendrickson commented on SPARK-7159: - Status update: With DB's suggestions in

[jira] [Commented] (SPARK-15565) The default value of spark.sql.warehouse.dir needs to explicitly point to local filesystem

2016-05-26 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15302502#comment-15302502 ] Xiao Li commented on SPARK-15565: - Will do it soon. : ) > The default value of spark.sql

[jira] [Commented] (SPARK-15565) The default value of spark.sql.warehouse.dir needs to explicitly point to local filesystem

2016-05-26 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15302499#comment-15302499 ] Yin Huai commented on SPARK-15565: -- Thanks! I was thinking about asking you :) > The de

[jira] [Commented] (SPARK-15565) The default value of spark.sql.warehouse.dir needs to explicitly point to local filesystem

2016-05-26 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15302486#comment-15302486 ] Xiao Li commented on SPARK-15565: - If nobody takes it, I can do it. : ) > The default va

[jira] [Created] (SPARK-15567) Refactor ExecutorAllocationManager

2016-05-26 Thread Kevin Cox (JIRA)
Kevin Cox created SPARK-15567: - Summary: Refactor ExecutorAllocationManager Key: SPARK-15567 URL: https://issues.apache.org/jira/browse/SPARK-15567 Project: Spark Issue Type: Task Rep

[jira] [Assigned] (SPARK-15567) Refactor ExecutorAllocationManager

2016-05-26 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15567?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15567: Assignee: (was: Apache Spark) > Refactor ExecutorAllocationManager > -

[jira] [Commented] (SPARK-15567) Refactor ExecutorAllocationManager

2016-05-26 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15567?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15302451#comment-15302451 ] Apache Spark commented on SPARK-15567: -- User 'kevincox' has created a pull request f

[jira] [Assigned] (SPARK-15567) Refactor ExecutorAllocationManager

2016-05-26 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15567?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15567: Assignee: Apache Spark > Refactor ExecutorAllocationManager >

[jira] [Assigned] (SPARK-15566) Expose null checking function to Python land.

2016-05-26 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15566?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15566: Assignee: Apache Spark > Expose null checking function to Python land. > -

[jira] [Assigned] (SPARK-15566) Expose null checking function to Python land.

2016-05-26 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15566?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15566: Assignee: (was: Apache Spark) > Expose null checking function to Python land. > --

[jira] [Commented] (SPARK-15566) Expose null checking function to Python land.

2016-05-26 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15566?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15302438#comment-15302438 ] Apache Spark commented on SPARK-15566: -- User 'kevincox' has created a pull request f

[jira] [Created] (SPARK-15566) Expose null checking function to Python land.

2016-05-26 Thread Kevin Cox (JIRA)
Kevin Cox created SPARK-15566: - Summary: Expose null checking function to Python land. Key: SPARK-15566 URL: https://issues.apache.org/jira/browse/SPARK-15566 Project: Spark Issue Type: Improveme

[jira] [Commented] (SPARK-13983) HiveThriftServer2 can not get "--hiveconf" or ''--hivevar" variables since 1.6 version (both multi-session and single session)

2016-05-26 Thread Teng Qiu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15302429#comment-15302429 ] Teng Qiu commented on SPARK-13983: -- Hi, we tried put spark.sql.hive.thriftServer.singleS

[jira] [Updated] (SPARK-15547) Encoder validation is too strict for inner nested structs

2016-05-26 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15547?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-15547: --- Description: The following Spark shell snippet reproduces this issue: {code} case class ClassData(a:

[jira] [Comment Edited] (SPARK-15564) App name is the main class name in Spark streaming jobs

2016-05-26 Thread Steven Lowenthal (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15564?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15302358#comment-15302358 ] Steven Lowenthal edited comment on SPARK-15564 at 5/26/16 4:34 PM:

[jira] [Commented] (SPARK-15506) only one notebook can define a UDF; java.sql.SQLException: Another instance of Derby may have already booted the database

2016-05-26 Thread Andrew Davidson (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15506?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15302359#comment-15302359 ] Andrew Davidson commented on SPARK-15506: - Hi Jeff Here is how I start the noteb

[jira] [Commented] (SPARK-15564) App name is the main class name in Spark streaming jobs

2016-05-26 Thread Steven Lowenthal (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15564?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15302358#comment-15302358 ] Steven Lowenthal commented on SPARK-15564: -- It looks like it works using the alt

[jira] [Updated] (SPARK-15564) App name is the main class name in Spark streaming jobs

2016-05-26 Thread Steven Lowenthal (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15564?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steven Lowenthal updated SPARK-15564: - Description: I've tried everything to set the app name to something other than the class

[jira] [Updated] (SPARK-15564) App name is the main class name in Spark streaming jobs

2016-05-26 Thread Steven Lowenthal (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15564?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steven Lowenthal updated SPARK-15564: - Description: I've tried everything to set the app name to something other than the class

[jira] [Created] (SPARK-15565) The default value of spark.sql.warehouse.dir needs to explicitly point to local filesystem

2016-05-26 Thread Yin Huai (JIRA)
Yin Huai created SPARK-15565: Summary: The default value of spark.sql.warehouse.dir needs to explicitly point to local filesystem Key: SPARK-15565 URL: https://issues.apache.org/jira/browse/SPARK-15565 Pr

[jira] [Commented] (SPARK-15034) Use the value of spark.sql.warehouse.dir as the warehouse location instead of using hive.metastore.warehouse.dir

2016-05-26 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15034?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15302343#comment-15302343 ] Yin Huai commented on SPARK-15034: -- Sure. It will be tracked by https://issues.apache.or

[jira] [Commented] (SPARK-15034) Use the value of spark.sql.warehouse.dir as the warehouse location instead of using hive.metastore.warehouse.dir

2016-05-26 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15034?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15302335#comment-15302335 ] Yin Huai commented on SPARK-15034: -- Spark 2.0 will not load hive-site.xml. Can you put a

[jira] [Updated] (SPARK-15529) Replace SQLContext and HiveContext with SparkSession in Test

2016-05-26 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15529?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-15529: Description: Use the latest Sparksession to replace the existing SQLContext and HiveContext in test cases.

[jira] [Created] (SPARK-15564) App name is the main class name in Spark streaming jobs

2016-05-26 Thread Steven Lowenthal (JIRA)
Steven Lowenthal created SPARK-15564: Summary: App name is the main class name in Spark streaming jobs Key: SPARK-15564 URL: https://issues.apache.org/jira/browse/SPARK-15564 Project: Spark

[jira] [Commented] (SPARK-15034) Use the value of spark.sql.warehouse.dir as the warehouse location instead of using hive.metastore.warehouse.dir

2016-05-26 Thread koert kuipers (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15034?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15302314#comment-15302314 ] koert kuipers commented on SPARK-15034: --- since the system property user.dir is a lo

[jira] [Commented] (SPARK-15563) Packages specification

2016-05-26 Thread jay vyas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15563?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15302290#comment-15302290 ] jay vyas commented on SPARK-15563: -- ok, makes sense. In any case, im pretty sure that,

[jira] [Commented] (SPARK-15563) Packages specification

2016-05-26 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15563?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15302285#comment-15302285 ] Sean Owen commented on SPARK-15563: --- I don't understand your comment. I'm saying this i

[jira] [Closed] (SPARK-15563) Packages specification

2016-05-26 Thread jay vyas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15563?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jay vyas closed SPARK-15563. Resolution: Fixed > Packages specification > -- > > Key: SPARK-15563 >

[jira] [Comment Edited] (SPARK-13599) Groovy-all ends up in spark-assembly if hive profile set

2016-05-26 Thread Catalin Alexandru Zamfir (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13599?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15302192#comment-15302192 ] Catalin Alexandru Zamfir edited comment on SPARK-13599 at 5/26/16 3:50 PM:

[jira] [Comment Edited] (SPARK-13599) Groovy-all ends up in spark-assembly if hive profile set

2016-05-26 Thread Catalin Alexandru Zamfir (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13599?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15302192#comment-15302192 ] Catalin Alexandru Zamfir edited comment on SPARK-13599 at 5/26/16 3:49 PM:

[jira] [Comment Edited] (SPARK-13599) Groovy-all ends up in spark-assembly if hive profile set

2016-05-26 Thread Catalin Alexandru Zamfir (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13599?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15302192#comment-15302192 ] Catalin Alexandru Zamfir edited comment on SPARK-13599 at 5/26/16 3:47 PM:

[jira] [Commented] (SPARK-15563) Packages specification

2016-05-26 Thread jay vyas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15563?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15302269#comment-15302269 ] jay vyas commented on SPARK-15563: -- ok. and, so the ASF Spark has no long term interest

[jira] [Commented] (SPARK-15563) Packages specification

2016-05-26 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15563?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15302259#comment-15302259 ] Sean Owen commented on SPARK-15563: --- I think it should go on the spark-packages site, b

[jira] [Updated] (SPARK-15563) Packages specification

2016-05-26 Thread jay vyas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15563?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jay vyas updated SPARK-15563: - Description: PROBLEM Spark packages should be extensible as a mechanism for individual application deve

[jira] [Updated] (SPARK-15563) Packages specification

2016-05-26 Thread jay vyas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15563?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jay vyas updated SPARK-15563: - Description: PROBLEM Spark packages should be extensible as a mechanism for individual application deve

[jira] [Updated] (SPARK-15563) Packages specification

2016-05-26 Thread jay vyas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15563?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jay vyas updated SPARK-15563: - Description: PROBLEM Spark packages should be extensible as a mechanism for individual application deve

[jira] [Created] (SPARK-15563) Packages specification

2016-05-26 Thread jay vyas (JIRA)
jay vyas created SPARK-15563: Summary: Packages specification Key: SPARK-15563 URL: https://issues.apache.org/jira/browse/SPARK-15563 Project: Spark Issue Type: Improvement Components:

[jira] [Commented] (SPARK-13599) Groovy-all ends up in spark-assembly if hive profile set

2016-05-26 Thread Catalin Alexandru Zamfir (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13599?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15302192#comment-15302192 ] Catalin Alexandru Zamfir commented on SPARK-13599: -- Coming back with som

[jira] [Updated] (SPARK-15526) Shade JPMML

2016-05-26 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15526?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-15526: -- Target Version/s: (was: 2.0.0) > Shade JPMML > --- > > Key: SPARK-15526 >

[jira] [Commented] (SPARK-15562) Temp directory is not deleted after program exit in DataFrameExample

2016-05-26 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15562?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15302128#comment-15302128 ] Sean Owen commented on SPARK-15562: --- @ding this is an example of proliferation of minor

[jira] [Commented] (SPARK-15562) Temp directory is not deleted after program exit in DataFrameExample

2016-05-26 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15562?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15302097#comment-15302097 ] Apache Spark commented on SPARK-15562: -- User 'dding3' has created a pull request for

[jira] [Assigned] (SPARK-15562) Temp directory is not deleted after program exit in DataFrameExample

2016-05-26 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15562?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15562: Assignee: (was: Apache Spark) > Temp directory is not deleted after program exit in Da

[jira] [Assigned] (SPARK-15562) Temp directory is not deleted after program exit in DataFrameExample

2016-05-26 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15562?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15562: Assignee: Apache Spark > Temp directory is not deleted after program exit in DataFrameExam

[jira] [Created] (SPARK-15562) Temp directory is not deleted after program exit in DataFrameExample

2016-05-26 Thread ding (JIRA)
ding created SPARK-15562: Summary: Temp directory is not deleted after program exit in DataFrameExample Key: SPARK-15562 URL: https://issues.apache.org/jira/browse/SPARK-15562 Project: Spark Issue T

[jira] [Updated] (SPARK-10722) Uncaught exception: RDDBlockId not found in driver-heartbeater

2016-05-26 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10722?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-10722: -- Assignee: Simon Scott > Uncaught exception: RDDBlockId not found in driver-heartbeater > --

[jira] [Resolved] (SPARK-10722) Uncaught exception: RDDBlockId not found in driver-heartbeater

2016-05-26 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10722?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-10722. --- Resolution: Fixed Fix Version/s: 1.6.2 Issue resolved by pull request 13222 [https://github.co

[jira] [Updated] (SPARK-15523) Update JPMML to 1.2.15

2016-05-26 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15523?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-15523: -- Assignee: Villu Ruusmann > Update JPMML to 1.2.15 > -- > > Key: SPA

[jira] [Resolved] (SPARK-15523) Update JPMML to 1.2.15

2016-05-26 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15523?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-15523. --- Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 13297 [https://github.co

[jira] [Resolved] (SPARK-15541) SparkContext.stop throws error

2016-05-26 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15541?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-15541. --- Resolution: Not A Problem Although I'm closing as "Not A Problem", it is a subtle and real problem fo

[jira] [Commented] (SPARK-15479) Spark job doesn't shut gracefully in yarn mode.

2016-05-26 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15479?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15302015#comment-15302015 ] Sean Owen commented on SPARK-15479: --- Please show more code, like how you trigger shutdo

[jira] [Commented] (SPARK-15479) Spark job doesn't shut gracefully in yarn mode.

2016-05-26 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15479?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15302016#comment-15302016 ] Sean Owen commented on SPARK-15479: --- Also, please try vs master, not 1.5 > Spark job d

[jira] [Commented] (SPARK-15558) Deadlock when retreiving shuffled cached data

2016-05-26 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15558?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15302003#comment-15302003 ] Sean Owen commented on SPARK-15558: --- Please try vs master if possible; so much has chan

[jira] [Commented] (SPARK-12550) sbt-launch-lib.bash: line 72: 2404 Killed "$@"

2016-05-26 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12550?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15302001#comment-15302001 ] Sean Owen commented on SPARK-12550: --- For avoidance of doubt, what docs are you referrin

[jira] [Resolved] (SPARK-15548) Shuffle file remains with dynamic allocation on YARN

2016-05-26 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15548?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-15548. --- Resolution: Not A Problem Agree, this is by design > Shuffle file remains with dynamic allocation on

[jira] [Resolved] (SPARK-15341) Add documentation for `model.write` to clarify `summary` was not saved

2016-05-26 Thread Yanbo Liang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15341?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yanbo Liang resolved SPARK-15341. - Resolution: Fixed > Add documentation for `model.write` to clarify `summary` was not saved > ---

[jira] [Commented] (SPARK-15034) Use the value of spark.sql.warehouse.dir as the warehouse location instead of using hive.metastore.warehouse.dir

2016-05-26 Thread Yi Zhou (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15034?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15301988#comment-15301988 ] Yi Zhou commented on SPARK-15034: - Hi [~yhuai] I can integrate Spark SQL with hive metast

[jira] [Commented] (SPARK-15558) Deadlock when retreiving shuffled cached data

2016-05-26 Thread Bertrand Bossy (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15558?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15301986#comment-15301986 ] Bertrand Bossy commented on SPARK-15558: IMHO, these are the relevant threads: {c

[jira] [Commented] (SPARK-15407) Floor division

2016-05-26 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15407?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15301957#comment-15301957 ] Herman van Hovell commented on SPARK-15407: --- You can use {{div}} (this will ret

[jira] [Closed] (SPARK-15561) Spark 2.0.0-preview hadoop-provided distribution is missing slf4j dependency

2016-05-26 Thread Jakub Liska (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15561?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jakub Liska closed SPARK-15561. --- Resolution: Invalid > Spark 2.0.0-preview hadoop-provided distribution is missing slf4j dependency >

[jira] [Commented] (SPARK-15561) Spark 2.0.0-preview hadoop-provided distribution is missing slf4j dependency

2016-05-26 Thread Jakub Liska (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15561?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15301954#comment-15301954 ] Jakub Liska commented on SPARK-15561: - Ah, my bad. I double checked that because it s

[jira] [Commented] (SPARK-15553) Dataset.createTempView should use CreateViewCommand

2016-05-26 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15553?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15301931#comment-15301931 ] Apache Spark commented on SPARK-15553: -- User 'viirya' has created a pull request for

[jira] [Assigned] (SPARK-15553) Dataset.createTempView should use CreateViewCommand

2016-05-26 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15553?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15553: Assignee: (was: Apache Spark) > Dataset.createTempView should use CreateViewCommand >

[jira] [Assigned] (SPARK-15553) Dataset.createTempView should use CreateViewCommand

2016-05-26 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15553?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15553: Assignee: Apache Spark > Dataset.createTempView should use CreateViewCommand > ---

[jira] [Resolved] (SPARK-15492) Binarization scala example copy & paste to spark-shell error

2016-05-26 Thread Nick Pentreath (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15492?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Pentreath resolved SPARK-15492. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 13266 [https:/

[jira] [Created] (SPARK-15561) Spark 2.0.0-preview hadoop-provided distribution is missing slf4j dependency

2016-05-26 Thread Jakub Liska (JIRA)
Jakub Liska created SPARK-15561: --- Summary: Spark 2.0.0-preview hadoop-provided distribution is missing slf4j dependency Key: SPARK-15561 URL: https://issues.apache.org/jira/browse/SPARK-15561 Project: S

[jira] [Assigned] (SPARK-15560) Queued/Supervise drivers waiting for retry drivers disappear for kill command in Mesos mode

2016-05-26 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15560?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15560: Assignee: Apache Spark > Queued/Supervise drivers waiting for retry drivers disappear for

[jira] [Commented] (SPARK-15560) Queued/Supervise drivers waiting for retry drivers disappear for kill command in Mesos mode

2016-05-26 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15560?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15301907#comment-15301907 ] Apache Spark commented on SPARK-15560: -- User 'devaraj-kavali' has created a pull req

[jira] [Assigned] (SPARK-15560) Queued/Supervise drivers waiting for retry drivers disappear for kill command in Mesos mode

2016-05-26 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15560?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15560: Assignee: (was: Apache Spark) > Queued/Supervise drivers waiting for retry drivers dis

[jira] [Created] (SPARK-15560) Queued/Supervise drivers waiting for retry drivers disappear for kill command in Mesos mode

2016-05-26 Thread Devaraj K (JIRA)
Devaraj K created SPARK-15560: - Summary: Queued/Supervise drivers waiting for retry drivers disappear for kill command in Mesos mode Key: SPARK-15560 URL: https://issues.apache.org/jira/browse/SPARK-15560

[jira] [Commented] (SPARK-15479) Spark job doesn't shut gracefully in yarn mode.

2016-05-26 Thread Rakesh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15479?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15301873#comment-15301873 ] Rakesh commented on SPARK-15479: Please note that setting for graceful shutdown is set to

[jira] [Updated] (SPARK-15559) TopicAndPartition should provide __hash__ method

2016-05-26 Thread Maciej Szymkiewicz (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15559?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maciej Szymkiewicz updated SPARK-15559: --- Description: In Python 3.x any object that provides eq method requires hash method f

[jira] [Updated] (SPARK-15559) TopicAndPartition should provide __hash__ method

2016-05-26 Thread Maciej Szymkiewicz (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15559?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maciej Szymkiewicz updated SPARK-15559: --- Description: In Python 3.x any object that provides {{__eq__}} method requires {{__ha

[jira] [Updated] (SPARK-15559) TopicAndPartition should provide __hash__ method

2016-05-26 Thread Maciej Szymkiewicz (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15559?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maciej Szymkiewicz updated SPARK-15559: --- Description: In Python 3.x any object that provides `__eq__` method requires {__hash_

[jira] [Commented] (SPARK-15559) TopicAndPartition should provide __hash__ method

2016-05-26 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15559?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15301778#comment-15301778 ] Apache Spark commented on SPARK-15559: -- User 'zero323' has created a pull request fo

[jira] [Assigned] (SPARK-15559) TopicAndPartition should provide __hash__ method

2016-05-26 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15559?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15559: Assignee: (was: Apache Spark) > TopicAndPartition should provide __hash__ method > ---

[jira] [Assigned] (SPARK-15559) TopicAndPartition should provide __hash__ method

2016-05-26 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15559?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15559: Assignee: Apache Spark > TopicAndPartition should provide __hash__ method > --

[jira] [Created] (SPARK-15559) TopicAndPartition should provide __hash__ method

2016-05-26 Thread Maciej Szymkiewicz (JIRA)
Maciej Szymkiewicz created SPARK-15559: -- Summary: TopicAndPartition should provide __hash__ method Key: SPARK-15559 URL: https://issues.apache.org/jira/browse/SPARK-15559 Project: Spark

[jira] [Updated] (SPARK-15558) Deadlock when retreiving shuffled cached data

2016-05-26 Thread Fabiano Francesconi (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15558?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fabiano Francesconi updated SPARK-15558: Attachment: screenshot-1.png > Deadlock when retreiving shuffled cached data >

[jira] [Created] (SPARK-15558) Deadlock when retreiving shuffled cached data

2016-05-26 Thread Fabiano Francesconi (JIRA)
Fabiano Francesconi created SPARK-15558: --- Summary: Deadlock when retreiving shuffled cached data Key: SPARK-15558 URL: https://issues.apache.org/jira/browse/SPARK-15558 Project: Spark I

[jira] [Comment Edited] (SPARK-15407) Floor division

2016-05-26 Thread Joao Ferreira (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15407?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15301759#comment-15301759 ] Joao Ferreira edited comment on SPARK-15407 at 5/26/16 8:40 AM: ---

[jira] [Commented] (SPARK-15407) Floor division

2016-05-26 Thread Joao Ferreira (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15407?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15301759#comment-15301759 ] Joao Ferreira commented on SPARK-15407: --- Hi Marc, In the end we came up with a solu

[jira] [Commented] (SPARK-15407) Floor division

2016-05-26 Thread Marc (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15407?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15301751#comment-15301751 ] Marc commented on SPARK-15407: -- I also got this and it would be very nice to have this featu

[jira] [Created] (SPARK-15557) expression ((cast(99 as decimal) + '3') * '2.3' ) return null

2016-05-26 Thread cen yuhai (JIRA)
cen yuhai created SPARK-15557: - Summary: expression ((cast(99 as decimal) + '3') * '2.3' ) return null Key: SPARK-15557 URL: https://issues.apache.org/jira/browse/SPARK-15557 Project: Spark Issu

[jira] [Comment Edited] (SPARK-15439) Failed to run unit test in SparkR

2016-05-26 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15439?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15301735#comment-15301735 ] Felix Cheung edited comment on SPARK-15439 at 5/26/16 8:22 AM:

[jira] [Commented] (SPARK-15556) Add new Dataset API to get the codegen info for Dataset

2016-05-26 Thread Saisai Shao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15556?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15301738#comment-15301738 ] Saisai Shao commented on SPARK-15556: - I see, thanks a lot for your comment. sorry to

[jira] [Commented] (SPARK-15548) Shuffle file remains with dynamic allocation on YARN

2016-05-26 Thread Saisai Shao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15548?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15301734#comment-15301734 ] Saisai Shao commented on SPARK-15548: - I did a quick test on the shuffle file removin

[jira] [Commented] (SPARK-15439) Failed to run unit test in SparkR

2016-05-26 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15439?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15301735#comment-15301735 ] Felix Cheung commented on SPARK-15439: -- I think we have an issue here - I"m running

[jira] [Commented] (SPARK-13184) Support minPartitions parameter for JSON and CSV datasources as options

2016-05-26 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13184?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15301727#comment-15301727 ] Takeshi Yamamuro commented on SPARK-13184: -- okay, I got your point. I'll fix my

[jira] [Closed] (SPARK-15556) Add new Dataset API to get the codegen info for Dataset

2016-05-26 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15556?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin closed SPARK-15556. --- Resolution: Not A Problem It's already there in queryExecution.debug. This is anyway a developer faci

[jira] [Commented] (SPARK-13184) Support minPartitions parameter for JSON and CSV datasources as options

2016-05-26 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13184?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15301724#comment-15301724 ] Reynold Xin commented on SPARK-13184: - No you can't do that (that will break APIs). Y

[jira] [Commented] (SPARK-13184) Support minPartitions parameter for JSON and CSV datasources as options

2016-05-26 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13184?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15301717#comment-15301717 ] Takeshi Yamamuro commented on SPARK-13184: -- If `SQLConf` is directly passed into

[jira] [Commented] (SPARK-15548) Shuffle file remains with dynamic allocation on YARN

2016-05-26 Thread He Tianyi (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15548?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15301715#comment-15301715 ] He Tianyi commented on SPARK-15548: --- Yes, but we have to restart application every few

[jira] [Created] (SPARK-15556) Add new Dataset API to get the codegen info for Dataset

2016-05-26 Thread Saisai Shao (JIRA)
Saisai Shao created SPARK-15556: --- Summary: Add new Dataset API to get the codegen info for Dataset Key: SPARK-15556 URL: https://issues.apache.org/jira/browse/SPARK-15556 Project: Spark Issue T

[jira] [Commented] (SPARK-15528) conv function returns inconsistent result for the same data

2016-05-26 Thread Lior Regev (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15528?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15301708#comment-15301708 ] Lior Regev commented on SPARK-15528: After further research I found out the if I chan

[jira] [Commented] (SPARK-15555) Driver with --supervise option cannot be killed in Mesos mode

2016-05-26 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15301704#comment-15301704 ] Apache Spark commented on SPARK-1: -- User 'devaraj-kavali' has created a pull req

[jira] [Assigned] (SPARK-15555) Driver with --supervise option cannot be killed in Mesos mode

2016-05-26 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-1: Assignee: (was: Apache Spark) > Driver with --supervise option cannot be killed in Mes

[jira] [Assigned] (SPARK-15555) Driver with --supervise option cannot be killed in Mesos mode

2016-05-26 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-1: Assignee: Apache Spark > Driver with --supervise option cannot be killed in Mesos mode > -

[jira] [Commented] (SPARK-13184) Support minPartitions parameter for JSON and CSV datasources as options

2016-05-26 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13184?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15301697#comment-15301697 ] Reynold Xin commented on SPARK-13184: - I was thinking we can just add a new method to

[jira] [Created] (SPARK-15555) Driver with --supervise option cannot be killed in Mesos mode

2016-05-26 Thread Devaraj K (JIRA)
Devaraj K created SPARK-1: - Summary: Driver with --supervise option cannot be killed in Mesos mode Key: SPARK-1 URL: https://issues.apache.org/jira/browse/SPARK-1 Project: Spark Issu

<    1   2   3   4   >