[jira] [Updated] (SPARK-17692) Document ML/MLlib behavior changes in Spark 2.1

2016-10-13 Thread Yanbo Liang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17692?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yanbo Liang updated SPARK-17692: Description: This JIRA records behavior changes of ML/MLlib between 2.0 and 2.1, so we can note th

[jira] [Commented] (SPARK-12664) Expose raw prediction scores in MultilayerPerceptronClassificationModel

2016-10-13 Thread Yanbo Liang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12664?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15571105#comment-15571105 ] Yanbo Liang commented on SPARK-12664: - [~GayathriMurali] Are you still working on thi

[jira] [Updated] (SPARK-12664) Expose raw prediction scores in MultilayerPerceptronClassificationModel

2016-10-13 Thread Yanbo Liang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12664?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yanbo Liang updated SPARK-12664: Shepherd: Yanbo Liang Target Version/s: 2.1.0 > Expose raw prediction scores in Multila

[jira] [Resolved] (SPARK-11272) Support importing and exporting event logs from HistoryServer web portal

2016-10-13 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11272?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or resolved SPARK-11272. --- Resolution: Fixed Assignee: Alex Bozarth Fix Version/s: 2.1.0 Target Vers

[jira] [Resolved] (SPARK-17899) add a debug mode to keep raw table properties in HiveExternalCatalog

2016-10-13 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17899?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or resolved SPARK-17899. --- Resolution: Fixed Fix Version/s: 2.1.0 > add a debug mode to keep raw table properties in Hive

[jira] [Resolved] (SPARK-17686) Propose to print Scala version in "spark-submit --version" command

2016-10-13 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17686?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or resolved SPARK-17686. --- Resolution: Fixed Assignee: Saisai Shao Fix Version/s: 2.1.0 Target Versi

[jira] [Commented] (SPARK-17409) Query in CTAS is Optimized Twice

2016-10-13 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17409?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15571196#comment-15571196 ] Apache Spark commented on SPARK-17409: -- User 'gatorsmile' has created a pull request

[jira] [Created] (SPARK-17903) MetastoreRelation should talk to external catalog instead of hive client

2016-10-13 Thread Wenchen Fan (JIRA)
Wenchen Fan created SPARK-17903: --- Summary: MetastoreRelation should talk to external catalog instead of hive client Key: SPARK-17903 URL: https://issues.apache.org/jira/browse/SPARK-17903 Project: Spark

[jira] [Created] (SPARK-17904) Add a wrapper function to download and install R packages on each executors.

2016-10-13 Thread Yanbo Liang (JIRA)
Yanbo Liang created SPARK-17904: --- Summary: Add a wrapper function to download and install R packages on each executors. Key: SPARK-17904 URL: https://issues.apache.org/jira/browse/SPARK-17904 Project: S

[jira] [Updated] (SPARK-17904) Add a wrapper function to install R packages on each executors.

2016-10-13 Thread Yanbo Liang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17904?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yanbo Liang updated SPARK-17904: Summary: Add a wrapper function to install R packages on each executors. (was: Add a wrapper funct

[jira] [Updated] (SPARK-17904) Add a wrapper function to install R packages on each executors.

2016-10-13 Thread Yanbo Liang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17904?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yanbo Liang updated SPARK-17904: Description: SparkR provides {{spark.lappy}} to run local R functions in distributed environment,

[jira] [Updated] (SPARK-17904) Add a wrapper function to install R packages on each executors.

2016-10-13 Thread Yanbo Liang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17904?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yanbo Liang updated SPARK-17904: Description: SparkR provides {{spark.lappy}} to run local R functions in distributed environment,

[jira] [Updated] (SPARK-17904) Add a wrapper function to install R packages on each executors.

2016-10-13 Thread Yanbo Liang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17904?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yanbo Liang updated SPARK-17904: Description: SparkR provides {{spark.lappy}} to run local R functions in distributed environment,

[jira] [Commented] (SPARK-17904) Add a wrapper function to install R packages on each executors.

2016-10-13 Thread Yanbo Liang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15571337#comment-15571337 ] Yanbo Liang commented on SPARK-17904: - cc [~shivaram] [~felixcheung] [~sunrui] > Add

[jira] [Assigned] (SPARK-17903) MetastoreRelation should talk to external catalog instead of hive client

2016-10-13 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17903?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17903: Assignee: Wenchen Fan (was: Apache Spark) > MetastoreRelation should talk to external cat

[jira] [Comment Edited] (SPARK-17904) Add a wrapper function to install R packages on each executors.

2016-10-13 Thread Yanbo Liang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15571337#comment-15571337 ] Yanbo Liang edited comment on SPARK-17904 at 10/13/16 9:07 AM:

[jira] [Commented] (SPARK-17903) MetastoreRelation should talk to external catalog instead of hive client

2016-10-13 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17903?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15571342#comment-15571342 ] Apache Spark commented on SPARK-17903: -- User 'cloud-fan' has created a pull request

[jira] [Assigned] (SPARK-17903) MetastoreRelation should talk to external catalog instead of hive client

2016-10-13 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17903?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17903: Assignee: Apache Spark (was: Wenchen Fan) > MetastoreRelation should talk to external cat

[jira] [Commented] (SPARK-17827) StatisticsColumnSuite failures on big endian platforms

2016-10-13 Thread Pete Robbins (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17827?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15571348#comment-15571348 ] Pete Robbins commented on SPARK-17827: -- In Statistics,scala case class StringColumn

[jira] [Comment Edited] (SPARK-17827) StatisticsColumnSuite failures on big endian platforms

2016-10-13 Thread Pete Robbins (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17827?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15571348#comment-15571348 ] Pete Robbins edited comment on SPARK-17827 at 10/13/16 9:10 AM: ---

[jira] [Comment Edited] (SPARK-17827) StatisticsColumnSuite failures on big endian platforms

2016-10-13 Thread Pete Robbins (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17827?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15571348#comment-15571348 ] Pete Robbins edited comment on SPARK-17827 at 10/13/16 9:11 AM: ---

[jira] [Commented] (SPARK-17904) Add a wrapper function to install R packages on each executors.

2016-10-13 Thread Jeff Zhang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15571366#comment-15571366 ] Jeff Zhang commented on SPARK-17904: It make sense to provide such api to install pac

[jira] [Created] (SPARK-17905) Added test cases for InMemoryRelation

2016-10-13 Thread Kazuaki Ishizaki (JIRA)
Kazuaki Ishizaki created SPARK-17905: Summary: Added test cases for InMemoryRelation Key: SPARK-17905 URL: https://issues.apache.org/jira/browse/SPARK-17905 Project: Spark Issue Type: Tes

[jira] [Commented] (SPARK-17904) Add a wrapper function to install R packages on each executors.

2016-10-13 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15571377#comment-15571377 ] Sean Owen commented on SPARK-17904: --- I think this is inconsistent with how Python packa

[jira] [Assigned] (SPARK-16845) org.apache.spark.sql.catalyst.expressions.GeneratedClass$SpecificOrdering" grows beyond 64 KB

2016-10-13 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16845?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16845: Assignee: Apache Spark > org.apache.spark.sql.catalyst.expressions.GeneratedClass$Specific

[jira] [Assigned] (SPARK-16845) org.apache.spark.sql.catalyst.expressions.GeneratedClass$SpecificOrdering" grows beyond 64 KB

2016-10-13 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16845?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16845: Assignee: (was: Apache Spark) > org.apache.spark.sql.catalyst.expressions.GeneratedCla

[jira] [Commented] (SPARK-16845) org.apache.spark.sql.catalyst.expressions.GeneratedClass$SpecificOrdering" grows beyond 64 KB

2016-10-13 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16845?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15571380#comment-15571380 ] Apache Spark commented on SPARK-16845: -- User 'lw-lin' has created a pull request for

[jira] [Commented] (SPARK-16845) org.apache.spark.sql.catalyst.expressions.GeneratedClass$SpecificOrdering" grows beyond 64 KB

2016-10-13 Thread Liwei Lin(Inactive) (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16845?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15571388#comment-15571388 ] Liwei Lin(Inactive) commented on SPARK-16845: - thank you [~kiszk] this is rea

[jira] [Commented] (SPARK-17904) Add a wrapper function to install R packages on each executors.

2016-10-13 Thread Yanbo Liang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15571387#comment-15571387 ] Yanbo Liang commented on SPARK-17904: - [~zjffdu] Thanks for your reply. For R users,

[jira] [Resolved] (SPARK-17883) Possible typo in comments of Row.scala

2016-10-13 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17883?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-17883. --- Resolution: Fixed Assignee: Weiluo Ren Fix Version/s: 2.1.0 OK, have a look at https

[jira] [Updated] (SPARK-17883) Possible typo in comments of Row.scala

2016-10-13 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17883?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-17883: -- Fix Version/s: 2.0.2 > Possible typo in comments of Row.scala > --

[jira] [Updated] (SPARK-17894) Ensure uniqueness of TaskSetManager name

2016-10-13 Thread Eren Avsarogullari (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17894?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eren Avsarogullari updated SPARK-17894: --- Summary: Ensure uniqueness of TaskSetManager name (was: Uniqueness of TaskSetManager

[jira] [Resolved] (SPARK-17893) Window functions should also allow looking back in time

2016-10-13 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17893?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-17893. --- Resolution: Not A Problem This might be better as a question on the mailing list, not a JIRA. THis is

[jira] [Updated] (SPARK-17894) Ensure uniqueness of TaskSetManager name

2016-10-13 Thread Eren Avsarogullari (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17894?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eren Avsarogullari updated SPARK-17894: --- Description: TaskSetManager should have unique name to avoid adding duplicate ones to

[jira] [Commented] (SPARK-9487) Use the same num. worker threads in Scala/Python unit tests

2016-10-13 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9487?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15571400#comment-15571400 ] Sean Owen commented on SPARK-9487: -- I think you can just make the change and see what hap

[jira] [Resolved] (SPARK-8337) KafkaUtils.createDirectStream for python is lacking API/feature parity with the Scala/Java version

2016-10-13 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8337?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-8337. -- Resolution: Done > KafkaUtils.createDirectStream for python is lacking API/feature parity with > the Sc

[jira] [Commented] (SPARK-17904) Add a wrapper function to install R packages on each executors.

2016-10-13 Thread Yanbo Liang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15571414#comment-15571414 ] Yanbo Liang commented on SPARK-17904: - [~srowen] Yeah, this proposal is different fro

[jira] [Commented] (SPARK-17680) Unicode Character Support for Column Names and Comments

2016-10-13 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17680?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15571425#comment-15571425 ] Apache Spark commented on SPARK-17680: -- User 'kiszk' has created a pull request for

[jira] [Commented] (SPARK-17890) scala.ScalaReflectionException

2016-10-13 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17890?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15571437#comment-15571437 ] Sean Owen commented on SPARK-17890: --- I think this ends up being yet another issue with

[jira] [Commented] (SPARK-17898) --repositories needs username and password

2016-10-13 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17898?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15571440#comment-15571440 ] Sean Owen commented on SPARK-17898: --- I don't think the goal of this feature is to suppo

[jira] [Commented] (SPARK-17827) StatisticsColumnSuite failures on big endian platforms

2016-10-13 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17827?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15571448#comment-15571448 ] Sean Owen commented on SPARK-17827: --- [~ZenWzh] I think you added this? what do you thin

[jira] [Commented] (SPARK-10590) Spark with YARN build is broken

2016-10-13 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10590?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15571470#comment-15571470 ] Sean Owen commented on SPARK-10590: --- I have not seen this error and all Jenkins builds

[jira] [Resolved] (SPARK-17901) NettyRpcEndpointRef: Error sending message and Caused by: java.util.ConcurrentModificationException

2016-10-13 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17901?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-17901. --- Resolution: Duplicate > NettyRpcEndpointRef: Error sending message and Caused by: > java.util.Concur

[jira] [Commented] (SPARK-17888) Memory leak in streaming driver when use SparkSQL in Streaming

2016-10-13 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17888?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15571476#comment-15571476 ] Sean Owen commented on SPARK-17888: --- What makes you believe there is a leak? if you're

[jira] [Commented] (SPARK-16599) java.util.NoSuchElementException: None.get at at org.apache.spark.storage.BlockInfoManager.releaseAllLocksForTask(BlockInfoManager.scala:343)

2016-10-13 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16599?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15571478#comment-15571478 ] Sean Owen commented on SPARK-16599: --- What You See Is What You Get here. Nobody is worki

[jira] [Commented] (SPARK-17827) StatisticsColumnSuite failures on big endian platforms

2016-10-13 Thread Pete Robbins (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17827?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15571546#comment-15571546 ] Pete Robbins commented on SPARK-17827: -- right so in these two cases maxLength in A

[jira] [Commented] (SPARK-17893) Window functions should also allow looking back in time

2016-10-13 Thread Raviteja Lokineni (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17893?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15571556#comment-15571556 ] Raviteja Lokineni commented on SPARK-17893: --- [~srowen] I don't find anything in

[jira] [Commented] (SPARK-17893) Window functions should also allow looking back in time

2016-10-13 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17893?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15571570#comment-15571570 ] Sean Owen commented on SPARK-17893: --- OK, this is about streaming? the syntax lets you s

[jira] [Commented] (SPARK-17893) Window functions should also allow looking back in time

2016-10-13 Thread Raviteja Lokineni (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17893?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15571582#comment-15571582 ] Raviteja Lokineni commented on SPARK-17893: --- [~srowen] No it's not actually str

[jira] [Comment Edited] (SPARK-17893) Window functions should also allow looking back in time

2016-10-13 Thread Raviteja Lokineni (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17893?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15571582#comment-15571582 ] Raviteja Lokineni edited comment on SPARK-17893 at 10/13/16 10:51 AM: -

[jira] [Comment Edited] (SPARK-17893) Window functions should also allow looking back in time

2016-10-13 Thread Raviteja Lokineni (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17893?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15571582#comment-15571582 ] Raviteja Lokineni edited comment on SPARK-17893 at 10/13/16 10:52 AM: -

[jira] [Commented] (SPARK-17893) Window functions should also allow looking back in time

2016-10-13 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17893?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15571599#comment-15571599 ] Sean Owen commented on SPARK-17893: --- OK, well windows here aren't inherently including

[jira] [Commented] (SPARK-17893) Window functions should also allow looking back in time

2016-10-13 Thread Raviteja Lokineni (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17893?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15571608#comment-15571608 ] Raviteja Lokineni commented on SPARK-17893: --- The tricky part is we don't a reco

[jira] [Commented] (SPARK-17893) Window functions should also allow looking back in time

2016-10-13 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17893?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15571623#comment-15571623 ] Sean Owen commented on SPARK-17893: --- Maybe: - Aggregate by day - Generate a DataFrame c

[jira] [Commented] (SPARK-17894) Ensure uniqueness of TaskSetManager name

2016-10-13 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17894?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15571690#comment-15571690 ] Apache Spark commented on SPARK-17894: -- User 'erenavsarogullari' has created a pull

[jira] [Assigned] (SPARK-17894) Ensure uniqueness of TaskSetManager name

2016-10-13 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17894?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17894: Assignee: Apache Spark > Ensure uniqueness of TaskSetManager name > --

[jira] [Assigned] (SPARK-17894) Ensure uniqueness of TaskSetManager name

2016-10-13 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17894?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17894: Assignee: (was: Apache Spark) > Ensure uniqueness of TaskSetManager name > ---

[jira] [Updated] (SPARK-17759) FairSchedulableBuilder should avoid to create duplicate fair scheduler-pools.

2016-10-13 Thread Eren Avsarogullari (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17759?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eren Avsarogullari updated SPARK-17759: --- Description: If _spark.scheduler.allocation.file_ has duplicate pools, all of them ar

[jira] [Updated] (SPARK-17759) Avoid adding duplicate schedulables

2016-10-13 Thread Eren Avsarogullari (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17759?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eren Avsarogullari updated SPARK-17759: --- Summary: Avoid adding duplicate schedulables (was: FairSchedulableBuilder should avo

[jira] [Commented] (SPARK-17893) Window functions should also allow looking back in time

2016-10-13 Thread Raviteja Lokineni (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17893?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15571728#comment-15571728 ] Raviteja Lokineni commented on SPARK-17893: --- I just hope that aggregates won't

[jira] [Commented] (SPARK-17904) Add a wrapper function to install R packages on each executors.

2016-10-13 Thread Sun Rui (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15571785#comment-15571785 ] Sun Rui commented on SPARK-17904: - this is a little bit tricky. You don't know the exact

[jira] [Updated] (SPARK-17906) MulticlassClassificationEvaluator support target label

2016-10-13 Thread zhengruifeng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17906?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhengruifeng updated SPARK-17906: - Description: In practice, I sometime only focus on metric of one special label. For example, in C

[jira] [Created] (SPARK-17906) MulticlassClassificationEvaluator support target label

2016-10-13 Thread zhengruifeng (JIRA)
zhengruifeng created SPARK-17906: Summary: MulticlassClassificationEvaluator support target label Key: SPARK-17906 URL: https://issues.apache.org/jira/browse/SPARK-17906 Project: Spark Issue

[jira] [Commented] (SPARK-17606) New batches are not created when there are 1000 created after restarting streaming from checkpoint.

2016-10-13 Thread etienne (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17606?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15571870#comment-15571870 ] etienne commented on SPARK-17606: - Theses string are log as DEBUG or in upper logs level

[jira] [Commented] (SPARK-17827) StatisticsColumnSuite failures on big endian platforms

2016-10-13 Thread Zhenhua Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17827?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15571881#comment-15571881 ] Zhenhua Wang commented on SPARK-17827: -- [~srowen] Thanks for notification. [~robbins

[jira] [Comment Edited] (SPARK-17606) New batches are not created when there are 1000 created after restarting streaming from checkpoint.

2016-10-13 Thread etienne (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17606?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15571870#comment-15571870 ] etienne edited comment on SPARK-17606 at 10/13/16 1:17 PM: --- The

[jira] [Commented] (SPARK-16599) java.util.NoSuchElementException: None.get at at org.apache.spark.storage.BlockInfoManager.releaseAllLocksForTask(BlockInfoManager.scala:343)

2016-10-13 Thread Joseph Naegele (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16599?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15571885#comment-15571885 ] Joseph Naegele commented on SPARK-16599: I apologize for not replying. I don't ha

[jira] [Commented] (SPARK-17827) StatisticsColumnSuite failures on big endian platforms

2016-10-13 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17827?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15571901#comment-15571901 ] Apache Spark commented on SPARK-17827: -- User 'robbinspg' has created a pull request

[jira] [Assigned] (SPARK-17827) StatisticsColumnSuite failures on big endian platforms

2016-10-13 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17827?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17827: Assignee: (was: Apache Spark) > StatisticsColumnSuite failures on big endian platforms

[jira] [Assigned] (SPARK-17827) StatisticsColumnSuite failures on big endian platforms

2016-10-13 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17827?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17827: Assignee: Apache Spark > StatisticsColumnSuite failures on big endian platforms >

[jira] [Commented] (SPARK-17827) StatisticsColumnSuite failures on big endian platforms

2016-10-13 Thread Pete Robbins (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17827?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15571890#comment-15571890 ] Pete Robbins commented on SPARK-17827: -- I have a PR ready which I will submit as soo

[jira] [Updated] (SPARK-17657) Disallow Users to Change Table Type

2016-10-13 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17657?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan updated SPARK-17657: Assignee: Xiao Li > Disallow Users to Change Table Type > > >

[jira] [Commented] (SPARK-17906) MulticlassClassificationEvaluator support target label

2016-10-13 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17906?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15571946#comment-15571946 ] Sean Owen commented on SPARK-17906: --- Are you describing what the old MulticlassMetrics

[jira] [Reopened] (SPARK-17572) Write.df is failing on spark cluster

2016-10-13 Thread Sankar Mittapally (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17572?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sankar Mittapally reopened SPARK-17572: --- I didn't see any issue from NFS side, I don't know how you confirmed it is not related t

[jira] [Commented] (SPARK-17572) Write.df is failing on spark cluster

2016-10-13 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17572?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15571960#comment-15571960 ] Sean Owen commented on SPARK-17572: --- [~sankar.mittapa...@creditvidya.com] see the error

[jira] [Created] (SPARK-17907) Not allowing more spark console

2016-10-13 Thread Sankar Mittapally (JIRA)
Sankar Mittapally created SPARK-17907: - Summary: Not allowing more spark console Key: SPARK-17907 URL: https://issues.apache.org/jira/browse/SPARK-17907 Project: Spark Issue Type: Bug

[jira] [Resolved] (SPARK-17657) Disallow Users to Change Table Type

2016-10-13 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17657?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-17657. - Resolution: Fixed Fix Version/s: 2.1.0 Issue resolved by pull request 15230 [https://githu

[jira] [Resolved] (SPARK-17907) Not allowing more spark console

2016-10-13 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17907?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-17907. --- Resolution: Invalid The error tells you the problem: {code} ValueError: Cannot run multiple SparkCo

[jira] [Commented] (SPARK-17904) Add a wrapper function to install R packages on each executors.

2016-10-13 Thread Yanbo Liang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15572108#comment-15572108 ] Yanbo Liang commented on SPARK-17904: - Thanks your comments. Yeah, I agree it's trick

[jira] [Updated] (SPARK-9862) Join: Handling data skew

2016-10-13 Thread wangyuhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9862?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wangyuhu updated SPARK-9862: Attachment: Handling skew data in join.pdf > Join: Handling data skew > > >

[jira] [Updated] (SPARK-17904) Add a wrapper function to install R packages on each executors.

2016-10-13 Thread Yanbo Liang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17904?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yanbo Liang updated SPARK-17904: Description: SparkR provides {{spark.lappy}} to run local R functions in distributed environment,

[jira] [Updated] (SPARK-17904) Add a wrapper function to install R packages on each executors.

2016-10-13 Thread Yanbo Liang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17904?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yanbo Liang updated SPARK-17904: Description: SparkR provides {{spark.lappy}} to run local R functions in distributed environment,

[jira] [Updated] (SPARK-17904) Add a wrapper function to install R packages on each executors.

2016-10-13 Thread Yanbo Liang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17904?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yanbo Liang updated SPARK-17904: Description: SparkR provides {{spark.lappy}} to run local R functions in distributed environment,

[jira] [Commented] (SPARK-17900) Mark the following Spark SQL APIs as stable

2016-10-13 Thread Cody Koeninger (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17900?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15572119#comment-15572119 ] Cody Koeninger commented on SPARK-17900: Thanks for doing this, should make thing

[jira] [Updated] (SPARK-17904) Add a wrapper function to install R packages on each executors.

2016-10-13 Thread Yanbo Liang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17904?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yanbo Liang updated SPARK-17904: Description: SparkR provides {{spark.lappy}} to run local R functions in distributed environment,

[jira] [Commented] (SPARK-17906) MulticlassClassificationEvaluator support target label

2016-10-13 Thread Seth Hendrickson (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17906?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15572154#comment-15572154 ] Seth Hendrickson commented on SPARK-17906: -- We are adding model summaries that w

[jira] [Created] (SPARK-17908) Column names Corrupted in pysaprk dataframe groupBy

2016-10-13 Thread Harish (JIRA)
Harish created SPARK-17908: -- Summary: Column names Corrupted in pysaprk dataframe groupBy Key: SPARK-17908 URL: https://issues.apache.org/jira/browse/SPARK-17908 Project: Spark Issue Type: Bug

[jira] [Created] (SPARK-17909) we should create table before writing out the data in CTAS

2016-10-13 Thread Wenchen Fan (JIRA)
Wenchen Fan created SPARK-17909: --- Summary: we should create table before writing out the data in CTAS Key: SPARK-17909 URL: https://issues.apache.org/jira/browse/SPARK-17909 Project: Spark Issu

[jira] [Assigned] (SPARK-17909) we should create table before writing out the data in CTAS

2016-10-13 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17909?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17909: Assignee: Wenchen Fan (was: Apache Spark) > we should create table before writing out the

[jira] [Commented] (SPARK-17909) we should create table before writing out the data in CTAS

2016-10-13 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17909?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15572179#comment-15572179 ] Apache Spark commented on SPARK-17909: -- User 'cloud-fan' has created a pull request

[jira] [Assigned] (SPARK-17909) we should create table before writing out the data in CTAS

2016-10-13 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17909?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17909: Assignee: Apache Spark (was: Wenchen Fan) > we should create table before writing out the

[jira] [Commented] (SPARK-17908) Column names Corrupted in pysaprk dataframe groupBy

2016-10-13 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17908?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15572189#comment-15572189 ] Sean Owen commented on SPARK-17908: --- Can you provide the error and a self-contained rep

[jira] [Commented] (SPARK-17890) scala.ScalaReflectionException

2016-10-13 Thread Khalid Reid (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17890?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15572217#comment-15572217 ] Khalid Reid commented on SPARK-17890: - Hi Sean, I've created a small project to repr

[jira] [Comment Edited] (SPARK-17890) scala.ScalaReflectionException

2016-10-13 Thread Khalid Reid (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17890?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15572217#comment-15572217 ] Khalid Reid edited comment on SPARK-17890 at 10/13/16 3:17 PM:

[jira] [Comment Edited] (SPARK-17890) scala.ScalaReflectionException

2016-10-13 Thread Khalid Reid (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17890?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15572217#comment-15572217 ] Khalid Reid edited comment on SPARK-17890 at 10/13/16 3:19 PM:

[jira] [Comment Edited] (SPARK-17890) scala.ScalaReflectionException

2016-10-13 Thread Khalid Reid (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17890?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15572217#comment-15572217 ] Khalid Reid edited comment on SPARK-17890 at 10/13/16 3:18 PM:

[jira] [Created] (SPARK-17910) Allow users to update the comment of a column

2016-10-13 Thread Yin Huai (JIRA)
Yin Huai created SPARK-17910: Summary: Allow users to update the comment of a column Key: SPARK-17910 URL: https://issues.apache.org/jira/browse/SPARK-17910 Project: Spark Issue Type: Bug

[jira] [Assigned] (SPARK-13983) HiveThriftServer2 can not get "--hiveconf" or ''--hivevar" variables since 1.6 version (both multi-session and single session)

2016-10-13 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13983?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13983: Assignee: Cheng Lian (was: Apache Spark) > HiveThriftServer2 can not get "--hiveconf" or

[jira] [Commented] (SPARK-13983) HiveThriftServer2 can not get "--hiveconf" or ''--hivevar" variables since 1.6 version (both multi-session and single session)

2016-10-13 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15572249#comment-15572249 ] Apache Spark commented on SPARK-13983: -- User 'wangyum' has created a pull request fo

[jira] [Assigned] (SPARK-13983) HiveThriftServer2 can not get "--hiveconf" or ''--hivevar" variables since 1.6 version (both multi-session and single session)

2016-10-13 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13983?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13983: Assignee: Apache Spark (was: Cheng Lian) > HiveThriftServer2 can not get "--hiveconf" or

  1   2   3   >