[jira] [Commented] (SPARK-6996) DataFrame should support collection types when creating DFs from JavaBeans.

2015-04-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6996?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14501752#comment-14501752 ] Apache Spark commented on SPARK-6996: - User 'punya' has created a pull request for

[jira] [Assigned] (SPARK-6996) DataFrame should support collection types when creating DFs from JavaBeans.

2015-04-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6996?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-6996: --- Assignee: Apache Spark DataFrame should support collection types when creating DFs from

[jira] [Assigned] (SPARK-6996) DataFrame should support collection types when creating DFs from JavaBeans.

2015-04-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6996?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-6996: --- Assignee: (was: Apache Spark) DataFrame should support collection types when creating

[jira] [Updated] (SPARK-6995) Spark driver program throws exception cant get Master Kerberose principal for use as renewer

2015-04-19 Thread Umesh Kacha (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6995?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Umesh Kacha updated SPARK-6995: --- Description: I am having thousands of files residing in HDFS which I would like to

[jira] [Updated] (SPARK-6996) DataFrame should support map types when creating DFs from JavaBeans.

2015-04-19 Thread Punya Biswal (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6996?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Punya Biswal updated SPARK-6996: Description: If we have a JavaBean class with fields of map types, SQL throws an exception in

[jira] [Commented] (SPARK-6977) PARSING_ERROR(2) in Spark Streaming

2015-04-19 Thread Saisai Shao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6977?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14501729#comment-14501729 ] Saisai Shao commented on SPARK-6977: Hi [~397090770], which Spark version do you use,

[jira] [Updated] (SPARK-6995) Spark driver program throws exception cant get Master Kerberose principal for use as renewer

2015-04-19 Thread Umesh Kacha (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6995?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Umesh Kacha updated SPARK-6995: --- Description: I am having thousands of files residing in HDFS which I would like to

[jira] [Commented] (SPARK-5923) Very slow query when using Oracle hive metastore and table has lots of partitions

2015-04-19 Thread Matthew Taylor (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5923?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14501800#comment-14501800 ] Matthew Taylor commented on SPARK-5923: --- Postgresql does not have a limitation on IN

[jira] [Updated] (SPARK-6995) Spark driver program throws exception cant get Master Kerberose principal for use as renewer

2015-04-19 Thread Umesh Kacha (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6995?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Umesh Kacha updated SPARK-6995: --- Description: I am having thousands of files residing in HDFS which I would like to

[jira] [Resolved] (SPARK-6963) Flaky test: o.a.s.ContextCleanerSuite automatically cleanup checkpoint

2015-04-19 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6963?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-6963. -- Resolution: Fixed Fix Version/s: 1.4.0 Flaky test: o.a.s.ContextCleanerSuite automatically

[jira] [Updated] (SPARK-6963) Flaky test: o.a.s.ContextCleanerSuite automatically cleanup checkpoint

2015-04-19 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6963?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-6963: - Priority: Major (was: Critical) Flaky test: o.a.s.ContextCleanerSuite automatically cleanup checkpoint

[jira] [Resolved] (SPARK-6156) Not cache in memory again if put memory_and_disk level block after put it in disk after unroll unsuccess in memory.

2015-04-19 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6156?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-6156. -- Resolution: Won't Fix Not cache in memory again if put memory_and_disk level block after put it in

[jira] [Created] (SPARK-6996) DataFrame should support collection types when creating DFs from JavaBeans.

2015-04-19 Thread Punya Biswal (JIRA)
Punya Biswal created SPARK-6996: --- Summary: DataFrame should support collection types when creating DFs from JavaBeans. Key: SPARK-6996 URL: https://issues.apache.org/jira/browse/SPARK-6996 Project:

[jira] [Updated] (SPARK-6999) infinite recursion with createDataFrame(JavaRDD[Row], java.util.List[String])

2015-04-19 Thread Justin Uang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6999?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Justin Uang updated SPARK-6999: --- Description: It looks like {code} def createDataFrame(rowRDD: JavaRDD[Row], columns:

[jira] [Updated] (SPARK-6999) infinite recursion with createDataFrame(JavaRDD[Row], java.util.List[String])

2015-04-19 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6999?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-6999: --- Priority: Critical (was: Major) infinite recursion with createDataFrame(JavaRDD[Row],

[jira] [Updated] (SPARK-6999) infinite recursion with createDataFrame(JavaRDD[Row], java.util.List[String])

2015-04-19 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6999?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-6999: --- Target Version/s: 1.3.2, 1.4.0 infinite recursion with createDataFrame(JavaRDD[Row],

[jira] [Created] (SPARK-6999) infinite recursion with createDataFrame(JavaRDD[Row], java.util.List[String])

2015-04-19 Thread Justin Uang (JIRA)
Justin Uang created SPARK-6999: -- Summary: infinite recursion with createDataFrame(JavaRDD[Row], java.util.List[String]) Key: SPARK-6999 URL: https://issues.apache.org/jira/browse/SPARK-6999 Project:

[jira] [Commented] (SPARK-2243) Support multiple SparkContexts in the same JVM

2015-04-19 Thread Marko Bonaci (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2243?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14502139#comment-14502139 ] Marko Bonaci commented on SPARK-2243: - Your fleet of hosts shares the same JVM?

[jira] [Created] (SPARK-7001) Partitions for a long single line file

2015-04-19 Thread Victor Bashurov (JIRA)
Victor Bashurov created SPARK-7001: -- Summary: Partitions for a long single line file Key: SPARK-7001 URL: https://issues.apache.org/jira/browse/SPARK-7001 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-6998) Make StreamingKMeans `Serializable`

2015-04-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6998?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14502034#comment-14502034 ] Apache Spark commented on SPARK-6998: - User 'zsxwing' has created a pull request for

[jira] [Updated] (SPARK-6998) Make StreamingKMeans `Serializable`

2015-04-19 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6998?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-6998: Component/s: MLlib Make StreamingKMeans `Serializable` ---

[jira] [Assigned] (SPARK-6998) Make StreamingKMeans `Serializable`

2015-04-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6998?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-6998: --- Assignee: Apache Spark Make StreamingKMeans `Serializable`

[jira] [Assigned] (SPARK-6998) Make StreamingKMeans `Serializable`

2015-04-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6998?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-6998: --- Assignee: (was: Apache Spark) Make StreamingKMeans `Serializable`

[jira] [Created] (SPARK-6998) Make StreamingKMeans `Serializable`

2015-04-19 Thread Shixiong Zhu (JIRA)
Shixiong Zhu created SPARK-6998: --- Summary: Make StreamingKMeans `Serializable` Key: SPARK-6998 URL: https://issues.apache.org/jira/browse/SPARK-6998 Project: Spark Issue Type: Bug Affects

[jira] [Created] (SPARK-6997) Convert StringType in LocalTableScan

2015-04-19 Thread Fei Wang (JIRA)
Fei Wang created SPARK-6997: --- Summary: Convert StringType in LocalTableScan Key: SPARK-6997 URL: https://issues.apache.org/jira/browse/SPARK-6997 Project: Spark Issue Type: Bug

[jira] [Assigned] (SPARK-6997) Convert StringType in LocalTableScan

2015-04-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6997?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-6997: --- Assignee: Apache Spark Convert StringType in LocalTableScan

[jira] [Assigned] (SPARK-6997) Convert StringType in LocalTableScan

2015-04-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6997?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-6997: --- Assignee: (was: Apache Spark) Convert StringType in LocalTableScan

[jira] [Commented] (SPARK-6997) Convert StringType in LocalTableScan

2015-04-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6997?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14501856#comment-14501856 ] Apache Spark commented on SPARK-6997: - User 'scwf' has created a pull request for this

[jira] [Commented] (SPARK-6989) Spark 1.3 REPL for Scala 2.11 (2.11.2) fails to start, emitting various arcane compiler errors

2015-04-19 Thread Prashant Sharma (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6989?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14501990#comment-14501990 ] Prashant Sharma commented on SPARK-6989: I am sure, I did not observe these back

[jira] [Commented] (SPARK-6969) Refresh the cached table when REFRESH TABLE is used

2015-04-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6969?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14502194#comment-14502194 ] Apache Spark commented on SPARK-6969: - User 'yhuai' has created a pull request for

[jira] [Assigned] (SPARK-6969) Refresh the cached table when REFRESH TABLE is used

2015-04-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6969?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-6969: --- Assignee: Apache Spark Refresh the cached table when REFRESH TABLE is used

[jira] [Updated] (SPARK-7002) Persist on RDD fails the second time if the action is called on a child RDD without showing a FAILED message

2015-04-19 Thread Tom Hubregtsen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7002?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tom Hubregtsen updated SPARK-7002: -- Description: The major issue is: Persist on RDD fails the second time if the action is called

[jira] [Updated] (SPARK-7002) Persist on RDD fails the second time if the action is called on a child RDD without showing a FAILED message

2015-04-19 Thread Tom Hubregtsen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7002?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tom Hubregtsen updated SPARK-7002: -- Description: The major issue is: Persist on RDD fails the second time if the action is called

[jira] [Updated] (SPARK-7002) Persist on RDD fails the second time if the action is called on a child RDD without showing a FAILED message

2015-04-19 Thread Tom Hubregtsen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7002?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tom Hubregtsen updated SPARK-7002: -- Description: The major issue is: Persist on RDD fails the second time if the action is called

[jira] [Created] (SPARK-7003) Improve reliability of connection failure between Netty block transfer service endpoints

2015-04-19 Thread Aaron Davidson (JIRA)
Aaron Davidson created SPARK-7003: - Summary: Improve reliability of connection failure between Netty block transfer service endpoints Key: SPARK-7003 URL: https://issues.apache.org/jira/browse/SPARK-7003

[jira] [Updated] (SPARK-7003) Improve reliability of connection failure detection between Netty block transfer service endpoints

2015-04-19 Thread Aaron Davidson (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7003?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aaron Davidson updated SPARK-7003: -- Summary: Improve reliability of connection failure detection between Netty block transfer

[jira] [Created] (SPARK-7004) resetProb error in pagerank

2015-04-19 Thread lisendong (JIRA)
lisendong created SPARK-7004: Summary: resetProb error in pagerank Key: SPARK-7004 URL: https://issues.apache.org/jira/browse/SPARK-7004 Project: Spark Issue Type: Bug Reporter:

[jira] [Created] (SPARK-7005) resetProb error in pagerank

2015-04-19 Thread lisendong (JIRA)
lisendong created SPARK-7005: Summary: resetProb error in pagerank Key: SPARK-7005 URL: https://issues.apache.org/jira/browse/SPARK-7005 Project: Spark Issue Type: Bug Components:

[jira] [Commented] (SPARK-6977) PARSING_ERROR(2) in Spark Streaming

2015-04-19 Thread yangping wu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6977?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14502241#comment-14502241 ] yangping wu commented on SPARK-6977: Hi [~jerryshao], Thank you for your attention.

[jira] [Comment Edited] (SPARK-6977) PARSING_ERROR(2) in Spark Streaming

2015-04-19 Thread yangping wu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6977?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14502241#comment-14502241 ] yangping wu edited comment on SPARK-6977 at 4/20/15 2:25 AM: -

[jira] [Commented] (SPARK-6962) Netty BlockTransferService hangs in the middle of SQL query

2015-04-19 Thread Aaron Davidson (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6962?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14502270#comment-14502270 ] Aaron Davidson commented on SPARK-6962: --- I created SPARK-7003 to track a fix to the

[jira] [Commented] (SPARK-7003) Improve reliability of connection failure detection between Netty block transfer service endpoints

2015-04-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7003?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14502268#comment-14502268 ] Apache Spark commented on SPARK-7003: - User 'aarondav' has created a pull request for

[jira] [Assigned] (SPARK-7003) Improve reliability of connection failure detection between Netty block transfer service endpoints

2015-04-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7003?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-7003: --- Assignee: Apache Spark (was: Aaron Davidson) Improve reliability of connection failure

[jira] [Commented] (SPARK-6932) A Prototype of Parameter Server

2015-04-19 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6932?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14502274#comment-14502274 ] Reynold Xin commented on SPARK-6932: [~srowen] why don't we keep this alive for

[jira] [Updated] (SPARK-7005) resetProb error in pagerank

2015-04-19 Thread lisendong (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7005?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] lisendong updated SPARK-7005: - Description: in the page rank code, the resetProb should be divided by #vertex according to the

[jira] [Commented] (SPARK-7004) resetProb error in pagerank

2015-04-19 Thread lisendong (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14502300#comment-14502300 ] lisendong commented on SPARK-7004: -- oh, sorry resetProb error in pagerank

[jira] [Commented] (SPARK-7004) resetProb error in pagerank

2015-04-19 Thread lisendong (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14502299#comment-14502299 ] lisendong commented on SPARK-7004: -- oh, sorry resetProb error in pagerank

[jira] [Resolved] (SPARK-6998) Make StreamingKMeans `Serializable`

2015-04-19 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6998?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-6998. Resolution: Fixed Fix Version/s: 1.4.0 1.3.2 1.2.3

[jira] [Commented] (SPARK-7005) resetProb error in pagerank

2015-04-19 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7005?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14502302#comment-14502302 ] Sean Owen commented on SPARK-7005: -- I don't think this is a bug. All it does is mean that

[jira] [Commented] (SPARK-7005) resetProb error in pagerank

2015-04-19 Thread lisendong (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7005?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14502310#comment-14502310 ] lisendong commented on SPARK-7005: -- I don't think so. because pagerank is a iteration

[jira] [Comment Edited] (SPARK-7005) resetProb error in pagerank

2015-04-19 Thread lisendong (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7005?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14502310#comment-14502310 ] lisendong edited comment on SPARK-7005 at 4/20/15 3:45 AM: --- I

[jira] [Created] (SPARK-7002) Persist on RDD fails the second time if the action is called on a child RDD without showing a FAILED message

2015-04-19 Thread Tom Hubregtsen (JIRA)
Tom Hubregtsen created SPARK-7002: - Summary: Persist on RDD fails the second time if the action is called on a child RDD without showing a FAILED message Key: SPARK-7002 URL:

[jira] [Assigned] (SPARK-6969) Refresh the cached table when REFRESH TABLE is used

2015-04-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6969?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-6969: --- Assignee: (was: Apache Spark) Refresh the cached table when REFRESH TABLE is used

[jira] [Comment Edited] (SPARK-6977) PARSING_ERROR(2) in Spark Streaming

2015-04-19 Thread yangping wu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6977?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14502241#comment-14502241 ] yangping wu edited comment on SPARK-6977 at 4/20/15 2:20 AM: -

[jira] [Comment Edited] (SPARK-6977) PARSING_ERROR(2) in Spark Streaming

2015-04-19 Thread yangping wu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6977?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14502241#comment-14502241 ] yangping wu edited comment on SPARK-6977 at 4/20/15 2:21 AM: -

[jira] [Commented] (SPARK-7000) spark.ml Prediction abstractions should reside in ml.prediction subpackage

2015-04-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7000?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14502291#comment-14502291 ] Apache Spark commented on SPARK-7000: - User 'jkbradley' has created a pull request for

[jira] [Commented] (SPARK-6932) A Prototype of Parameter Server

2015-04-19 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6932?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14502289#comment-14502289 ] Sean Owen commented on SPARK-6932: -- Sure, discussion is good. I wanted to scale back the

[jira] [Assigned] (SPARK-7000) spark.ml Prediction abstractions should reside in ml.prediction subpackage

2015-04-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7000?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-7000: --- Assignee: Joseph K. Bradley (was: Apache Spark) spark.ml Prediction abstractions should

[jira] [Resolved] (SPARK-7004) resetProb error in pagerank

2015-04-19 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7004?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-7004. -- Resolution: Duplicate You created this twice. resetProb error in pagerank

[jira] [Assigned] (SPARK-7000) spark.ml Prediction abstractions should reside in ml.prediction subpackage

2015-04-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7000?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-7000: --- Assignee: Apache Spark (was: Joseph K. Bradley) spark.ml Prediction abstractions should

[jira] [Commented] (SPARK-6932) A Prototype of Parameter Server

2015-04-19 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6932?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14502296#comment-14502296 ] Reynold Xin commented on SPARK-6932: Yup - I just read through the proposal (is there

[jira] [Comment Edited] (SPARK-6932) A Prototype of Parameter Server

2015-04-19 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6932?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14502296#comment-14502296 ] Reynold Xin edited comment on SPARK-6932 at 4/20/15 3:30 AM: -

[jira] [Issue Comment Deleted] (SPARK-7004) resetProb error in pagerank

2015-04-19 Thread lisendong (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7004?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] lisendong updated SPARK-7004: - Comment: was deleted (was: oh, sorry) resetProb error in pagerank ---

[jira] [Issue Comment Deleted] (SPARK-7004) resetProb error in pagerank

2015-04-19 Thread lisendong (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7004?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] lisendong updated SPARK-7004: - Comment: was deleted (was: oh, sorry) resetProb error in pagerank ---

[jira] [Resolved] (SPARK-6983) Update ReceiverTrackerActor to use the new Rpc interface

2015-04-19 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6983?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-6983. Resolution: Fixed Fix Version/s: 1.4.0 Assignee: Shixiong Zhu Update

[jira] [Commented] (SPARK-7005) resetProb error in pagerank

2015-04-19 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7005?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14502311#comment-14502311 ] Sean Owen commented on SPARK-7005: -- It's actually mentioned in the same wikipedia page:

[jira] [Resolved] (SPARK-6979) Replace JobScheduler.eventActor and JobGenerator.eventActor with EventLoop

2015-04-19 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6979?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-6979. Resolution: Fixed Fix Version/s: 1.4.0 Assignee: Shixiong Zhu Replace

[jira] [Updated] (SPARK-6999) infinite recursion with createDataFrame(JavaRDD[Row], java.util.List[String])

2015-04-19 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6999?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-6999: --- Component/s: SQL infinite recursion with createDataFrame(JavaRDD[Row],

[jira] [Updated] (SPARK-7003) Improve reliability of connection failure detection between Netty block transfer service endpoints

2015-04-19 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7003?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-7003: --- Component/s: Spark Core Improve reliability of connection failure detection between Netty

[jira] [Commented] (SPARK-6891) ExecutorAllocationManager will request negative number executors

2015-04-19 Thread Ilya Ganelin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6891?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14502322#comment-14502322 ] Ilya Ganelin commented on SPARK-6891: - [~meiyoula] Any hints on reproducing this aside

[jira] [Commented] (SPARK-3630) Identify cause of Kryo+Snappy PARSING_ERROR

2015-04-19 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3630?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14502323#comment-14502323 ] Patrick Wendell commented on SPARK-3630: [~arahuja] If you are able to upgrade

[jira] [Assigned] (SPARK-6918) Secure HBase with Kerberos does not work over YARN

2015-04-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6918?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-6918: --- Assignee: Apache Spark Secure HBase with Kerberos does not work over YARN

[jira] [Commented] (SPARK-6918) Secure HBase with Kerberos does not work over YARN

2015-04-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6918?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14502324#comment-14502324 ] Apache Spark commented on SPARK-6918: - User 'deanchen' has created a pull request for

[jira] [Assigned] (SPARK-6918) Secure HBase with Kerberos does not work over YARN

2015-04-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6918?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-6918: --- Assignee: (was: Apache Spark) Secure HBase with Kerberos does not work over YARN

[jira] [Commented] (SPARK-2243) Support multiple SparkContexts in the same JVM

2015-04-19 Thread Swaranga Sarma (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2243?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14502326#comment-14502326 ] Swaranga Sarma commented on SPARK-2243: --- No, they are different physical/virtual

[jira] [Commented] (SPARK-7002) Persist on RDD fails the second time if the action is called on a child RDD without showing a FAILED message

2015-04-19 Thread Tom Hubregtsen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7002?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14502329#comment-14502329 ] Tom Hubregtsen commented on SPARK-7002: --- I disagree. I first also thought it was

[jira] [Commented] (SPARK-6932) A Prototype of Parameter Server

2015-04-19 Thread Guoqiang Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6932?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14502332#comment-14502332 ] Guoqiang Li commented on SPARK-6932: I have three questions: 1. {code:none} def

[jira] [Commented] (SPARK-7001) Partitions for a long single line file

2015-04-19 Thread Victor Bashurov (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14502335#comment-14502335 ] Victor Bashurov commented on SPARK-7001: Thanks for your answer, please see my

[jira] [Commented] (SPARK-6977) PARSING_ERROR(2) in Spark Streaming

2015-04-19 Thread Saisai Shao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6977?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14502356#comment-14502356 ] Saisai Shao commented on SPARK-6977: Hi [~397090770], from the code you pasted, seems

[jira] [Updated] (SPARK-6968) Make maniuplating an underlying RDD of a DataFrame easier

2015-04-19 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6968?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-6968: --- Priority: Critical (was: Minor) Make maniuplating an underlying RDD of a DataFrame easier

[jira] [Updated] (SPARK-6968) Make maniuplating an underlying RDD of a DataFrame easier

2015-04-19 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6968?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-6968: --- Labels: (was: DataFrame) Make maniuplating an underlying RDD of a DataFrame easier

[jira] [Updated] (SPARK-6968) Make maniuplating an underlying RDD of a DataFrame easier

2015-04-19 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6968?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-6968: --- Component/s: SQL Make maniuplating an underlying RDD of a DataFrame easier

[jira] [Updated] (SPARK-6968) Make maniuplating an underlying RDD of a DataFrame easier

2015-04-19 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6968?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-6968: --- Component/s: (was: Spark Core) Make maniuplating an underlying RDD of a DataFrame easier

[jira] [Updated] (SPARK-6968) Make maniuplating an underlying RDD of a DataFrame easier

2015-04-19 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6968?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-6968: --- Priority: Minor (was: Critical) Make maniuplating an underlying RDD of a DataFrame easier

[jira] [Commented] (SPARK-6968) Make maniuplating an underlying RDD of a DataFrame easier

2015-04-19 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6968?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14502389#comment-14502389 ] Reynold Xin commented on SPARK-6968: For now you can use this code snippet:

[jira] [Commented] (SPARK-6977) PARSING_ERROR(2) in Spark Streaming

2015-04-19 Thread yangping wu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6977?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14502388#comment-14502388 ] yangping wu commented on SPARK-6977: Hi, [~jerryshao], *var sdb: (String, Obj) =

[jira] [Comment Edited] (SPARK-6977) PARSING_ERROR(2) in Spark Streaming

2015-04-19 Thread yangping wu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6977?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14502388#comment-14502388 ] yangping wu edited comment on SPARK-6977 at 4/20/15 5:55 AM: -

[jira] [Commented] (SPARK-6968) Make maniuplating an underlying RDD of a DataFrame easier

2015-04-19 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6968?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14502386#comment-14502386 ] Patrick Wendell commented on SPARK-6968: So coalesce() is added to the DataFrame

[jira] [Updated] (SPARK-7002) Persist on RDD fails the second time if the action is called on a child RDD without showing a FAILED message

2015-04-19 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7002?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-7002: - Component/s: (was: Spark Shell) Persist on RDD fails the second time if the action is called on a

[jira] [Updated] (SPARK-7002) Persist on RDD fails the second time if the action is called on a child RDD without showing a FAILED message

2015-04-19 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7002?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-7002: - Priority: Minor (was: Major) This looks like a purely cosmetic issue in the debug output, right?

[jira] [Commented] (SPARK-7001) Partitions for a long single line file

2015-04-19 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14502314#comment-14502314 ] Sean Owen commented on SPARK-7001: -- First, it's probably important to note that this is

[jira] [Commented] (SPARK-6961) Cannot save data to parquet files when executing from Windows from a Maven Project

2015-04-19 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6961?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14502316#comment-14502316 ] Patrick Wendell commented on SPARK-6961: Based on looking at the stack trace, I

[jira] [Updated] (SPARK-6959) Support for datetime comparisions in filter for dataframes in pyspark

2015-04-19 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6959?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-6959: --- Issue Type: Improvement (was: Bug) Support for datetime comparisions in filter for

[jira] [Commented] (SPARK-3530) Pipeline and Parameters

2015-04-19 Thread Fan Jiang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3530?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14502364#comment-14502364 ] Fan Jiang commented on SPARK-3530: -- Hi Xiangrui, Which part of this pipeline project

[jira] [Closed] (SPARK-6968) Make maniuplating an underlying RDD of a DataFrame easier

2015-04-19 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6968?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin closed SPARK-6968. -- Resolution: Later Make maniuplating an underlying RDD of a DataFrame easier

[jira] [Comment Edited] (SPARK-6968) Make maniuplating an underlying RDD of a DataFrame easier

2015-04-19 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6968?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14502386#comment-14502386 ] Patrick Wendell edited comment on SPARK-6968 at 4/20/15 5:56 AM:

[jira] [Commented] (SPARK-6869) Add pyspark archives path to PYTHONPATH

2015-04-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6869?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14501860#comment-14501860 ] Apache Spark commented on SPARK-6869: - User 'lianhuiwang' has created a pull request