[jira] [Assigned] (SPARK-5681) Calling graceful stop() immediately after start() on StreamingContext should not get stuck indefinitely

2015-04-03 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5681?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-5681: --- Assignee: (was: Apache Spark) Calling graceful stop() immediately after start() on

[jira] [Assigned] (SPARK-5681) Calling graceful stop() immediately after start() on StreamingContext should not get stuck indefinitely

2015-04-03 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5681?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-5681: --- Assignee: Apache Spark Calling graceful stop() immediately after start() on

[jira] [Assigned] (SPARK-6694) SparkSQL CLI must be able to specify an option --database on the command line.

2015-04-03 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6694?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-6694: --- Assignee: (was: Apache Spark) SparkSQL CLI must be able to specify an option --database

[jira] [Commented] (SPARK-6694) SparkSQL CLI must be able to specify an option --database on the command line.

2015-04-03 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6694?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14394191#comment-14394191 ] Apache Spark commented on SPARK-6694: - User 'adachij2002' has created a pull request

[jira] [Commented] (SPARK-6428) Add to style checker public method must have explicit type defined

2015-04-03 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6428?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14394121#comment-14394121 ] Apache Spark commented on SPARK-6428: - User 'rxin' has created a pull request for this

[jira] [Assigned] (SPARK-6428) Add to style checker public method must have explicit type defined

2015-04-03 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6428?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-6428: --- Assignee: Apache Spark (was: Reynold Xin) Add to style checker public method must have

[jira] [Reopened] (SPARK-1095) Ensure all public methods return explicit types

2015-04-03 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1095?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin reopened SPARK-1095: Assignee: Reynold Xin (was: prashant) Ensure all public methods return explicit types

[jira] [Updated] (SPARK-6692) Make it possible to kill AM in YARN cluster mode when the client is terminated

2015-04-03 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6692?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-6692: --- Assignee: Cheolsoo Park Make it possible to kill AM in YARN cluster mode when the client is

[jira] [Commented] (SPARK-6693) add to string with max lines and width for matrix

2015-04-03 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14394146#comment-14394146 ] Apache Spark commented on SPARK-6693: - User 'hhbyyh' has created a pull request for

[jira] [Assigned] (SPARK-6693) add to string with max lines and width for matrix

2015-04-03 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6693?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-6693: --- Assignee: (was: Apache Spark) add to string with max lines and width for matrix

[jira] [Commented] (SPARK-6664) Split Ordered RDD into multiple RDDs by keys (boundaries or intervals)

2015-04-03 Thread Florian Verhein (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6664?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14394147#comment-14394147 ] Florian Verhein commented on SPARK-6664: I guess the other thing is - we can union

[jira] [Created] (SPARK-6693) add to string with max lines and width for matrix

2015-04-03 Thread yuhao yang (JIRA)
yuhao yang created SPARK-6693: - Summary: add to string with max lines and width for matrix Key: SPARK-6693 URL: https://issues.apache.org/jira/browse/SPARK-6693 Project: Spark Issue Type:

[jira] [Assigned] (SPARK-6693) add to string with max lines and width for matrix

2015-04-03 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6693?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-6693: --- Assignee: Apache Spark add to string with max lines and width for matrix

[jira] [Updated] (SPARK-6693) add toString with max lines and width for matrix

2015-04-03 Thread yuhao yang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6693?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] yuhao yang updated SPARK-6693: -- Summary: add toString with max lines and width for matrix (was: add to string with max lines and width

[jira] [Assigned] (SPARK-6211) Test Python Kafka API using Python unit tests

2015-04-03 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6211?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-6211: --- Assignee: Saisai Shao (was: Apache Spark) Test Python Kafka API using Python unit tests

[jira] [Assigned] (SPARK-6211) Test Python Kafka API using Python unit tests

2015-04-03 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6211?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-6211: --- Assignee: Apache Spark (was: Saisai Shao) Test Python Kafka API using Python unit tests

[jira] [Created] (SPARK-6691) Abstract and add a dynamic RateLimiter for Spark Streaming

2015-04-03 Thread Saisai Shao (JIRA)
Saisai Shao created SPARK-6691: -- Summary: Abstract and add a dynamic RateLimiter for Spark Streaming Key: SPARK-6691 URL: https://issues.apache.org/jira/browse/SPARK-6691 Project: Spark Issue

[jira] [Assigned] (SPARK-6428) Add to style checker public method must have explicit type defined

2015-04-03 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6428?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-6428: --- Assignee: Reynold Xin (was: Apache Spark) Add to style checker public method must have

[jira] [Assigned] (SPARK-5523) TaskMetrics and TaskInfo have innumerable copies of the hostname string

2015-04-03 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5523?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-5523: --- Assignee: (was: Apache Spark) TaskMetrics and TaskInfo have innumerable copies of the

[jira] [Commented] (SPARK-6664) Split Ordered RDD into multiple RDDs by keys (boundaries or intervals)

2015-04-03 Thread Florian Verhein (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6664?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14394141#comment-14394141 ] Florian Verhein commented on SPARK-6664: Thanks [~sowen]. I disagree :-) ...If

[jira] [Created] (SPARK-6694) SparkSQL CLI must be able to specify an option --database on the command line.

2015-04-03 Thread Jin Adachi (JIRA)
Jin Adachi created SPARK-6694: - Summary: SparkSQL CLI must be able to specify an option --database on the command line. Key: SPARK-6694 URL: https://issues.apache.org/jira/browse/SPARK-6694 Project:

[jira] [Created] (SPARK-6692) Make it possible to kill AM in YARN cluster mode when the client is terminated

2015-04-03 Thread Cheolsoo Park (JIRA)
Cheolsoo Park created SPARK-6692: Summary: Make it possible to kill AM in YARN cluster mode when the client is terminated Key: SPARK-6692 URL: https://issues.apache.org/jira/browse/SPARK-6692

[jira] [Assigned] (SPARK-5523) TaskMetrics and TaskInfo have innumerable copies of the hostname string

2015-04-03 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5523?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-5523: --- Assignee: Apache Spark TaskMetrics and TaskInfo have innumerable copies of the hostname

[jira] [Commented] (SPARK-6694) SparkSQL CLI must be able to specify an option --database on the command line.

2015-04-03 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6694?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14394211#comment-14394211 ] Sean Owen commented on SPARK-6694: -- What problem do you encounter? You only showed the

[jira] [Resolved] (SPARK-6560) PairRDDFunctions suppresses exceptions in writeFile

2015-04-03 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6560?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-6560. -- Resolution: Fixed Fix Version/s: 1.4.0 Issue resolved by pull request 5223

[jira] [Commented] (SPARK-6568) spark-shell.cmd --jars option does not accept the jar that has space in its path

2015-04-03 Thread Masayoshi TSUZUKI (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6568?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14394240#comment-14394240 ] Masayoshi TSUZUKI commented on SPARK-6568: -- {code} bin\spark-shell.cmd --jars

[jira] [Updated] (SPARK-6694) SparkSQL CLI must be able to specify an option --database on the command line.

2015-04-03 Thread Jin Adachi (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6694?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jin Adachi updated SPARK-6694: -- Description: SparkSQL CLI has an option --database as follows. But, the option --database is ignored.

[jira] [Commented] (SPARK-6569) Kafka directInputStream logs what appear to be incorrect warnings

2015-04-03 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6569?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14394262#comment-14394262 ] Sean Owen commented on SPARK-6569: -- [~c...@koeninger.org] what do you think about just

[jira] [Updated] (SPARK-3468) WebUI Timeline-View feature

2015-04-03 Thread Kousuke Saruta (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3468?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta updated SPARK-3468: -- Attachment: (was: stages.png) WebUI Timeline-View feature ---

[jira] [Updated] (SPARK-3468) WebUI Timeline-View feature

2015-04-03 Thread Kousuke Saruta (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3468?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta updated SPARK-3468: -- Attachment: (was: taskDetails.png) WebUI Timeline-View feature

[jira] [Updated] (SPARK-3468) WebUI Timeline-View feature

2015-04-03 Thread Kousuke Saruta (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3468?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta updated SPARK-3468: -- Attachment: (was: stage-timeline.png) WebUI Timeline-View feature

[jira] [Updated] (SPARK-3468) WebUI Timeline-View feature

2015-04-03 Thread Kousuke Saruta (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3468?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta updated SPARK-3468: -- Attachment: (was: executors.png) WebUI Timeline-View feature ---

[jira] [Updated] (SPARK-3468) WebUI Timeline-View feature

2015-04-03 Thread Kousuke Saruta (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3468?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta updated SPARK-3468: -- Attachment: (was: tasks.png) WebUI Timeline-View feature ---

[jira] [Issue Comment Deleted] (SPARK-3468) WebUI Timeline-View feature

2015-04-03 Thread Kousuke Saruta (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3468?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta updated SPARK-3468: -- Comment: was deleted (was: Sorry for pending this ticket for a long time. I've re considered

[jira] [Updated] (SPARK-3468) WebUI Timeline-View feature

2015-04-03 Thread Kousuke Saruta (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3468?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta updated SPARK-3468: -- Attachment: TaskAssignmentTimelineView.png JobTimelineView.png

[jira] [Assigned] (SPARK-6489) Optimize lateral view with explode to not read unnecessary columns

2015-04-03 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6489?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-6489: --- Assignee: (was: Apache Spark) Optimize lateral view with explode to not read

[jira] [Updated] (SPARK-6695) Add an external iterator: a hadoop-like output collector

2015-04-03 Thread uncleGen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6695?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] uncleGen updated SPARK-6695: Description: In practical use, we usually need to create a big iterator, which means too big in `memory

[jira] [Assigned] (SPARK-6568) spark-shell.cmd --jars option does not accept the jar that has space in its path

2015-04-03 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6568?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-6568: --- Assignee: (was: Apache Spark) spark-shell.cmd --jars option does not accept the jar

[jira] [Assigned] (SPARK-6568) spark-shell.cmd --jars option does not accept the jar that has space in its path

2015-04-03 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6568?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-6568: --- Assignee: Apache Spark spark-shell.cmd --jars option does not accept the jar that has space

[jira] [Commented] (SPARK-6239) Spark MLlib fpm#FPGrowth minSupport should use long instead

2015-04-03 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6239?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14394247#comment-14394247 ] Apache Spark commented on SPARK-6239: - User 'kretes' has created a pull request for

[jira] [Commented] (SPARK-6687) In the hadoop 0.23 profile, hadoop pulls in an older version of netty which conflicts with akka's netty

2015-04-03 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6687?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14394265#comment-14394265 ] Sean Owen commented on SPARK-6687: -- Does this cause any problem? I expect a lot of things

[jira] [Commented] (SPARK-6681) JAVA_HOME error with upgrade to Spark 1.3.0

2015-04-03 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6681?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14394290#comment-14394290 ] Sean Owen commented on SPARK-6681: -- That literal doesn't occur in Spark. That looks like

[jira] [Updated] (SPARK-6691) Abstract and add a dynamic RateLimiter for Spark Streaming

2015-04-03 Thread Saisai Shao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6691?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao updated SPARK-6691: --- Issue Type: Improvement (was: New Feature) Abstract and add a dynamic RateLimiter for Spark

[jira] [Commented] (SPARK-6692) Make it possible to kill AM in YARN cluster mode when the client is terminated

2015-04-03 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6692?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14394135#comment-14394135 ] Apache Spark commented on SPARK-6692: - User 'piaozhexiu' has created a pull request

[jira] [Assigned] (SPARK-6692) Make it possible to kill AM in YARN cluster mode when the client is terminated

2015-04-03 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6692?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-6692: --- Assignee: (was: Apache Spark) Make it possible to kill AM in YARN cluster mode when the

[jira] [Commented] (SPARK-2489) Unsupported parquet datatype optional fixed_len_byte_array

2015-04-03 Thread Ishaaq Chandy (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2489?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14394326#comment-14394326 ] Ishaaq Chandy commented on SPARK-2489: -- I see [~joesu]'s pull request got closed

[jira] [Updated] (SPARK-6689) MiniYarnCLuster still test failed with hadoop-2.2

2015-04-03 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6689?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-6689: - Priority: Minor (was: Major) I imagine this is a problem because you are building with SBT, and it can't

[jira] [Created] (SPARK-6697) PeriodicGraphCheckpointer is not clear Edges.

2015-04-03 Thread Guoqiang Li (JIRA)
Guoqiang Li created SPARK-6697: -- Summary: PeriodicGraphCheckpointer is not clear Edges. Key: SPARK-6697 URL: https://issues.apache.org/jira/browse/SPARK-6697 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-6664) Split Ordered RDD into multiple RDDs by keys (boundaries or intervals)

2015-04-03 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6664?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14394231#comment-14394231 ] Sean Owen commented on SPARK-6664: -- Yes _k_ estimates is better than 1; this is both more

[jira] [Created] (SPARK-6695) Add an external iterator: a hadoop-like output collector

2015-04-03 Thread uncleGen (JIRA)
uncleGen created SPARK-6695: --- Summary: Add an external iterator: a hadoop-like output collector Key: SPARK-6695 URL: https://issues.apache.org/jira/browse/SPARK-6695 Project: Spark Issue Type: New

[jira] [Updated] (SPARK-6695) Add an external iterator: a hadoop-like output collector

2015-04-03 Thread uncleGen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6695?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] uncleGen updated SPARK-6695: Description: In practical use, we usually need to create a big iterator, which means too big in `memory

[jira] [Commented] (SPARK-6695) Add an external iterator: a hadoop-like output collector

2015-04-03 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6695?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14394236#comment-14394236 ] Sean Owen commented on SPARK-6695: -- I am not sure what the use case is here. You already

[jira] [Commented] (SPARK-6694) SparkSQL CLI must be able to specify an option --database on the command line.

2015-04-03 Thread Jin Adachi (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6694?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14394251#comment-14394251 ] Jin Adachi commented on SPARK-6694: --- SparkSQL CLI doesn't work option --database, and

[jira] [Commented] (SPARK-6665) Randomly Shuffle an RDD

2015-04-03 Thread Florian Verhein (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6665?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14394291#comment-14394291 ] Florian Verhein commented on SPARK-6665: Fair enough. I'll have to implement it

[jira] [Commented] (SPARK-6638) optimize StringType in SQL

2015-04-03 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6638?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14394614#comment-14394614 ] Apache Spark commented on SPARK-6638: - User 'davies' has created a pull request for

[jira] [Commented] (SPARK-6330) newParquetRelation gets incorrect FileSystem

2015-04-03 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6330?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14394839#comment-14394839 ] Apache Spark commented on SPARK-6330: - User 'yhuai' has created a pull request for

[jira] [Commented] (SPARK-6697) PeriodicGraphCheckpointer is not clear edges.

2015-04-03 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6697?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14394785#comment-14394785 ] Joseph K. Bradley commented on SPARK-6697: -- Thanks for pointing this out. I

[jira] [Updated] (SPARK-6615) Add missing methods to Word2Vec's Python API

2015-04-03 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6615?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-6615: - Assignee: Kai Sasaki Add missing methods to Word2Vec's Python API

[jira] [Resolved] (SPARK-6615) Python API for Word2Vec

2015-04-03 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6615?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng resolved SPARK-6615. -- Resolution: Fixed Issue resolved by pull request 5296

[jira] [Updated] (SPARK-6615) Add missing methods to Word2Vec's Python API

2015-04-03 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6615?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-6615: - Summary: Add missing methods to Word2Vec's Python API (was: Python API for Word2Vec) Add

[jira] [Updated] (SPARK-6698) RandomForest.scala (et al) hardcodes usage of StorageLevel.MEMORY_AND_DISK

2015-04-03 Thread Michael Bieniosek (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6698?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Bieniosek updated SPARK-6698: - Attachment: SPARK-6698.patch Attaching proposed patch to copy StorageLevel from input RDD

[jira] [Updated] (SPARK-6698) RandomForest.scala (et al) hardcodes usage of StorageLevel.MEMORY_AND_DISK

2015-04-03 Thread Michael Bieniosek (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6698?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Bieniosek updated SPARK-6698: - Attachment: (was: SPARK-6698.patch) RandomForest.scala (et al) hardcodes usage of

[jira] [Updated] (SPARK-6682) Deprecate static train and use builder instead for Scala/Java

2015-04-03 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6682?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-6682: - Description: In MLlib, we have for some time been unofficially moving away from the old

[jira] [Resolved] (SPARK-6330) newParquetRelation gets incorrect FileSystem

2015-04-03 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6330?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai resolved SPARK-6330. - Resolution: Fixed newParquetRelation gets incorrect FileSystem

[jira] [Commented] (SPARK-6330) newParquetRelation gets incorrect FileSystem

2015-04-03 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6330?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14394864#comment-14394864 ] Yin Huai commented on SPARK-6330: - Please ignore my comment. newParquetRelation gets

[jira] [Commented] (SPARK-6682) Deprecate static train and use builder instead for Scala/Java

2015-04-03 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6682?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14394776#comment-14394776 ] Joseph K. Bradley commented on SPARK-6682: -- Note: We could keep 2 APIs for

[jira] [Commented] (SPARK-6682) Deprecate static train and use builder instead for Scala/Java

2015-04-03 Thread Yu Ishikawa (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6682?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14394843#comment-14394843 ] Yu Ishikawa commented on SPARK-6682: Hi [~josephkb], Thank you for your proposal. That

[jira] [Reopened] (SPARK-6330) newParquetRelation gets incorrect FileSystem

2015-04-03 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6330?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai reopened SPARK-6330: - I am reopening the issue since for s3n, {{fs.makeQualified(qualifiedPath)}} does not. It will throw a very

[jira] [Resolved] (SPARK-6492) SparkContext.stop() can deadlock when DAGSchedulerEventProcessLoop dies

2015-04-03 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6492?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-6492. -- Resolution: Fixed Fix Version/s: 1.4.0 Issue resolved by pull request 5277

[jira] [Updated] (SPARK-6492) SparkContext.stop() can deadlock when DAGSchedulerEventProcessLoop dies

2015-04-03 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6492?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-6492: - Assignee: Ilya Ganelin SparkContext.stop() can deadlock when DAGSchedulerEventProcessLoop dies

[jira] [Assigned] (SPARK-4205) Timestamp and Date objects with comparison operators

2015-04-03 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4205?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-4205: --- Assignee: Apache Spark Timestamp and Date objects with comparison operators

[jira] [Assigned] (SPARK-4205) Timestamp and Date objects with comparison operators

2015-04-03 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4205?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-4205: --- Assignee: (was: Apache Spark) Timestamp and Date objects with comparison operators

[jira] [Created] (SPARK-6698) RandomForest.scala (et al) hardcodes usage of StorageLevel.MEMORY_AND_DISK

2015-04-03 Thread Michael Bieniosek (JIRA)
Michael Bieniosek created SPARK-6698: Summary: RandomForest.scala (et al) hardcodes usage of StorageLevel.MEMORY_AND_DISK Key: SPARK-6698 URL: https://issues.apache.org/jira/browse/SPARK-6698

[jira] [Resolved] (SPARK-5203) union with different decimal type report error

2015-04-03 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5203?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian resolved SPARK-5203. --- Resolution: Fixed Fix Version/s: 1.4.0 Issue resolved by pull request 4004

[jira] [Updated] (SPARK-6330) newParquetRelation gets incorrect FileSystem

2015-04-03 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6330?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai updated SPARK-6330: Priority: Blocker (was: Major) newParquetRelation gets incorrect FileSystem

[jira] [Commented] (SPARK-4258) NPE with new Parquet Filters

2015-04-03 Thread Yash Datta (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4258?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14394670#comment-14394670 ] Yash Datta commented on SPARK-4258: --- [~yhuai] No it does not. I fixed this in parquet

[jira] [Closed] (SPARK-6640) Executor may connect to HeartbeartReceiver before it's setup in the driver side

2015-04-03 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6640?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or closed SPARK-6640. Resolution: Fixed Fix Version/s: 1.4.0 Target Version/s: 1.4.0 Executor may connect to

[jira] [Updated] (SPARK-6698) RandomForest.scala (et al) hardcodes usage of StorageLevel.MEMORY_AND_DISK

2015-04-03 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6698?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-6698: - Priority: Minor (was: Major) Issue Type: Improvement (was: Bug) (Open a PR; changes aren't

[jira] [Assigned] (SPARK-6698) RandomForest.scala (et al) hardcodes usage of StorageLevel.MEMORY_AND_DISK

2015-04-03 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6698?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-6698: --- Assignee: (was: Apache Spark) RandomForest.scala (et al) hardcodes usage of

[jira] [Assigned] (SPARK-6698) RandomForest.scala (et al) hardcodes usage of StorageLevel.MEMORY_AND_DISK

2015-04-03 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6698?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-6698: --- Assignee: Apache Spark RandomForest.scala (et al) hardcodes usage of

[jira] [Commented] (SPARK-6698) RandomForest.scala (et al) hardcodes usage of StorageLevel.MEMORY_AND_DISK

2015-04-03 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14394681#comment-14394681 ] Apache Spark commented on SPARK-6698: - User 'bien' has created a pull request for this

[jira] [Closed] (SPARK-6688) EventLoggingListener should always operate on resolved URIs

2015-04-03 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6688?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or closed SPARK-6688. Resolution: Fixed Fix Version/s: 1.4.0 1.3.1 Assignee: Marcelo Vanzin

[jira] [Resolved] (SPARK-6647) Make trait StringComparison as BinaryPredicate and throw error when Predicate can't translate to data source Filter

2015-04-03 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6647?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-6647. - Resolution: Fixed Fix Version/s: 1.4.0 Issue resolved by pull request 5309

[jira] [Updated] (SPARK-6683) Handling feature scaling properly for GLMs

2015-04-03 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6683?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-6683: - Description: GeneralizedLinearAlgorithm can scale features. This has 2 effects: *

[jira] [Commented] (SPARK-6683) Handling feature scaling properly for GLMs

2015-04-03 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6683?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14395122#comment-14395122 ] Joseph K. Bradley commented on SPARK-6683: -- Great, it sounds like we're in

[jira] [Assigned] (SPARK-6700) flaky test: run Python application in yarn-cluster mode

2015-04-03 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6700?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-6700: --- Assignee: Lianhui Wang (was: Apache Spark) flaky test: run Python application in

[jira] [Commented] (SPARK-6700) flaky test: run Python application in yarn-cluster mode

2015-04-03 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6700?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14395127#comment-14395127 ] Apache Spark commented on SPARK-6700: - User 'davies' has created a pull request for

[jira] [Assigned] (SPARK-6700) flaky test: run Python application in yarn-cluster mode

2015-04-03 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6700?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-6700: --- Assignee: Apache Spark (was: Lianhui Wang) flaky test: run Python application in

[jira] [Updated] (SPARK-6700) flaky test: run Python application in yarn-cluster mode

2015-04-03 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6700?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-6700: -- Labels: test yarn (was: ) flaky test: run Python application in yarn-cluster mode

[jira] [Commented] (SPARK-6682) Deprecate static train and use builder instead for Scala/Java

2015-04-03 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6682?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14394998#comment-14394998 ] Joseph K. Bradley commented on SPARK-6682: -- I don't know of an automatic

[jira] [Assigned] (SPARK-6577) SparseMatrix should be supported in PySpark

2015-04-03 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6577?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-6577: --- Assignee: Apache Spark SparseMatrix should be supported in PySpark

[jira] [Assigned] (SPARK-6577) SparseMatrix should be supported in PySpark

2015-04-03 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6577?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-6577: --- Assignee: (was: Apache Spark) SparseMatrix should be supported in PySpark

[jira] [Commented] (SPARK-6577) SparseMatrix should be supported in PySpark

2015-04-03 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6577?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14395099#comment-14395099 ] Apache Spark commented on SPARK-6577: - User 'MechCoder' has created a pull request for

[jira] [Closed] (SPARK-6700) flaky test: run Python application in yarn-cluster mode

2015-04-03 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6700?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or closed SPARK-6700. Resolution: Fixed flaky test: run Python application in yarn-cluster mode

[jira] [Commented] (SPARK-6683) Handling feature scaling properly for GLMs

2015-04-03 Thread DB Tsai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6683?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14395088#comment-14395088 ] DB Tsai commented on SPARK-6683: I have this implemented in our lab including handling the

[jira] [Created] (SPARK-6703) Provide a way to discover existing SparkContext's

2015-04-03 Thread Patrick Wendell (JIRA)
Patrick Wendell created SPARK-6703: -- Summary: Provide a way to discover existing SparkContext's Key: SPARK-6703 URL: https://issues.apache.org/jira/browse/SPARK-6703 Project: Spark Issue

[jira] [Updated] (SPARK-5992) Locality Sensitive Hashing (LSH) for MLlib

2015-04-03 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5992?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-5992: - Shepherd: Xiangrui Meng Locality Sensitive Hashing (LSH) for MLlib

[jira] [Created] (SPARK-6701) Flaky test: o.a.s.deploy.yarn.YarnClusterSuite Python application

2015-04-03 Thread Andrew Or (JIRA)
Andrew Or created SPARK-6701: Summary: Flaky test: o.a.s.deploy.yarn.YarnClusterSuite Python application Key: SPARK-6701 URL: https://issues.apache.org/jira/browse/SPARK-6701 Project: Spark

[jira] [Commented] (SPARK-6683) Handling feature scaling properly for GLMs

2015-04-03 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6683?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14395145#comment-14395145 ] Joseph K. Bradley commented on SPARK-6683: -- If you're referring to what I was

[jira] [Commented] (SPARK-6673) spark-shell.cmd can't start even when spark was built in Windows

2015-04-03 Thread Alexander Ulanov (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14395001#comment-14395001 ] Alexander Ulanov commented on SPARK-6673: - Probably similar issue: I am trying to

  1   2   >