[jira] [Commented] (SPARK-15678) Not use cache on appends and overwrites

2017-02-21 Thread Fei Shao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15678?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15877618#comment-15877618 ] Fei Shao commented on SPARK-15678: -- I checked code with refreshByPath, it worked well. But I do not

[jira] [Commented] (SPARK-19688) Spark on Yarn Credentials File set to different application directory

2017-02-21 Thread Saisai Shao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15877587#comment-15877587 ] Saisai Shao commented on SPARK-19688: - Can you please elaborate the problem you met, otherwise it is

[jira] [Commented] (SPARK-19617) Fix the race condition when starting and stopping a query quickly

2017-02-21 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19617?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15877578#comment-15877578 ] Apache Spark commented on SPARK-19617: -- User 'gf53520' has created a pull request for this issue:

[jira] [Commented] (SPARK-19639) Add spark.svmLinear example and update vignettes

2017-02-21 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19639?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15877576#comment-15877576 ] Felix Cheung commented on SPARK-19639: -- right, I left it out because of the programming guide which

[jira] [Commented] (SPARK-15678) Not use cache on appends and overwrites

2017-02-21 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15678?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15877575#comment-15877575 ] Kazuaki Ishizaki commented on SPARK-15678: -- How about insert {{spark.catalog.refreshByPath()}}

[jira] [Resolved] (SPARK-19639) Add spark.svmLinear example and update vignettes

2017-02-21 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19639?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung resolved SPARK-19639. -- Resolution: Fixed Assignee: Miao Wang Fix Version/s: 2.2.0 Target

[jira] [Comment Edited] (SPARK-15678) Not use cache on appends and overwrites

2017-02-21 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15678?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15877575#comment-15877575 ] Kazuaki Ishizaki edited comment on SPARK-15678 at 2/22/17 6:37 AM: --- How

[jira] [Closed] (SPARK-19642) Improve the security guarantee for rest api and ui

2017-02-21 Thread Genmao Yu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19642?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Genmao Yu closed SPARK-19642. - Resolution: Won't Fix > Improve the security guarantee for rest api and ui >

[jira] [Comment Edited] (SPARK-15678) Not use cache on appends and overwrites

2017-02-21 Thread Gen TANG (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15678?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15877513#comment-15877513 ] Gen TANG edited comment on SPARK-15678 at 2/22/17 5:41 AM: --- hi, I found a bug

[jira] [Commented] (SPARK-15678) Not use cache on appends and overwrites

2017-02-21 Thread Gen TANG (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15678?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15877513#comment-15877513 ] Gen TANG commented on SPARK-15678: -- hi, I found a bug which is probably related with this

[jira] [Commented] (SPARK-19525) Enable Compression of RDD Checkpoints

2017-02-21 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19525?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15877501#comment-15877501 ] Apache Spark commented on SPARK-19525: -- User 'aramesh117' has created a pull request for this issue:

[jira] [Assigned] (SPARK-19525) Enable Compression of RDD Checkpoints

2017-02-21 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19525?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19525: Assignee: Apache Spark > Enable Compression of RDD Checkpoints >

[jira] [Assigned] (SPARK-19525) Enable Compression of RDD Checkpoints

2017-02-21 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19525?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19525: Assignee: (was: Apache Spark) > Enable Compression of RDD Checkpoints >

[jira] [Created] (SPARK-19696) Wrong Documentation for Java Word Count Example

2017-02-21 Thread gaurav gupta (JIRA)
gaurav gupta created SPARK-19696: Summary: Wrong Documentation for Java Word Count Example Key: SPARK-19696 URL: https://issues.apache.org/jira/browse/SPARK-19696 Project: Spark Issue Type:

[jira] [Updated] (SPARK-19695) Throw an exception if a `columnNameOfCorruptRecord` field violates requirements in Json formats

2017-02-21 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19695?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takeshi Yamamuro updated SPARK-19695: - Summary: Throw an exception if a `columnNameOfCorruptRecord` field violates requirements

[jira] [Comment Edited] (SPARK-19661) Spark-2.1.0 can not connect hbase

2017-02-21 Thread sydt (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19661?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15877441#comment-15877441 ] sydt edited comment on SPARK-19661 at 2/22/17 4:18 AM: --- Yeah,you are right.It is

[jira] [Resolved] (SPARK-19617) Fix the race condition when starting and stopping a query quickly

2017-02-21 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19617?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu resolved SPARK-19617. -- Resolution: Fixed Fix Version/s: 2.1.1 > Fix the race condition when starting and

[jira] [Commented] (SPARK-19661) Spark-2.1.0 can not connect hbase

2017-02-21 Thread sydt (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19661?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15877441#comment-15877441 ] sydt commented on SPARK-19661: -- Yeah,you are right.It is accomplished by hive-hbase-handler-0.13.1.jar

[jira] [Assigned] (SPARK-19695) Throw an exception if a `columnNameOfCorruptRecord` field violates requirements

2017-02-21 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19695?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19695: Assignee: (was: Apache Spark) > Throw an exception if a `columnNameOfCorruptRecord`

[jira] [Assigned] (SPARK-19695) Throw an exception if a `columnNameOfCorruptRecord` field violates requirements

2017-02-21 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19695?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19695: Assignee: Apache Spark > Throw an exception if a `columnNameOfCorruptRecord` field

[jira] [Commented] (SPARK-19695) Throw an exception if a `columnNameOfCorruptRecord` field violates requirements

2017-02-21 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19695?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15877440#comment-15877440 ] Apache Spark commented on SPARK-19695: -- User 'maropu' has created a pull request for this issue:

[jira] [Created] (SPARK-19695) Throw an exception if a `columnNameOfCorruptRecord` field violates requirements

2017-02-21 Thread Takeshi Yamamuro (JIRA)
Takeshi Yamamuro created SPARK-19695: Summary: Throw an exception if a `columnNameOfCorruptRecord` field violates requirements Key: SPARK-19695 URL: https://issues.apache.org/jira/browse/SPARK-19695

[jira] [Assigned] (SPARK-19694) Add missing 'setTopicDistributionCol' for LDAModel

2017-02-21 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19694?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19694: Assignee: Apache Spark > Add missing 'setTopicDistributionCol' for LDAModel >

[jira] [Commented] (SPARK-19694) Add missing 'setTopicDistributionCol' for LDAModel

2017-02-21 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19694?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15877417#comment-15877417 ] Apache Spark commented on SPARK-19694: -- User 'zhengruifeng' has created a pull request for this

[jira] [Assigned] (SPARK-19694) Add missing 'setTopicDistributionCol' for LDAModel

2017-02-21 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19694?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19694: Assignee: (was: Apache Spark) > Add missing 'setTopicDistributionCol' for LDAModel >

[jira] [Created] (SPARK-19694) Add missing 'setTopicDistributionCol' for LDAModel

2017-02-21 Thread zhengruifeng (JIRA)
zhengruifeng created SPARK-19694: Summary: Add missing 'setTopicDistributionCol' for LDAModel Key: SPARK-19694 URL: https://issues.apache.org/jira/browse/SPARK-19694 Project: Spark Issue

[jira] [Resolved] (SPARK-19670) Enable Bucketed Table Reading and Writing Testing Without Hive Support

2017-02-21 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19670?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-19670. - Resolution: Fixed Fix Version/s: 2.2.0 > Enable Bucketed Table Reading and Writing Testing

[jira] [Commented] (SPARK-19693) SET mapreduce.job.reduces automatically converted to spark.sql.shuffle.partitions

2017-02-21 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15877391#comment-15877391 ] Apache Spark commented on SPARK-19693: -- User 'wangyum' has created a pull request for this issue:

[jira] [Issue Comment Deleted] (SPARK-19693) SET mapreduce.job.reduces automatically converted to spark.sql.shuffle.partitions

2017-02-21 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19693?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-19693: Comment: was deleted (was: I'm working on.) > SET mapreduce.job.reduces automatically converted

[jira] [Assigned] (SPARK-19693) SET mapreduce.job.reduces automatically converted to spark.sql.shuffle.partitions

2017-02-21 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19693?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19693: Assignee: (was: Apache Spark) > SET mapreduce.job.reduces automatically converted to

[jira] [Assigned] (SPARK-19693) SET mapreduce.job.reduces automatically converted to spark.sql.shuffle.partitions

2017-02-21 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19693?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19693: Assignee: Apache Spark > SET mapreduce.job.reduces automatically converted to >

[jira] [Commented] (SPARK-19693) SET mapreduce.job.reduces automatically converted to spark.sql.shuffle.partitions

2017-02-21 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15877382#comment-15877382 ] Yuming Wang commented on SPARK-19693: - I'm working on. > SET mapreduce.job.reduces automatically

[jira] [Created] (SPARK-19693) SET mapreduce.job.reduces automatically converted to spark.sql.shuffle.partitions

2017-02-21 Thread Yuming Wang (JIRA)
Yuming Wang created SPARK-19693: --- Summary: SET mapreduce.job.reduces automatically converted to spark.sql.shuffle.partitions Key: SPARK-19693 URL: https://issues.apache.org/jira/browse/SPARK-19693

[jira] [Commented] (SPARK-19675) ExecutorClassLoader loads classes from SystemClassLoader

2017-02-21 Thread Kohki Nishio (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19675?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15877350#comment-15877350 ] Kohki Nishio commented on SPARK-19675: -- Turned out that this is a dup of

[jira] [Updated] (SPARK-19692) Comparison on BinaryType has incorrect results

2017-02-21 Thread Don Smith (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19692?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Don Smith updated SPARK-19692: --- Summary: Comparison on BinaryType has incorrect results (was: Comparison on BinaryType returns no

[jira] [Updated] (SPARK-19692) Comparison on BinaryType returns no results

2017-02-21 Thread Don Smith (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19692?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Don Smith updated SPARK-19692: --- Description: I believe there is an issue with comparisons on binary fields: {code} val sc =

[jira] [Created] (SPARK-19692) Comparison on BinaryType returns no results

2017-02-21 Thread Don Smith (JIRA)
Don Smith created SPARK-19692: -- Summary: Comparison on BinaryType returns no results Key: SPARK-19692 URL: https://issues.apache.org/jira/browse/SPARK-19692 Project: Spark Issue Type: Bug

[jira] [Resolved] (SPARK-19676) Flaky test: FsHistoryProviderSuite.SPARK-3697: ignore directories that cannot be read.

2017-02-21 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19676?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-19676. Resolution: Invalid Reporter said he was running tests as root. > Flaky test:

[jira] [Comment Edited] (SPARK-19675) ExecutorClassLoader loads classes from SystemClassLoader

2017-02-21 Thread Kohki Nishio (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19675?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15877290#comment-15877290 ] Kohki Nishio edited comment on SPARK-19675 at 2/22/17 2:11 AM: --- not in a

[jira] [Commented] (SPARK-19675) ExecutorClassLoader loads classes from SystemClassLoader

2017-02-21 Thread Kohki Nishio (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19675?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15877290#comment-15877290 ] Kohki Nishio commented on SPARK-19675: -- not in a separate process, I'm running it in local mode, so

[jira] [Comment Edited] (SPARK-19675) ExecutorClassLoader loads classes from SystemClassLoader

2017-02-21 Thread Kohki Nishio (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19675?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15877273#comment-15877273 ] Kohki Nishio edited comment on SPARK-19675 at 2/22/17 2:00 AM: --- [~zsxwing]

[jira] [Commented] (SPARK-19675) ExecutorClassLoader loads classes from SystemClassLoader

2017-02-21 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19675?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15877276#comment-15877276 ] Shixiong Zhu commented on SPARK-19675: -- Just to clarify one thing: Executors in your case will run

[jira] [Commented] (SPARK-19675) ExecutorClassLoader loads classes from SystemClassLoader

2017-02-21 Thread Kohki Nishio (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19675?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15877273#comment-15877273 ] Kohki Nishio commented on SPARK-19675: -- [~zsxwing] That's only when findClass is used, it doesn't do

[jira] [Comment Edited] (SPARK-19675) ExecutorClassLoader loads classes from SystemClassLoader

2017-02-21 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19675?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15877262#comment-15877262 ] Shixiong Zhu edited comment on SPARK-19675 at 2/22/17 1:59 AM: --- [~taroplus]

[jira] [Comment Edited] (SPARK-19675) ExecutorClassLoader loads classes from SystemClassLoader

2017-02-21 Thread Kohki Nishio (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19675?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15877256#comment-15877256 ] Kohki Nishio edited comment on SPARK-19675 at 2/22/17 1:58 AM: --- My

[jira] [Commented] (SPARK-19675) ExecutorClassLoader loads classes from SystemClassLoader

2017-02-21 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19675?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15877262#comment-15877262 ] Shixiong Zhu commented on SPARK-19675: -- [~taroplus] ExecutorClassLoader does try to load from its

[jira] [Comment Edited] (SPARK-19675) ExecutorClassLoader loads classes from SystemClassLoader

2017-02-21 Thread Kohki Nishio (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19675?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15877256#comment-15877256 ] Kohki Nishio edited comment on SPARK-19675 at 2/22/17 1:54 AM: --- My

[jira] [Comment Edited] (SPARK-19675) ExecutorClassLoader loads classes from SystemClassLoader

2017-02-21 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19675?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15877255#comment-15877255 ] Shixiong Zhu edited comment on SPARK-19675 at 2/22/17 1:53 AM: --- [~taroplus]

[jira] [Commented] (SPARK-19675) ExecutorClassLoader loads classes from SystemClassLoader

2017-02-21 Thread Kohki Nishio (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19675?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15877256#comment-15877256 ] Kohki Nishio commented on SPARK-19675: -- My application does notebook thing, this is about Zeppelin

[jira] [Comment Edited] (SPARK-19675) ExecutorClassLoader loads classes from SystemClassLoader

2017-02-21 Thread Kohki Nishio (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19675?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15877256#comment-15877256 ] Kohki Nishio edited comment on SPARK-19675 at 2/22/17 1:54 AM: --- My

[jira] [Commented] (SPARK-19675) ExecutorClassLoader loads classes from SystemClassLoader

2017-02-21 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19675?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15877255#comment-15877255 ] Shixiong Zhu commented on SPARK-19675: -- [~taroplus] Yeah, I should have checked `sbt run` with

[jira] [Updated] (SPARK-19675) ExecutorClassLoader loads classes from SystemClassLoader

2017-02-21 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19675?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-19675: - Issue Type: Improvement (was: Bug) > ExecutorClassLoader loads classes from SystemClassLoader >

[jira] [Commented] (SPARK-13767) py4j.protocol.Py4JNetworkError: An error occurred while trying to connect to the Java server

2017-02-21 Thread yb (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13767?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15877251#comment-15877251 ] yb commented on SPARK-13767: i use spark 1.6.2 python 2.7 I'm seeing the error that Venkata showed as well,

[jira] [Commented] (SPARK-19675) ExecutorClassLoader loads classes from SystemClassLoader

2017-02-21 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19675?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15877239#comment-15877239 ] Sean Owen commented on SPARK-19675: --- No, this isn't even a supported way to run an executor. None of

[jira] [Commented] (SPARK-19675) ExecutorClassLoader loads classes from SystemClassLoader

2017-02-21 Thread Kohki Nishio (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19675?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15877235#comment-15877235 ] Kohki Nishio commented on SPARK-19675: -- I don't think this is something clearly a wrong bug report,

[jira] [Commented] (SPARK-19675) ExecutorClassLoader loads classes from SystemClassLoader

2017-02-21 Thread Kohki Nishio (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19675?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15877234#comment-15877234 ] Kohki Nishio commented on SPARK-19675: -- Overriding loadClass fixes my scenario {code} import

[jira] [Created] (SPARK-19691) Calculating percentile of decimal column fails with ClassCastException

2017-02-21 Thread Josh Rosen (JIRA)
Josh Rosen created SPARK-19691: -- Summary: Calculating percentile of decimal column fails with ClassCastException Key: SPARK-19691 URL: https://issues.apache.org/jira/browse/SPARK-19691 Project: Spark

[jira] [Commented] (SPARK-19675) ExecutorClassLoader loads classes from SystemClassLoader

2017-02-21 Thread Kohki Nishio (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19675?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15877219#comment-15877219 ] Kohki Nishio commented on SPARK-19675: -- First of all, sbt (0.13) is built with Scala2.10 and it can

[jira] [Created] (SPARK-19690) Join a streaming DataFrame with a batch DataFrame may not work

2017-02-21 Thread Shixiong Zhu (JIRA)
Shixiong Zhu created SPARK-19690: Summary: Join a streaming DataFrame with a batch DataFrame may not work Key: SPARK-19690 URL: https://issues.apache.org/jira/browse/SPARK-19690 Project: Spark

[jira] [Created] (SPARK-19689) Job Details page doesn't show 'Tasks: Succeeded/Total' progress bar text properly

2017-02-21 Thread Devaraj K (JIRA)
Devaraj K created SPARK-19689: - Summary: Job Details page doesn't show 'Tasks: Succeeded/Total' progress bar text properly Key: SPARK-19689 URL: https://issues.apache.org/jira/browse/SPARK-19689 Project:

[jira] [Updated] (SPARK-19689) Job Details page doesn't show 'Tasks: Succeeded/Total' progress bar text properly

2017-02-21 Thread Devaraj K (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19689?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Devaraj K updated SPARK-19689: -- Attachment: Tasks Progress bar - Job Details Page.png > Job Details page doesn't show 'Tasks:

[jira] [Commented] (SPARK-19636) Feature parity for correlation statistics in MLlib

2017-02-21 Thread Timothy Hunter (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19636?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15877128#comment-15877128 ] Timothy Hunter commented on SPARK-19636: Looking more closely at the code, it makes sense to

[jira] [Commented] (SPARK-19652) REST API does not perform user auth for individual apps

2017-02-21 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19652?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15877073#comment-15877073 ] Apache Spark commented on SPARK-19652: -- User 'vanzin' has created a pull request for this issue:

[jira] [Commented] (SPARK-19681) save and load pipeline and then use it yield java.lang.RuntimeException

2017-02-21 Thread JIRA
[ https://issues.apache.org/jira/browse/SPARK-19681?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15877033#comment-15877033 ] Boris Clémençon commented on SPARK-19681: -- EDIT: the problem is in

[jira] [Closed] (SPARK-19681) save and load pipeline and then use it yield java.lang.RuntimeException

2017-02-21 Thread JIRA
[ https://issues.apache.org/jira/browse/SPARK-19681?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Boris Clémençon closed SPARK-19681. Resolution: Fixed > save and load pipeline and then use it yield

[jira] [Commented] (SPARK-19635) Feature parity for Chi-square hypothesis testing in MLlib

2017-02-21 Thread Miao Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19635?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15876979#comment-15876979 ] Miao Wang commented on SPARK-19635: --- https://github.com/apache/spark/pull/13440 [~timhunter] This one

[jira] [Commented] (SPARK-19635) Feature parity for Chi-square hypothesis testing in MLlib

2017-02-21 Thread Miao Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19635?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15876977#comment-15876977 ] Miao Wang commented on SPARK-19635: --- I think there is a related PR opened for quite a while. Let me

[jira] [Resolved] (SPARK-19687) Does SPARK supports for Postgres JSONB data type to store JSON data, if yes, kindly please help us with any examples.

2017-02-21 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19687?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-19687. --- Resolution: Invalid Questions should go to u...@spark.apache.org > Does SPARK supports for Postgres

[jira] [Created] (SPARK-19688) Spark on Yarn Credentials File set to different application directory

2017-02-21 Thread Devaraj Jonnadula (JIRA)
Devaraj Jonnadula created SPARK-19688: - Summary: Spark on Yarn Credentials File set to different application directory Key: SPARK-19688 URL: https://issues.apache.org/jira/browse/SPARK-19688

[jira] [Updated] (SPARK-19687) Does SPARK supports for Postgres JSONB data type to store JSON data, if yes, kindly please help us with any examples.

2017-02-21 Thread Praveen Tallapudi (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19687?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Praveen Tallapudi updated SPARK-19687: -- Summary: Does SPARK supports for Postgres JSONB data type to store JSON data, if yes,

[jira] [Updated] (SPARK-19687) Does SPARK supports for Postgres JSONB to store JSON data, if yes, kindly please help us with any examples.

2017-02-21 Thread Praveen Tallapudi (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19687?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Praveen Tallapudi updated SPARK-19687: -- Summary: Does SPARK supports for Postgres JSONB to store JSON data, if yes, kindly

[jira] [Created] (SPARK-19687) SPARK supports for Postgres JSONB to store JSON data, if yes, kindly please help us with any examples.

2017-02-21 Thread Praveen Tallapudi (JIRA)
Praveen Tallapudi created SPARK-19687: - Summary: SPARK supports for Postgres JSONB to store JSON data, if yes, kindly please help us with any examples. Key: SPARK-19687 URL:

[jira] [Commented] (SPARK-19639) Add spark.svmLinear example and update vignettes

2017-02-21 Thread Miao Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19639?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15876936#comment-15876936 ] Miao Wang commented on SPARK-19639: --- This JIRA should be closed as the PR is merged. Thanks! cc

[jira] [Commented] (SPARK-19636) Feature parity for correlation statistics in MLlib

2017-02-21 Thread Timothy Hunter (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19636?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15876929#comment-15876929 ] Timothy Hunter commented on SPARK-19636: Unless someone has started to work on this task, I will

[jira] [Resolved] (SPARK-19686) Spark poc projects

2017-02-21 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19686?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-19686. --- Resolution: Invalid > Spark poc projects > -- > > Key: SPARK-19686 >

[jira] [Created] (SPARK-19686) Spark poc projects

2017-02-21 Thread Srinivasa Kalyanachakravarthy Vasagiri (JIRA)
Srinivasa Kalyanachakravarthy Vasagiri created SPARK-19686: -- Summary: Spark poc projects Key: SPARK-19686 URL: https://issues.apache.org/jira/browse/SPARK-19686 Project: Spark

[jira] [Comment Edited] (SPARK-19675) ExecutorClassLoader loads classes from SystemClassLoader

2017-02-21 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19675?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15876854#comment-15876854 ] Shixiong Zhu edited comment on SPARK-19675 at 2/21/17 10:17 PM:

[jira] [Commented] (SPARK-19675) ExecutorClassLoader loads classes from SystemClassLoader

2017-02-21 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19675?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15876854#comment-15876854 ] Shixiong Zhu commented on SPARK-19675: -- [~taroplus] If I understand correctly, SBT launches your

[jira] [Commented] (SPARK-19684) Move info about running specific tests to developer website

2017-02-21 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19684?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15876846#comment-15876846 ] Apache Spark commented on SPARK-19684: -- User 'kayousterhout' has created a pull request for this

[jira] [Assigned] (SPARK-19684) Move info about running specific tests to developer website

2017-02-21 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19684?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19684: Assignee: Apache Spark (was: Kay Ousterhout) > Move info about running specific tests to

[jira] [Assigned] (SPARK-19684) Move info about running specific tests to developer website

2017-02-21 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19684?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19684: Assignee: Kay Ousterhout (was: Apache Spark) > Move info about running specific tests to

[jira] [Commented] (SPARK-19685) PipedRDD tasks should not hang on interruption / errors

2017-02-21 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19685?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15876828#comment-15876828 ] Josh Rosen commented on SPARK-19685: By the way, one simple fix here might be to use the "soft-kill

[jira] [Created] (SPARK-19685) PipedRDD tasks should not hang on interruption / errors

2017-02-21 Thread Josh Rosen (JIRA)
Josh Rosen created SPARK-19685: -- Summary: PipedRDD tasks should not hang on interruption / errors Key: SPARK-19685 URL: https://issues.apache.org/jira/browse/SPARK-19685 Project: Spark Issue

[jira] [Commented] (SPARK-19683) Support for libsvm-based learning-to-rank format

2017-02-21 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19683?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15876812#comment-15876812 ] Sean Owen commented on SPARK-19683: --- This is trivial for an application to implement. Unless it is a

[jira] [Comment Edited] (SPARK-18454) Changes to improve Nearest Neighbor Search for LSH

2017-02-21 Thread Yun Ni (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18454?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15874956#comment-15874956 ] Yun Ni edited comment on SPARK-18454 at 2/21/17 9:39 PM: - [~josephkb] [~sethah]

[jira] [Issue Comment Deleted] (SPARK-19617) Fix the race condition when starting and stopping a query quickly

2017-02-21 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19617?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-19617: - Comment: was deleted (was: User 'gf53520' has created a pull request for this issue:

[jira] [Commented] (SPARK-19677) HDFSBackedStateStoreProvider fails to overwrite existing file

2017-02-21 Thread Heji Kim (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19677?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15876747#comment-15876747 ] Heji Kim commented on SPARK-19677: -- Thank you for reporting this issue! I just wanted to add that we

[jira] [Updated] (SPARK-19680) Offsets out of range with no configured reset policy for partitions

2017-02-21 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19680?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-19680: - Component/s: (was: Structured Streaming) DStreams > Offsets out of range

[jira] [Created] (SPARK-19684) Move info about running specific tests to developer website

2017-02-21 Thread Kay Ousterhout (JIRA)
Kay Ousterhout created SPARK-19684: -- Summary: Move info about running specific tests to developer website Key: SPARK-19684 URL: https://issues.apache.org/jira/browse/SPARK-19684 Project: Spark

[jira] [Created] (SPARK-19683) Support for libsvm-based learning-to-rank format

2017-02-21 Thread Craig Macdonald (JIRA)
Craig Macdonald created SPARK-19683: --- Summary: Support for libsvm-based learning-to-rank format Key: SPARK-19683 URL: https://issues.apache.org/jira/browse/SPARK-19683 Project: Spark Issue

[jira] [Updated] (SPARK-19497) dropDuplicates with watermark

2017-02-21 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19497?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-19497: - Labels: release_notes (was: ) > dropDuplicates with watermark > - >

[jira] [Assigned] (SPARK-19682) Issue warning (or error) when subset method "[[" takes vector index

2017-02-21 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19682?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19682: Assignee: (was: Apache Spark) > Issue warning (or error) when subset method "[["

[jira] [Assigned] (SPARK-19682) Issue warning (or error) when subset method "[[" takes vector index

2017-02-21 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19682?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19682: Assignee: Apache Spark > Issue warning (or error) when subset method "[[" takes vector

[jira] [Commented] (SPARK-19682) Issue warning (or error) when subset method "[[" takes vector index

2017-02-21 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19682?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15876497#comment-15876497 ] Apache Spark commented on SPARK-19682: -- User 'actuaryzhang' has created a pull request for this

[jira] [Created] (SPARK-19682) Issue warning (or error) when subset method "[[" takes vector index

2017-02-21 Thread Wayne Zhang (JIRA)
Wayne Zhang created SPARK-19682: --- Summary: Issue warning (or error) when subset method "[[" takes vector index Key: SPARK-19682 URL: https://issues.apache.org/jira/browse/SPARK-19682 Project: Spark

[jira] [Commented] (SPARK-19659) Fetch big blocks to disk when shuffle-read

2017-02-21 Thread Imran Rashid (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19659?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15876466#comment-15876466 ] Imran Rashid commented on SPARK-19659: -- [~jinxing6...@126.com] Thanks for taking this on, I think

[jira] [Created] (SPARK-19681) save and load pipeline and then use it yield java.lang.RuntimeException

2017-02-21 Thread JIRA
Boris Clémençon created SPARK-19681: Summary: save and load pipeline and then use it yield java.lang.RuntimeException Key: SPARK-19681 URL: https://issues.apache.org/jira/browse/SPARK-19681

[jira] [Resolved] (SPARK-19626) Configuration `spark.yarn.credentials.updateTime` takes no effect

2017-02-21 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19626?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-19626. Resolution: Fixed Assignee: Kent Yao Fix Version/s: 2.2.0

[jira] [Resolved] (SPARK-19337) Documentation and examples for LinearSVC

2017-02-21 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19337?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung resolved SPARK-19337. -- Resolution: Fixed Assignee: yuhao yang Fix Version/s: 2.2.0 > Documentation

[jira] [Updated] (SPARK-19680) Offsets out of range with no configured reset policy for partitions

2017-02-21 Thread Schakmann Rene (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19680?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Schakmann Rene updated SPARK-19680: --- Description: I'm using spark streaming with kafka to acutally create a toplist. I want to

  1   2   >