[jira] [Commented] (SPARK-16609) Single function for parsing timestamps/dates

2016-08-05 Thread Sandeep Singh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16609?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15410501#comment-15410501 ] Sandeep Singh commented on SPARK-16609: --- I can work on this. > Single function for parsing

[jira] [Commented] (SPARK-16932) Programming-guide Accumulator section should be more clear w.r.t new API

2016-08-05 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16932?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15410488#comment-15410488 ] Apache Spark commented on SPARK-16932: -- User 'BryanCutler' has created a pull request for this

[jira] [Assigned] (SPARK-16932) Programming-guide Accumulator section should be more clear w.r.t new API

2016-08-05 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16932?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16932: Assignee: Apache Spark > Programming-guide Accumulator section should be more clear w.r.t

[jira] [Assigned] (SPARK-16932) Programming-guide Accumulator section should be more clear w.r.t new API

2016-08-05 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16932?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16932: Assignee: (was: Apache Spark) > Programming-guide Accumulator section should be more

[jira] [Created] (SPARK-16932) Programming-guide Accumulator section should be more clear w.r.t new API

2016-08-05 Thread Bryan Cutler (JIRA)
Bryan Cutler created SPARK-16932: Summary: Programming-guide Accumulator section should be more clear w.r.t new API Key: SPARK-16932 URL: https://issues.apache.org/jira/browse/SPARK-16932 Project:

[jira] [Closed] (SPARK-15702) Update document programming-guide accumulator section

2016-08-05 Thread Bryan Cutler (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15702?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bryan Cutler closed SPARK-15702. Resolution: Fixed > Update document programming-guide accumulator section >

[jira] [Commented] (SPARK-16889) Add formatMessage Column expression for formatting strings in java.text.MessageFormat style in Scala API

2016-08-05 Thread Sandeep Singh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16889?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15410484#comment-15410484 ] Sandeep Singh commented on SPARK-16889: --- Why not use something like {code} "Argument '%s' shall

[jira] [Commented] (SPARK-16852) RejectedExecutionException when exit at some times

2016-08-05 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16852?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15410481#comment-15410481 ] Sean Owen commented on SPARK-16852: --- Does it cause any problem? > RejectedExecutionException when exit

[jira] [Commented] (SPARK-16326) Evaluate sparklyr package from RStudio

2016-08-05 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16326?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15410480#comment-15410480 ] Sean Owen commented on SPARK-16326: --- Is there an action for the Spark code here? I'm inclined to close

[jira] [Updated] (SPARK-16717) Dataframe (jdbc) is missing a way to link and external function to get a connection

2016-08-05 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16717?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-16717: -- Priority: Minor (was: Major) > Dataframe (jdbc) is missing a way to link and external function to get

[jira] [Commented] (SPARK-16666) Kryo encoder for custom complex classes

2016-08-05 Thread Sam (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15410454#comment-15410454 ] Sam commented on SPARK-1: - [~clockfly] in your code sample, there is a case class for Point, not esri's

[jira] [Comment Edited] (SPARK-16929) Bad synchronization with regard to speculation

2016-08-05 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16929?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15410441#comment-15410441 ] Sean Owen edited comment on SPARK-16929 at 8/6/16 3:55 AM: --- At least, one easy

[jira] [Commented] (SPARK-16929) Bad synchronization with regard to speculation

2016-08-05 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16929?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15410441#comment-15410441 ] Sean Owen commented on SPARK-16929: --- At least, one easy optimization we could make is to let

[jira] [Updated] (SPARK-15899) file scheme should be used correctly

2016-08-05 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15899?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-15899: -- Assignee: Alexander Ulanov Priority: Major (was: Minor) Issue Type: Bug (was:

[jira] [Updated] (SPARK-16847) Prevent to potentially read corrupt statstics on binary in Parquet via VectorizedReader

2016-08-05 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16847?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-16847: -- Assignee: Hyukjin Kwon > Prevent to potentially read corrupt statstics on binary in Parquet via >

[jira] [Resolved] (SPARK-16847) Prevent to potentially read corrupt statstics on binary in Parquet via VectorizedReader

2016-08-05 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16847?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-16847. --- Resolution: Fixed Fix Version/s: 2.1.0 Issue resolved by pull request 14450

[jira] [Updated] (SPARK-16928) Recursive call of ColumnVector::getInt() breaks JIT inlining

2016-08-05 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16928?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-16928: -- Priority: Minor (was: Major) Description: In both OnHeapColumnVector and OffHeapColumnVector,

[jira] [Commented] (SPARK-15702) Update document programming-guide accumulator section

2016-08-05 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15702?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15410431#comment-15410431 ] Sean Owen commented on SPARK-15702: --- I don't think this should be reopened to add a new and somewhat

[jira] [Commented] (SPARK-16864) Comprehensive version info

2016-08-05 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16864?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15410430#comment-15410430 ] Sean Owen commented on SPARK-16864: --- What would an app do with that info at runtime -- you'd really

[jira] [Assigned] (SPARK-16610) When writing ORC files, orc.compress should not be overridden if users do not set "compression" in the options

2016-08-05 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16610?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16610: Assignee: Apache Spark > When writing ORC files, orc.compress should not be overridden if

[jira] [Assigned] (SPARK-16610) When writing ORC files, orc.compress should not be overridden if users do not set "compression" in the options

2016-08-05 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16610?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16610: Assignee: (was: Apache Spark) > When writing ORC files, orc.compress should not be

[jira] [Commented] (SPARK-16610) When writing ORC files, orc.compress should not be overridden if users do not set "compression" in the options

2016-08-05 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16610?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15410393#comment-15410393 ] Apache Spark commented on SPARK-16610: -- User 'HyukjinKwon' has created a pull request for this

[jira] [Updated] (SPARK-16856) Link the application's executor page to the master's UI

2016-08-05 Thread Tao Lin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16856?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tao Lin updated SPARK-16856: Summary: Link the application's executor page to the master's UI (was: Link application summary page and

[jira] [Assigned] (SPARK-16931) PySpark access to data-frame bucketing api

2016-08-05 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16931?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16931: Assignee: (was: Apache Spark) > PySpark access to data-frame bucketing api >

[jira] [Commented] (SPARK-16931) PySpark access to data-frame bucketing api

2016-08-05 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16931?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15410351#comment-15410351 ] Apache Spark commented on SPARK-16931: -- User 'GregBowyer' has created a pull request for this issue:

[jira] [Assigned] (SPARK-16931) PySpark access to data-frame bucketing api

2016-08-05 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16931?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16931: Assignee: Apache Spark > PySpark access to data-frame bucketing api >

[jira] [Created] (SPARK-16931) PySpark access to data-frame bucketing api

2016-08-05 Thread Greg Bowyer (JIRA)
Greg Bowyer created SPARK-16931: --- Summary: PySpark access to data-frame bucketing api Key: SPARK-16931 URL: https://issues.apache.org/jira/browse/SPARK-16931 Project: Spark Issue Type:

[jira] [Commented] (SPARK-16508) Fix documentation warnings found by R CMD check

2016-08-05 Thread Junyang Qian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16508?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15410296#comment-15410296 ] Junyang Qian commented on SPARK-16508: -- It seems that there are still some warnings in my local

[jira] [Created] (SPARK-16930) ApplicationMaster's code that waits for SparkContext is race-prone

2016-08-05 Thread Marcelo Vanzin (JIRA)
Marcelo Vanzin created SPARK-16930: -- Summary: ApplicationMaster's code that waits for SparkContext is race-prone Key: SPARK-16930 URL: https://issues.apache.org/jira/browse/SPARK-16930 Project:

[jira] [Created] (SPARK-16929) Bad synchronization with regard to speculation

2016-08-05 Thread Nicholas Brown (JIRA)
Nicholas Brown created SPARK-16929: -- Summary: Bad synchronization with regard to speculation Key: SPARK-16929 URL: https://issues.apache.org/jira/browse/SPARK-16929 Project: Spark Issue

[jira] [Commented] (SPARK-15354) Topology aware block replication strategies

2016-08-05 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15354?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15410269#comment-15410269 ] Davies Liu commented on SPARK-15354: This strategy used in HDFS is to balance the write traffic (for

[jira] [Comment Edited] (SPARK-11638) Run Spark on Mesos with bridge networking

2016-08-05 Thread Stavros Kontopoulos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11638?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15410261#comment-15410261 ] Stavros Kontopoulos edited comment on SPARK-11638 at 8/5/16 11:08 PM:

[jira] [Commented] (SPARK-11638) Run Spark on Mesos with bridge networking

2016-08-05 Thread Stavros Kontopoulos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11638?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15410261#comment-15410261 ] Stavros Kontopoulos commented on SPARK-11638: - [~radekg][~mgummelt] what do you think? > Run

[jira] [Resolved] (SPARK-16901) Hive settings in hive-site.xml may be overridden by Hive's default values

2016-08-05 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16901?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai resolved SPARK-16901. -- Resolution: Fixed Fix Version/s: 2.1.0 2.0.1 Issue resolved by pull request

[jira] [Comment Edited] (SPARK-16864) Comprehensive version info

2016-08-05 Thread Jan Gorecki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16864?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15410238#comment-15410238 ] Jan Gorecki edited comment on SPARK-16864 at 8/5/16 10:52 PM: -- Hi, git

[jira] [Comment Edited] (SPARK-16864) Comprehensive version info

2016-08-05 Thread Jan Gorecki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16864?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15410238#comment-15410238 ] Jan Gorecki edited comment on SPARK-16864 at 8/5/16 10:51 PM: -- Hi, git

[jira] [Commented] (SPARK-16864) Comprehensive version info

2016-08-05 Thread Jan Gorecki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16864?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15410238#comment-15410238 ] Jan Gorecki commented on SPARK-16864: - Hi, git commit is relevant to applications at runtime as long

[jira] [Assigned] (SPARK-15702) Update document programming-guide accumulator section

2016-08-05 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15702?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15702: Assignee: Weichen Xu (was: Apache Spark) > Update document programming-guide accumulator

[jira] [Commented] (SPARK-15702) Update document programming-guide accumulator section

2016-08-05 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15702?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15410239#comment-15410239 ] Apache Spark commented on SPARK-15702: -- User 'BryanCutler' has created a pull request for this

[jira] [Assigned] (SPARK-15702) Update document programming-guide accumulator section

2016-08-05 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15702?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15702: Assignee: Apache Spark (was: Weichen Xu) > Update document programming-guide accumulator

[jira] [Reopened] (SPARK-15702) Update document programming-guide accumulator section

2016-08-05 Thread Bryan Cutler (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15702?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bryan Cutler reopened SPARK-15702: -- I'm reopening this because I think the current programming guide accumulator section is

[jira] [Updated] (SPARK-16924) DataStreamReader can not support option("inferSchema", true/false) for csv and json file source

2016-08-05 Thread Xin Wu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16924?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xin Wu updated SPARK-16924: --- Issue Type: Improvement (was: Bug) > DataStreamReader can not support option("inferSchema", true/false) for

[jira] [Commented] (SPARK-16926) Partition columns are present in columns metadata for partition but not table

2016-08-05 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16926?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15410202#comment-15410202 ] Apache Spark commented on SPARK-16926: -- User 'dafrista' has created a pull request for this issue:

[jira] [Assigned] (SPARK-16926) Partition columns are present in columns metadata for partition but not table

2016-08-05 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16926?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16926: Assignee: Apache Spark > Partition columns are present in columns metadata for partition

[jira] [Assigned] (SPARK-16926) Partition columns are present in columns metadata for partition but not table

2016-08-05 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16926?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16926: Assignee: (was: Apache Spark) > Partition columns are present in columns metadata for

[jira] [Assigned] (SPARK-16928) Recursive call of ColumnVector::getInt() breaks JIT inlining

2016-08-05 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16928?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16928: Assignee: Apache Spark > Recursive call of ColumnVector::getInt() breaks JIT inlining >

[jira] [Commented] (SPARK-16928) Recursive call of ColumnVector::getInt() breaks JIT inlining

2016-08-05 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16928?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15410173#comment-15410173 ] Apache Spark commented on SPARK-16928: -- User 'ooq' has created a pull request for this issue:

[jira] [Assigned] (SPARK-16928) Recursive call of ColumnVector::getInt() breaks JIT inlining

2016-08-05 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16928?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16928: Assignee: (was: Apache Spark) > Recursive call of ColumnVector::getInt() breaks JIT

[jira] [Created] (SPARK-16928) Recursive call of ColumnVector::getInt() breaks JIT inlining

2016-08-05 Thread Qifan Pu (JIRA)
Qifan Pu created SPARK-16928: Summary: Recursive call of ColumnVector::getInt() breaks JIT inlining Key: SPARK-16928 URL: https://issues.apache.org/jira/browse/SPARK-16928 Project: Spark Issue

[jira] [Assigned] (SPARK-16927) Mesos Cluster Dispatcher default properties

2016-08-05 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16927?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16927: Assignee: Apache Spark > Mesos Cluster Dispatcher default properties >

[jira] [Assigned] (SPARK-16927) Mesos Cluster Dispatcher default properties

2016-08-05 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16927?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16927: Assignee: (was: Apache Spark) > Mesos Cluster Dispatcher default properties >

[jira] [Assigned] (SPARK-16923) Mesos cluster scheduler duplicates config vars by setting them in the environment and as --conf

2016-08-05 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16923?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16923: Assignee: Apache Spark > Mesos cluster scheduler duplicates config vars by setting them

[jira] [Commented] (SPARK-16927) Mesos Cluster Dispatcher default properties

2016-08-05 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16927?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15410129#comment-15410129 ] Apache Spark commented on SPARK-16927: -- User 'mgummelt' has created a pull request for this issue:

[jira] [Assigned] (SPARK-16923) Mesos cluster scheduler duplicates config vars by setting them in the environment and as --conf

2016-08-05 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16923?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16923: Assignee: (was: Apache Spark) > Mesos cluster scheduler duplicates config vars by

[jira] [Commented] (SPARK-16923) Mesos cluster scheduler duplicates config vars by setting them in the environment and as --conf

2016-08-05 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16923?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15410130#comment-15410130 ] Apache Spark commented on SPARK-16923: -- User 'mgummelt' has created a pull request for this issue:

[jira] [Created] (SPARK-16927) Mesos Cluster Dispatcher default properties

2016-08-05 Thread Michael Gummelt (JIRA)
Michael Gummelt created SPARK-16927: --- Summary: Mesos Cluster Dispatcher default properties Key: SPARK-16927 URL: https://issues.apache.org/jira/browse/SPARK-16927 Project: Spark Issue

[jira] [Updated] (SPARK-16260) ML Example Improvements and Cleanup

2016-08-05 Thread Bryan Cutler (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16260?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bryan Cutler updated SPARK-16260: - Description: This parent task is to track a few possible improvements and cleanup for PySpark

[jira] [Updated] (SPARK-16260) ML Example Improvements and Cleanup

2016-08-05 Thread Bryan Cutler (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16260?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bryan Cutler updated SPARK-16260: - Summary: ML Example Improvements and Cleanup (was: PySpark ML Example Improvements and Cleanup)

[jira] [Commented] (SPARK-16883) SQL decimal type is not properly cast to number when collecting SparkDataFrame

2016-08-05 Thread Hossein Falaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16883?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15410105#comment-15410105 ] Hossein Falaki commented on SPARK-16883: Thanks [~shivaram]! This may require changing the

[jira] [Resolved] (SPARK-16817) Enable storing of shuffle data in Alluxio

2016-08-05 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16817?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-16817. --- Resolution: Won't Fix We removed the Tachyon dependency a while ago, although its purpose was

[jira] [Resolved] (SPARK-16784) Configurable log4j settings

2016-08-05 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16784?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-16784. --- Resolution: Not A Problem Reopen if that's not what you meant > Configurable log4j settings >

[jira] [Commented] (SPARK-16925) Spark tasks which cause JVM to exit with a zero exit code may cause app to hang in Standalone mode

2016-08-05 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16925?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15410078#comment-15410078 ] Apache Spark commented on SPARK-16925: -- User 'JoshRosen' has created a pull request for this issue:

[jira] [Updated] (SPARK-16925) Spark tasks which cause JVM to exit with a zero exit code may cause app to hang in Standalone mode

2016-08-05 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16925?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-16925: --- Description: If you have a Spark standalone cluster which runs a single application and you have a

[jira] [Resolved] (SPARK-16709) Task with commit failed will retry infinite when speculation set to true

2016-08-05 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16709?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-16709. --- Resolution: Duplicate > Task with commit failed will retry infinite when speculation set to true >

[jira] [Resolved] (SPARK-16499) Improve applyInPlace function for matrix in ANN code

2016-08-05 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16499?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-16499. --- Resolution: Won't Fix > Improve applyInPlace function for matrix in ANN code >

[jira] [Resolved] (SPARK-16455) Add a new hook in CoarseGrainedSchedulerBackend in order to stop scheduling new tasks when cluster is restarting

2016-08-05 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16455?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-16455. --- Resolution: Won't Fix > Add a new hook in CoarseGrainedSchedulerBackend in order to stop scheduling

[jira] [Updated] (SPARK-16925) Spark tasks which cause JVM to exit with a zero exit code may cause app to hang in Standalone mode

2016-08-05 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16925?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-16925: --- Summary: Spark tasks which cause JVM to exit with a zero exit code may cause app to hang in

[jira] [Created] (SPARK-16925) Spark tasks which cause JVM to exit with a zero exit code may cause app to hang

2016-08-05 Thread Josh Rosen (JIRA)
Josh Rosen created SPARK-16925: -- Summary: Spark tasks which cause JVM to exit with a zero exit code may cause app to hang Key: SPARK-16925 URL: https://issues.apache.org/jira/browse/SPARK-16925 Project:

[jira] [Created] (SPARK-16926) Partition columns are present in columns metadata for partition but not table

2016-08-05 Thread Brian Cho (JIRA)
Brian Cho created SPARK-16926: - Summary: Partition columns are present in columns metadata for partition but not table Key: SPARK-16926 URL: https://issues.apache.org/jira/browse/SPARK-16926 Project:

[jira] [Assigned] (SPARK-16924) DataStreamReader can not support option("inferSchema", true/false) for csv and json file source

2016-08-05 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16924?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16924: Assignee: Apache Spark > DataStreamReader can not support option("inferSchema",

[jira] [Assigned] (SPARK-16924) DataStreamReader can not support option("inferSchema", true/false) for csv and json file source

2016-08-05 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16924?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16924: Assignee: (was: Apache Spark) > DataStreamReader can not support

[jira] [Commented] (SPARK-16924) DataStreamReader can not support option("inferSchema", true/false) for csv and json file source

2016-08-05 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15410059#comment-15410059 ] Apache Spark commented on SPARK-16924: -- User 'xwu0226' has created a pull request for this issue:

[jira] [Created] (SPARK-16924) DataStreamReader can not support option("inferSchema", true/false) for csv and json file source

2016-08-05 Thread Xin Wu (JIRA)
Xin Wu created SPARK-16924: -- Summary: DataStreamReader can not support option("inferSchema", true/false) for csv and json file source Key: SPARK-16924 URL: https://issues.apache.org/jira/browse/SPARK-16924

[jira] [Created] (SPARK-16923) Mesos cluster scheduler duplicates config vars by setting them in the environment and as --conf

2016-08-05 Thread Michael Gummelt (JIRA)
Michael Gummelt created SPARK-16923: --- Summary: Mesos cluster scheduler duplicates config vars by setting them in the environment and as --conf Key: SPARK-16923 URL:

[jira] [Resolved] (SPARK-13238) Add ganglia dmax parameter

2016-08-05 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13238?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-13238. Resolution: Fixed Assignee: Ekasit Kijsipongse Fix Version/s: 2.1.0 > Add

[jira] [Commented] (SPARK-16586) spark-class crash with "[: too many arguments" instead of displaying the correct error message

2016-08-05 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16586?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15409984#comment-15409984 ] Apache Spark commented on SPARK-16586: -- User 'vanzin' has created a pull request for this issue:

[jira] [Resolved] (SPARK-16260) PySpark ML Example Improvements and Cleanup

2016-08-05 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16260?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-16260. --- Resolution: Done Assignee: Bryan Cutler > PySpark ML Example Improvements and Cleanup >

[jira] [Updated] (SPARK-16826) java.util.Hashtable limits the throughput of PARSE_URL()

2016-08-05 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16826?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-16826: -- Assignee: Sylvain Zimmer > java.util.Hashtable limits the throughput of PARSE_URL() >

[jira] [Updated] (SPARK-16421) Improve output from ML examples

2016-08-05 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16421?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-16421: -- Assignee: Bryan Cutler Priority: Major (was: Trivial) > Improve output from ML examples >

[jira] [Resolved] (SPARK-16421) Improve output from ML examples

2016-08-05 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16421?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-16421. --- Resolution: Fixed Fix Version/s: 2.1.0 Issue resolved by pull request 14308

[jira] [Resolved] (SPARK-16826) java.util.Hashtable limits the throughput of PARSE_URL()

2016-08-05 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16826?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-16826. --- Resolution: Fixed Fix Version/s: 2.1.0 Issue resolved by pull request 14488

[jira] [Commented] (SPARK-16922) Query failure due to executor OOM in Spark 2.0

2016-08-05 Thread Sital Kedia (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16922?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15409878#comment-15409878 ] Sital Kedia commented on SPARK-16922: - PS - Rerunning the query with

[jira] [Assigned] (SPARK-16905) Support SQL DDL: MSCK REPAIR TABLE

2016-08-05 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16905?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16905: Assignee: Apache Spark (was: Davies Liu) > Support SQL DDL: MSCK REPAIR TABLE >

[jira] [Commented] (SPARK-6305) Add support for log4j 2.x to Spark

2016-08-05 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6305?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15409829#comment-15409829 ] Sean Owen commented on SPARK-6305: -- The problem, I think, is that then this would not take effect if the

[jira] [Commented] (SPARK-16905) Support SQL DDL: MSCK REPAIR TABLE

2016-08-05 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16905?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15409830#comment-15409830 ] Apache Spark commented on SPARK-16905: -- User 'davies' has created a pull request for this issue:

[jira] [Assigned] (SPARK-16905) Support SQL DDL: MSCK REPAIR TABLE

2016-08-05 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16905?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16905: Assignee: Davies Liu (was: Apache Spark) > Support SQL DDL: MSCK REPAIR TABLE >

[jira] [Commented] (SPARK-16922) Query failure due to executor OOM in Spark 2.0

2016-08-05 Thread Sital Kedia (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16922?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15409813#comment-15409813 ] Sital Kedia commented on SPARK-16922: - cc - [~rxin] > Query failure due to executor OOM in Spark

[jira] [Created] (SPARK-16922) Query failure due to executor OOM in Spark 2.0

2016-08-05 Thread Sital Kedia (JIRA)
Sital Kedia created SPARK-16922: --- Summary: Query failure due to executor OOM in Spark 2.0 Key: SPARK-16922 URL: https://issues.apache.org/jira/browse/SPARK-16922 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-16610) When writing ORC files, orc.compress should not be overridden if users do not set "compression" in the options

2016-08-05 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16610?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15409801#comment-15409801 ] Yin Huai commented on SPARK-16610: -- Sure. For ORC, {{orc.compression}} is the actual conf that lets ORC

[jira] [Resolved] (SPARK-5312) Use sbt to detect new or changed public classes in PRs

2016-08-05 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5312?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-5312. -- Resolution: Won't Fix > Use sbt to detect new or changed public classes in PRs >

[jira] [Created] (SPARK-16921) RDD/DataFrame persist() and cache() should return Python context managers

2016-08-05 Thread Nicholas Chammas (JIRA)
Nicholas Chammas created SPARK-16921: Summary: RDD/DataFrame persist() and cache() should return Python context managers Key: SPARK-16921 URL: https://issues.apache.org/jira/browse/SPARK-16921

[jira] [Closed] (SPARK-7505) Update PySpark DataFrame docs: encourage __getitem__, mark as experimental, etc.

2016-08-05 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7505?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicholas Chammas closed SPARK-7505. --- Resolution: Invalid Closing this as invalid as I believe these issues are no longer

[jira] [Commented] (SPARK-5312) Use sbt to detect new or changed public classes in PRs

2016-08-05 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5312?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15409767#comment-15409767 ] Nicholas Chammas commented on SPARK-5312: - [~boyork] - Shall we close this? It doesn't look like

[jira] [Updated] (SPARK-16920) Investigate and fix issues introduced in SPARK-15858

2016-08-05 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16920?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-16920: -- Issue Type: Improvement (was: Bug) > Investigate and fix issues introduced in

[jira] [Created] (SPARK-16920) Investigate and fix issues introduced in SPARK-15858

2016-08-05 Thread Vladimir Feinberg (JIRA)
Vladimir Feinberg created SPARK-16920: - Summary: Investigate and fix issues introduced in SPARK-15858 Key: SPARK-16920 URL: https://issues.apache.org/jira/browse/SPARK-16920 Project: Spark

[jira] [Commented] (SPARK-16883) SQL decimal type is not properly cast to number when collecting SparkDataFrame

2016-08-05 Thread Shivaram Venkataraman (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16883?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15409691#comment-15409691 ] Shivaram Venkataraman commented on SPARK-16883: --- The thing to check then is how the

[jira] [Commented] (SPARK-6305) Add support for log4j 2.x to Spark

2016-08-05 Thread JIRA
[ https://issues.apache.org/jira/browse/SPARK-6305?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15409660#comment-15409660 ] Mikael Ståldal commented on SPARK-6305: --- For unit tests, you can put a logging configuration file in

[jira] [Commented] (SPARK-6305) Add support for log4j 2.x to Spark

2016-08-05 Thread JIRA
[ https://issues.apache.org/jira/browse/SPARK-6305?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15409655#comment-15409655 ] Mikael Ståldal commented on SPARK-6305: --- It should not be necessary to do explicit logging

[jira] [Commented] (SPARK-6305) Add support for log4j 2.x to Spark

2016-08-05 Thread JIRA
[ https://issues.apache.org/jira/browse/SPARK-6305?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15409646#comment-15409646 ] Mikael Ståldal commented on SPARK-6305: --- It might not be necessary to load logging configuration

[jira] [Updated] (SPARK-16917) Spark streaming kafka version compatibility.

2016-08-05 Thread Sudev (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16917?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sudev updated SPARK-16917: -- Description: It would be nice to have Kafka version compatibility information in the official documentation.

  1   2   >