[jira] [Closed] (SPARK-19057) Instance weights must be non-negative

2017-06-07 Thread zhengruifeng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19057?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhengruifeng closed SPARK-19057. Resolution: Won't Fix > Instance weights must be non-negative >

[jira] [Comment Edited] (SPARK-21001) Staging folders from Hive table are not being cleared.

2017-06-07 Thread Liang-Chi Hsieh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16040341#comment-16040341 ] Liang-Chi Hsieh edited comment on SPARK-21001 at 6/8/17 5:20 AM: - I found

[jira] [Commented] (SPARK-19547) KafkaUtil throw 'No current assignment for partition' Exception

2017-06-07 Thread Utkarsh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19547?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16042241#comment-16042241 ] Utkarsh commented on SPARK-19547: - Hi, I'm using Kafka 0.10.0 and kafka-clients 0.10.0.0 and am facing

[jira] [Updated] (SPARK-21012) Support glob path for resources adding to Spark

2017-06-07 Thread Saisai Shao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21012?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao updated SPARK-21012: Description: Current "\-\-jars (spark.jars)", "\-\-files (spark.files)", "\-\-py-files

[jira] [Commented] (SPARK-19307) SPARK-17387 caused ignorance of conf object passed to SparkContext:

2017-06-07 Thread Irina Truong (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19307?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16041040#comment-16041040 ] Irina Truong commented on SPARK-19307: -- Is this available in 2.1.1? I could not find it in release

[jira] [Commented] (SPARK-21009) SparkListenerTaskEnd.taskInfo.accumulables might not be accurate

2017-06-07 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21009?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16041039#comment-16041039 ] Yuming Wang commented on SPARK-21009: - This could be duplicate of

[jira] [Commented] (SPARK-21010) Spark-Sql Can't Handle char() type Well

2017-06-07 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21010?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16041021#comment-16041021 ] Yuming Wang commented on SPARK-21010: - The problem is that you define {{char(20)}},

[jira] [Updated] (SPARK-19360) Spark 2.X does not support stored by clause

2017-06-07 Thread Artur Sukhenko (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19360?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Artur Sukhenko updated SPARK-19360: --- Summary: Spark 2.X does not support stored by clause (was: Spark 2.X does not support

[jira] [Commented] (SPARK-20342) DAGScheduler sends SparkListenerTaskEnd before updating task's accumulators

2017-06-07 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20342?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16041198#comment-16041198 ] Marcelo Vanzin commented on SPARK-20342: I have a fix for this, might as well use a variant of

[jira] [Commented] (SPARK-20342) DAGScheduler sends SparkListenerTaskEnd before updating task's accumulators

2017-06-07 Thread Bogdan Raducanu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20342?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16041054#comment-16041054 ] Bogdan Raducanu commented on SPARK-20342: - This code fails because of this issue: {code}

[jira] [Commented] (SPARK-21009) SparkListenerTaskEnd.taskInfo.accumulables might not be accurate

2017-06-07 Thread Bogdan Raducanu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21009?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16041055#comment-16041055 ] Bogdan Raducanu commented on SPARK-21009: - Yes, looks like duplicate. I posted the repro code in

[jira] [Commented] (SPARK-18085) Better History Server scalability for many / large applications

2017-06-07 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18085?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16041153#comment-16041153 ] Marcelo Vanzin commented on SPARK-18085: bq. the procedure of loading the history summary page is

[jira] [Commented] (SPARK-20760) Memory Leak of RDD blocks

2017-06-07 Thread Jose Soltren (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20760?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16041027#comment-16041027 ] Jose Soltren commented on SPARK-20760: -- Hi - I've been researching this issue, and I agree with

[jira] [Resolved] (SPARK-21010) Spark-Sql Can't Handle char() type Well

2017-06-07 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21010?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-21010. --- Resolution: Not A Problem Fix Version/s: (was: 2.1.1) Target Version/s: (was:

[jira] [Resolved] (SPARK-21009) SparkListenerTaskEnd.taskInfo.accumulables might not be accurate

2017-06-07 Thread Bogdan Raducanu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21009?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bogdan Raducanu resolved SPARK-21009. - Resolution: Duplicate > SparkListenerTaskEnd.taskInfo.accumulables might not be accurate

[jira] [Commented] (SPARK-20691) Difference between Storage Memory as seen internally and in web UI

2017-06-07 Thread Maarten Kesselaers (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20691?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16040263#comment-16040263 ] Maarten Kesselaers commented on SPARK-20691: [~srowen], I would like to work on this one. As

[jira] [Commented] (SPARK-20691) Difference between Storage Memory as seen internally and in web UI

2017-06-07 Thread Maarten Kesselaers (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20691?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16040294#comment-16040294 ] Maarten Kesselaers commented on SPARK-20691: OK, some components use the function {noformat}

[jira] [Commented] (SPARK-21001) Staging folders from Hive table are not being cleared.

2017-06-07 Thread Liang-Chi Hsieh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16040341#comment-16040341 ] Liang-Chi Hsieh commented on SPARK-21001: - I found a PR which backports the related fix to 2.0

[jira] [Commented] (SPARK-20998) BroadcastHashJoin producing wrong results

2017-06-07 Thread Liang-Chi Hsieh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20998?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16040321#comment-16040321 ] Liang-Chi Hsieh commented on SPARK-20998: - Can you provide a sample data to reproduce this issue?

[jira] [Comment Edited] (SPARK-20691) Difference between Storage Memory as seen internally and in web UI

2017-06-07 Thread Maarten Kesselaers (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20691?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16040263#comment-16040263 ] Maarten Kesselaers edited comment on SPARK-20691 at 6/7/17 6:24 AM:

[jira] [Updated] (SPARK-20977) NPE in CollectionAccumulator

2017-06-07 Thread sharkd tu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20977?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sharkd tu updated SPARK-20977: -- Environment: OpenJDK 64-Bit Server VM (25.71-b00) for linux-ppc64 JRE

[jira] [Created] (SPARK-21003) Spark Packages not working properly

2017-06-07 Thread Dulaj Rajitha (JIRA)
Dulaj Rajitha created SPARK-21003: - Summary: Spark Packages not working properly Key: SPARK-21003 URL: https://issues.apache.org/jira/browse/SPARK-21003 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-20342) DAGScheduler sends SparkListenerTaskEnd before updating task's accumulators

2017-06-07 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20342?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16041256#comment-16041256 ] Apache Spark commented on SPARK-20342: -- User 'vanzin' has created a pull request for this issue:

[jira] [Assigned] (SPARK-20342) DAGScheduler sends SparkListenerTaskEnd before updating task's accumulators

2017-06-07 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20342?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-20342: Assignee: Apache Spark > DAGScheduler sends SparkListenerTaskEnd before updating task's

[jira] [Assigned] (SPARK-20342) DAGScheduler sends SparkListenerTaskEnd before updating task's accumulators

2017-06-07 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20342?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-20342: Assignee: (was: Apache Spark) > DAGScheduler sends SparkListenerTaskEnd before

[jira] [Commented] (SPARK-17237) DataFrame fill after pivot causing org.apache.spark.sql.AnalysisException

2017-06-07 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17237?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16041434#comment-16041434 ] Takeshi Yamamuro commented on SPARK-17237: -- Thanks for the report. I think there are two points

[jira] [Commented] (SPARK-19185) ConcurrentModificationExceptions with CachedKafkaConsumers when Windowing

2017-06-07 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19185?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16041271#comment-16041271 ] Apache Spark commented on SPARK-19185: -- User 'markgrover' has created a pull request for this issue:

[jira] [Resolved] (SPARK-19800) Implement one kind of streaming sampling - reservoir sampling

2017-06-07 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19800?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-19800. --- Resolution: Won't Fix > Implement one kind of streaming sampling - reservoir sampling >

[jira] [Comment Edited] (SPARK-17237) DataFrame fill after pivot causing org.apache.spark.sql.AnalysisException

2017-06-07 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17237?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16041434#comment-16041434 ] Takeshi Yamamuro edited comment on SPARK-17237 at 6/7/17 7:04 PM: --

[jira] [Updated] (SPARK-21003) Spark Java Configuration : spark.jars.packages not working properly

2017-06-07 Thread Dulaj Rajitha (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21003?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dulaj Rajitha updated SPARK-21003: -- Description: I am unable to load maven dependencies for spark executors using

[jira] [Created] (SPARK-21005) VectorIndexerModel does not prepare output column field correctly

2017-06-07 Thread Chen Lin (JIRA)
Chen Lin created SPARK-21005: Summary: VectorIndexerModel does not prepare output column field correctly Key: SPARK-21005 URL: https://issues.apache.org/jira/browse/SPARK-21005 Project: Spark

[jira] [Resolved] (SPARK-20512) SparkR 2.2 QA: Programming guide, migration guide, vignettes updates

2017-06-07 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20512?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-20512. --- Resolution: Done > SparkR 2.2 QA: Programming guide, migration guide, vignettes updates >

[jira] [Resolved] (SPARK-20378) StreamSinkProvider should provide schema in createSink.

2017-06-07 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20378?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-20378. --- Resolution: Won't Fix > StreamSinkProvider should provide schema in createSink. >

[jira] [Resolved] (SPARK-14460) DataFrameWriter JDBC doesn't Quote/Escape column names

2017-06-07 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14460?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-14460. --- Resolution: Duplicate > DataFrameWriter JDBC doesn't Quote/Escape column names >

[jira] [Resolved] (SPARK-14686) Implement a non-inheritable localProperty facility

2017-06-07 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14686?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-14686. --- Resolution: Won't Fix > Implement a non-inheritable localProperty facility >

[jira] [Assigned] (SPARK-20691) Difference between Storage Memory as seen internally and in web UI

2017-06-07 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20691?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-20691: Assignee: (was: Apache Spark) > Difference between Storage Memory as seen internally

[jira] [Commented] (SPARK-21008) Streaming applications read stale credentials file when recovering from checkpoint.

2017-06-07 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21008?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16040565#comment-16040565 ] Apache Spark commented on SPARK-21008: -- User 'saturday-shi' has created a pull request for this

[jira] [Assigned] (SPARK-21008) Streaming applications read stale credentials file when recovering from checkpoint.

2017-06-07 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21008?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21008: Assignee: (was: Apache Spark) > Streaming applications read stale credentials file

[jira] [Assigned] (SPARK-21008) Streaming applications read stale credentials file when recovering from checkpoint.

2017-06-07 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21008?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21008: Assignee: Apache Spark > Streaming applications read stale credentials file when

[jira] [Resolved] (SPARK-20149) Audit PySpark code base for 2.6 specific work arounds

2017-06-07 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20149?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-20149. --- Resolution: Done > Audit PySpark code base for 2.6 specific work arounds >

[jira] [Created] (SPARK-21006) Create rpcEnv and run later needs shutdown and awaitTermination

2017-06-07 Thread wangjiaochun (JIRA)
wangjiaochun created SPARK-21006: Summary: Create rpcEnv and run later needs shutdown and awaitTermination Key: SPARK-21006 URL: https://issues.apache.org/jira/browse/SPARK-21006 Project: Spark

[jira] [Resolved] (SPARK-20515) Issue with reading Hive ORC tables having char/varchar columns in Spark SQL

2017-06-07 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20515?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-20515. --- Resolution: Duplicate > Issue with reading Hive ORC tables having char/varchar columns in Spark SQL

[jira] [Resolved] (SPARK-16124) Throws exception when executing query on `build/sbt hive/console`

2017-06-07 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16124?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-16124. --- Resolution: Not A Problem > Throws exception when executing query on `build/sbt hive/console` >

[jira] [Created] (SPARK-21007) Add SQL function - RIGHT && LEFT

2017-06-07 Thread liuxian (JIRA)
liuxian created SPARK-21007: --- Summary: Add SQL function - RIGHT && LEFT Key: SPARK-21007 URL: https://issues.apache.org/jira/browse/SPARK-21007 Project: Spark Issue Type: Improvement

[jira] [Updated] (SPARK-21007) Add SQL function - RIGHT && LEFT

2017-06-07 Thread liuxian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21007?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liuxian updated SPARK-21007: Description: Add SQL function - RIGHT && LEFT, same as MySQL:

[jira] [Commented] (SPARK-20622) Parquet partition discovery for non key=value named directories

2017-06-07 Thread Noam Asor (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20622?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16040515#comment-16040515 ] Noam Asor commented on SPARK-20622: --- The provided pull request is not complete and is rather in a POC

[jira] [Commented] (SPARK-20928) Continuous Processing Mode for Structured Streaming

2017-06-07 Thread Wladimir Schmidt (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20928?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16040516#comment-16040516 ] Wladimir Schmidt commented on SPARK-20928: -- I am really looking forward for it! This sub

[jira] [Updated] (SPARK-21005) VectorIndexerModel does not prepare output column field correctly

2017-06-07 Thread Chen Lin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21005?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chen Lin updated SPARK-21005: - Description: >From my understanding through reading the documentation, VectorIndexer >decides which

[jira] [Updated] (SPARK-20993) The configuration item about 'Spark.blacklist.enabled' need to set the default value 'false'

2017-06-07 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20993?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-20993: -- Priority: Trivial (was: Minor) > The configuration item about 'Spark.blacklist.enabled' need to set

[jira] [Updated] (SPARK-20980) Rename the option `wholeFile` to `multiLine` for JSON and CSV

2017-06-07 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20980?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-20980: -- Target Version/s: (was: 2.2.0) Priority: Major (was: Blocker) Issue Type: Task

[jira] [Commented] (SPARK-20427) Issue with Spark interpreting Oracle datatype NUMBER

2017-06-07 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20427?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16040484#comment-16040484 ] Yuming Wang commented on SPARK-20427: - [~srowen], [~smilegator] *Workaround*: # Increase DecimalType

[jira] [Resolved] (SPARK-18385) Make the transformer's natively in ml framework to avoid extra conversion

2017-06-07 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18385?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-18385. --- Resolution: Won't Fix > Make the transformer's natively in ml framework to avoid extra conversion >

[jira] [Resolved] (SPARK-16856) Link the application's executor page to the master's UI

2017-06-07 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16856?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-16856. --- Resolution: Won't Fix > Link the application's executor page to the master's UI >

[jira] [Created] (SPARK-21008) Streaming applications read stale credentials file when recovering from checkpoint.

2017-06-07 Thread Xing Shi (JIRA)
Xing Shi created SPARK-21008: Summary: Streaming applications read stale credentials file when recovering from checkpoint. Key: SPARK-21008 URL: https://issues.apache.org/jira/browse/SPARK-21008 Project:

[jira] [Commented] (SPARK-20691) Difference between Storage Memory as seen internally and in web UI

2017-06-07 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20691?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16040556#comment-16040556 ] Apache Spark commented on SPARK-20691: -- User 'mkesselaers' has created a pull request for this

[jira] [Assigned] (SPARK-20691) Difference between Storage Memory as seen internally and in web UI

2017-06-07 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20691?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-20691: Assignee: Apache Spark > Difference between Storage Memory as seen internally and in web

[jira] [Resolved] (SPARK-20966) Table data is not sorted by startTime time desc, time is not formatted and redundant code in JDBC/ODBC Server page.

2017-06-07 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20966?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-20966. --- Resolution: Fixed Fix Version/s: 2.3.0 Issue resolved by pull request 18186

[jira] [Assigned] (SPARK-20966) Table data is not sorted by startTime time desc, time is not formatted and redundant code in JDBC/ODBC Server page.

2017-06-07 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20966?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen reassigned SPARK-20966: - Assignee: guoxiaolongzte > Table data is not sorted by startTime time desc, time is not

[jira] [Updated] (SPARK-20448) Document how FileInputDStream works with object storage

2017-06-07 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20448?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-20448: -- Target Version/s: (was: 2.2.0) > Document how FileInputDStream works with object storage >

[jira] [Updated] (SPARK-14151) Propose to refactor and expose Metrics Sink and Source interface

2017-06-07 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14151?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-14151: -- Target Version/s: (was: 2.2.0) > Propose to refactor and expose Metrics Sink and Source interface >

[jira] [Updated] (SPARK-20929) LinearSVC should not use shared Param HasThresholds

2017-06-07 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20929?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-20929: -- Target Version/s: (was: 2.2.0) > LinearSVC should not use shared Param HasThresholds >

[jira] [Updated] (SPARK-17637) Packed scheduling for Spark tasks across executors

2017-06-07 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17637?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-17637: -- Target Version/s: (was: 2.2.0) > Packed scheduling for Spark tasks across executors >

[jira] [Updated] (SPARK-21003) Spark Java Configuration : spark.jars.packages not working properly

2017-06-07 Thread Dulaj Rajitha (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21003?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dulaj Rajitha updated SPARK-21003: -- Summary: Spark Java Configuration : spark.jars.packages not working properly (was: Spark

[jira] [Resolved] (SPARK-21004) remove the never used declaration in function warnDeprecatedVersions

2017-06-07 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21004?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-21004. --- Resolution: Won't Fix Agree, but, this is really too minor to bother with in one issue, and, this

[jira] [Assigned] (SPARK-21006) Create rpcEnv and run later needs shutdown and awaitTermination

2017-06-07 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21006?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21006: Assignee: Apache Spark > Create rpcEnv and run later needs shutdown and awaitTermination

[jira] [Assigned] (SPARK-21006) Create rpcEnv and run later needs shutdown and awaitTermination

2017-06-07 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21006?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21006: Assignee: (was: Apache Spark) > Create rpcEnv and run later needs shutdown and

[jira] [Commented] (SPARK-21006) Create rpcEnv and run later needs shutdown and awaitTermination

2017-06-07 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21006?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16040474#comment-16040474 ] Apache Spark commented on SPARK-21006: -- User 'wangjiaochun' has created a pull request for this

[jira] [Resolved] (SPARK-17799) InterfaceStability annotation

2017-06-07 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17799?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-17799. --- Resolution: Done > InterfaceStability annotation > - > >

[jira] [Commented] (SPARK-20676) Upload to PyPi

2017-06-07 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20676?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16040579#comment-16040579 ] Sean Owen commented on SPARK-20676: --- [~holdenk] are all the PyPi changes done? > Upload to PyPi >

[jira] [Resolved] (SPARK-21003) Spark Java Configuration : spark.jars.packages not working properly

2017-06-07 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21003?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-21003. --- Resolution: Invalid There isn't info here to understand the problem. This should start as a

[jira] [Commented] (SPARK-20691) Difference between Storage Memory as seen internally and in web UI

2017-06-07 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20691?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16040409#comment-16040409 ] Sean Owen commented on SPARK-20691: --- [~maartenkessela...@gmail.com] what I don't want to do is change

[jira] [Created] (SPARK-21004) remove the never used declaration in function warnDeprecatedVersions

2017-06-07 Thread yangZhiguo (JIRA)
yangZhiguo created SPARK-21004: -- Summary: remove the never used declaration in function warnDeprecatedVersions Key: SPARK-21004 URL: https://issues.apache.org/jira/browse/SPARK-21004 Project: Spark

[jira] [Commented] (SPARK-21004) remove the never used declaration in function warnDeprecatedVersions

2017-06-07 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16040426#comment-16040426 ] Apache Spark commented on SPARK-21004: -- User 'JackYangzg' has created a pull request for this issue:

[jira] [Assigned] (SPARK-21005) VectorIndexerModel does not prepare output column field correctly

2017-06-07 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21005?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21005: Assignee: (was: Apache Spark) > VectorIndexerModel does not prepare output column

[jira] [Assigned] (SPARK-21005) VectorIndexerModel does not prepare output column field correctly

2017-06-07 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21005?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21005: Assignee: Apache Spark > VectorIndexerModel does not prepare output column field

[jira] [Commented] (SPARK-21005) VectorIndexerModel does not prepare output column field correctly

2017-06-07 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21005?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16040478#comment-16040478 ] Apache Spark commented on SPARK-21005: -- User 'hibayesian' has created a pull request for this issue:

[jira] [Assigned] (SPARK-21007) Add SQL function - RIGHT && LEFT

2017-06-07 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21007?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21007: Assignee: (was: Apache Spark) > Add SQL function - RIGHT && LEFT >

[jira] [Assigned] (SPARK-21007) Add SQL function - RIGHT && LEFT

2017-06-07 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21007?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21007: Assignee: Apache Spark > Add SQL function - RIGHT && LEFT >

[jira] [Updated] (SPARK-20622) Parquet partition discovery for non key=value named directories

2017-06-07 Thread Noam Asor (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20622?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Noam Asor updated SPARK-20622: -- Priority: Minor (was: Major) > Parquet partition discovery for non key=value named directories >

[jira] [Commented] (SPARK-21007) Add SQL function - RIGHT && LEFT

2017-06-07 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21007?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16040510#comment-16040510 ] Apache Spark commented on SPARK-21007: -- User '10110346' has created a pull request for this issue:

[jira] [Resolved] (SPARK-12661) Drop Python 2.6 support in PySpark

2017-06-07 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12661?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-12661. --- Resolution: Done > Drop Python 2.6 support in PySpark > -- > >

[jira] [Updated] (SPARK-20947) Encoding/decoding issue in PySpark pipe implementation

2017-06-07 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20947?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-20947: -- Target Version/s: (was: 1.6.3) > Encoding/decoding issue in PySpark pipe implementation >

[jira] [Updated] (SPARK-20491) Synonym handling replacement issue in Apache Spark

2017-06-07 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20491?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-20491: -- Target Version/s: (was: 2.0.2) > Synonym handling replacement issue in Apache Spark >

[jira] [Updated] (SPARK-21011) RDD filter can combine/corrupt columns

2017-06-07 Thread Steven Landes (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21011?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steven Landes updated SPARK-21011: -- Description: I used PySpark to read in some CSV files (actually separated by backspace, might

[jira] [Created] (SPARK-21011) RDD filter can combine/corrupt columns

2017-06-07 Thread Steven Landes (JIRA)
Steven Landes created SPARK-21011: - Summary: RDD filter can combine/corrupt columns Key: SPARK-21011 URL: https://issues.apache.org/jira/browse/SPARK-21011 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-18085) Better History Server scalability for many / large applications

2017-06-07 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18085?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16041498#comment-16041498 ] Marcelo Vanzin commented on SPARK-18085: BTW for the last issue I had already filed SPARK-20656

[jira] [Commented] (SPARK-2868) Support named accumulators in Python

2017-06-07 Thread Kyle Heath (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2868?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16041557#comment-16041557 ] Kyle Heath commented on SPARK-2868: --- @[~holdenk]: I would love to better understand the scope of the

[jira] [Commented] (SPARK-12559) Cluster mode doesn't work with --packages

2017-06-07 Thread Stavros Kontopoulos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12559?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16041622#comment-16041622 ] Stavros Kontopoulos commented on SPARK-12559: - Is anyone working on this? Some people may

[jira] [Updated] (SPARK-20962) Support subquery column aliases in FROM clause

2017-06-07 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20962?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-20962: -- Fix Version/s: (was: 2.3.0) > Support subquery column aliases in FROM clause >

[jira] [Updated] (SPARK-17495) Hive hash implementation

2017-06-07 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17495?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-17495: -- Fix Version/s: (was: 2.2.0) > Hive hash implementation > > >

[jira] [Updated] (SPARK-20894) Error while checkpointing to HDFS

2017-06-07 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20894?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-20894: -- Fix Version/s: (was: 2.3.0) > Error while checkpointing to HDFS >

[jira] [Updated] (SPARK-20994) Alleviate memory pressure in StreamManager

2017-06-07 Thread jin xing (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20994?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jin xing updated SPARK-20994: - Description: In my cluster, we are suffering from OOM of shuffle-service. We found that a lot of

[jira] [Updated] (SPARK-20963) Support column aliases for aliased relation in FROM clause

2017-06-07 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20963?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-20963: -- Fix Version/s: (was: 2.3.0) > Support column aliases for aliased relation in FROM clause >

[jira] [Commented] (SPARK-20997) spark-submit's --driver-cores marked as "YARN-only" but listed under "Spark standalone with cluster deploy mode only"

2017-06-07 Thread Jacek Laskowski (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20997?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16040594#comment-16040594 ] Jacek Laskowski commented on SPARK-20997: - Go ahead! Thanks [~guoxiaolongzte]! > spark-submit's

[jira] [Commented] (SPARK-20994) Alleviate memory pressure in StreamManager

2017-06-07 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20994?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16040612#comment-16040612 ] Apache Spark commented on SPARK-20994: -- User 'jinxing64' has created a pull request for this issue:

[jira] [Commented] (SPARK-20998) BroadcastHashJoin producing wrong results

2017-06-07 Thread Mohit (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20998?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16040687#comment-16040687 ] Mohit commented on SPARK-20998: --- [~viirya] Attached. > BroadcastHashJoin producing wrong results >

[jira] [Comment Edited] (SPARK-20998) BroadcastHashJoin producing wrong results

2017-06-07 Thread Mohit (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20998?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16040687#comment-16040687 ] Mohit edited comment on SPARK-20998 at 6/7/17 10:41 AM: [~viirya] Attached an orc

[jira] [Commented] (SPARK-18085) Better History Server scalability for many / large applications

2017-06-07 Thread DjvuLee (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18085?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16040633#comment-16040633 ] DjvuLee commented on SPARK-18085: - [~vanzin] the procedure of loading the history summary page is still a

[jira] [Commented] (SPARK-17237) DataFrame fill after pivot causing org.apache.spark.sql.AnalysisException

2017-06-07 Thread JIRA
[ https://issues.apache.org/jira/browse/SPARK-17237?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16040716#comment-16040716 ] Alberto Fernández commented on SPARK-17237: --- Hi there, I think this change introduced a

[jira] [Resolved] (SPARK-20491) Synonym handling replacement issue in Apache Spark

2017-06-07 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20491?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-20491. --- Resolution: Invalid This looks like a question for the mailing list or somewhere else, not a JIRA >

  1   2   >