[jira] [Created] (SPARK-22769) When driver stopping, there is problem: RpcEnv already stopped

2017-12-12 Thread KaiXinXIaoLei (JIRA)
KaiXinXIaoLei created SPARK-22769: - Summary: When driver stopping, there is problem: RpcEnv already stopped Key: SPARK-22769 URL: https://issues.apache.org/jira/browse/SPARK-22769 Project: Spark

[jira] [Resolved] (SPARK-22042) ReorderJoinPredicates can break when child's partitioning is not decided

2017-12-12 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22042?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-22042. - Resolution: Fixed Assignee: Tejas Patil Fix Version/s: 2.3.0 > ReorderJoinPredicates can

[jira] [Resolved] (SPARK-22700) Bucketizer.transform incorrectly drops row containing NaN

2017-12-12 Thread Nick Pentreath (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22700?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Pentreath resolved SPARK-22700. Resolution: Fixed Fix Version/s: 2.3.0 Issue resolved by pull request 19894

[jira] [Assigned] (SPARK-22700) Bucketizer.transform incorrectly drops row containing NaN

2017-12-12 Thread Nick Pentreath (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22700?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Pentreath reassigned SPARK-22700: -- Assignee: zhengruifeng > Bucketizer.transform incorrectly drops row containing NaN >

[jira] [Updated] (SPARK-19809) NullPointerException on zero-size ORC file

2017-12-12 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-19809: Component/s: (was: Input/Output) SQL > NullPointerException on zero-size ORC file >

[jira] [Resolved] (SPARK-19809) NullPointerException on zero-size ORC file

2017-12-12 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-19809. - Resolution: Fixed > NullPointerException on zero-size ORC file >

[jira] [Commented] (SPARK-20849) Document R DecisionTree

2017-12-12 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20849?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16288740#comment-16288740 ] Apache Spark commented on SPARK-20849: -- User 'zhengruifeng' has created a pull request for this

[jira] [Resolved] (SPARK-22644) Make ML testsuite support StructuredStreaming test

2017-12-12 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22644?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley resolved SPARK-22644. --- Resolution: Fixed Fix Version/s: 2.3.0 Issue resolved by pull request 19843

[jira] [Updated] (SPARK-22711) _pickle.PicklingError: args[0] from __newobj__ args has the wrong class from cloudpickle.py

2017-12-12 Thread Prateek (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22711?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prateek updated SPARK-22711: Affects Version/s: 2.2.1 > _pickle.PicklingError: args[0] from __newobj__ args has the wrong class from >

[jira] [Commented] (SPARK-22711) _pickle.PicklingError: args[0] from __newobj__ args has the wrong class from cloudpickle.py

2017-12-12 Thread Prateek (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22711?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16288722#comment-16288722 ] Prateek commented on SPARK-22711: - This is not linear regression, do you want me to send the code? >

[jira] [Commented] (SPARK-22711) _pickle.PicklingError: args[0] from __newobj__ args has the wrong class from cloudpickle.py

2017-12-12 Thread Prateek (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22711?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16288714#comment-16288714 ] Prateek commented on SPARK-22711: - with regular python its working fine. I tried with latest spark also

[jira] [Resolved] (SPARK-22768) Spark SQL alter table and beeline alter table Metadata is different

2017-12-12 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22768?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-22768. --- Resolution: Invalid It's very unclear what you're asking, but this should start on the mailing list.

[jira] [Created] (SPARK-22768) Spark SQL alter table and beeline alter table Metadata is different

2017-12-12 Thread JIRA
吴志龙 created SPARK-22768: --- Summary: Spark SQL alter table and beeline alter table Metadata is different Key: SPARK-22768 URL: https://issues.apache.org/jira/browse/SPARK-22768 Project: Spark Issue

[jira] [Assigned] (SPARK-22767) use ctx.addReferenceObj in InSet and ScalaUDF

2017-12-12 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22767?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-22767: Assignee: Apache Spark (was: Wenchen Fan) > use ctx.addReferenceObj in InSet and

[jira] [Assigned] (SPARK-22767) use ctx.addReferenceObj in InSet and ScalaUDF

2017-12-12 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22767?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-22767: Assignee: Wenchen Fan (was: Apache Spark) > use ctx.addReferenceObj in InSet and

[jira] [Commented] (SPARK-22767) use ctx.addReferenceObj in InSet and ScalaUDF

2017-12-12 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22767?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16288670#comment-16288670 ] Apache Spark commented on SPARK-22767: -- User 'cloud-fan' has created a pull request for this issue:

[jira] [Created] (SPARK-22767) use ctx.addReferenceObj in InSet and ScalaUDF

2017-12-12 Thread Wenchen Fan (JIRA)
Wenchen Fan created SPARK-22767: --- Summary: use ctx.addReferenceObj in InSet and ScalaUDF Key: SPARK-22767 URL: https://issues.apache.org/jira/browse/SPARK-22767 Project: Spark Issue Type:

[jira] [Commented] (SPARK-22496) beeline display operation log

2017-12-12 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22496?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16288642#comment-16288642 ] Apache Spark commented on SPARK-22496: -- User 'ChenjunZou' has created a pull request for this issue:

[jira] [Assigned] (SPARK-22600) Fix 64kb limit for deeply nested expressions under wholestage codegen

2017-12-12 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22600?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-22600: --- Assignee: Liang-Chi Hsieh > Fix 64kb limit for deeply nested expressions under wholestage

[jira] [Resolved] (SPARK-22600) Fix 64kb limit for deeply nested expressions under wholestage codegen

2017-12-12 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22600?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-22600. - Resolution: Fixed Fix Version/s: 2.3.0 Issue resolved by pull request 19813

[jira] [Commented] (SPARK-19809) NullPointerException on zero-size ORC file

2017-12-12 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16288623#comment-16288623 ] Dongjoon Hyun commented on SPARK-19809: --- Since Hive 1.2.1 library code path still has this problem,

[jira] [Assigned] (SPARK-19809) NullPointerException on zero-size ORC file

2017-12-12 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19809: Assignee: Apache Spark (was: Dongjoon Hyun) > NullPointerException on zero-size ORC file

[jira] [Commented] (SPARK-19809) NullPointerException on zero-size ORC file

2017-12-12 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16288622#comment-16288622 ] Apache Spark commented on SPARK-19809: -- User 'dongjoon-hyun' has created a pull request for this

[jira] [Assigned] (SPARK-19809) NullPointerException on zero-size ORC file

2017-12-12 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19809: Assignee: Dongjoon Hyun (was: Apache Spark) > NullPointerException on zero-size ORC file

[jira] [Resolved] (SPARK-22716) revisit ctx.addReferenceObj

2017-12-12 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22716?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-22716. - Resolution: Fixed Fix Version/s: 2.3.0 Issue resolved by pull request 19916

[jira] [Assigned] (SPARK-22716) Avoid the creation of mutable states in addReferenceObj

2017-12-12 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22716?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-22716: --- Assignee: Marco Gaido > Avoid the creation of mutable states in addReferenceObj >

[jira] [Updated] (SPARK-22716) Avoid the creation of mutable states in addReferenceObj

2017-12-12 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22716?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan updated SPARK-22716: Summary: Avoid the creation of mutable states in addReferenceObj (was: revisit

[jira] [Commented] (SPARK-22755) Expression (946-885)*1.0/946 < 0.1 and (946-885)*1.000/946 < 0.1 return different results

2017-12-12 Thread Kevin Zhang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22755?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16288608#comment-16288608 ] Kevin Zhang commented on SPARK-22755: - [~ksunitha] Thanks, it helps a lot > Expression

[jira] [Reopened] (SPARK-19809) NullPointerException on zero-size ORC file

2017-12-12 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li reopened SPARK-19809: - > NullPointerException on zero-size ORC file > -- > >

[jira] [Updated] (SPARK-22574) Wrong request causing Spark Dispatcher going inactive

2017-12-12 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22574?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin updated SPARK-22574: --- Fix Version/s: (was: 2.2.2) (was: 2.3.0) > Wrong request causing

[jira] [Reopened] (SPARK-22574) Wrong request causing Spark Dispatcher going inactive

2017-12-12 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22574?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin reopened SPARK-22574: > Wrong request causing Spark Dispatcher going inactive >

[jira] [Assigned] (SPARK-22574) Wrong request causing Spark Dispatcher going inactive

2017-12-12 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22574?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin reassigned SPARK-22574: -- Assignee: (was: German Schiavon Matteo) > Wrong request causing Spark Dispatcher

[jira] [Commented] (SPARK-22574) Wrong request causing Spark Dispatcher going inactive

2017-12-12 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22574?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16288506#comment-16288506 ] Marcelo Vanzin commented on SPARK-22574: (Commit above was reverted.) > Wrong request causing

[jira] [Assigned] (SPARK-22766) Install R linter package in spark lib directory

2017-12-12 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22766?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-22766: Assignee: Apache Spark > Install R linter package in spark lib directory >

[jira] [Commented] (SPARK-22766) Install R linter package in spark lib directory

2017-12-12 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22766?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16288428#comment-16288428 ] Apache Spark commented on SPARK-22766: -- User 'falaki' has created a pull request for this issue:

[jira] [Assigned] (SPARK-22766) Install R linter package in spark lib directory

2017-12-12 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22766?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-22766: Assignee: (was: Apache Spark) > Install R linter package in spark lib directory >

[jira] [Commented] (SPARK-21087) CrossValidator, TrainValidationSplit should collect all models when fitting: Scala API

2017-12-12 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21087?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16288426#comment-16288426 ] Apache Spark commented on SPARK-21087: -- User 'yanboliang' has created a pull request for this issue:

[jira] [Created] (SPARK-22766) Install R linter package in spark lib directory

2017-12-12 Thread Hossein Falaki (JIRA)
Hossein Falaki created SPARK-22766: -- Summary: Install R linter package in spark lib directory Key: SPARK-22766 URL: https://issues.apache.org/jira/browse/SPARK-22766 Project: Spark Issue

[jira] [Commented] (SPARK-22722) Test Coverage for Type Coercion Compatibility

2017-12-12 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16288425#comment-16288425 ] Xiao Li commented on SPARK-22722: - We need to run the same test cases in Hive. Thus, please ensure the

[jira] [Commented] (SPARK-22722) Test Coverage for Type Coercion Compatibility

2017-12-12 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16288422#comment-16288422 ] Xiao Li commented on SPARK-22722: - [~q79969786] I assigned this ticket to you. Thanks for your work! >

[jira] [Assigned] (SPARK-22722) Test Coverage for Type Coercion Compatibility

2017-12-12 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22722?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li reassigned SPARK-22722: --- Assignee: Yuming Wang > Test Coverage for Type Coercion Compatibility >

[jira] [Commented] (SPARK-22765) Create a new executor allocation scheme based on that of MR

2017-12-12 Thread Xuefu Zhang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22765?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16288414#comment-16288414 ] Xuefu Zhang commented on SPARK-22765: - I wouldn't say that MR is static, at lease not static in

[jira] [Assigned] (SPARK-19809) NullPointerException on zero-size ORC file

2017-12-12 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-19809: Assignee: Dongjoon Hyun > NullPointerException on zero-size ORC file >

[jira] [Resolved] (SPARK-19809) NullPointerException on zero-size ORC file

2017-12-12 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-19809. -- Resolution: Fixed Fix Version/s: 2.3.0 Issue resolved by pull request 19948

[jira] [Commented] (SPARK-20928) SPIP: Continuous Processing Mode for Structured Streaming

2017-12-12 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20928?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16288391#comment-16288391 ] Michael Armbrust commented on SPARK-20928: -- An update on this. We've started to create subtasks

[jira] [Assigned] (SPARK-22764) Flaky test: SparkContextSuite "Cancelling stages/jobs with custom reasons"

2017-12-12 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22764?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-22764: Assignee: (was: Apache Spark) > Flaky test: SparkContextSuite "Cancelling stages/jobs

[jira] [Assigned] (SPARK-22764) Flaky test: SparkContextSuite "Cancelling stages/jobs with custom reasons"

2017-12-12 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22764?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-22764: Assignee: Apache Spark > Flaky test: SparkContextSuite "Cancelling stages/jobs with

[jira] [Assigned] (SPARK-21867) Support async spilling in UnsafeShuffleWriter

2017-12-12 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21867?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21867: Assignee: Apache Spark > Support async spilling in UnsafeShuffleWriter >

[jira] [Commented] (SPARK-21867) Support async spilling in UnsafeShuffleWriter

2017-12-12 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21867?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16288363#comment-16288363 ] Apache Spark commented on SPARK-21867: -- User 'ericvandenbergfb' has created a pull request for this

[jira] [Assigned] (SPARK-21867) Support async spilling in UnsafeShuffleWriter

2017-12-12 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21867?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21867: Assignee: (was: Apache Spark) > Support async spilling in UnsafeShuffleWriter >

[jira] [Commented] (SPARK-22764) Flaky test: SparkContextSuite "Cancelling stages/jobs with custom reasons"

2017-12-12 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22764?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16288364#comment-16288364 ] Apache Spark commented on SPARK-22764: -- User 'vanzin' has created a pull request for this issue:

[jira] [Commented] (SPARK-22765) Create a new executor allocation scheme based on that of MR

2017-12-12 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22765?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16288353#comment-16288353 ] Sean Owen commented on SPARK-22765: --- I'm not clear why this needs another allocation scheme. You say

[jira] [Created] (SPARK-22765) Create a new executor allocation scheme based on that of MR

2017-12-12 Thread Xuefu Zhang (JIRA)
Xuefu Zhang created SPARK-22765: --- Summary: Create a new executor allocation scheme based on that of MR Key: SPARK-22765 URL: https://issues.apache.org/jira/browse/SPARK-22765 Project: Spark

[jira] [Created] (SPARK-22764) Flaky test: SparkContextSuite "Cancelling stages/jobs with custom reasons"

2017-12-12 Thread Marcelo Vanzin (JIRA)
Marcelo Vanzin created SPARK-22764: -- Summary: Flaky test: SparkContextSuite "Cancelling stages/jobs with custom reasons" Key: SPARK-22764 URL: https://issues.apache.org/jira/browse/SPARK-22764

[jira] [Updated] (SPARK-22644) Make ML testsuite support StructuredStreaming test

2017-12-12 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22644?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-22644: -- Target Version/s: 2.3.0 > Make ML testsuite support StructuredStreaming test >

[jira] [Assigned] (SPARK-22644) Make ML testsuite support StructuredStreaming test

2017-12-12 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22644?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley reassigned SPARK-22644: - Assignee: Weichen Xu > Make ML testsuite support StructuredStreaming test >

[jira] [Updated] (SPARK-22644) Make ML testsuite support StructuredStreaming test

2017-12-12 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22644?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-22644: -- Shepherd: Joseph K. Bradley > Make ML testsuite support StructuredStreaming test >

[jira] [Commented] (SPARK-22126) Fix model-specific optimization support for ML tuning

2017-12-12 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22126?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16288305#comment-16288305 ] Joseph K. Bradley commented on SPARK-22126: --- Hi all, thanks for the comments. I realized my

[jira] [Commented] (SPARK-21867) Support async spilling in UnsafeShuffleWriter

2017-12-12 Thread Eric Vandenberg (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21867?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16288298#comment-16288298 ] Eric Vandenberg commented on SPARK-21867: - 1. The default would be 1 so does not change default

[jira] [Commented] (SPARK-22683) Allow tuning the number of dynamically allocated executors wrt task number

2017-12-12 Thread Xuefu Zhang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22683?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16288262#comment-16288262 ] Xuefu Zhang commented on SPARK-22683: - Hi [~jcuquemelle], Thanks for working on this and bringing up

[jira] [Assigned] (SPARK-22757) Init-container in the driver/executor pods for downloading remote dependencies

2017-12-12 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22757?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-22757: Assignee: Apache Spark > Init-container in the driver/executor pods for downloading

[jira] [Assigned] (SPARK-22757) Init-container in the driver/executor pods for downloading remote dependencies

2017-12-12 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22757?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-22757: Assignee: (was: Apache Spark) > Init-container in the driver/executor pods for

[jira] [Commented] (SPARK-22757) Init-container in the driver/executor pods for downloading remote dependencies

2017-12-12 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22757?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16288213#comment-16288213 ] Apache Spark commented on SPARK-22757: -- User 'liyinan926' has created a pull request for this issue:

[jira] [Resolved] (SPARK-22574) Wrong request causing Spark Dispatcher going inactive

2017-12-12 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22574?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-22574. Resolution: Fixed Fix Version/s: 2.3.0 2.2.2 Issue resolved by

[jira] [Assigned] (SPARK-22574) Wrong request causing Spark Dispatcher going inactive

2017-12-12 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22574?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin reassigned SPARK-22574: -- Assignee: German Schiavon Matteo > Wrong request causing Spark Dispatcher going

[jira] [Resolved] (SPARK-22289) Cannot save LogisticRegressionClassificationModel with bounds on coefficients

2017-12-12 Thread Yanbo Liang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22289?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yanbo Liang resolved SPARK-22289. - Resolution: Fixed Fix Version/s: 2.3.0 2.2.2 > Cannot save

[jira] [Resolved] (SPARK-22755) Expression (946-885)*1.0/946 < 0.1 and (946-885)*1.000/946 < 0.1 return different results

2017-12-12 Thread Sunitha Kambhampati (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22755?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sunitha Kambhampati resolved SPARK-22755. - Resolution: Fixed Fix Version/s: 2.2.1 > Expression (946-885)*1.0/946 <

[jira] [Commented] (SPARK-22755) Expression (946-885)*1.0/946 < 0.1 and (946-885)*1.000/946 < 0.1 return different results

2017-12-12 Thread Sunitha Kambhampati (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22755?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16288061#comment-16288061 ] Sunitha Kambhampati commented on SPARK-22755: - SPARK-21332 fixed this issue. The changes

[jira] [Updated] (SPARK-22729) Add getTruncateQuery to JdbcDialect

2017-12-12 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22729?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-22729: Issue Type: Improvement (was: New Feature) > Add getTruncateQuery to JdbcDialect >

[jira] [Resolved] (SPARK-22729) Add getTruncateQuery to JdbcDialect

2017-12-12 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22729?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-22729. - Resolution: Fixed Fix Version/s: 2.3.0 > Add getTruncateQuery to JdbcDialect >

[jira] [Assigned] (SPARK-22729) Add getTruncateQuery to JdbcDialect

2017-12-12 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22729?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li reassigned SPARK-22729: --- Assignee: Daniel van der Ende > Add getTruncateQuery to JdbcDialect >

[jira] [Commented] (SPARK-22742) Spark2.x does not support read data from Hive 2.2 and 2.3

2017-12-12 Thread Zhongting Hu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16288044#comment-16288044 ] Zhongting Hu commented on SPARK-22742: -- Thanks [~smilegator] to reopen it, can I get a rough

[jira] [Commented] (SPARK-19834) csv escape of quote escape

2017-12-12 Thread Soonmok Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19834?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16288025#comment-16288025 ] Soonmok Kwon commented on SPARK-19834: -- I will re-open this soon (in a week). > csv escape of quote

[jira] [Commented] (SPARK-22755) Expression (946-885)*1.0/946 < 0.1 and (946-885)*1.000/946 < 0.1 return different results

2017-12-12 Thread Sunitha Kambhampati (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22755?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16287994#comment-16287994 ] Sunitha Kambhampati commented on SPARK-22755: - A) TRUNK: - My trunk codeline is sync'd up to

[jira] [Resolved] (SPARK-16986) "Started" time, "Completed" time and "Last Updated" time in history server UI are not user local time

2017-12-12 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16986?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-16986. Resolution: Fixed Assignee: Yuming Wang Fix Version/s: 2.3.0 > "Started"

[jira] [Reopened] (SPARK-16986) "Started" time, "Completed" time and "Last Updated" time in history server UI are not user local time

2017-12-12 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16986?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin reopened SPARK-16986: > "Started" time, "Completed" time and "Last Updated" time in history server UI > are not

[jira] [Commented] (SPARK-22763) SHS: Ignore unknown events and parse through the file

2017-12-12 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22763?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16287924#comment-16287924 ] Apache Spark commented on SPARK-22763: -- User 'gengliangwang' has created a pull request for this

[jira] [Assigned] (SPARK-22763) SHS: Ignore unknown events and parse through the file

2017-12-12 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22763?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-22763: Assignee: Apache Spark > SHS: Ignore unknown events and parse through the file >

[jira] [Updated] (SPARK-19809) NullPointerException on zero-size ORC file

2017-12-12 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-19809: -- Affects Version/s: (was: 2.2.0) 2.2.1 > NullPointerException on

[jira] [Assigned] (SPARK-22763) SHS: Ignore unknown events and parse through the file

2017-12-12 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22763?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-22763: Assignee: (was: Apache Spark) > SHS: Ignore unknown events and parse through the file

[jira] [Created] (SPARK-22763) SHS: Ignore unknown events and parse through the file

2017-12-12 Thread Gengliang Wang (JIRA)
Gengliang Wang created SPARK-22763: -- Summary: SHS: Ignore unknown events and parse through the file Key: SPARK-22763 URL: https://issues.apache.org/jira/browse/SPARK-22763 Project: Spark

[jira] [Commented] (SPARK-22351) Support user-created custom Encoders for Datasets

2017-12-12 Thread Adamos Loizou (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22351?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16287735#comment-16287735 ] Adamos Loizou commented on SPARK-22351: --- Hello guys, once more I've run against this problem now

[jira] [Comment Edited] (SPARK-22683) Allow tuning the number of dynamically allocated executors wrt task number

2017-12-12 Thread Julien Cuquemelle (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22683?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16286253#comment-16286253 ] Julien Cuquemelle edited comment on SPARK-22683 at 12/12/17 3:14 PM: -

[jira] [Commented] (SPARK-21322) support histogram in filter cardinality estimation

2017-12-12 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21322?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16287691#comment-16287691 ] Apache Spark commented on SPARK-21322: -- User 'cloud-fan' has created a pull request for this issue:

[jira] [Resolved] (SPARK-22761) 64KB JVM bytecode limit problem with GLM

2017-12-12 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22761?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-22761. --- Resolution: Duplicate Without more info this sounds like a possible duplicate of several other

[jira] [Commented] (SPARK-21914) Running examples as tests in SQL builtin function documentation

2017-12-12 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21914?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16287538#comment-16287538 ] Hyukjin Kwon commented on SPARK-21914: -- I totally misunderstood your comment [~rxin]. I was so

[jira] [Commented] (SPARK-18580) Use spark.streaming.backpressure.initialRate in DirectKafkaInputDStream

2017-12-12 Thread Alexey Lipodat (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18580?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16287496#comment-16287496 ] Alexey Lipodat commented on SPARK-18580: Guys, Could you please review Oleksandr's pull ^ ? This

[jira] [Commented] (SPARK-22760) where driver is stopping, and some executors lost because of YarnSchedulerBackend.stop, then there is a problem.

2017-12-12 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22760?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16287489#comment-16287489 ] Apache Spark commented on SPARK-22760: -- User 'KaiXinXiaoLei' has created a pull request for this

[jira] [Commented] (SPARK-22450) Safely register class for mllib

2017-12-12 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22450?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16287483#comment-16287483 ] Apache Spark commented on SPARK-22450: -- User 'zhengruifeng' has created a pull request for this

[jira] [Commented] (SPARK-21867) Support async spilling in UnsafeShuffleWriter

2017-12-12 Thread Eyal Farago (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21867?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16287461#comment-16287461 ] Eyal Farago commented on SPARK-21867: - [~ericvandenbergfb], looks good few questions though: 1.

[jira] [Commented] (SPARK-22761) 64KB JVM bytecode limit problem with GLM

2017-12-12 Thread Marco Gaido (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22761?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16287394#comment-16287394 ] Marco Gaido commented on SPARK-22761: - this is probably solved by SPARK-6. Can you try to

[jira] [Updated] (SPARK-22760) where driver is stopping, and some executors lost because of YarnSchedulerBackend.stop, then there is a problem.

2017-12-12 Thread KaiXinXIaoLei (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22760?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] KaiXinXIaoLei updated SPARK-22760: -- Description: Using {{SPARK-14228}} , i still find a problem: {noformat} 17/12/12 15:34:45

[jira] [Updated] (SPARK-22760) where driver is stopping, and some executors lost because of YarnSchedulerBackend.stop, then there is a problem.

2017-12-12 Thread KaiXinXIaoLei (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22760?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] KaiXinXIaoLei updated SPARK-22760: -- Description: Using SPARK-14228 {{monospaced text}} , i still find a problem: {noformat}

[jira] [Updated] (SPARK-22760) where driver is stopping, and some executors lost because of YarnSchedulerBackend.stop, then there is a problem.

2017-12-12 Thread KaiXinXIaoLei (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22760?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] KaiXinXIaoLei updated SPARK-22760: -- Description: Using SPARK-14228 , i still find a problem: {noformat} 17/12/12 15:34:45 INFO

[jira] [Updated] (SPARK-22760) where driver is stopping, and some executors lost because of YarnSchedulerBackend.stop, then there is a problem.

2017-12-12 Thread KaiXinXIaoLei (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22760?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] KaiXinXIaoLei updated SPARK-22760: -- Description: Using SPARK-14228 , i still find a problem: {noformat} 17/12/12 15:34:45 INFO

[jira] [Updated] (SPARK-22760) where driver is stopping, and some executors lost because of YarnSchedulerBackend.stop, then there is a problem.

2017-12-12 Thread KaiXinXIaoLei (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22760?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] KaiXinXIaoLei updated SPARK-22760: -- Description: Use SPARK-14228 , i find a problem: {panel:title=My title} Some text with a

[jira] [Updated] (SPARK-22760) where driver is stopping, and some executors lost because of YarnSchedulerBackend.stop, then there is a problem.

2017-12-12 Thread KaiXinXIaoLei (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22760?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] KaiXinXIaoLei updated SPARK-22760: -- Description: Use SPARK-14228 , i find a problem: ^superscript text^ ~subscript text~

[jira] [Updated] (SPARK-22760) where driver is stopping, and some executors lost because of YarnSchedulerBackend.stop, then there is a problem.

2017-12-12 Thread KaiXinXIaoLei (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22760?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] KaiXinXIaoLei updated SPARK-22760: -- Description: Use SPARK-14228 , i find a problem: ^ 17/12/12 15:34:45 INFO

[jira] [Commented] (SPARK-22752) FileNotFoundException while reading from Kafka

2017-12-12 Thread Marco Gaido (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22752?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16287305#comment-16287305 ] Marco Gaido commented on SPARK-22752: - Hi [~zsxwing], thanks for looking at this. The checkpointDir

[jira] [Commented] (SPARK-22749) df.na().fill(0) over an unnamed FIRST() fails the query

2017-12-12 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22749?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16287251#comment-16287251 ] Hyukjin Kwon commented on SPARK-22749: -- ping [~efrat.s] > df.na().fill(0) over an unnamed FIRST()

  1   2   >