[jira] [Created] (SPARK-21745) Refactor ColumnVector hierarchy to make ColumnVector read-only and to introduce MutableColumnVector.

2017-08-15 Thread Takuya Ueshin (JIRA)
Takuya Ueshin created SPARK-21745: - Summary: Refactor ColumnVector hierarchy to make ColumnVector read-only and to introduce MutableColumnVector. Key: SPARK-21745 URL:

[jira] [Commented] (SPARK-21736) Spark 2.2 in Windows does not recognize the URI "file:Z:\SIT1\TreatmentManager\app.properties"

2017-08-15 Thread Ross Brigoli (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21736?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16128336#comment-16128336 ] Ross Brigoli commented on SPARK-21736: -- You're right file:/z:/works. But the issue is that

[jira] [Assigned] (SPARK-14516) Clustering evaluator

2017-08-15 Thread Yanbo Liang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14516?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yanbo Liang reassigned SPARK-14516: --- Assignee: Marco Gaido > Clustering evaluator > > >

[jira] [Updated] (SPARK-14516) Clustering evaluator

2017-08-15 Thread Yanbo Liang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14516?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yanbo Liang updated SPARK-14516: Shepherd: Yanbo Liang Affects Version/s: 2.2.0 > Clustering evaluator >

[jira] [Updated] (SPARK-21744) Add retry logic when create new broadcast

2017-08-15 Thread zhoukang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21744?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhoukang updated SPARK-21744: - Description: When driver submit new stage and there is a bad disk before spark,then driver may will

[jira] [Commented] (SPARK-21726) Check for structural integrity of the plan in QO in test mode

2017-08-15 Thread Liang-Chi Hsieh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21726?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16128327#comment-16128327 ] Liang-Chi Hsieh commented on SPARK-21726: - Submitted a PR at

[jira] [Created] (SPARK-21744) Add retry logic when create new broadcast

2017-08-15 Thread zhoukang (JIRA)
zhoukang created SPARK-21744: Summary: Add retry logic when create new broadcast Key: SPARK-21744 URL: https://issues.apache.org/jira/browse/SPARK-21744 Project: Spark Issue Type: Improvement

[jira] [Commented] (SPARK-4502) Spark SQL reads unneccesary nested fields from Parquet

2017-08-15 Thread Gaurav Shah (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4502?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16128318#comment-16128318 ] Gaurav Shah commented on SPARK-4502: [~marmbrus] Do you have some time to review this pull request ?

[jira] [Commented] (SPARK-19256) Hive bucketing support

2017-08-15 Thread Tejas Patil (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19256?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16128305#comment-16128305 ] Tejas Patil commented on SPARK-19256: - PR for writer side changes is out :

[jira] [Created] (SPARK-21743) top-most limit should not cause memory leak

2017-08-15 Thread Wenchen Fan (JIRA)
Wenchen Fan created SPARK-21743: --- Summary: top-most limit should not cause memory leak Key: SPARK-21743 URL: https://issues.apache.org/jira/browse/SPARK-21743 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-17025) Cannot persist PySpark ML Pipeline model that includes custom Transformer

2017-08-15 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17025?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16128271#comment-16128271 ] Nicholas Chammas commented on SPARK-17025: -- I'm still interested in this but I won't be able to

[jira] [Commented] (SPARK-21739) timestamp partition would fail in v2.2.0

2017-08-15 Thread Feng Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21739?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16128267#comment-16128267 ] Feng Zhu commented on SPARK-21739: -- In such case, the Cast expression is evaluated directly without

[jira] [Updated] (SPARK-21742) BisectingKMeans generate different models with/without caching

2017-08-15 Thread zhengruifeng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21742?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhengruifeng updated SPARK-21742: - Summary: BisectingKMeans generate different models with/without caching (was: BisectingKMeans

[jira] [Created] (SPARK-21742) BisectingKMeans generate different results with/without caching

2017-08-15 Thread zhengruifeng (JIRA)
zhengruifeng created SPARK-21742: Summary: BisectingKMeans generate different results with/without caching Key: SPARK-21742 URL: https://issues.apache.org/jira/browse/SPARK-21742 Project: Spark

[jira] [Commented] (SPARK-18394) Executing the same query twice in a row results in CodeGenerator cache misses

2017-08-15 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18394?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16128243#comment-16128243 ] Takeshi Yamamuro commented on SPARK-18394: -- Any update? I checked and I found the master still

[jira] [Commented] (SPARK-21657) Spark has exponential time complexity to explode(array of structs)

2017-08-15 Thread Liang-Chi Hsieh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21657?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16128242#comment-16128242 ] Liang-Chi Hsieh commented on SPARK-21657: - [~maropu] I've noticed that change. There is a hotfix

[jira] [Updated] (SPARK-21716) The time-range window can't be applied on the reduce operator

2017-08-15 Thread Fan Donglai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21716?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fan Donglai updated SPARK-21716: Description: I can't use GroupBy + Window operator to get the newest(the maximum event time) row

[jira] [Commented] (SPARK-21741) Python API for DataFrame-based multivariate summarizer

2017-08-15 Thread Weichen Xu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21741?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16128226#comment-16128226 ] Weichen Xu commented on SPARK-21741: OK I will work on this. I will post a design doc first. >

[jira] [Created] (SPARK-21741) Python API for DataFrame-based multivariate summarizer

2017-08-15 Thread Yanbo Liang (JIRA)
Yanbo Liang created SPARK-21741: --- Summary: Python API for DataFrame-based multivariate summarizer Key: SPARK-21741 URL: https://issues.apache.org/jira/browse/SPARK-21741 Project: Spark Issue

[jira] [Commented] (SPARK-21741) Python API for DataFrame-based multivariate summarizer

2017-08-15 Thread Yanbo Liang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21741?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16128223#comment-16128223 ] Yanbo Liang commented on SPARK-21741: - [~WeichenXu123] Would you like to work on this? Thanks. >

[jira] [Resolved] (SPARK-19634) Feature parity for descriptive statistics in MLlib

2017-08-15 Thread Yanbo Liang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19634?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yanbo Liang resolved SPARK-19634. - Resolution: Fixed Fix Version/s: 2.3.0 > Feature parity for descriptive statistics in

[jira] [Assigned] (SPARK-21712) Clarify PySpark Column.substr() type checking error message

2017-08-15 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21712?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-21712: Assignee: Nicholas Chammas > Clarify PySpark Column.substr() type checking error message

[jira] [Resolved] (SPARK-21712) Clarify PySpark Column.substr() type checking error message

2017-08-15 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21712?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-21712. -- Resolution: Fixed Fix Version/s: 2.3.0 Issue resolved by pull request 18926

[jira] [Updated] (SPARK-21681) MLOR do not work correctly when featureStd contains zero

2017-08-15 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21681?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-21681: -- Shepherd: Joseph K. Bradley > MLOR do not work correctly when featureStd contains zero

[jira] [Updated] (SPARK-21681) MLOR do not work correctly when featureStd contains zero

2017-08-15 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21681?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-21681: -- Affects Version/s: 2.3.0 > MLOR do not work correctly when featureStd contains zero >

[jira] [Updated] (SPARK-21681) MLOR do not work correctly when featureStd contains zero

2017-08-15 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21681?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-21681: -- Target Version/s: 2.2.1, 2.3.0 > MLOR do not work correctly when featureStd contains

[jira] [Assigned] (SPARK-21681) MLOR do not work correctly when featureStd contains zero

2017-08-15 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21681?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley reassigned SPARK-21681: - Assignee: Weichen Xu > MLOR do not work correctly when featureStd contains zero

[jira] [Commented] (SPARK-20589) Allow limiting task concurrency per stage

2017-08-15 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20589?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16127942#comment-16127942 ] Thomas Graves commented on SPARK-20589: --- Note that this type of option is also already supported in

[jira] [Commented] (SPARK-20589) Allow limiting task concurrency per stage

2017-08-15 Thread Amit Kumar (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20589?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16127930#comment-16127930 ] Amit Kumar commented on SPARK-20589: [~imranr] Like you said, we could restrict the number of

[jira] [Commented] (SPARK-20589) Allow limiting task concurrency per stage

2017-08-15 Thread Imran Rashid (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20589?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16127911#comment-16127911 ] Imran Rashid commented on SPARK-20589: -- couldn't you just limit the number of executors of your

[jira] [Resolved] (SPARK-21731) Upgrade scalastyle to 0.9

2017-08-15 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21731?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-21731. Resolution: Fixed Assignee: Marcelo Vanzin Fix Version/s: 2.3.0

[jira] [Created] (SPARK-21740) DataFrame.write does not work with Phoenix JDBC Driver

2017-08-15 Thread Paul Wu (JIRA)
Paul Wu created SPARK-21740: --- Summary: DataFrame.write does not work with Phoenix JDBC Driver Key: SPARK-21740 URL: https://issues.apache.org/jira/browse/SPARK-21740 Project: Spark Issue Type: Bug

[jira] [Resolved] (SPARK-17742) Spark Launcher does not get failed state in Listener

2017-08-15 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17742?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-17742. Resolution: Fixed Fix Version/s: 2.3.0 https://github.com/apache/spark/pull/18877

[jira] [Comment Edited] (SPARK-21720) Filter predicate with many conditions throw stackoverflow error

2017-08-15 Thread poplav (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21720?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16127657#comment-16127657 ] poplav edited comment on SPARK-21720 at 8/15/17 6:10 PM: - [~kiszk] So, to clarify

[jira] [Comment Edited] (SPARK-21720) Filter predicate with many conditions throw stackoverflow error

2017-08-15 Thread poplav (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21720?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16127657#comment-16127657 ] poplav edited comment on SPARK-21720 at 8/15/17 6:10 PM: - [~kiszk] So, to clarify

[jira] [Commented] (SPARK-21720) Filter predicate with many conditions throw stackoverflow error

2017-08-15 Thread poplav (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21720?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16127657#comment-16127657 ] poplav commented on SPARK-21720: So, to clarify did the fix simply bump up the number of fields before we

[jira] [Commented] (SPARK-21738) Thriftserver doesn't cancel jobs when session is closed

2017-08-15 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21738?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16127542#comment-16127542 ] Xiao Li commented on SPARK-21738: - https://github.com/apache/spark/pull/18951 > Thriftserver doesn't

[jira] [Comment Edited] (SPARK-21657) Spark has exponential time complexity to explode(array of structs)

2017-08-15 Thread Ruslan Dautkhanov (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21657?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16127511#comment-16127511 ] Ruslan Dautkhanov edited comment on SPARK-21657 at 8/15/17 4:59 PM:

[jira] [Commented] (SPARK-21657) Spark has exponential time complexity to explode(array of structs)

2017-08-15 Thread Ruslan Dautkhanov (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21657?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16127511#comment-16127511 ] Ruslan Dautkhanov commented on SPARK-21657: --- Thank you [~maropu] and [~viirya], that commit is

[jira] [Comment Edited] (SPARK-21720) Filter predicate with many conditions throw stackoverflow error

2017-08-15 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21720?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16127477#comment-16127477 ] Kazuaki Ishizaki edited comment on SPARK-21720 at 8/15/17 4:26 PM: --- In

[jira] [Commented] (SPARK-21720) Filter predicate with many conditions throw stackoverflow error

2017-08-15 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21720?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16127477#comment-16127477 ] Kazuaki Ishizaki commented on SPARK-21720: -- In this case, to add JVM option {{-Xss512m}}

[jira] [Resolved] (SPARK-21034) Allow filter pushdown filters through non deterministic functions for columns involved in groupby / join

2017-08-15 Thread Abhijit Bhole (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21034?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Abhijit Bhole resolved SPARK-21034. --- Resolution: Duplicate > Allow filter pushdown filters through non deterministic functions

[jira] [Updated] (SPARK-21739) timestamp partition would fail in v2.2.0

2017-08-15 Thread wangzhihao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21739?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wangzhihao updated SPARK-21739: --- Description: The spark v2.2.0 introduce TimeZoneAwareExpression, which causes bugs if we select

[jira] [Updated] (SPARK-21739) timestamp partition would fail in v2.2.0

2017-08-15 Thread wangzhihao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21739?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wangzhihao updated SPARK-21739: --- Description: The spark v2.2.0 introduce TimeZoneAwareExpression, which causes bugs if we select

[jira] [Commented] (SPARK-21034) Allow filter pushdown filters through non deterministic functions for columns involved in groupby / join

2017-08-15 Thread Abhijit Bhole (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21034?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16127454#comment-16127454 ] Abhijit Bhole commented on SPARK-21034: --- Thank you !! > Allow filter pushdown filters through non

[jira] [Updated] (SPARK-21739) timestamp partition would fail in v2.2.0

2017-08-15 Thread wangzhihao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21739?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wangzhihao updated SPARK-21739: --- Description: The spark v2.2.0 introduce TimeZoneAwareExpression, which causes bugs if we select

[jira] [Updated] (SPARK-21739) timestamp partition would fail in v2.2.0

2017-08-15 Thread wangzhihao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21739?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wangzhihao updated SPARK-21739: --- Description: The spark v2.2.0 introduce TimeZoneAwareExpression, which causes bugs if we select

[jira] [Updated] (SPARK-21739) timestamp partition would fail in v2.2.0

2017-08-15 Thread wangzhihao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21739?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wangzhihao updated SPARK-21739: --- Description: The spark v2.2.0 introduce TimeZoneAwareExpression, which causes bugs if we select

[jira] [Updated] (SPARK-21739) timestamp partition would fail in v2.2.0

2017-08-15 Thread wangzhihao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21739?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wangzhihao updated SPARK-21739: --- Description: The spark v2.2.0 introduce TimeZoneAwareExpression, which causes bugs if we select

[jira] [Updated] (SPARK-21739) timestamp partition would fail in v2.2.0

2017-08-15 Thread wangzhihao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21739?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wangzhihao updated SPARK-21739: --- Description: The spark v2.2.0 introduce TimeZoneAwareExpression, which causes bugs if we select

[jira] [Updated] (SPARK-21739) timestamp partition would fail in v2.2.0

2017-08-15 Thread wangzhihao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21739?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wangzhihao updated SPARK-21739: --- Description: The spark v2.2.0 introduce TimeZoneAwareExpression, which causes bugs if we select

[jira] [Updated] (SPARK-21739) timestamp partition would fail in v2.2.0

2017-08-15 Thread wangzhihao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21739?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wangzhihao updated SPARK-21739: --- Docs Text: (was: The spark v2.2.0 introduce TimeZoneAwareExpression, which causes bugs if we

[jira] [Updated] (SPARK-21739) timestamp partition would fail in v2.2.0

2017-08-15 Thread wangzhihao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21739?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wangzhihao updated SPARK-21739: --- Docs Text: The spark v2.2.0 introduce TimeZoneAwareExpression, which causes bugs if we select data

[jira] [Created] (SPARK-21739) timestamp partition would fail in v2.2.0

2017-08-15 Thread wangzhihao (JIRA)
wangzhihao created SPARK-21739: -- Summary: timestamp partition would fail in v2.2.0 Key: SPARK-21739 URL: https://issues.apache.org/jira/browse/SPARK-21739 Project: Spark Issue Type: Bug

[jira] [Resolved] (SPARK-21721) Memory leak in org.apache.spark.sql.hive.execution.InsertIntoHiveTable

2017-08-15 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21721?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-21721. - Resolution: Fixed Assignee: Liang-Chi Hsieh Fix Version/s: 2.3.0 2.2.1

[jira] [Created] (SPARK-21738) Thriftserver doesn't cancel jobs when session is closed

2017-08-15 Thread Marco Gaido (JIRA)
Marco Gaido created SPARK-21738: --- Summary: Thriftserver doesn't cancel jobs when session is closed Key: SPARK-21738 URL: https://issues.apache.org/jira/browse/SPARK-21738 Project: Spark Issue

[jira] [Commented] (SPARK-19019) PySpark does not work with Python 3.6.0

2017-08-15 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19019?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16127339#comment-16127339 ] Hyukjin Kwon commented on SPARK-19019: -- I think this was backported into Spark 2.1.1. Was your Spark

[jira] [Commented] (SPARK-19019) PySpark does not work with Python 3.6.0

2017-08-15 Thread Mathias M. Andersen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19019?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16127330#comment-16127330 ] Mathias M. Andersen commented on SPARK-19019: - Just got this error post fix on spark 2.1:

[jira] [Comment Edited] (SPARK-19019) PySpark does not work with Python 3.6.0

2017-08-15 Thread Mathias M. Andersen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19019?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16127330#comment-16127330 ] Mathias M. Andersen edited comment on SPARK-19019 at 8/15/17 2:51 PM:

[jira] [Created] (SPARK-21737) Create communication channel between arbitrary clients and the Spark AM in YARN mode

2017-08-15 Thread Jong Yoon Lee (JIRA)
Jong Yoon Lee created SPARK-21737: - Summary: Create communication channel between arbitrary clients and the Spark AM in YARN mode Key: SPARK-21737 URL: https://issues.apache.org/jira/browse/SPARK-21737

[jira] [Resolved] (SPARK-21736) Spark 2.2 in Windows does not recognize the URI "file:Z:\SIT1\TreatmentManager\app.properties"

2017-08-15 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21736?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-21736. --- Resolution: Not A Problem > Spark 2.2 in Windows does not recognize the URI >

[jira] [Updated] (SPARK-21736) Spark 2.2 in Windows does not recognize the URI "file:Z:\SIT1\TreatmentManager\app.properties"

2017-08-15 Thread Ross Brigoli (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21736?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ross Brigoli updated SPARK-21736: - Description: We are currently running Spark 2.1 in Windows Server 2012 in Production. While

[jira] [Commented] (SPARK-21736) Spark 2.2 in Windows does not recognize the URI "file:Z:\SIT1\TreatmentManager\app.properties"

2017-08-15 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21736?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16127309#comment-16127309 ] Sean Owen commented on SPARK-21736: --- This isn't related to Spark though. The value, property are

[jira] [Updated] (SPARK-21736) Spark 2.2 in Windows does not recognize the URI "file:Z:\SIT1\TreatmentManager\app.properties"

2017-08-15 Thread Ross Brigoli (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21736?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ross Brigoli updated SPARK-21736: - Description: We are currently running Spark 2.1 in Windows Server 2012 in Production. While

[jira] [Commented] (SPARK-21726) Check for structural integrity of the plan in QO in test mode

2017-08-15 Thread Liang-Chi Hsieh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21726?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16127288#comment-16127288 ] Liang-Chi Hsieh commented on SPARK-21726: - [~rxin] Thanks for pinging me! Yes, I'm interested in

[jira] [Resolved] (SPARK-21734) spark job blocked by updateDependencies

2017-08-15 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21734?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-21734. --- Resolution: Invalid Not sure what you mean, but this should start maybe as a question on the

[jira] [Commented] (SPARK-21714) SparkSubmit in Yarn Client mode downloads remote files and then reuploads them again

2017-08-15 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21714?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16127204#comment-16127204 ] Thomas Graves commented on SPARK-21714: --- I haven't had time to get to it, so it would be great if

[jira] [Commented] (SPARK-21734) spark job blocked by updateDependencies

2017-08-15 Thread Cheng jin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21734?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16127203#comment-16127203 ] Cheng jin commented on SPARK-21734: --- we met this problem, in fact , it is like this: {code:java}

[jira] [Commented] (SPARK-21714) SparkSubmit in Yarn Client mode downloads remote files and then reuploads them again

2017-08-15 Thread Saisai Shao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21714?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16127199#comment-16127199 ] Saisai Shao commented on SPARK-21714: - Let me take a crack on this if no one is working on it. >

[jira] [Commented] (SPARK-21736) Spark 2.2 in Windows does not recognize the URI file:/z:/sparklogs from spark-defaults.conf

2017-08-15 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21736?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16127172#comment-16127172 ] Sean Owen commented on SPARK-21736: --- Hm, isn't it {{file:///z:/...}}? not sure if that makes a

[jira] [Created] (SPARK-21736) Spark 2.2 in Windows does not recognize the URI file:/z:/sparklogs from spark-defaults.conf

2017-08-15 Thread Ross Brigoli (JIRA)
Ross Brigoli created SPARK-21736: Summary: Spark 2.2 in Windows does not recognize the URI file:/z:/sparklogs from spark-defaults.conf Key: SPARK-21736 URL: https://issues.apache.org/jira/browse/SPARK-21736

[jira] [Resolved] (SPARK-21704) Add the description of 'sbin/stop-slave.sh' in spark-standalone.html.

2017-08-15 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21704?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-21704. --- Resolution: Not A Problem > Add the description of 'sbin/stop-slave.sh' in spark-standalone.html. >

[jira] [Updated] (SPARK-21734) spark job blocked by updateDependencies

2017-08-15 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21734?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-21734: -- Flags: (was: Patch) Target Version/s: (was: 2.2.0) Labels: (was:

[jira] [Resolved] (SPARK-21735) spark job blocked by updateDependencies

2017-08-15 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21735?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-21735. --- Resolution: Duplicate Target Version/s: (was: 2.2.0) > spark job blocked by

[jira] [Commented] (SPARK-21702) Structured Streaming S3A SSE Encryption Not Applied when PartitionBy Used

2017-08-15 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21702?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16127062#comment-16127062 ] Steve Loughran commented on SPARK-21702: ps, you shouldn't need to set the s3a.impl field; that's

[jira] [Created] (SPARK-21735) spark job blocked by updateDependencies

2017-08-15 Thread Cheng jin (JIRA)
Cheng jin created SPARK-21735: - Summary: spark job blocked by updateDependencies Key: SPARK-21735 URL: https://issues.apache.org/jira/browse/SPARK-21735 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-21702) Structured Streaming S3A SSE Encryption Not Applied when PartitionBy Used

2017-08-15 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21702?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16127061#comment-16127061 ] Steve Loughran commented on SPARK-21702: This is interesting. What may be happening is that

[jira] [Created] (SPARK-21734) spark job blocked by updateDependencies

2017-08-15 Thread Cheng jin (JIRA)
Cheng jin created SPARK-21734: - Summary: spark job blocked by updateDependencies Key: SPARK-21734 URL: https://issues.apache.org/jira/browse/SPARK-21734 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-21681) MLOR do not work correctly when featureStd contains zero

2017-08-15 Thread Weichen Xu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21681?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Weichen Xu updated SPARK-21681: --- Description: MLOR do not work correctly when featureStd contains zero. We can reproduce the bug

[jira] [Commented] (SPARK-21733) ERROR executor.CoarseGrainedExecutorBackend: RECEIVED SIGNAL TERM

2017-08-15 Thread Jepson (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21733?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16126978#comment-16126978 ] Jepson commented on SPARK-21733: [~sowen], thank you for correcting me, I will notice it at the next

[jira] [Resolved] (SPARK-21733) ERROR executor.CoarseGrainedExecutorBackend: RECEIVED SIGNAL TERM

2017-08-15 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21733?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-21733. --- Resolution: Not A Problem Fix Version/s: (was: 2.1.1) It wasn't "Fixed"; there was no

[jira] [Reopened] (SPARK-21733) ERROR executor.CoarseGrainedExecutorBackend: RECEIVED SIGNAL TERM

2017-08-15 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21733?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen reopened SPARK-21733: --- > ERROR executor.CoarseGrainedExecutorBackend: RECEIVED SIGNAL TERM >

[jira] [Commented] (SPARK-21733) ERROR executor.CoarseGrainedExecutorBackend: RECEIVED SIGNAL TERM

2017-08-15 Thread Saisai Shao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21733?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16126974#comment-16126974 ] Saisai Shao commented on SPARK-21733: - Should the resolution of this JIRA be "invalid" or else?

[jira] [Resolved] (SPARK-21733) ERROR executor.CoarseGrainedExecutorBackend: RECEIVED SIGNAL TERM

2017-08-15 Thread Jepson (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21733?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jepson resolved SPARK-21733. Resolution: Fixed Fix Version/s: 2.1.1 > ERROR executor.CoarseGrainedExecutorBackend: RECEIVED

[jira] [Commented] (SPARK-21733) ERROR executor.CoarseGrainedExecutorBackend: RECEIVED SIGNAL TERM

2017-08-15 Thread Jepson (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21733?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16126970#comment-16126970 ] Jepson commented on SPARK-21733: I have resolve this issue.Thanks for [~jerryshao] and [~srowen]

[jira] [Updated] (SPARK-21733) ERROR executor.CoarseGrainedExecutorBackend: RECEIVED SIGNAL TERM

2017-08-15 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21733?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-21733: -- Shepherd: (was: -) Flags: (was: Patch,Important) Labels: (was: patch) Yes, this

[jira] [Updated] (SPARK-21723) Can't write LibSVM - key not found: numFeatures

2017-08-15 Thread JIRA
[ https://issues.apache.org/jira/browse/SPARK-21723?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jan Vršovský updated SPARK-21723: - Description: Writing a dataset to LibSVM format raises an exception

[jira] [Commented] (SPARK-21723) Can't write LibSVM - key not found: numFeatures

2017-08-15 Thread Nick Pentreath (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21723?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16126939#comment-16126939 ] Nick Pentreath commented on SPARK-21723: Yes, we should definitely be able to write LibSVM format

[jira] [Updated] (SPARK-21723) Can't write LibSVM - key not found: numFeatures

2017-08-15 Thread JIRA
[ https://issues.apache.org/jira/browse/SPARK-21723?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jan Vršovský updated SPARK-21723: - Description: Writing a dataset to LibSVM format raises an exception

[jira] [Commented] (SPARK-21691) Accessing canonicalized plan for query with limit throws exception

2017-08-15 Thread Bjoern Toldbod (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21691?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16126947#comment-16126947 ] Bjoern Toldbod commented on SPARK-21691: The suggested solution works for me. Thanks >

[jira] [Comment Edited] (SPARK-21652) Optimizer cannot reach a fixed point on certain queries

2017-08-15 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21652?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16126917#comment-16126917 ] Takeshi Yamamuro edited comment on SPARK-21652 at 8/15/17 8:08 AM: ---

[jira] [Comment Edited] (SPARK-21363) Prevent column name duplication in temporary view

2017-08-15 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21363?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16126847#comment-16126847 ] Takeshi Yamamuro edited comment on SPARK-21363 at 8/15/17 8:08 AM: ---

[jira] [Commented] (SPARK-21652) Optimizer cannot reach a fixed point on certain queries

2017-08-15 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21652?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16126917#comment-16126917 ] Takeshi Yamamuro commented on SPARK-21652: -- The pr is:

[jira] [Commented] (SPARK-21481) Add indexOf method in ml.feature.HashingTF similar to mllib.feature.HashingTF

2017-08-15 Thread Yanbo Liang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21481?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16126914#comment-16126914 ] Yanbo Liang commented on SPARK-21481: - See my comments at https://github.com/apache/spark/pull/18736

[jira] [Commented] (SPARK-21034) Allow filter pushdown filters through non deterministic functions for columns involved in groupby / join

2017-08-15 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21034?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16126902#comment-16126902 ] Takeshi Yamamuro commented on SPARK-21034: -- It seems this is duplicate of SPARK-21707. If so, we

[jira] [Commented] (SPARK-21733) ERROR executor.CoarseGrainedExecutorBackend: RECEIVED SIGNAL TERM

2017-08-15 Thread Saisai Shao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21733?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16126894#comment-16126894 ] Saisai Shao commented on SPARK-21733: - Is there any issue here? > ERROR

[jira] [Commented] (SPARK-21657) Spark has exponential time complexity to explode(array of structs)

2017-08-15 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21657?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16126877#comment-16126877 ] Takeshi Yamamuro commented on SPARK-21657: -- [~viirya] Probably, this is what you want?

[jira] [Updated] (SPARK-21363) Prevent column name duplication in temporary view

2017-08-15 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21363?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-21363: Target Version/s: 3.0.0 > Prevent column name duplication in temporary view >

[jira] [Commented] (SPARK-21363) Prevent column name duplication in temporary view

2017-08-15 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21363?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16126869#comment-16126869 ] Xiao Li commented on SPARK-21363: - We need to revisit whether we should support duplicate column names in

[jira] [Commented] (SPARK-21732) Lazily init hive metastore client

2017-08-15 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21732?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16126866#comment-16126866 ] Xiao Li commented on SPARK-21732: - https://github.com/apache/spark/pull/18944 > Lazily init hive

  1   2   >