[jira] [Commented] (SPARK-23109) ML 2.3 QA: API: Python API coverage

[ https://issues.apache.org/jira/browse/SPARK-23109?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16340653#comment-16340653 ] Nick Pentreath commented on SPARK-23109: [~bryanc] can you add a Jira for adding {{columnSchema}}

[jira] [Assigned] (SPARK-23200) Reset configuration when restarting from checkpoints

[ https://issues.apache.org/jira/browse/SPARK-23200?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao reassigned SPARK-23200: --- Assignee: Anirudh Ramanathan > Reset configuration when restarting from checkpoints >

[jira] [Updated] (SPARK-22799) Bucketizer should throw exception if single- and multi-column params are both set

[ https://issues.apache.org/jira/browse/SPARK-22799?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Pentreath updated SPARK-22799: --- Target Version/s: 2.3.0 (was: 2.4.0) > Bucketizer should throw exception if single- and

[jira] [Updated] (SPARK-23106) ML, Graph 2.3 QA: API: Binary incompatible changes

[ https://issues.apache.org/jira/browse/SPARK-23106?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Pentreath updated SPARK-23106: --- Affects Version/s: 2.3.0 Target Version/s: 2.3.0 > ML, Graph 2.3 QA: API: Binary

[jira] [Commented] (SPARK-23106) ML, Graph 2.3 QA: API: Binary incompatible changes

[ https://issues.apache.org/jira/browse/SPARK-23106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16340645#comment-16340645 ] Nick Pentreath commented on SPARK-23106: Thanks [~bago.amirbekian]. However, running MiMa is not

[jira] [Updated] (SPARK-23109) ML 2.3 QA: API: Python API coverage

[ https://issues.apache.org/jira/browse/SPARK-23109?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Pentreath updated SPARK-23109: --- Affects Version/s: 2.3.0 Target Version/s: 2.3.0 > ML 2.3 QA: API: Python API coverage

[jira] [Commented] (SPARK-23200) Reset configuration when restarting from checkpoints

[ https://issues.apache.org/jira/browse/SPARK-23200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16340648#comment-16340648 ] Saisai Shao commented on SPARK-23200: - Issue resolved by pull request 20383

[jira] [Assigned] (SPARK-23163) Sync Python ML API docs with Scala

[ https://issues.apache.org/jira/browse/SPARK-23163?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Pentreath reassigned SPARK-23163: -- Assignee: Bryan Cutler > Sync Python ML API docs with Scala >

[jira] [Resolved] (SPARK-23163) Sync Python ML API docs with Scala

[ https://issues.apache.org/jira/browse/SPARK-23163?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Pentreath resolved SPARK-23163. Resolution: Fixed Fix Version/s: 2.3.0 > Sync Python ML API docs with Scala >

[jira] [Resolved] (SPARK-23200) Reset configuration when restarting from checkpoints

[ https://issues.apache.org/jira/browse/SPARK-23200?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao resolved SPARK-23200. - Resolution: Fixed Fix Version/s: 2.4.0 > Reset configuration when restarting from

[jira] [Updated] (SPARK-22799) Bucketizer should throw exception if single- and multi-column params are both set

[ https://issues.apache.org/jira/browse/SPARK-22799?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Pentreath updated SPARK-22799: --- Priority: Blocker (was: Major) > Bucketizer should throw exception if single- and

[jira] [Created] (SPARK-23224) union all will throw gramma exception

chenyukang created SPARK-23224: -- Summary: union all will throw gramma exception Key: SPARK-23224 URL: https://issues.apache.org/jira/browse/SPARK-23224 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-21601) Modify the JDK version of the Maven compilation

[ https://issues.apache.org/jira/browse/SPARK-21601?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16340568#comment-16340568 ] jifei_yang commented on SPARK-21601: Thanks. > Modify the JDK version of the Maven compilation >

[jira] [Commented] (SPARK-4502) Spark SQL reads unneccesary nested fields from Parquet

[ https://issues.apache.org/jira/browse/SPARK-4502?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16340544#comment-16340544 ] Yin Huai commented on SPARK-4502: - I think it makes sense to target for 2.4.0. 2.3.1 is a maintenance

[jira] [Commented] (SPARK-23206) Additional Memory Tuning Metrics

[ https://issues.apache.org/jira/browse/SPARK-23206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16340536#comment-16340536 ] Saisai Shao commented on SPARK-23206: - Would you please summarize what kind of metrics you want to

[jira] [Resolved] (SPARK-23187) Accumulator object can not be sent from Executor to Driver

[ https://issues.apache.org/jira/browse/SPARK-23187?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao resolved SPARK-23187. - Resolution: Not A Problem > Accumulator object can not be sent from Executor to Driver >

[jira] [Commented] (SPARK-23187) Accumulator object can not be sent from Executor to Driver

[ https://issues.apache.org/jira/browse/SPARK-23187?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16340522#comment-16340522 ] Saisai Shao commented on SPARK-23187: - I'm going to close this JIRA, as there's no issue in reporting

[jira] [Assigned] (SPARK-23223) Stacking dataset transforms performs poorly

[ https://issues.apache.org/jira/browse/SPARK-23223?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23223: Assignee: Apache Spark (was: Herman van Hovell) > Stacking dataset transforms performs

[jira] [Assigned] (SPARK-23223) Stacking dataset transforms performs poorly

[ https://issues.apache.org/jira/browse/SPARK-23223?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23223: Assignee: Herman van Hovell (was: Apache Spark) > Stacking dataset transforms performs

[jira] [Commented] (SPARK-23187) Accumulator object can not be sent from Executor to Driver

[ https://issues.apache.org/jira/browse/SPARK-23187?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16340521#comment-16340521 ] Saisai Shao commented on SPARK-23187: - I'm going to close this JIRA, as there's no issue in reporting

[jira] [Commented] (SPARK-23223) Stacking dataset transforms performs poorly

[ https://issues.apache.org/jira/browse/SPARK-23223?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16340520#comment-16340520 ] Apache Spark commented on SPARK-23223: -- User 'hvanhovell' has created a pull request for this issue:

[jira] [Created] (SPARK-23223) Stacking dataset transforms performs poorly

Herman van Hovell created SPARK-23223: - Summary: Stacking dataset transforms performs poorly Key: SPARK-23223 URL: https://issues.apache.org/jira/browse/SPARK-23223 Project: Spark Issue

[jira] [Commented] (SPARK-4502) Spark SQL reads unneccesary nested fields from Parquet

[ https://issues.apache.org/jira/browse/SPARK-4502?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16340515#comment-16340515 ] Simeon Simeonov commented on SPARK-4502: +1 [~holdenk] this should be a big boost for any Spark

[jira] [Commented] (SPARK-23220) broadcast hint not applied in a streaming left anti join

[ https://issues.apache.org/jira/browse/SPARK-23220?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16340495#comment-16340495 ] Liang-Chi Hsieh commented on SPARK-23220: - I can't re-produce it locally. I join a stream with a

[jira] [Commented] (SPARK-4502) Spark SQL reads unneccesary nested fields from Parquet

[ https://issues.apache.org/jira/browse/SPARK-4502?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16340488#comment-16340488 ] holdenk commented on SPARK-4502: [~sameerag] understand this is a pretty big change to try and get in at

[jira] [Commented] (SPARK-23206) Additional Memory Tuning Metrics

[ https://issues.apache.org/jira/browse/SPARK-23206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16340474#comment-16340474 ] Edwina Lu commented on SPARK-23206: --- [~jerryshao], yes  SPARK-9103 is similar, and this proposal

[jira] [Comment Edited] (SPARK-9103) Tracking spark's memory usage

[ https://issues.apache.org/jira/browse/SPARK-9103?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16340449#comment-16340449 ] Edwina Lu edited comment on SPARK-9103 at 1/26/18 2:08 AM: --- We (at LinkedIn) are

[jira] [Commented] (SPARK-9103) Tracking spark's memory usage

[ https://issues.apache.org/jira/browse/SPARK-9103?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16340449#comment-16340449 ] Edwina Lu commented on SPARK-9103: -- We (at LinkedIn) are interested in gathering more memory metrics as

[jira] [Commented] (SPARK-23105) Spark MLlib, GraphX 2.3 QA umbrella

[ https://issues.apache.org/jira/browse/SPARK-23105?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16340448#comment-16340448 ] Bago Amirbekian commented on SPARK-23105: - [~mlnick] We can update the sub tasks to target 2.3 if

[jira] [Commented] (SPARK-23157) withColumn fails for a column that is a result of mapped DataSet

[ https://issues.apache.org/jira/browse/SPARK-23157?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16340412#comment-16340412 ] Henry Robinson commented on SPARK-23157: I'm not sure if this should actually be expected to

[jira] [Commented] (SPARK-23109) ML 2.3 QA: API: Python API coverage

[ https://issues.apache.org/jira/browse/SPARK-23109?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16340399#comment-16340399 ] Bago Amirbekian commented on SPARK-23109: - [~bryanc] One reason the python API might be different

[jira] [Commented] (SPARK-22809) pyspark is sensitive to imports with dots

[ https://issues.apache.org/jira/browse/SPARK-22809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16340344#comment-16340344 ] Sean Owen commented on SPARK-22809: --- Might duplicate SPARK-23159 but I wasn't sure. > pyspark is

[jira] [Assigned] (SPARK-23032) Add a per-query codegenStageId to WholeStageCodegenExec

[ https://issues.apache.org/jira/browse/SPARK-23032?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li reassigned SPARK-23032: --- Assignee: Kris Mok > Add a per-query codegenStageId to WholeStageCodegenExec >

[jira] [Resolved] (SPARK-23032) Add a per-query codegenStageId to WholeStageCodegenExec

[ https://issues.apache.org/jira/browse/SPARK-23032?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-23032. - Resolution: Fixed Fix Version/s: 2.3.0 > Add a per-query codegenStageId to WholeStageCodegenExec

[jira] [Assigned] (SPARK-23205) ImageSchema.readImages incorrectly sets alpha channel to 255 for four-channel images

[ https://issues.apache.org/jira/browse/SPARK-23205?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen reassigned SPARK-23205: - Assignee: Siddharth Murching > ImageSchema.readImages incorrectly sets alpha channel to 255 for

[jira] [Resolved] (SPARK-23205) ImageSchema.readImages incorrectly sets alpha channel to 255 for four-channel images

[ https://issues.apache.org/jira/browse/SPARK-23205?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-23205. --- Resolution: Fixed Fix Version/s: 2.3.0 Issue resolved by pull request 20389

[jira] [Created] (SPARK-23222) Flaky test: DataFrameRangeSuite

Marcelo Vanzin created SPARK-23222: -- Summary: Flaky test: DataFrameRangeSuite Key: SPARK-23222 URL: https://issues.apache.org/jira/browse/SPARK-23222 Project: Spark Issue Type: Bug

[jira] [Resolved] (SPARK-23081) Add colRegex API to PySpark

[ https://issues.apache.org/jira/browse/SPARK-23081?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-23081. -- Resolution: Fixed Fix Version/s: 2.3.0 Fixed in 

[jira] [Assigned] (SPARK-23081) Add colRegex API to PySpark

[ https://issues.apache.org/jira/browse/SPARK-23081?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-23081: Assignee: Huaxin Gao > Add colRegex API to PySpark > --- > >

[jira] [Comment Edited] (SPARK-23106) ML, Graph 2.3 QA: API: Binary incompatible changes

[ https://issues.apache.org/jira/browse/SPARK-23106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16340234#comment-16340234 ] Bago Amirbekian edited comment on SPARK-23106 at 1/25/18 10:49 PM: --- I

[jira] [Resolved] (SPARK-23106) ML, Graph 2.3 QA: API: Binary incompatible changes

[ https://issues.apache.org/jira/browse/SPARK-23106?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bago Amirbekian resolved SPARK-23106. - Resolution: Resolved > ML, Graph 2.3 QA: API: Binary incompatible changes >

[jira] [Comment Edited] (SPARK-23106) ML, Graph 2.3 QA: API: Binary incompatible changes

[ https://issues.apache.org/jira/browse/SPARK-23106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16340234#comment-16340234 ] Bago Amirbekian edited comment on SPARK-23106 at 1/25/18 10:49 PM: --- I

[jira] [Comment Edited] (SPARK-23106) ML, Graph 2.3 QA: API: Binary incompatible changes

[ https://issues.apache.org/jira/browse/SPARK-23106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16340234#comment-16340234 ] Bago Amirbekian edited comment on SPARK-23106 at 1/25/18 10:49 PM: --- I

[jira] [Commented] (SPARK-23106) ML, Graph 2.3 QA: API: Binary incompatible changes

[ https://issues.apache.org/jira/browse/SPARK-23106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16340234#comment-16340234 ] Bago Amirbekian commented on SPARK-23106: - I ran mina in branch-2.3 and got the following output:

[jira] [Commented] (SPARK-23084) Add unboundedPreceding(), unboundedFollowing() and currentRow() to PySpark

[ https://issues.apache.org/jira/browse/SPARK-23084?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16340163#comment-16340163 ] Apache Spark commented on SPARK-23084: -- User 'huaxingao' has created a pull request for this issue:

[jira] [Assigned] (SPARK-23084) Add unboundedPreceding(), unboundedFollowing() and currentRow() to PySpark

[ https://issues.apache.org/jira/browse/SPARK-23084?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23084: Assignee: (was: Apache Spark) > Add unboundedPreceding(), unboundedFollowing() and

[jira] [Assigned] (SPARK-23084) Add unboundedPreceding(), unboundedFollowing() and currentRow() to PySpark

[ https://issues.apache.org/jira/browse/SPARK-23084?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23084: Assignee: Apache Spark > Add unboundedPreceding(), unboundedFollowing() and currentRow()

[jira] [Assigned] (SPARK-23209) HiveDelegationTokenProvider throws an exception if Hive jars are not the classpath

[ https://issues.apache.org/jira/browse/SPARK-23209?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23209: Assignee: Apache Spark > HiveDelegationTokenProvider throws an exception if Hive jars are

[jira] [Commented] (SPARK-23209) HiveDelegationTokenProvider throws an exception if Hive jars are not the classpath

[ https://issues.apache.org/jira/browse/SPARK-23209?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16340111#comment-16340111 ] Apache Spark commented on SPARK-23209: -- User 'vanzin' has created a pull request for this issue:

[jira] [Assigned] (SPARK-23209) HiveDelegationTokenProvider throws an exception if Hive jars are not the classpath

[ https://issues.apache.org/jira/browse/SPARK-23209?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23209: Assignee: (was: Apache Spark) > HiveDelegationTokenProvider throws an exception if

[jira] [Commented] (SPARK-18165) Kinesis support in Structured Streaming

[ https://issues.apache.org/jira/browse/SPARK-18165?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16339812#comment-16339812 ] Swaranga Sarma commented on SPARK-18165: Any updates? > Kinesis support in Structured Streaming

[jira] [Commented] (SPARK-23221) Fix KafkaContinuousSourceStressForDontFailOnDataLossSuite to run with enough cores

[ https://issues.apache.org/jira/browse/SPARK-23221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16339810#comment-16339810 ] Apache Spark commented on SPARK-23221: -- User 'jose-torres' has created a pull request for this

[jira] [Assigned] (SPARK-23221) Fix KafkaContinuousSourceStressForDontFailOnDataLossSuite to run with enough cores

[ https://issues.apache.org/jira/browse/SPARK-23221?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23221: Assignee: Apache Spark > Fix KafkaContinuousSourceStressForDontFailOnDataLossSuite to run

[jira] [Assigned] (SPARK-23221) Fix KafkaContinuousSourceStressForDontFailOnDataLossSuite to run with enough cores

[ https://issues.apache.org/jira/browse/SPARK-23221?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23221: Assignee: (was: Apache Spark) > Fix

[jira] [Updated] (SPARK-23221) Fix KafkaContinuousSourceStressForDontFailOnDataLossSuite to run with enough cores

[ https://issues.apache.org/jira/browse/SPARK-23221?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jose Torres updated SPARK-23221: Description: Currently, `KafkaContinuousSourceStressForDontFailOnDataLossSuite` runs on only 2

[jira] [Created] (SPARK-23221) Fix KafkaContinuousSourceStressForDontFailOnDataLossSuite to run with enough cores

Jose Torres created SPARK-23221: --- Summary: Fix KafkaContinuousSourceStressForDontFailOnDataLossSuite to run with enough cores Key: SPARK-23221 URL: https://issues.apache.org/jira/browse/SPARK-23221

[jira] [Commented] (SPARK-23173) from_json can produce nulls for fields which are marked as non-nullable

[ https://issues.apache.org/jira/browse/SPARK-23173?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16339794#comment-16339794 ] Reynold Xin commented on SPARK-23173: - Yea I agree with you Herman. On Sun, Jan 21, 2018 at 5:44 PM

[jira] [Commented] (SPARK-23213) SparkR:::textFile(sc1,"/opt/test333") can not work on spark2.2.1

[ https://issues.apache.org/jira/browse/SPARK-23213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16339748#comment-16339748 ] Felix Cheung commented on SPARK-23213: -- To clarify we don’t support RDD in R. Anything you access

[jira] [Commented] (SPARK-23201) Cannot create view when duplicate columns exist in subquery

[ https://issues.apache.org/jira/browse/SPARK-23201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16339687#comment-16339687 ] Herman van Hovell commented on SPARK-23201: --- +1 on closing. > Cannot create view when

[jira] [Commented] (SPARK-23198) Fix KafkaContinuousSourceStressForDontFailOnDataLossSuite to test ContinuousExecution

[ https://issues.apache.org/jira/browse/SPARK-23198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16339664#comment-16339664 ] Apache Spark commented on SPARK-23198: -- User 'jose-torres' has created a pull request for this

[jira] [Commented] (SPARK-23201) Cannot create view when duplicate columns exist in subquery

[ https://issues.apache.org/jira/browse/SPARK-23201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16339617#comment-16339617 ] Dongjoon Hyun commented on SPARK-23201: --- +1 for [~hyukjin.kwon]. > Cannot create view when

[jira] [Commented] (SPARK-23213) SparkR:::textFile(sc1,"/opt/test333") can not work on spark2.2.1

[ https://issues.apache.org/jira/browse/SPARK-23213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16339603#comment-16339603 ] Tony commented on SPARK-23213: --- [~felixcheung]  The only way to using method in  *RDD.R*  *is*  called by 

[jira] [Commented] (SPARK-23215) Dataset Grouping: Index out of bounds error

[ https://issues.apache.org/jira/browse/SPARK-23215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16339595#comment-16339595 ] Nikhil commented on SPARK-23215: [~srowen] I have attached the complete stack trace(it was incomplete

[jira] [Updated] (SPARK-23215) Dataset Grouping: Index out of bounds error

[ https://issues.apache.org/jira/browse/SPARK-23215?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nikhil updated SPARK-23215: --- Description: Peforming groupByKey operation followed by reduceGroups on dataset results in

[jira] [Updated] (SPARK-23215) Dataset Grouping: Index out of bounds error

[ https://issues.apache.org/jira/browse/SPARK-23215?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nikhil updated SPARK-23215: --- Description: Peforming groupByKey operation followed by reduceGroups on dataset results in

[jira] [Updated] (SPARK-23215) Dataset Grouping: Index out of bounds error

[ https://issues.apache.org/jira/browse/SPARK-23215?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nikhil updated SPARK-23215: --- Description: Peforming groupByKey operation followed by reduceGroups on dataset results in

[jira] [Updated] (SPARK-23215) Dataset Grouping: Index out of bounds error

[ https://issues.apache.org/jira/browse/SPARK-23215?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nikhil updated SPARK-23215: --- Description: Peforming groupByKey operation followed by reduceGroups on dataset results in

[jira] [Updated] (SPARK-23215) Dataset Grouping: Index out of bounds error

[ https://issues.apache.org/jira/browse/SPARK-23215?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nikhil updated SPARK-23215: --- Attachment: spark_issue.csv > Dataset Grouping: Index out of bounds error >

[jira] [Updated] (SPARK-23215) Dataset Grouping: Index out of bounds error

[ https://issues.apache.org/jira/browse/SPARK-23215?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nikhil updated SPARK-23215: --- Attachment: SparkIndexOutOfBoundsIssue.java > Dataset Grouping: Index out of bounds error >

[jira] [Updated] (SPARK-23215) Dataset Grouping: Index out of bounds error

[ https://issues.apache.org/jira/browse/SPARK-23215?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nikhil updated SPARK-23215: --- Attachment: stack_trace.txt > Dataset Grouping: Index out of bounds error >

[jira] [Updated] (SPARK-23215) Dataset Grouping: Index out of bounds error

[ https://issues.apache.org/jira/browse/SPARK-23215?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nikhil updated SPARK-23215: --- Description: (was: Peforming groupByKey operation followed by reduceGroups on dataset results in

[jira] [Closed] (SPARK-23213) SparkR:::textFile(sc1,"/opt/test333") can not work on spark2.2.1

[ https://issues.apache.org/jira/browse/SPARK-23213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen closed SPARK-23213. - > SparkR:::textFile(sc1,"/opt/test333") can not work on spark2.2.1 >

[jira] [Resolved] (SPARK-23213) SparkR:::textFile(sc1,"/opt/test333") can not work on spark2.2.1

[ https://issues.apache.org/jira/browse/SPARK-23213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-23213. --- Resolution: Not A Problem > SparkR:::textFile(sc1,"/opt/test333") can not work on spark2.2.1 >

[jira] [Commented] (SPARK-23213) SparkR:::textFile(sc1,"/opt/test333") can not work on spark2.2.1

[ https://issues.apache.org/jira/browse/SPARK-23213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16339559#comment-16339559 ] Felix Cheung commented on SPARK-23213: -- If you have any specific on what you need - we should have

[jira] [Commented] (SPARK-23213) SparkR:::textFile(sc1,"/opt/test333") can not work on spark2.2.1

[ https://issues.apache.org/jira/browse/SPARK-23213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16339551#comment-16339551 ] Tony commented on SPARK-23213: --- [~felixcheung] Thanks for you reply. Some more questions need your further

[jira] [Commented] (SPARK-23215) Dataset Grouping: Index out of bounds error

[ https://issues.apache.org/jira/browse/SPARK-23215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16339533#comment-16339533 ] Sean Owen commented on SPARK-23215: --- It doesn't seem to get past count(), so this is likely a mismatch

[jira] [Updated] (SPARK-23209) HiveDelegationTokenProvider throws an exception if Hive jars are not the classpath

[ https://issues.apache.org/jira/browse/SPARK-23209?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin updated SPARK-23209: --- Target Version/s: 2.3.0 Priority: Blocker (was: Major) >

[jira] [Commented] (SPARK-23213) SparkR:::textFile(sc1,"/opt/test333") can not work on spark2.2.1

[ https://issues.apache.org/jira/browse/SPARK-23213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16339522#comment-16339522 ] Felix Cheung commented on SPARK-23213: -- You can convert DataFrame into RDD But again textFile and

[jira] [Commented] (SPARK-23189) reflect stage level blacklisting on executor tab

[ https://issues.apache.org/jira/browse/SPARK-23189?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16339510#comment-16339510 ] Attila Zsolt Piros commented on SPARK-23189: I have introduced a new attribute for live

[jira] [Commented] (SPARK-23219) Rename ReadTask to DataReaderFactory

[ https://issues.apache.org/jira/browse/SPARK-23219?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16339496#comment-16339496 ] Apache Spark commented on SPARK-23219: -- User 'gengliangwang' has created a pull request for this

[jira] [Assigned] (SPARK-23219) Rename ReadTask to DataReaderFactory

[ https://issues.apache.org/jira/browse/SPARK-23219?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23219: Assignee: Apache Spark > Rename ReadTask to DataReaderFactory >

[jira] [Assigned] (SPARK-23219) Rename ReadTask to DataReaderFactory

[ https://issues.apache.org/jira/browse/SPARK-23219?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23219: Assignee: (was: Apache Spark) > Rename ReadTask to DataReaderFactory >

[jira] [Updated] (SPARK-23220) broadcast hint not applied in a streaming left anti join

[ https://issues.apache.org/jira/browse/SPARK-23220?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mathieu DESPRIEE updated SPARK-23220: - Description: We have a structured streaming app doing a left anti-join between a stream,

[jira] [Updated] (SPARK-23189) reflect stage level blacklisting on executor tab

[ https://issues.apache.org/jira/browse/SPARK-23189?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Attila Zsolt Piros updated SPARK-23189: --- Attachment: multiple_stages.png > reflect stage level blacklisting on executor tab

[jira] [Reopened] (SPARK-23213) SparkR:::textFile(sc1,"/opt/test333") can not work on spark2.2.1

[ https://issues.apache.org/jira/browse/SPARK-23213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tony reopened SPARK-23213: --- Repen , > SparkR:::textFile(sc1,"/opt/test333") can not work on spark2.2.1 >

[jira] [Commented] (SPARK-23213) SparkR:::textFile(sc1,"/opt/test333") can not work on spark2.2.1

[ https://issues.apache.org/jira/browse/SPARK-23213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16339466#comment-16339466 ] Tony commented on SPARK-23213: --- [~felixcheung] Please  help to have a look. >

[jira] [Updated] (SPARK-23220) broadcast hint not applied in a streaming left anti join

[ https://issues.apache.org/jira/browse/SPARK-23220?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mathieu DESPRIEE updated SPARK-23220: - Description: We have a structured streaming app doing a left anti-join between a stream,

[jira] [Updated] (SPARK-23220) broadcast hint not applied in a streaming left anti join

[ https://issues.apache.org/jira/browse/SPARK-23220?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mathieu DESPRIEE updated SPARK-23220: - Attachment: Screenshot from 2018-01-25 17-32-45.png > broadcast hint not applied in a

[jira] [Created] (SPARK-23220) broadcast hint not applied in a streaming left anti join

Mathieu DESPRIEE created SPARK-23220: Summary: broadcast hint not applied in a streaming left anti join Key: SPARK-23220 URL: https://issues.apache.org/jira/browse/SPARK-23220 Project: Spark

[jira] [Commented] (SPARK-23213) SparkR:::textFile(sc1,"/opt/test333") can not work on spark2.2.1

[ https://issues.apache.org/jira/browse/SPARK-23213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16339461#comment-16339461 ] Tony commented on SPARK-23213: --- Hi,*[~hyukjin.kwon]  ,*I  tried read.text before ,but read.text  will

[jira] [Commented] (SPARK-23209) HiveDelegationTokenProvider throws an exception if Hive jars are not the classpath

[ https://issues.apache.org/jira/browse/SPARK-23209?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16339454#comment-16339454 ] Sahil Takiar commented on SPARK-23209: -- Setting {{spark.security.credentials.hive.enabled}} doesn't

[jira] [Created] (SPARK-23219) Rename ReadTask to DataReaderFactory

Gengliang Wang created SPARK-23219: -- Summary: Rename ReadTask to DataReaderFactory Key: SPARK-23219 URL: https://issues.apache.org/jira/browse/SPARK-23219 Project: Spark Issue Type:

[jira] [Commented] (SPARK-23189) reflect stage level blacklisting on executor tab

[ https://issues.apache.org/jira/browse/SPARK-23189?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16339442#comment-16339442 ] Imran Rashid commented on SPARK-23189: -- looks reasonable to me. How do you differentiate

[jira] [Comment Edited] (SPARK-23189) reflect stage level blacklisting on executor tab

[ https://issues.apache.org/jira/browse/SPARK-23189?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16339389#comment-16339389 ] Attila Zsolt Piros edited comment on SPARK-23189 at 1/25/18 3:38 PM: -

[jira] [Commented] (SPARK-23189) reflect stage level blacklisting on executor tab

[ https://issues.apache.org/jira/browse/SPARK-23189?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16339389#comment-16339389 ] Attila Zsolt Piros commented on SPARK-23189: I have uploaded some images about the very first

[jira] [Updated] (SPARK-23189) reflect stage level blacklisting on executor tab

[ https://issues.apache.org/jira/browse/SPARK-23189?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Attila Zsolt Piros updated SPARK-23189: --- Attachment: afterStageCompleted.png backlisted.png > reflect stage

[jira] [Resolved] (SPARK-23213) SparkR:::textFile(sc1,"/opt/test333") can not work on spark2.2.1

[ https://issues.apache.org/jira/browse/SPARK-23213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-23213. --- Resolution: Not A Problem > SparkR:::textFile(sc1,"/opt/test333") can not work on spark2.2.1 >

[jira] [Assigned] (SPARK-23217) Add cosine distance measure to ClusteringEvaluator

[ https://issues.apache.org/jira/browse/SPARK-23217?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23217: Assignee: (was: Apache Spark) > Add cosine distance measure to ClusteringEvaluator >

[jira] [Commented] (SPARK-23217) Add cosine distance measure to ClusteringEvaluator

[ https://issues.apache.org/jira/browse/SPARK-23217?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16339336#comment-16339336 ] Apache Spark commented on SPARK-23217: -- User 'mgaido91' has created a pull request for this issue:

[jira] [Assigned] (SPARK-23217) Add cosine distance measure to ClusteringEvaluator

[ https://issues.apache.org/jira/browse/SPARK-23217?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23217: Assignee: Apache Spark > Add cosine distance measure to ClusteringEvaluator >

  1   2   >