[jira] [Updated] (SPARK-21828) org.apache.spark.sql.catalyst.expressions.GeneratedClass$SpecificOrdering" grows beyond 64 KB...again

[ https://issues.apache.org/jira/browse/SPARK-21828?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takeshi Yamamuro updated SPARK-21828: - Flags: (was: Important) >

[jira] [Updated] (SPARK-21828) org.apache.spark.sql.catalyst.expressions.GeneratedClass$SpecificOrdering" grows beyond 64 KB...again

[ https://issues.apache.org/jira/browse/SPARK-21828?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takeshi Yamamuro updated SPARK-21828: - Component/s: (was: ML) >

[jira] [Commented] (SPARK-21828) org.apache.spark.sql.catalyst.expressions.GeneratedClass$SpecificOrdering" grows beyond 64 KB...again

[ https://issues.apache.org/jira/browse/SPARK-21828?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16141234#comment-16141234 ] Takeshi Yamamuro commented on SPARK-21828: -- You can't set target version or something and these

[jira] [Commented] (SPARK-21835) RewritePredicateSubquery should not produce unresolved query plans

[ https://issues.apache.org/jira/browse/SPARK-21835?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16141233#comment-16141233 ] Liang-Chi Hsieh commented on SPARK-21835: - Submitted PR at

[jira] [Updated] (SPARK-21828) org.apache.spark.sql.catalyst.expressions.GeneratedClass$SpecificOrdering" grows beyond 64 KB...again

[ https://issues.apache.org/jira/browse/SPARK-21828?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takeshi Yamamuro updated SPARK-21828: - Target Version/s: (was: 2.1.0, 2.2.0) >

[jira] [Created] (SPARK-21835) RewritePredicateSubquery should not produce unresolved query plans

Liang-Chi Hsieh created SPARK-21835: --- Summary: RewritePredicateSubquery should not produce unresolved query plans Key: SPARK-21835 URL: https://issues.apache.org/jira/browse/SPARK-21835 Project:

[jira] [Comment Edited] (SPARK-21828) org.apache.spark.sql.catalyst.expressions.GeneratedClass$SpecificOrdering" grows beyond 64 KB...again

[ https://issues.apache.org/jira/browse/SPARK-21828?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16141138#comment-16141138 ] Otis Smart edited comment on SPARK-21828 at 8/25/17 4:19 AM: - Hi KI: I thank

[jira] [Commented] (SPARK-21828) org.apache.spark.sql.catalyst.expressions.GeneratedClass$SpecificOrdering" grows beyond 64 KB...again

[ https://issues.apache.org/jira/browse/SPARK-21828?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16141138#comment-16141138 ] Otis Smart commented on SPARK-21828: Hi KI: I thank you for the expedient reply! * Here (below text)

[jira] [Updated] (SPARK-21828) org.apache.spark.sql.catalyst.expressions.GeneratedClass$SpecificOrdering" grows beyond 64 KB...again

[ https://issues.apache.org/jira/browse/SPARK-21828?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Otis Smart updated SPARK-21828: --- Affects Version/s: (was: 2.2.0) 2.1.0 >

[jira] [Updated] (SPARK-21828) org.apache.spark.sql.catalyst.expressions.GeneratedClass$SpecificOrdering" grows beyond 64 KB...again

[ https://issues.apache.org/jira/browse/SPARK-21828?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Otis Smart updated SPARK-21828: --- Target Version/s: 2.2.0, 2.1.0 (was: 2.2.0) >

[jira] [Commented] (SPARK-18085) SPIP: Better History Server scalability for many / large applications

[ https://issues.apache.org/jira/browse/SPARK-18085?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16141091#comment-16141091 ] jincheng commented on SPARK-18085: -- it is really a good idea to speed up history server with levelDB. I

[jira] [Commented] (SPARK-21829) Enable config to permanently blacklist a list of nodes

[ https://issues.apache.org/jira/browse/SPARK-21829?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16141081#comment-16141081 ] Saisai Shao commented on SPARK-21829: - Cross post the comment here. Since you're running Spark on

[jira] [Comment Edited] (SPARK-21822) When insert Hive Table is finished, it is better to clean out the tmpLocation dir

[ https://issues.apache.org/jira/browse/SPARK-21822?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16141030#comment-16141030 ] lufei edited comment on SPARK-21822 at 8/25/17 1:26 AM: I'm so sorry for close

[jira] [Reopened] (SPARK-21822) When insert Hive Table is finished, it is better to clean out the tmpLocation dir

[ https://issues.apache.org/jira/browse/SPARK-21822?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] lufei reopened SPARK-21822: --- I close this issue by mistake. > When insert Hive Table is finished, it is better to clean out the tmpLocation

[jira] [Commented] (SPARK-21822) When insert Hive Table is finished, it is better to clean out the tmpLocation dir

[ https://issues.apache.org/jira/browse/SPARK-21822?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16141026#comment-16141026 ] lufei commented on SPARK-21822: --- [~sowen]I'm sorry,I didn't get your meaning before, I have already

[jira] [Issue Comment Deleted] (SPARK-21822) When insert Hive Table is finished, it is better to clean out the tmpLocation dir

[ https://issues.apache.org/jira/browse/SPARK-21822?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] lufei updated SPARK-21822: -- Comment: was deleted (was: [~sowen]Ok,I got it.I will close this issue immediately,thanks.) > When insert

[jira] [Commented] (SPARK-19571) tests are failing to run on Windows with another instance Derby error with Hadoop 2.6.5

[ https://issues.apache.org/jira/browse/SPARK-19571?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16141014#comment-16141014 ] Hyukjin Kwon commented on SPARK-19571: -- Sure, thanks! > tests are failing to run on Windows with

[jira] [Created] (SPARK-21834) Incorrect executor request in case of dynamic allocation

Sital Kedia created SPARK-21834: --- Summary: Incorrect executor request in case of dynamic allocation Key: SPARK-21834 URL: https://issues.apache.org/jira/browse/SPARK-21834 Project: Spark Issue

[jira] [Updated] (SPARK-19571) tests are failing to run on Windows with another instance Derby error with Hadoop 2.6.5

[ https://issues.apache.org/jira/browse/SPARK-19571?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-19571: - Fix Version/s: 2.2.0 > tests are failing to run on Windows with another instance Derby error

[jira] [Closed] (SPARK-21822) When insert Hive Table is finished, it is better to clean out the tmpLocation dir

[ https://issues.apache.org/jira/browse/SPARK-21822?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] lufei closed SPARK-21822. - Resolution: Invalid > When insert Hive Table is finished, it is better to clean out the tmpLocation > dir >

[jira] [Commented] (SPARK-21822) When insert Hive Table is finished, it is better to clean out the tmpLocation dir

[ https://issues.apache.org/jira/browse/SPARK-21822?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16140983#comment-16140983 ] lufei commented on SPARK-21822: --- [~sowen]Ok,I got it.I will close this issue immediately,thanks. > When

[jira] [Resolved] (SPARK-21830) Bump the dependency of ANTLR to version 4.7

[ https://issues.apache.org/jira/browse/SPARK-21830?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-21830. - Resolution: Fixed Fix Version/s: 2.3.0 > Bump the dependency of ANTLR to version 4.7 >

[jira] [Commented] (SPARK-21830) Bump the dependency of ANTLR to version 4.7

[ https://issues.apache.org/jira/browse/SPARK-21830?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16140939#comment-16140939 ] Xiao Li commented on SPARK-21830: - https://github.com/apache/spark/pull/19042 > Bump the dependency of

[jira] [Commented] (SPARK-21798) No config to replace deprecated SPARK_CLASSPATH config for launching daemons like History Server

[ https://issues.apache.org/jira/browse/SPARK-21798?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16140833#comment-16140833 ] Parth Gandhi commented on SPARK-21798: -- Filed a pull request for this ticket:

[jira] [Assigned] (SPARK-21701) Add TCP send/rcv buffer size support for RPC client

[ https://issues.apache.org/jira/browse/SPARK-21701?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu reassigned SPARK-21701: Assignee: Xu Zhang > Add TCP send/rcv buffer size support for RPC client >

[jira] [Resolved] (SPARK-21701) Add TCP send/rcv buffer size support for RPC client

[ https://issues.apache.org/jira/browse/SPARK-21701?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu resolved SPARK-21701. -- Resolution: Fixed Fix Version/s: 2.3.0 > Add TCP send/rcv buffer size support for RPC

[jira] [Commented] (SPARK-18355) Spark SQL fails to read data from a ORC hive table that has a new column added to it

[ https://issues.apache.org/jira/browse/SPARK-18355?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16140552#comment-16140552 ] Dongjoon Hyun commented on SPARK-18355: --- This will be resolved via Apache ORC 1.4.0. > Spark SQL

[jira] [Closed] (SPARK-21833) CoarseGrainedSchedulerBackend leaks executors in case of dynamic allocation

[ https://issues.apache.org/jira/browse/SPARK-21833?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sital Kedia closed SPARK-21833. --- Resolution: Duplicate Duplicate of SPARK-20540 > CoarseGrainedSchedulerBackend leaks executors in

[jira] [Updated] (SPARK-21833) CoarseGrainedSchedulerBackend leaks executors in case of dynamic allocation

[ https://issues.apache.org/jira/browse/SPARK-21833?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sital Kedia updated SPARK-21833: Description: We have seen this issue in coarse grained scheduler that in case of dynamic executor

[jira] [Commented] (SPARK-21833) CoarseGrainedSchedulerBackend leaks executors in case of dynamic allocation

[ https://issues.apache.org/jira/browse/SPARK-21833?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16140533#comment-16140533 ] Sital Kedia commented on SPARK-21833: - Actually, SPARK-20540 already addressed this issue on latest

[jira] [Updated] (SPARK-21833) CoarseGrainedSchedulerBackend leaks executors in case of dynamic allocation

[ https://issues.apache.org/jira/browse/SPARK-21833?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sital Kedia updated SPARK-21833: Description: We have seen this issue in coarse grained scheduler that in case of dynamic executor

[jira] [Updated] (SPARK-21833) CoarseGrainedSchedulerBackend leaks executors in case of dynamic allocation

[ https://issues.apache.org/jira/browse/SPARK-21833?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sital Kedia updated SPARK-21833: Description: We have seen this issue in coarse grained scheduler that in case of dynamic executor

[jira] [Created] (SPARK-21833) CoarseGrainedSchedulerBackend leaks executors in case of dynamic allocation

Sital Kedia created SPARK-21833: --- Summary: CoarseGrainedSchedulerBackend leaks executors in case of dynamic allocation Key: SPARK-21833 URL: https://issues.apache.org/jira/browse/SPARK-21833 Project:

[jira] [Created] (SPARK-21832) Merge SQLBuilderTest into ExpressionSQLBuilderSuite

Dongjoon Hyun created SPARK-21832: - Summary: Merge SQLBuilderTest into ExpressionSQLBuilderSuite Key: SPARK-21832 URL: https://issues.apache.org/jira/browse/SPARK-21832 Project: Spark Issue

[jira] [Updated] (SPARK-21788) Handle more exceptions when stopping a streaming query

[ https://issues.apache.org/jira/browse/SPARK-21788?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-21788: - Fix Version/s: (was: 3.0.0) 2.3.0 > Handle more exceptions when stopping

[jira] [Updated] (SPARK-21831) Remove `spark.sql.hive.convertMetastoreOrc` config in HiveCompatibilitySuite

[ https://issues.apache.org/jira/browse/SPARK-21831?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-21831: -- Summary: Remove `spark.sql.hive.convertMetastoreOrc` config in HiveCompatibilitySuite (was:

[jira] [Created] (SPARK-21831) Remove obsolete `spark.sql.hive.convertMetastoreOrc` config in HiveCompatibilitySuite

Dongjoon Hyun created SPARK-21831: - Summary: Remove obsolete `spark.sql.hive.convertMetastoreOrc` config in HiveCompatibilitySuite Key: SPARK-21831 URL: https://issues.apache.org/jira/browse/SPARK-21831

[jira] [Updated] (SPARK-21797) spark cannot read partitioned data in S3 that are partly in glacier

[ https://issues.apache.org/jira/browse/SPARK-21797?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated SPARK-21797: --- Environment: Amazon EMR > spark cannot read partitioned data in S3 that are partly in

[jira] [Comment Edited] (SPARK-21797) spark cannot read partitioned data in S3 that are partly in glacier

[ https://issues.apache.org/jira/browse/SPARK-21797?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16140408#comment-16140408 ] Steve Loughran edited comment on SPARK-21797 at 8/24/17 5:56 PM: - This is

[jira] [Commented] (SPARK-21797) spark cannot read partitioned data in S3 that are partly in glacier

[ https://issues.apache.org/jira/browse/SPARK-21797?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16140408#comment-16140408 ] Steve Loughran commented on SPARK-21797: This is happening deep the Amazon EMR team's closed

[jira] [Resolved] (SPARK-21788) Handle more exceptions when stopping a streaming query

[ https://issues.apache.org/jira/browse/SPARK-21788?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tathagata Das resolved SPARK-21788. --- Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 18997

[jira] [Updated] (SPARK-21681) MLOR do not work correctly when featureStd contains zero

[ https://issues.apache.org/jira/browse/SPARK-21681?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-21681: -- Labels: correctness (was: ) > MLOR do not work correctly when featureStd contains

[jira] [Resolved] (SPARK-21681) MLOR do not work correctly when featureStd contains zero

[ https://issues.apache.org/jira/browse/SPARK-21681?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley resolved SPARK-21681. --- Resolution: Fixed Fix Version/s: 2.2.1 Issue resolved by pull request 19026

[jira] [Commented] (SPARK-16742) Kerberos support for Spark on Mesos

[ https://issues.apache.org/jira/browse/SPARK-16742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16140293#comment-16140293 ] Marcelo Vanzin commented on SPARK-16742: Both renewal and creating new tickets after the TTL

[jira] [Commented] (SPARK-21828) org.apache.spark.sql.catalyst.expressions.GeneratedClass$SpecificOrdering" grows beyond 64 KB...again

[ https://issues.apache.org/jira/browse/SPARK-21828?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16140282#comment-16140282 ] Kazuaki Ishizaki commented on SPARK-21828: -- Thank you for reporting a problem. First, IIUC, this

[jira] [Commented] (SPARK-21797) spark cannot read partitioned data in S3 that are partly in glacier

[ https://issues.apache.org/jira/browse/SPARK-21797?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16140261#comment-16140261 ] Boris Clémençon commented on SPARK-21797: -- Steve, This is the stacks: {noformat} WARN

[jira] [Created] (SPARK-21830) Bump the dependency of ANTLR to version 4.7

Herman van Hovell created SPARK-21830: - Summary: Bump the dependency of ANTLR to version 4.7 Key: SPARK-21830 URL: https://issues.apache.org/jira/browse/SPARK-21830 Project: Spark Issue

[jira] [Commented] (SPARK-21799) KMeans performance regression (5-6x slowdown) in Spark 2.2

[ https://issues.apache.org/jira/browse/SPARK-21799?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16140183#comment-16140183 ] zakaria hili commented on SPARK-21799: -- [~WeichenXu123], df.rdd.getStorageLevel return none even if

[jira] [Updated] (SPARK-21829) Enable config to permanently blacklist a list of nodes

[ https://issues.apache.org/jira/browse/SPARK-21829?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Luca Canali updated SPARK-21829: Summary: Enable config to permanently blacklist a list of nodes (was: Prevent running

[jira] [Commented] (SPARK-21799) KMeans performance regression (5-6x slowdown) in Spark 2.2

[ https://issues.apache.org/jira/browse/SPARK-21799?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16140162#comment-16140162 ] Weichen Xu commented on SPARK-21799: I suggest check both `df.storageLevel` and

[jira] [Commented] (SPARK-21817) Pass FSPermissions to LocatedFileStatus from InMemoryFileIndex

[ https://issues.apache.org/jira/browse/SPARK-21817?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16140161#comment-16140161 ] Steve Loughran commented on SPARK-21817: FYI, this is now fixed in hadoop trunk/3.0-beta-1 >

[jira] [Commented] (SPARK-21797) spark cannot read partitioned data in S3 that are partly in glacier

[ https://issues.apache.org/jira/browse/SPARK-21797?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16140147#comment-16140147 ] Steve Loughran commented on SPARK-21797: Note that if it is just during spark partition

[jira] [Commented] (SPARK-21797) spark cannot read partitioned data in S3 that are partly in glacier

[ https://issues.apache.org/jira/browse/SPARK-21797?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16140138#comment-16140138 ] Steve Loughran commented on SPARK-21797: I was talking about the cost and time of getting data

[jira] [Resolved] (SPARK-21826) outer broadcast hash join should not throw NPE

[ https://issues.apache.org/jira/browse/SPARK-21826?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell resolved SPARK-21826. --- Resolution: Fixed Fix Version/s: 2.3.0 2.2.1 Fixed per

[jira] [Assigned] (SPARK-21826) outer broadcast hash join should not throw NPE

[ https://issues.apache.org/jira/browse/SPARK-21826?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell reassigned SPARK-21826: - Assignee: Wenchen Fan > outer broadcast hash join should not throw NPE >

[jira] [Commented] (SPARK-16742) Kerberos support for Spark on Mesos

[ https://issues.apache.org/jira/browse/SPARK-16742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16140111#comment-16140111 ] Arthur Rand commented on SPARK-16742: - Hello [~vanzin], I'm assuming you're talking about automatic

[jira] [Comment Edited] (SPARK-21190) SPIP: Vectorized UDFs in Python

[ https://issues.apache.org/jira/browse/SPARK-21190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16140094#comment-16140094 ] Li Jin edited comment on SPARK-21190 at 8/24/17 2:33 PM: - [~ueshin], Got it. I'd

[jira] [Comment Edited] (SPARK-21190) SPIP: Vectorized UDFs in Python

[ https://issues.apache.org/jira/browse/SPARK-21190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16140094#comment-16140094 ] Li Jin edited comment on SPARK-21190 at 8/24/17 2:33 PM: - [~ueshin], Got it. I'd

[jira] [Commented] (SPARK-21190) SPIP: Vectorized UDFs in Python

[ https://issues.apache.org/jira/browse/SPARK-21190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16140094#comment-16140094 ] Li Jin commented on SPARK-21190: [~ueshin], Got it. I'd actually prefer doing it this way: {code}

[jira] [Commented] (SPARK-21829) Prevent running executors/tasks on a user-specified list of cluster nodes

[ https://issues.apache.org/jira/browse/SPARK-21829?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16140065#comment-16140065 ] Luca Canali commented on SPARK-21829: - https://github.com/apache/spark/pull/19039 > Prevent running

[jira] [Reopened] (SPARK-21527) Use buffer limit in order to take advantage of JAVA NIO Util's buffercache

[ https://issues.apache.org/jira/browse/SPARK-21527?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhoukang reopened SPARK-21527: -- > Use buffer limit in order to take advantage of JAVA NIO Util's buffercache >

[jira] [Commented] (SPARK-21527) Use buffer limit in order to take advantage of JAVA NIO Util's buffercache

[ https://issues.apache.org/jira/browse/SPARK-21527?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16140062#comment-16140062 ] zhoukang commented on SPARK-21527: -- https://github.com/apache/spark/pull/18730 > Use buffer limit in

[jira] [Assigned] (SPARK-21759) In.checkInputDataTypes should not wrongly report unresolved plans for IN correlated subquery

[ https://issues.apache.org/jira/browse/SPARK-21759?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-21759: --- Assignee: Liang-Chi Hsieh > In.checkInputDataTypes should not wrongly report unresolved

[jira] [Resolved] (SPARK-21759) In.checkInputDataTypes should not wrongly report unresolved plans for IN correlated subquery

[ https://issues.apache.org/jira/browse/SPARK-21759?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-21759. - Resolution: Fixed Fix Version/s: 2.3.0 Issue resolved by pull request 18968

[jira] [Commented] (SPARK-21829) Prevent running executors/tasks on a user-specified list of cluster nodes

[ https://issues.apache.org/jira/browse/SPARK-21829?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16140041#comment-16140041 ] Luca Canali commented on SPARK-21829: - I think what I am addressing may be a rare case (but it

[jira] [Commented] (SPARK-21829) Prevent running executors/tasks on a user-specified list of cluster nodes

[ https://issues.apache.org/jira/browse/SPARK-21829?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16140030#comment-16140030 ] Sean Owen commented on SPARK-21829: --- Why not just take them out of the resource manager entirely? This

[jira] [Created] (SPARK-21829) Prevent running executors/tasks on a user-specified list of cluster nodes

Luca Canali created SPARK-21829: --- Summary: Prevent running executors/tasks on a user-specified list of cluster nodes Key: SPARK-21829 URL: https://issues.apache.org/jira/browse/SPARK-21829 Project:

[jira] [Resolved] (SPARK-21745) Refactor ColumnVector hierarchy to make ColumnVector read-only and to introduce WritableColumnVector.

[ https://issues.apache.org/jira/browse/SPARK-21745?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-21745. - Resolution: Fixed Fix Version/s: 2.3.0 Issue resolved by pull request 18958

[jira] [Updated] (SPARK-21822) When insert Hive Table is finished, it is better to clean out the tmpLocation dir

[ https://issues.apache.org/jira/browse/SPARK-21822?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-21822: -- Priority: Minor (was: Major) [~figo] although the exact meanings of priorities are a little

[jira] [Assigned] (SPARK-21745) Refactor ColumnVector hierarchy to make ColumnVector read-only and to introduce WritableColumnVector.

[ https://issues.apache.org/jira/browse/SPARK-21745?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-21745: --- Assignee: Takuya Ueshin > Refactor ColumnVector hierarchy to make ColumnVector read-only

[jira] [Updated] (SPARK-21828) org.apache.spark.sql.catalyst.expressions.GeneratedClass$SpecificOrdering" grows beyond 64 KB...again

[ https://issues.apache.org/jira/browse/SPARK-21828?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Otis Smart updated SPARK-21828: --- Shepherd: Kazuaki Ishizaki (was: Liwei Lin) >

[jira] [Updated] (SPARK-21828) org.apache.spark.sql.catalyst.expressions.GeneratedClass$SpecificOrdering" grows beyond 64 KB...again

[ https://issues.apache.org/jira/browse/SPARK-21828?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Otis Smart updated SPARK-21828: --- Component/s: SQL > org.apache.spark.sql.catalyst.expressions.GeneratedClass$SpecificOrdering" >

[jira] [Created] (SPARK-21828) org.apache.spark.sql.catalyst.expressions.GeneratedClass$SpecificOrdering" grows beyond 64 KB...again

Otis Smart created SPARK-21828: -- Summary: org.apache.spark.sql.catalyst.expressions.GeneratedClass$SpecificOrdering" grows beyond 64 KB...again Key: SPARK-21828 URL:

[jira] [Updated] (SPARK-21822) When insert Hive Table is finished, it is better to clean out the tmpLocation dir

[ https://issues.apache.org/jira/browse/SPARK-21822?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] lufei updated SPARK-21822: -- Priority: Major (was: Minor) > When insert Hive Table is finished, it is better to clean out the tmpLocation

[jira] [Commented] (SPARK-21797) spark cannot read partitioned data in S3 that are partly in glacier

[ https://issues.apache.org/jira/browse/SPARK-21797?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16139986#comment-16139986 ] Sean Owen commented on SPARK-21797: --- Sure, but in all events, this is an operation that is fine with

[jira] [Comment Edited] (SPARK-21797) spark cannot read partitioned data in S3 that are partly in glacier

[ https://issues.apache.org/jira/browse/SPARK-21797?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16139978#comment-16139978 ] Boris Clémençon edited comment on SPARK-21797 at 8/24/17 12:37 PM:

[jira] [Comment Edited] (SPARK-16845) org.apache.spark.sql.catalyst.expressions.GeneratedClass$SpecificOrdering" grows beyond 64 KB

[ https://issues.apache.org/jira/browse/SPARK-16845?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16139969#comment-16139969 ] Otis Smart edited comment on SPARK-16845 at 8/24/17 12:37 PM: -- Hello! 1. I

[jira] [Comment Edited] (SPARK-21797) spark cannot read partitioned data in S3 that are partly in glacier

[ https://issues.apache.org/jira/browse/SPARK-21797?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16139978#comment-16139978 ] Boris Clémençon edited comment on SPARK-21797 at 8/24/17 12:37 PM:

[jira] [Comment Edited] (SPARK-21797) spark cannot read partitioned data in S3 that are partly in glacier

[ https://issues.apache.org/jira/browse/SPARK-21797?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16139978#comment-16139978 ] Boris Clémençon edited comment on SPARK-21797 at 8/24/17 12:36 PM:

[jira] [Commented] (SPARK-21797) spark cannot read partitioned data in S3 that are partly in glacier

[ https://issues.apache.org/jira/browse/SPARK-21797?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16139978#comment-16139978 ] Boris Clémençon commented on SPARK-21797: -- Hi Steve, to be sure we understand each other, *I

[jira] [Comment Edited] (SPARK-16845) org.apache.spark.sql.catalyst.expressions.GeneratedClass$SpecificOrdering" grows beyond 64 KB

[ https://issues.apache.org/jira/browse/SPARK-16845?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16139969#comment-16139969 ] Otis Smart edited comment on SPARK-16845 at 8/24/17 12:31 PM: -- Hello! 1. I

[jira] [Commented] (SPARK-16845) org.apache.spark.sql.catalyst.expressions.GeneratedClass$SpecificOrdering" grows beyond 64 KB

[ https://issues.apache.org/jira/browse/SPARK-16845?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16139969#comment-16139969 ] Otis Smart commented on SPARK-16845: Hello! 1. I encounter a similar issue (see below text) on

[jira] [Commented] (SPARK-21825) improving "assert(exchanges.map(_.outputPartitioning.numPartitions)" in ExchangeCoordinatorSuite

[ https://issues.apache.org/jira/browse/SPARK-21825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16139957#comment-16139957 ] Sean Owen commented on SPARK-21825: --- I don't think this is even something we should merge. > improving

[jira] [Commented] (SPARK-21825) improving "assert(exchanges.map(_.outputPartitioning.numPartitions)" in ExchangeCoordinatorSuite

[ https://issues.apache.org/jira/browse/SPARK-21825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16139953#comment-16139953 ] Nikhil Bhide commented on SPARK-21825: -- Please assign this issue to me. > improving

[jira] [Commented] (SPARK-19123) KeyProviderException when reading Azure Blobs from Apache Spark

[ https://issues.apache.org/jira/browse/SPARK-19123?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16139950#comment-16139950 ] Davis commented on SPARK-19123: --- Please add the following config entry to the SparkSessionBuilder to skip

[jira] [Assigned] (SPARK-19159) PySpark UDF API improvements

[ https://issues.apache.org/jira/browse/SPARK-19159?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-19159: Assignee: Maciej Szymkiewicz > PySpark UDF API improvements >

[jira] [Commented] (SPARK-21797) spark cannot read partitioned data in S3 that are partly in glacier

[ https://issues.apache.org/jira/browse/SPARK-21797?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16139919#comment-16139919 ] Steve Loughran commented on SPARK-21797: If you are using S3// URLs then its the AWS team's

[jira] [Assigned] (SPARK-18777) Return UDF objects when registering from Python

[ https://issues.apache.org/jira/browse/SPARK-18777?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-18777: Assignee: Maciej Szymkiewicz > Return UDF objects when registering from Python >

[jira] [Resolved] (SPARK-18777) Return UDF objects when registering from Python

[ https://issues.apache.org/jira/browse/SPARK-18777?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-18777. -- Resolution: Fixed Fixed in https://github.com/apache/spark/pull/17831 > Return UDF objects

[jira] [Updated] (SPARK-19159) PySpark UDF API improvements

[ https://issues.apache.org/jira/browse/SPARK-19159?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-19159: - Fix Version/s: 2.3.0 > PySpark UDF API improvements > > >

[jira] [Resolved] (SPARK-19159) PySpark UDF API improvements

[ https://issues.apache.org/jira/browse/SPARK-19159?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-19159. -- Resolution: Done > PySpark UDF API improvements > > >

[jira] [Assigned] (SPARK-19165) UserDefinedFunction should verify call arguments and provide readable exception in case of mismatch

[ https://issues.apache.org/jira/browse/SPARK-19165?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-19165: Assignee: Hyukjin Kwon > UserDefinedFunction should verify call arguments and provide

[jira] [Resolved] (SPARK-19165) UserDefinedFunction should verify call arguments and provide readable exception in case of mismatch

[ https://issues.apache.org/jira/browse/SPARK-19165?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-19165. -- Resolution: Fixed Fix Version/s: 2.3.0 Issue resolved by pull request 19027

[jira] [Commented] (SPARK-21172) EOFException reached end of stream in UnsafeRowSerializer

[ https://issues.apache.org/jira/browse/SPARK-21172?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16139910#comment-16139910 ] liupengcheng commented on SPARK-21172: -- [~lasanthafdo] This may caused by packets bits error in

[jira] [Commented] (SPARK-21702) Structured Streaming S3A SSE Encryption Not Visible through AWS S3 GUI when PartitionBy Used

[ https://issues.apache.org/jira/browse/SPARK-21702?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16139898#comment-16139898 ] Steve Loughran commented on SPARK-21702: IF this is just "directories", then there are no

[jira] [Resolved] (SPARK-21702) Structured Streaming S3A SSE Encryption Not Visible through AWS S3 GUI when PartitionBy Used

[ https://issues.apache.org/jira/browse/SPARK-21702?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved SPARK-21702. Resolution: Invalid > Structured Streaming S3A SSE Encryption Not Visible through AWS S3

[jira] [Assigned] (SPARK-21804) json_tuple returns null values within repeated columns except the first one

[ https://issues.apache.org/jira/browse/SPARK-21804?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-21804: Assignee: Jen-Ming Chung > json_tuple returns null values within repeated columns except

[jira] [Updated] (SPARK-21825) improving "assert(exchanges.map(_.outputPartitioning.numPartitions)" in ExchangeCoordinatorSuite

[ https://issues.apache.org/jira/browse/SPARK-21825?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] iamhumanbeing updated SPARK-21825: -- Description: ExchangeCoordinatorSuite.scala Line 424:

[jira] [Resolved] (SPARK-21804) json_tuple returns null values within repeated columns except the first one

[ https://issues.apache.org/jira/browse/SPARK-21804?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-21804. -- Resolution: Fixed Fix Version/s: 2.3.0 Issue resolved by pull request 19017

[jira] [Commented] (SPARK-21495) DIGEST-MD5: Out of order sequencing of messages from server

[ https://issues.apache.org/jira/browse/SPARK-21495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16139861#comment-16139861 ] Sean Owen commented on SPARK-21495: --- Maybe, but it doesn't sound like it's established that it's not

  1   2   >