[jira] [Commented] (SPARK-26602) Insert into table fails after querying the UDF which is loaded with wrong hdfs path

2019-03-05 Thread Ajith S (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26602?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16784506#comment-16784506 ] Ajith S commented on SPARK-26602: - [~chakravarthi] Hi, thanks for reporting the issue, From your example

[jira] [Comment Edited] (SPARK-26602) Insert into table fails after querying the UDF which is loaded with wrong hdfs path

2019-03-05 Thread Chakravarthi (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26602?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16784134#comment-16784134 ] Chakravarthi edited comment on SPARK-26602 at 3/5/19 2:31 PM: -- Hi [~srowen]

[jira] [Commented] (SPARK-27060) DDL Commands are accepting Keywords like create, drop as tableName

2019-03-05 Thread Sujith Chacko (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27060?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16784503#comment-16784503 ] Sujith Chacko commented on SPARK-27060: --- This looks like a compatibility issue with other engines.

[jira] [Created] (SPARK-27060) DDL Commands are accepting Keywords like create, drop as tableName

2019-03-05 Thread Sachin Ramachandra Setty (JIRA)
Sachin Ramachandra Setty created SPARK-27060: Summary: DDL Commands are accepting Keywords like create, drop as tableName Key: SPARK-27060 URL: https://issues.apache.org/jira/browse/SPARK-27060

[jira] [Commented] (SPARK-23986) CompileException when using too many avg aggregation after joining

2019-03-05 Thread Pedro Fernandes (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23986?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16784482#comment-16784482 ] Pedro Fernandes commented on SPARK-23986: - Guys, Is there a workaround for the folks that can't 

[jira] [Commented] (SPARK-20415) SPARK job hangs while writing DataFrame to HDFS

2019-03-05 Thread Martin Studer (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20415?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16784475#comment-16784475 ] Martin Studer commented on SPARK-20415: --- In fact, I believe the underlying issue is actually

[jira] [Commented] (SPARK-26602) Insert into table fails after querying the UDF which is loaded with wrong hdfs path

2019-03-05 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26602?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16784446#comment-16784446 ] Sean Owen commented on SPARK-26602: --- That sounds like user error. I'd close this as NotAProblem. It

[jira] [Comment Edited] (SPARK-26602) Insert into table fails after querying the UDF which is loaded with wrong hdfs path

2019-03-05 Thread Chakravarthi (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26602?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16784134#comment-16784134 ] Chakravarthi edited comment on SPARK-26602 at 3/5/19 1:52 PM: -- Hi [~srowen]

[jira] [Updated] (SPARK-27059) spark-submit on kubernetes cluster does not recognise k8s --master property

2019-03-05 Thread Andreas Adamides (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27059?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andreas Adamides updated SPARK-27059: - Description: I have successfully installed a Kubernetes cluster and can verify this by:

[jira] [Assigned] (SPARK-26923) Refactor ArrowRRunner and RRunner to share the same base

2019-03-05 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26923?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26923: Assignee: (was: Apache Spark) > Refactor ArrowRRunner and RRunner to share the same

[jira] [Commented] (SPARK-26972) Issue with CSV import and inferSchema set to true

2019-03-05 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26972?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16784453#comment-16784453 ] Sean Owen commented on SPARK-26972: --- I haven't checked when it's case-insensitive, but to be clear:

[jira] [Commented] (SPARK-26998) spark.ssl.keyStorePassword in plaintext on 'ps -ef' output of executor processes in Standalone mode

2019-03-05 Thread Jungtaek Lim (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26998?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16784434#comment-16784434 ] Jungtaek Lim commented on SPARK-26998: -- If I understand correctly, the PR would mitigate the issue

[jira] [Assigned] (SPARK-26923) Refactor ArrowRRunner and RRunner to share the same base

2019-03-05 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26923?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26923: Assignee: Apache Spark > Refactor ArrowRRunner and RRunner to share the same base >

[jira] [Updated] (SPARK-27059) spark-submit on kubernetes cluster does not recognise k8s --master property

2019-03-05 Thread Andreas Adamides (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27059?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andreas Adamides updated SPARK-27059: - Description: I have successfully installed a Kubernetes cluster and can verify this by:

[jira] [Created] (SPARK-27059) spark-submit on kubernetes cluster does not recognise k8s --master property

2019-03-05 Thread Andreas Adamides (JIRA)
Andreas Adamides created SPARK-27059: Summary: spark-submit on kubernetes cluster does not recognise k8s --master property Key: SPARK-27059 URL: https://issues.apache.org/jira/browse/SPARK-27059

[jira] [Updated] (SPARK-27059) spark-submit on kubernetes cluster does not recognise k8s --master property

2019-03-05 Thread Andreas Adamides (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27059?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andreas Adamides updated SPARK-27059: - Description: I have successfully installed a Kubernetes cluster and can verify this by:

[jira] [Updated] (SPARK-27059) spark-submit on kubernetes cluster does not recognise k8s --master property

2019-03-05 Thread Andreas Adamides (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27059?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andreas Adamides updated SPARK-27059: - Description: I have successfully installed a Kubernetes cluster and can verify this by:

[jira] [Commented] (SPARK-26998) spark.ssl.keyStorePassword in plaintext on 'ps -ef' output of executor processes in Standalone mode

2019-03-05 Thread Gabor Somogyi (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26998?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16784410#comment-16784410 ] Gabor Somogyi commented on SPARK-26998: --- Ahaaa, I see now. 2 problems tried to be solved in one

[jira] [Updated] (SPARK-27059) spark-submit on kubernetes cluster does not recognise k8s --master property

2019-03-05 Thread Andreas Adamides (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27059?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andreas Adamides updated SPARK-27059: - Description: I have successfully installed a Kubernetes cluster and can verify this by:

[jira] [Commented] (SPARK-26998) spark.ssl.keyStorePassword in plaintext on 'ps -ef' output of executor processes in Standalone mode

2019-03-05 Thread Gabor Somogyi (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26998?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16784406#comment-16784406 ] Gabor Somogyi commented on SPARK-26998: --- {quote} Can be resolved if below PR is merged: [[Github]

[jira] [Commented] (SPARK-26998) spark.ssl.keyStorePassword in plaintext on 'ps -ef' output of executor processes in Standalone mode

2019-03-05 Thread Gabor Somogyi (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26998?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16784391#comment-16784391 ] Gabor Somogyi commented on SPARK-26998: --- [~toopt4] thanks for the info. Are you working on this?

[jira] [Commented] (SPARK-27058) Support mounting host dirs for K8s tests

2019-03-05 Thread Stavros Kontopoulos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27058?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16784396#comment-16784396 ] Stavros Kontopoulos commented on SPARK-27058: - [~shaneknapp] added this to keep track of

[jira] [Created] (SPARK-27058) Support mounting host dirs for K8s tests

2019-03-05 Thread Stavros Kontopoulos (JIRA)
Stavros Kontopoulos created SPARK-27058: --- Summary: Support mounting host dirs for K8s tests Key: SPARK-27058 URL: https://issues.apache.org/jira/browse/SPARK-27058 Project: Spark Issue

[jira] [Assigned] (SPARK-27057) Common trait for limit exec operators

2019-03-05 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27057?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27057: Assignee: Apache Spark > Common trait for limit exec operators >

[jira] [Assigned] (SPARK-27057) Common trait for limit exec operators

2019-03-05 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27057?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27057: Assignee: (was: Apache Spark) > Common trait for limit exec operators >

[jira] [Comment Edited] (SPARK-26998) spark.ssl.keyStorePassword in plaintext on 'ps -ef' output of executor processes in Standalone mode

2019-03-05 Thread Gabor Somogyi (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26998?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16784391#comment-16784391 ] Gabor Somogyi edited comment on SPARK-26998 at 3/5/19 12:51 PM:

[jira] [Created] (SPARK-27057) Common trait for limit exec operators

2019-03-05 Thread Maxim Gekk (JIRA)
Maxim Gekk created SPARK-27057: -- Summary: Common trait for limit exec operators Key: SPARK-27057 URL: https://issues.apache.org/jira/browse/SPARK-27057 Project: Spark Issue Type: Improvement

[jira] [Updated] (SPARK-26923) Refactor ArrowRRunner and RRunner to share the same base

2019-03-05 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26923?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-26923: - Summary: Refactor ArrowRRunner and RRunner to share the same base (was: Refactor ArrowRRunner

[jira] [Assigned] (SPARK-27056) Remove `start-shuffle-service.sh`

2019-03-05 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27056?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27056: Assignee: Apache Spark > Remove `start-shuffle-service.sh` >

[jira] [Assigned] (SPARK-27056) Remove `start-shuffle-service.sh`

2019-03-05 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27056?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27056: Assignee: (was: Apache Spark) > Remove `start-shuffle-service.sh` >

[jira] [Updated] (SPARK-27056) Remove `start-shuffle-service.sh`

2019-03-05 Thread liuxian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27056?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liuxian updated SPARK-27056: Description: _start-shuffle-service.sh_ was only used by Mesos before _start-mesos-shuffle-service.sh_.

[jira] [Created] (SPARK-27056) Remove `start-shuffle-service.sh`

2019-03-05 Thread liuxian (JIRA)
liuxian created SPARK-27056: --- Summary: Remove `start-shuffle-service.sh` Key: SPARK-27056 URL: https://issues.apache.org/jira/browse/SPARK-27056 Project: Spark Issue Type: Improvement

[jira] [Updated] (SPARK-27055) Update Structured Streaming documentation because of DSv2 changes

2019-03-05 Thread Gabor Somogyi (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27055?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gabor Somogyi updated SPARK-27055: -- Description: Since SPARK-26956 has been merged the Structured Streaming documentation has to

[jira] [Created] (SPARK-27055) Update Structured Streaming documentation because of DSv2 changes

2019-03-05 Thread Gabor Somogyi (JIRA)
Gabor Somogyi created SPARK-27055: - Summary: Update Structured Streaming documentation because of DSv2 changes Key: SPARK-27055 URL: https://issues.apache.org/jira/browse/SPARK-27055 Project: Spark

[jira] [Commented] (SPARK-27019) Spark UI's SQL tab shows inconsistent values

2019-03-05 Thread peay (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27019?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16784260#comment-16784260 ] peay commented on SPARK-27019: -- Yes, I had edited my message above shortly after posting - cannot reproduce 

[jira] [Updated] (SPARK-26602) Insert into table fails after querying the UDF which is loaded with wrong hdfs path

2019-03-05 Thread Chakravarthi (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26602?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chakravarthi updated SPARK-26602: - Attachment: beforeFixUdf.txt > Insert into table fails after querying the UDF which is loaded

[jira] [Resolved] (SPARK-26850) Make EventLoggingListener LOG_FILE_PERMISSIONS configurable

2019-03-05 Thread Sandeep Katta (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26850?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandeep Katta resolved SPARK-26850. --- Resolution: Duplicate > Make EventLoggingListener LOG_FILE_PERMISSIONS configurable >

[jira] [Commented] (SPARK-23339) Spark UI not loading *.js/*.css files, only raw HTML

2019-03-05 Thread gary (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23339?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16784204#comment-16784204 ] gary commented on SPARK-23339: -- exclude servlet-api-2.5.jar, use servlet-api-3.1.0.jar. It works for me. >

[jira] [Commented] (SPARK-20415) SPARK job hangs while writing DataFrame to HDFS

2019-03-05 Thread Martin Studer (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20415?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16784200#comment-16784200 ] Martin Studer commented on SPARK-20415: --- I'm observing a similar issue where all executor tasks

[jira] [Comment Edited] (SPARK-27019) Spark UI's SQL tab shows inconsistent values

2019-03-05 Thread Shahid K I (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27019?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16784194#comment-16784194 ] Shahid K I edited comment on SPARK-27019 at 3/5/19 7:59 AM: Please upload

<    1   2