[jira] [Created] (SPARK-21714) SparkSubmit in Yarn Client mode downloads remote files and then reuploads them again

Thomas Graves created SPARK-21714: - Summary: SparkSubmit in Yarn Client mode downloads remote files and then reuploads them again Key: SPARK-21714 URL: https://issues.apache.org/jira/browse/SPARK-21714

[jira] [Created] (SPARK-21713) Replace LogicalPlan.isStreaming with OutputMode

Jose Torres created SPARK-21713: --- Summary: Replace LogicalPlan.isStreaming with OutputMode Key: SPARK-21713 URL: https://issues.apache.org/jira/browse/SPARK-21713 Project: Spark Issue Type:

[jira] [Commented] (SPARK-21711) spark-submit command should accept log4j configuration parameters for spark client logging.

[ https://issues.apache.org/jira/browse/SPARK-21711?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16123770#comment-16123770 ] Sean Owen commented on SPARK-21711: --- How about configuring the log4j config with

[jira] [Created] (SPARK-21712) Clarify PySpark Column.substr() type checking error message

Nicholas Chammas created SPARK-21712: Summary: Clarify PySpark Column.substr() type checking error message Key: SPARK-21712 URL: https://issues.apache.org/jira/browse/SPARK-21712 Project: Spark

[jira] [Commented] (SPARK-5564) Support sparse LDA solutions

[ https://issues.apache.org/jira/browse/SPARK-5564?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16123803#comment-16123803 ] Valeriy Avanesov commented on SPARK-5564: - I am considering working on this issue. The question is

[jira] [Issue Comment Deleted] (SPARK-14927) DataFrame. saveAsTable creates RDD partitions but not Hive partitions

[ https://issues.apache.org/jira/browse/SPARK-14927?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raul Saez Tapia updated SPARK-14927: Comment: was deleted (was: *SOMASUNDARAM SUDALAIMUTHU* I am sorry but I am out of project

[jira] [Resolved] (SPARK-21700) How can I get the MetricsSystem information

[ https://issues.apache.org/jira/browse/SPARK-21700?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-21700. --- Resolution: Invalid > How can I get the MetricsSystem information >

[jira] [Commented] (SPARK-21701) Add TCP send/rcv buffer size support for RPC client

[ https://issues.apache.org/jira/browse/SPARK-21701?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16123004#comment-16123004 ] Sean Owen commented on SPARK-21701: --- I think you'd have to show evidence that it's worth the extra

[jira] [Commented] (SPARK-19372) Code generation for Filter predicate including many OR conditions exceeds JVM method size limit

[ https://issues.apache.org/jira/browse/SPARK-19372?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16123029#comment-16123029 ] srinivasan commented on SPARK-19372: Hi [~kiszk], the fix does not work for select * from temp where

[jira] [Commented] (SPARK-21690) one-pass imputer

[ https://issues.apache.org/jira/browse/SPARK-21690?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16122904#comment-16122904 ] Yan Facai (颜发才) commented on SPARK-21690: - We can use `df.summary("mean")` and

[jira] [Comment Edited] (SPARK-21690) one-pass imputer

[ https://issues.apache.org/jira/browse/SPARK-21690?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16122904#comment-16122904 ] Yan Facai (颜发才) edited comment on SPARK-21690 at 8/11/17 6:02 AM: -- We

[jira] [Created] (SPARK-21706) Support Custom PartitionSpec Provider for Kinesis Firehose or similar

Sebastian Herold created SPARK-21706: Summary: Support Custom PartitionSpec Provider for Kinesis Firehose or similar Key: SPARK-21706 URL: https://issues.apache.org/jira/browse/SPARK-21706

[jira] [Updated] (SPARK-21520) Improvement a special case for non-deterministic projects in optimizer

[ https://issues.apache.org/jira/browse/SPARK-21520?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] caoxuewen updated SPARK-21520: -- Summary: Improvement a special case for non-deterministic projects in optimizer (was: Improvement a

[jira] [Commented] (SPARK-21690) one-pass imputer

[ https://issues.apache.org/jira/browse/SPARK-21690?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16122907#comment-16122907 ] zhengruifeng commented on SPARK-21690: -- [~facai] Thanks, but I already send a PR for this. However,

[jira] [Comment Edited] (SPARK-21690) one-pass imputer

[ https://issues.apache.org/jira/browse/SPARK-21690?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16122907#comment-16122907 ] zhengruifeng edited comment on SPARK-21690 at 8/11/17 6:06 AM: --- [~facai]

[jira] [Resolved] (SPARK-21703) Why RPC message are transferred with header and body separately in TCP frame

[ https://issues.apache.org/jira/browse/SPARK-21703?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-21703. --- Resolution: Invalid This belongs on the mailing list not JIRA > Why RPC message are transferred

[jira] [Comment Edited] (SPARK-19372) Code generation for Filter predicate including many OR conditions exceeds JVM method size limit

[ https://issues.apache.org/jira/browse/SPARK-19372?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16123029#comment-16123029 ] srinivasan edited comment on SPARK-19372 at 8/11/17 8:28 AM: - Hi [~kiszk],

[jira] [Comment Edited] (SPARK-19372) Code generation for Filter predicate including many OR conditions exceeds JVM method size limit

[ https://issues.apache.org/jira/browse/SPARK-19372?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16123029#comment-16123029 ] srinivasan edited comment on SPARK-19372 at 8/11/17 8:29 AM: - Hi [~kiszk],

[jira] [Comment Edited] (SPARK-21701) Add TCP send/rcv buffer size support for RPC client

[ https://issues.apache.org/jira/browse/SPARK-21701?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16123096#comment-16123096 ] neoremind edited comment on SPARK-21701 at 8/11/17 9:44 AM: Hi Sean, Thanks

[jira] [Commented] (SPARK-21690) one-pass imputer

[ https://issues.apache.org/jira/browse/SPARK-21690?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16122922#comment-16122922 ] Yan Facai (颜发才) commented on SPARK-21690: - Cool! Just go head. > one-pass imputer >

[jira] [Created] (SPARK-21705) Add spark.internal.config parameter description

caoxuewen created SPARK-21705: - Summary: Add spark.internal.config parameter description Key: SPARK-21705 URL: https://issues.apache.org/jira/browse/SPARK-21705 Project: Spark Issue Type:

[jira] [Commented] (SPARK-21703) Why RPC message are transferred with header and body separately in TCP frame

[ https://issues.apache.org/jira/browse/SPARK-21703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16123071#comment-16123071 ] neoremind commented on SPARK-21703: --- Thanks Sean to guide me to the right place. Here's the

[jira] [Comment Edited] (SPARK-21690) one-pass imputer

[ https://issues.apache.org/jira/browse/SPARK-21690?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16122904#comment-16122904 ] Yan Facai (颜发才) edited comment on SPARK-21690 at 8/11/17 6:02 AM: -- We

[jira] [Resolved] (SPARK-21600) The description of "this requires spark.shuffle.service.enabled to be set" for the spark.dynamicAllocation.enabled configuration item is not clear

[ https://issues.apache.org/jira/browse/SPARK-21600?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-21600. --- Resolution: Won't Fix This is not worth the review overhead. > The description of "this requires

[jira] [Comment Edited] (SPARK-21657) Spark has exponential time complexity to explode(array of structs)

[ https://issues.apache.org/jira/browse/SPARK-21657?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16123050#comment-16123050 ] Liang-Chi Hsieh edited comment on SPARK-21657 at 8/11/17 8:55 AM: -- Maybe

[jira] [Commented] (SPARK-21697) NPE & ExceptionInInitializerError trying to load UTF from HDFS

[ https://issues.apache.org/jira/browse/SPARK-21697?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16123063#comment-16123063 ] Steve Loughran commented on SPARK-21697: # I don't see anything which can be done in HDFS here;

[jira] [Updated] (SPARK-21520) Improvement a special case for non-deterministic projects in optimizer

[ https://issues.apache.org/jira/browse/SPARK-21520?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] caoxuewen updated SPARK-21520: -- Description: Currently, Did a lot of special handling for non-deterministic projects and filters in

[jira] [Comment Edited] (SPARK-21703) Why RPC message are transferred with header and body separately in TCP frame

[ https://issues.apache.org/jira/browse/SPARK-21703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16123071#comment-16123071 ] neoremind edited comment on SPARK-21703 at 8/11/17 9:18 AM: Thanks Sean to

[jira] [Commented] (SPARK-21701) Add TCP send/rcv buffer size support for RPC client

[ https://issues.apache.org/jira/browse/SPARK-21701?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16123096#comment-16123096 ] neoremind commented on SPARK-21701: --- Hi Sean, Thanks for your quick response. SO_RCVBUF and SO_SNDBUF

[jira] [Commented] (SPARK-21690) one-pass imputer

[ https://issues.apache.org/jira/browse/SPARK-21690?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16122909#comment-16122909 ] zhengruifeng commented on SPARK-21690: -- The corresponding PR is here

[jira] [Created] (SPARK-21704) Add the description of 'sbin/stop-slave.sh' in spark-standalone.html.

guoxiaolongzte created SPARK-21704: -- Summary: Add the description of 'sbin/stop-slave.sh' in spark-standalone.html. Key: SPARK-21704 URL: https://issues.apache.org/jira/browse/SPARK-21704 Project:

[jira] [Commented] (SPARK-21657) Spark has exponential time complexity to explode(array of structs)

[ https://issues.apache.org/jira/browse/SPARK-21657?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16123050#comment-16123050 ] Liang-Chi Hsieh commented on SPARK-21657: - Maybe not very related to this issue. But I'm

[jira] [Comment Edited] (SPARK-21701) Add TCP send/rcv buffer size support for RPC client

[ https://issues.apache.org/jira/browse/SPARK-21701?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16123096#comment-16123096 ] neoremind edited comment on SPARK-21701 at 8/11/17 9:46 AM: Hi Sean, Thanks

[jira] [Created] (SPARK-21708) use sbt 1.0.0

PJ Fanning created SPARK-21708: -- Summary: use sbt 1.0.0 Key: SPARK-21708 URL: https://issues.apache.org/jira/browse/SPARK-21708 Project: Spark Issue Type: Improvement Components:

[jira] [Updated] (SPARK-21708) use sbt 1.0.0

[ https://issues.apache.org/jira/browse/SPARK-21708?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] PJ Fanning updated SPARK-21708: --- Description: I had a quick look and I think we'll need to wait until sbt-launch 1.0 jar is

[jira] [Commented] (SPARK-14401) Switch to stock sbt-pom-reader plugin

[ https://issues.apache.org/jira/browse/SPARK-14401?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16123319#comment-16123319 ] PJ Fanning commented on SPARK-14401: This would be useful for a general upgrade to sbt 1.0.0 >

[jira] [Updated] (SPARK-21708) use sbt 1.0.0

[ https://issues.apache.org/jira/browse/SPARK-21708?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] PJ Fanning updated SPARK-21708: --- Description: I had a quick look and I think we'll need to wait until sbt-launch 1.0 jar is

[jira] [Commented] (SPARK-11574) Spark should support StatsD sink out of box

[ https://issues.apache.org/jira/browse/SPARK-11574?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16123344#comment-16123344 ] Na Zhao commented on SPARK-11574: - When can I expect this feature to release? > Spark should support

[jira] [Commented] (SPARK-21656) spark dynamic allocation should not idle timeout executors when there are enough tasks to run on them

[ https://issues.apache.org/jira/browse/SPARK-21656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16123358#comment-16123358 ] Thomas Graves commented on SPARK-21656: --- example of test results with this. We have production job

[jira] [Commented] (SPARK-21686) spark.sql.hive.convertMetastoreOrc is causing NullPointerException while reading ORC tables

[ https://issues.apache.org/jira/browse/SPARK-21686?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16123361#comment-16123361 ] Ernani Pereira de Mattos Junior commented on SPARK-21686: - Hello [~viirya] I

[jira] [Commented] (SPARK-21697) NPE & ExceptionInInitializerError trying to load UTF from HDFS

[ https://issues.apache.org/jira/browse/SPARK-21697?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16123193#comment-16123193 ] Steve Loughran commented on SPARK-21697: PS: right now, probably doesn't work at all > NPE &

[jira] [Commented] (SPARK-12868) ADD JAR via sparkSQL JDBC will fail when using a HDFS URL

[ https://issues.apache.org/jira/browse/SPARK-12868?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16123192#comment-16123192 ] Steve Loughran commented on SPARK-12868: SPARK-21697: harder than it would initially seem > ADD

[jira] [Comment Edited] (SPARK-21701) Add TCP send/rcv buffer size support for RPC client

[ https://issues.apache.org/jira/browse/SPARK-21701?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16123096#comment-16123096 ] Xu Zhang edited comment on SPARK-21701 at 8/11/17 9:50 AM: --- Hi Sean, Thanks

[jira] [Created] (SPARK-21707) Improvement a special case for non-deterministic filters in optimizer

caoxuewen created SPARK-21707: - Summary: Improvement a special case for non-deterministic filters in optimizer Key: SPARK-21707 URL: https://issues.apache.org/jira/browse/SPARK-21707 Project: Spark

[jira] [Comment Edited] (SPARK-21701) Add TCP send/rcv buffer size support for RPC client

[ https://issues.apache.org/jira/browse/SPARK-21701?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16123096#comment-16123096 ] Xu Zhang edited comment on SPARK-21701 at 8/11/17 9:53 AM: --- Hi Sean, Thanks

[jira] [Commented] (SPARK-21697) NPE & ExceptionInInitializerError trying to load UTF from HDFS

[ https://issues.apache.org/jira/browse/SPARK-21697?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16123208#comment-16123208 ] Steve Loughran commented on SPARK-21697: What would a test to replicate look like? # Create

[jira] [Resolved] (SPARK-21595) introduction of spark.sql.windowExec.buffer.spill.threshold in spark 2.2 breaks existing workflow

[ https://issues.apache.org/jira/browse/SPARK-21595?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell resolved SPARK-21595. --- Resolution: Fixed Assignee: Tejas Patil Fix Version/s: 2.3.0

[jira] [Resolved] (SPARK-19122) Unnecessary shuffle+sort added if join predicates ordering differ from bucketing and sorting order

[ https://issues.apache.org/jira/browse/SPARK-19122?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-19122. - Resolution: Fixed Assignee: Tejas Patil Fix Version/s: 2.3.0 > Unnecessary shuffle+sort

[jira] [Comment Edited] (SPARK-21711) spark-submit command should accept log4j configuration parameters for spark client logging.

[ https://issues.apache.org/jira/browse/SPARK-21711?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16123889#comment-16123889 ] Mahesh Ambule edited comment on SPARK-21711 at 8/11/17 7:27 PM: Sean

[jira] [Updated] (SPARK-21519) Add an option to the JDBC data source to initialize the environment of the remote database session

[ https://issues.apache.org/jira/browse/SPARK-21519?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Luca Canali updated SPARK-21519: Description: This proposes an option to the JDBC datasource, tentatively called

[jira] [Updated] (SPARK-21519) Add an option to the JDBC data source to initialize the environment of the remote database session

[ https://issues.apache.org/jira/browse/SPARK-21519?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Luca Canali updated SPARK-21519: Description: This proposes an option to the JDBC datasource, tentatively called

[jira] [Updated] (SPARK-21715) History Server respondes history page html content multiple times for only one http request

[ https://issues.apache.org/jira/browse/SPARK-21715?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ye Zhou updated SPARK-21715: Attachment: Performance.png > History Server respondes history page html content multiple times for only

[jira] [Created] (SPARK-21715) History Server respondes history page html content multiple times for only one http request

Ye Zhou created SPARK-21715: --- Summary: History Server respondes history page html content multiple times for only one http request Key: SPARK-21715 URL: https://issues.apache.org/jira/browse/SPARK-21715

[jira] [Commented] (SPARK-21698) write.partitionBy() is giving me garbage data

[ https://issues.apache.org/jira/browse/SPARK-21698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16124161#comment-16124161 ] Xiao Li commented on SPARK-21698: - {{insertInto}} is resolved by positions. You can printout the schema

[jira] [Comment Edited] (SPARK-21711) spark-submit command should accept log4j configuration parameters for spark client logging.

[ https://issues.apache.org/jira/browse/SPARK-21711?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16123889#comment-16123889 ] Mahesh Ambule edited comment on SPARK-21711 at 8/11/17 7:22 PM: Sean

[jira] [Commented] (SPARK-21711) spark-submit command should accept log4j configuration parameters for spark client logging.

[ https://issues.apache.org/jira/browse/SPARK-21711?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16123941#comment-16123941 ] Mahesh Ambule commented on SPARK-21711: --- I dont want to configure executor or driver java options.

[jira] [Commented] (SPARK-21711) spark-submit command should accept log4j configuration parameters for spark client logging.

[ https://issues.apache.org/jira/browse/SPARK-21711?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16123957#comment-16123957 ] Sean Owen commented on SPARK-21711: --- Oh, well that's your own application. You configure your own

[jira] [Updated] (SPARK-21519) Add an option to the JDBC data source to initialize the environment of the remote database session

[ https://issues.apache.org/jira/browse/SPARK-21519?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Luca Canali updated SPARK-21519: Description: This proposes an option to the JDBC datasource, tentatively called

[jira] [Comment Edited] (SPARK-21711) spark-submit command should accept log4j configuration parameters for spark client logging.

[ https://issues.apache.org/jira/browse/SPARK-21711?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16123967#comment-16123967 ] Mahesh Ambule edited comment on SPARK-21711 at 8/11/17 8:18 PM: Here by

[jira] [Updated] (SPARK-21715) History Server respondes history page html content multiple times for only one http request

[ https://issues.apache.org/jira/browse/SPARK-21715?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ye Zhou updated SPARK-21715: Description: UI looks fine for the home page. But we check the performance for each individual

[jira] [Updated] (SPARK-21715) History Server respondes history page html content multiple times for only one http request

[ https://issues.apache.org/jira/browse/SPARK-21715?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ye Zhou updated SPARK-21715: Description: UI looks fine for the home page. But we check the performance for each individual

[jira] [Updated] (SPARK-21715) History Server respondes history page html content multiple times for only one http request

[ https://issues.apache.org/jira/browse/SPARK-21715?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ye Zhou updated SPARK-21715: Attachment: ResponseContent.png > History Server respondes history page html content multiple times for

[jira] [Commented] (SPARK-21708) use sbt 1.0.0

[ https://issues.apache.org/jira/browse/SPARK-21708?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16124012#comment-16124012 ] PJ Fanning commented on SPARK-21708: [~srowen] Your point about IDEs is valid. IntelliJ IDEA has

[jira] [Updated] (SPARK-21200) Spark REST API is not working or Spark documentation is wrong.

[ https://issues.apache.org/jira/browse/SPARK-21200?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rahul Gupta updated SPARK-21200: Attachment: (was: Screen Shot 2017-08-11 at 2.29.38 PM.png) > Spark REST API is not working or

[jira] [Updated] (SPARK-21200) Spark REST API is not working or Spark documentation is wrong.

[ https://issues.apache.org/jira/browse/SPARK-21200?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rahul Gupta updated SPARK-21200: Attachment: Screen Shot 2017-08-11 at 2.29.38 PM.png sample of API working in spark 1.4 > Spark

[jira] [Comment Edited] (SPARK-21711) spark-submit command should accept log4j configuration parameters for spark client logging.

[ https://issues.apache.org/jira/browse/SPARK-21711?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16123889#comment-16123889 ] Mahesh Ambule edited comment on SPARK-21711 at 8/11/17 7:36 PM: @Sean

[jira] [Updated] (SPARK-21715) History Server respondes history page html content multiple times for only one http request

[ https://issues.apache.org/jira/browse/SPARK-21715?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ye Zhou updated SPARK-21715: Description: UI looks fine for the home page. But we check the performance for each individual

[jira] [Assigned] (SPARK-21519) Add an option to the JDBC data source to initialize the environment of the remote database session

[ https://issues.apache.org/jira/browse/SPARK-21519?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li reassigned SPARK-21519: --- Assignee: Luca Canali > Add an option to the JDBC data source to initialize the environment of the

[jira] [Issue Comment Deleted] (SPARK-21200) Spark REST API is not working or Spark documentation is wrong.

[ https://issues.apache.org/jira/browse/SPARK-21200?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rahul Gupta updated SPARK-21200: Comment: was deleted (was: sample of API working in spark 1.4) > Spark REST API is not working or

[jira] [Resolved] (SPARK-21698) write.partitionBy() is giving me garbage data

[ https://issues.apache.org/jira/browse/SPARK-21698?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-21698. - Resolution: Won't Fix > write.partitionBy() is giving me garbage data >

[jira] [Resolved] (SPARK-21519) Add an option to the JDBC data source to initialize the environment of the remote database session

[ https://issues.apache.org/jira/browse/SPARK-21519?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-21519. - Resolution: Fixed Fix Version/s: 2.3.0 > Add an option to the JDBC data source to initialize the

[jira] [Commented] (SPARK-21711) spark-submit command should accept log4j configuration parameters for spark client logging.

[ https://issues.apache.org/jira/browse/SPARK-21711?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16123889#comment-16123889 ] Mahesh Ambule commented on SPARK-21711: --- sean owen: Thanks for the reply. I tried to pass

[jira] [Updated] (SPARK-21711) spark-submit command should accept log4j configuration parameters for spark client logging.

[ https://issues.apache.org/jira/browse/SPARK-21711?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mahesh Ambule updated SPARK-21711: -- Attachment: spark-submit client logs.txt > spark-submit command should accept log4j

[jira] [Commented] (SPARK-21711) spark-submit command should accept log4j configuration parameters for spark client logging.

[ https://issues.apache.org/jira/browse/SPARK-21711?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16123911#comment-16123911 ] Sean Owen commented on SPARK-21711: --- It does, I'm referring to the spark.executor.extraJavaOptions

[jira] [Commented] (SPARK-21711) spark-submit command should accept log4j configuration parameters for spark client logging.

[ https://issues.apache.org/jira/browse/SPARK-21711?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16123967#comment-16123967 ] Mahesh Ambule commented on SPARK-21711: --- Here by spark client, I meant java client process started

[jira] [Commented] (SPARK-21200) Spark REST API is not working or Spark documentation is wrong.

[ https://issues.apache.org/jira/browse/SPARK-21200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16124116#comment-16124116 ] Rahul Gupta commented on SPARK-21200: - [~sowen] I was facing similar issue, and wanted to know the

[jira] [Resolved] (SPARK-12559) Cluster mode doesn't work with --packages

[ https://issues.apache.org/jira/browse/SPARK-12559?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-12559. Resolution: Fixed Assignee: Stavros Kontopoulos Fix Version/s: 2.3.0 >

[jira] [Updated] (SPARK-18278) SPIP: Support native submission of spark jobs to a kubernetes cluster

[ https://issues.apache.org/jira/browse/SPARK-18278?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anirudh Ramanathan updated SPARK-18278: --- Attachment: SPARK-18278 Spark on Kubernetes Design Proposal Revision 2 (1).pdf

[jira] [Commented] (SPARK-21710) ConsoleSink causes OOM crashes with large inputs.

[ https://issues.apache.org/jira/browse/SPARK-21710?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16124200#comment-16124200 ] Shixiong Zhu commented on SPARK-21710: -- `collect` is a workaround for

[jira] [Comment Edited] (SPARK-18085) SPIP: Better History Server scalability for many / large applications

[ https://issues.apache.org/jira/browse/SPARK-18085?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16124421#comment-16124421 ] duyanghao edited comment on SPARK-18085 at 8/12/17 3:11 AM: [~vanzin] Could

[jira] [Commented] (SPARK-18085) SPIP: Better History Server scalability for many / large applications

[ https://issues.apache.org/jira/browse/SPARK-18085?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16124421#comment-16124421 ] duyanghao commented on SPARK-18085: --- [~vanzin] Could you have a summary of your progress(solved

[jira] [Closed] (SPARK-21689) Spark submit will not get kerberos token token when hbase class not found

[ https://issues.apache.org/jira/browse/SPARK-21689?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhoukang closed SPARK-21689. Resolution: Won't Fix > Spark submit will not get kerberos token token when hbase class not found >

[jira] [Commented] (SPARK-21711) spark-submit command should accept log4j configuration parameters for spark client logging.

[ https://issues.apache.org/jira/browse/SPARK-21711?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16124457#comment-16124457 ] Mahesh Ambule commented on SPARK-21711: --- My Application > spark-submit > spark client --->

[jira] [Issue Comment Deleted] (SPARK-21711) spark-submit command should accept log4j configuration parameters for spark client logging.

[ https://issues.apache.org/jira/browse/SPARK-21711?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mahesh Ambule updated SPARK-21711: -- Comment: was deleted (was: Here by spark client, I meant java client process started by

[jira] [Resolved] (SPARK-21675) Add a navigation bar at the bottom of the Details for Stage Page

[ https://issues.apache.org/jira/browse/SPARK-21675?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-21675. --- Resolution: Fixed Fix Version/s: 2.3.0 Issue resolved by pull request 18893

[jira] [Updated] (SPARK-21709) use sbt 0.13.16 and update sbt plugins

[ https://issues.apache.org/jira/browse/SPARK-21709?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] PJ Fanning updated SPARK-21709: --- Description: A preliminary step to SPARK-21708. Quite a lot of sbt plugin changes needed to get to

[jira] [Updated] (SPARK-21709) use sbt 0.13.16 and update sbt plugins

[ https://issues.apache.org/jira/browse/SPARK-21709?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-21709: -- Priority: Minor (was: Major) OK, not sure if I'd treat these separately, as we have no burning need

[jira] [Created] (SPARK-21710) ConsoleSink causes OOM crashes with large inputs.

Gerard Maas created SPARK-21710: --- Summary: ConsoleSink causes OOM crashes with large inputs. Key: SPARK-21710 URL: https://issues.apache.org/jira/browse/SPARK-21710 Project: Spark Issue Type:

[jira] [Updated] (SPARK-21710) ConsoleSink causes OOM crashes with large inputs.

[ https://issues.apache.org/jira/browse/SPARK-21710?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gerard Maas updated SPARK-21710: Description: ConsoleSink does a full collect of the streaming dataset in order to show few lines

[jira] [Commented] (SPARK-21656) spark dynamic allocation should not idle timeout executors when there are enough tasks to run on them

[ https://issues.apache.org/jira/browse/SPARK-21656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16123435#comment-16123435 ] Thomas Graves commented on SPARK-21656: --- Yes there is a trade off here, use some more resource or

[jira] [Comment Edited] (SPARK-21656) spark dynamic allocation should not idle timeout executors when there are enough tasks to run on them

[ https://issues.apache.org/jira/browse/SPARK-21656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16123439#comment-16123439 ] Thomas Graves edited comment on SPARK-21656 at 8/11/17 2:43 PM: Note,

[jira] [Commented] (SPARK-21708) use sbt 1.0.0

[ https://issues.apache.org/jira/browse/SPARK-21708?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16123451#comment-16123451 ] PJ Fanning commented on SPARK-21708: [~srowen] the build/sbt scripting will download the preferred

[jira] [Updated] (SPARK-21711) spark-submit command should accept log4j configuration parameters for spark client logging.

[ https://issues.apache.org/jira/browse/SPARK-21711?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mahesh Ambule updated SPARK-21711: -- Description: Currently, log4j properties can be specified in spark 'conf' directory in

[jira] [Commented] (SPARK-21685) Params isSet in scala Transformer triggered by _setDefault in pyspark

[ https://issues.apache.org/jira/browse/SPARK-21685?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16123738#comment-16123738 ] Ratan Rai Sur commented on SPARK-21685: --- The python wrapper is generated so I've pasted it here so

[jira] [Commented] (SPARK-21710) ConsoleSink causes OOM crashes with large inputs.

[ https://issues.apache.org/jira/browse/SPARK-21710?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16123485#comment-16123485 ] Gerard Maas commented on SPARK-21710: - PR: https://github.com/apache/spark/pull/18923 > ConsoleSink

[jira] [Created] (SPARK-21711) spark-submit command should accept log4j configuration parameters for spark client logging.

Mahesh Ambule created SPARK-21711: - Summary: spark-submit command should accept log4j configuration parameters for spark client logging. Key: SPARK-21711 URL: https://issues.apache.org/jira/browse/SPARK-21711

[jira] [Updated] (SPARK-21711) spark-submit command should accept log4j configuration parameters for spark client logging.

[ https://issues.apache.org/jira/browse/SPARK-21711?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mahesh Ambule updated SPARK-21711: -- Description: Currently, log4j properties can be specified in spark 'conf' directory in

[jira] [Assigned] (SPARK-21675) Add a navigation bar at the bottom of the Details for Stage Page

[ https://issues.apache.org/jira/browse/SPARK-21675?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen reassigned SPARK-21675: - Assignee: Kent Yao > Add a navigation bar at the bottom of the Details for Stage Page >

[jira] [Created] (SPARK-21709) use sbt 0.13.16 and update sbt plugins

PJ Fanning created SPARK-21709: -- Summary: use sbt 0.13.16 and update sbt plugins Key: SPARK-21709 URL: https://issues.apache.org/jira/browse/SPARK-21709 Project: Spark Issue Type: Improvement

[jira] [Updated] (SPARK-21708) use sbt 1.0.0

[ https://issues.apache.org/jira/browse/SPARK-21708?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-21708: -- Priority: Minor (was: Major) Just wondering if this will cause end users to have to use SBT 1.0, or

  1   2   >