[jira] [Commented] (SPARK-21591) Implement treeAggregate on Dataset API

[ https://issues.apache.org/jira/browse/SPARK-21591?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16108476#comment-16108476 ] Yanbo Liang commented on SPARK-21591: - cc [~cloud_fan] > Implement treeAggregate on Dataset API >

[jira] [Updated] (SPARK-21591) Implement treeAggregate on Dataset API

[ https://issues.apache.org/jira/browse/SPARK-21591?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yanbo Liang updated SPARK-21591: Issue Type: Brainstorming (was: New Feature) > Implement treeAggregate on Dataset API >

[jira] [Updated] (SPARK-21591) Implement treeAggregate on Dataset API

[ https://issues.apache.org/jira/browse/SPARK-21591?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yanbo Liang updated SPARK-21591: Description: The Tungsten execution engine substantially improved the efficiency of memory and

[jira] [Commented] (SPARK-21586) Read CSV (SQL Context) Doesnt ignore delimiters within special types of quotes, other special characters

[ https://issues.apache.org/jira/browse/SPARK-21586?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16108471#comment-16108471 ] Sean Owen commented on SPARK-21586: --- Give an example of the input and how you read it. It is not clear

[jira] [Updated] (SPARK-21591) Implement treeAggregate on Dataset API

[ https://issues.apache.org/jira/browse/SPARK-21591?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yanbo Liang updated SPARK-21591: Description: The Tungsten execution engine substantially improved the efficiency of memory and

[jira] [Updated] (SPARK-21591) Implement treeAggregate on Dataset API

[ https://issues.apache.org/jira/browse/SPARK-21591?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yanbo Liang updated SPARK-21591: Description: The Tungsten execution engine substantially improved the efficiency of memory and

[jira] [Created] (SPARK-21591) Implement treeAggregate on Dataset API

Yanbo Liang created SPARK-21591: --- Summary: Implement treeAggregate on Dataset API Key: SPARK-21591 URL: https://issues.apache.org/jira/browse/SPARK-21591 Project: Spark Issue Type: New Feature

[jira] [Updated] (SPARK-21591) Implement treeAggregate on Dataset API

[ https://issues.apache.org/jira/browse/SPARK-21591?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yanbo Liang updated SPARK-21591: Description: The Tungsten execution engine substantially improved the efficiency of memory and

[jira] [Commented] (SPARK-21591) Implement treeAggregate on Dataset API

[ https://issues.apache.org/jira/browse/SPARK-21591?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16108644#comment-16108644 ] Kazuaki Ishizaki commented on SPARK-21591: -- I like this idea > Implement treeAggregate on

[jira] [Commented] (SPARK-21588) SQLContext.getConf(key, null) should return null, but it throws NPE

[ https://issues.apache.org/jira/browse/SPARK-21588?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16108622#comment-16108622 ] Vinod KC commented on SPARK-21588: -- [~brkyvz] Can you share sample code and NPE stack trace? >

[jira] [Assigned] (SPARK-21475) Change the usage of FileInputStream/OutputStream to Files.newInput/OutputStream in the critical path

[ https://issues.apache.org/jira/browse/SPARK-21475?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen reassigned SPARK-21475: - Assignee: Saisai Shao > Change the usage of FileInputStream/OutputStream to >

[jira] [Resolved] (SPARK-21475) Change the usage of FileInputStream/OutputStream to Files.newInput/OutputStream in the critical path

[ https://issues.apache.org/jira/browse/SPARK-21475?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-21475. --- Resolution: Fixed Fix Version/s: 2.3.0 Issue resolved by pull request 18684

[jira] [Resolved] (SPARK-21582) DataFrame.withColumnRenamed cause huge performance overhead

[ https://issues.apache.org/jira/browse/SPARK-21582?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-21582. --- Resolution: Not A Problem > DataFrame.withColumnRenamed cause huge performance overhead >

[jira] [Comment Edited] (SPARK-21591) Implement treeAggregate on Dataset API

[ https://issues.apache.org/jira/browse/SPARK-21591?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16108476#comment-16108476 ] Yanbo Liang edited comment on SPARK-21591 at 8/1/17 6:50 AM: - cc [~cloud_fan]

[jira] [Created] (SPARK-21599) Collecting column statistics for datasource tables may fail with java.util.NoSuchElementException

Dilip Biswal created SPARK-21599: Summary: Collecting column statistics for datasource tables may fail with java.util.NoSuchElementException Key: SPARK-21599 URL: https://issues.apache.org/jira/browse/SPARK-21599

[jira] [Updated] (SPARK-21565) aggregate query fails with watermark on eventTime but works with watermark on timestamp column generated by current_timestamp

[ https://issues.apache.org/jira/browse/SPARK-21565?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-21565: - Description: *Short Description: * Aggregation query fails with eventTime as watermark column

[jira] [Commented] (SPARK-21598) Collect usability/events information from Spark History Server

[ https://issues.apache.org/jira/browse/SPARK-21598?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16110028#comment-16110028 ] Marcelo Vanzin commented on SPARK-21598: [~steve_l] has been working for some time on

[jira] [Created] (SPARK-21601) Modify the JDK version of the Maven compilation

jifei_yang created SPARK-21601: -- Summary: Modify the JDK version of the Maven compilation Key: SPARK-21601 URL: https://issues.apache.org/jira/browse/SPARK-21601 Project: Spark Issue Type:

[jira] [Commented] (SPARK-21591) Implement treeAggregate on Dataset API

[ https://issues.apache.org/jira/browse/SPARK-21591?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16110196#comment-16110196 ] Yanbo Liang commented on SPARK-21591: - [~viirya] Yep, this is the way we are using, but we want to

[jira] [Updated] (SPARK-21604) if the object extends Logging, i suggest to remove the var LOG which is useless.

[ https://issues.apache.org/jira/browse/SPARK-21604?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zuotingbing updated SPARK-21604: Description: if the object extends Logging, i suggest to remove the var LOG which is useless.

[jira] [Created] (SPARK-21605) Let IntelliJ IDEA correctly detect Language level and Target byte code version

Chang chen created SPARK-21605: -- Summary: Let IntelliJ IDEA correctly detect Language level and Target byte code version Key: SPARK-21605 URL: https://issues.apache.org/jira/browse/SPARK-21605 Project:

[jira] [Created] (SPARK-21604) Error class name for log, and if the object extends Logging, i suggest to remove the var LOG which is useless.

zuotingbing created SPARK-21604: --- Summary: Error class name for log, and if the object extends Logging, i suggest to remove the var LOG which is useless. Key: SPARK-21604 URL:

[jira] [Created] (SPARK-21603) The wholestage codegen will be much slower then wholestage codegen is closed when the function is too long

eaton created SPARK-21603: - Summary: The wholestage codegen will be much slower then wholestage codegen is closed when the function is too long Key: SPARK-21603 URL: https://issues.apache.org/jira/browse/SPARK-21603

[jira] [Resolved] (SPARK-21578) Add JavaSparkContextSuite

[ https://issues.apache.org/jira/browse/SPARK-21578?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-21578. - Resolution: Fixed > Add JavaSparkContextSuite > - > > Key:

[jira] [Updated] (SPARK-21605) Let IntelliJ IDEA correctly detect Language level and Target byte code version

[ https://issues.apache.org/jira/browse/SPARK-21605?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chang chen updated SPARK-21605: --- Labels: IDE maven (was: ) > Let IntelliJ IDEA correctly detect Language level and Target byte code

[jira] [Updated] (SPARK-21604) if the object extends Logging, i suggest to remove the var LOG which is useless.

[ https://issues.apache.org/jira/browse/SPARK-21604?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zuotingbing updated SPARK-21604: Summary: if the object extends Logging, i suggest to remove the var LOG which is useless. (was:

[jira] [Commented] (SPARK-12717) pyspark broadcast fails when using multiple threads

[ https://issues.apache.org/jira/browse/SPARK-12717?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16110007#comment-16110007 ] Bryan Cutler commented on SPARK-12717: -- Thanks [~hyukjin.kwon]! What are your thoughts on

[jira] [Updated] (SPARK-21590) Structured Streaming window start time should support negative values to adjust time zone

[ https://issues.apache.org/jira/browse/SPARK-21590?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kevin Zhang updated SPARK-21590: Description: I want to calculate (unique) daily access count using structured streaming (2.2.0).

[jira] [Closed] (SPARK-21601) Modify the JDK version of the Maven compilation

[ https://issues.apache.org/jira/browse/SPARK-21601?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jifei_yang closed SPARK-21601. -- Resolution: Won't Do > Modify the JDK version of the Maven compilation >

[jira] [Commented] (SPARK-21601) Modify the JDK version of the Maven compilation

[ https://issues.apache.org/jira/browse/SPARK-21601?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16110250#comment-16110250 ] jifei_yang commented on SPARK-21601: Just learned that the spark team is now using

[jira] [Commented] (SPARK-650) Add a "setup hook" API for running initialization code on each executor

[ https://issues.apache.org/jira/browse/SPARK-650?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16110006#comment-16110006 ] Michael Schmeißer commented on SPARK-650: - Please see my comment from 05/Dec/16 12:39 and the

[jira] [Commented] (SPARK-21600) The description of "this requires spark.shuffle.service.enabled to be set" for the spark.dynamicAllocation.enabled configuration item is not clear

[ https://issues.apache.org/jira/browse/SPARK-21600?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16110095#comment-16110095 ] guoxiaolongzte commented on SPARK-21600: I will do it. > The description of "this requires

[jira] [Created] (SPARK-21600) The description of "this requires spark.shuffle.service.enabled to be set" for the spark.dynamicAllocation.enabled configuration item is not clear

guoxiaolongzte created SPARK-21600: -- Summary: The description of "this requires spark.shuffle.service.enabled to be set" for the spark.dynamicAllocation.enabled configuration item is not clear Key: SPARK-21600

[jira] [Commented] (SPARK-21591) Implement treeAggregate on Dataset API

[ https://issues.apache.org/jira/browse/SPARK-21591?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16110169#comment-16110169 ] Liang-Chi Hsieh commented on SPARK-21591: - The most straightforward way is similar to

[jira] [Commented] (SPARK-21591) Implement treeAggregate on Dataset API

[ https://issues.apache.org/jira/browse/SPARK-21591?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16110226#comment-16110226 ] Liang-Chi Hsieh commented on SPARK-21591: - IIUC, basically the aggregation in SparkSQL doesn't

[jira] [Assigned] (SPARK-21599) Collecting column statistics for datasource tables may fail with java.util.NoSuchElementException

[ https://issues.apache.org/jira/browse/SPARK-21599?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li reassigned SPARK-21599: --- Assignee: Dilip Biswal > Collecting column statistics for datasource tables may fail with >

[jira] [Commented] (SPARK-21599) Collecting column statistics for datasource tables may fail with java.util.NoSuchElementException

[ https://issues.apache.org/jira/browse/SPARK-21599?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16110231#comment-16110231 ] Xiao Li commented on SPARK-21599: - https://github.com/apache/spark/pull/18804 > Collecting column

[jira] [Commented] (SPARK-21590) Structured Streaming window start time should support negative values to adjust time zone

[ https://issues.apache.org/jira/browse/SPARK-21590?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1610#comment-1610 ] Shixiong Zhu commented on SPARK-21590: -- Yeah, this is a bug. A timestamp can be negative. cc

[jira] [Commented] (SPARK-12717) pyspark broadcast fails when using multiple threads

[ https://issues.apache.org/jira/browse/SPARK-12717?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16110014#comment-16110014 ] Hyukjin Kwon commented on SPARK-12717: -- Would you mind if I ask open a backport? Just want to check

[jira] [Commented] (SPARK-21601) Modify the JDK version of the Maven compilation

[ https://issues.apache.org/jira/browse/SPARK-21601?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16110204#comment-16110204 ] jifei_yang commented on SPARK-21601: As follows, 1.8 1.8 1.8 > Modify the JDK version of

[jira] [Created] (SPARK-21602) Add map_keys and map_values functions to R

Hyukjin Kwon created SPARK-21602: Summary: Add map_keys and map_values functions to R Key: SPARK-21602 URL: https://issues.apache.org/jira/browse/SPARK-21602 Project: Spark Issue Type:

[jira] [Commented] (SPARK-21601) Modify the JDK version of the Maven compilation

[ https://issues.apache.org/jira/browse/SPARK-21601?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16110234#comment-16110234 ] jifei_yang commented on SPARK-21601: [https://github.com/apache/spark/pull/18807] > Modify the JDK

[jira] [Comment Edited] (SPARK-21591) Implement treeAggregate on Dataset API

[ https://issues.apache.org/jira/browse/SPARK-21591?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16110169#comment-16110169 ] Liang-Chi Hsieh edited comment on SPARK-21591 at 8/2/17 2:41 AM: - The

[jira] [Comment Edited] (SPARK-21591) Implement treeAggregate on Dataset API

[ https://issues.apache.org/jira/browse/SPARK-21591?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16110169#comment-16110169 ] Liang-Chi Hsieh edited comment on SPARK-21591 at 8/2/17 2:41 AM: - The

[jira] [Commented] (SPARK-21595) introduction of spark.sql.windowExec.buffer.spill.threshold in spark 2.2 breaks existing workflow

[ https://issues.apache.org/jira/browse/SPARK-21595?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16110053#comment-16110053 ] Tejas Patil commented on SPARK-21595: - This config was introduced by me in SPARK-13450. The reason

[jira] [Updated] (SPARK-21601) Modify the JDK version of the Maven compilation

[ https://issues.apache.org/jira/browse/SPARK-21601?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jifei_yang updated SPARK-21601: --- Description: When using maven to compile spark, I want to add a modified jdk property. This is

[jira] [Commented] (SPARK-21590) Structured Streaming window start time should support negative values to adjust time zone

[ https://issues.apache.org/jira/browse/SPARK-21590?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16110209#comment-16110209 ] Kevin Zhang commented on SPARK-21590: - I think the only problem is the non-negative check of window

[jira] [Updated] (SPARK-13669) Job will always fail in the external shuffle service unavailable situation

[ https://issues.apache.org/jira/browse/SPARK-13669?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Imran Rashid updated SPARK-13669: - Component/s: Scheduler > Job will always fail in the external shuffle service unavailable

[jira] [Commented] (SPARK-21110) Structs should be usable in inequality filters

[ https://issues.apache.org/jira/browse/SPARK-21110?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16109734#comment-16109734 ] Andrew Ray commented on SPARK-21110: I'm working on this > Structs should be usable in inequality

[jira] [Commented] (SPARK-12717) pyspark broadcast fails when using multiple threads

[ https://issues.apache.org/jira/browse/SPARK-12717?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16109923#comment-16109923 ] Hyukjin Kwon commented on SPARK-12717: -- I see. Thank you. Yes, I am seeing now. > pyspark broadcast

[jira] [Resolved] (SPARK-21573) Tests failing with run-tests.py SyntaxError occasionally in Jenkins

[ https://issues.apache.org/jira/browse/SPARK-21573?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-21573. --- Resolution: Fixed Assignee: shane knapp Fix Version/s: 2.3.0 Looks good, in that the

[jira] [Resolved] (SPARK-12717) pyspark broadcast fails when using multiple threads

[ https://issues.apache.org/jira/browse/SPARK-12717?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-12717. -- Resolution: Fixed Fix Version/s: 2.3.0 > pyspark broadcast fails when using multiple

[jira] [Resolved] (SPARK-21339) spark-shell --packages option does not add jars to classpath on windows

[ https://issues.apache.org/jira/browse/SPARK-21339?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-21339. Resolution: Fixed Assignee: Devaraj K Fix Version/s: 2.3.0

[jira] [Commented] (SPARK-19112) add codec for ZStandard

[ https://issues.apache.org/jira/browse/SPARK-19112?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16109784#comment-16109784 ] Sital Kedia commented on SPARK-19112: - [~sowen], [~tgraves] - Using zstd compression for our Spark

[jira] [Created] (SPARK-21598) Collect usability/events information from Spark History Server

Eric Vandenberg created SPARK-21598: --- Summary: Collect usability/events information from Spark History Server Key: SPARK-21598 URL: https://issues.apache.org/jira/browse/SPARK-21598 Project: Spark

[jira] [Commented] (SPARK-21330) Bad partitioning does not allow to read a JDBC table with extreme values on the partition column

[ https://issues.apache.org/jira/browse/SPARK-21330?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16109568#comment-16109568 ] Andrew Ray commented on SPARK-21330: https://github.com/apache/spark/pull/18800 > Bad partitioning

[jira] [Created] (SPARK-21597) Avg event time calculated in progress may be wrong

Shixiong Zhu created SPARK-21597: Summary: Avg event time calculated in progress may be wrong Key: SPARK-21597 URL: https://issues.apache.org/jira/browse/SPARK-21597 Project: Spark Issue

[jira] [Commented] (SPARK-18580) Use spark.streaming.backpressure.initialRate in DirectKafkaInputDStream

[ https://issues.apache.org/jira/browse/SPARK-18580?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16109946#comment-16109946 ] Oleg Muravskiy commented on SPARK-18580: Thanks, [~ozzieba]! >From my point of view this patch

[jira] [Commented] (SPARK-10878) Race condition when resolving Maven coordinates via Ivy

[ https://issues.apache.org/jira/browse/SPARK-10878?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16109778#comment-16109778 ] Min Shen commented on SPARK-10878: -- We hit the same issue in our infrastructure where concurrent Livy

[jira] [Commented] (SPARK-21573) Tests failing with run-tests.py SyntaxError occasionally in Jenkins

[ https://issues.apache.org/jira/browse/SPARK-21573?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16109881#comment-16109881 ] shane knapp commented on SPARK-21573: - ok, we pushed that PR. it should fix things. i also found

[jira] [Assigned] (SPARK-12717) pyspark broadcast fails when using multiple threads

[ https://issues.apache.org/jira/browse/SPARK-12717?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen reassigned SPARK-12717: - Assignee: Bryan Cutler [~hyukjin.kwon] I added you to the Committers group in JIRA, maybe that

[jira] [Commented] (SPARK-12717) pyspark broadcast fails when using multiple threads

[ https://issues.apache.org/jira/browse/SPARK-12717?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16109911#comment-16109911 ] Hyukjin Kwon commented on SPARK-12717: -- Issue resolved by pull request 18695

[jira] [Commented] (SPARK-21594) Probability output from MutilayerPerceptronClassifier

[ https://issues.apache.org/jira/browse/SPARK-21594?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16109017#comment-16109017 ] Joseph Wang commented on SPARK-21594: - Hi, I have tested with the Spark2.2.0 prebuilt release. This

[jira] [Updated] (SPARK-21594) Missing probability output from MutilayerPerceptronClassifier

[ https://issues.apache.org/jira/browse/SPARK-21594?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph Wang updated SPARK-21594: Summary: Missing probability output from MutilayerPerceptronClassifier (was: Probability output

[jira] [Updated] (SPARK-21595) introduction of spark.sql.windowExec.buffer.spill.threshold in spark 2.2 breaks existing workflow

[ https://issues.apache.org/jira/browse/SPARK-21595?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stephan Reiling updated SPARK-21595: Description: My pyspark code has the following statement: {code:java} # assign row key

[jira] [Created] (SPARK-21595) introduction of spark.sql.windowExec.buffer.spill.threshold in spark 2.2 breaks existing workflow

Stephan Reiling created SPARK-21595: --- Summary: introduction of spark.sql.windowExec.buffer.spill.threshold in spark 2.2 breaks existing workflow Key: SPARK-21595 URL:

[jira] [Commented] (SPARK-21563) Race condition when serializing TaskDescriptions and adding jars

[ https://issues.apache.org/jira/browse/SPARK-21563?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16109060#comment-16109060 ] Imran Rashid commented on SPARK-21563: -- Thanks for the detailed report [~aash], makes perfect sense.

[jira] [Updated] (SPARK-21563) Race condition when serializing TaskDescriptions and adding jars

[ https://issues.apache.org/jira/browse/SPARK-21563?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Imran Rashid updated SPARK-21563: - Component/s: Scheduler > Race condition when serializing TaskDescriptions and adding jars >

[jira] [Commented] (SPARK-21593) Fix broken configuration page

[ https://issues.apache.org/jira/browse/SPARK-21593?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16109101#comment-16109101 ] Artur Sukhenko commented on SPARK-21593: [~srowen] Yes, {code}

[jira] [Commented] (SPARK-21374) Reading globbed paths from S3 into DF doesn't work if filesystem caching is disabled

[ https://issues.apache.org/jira/browse/SPARK-21374?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16108897#comment-16108897 ] Steve Loughran commented on SPARK-21374: I understand...the patch shows the issue. Its only

[jira] [Commented] (SPARK-21585) Application Master marking application status as Failed for Client Mode

[ https://issues.apache.org/jira/browse/SPARK-21585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16108911#comment-16108911 ] Sean Owen commented on SPARK-21585: --- Yeah I've noticed -- not sure why or how to fix it. A PR can be

[jira] [Resolved] (SPARK-21585) Application Master marking application status as Failed for Client Mode

[ https://issues.apache.org/jira/browse/SPARK-21585?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-21585. --- Resolution: Fixed Assignee: Parth Gandhi Fix Version/s: 2.3.0 > Application

[jira] [Created] (SPARK-21593) Fix broken configuration page

Artur Sukhenko created SPARK-21593: -- Summary: Fix broken configuration page Key: SPARK-21593 URL: https://issues.apache.org/jira/browse/SPARK-21593 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-21593) Fix broken configuration page

[ https://issues.apache.org/jira/browse/SPARK-21593?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Artur Sukhenko updated SPARK-21593: --- Description: Latest configuration page for Spark 2.2.0 has broken menu list and named

[jira] [Updated] (SPARK-21593) Fix broken configuration page

[ https://issues.apache.org/jira/browse/SPARK-21593?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Artur Sukhenko updated SPARK-21593: --- Description: Latest configuration page for Spark 2.2.0 has broken menu list and named

[jira] [Commented] (SPARK-21593) Fix broken configuration page

[ https://issues.apache.org/jira/browse/SPARK-21593?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16108960#comment-16108960 ] Sean Owen commented on SPARK-21593: --- I couldn't make out what you were reporting from this descripiton,

[jira] [Updated] (SPARK-21593) Fix broken configuration page

[ https://issues.apache.org/jira/browse/SPARK-21593?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Artur Sukhenko updated SPARK-21593: --- Description: Latest configuration page for Spark 2.2.0 has broken menu list and named

[jira] [Updated] (SPARK-21591) Implement treeAggregate on Dataset API

[ https://issues.apache.org/jira/browse/SPARK-21591?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yanbo Liang updated SPARK-21591: Description: The Tungsten execution engine substantially improved the efficiency of memory and

[jira] [Commented] (SPARK-21390) Dataset filter api inconsistency

[ https://issues.apache.org/jira/browse/SPARK-21390?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16108880#comment-16108880 ] Vinod KC commented on SPARK-21390: -- [~kiszk]case class is compiled differently in the spark shell . For

[jira] [Commented] (SPARK-21514) Hive has updated with new support for S3 and InsertIntoHiveTable.scala should update also

[ https://issues.apache.org/jira/browse/SPARK-21514?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16108882#comment-16108882 ] Steve Loughran commented on SPARK-21514: Can you link this JIRA to the specific HIVE work? >

[jira] [Commented] (SPARK-21585) Application Master marking application status as Failed for Client Mode

[ https://issues.apache.org/jira/browse/SPARK-21585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16108901#comment-16108901 ] Thomas Graves commented on SPARK-21585: --- [~srowen]do you know if the github pull request link isn't

[jira] [Commented] (SPARK-21585) Application Master marking application status as Failed for Client Mode

[ https://issues.apache.org/jira/browse/SPARK-21585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16108913#comment-16108913 ] Thomas Graves commented on SPARK-21585: --- https://github.com/apache/spark/pull/18788 > Application

[jira] [Updated] (SPARK-21593) Fix broken configuration page

[ https://issues.apache.org/jira/browse/SPARK-21593?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Artur Sukhenko updated SPARK-21593: --- Description: Latest configuration page for Spark 2.2.0 has broken menu list and named

[jira] [Commented] (SPARK-21593) Fix broken configuration page

[ https://issues.apache.org/jira/browse/SPARK-21593?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16108966#comment-16108966 ] Artur Sukhenko commented on SPARK-21593: Yes, as well as having {code}### Dynamic

[jira] [Updated] (SPARK-21593) Fix broken configuration page

[ https://issues.apache.org/jira/browse/SPARK-21593?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Artur Sukhenko updated SPARK-21593: --- Description: Latest configuration page for Spark 2.2.0 has broken menu list and named

[jira] [Resolved] (SPARK-21594) Probability output from MutilayerPerceptronClassifier

[ https://issues.apache.org/jira/browse/SPARK-21594?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-21594. --- Resolution: Invalid This should go to u...@spark.apache.org > Probability output from

[jira] [Resolved] (SPARK-21388) GBT inherit from HasStepSize & LInearSVC/Binarizer from HasThreshold

[ https://issues.apache.org/jira/browse/SPARK-21388?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yanbo Liang resolved SPARK-21388. - Resolution: Fixed Assignee: zhengruifeng Fix Version/s: 2.3.0 > GBT inherit from

[jira] [Updated] (SPARK-21593) Fix broken configuration page

[ https://issues.apache.org/jira/browse/SPARK-21593?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Artur Sukhenko updated SPARK-21593: --- Attachment: doc_latest.png doc_211.png Latest documentation page with broken

[jira] [Updated] (SPARK-21593) Fix broken configuration page

[ https://issues.apache.org/jira/browse/SPARK-21593?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Artur Sukhenko updated SPARK-21593: --- Attachment: doc_latest.jpg doc_211.jpg dyn_211.jpg

[jira] [Updated] (SPARK-21593) Fix broken configuration page

[ https://issues.apache.org/jira/browse/SPARK-21593?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Artur Sukhenko updated SPARK-21593: --- Attachment: (was: doc_latest.png) > Fix broken configuration page >

[jira] [Updated] (SPARK-21593) Fix broken configuration page

[ https://issues.apache.org/jira/browse/SPARK-21593?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Artur Sukhenko updated SPARK-21593: --- Attachment: (was: doc_211.png) > Fix broken configuration page >

[jira] [Created] (SPARK-21594) Probability output from MutilayerPerceptronClassifier

Joseph Wang created SPARK-21594: --- Summary: Probability output from MutilayerPerceptronClassifier Key: SPARK-21594 URL: https://issues.apache.org/jira/browse/SPARK-21594 Project: Spark Issue

[jira] [Updated] (SPARK-21591) Implement treeAggregate on Dataset API

[ https://issues.apache.org/jira/browse/SPARK-21591?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yanbo Liang updated SPARK-21591: Description: The Tungsten execution engine substantially improved the efficiency of memory and

[jira] [Updated] (SPARK-21591) Implement treeAggregate on Dataset API

[ https://issues.apache.org/jira/browse/SPARK-21591?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yanbo Liang updated SPARK-21591: Description: The Tungsten execution engine substantially improved the efficiency of memory and

[jira] [Issue Comment Deleted] (SPARK-18535) Redact sensitive information from Spark logs and UI

[ https://issues.apache.org/jira/browse/SPARK-18535?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Diogo Munaro Vieira updated SPARK-18535: Comment: was deleted (was: I did a merge request for version 2.1.2:

[jira] [Updated] (SPARK-14516) Clustering evaluator

[ https://issues.apache.org/jira/browse/SPARK-14516?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yanbo Liang updated SPARK-14516: Priority: Major (was: Minor) > Clustering evaluator > > >

[jira] [Updated] (SPARK-14516) Clustering evaluator

[ https://issues.apache.org/jira/browse/SPARK-14516?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yanbo Liang updated SPARK-14516: Issue Type: New Feature (was: Brainstorming) > Clustering evaluator > > >

[jira] [Commented] (SPARK-18580) Use spark.streaming.backpressure.initialRate in DirectKafkaInputDStream

[ https://issues.apache.org/jira/browse/SPARK-18580?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16109265#comment-16109265 ] Oz Ben-Ami commented on SPARK-18580: +1 We're using our own dynamic allocation to scale with incoming

[jira] [Commented] (SPARK-21571) Spark history server leaves incomplete or unreadable history files around forever.

[ https://issues.apache.org/jira/browse/SPARK-21571?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16109282#comment-16109282 ] Eric Vandenberg commented on SPARK-21571: - Link to pull request

[jira] [Commented] (SPARK-21522) Flaky test: LauncherServerSuite.testStreamFiltering

[ https://issues.apache.org/jira/browse/SPARK-21522?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16109298#comment-16109298 ] Marcelo Vanzin commented on SPARK-21522: Since the bot didn't add the link:

  1   2   >