[jira] [Commented] (SPARK-6305) Add support for log4j 2.x to Spark

[ https://issues.apache.org/jira/browse/SPARK-6305?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14362996#comment-14362996 ] Lior Chaga commented on SPARK-6305: --- Works by adding log4j 2.x jars with log4j1.2-api

[jira] [Created] (SPARK-7032) SparkSQL incorrect results when using UNION/EXCEPT with GROUP BY clause

Lior Chaga created SPARK-7032: - Summary: SparkSQL incorrect results when using UNION/EXCEPT with GROUP BY clause Key: SPARK-7032 URL: https://issues.apache.org/jira/browse/SPARK-7032 Project: Spark

[jira] [Commented] (SPARK-7032) SparkSQL incorrect results when using UNION/EXCEPT with GROUP BY clause

[ https://issues.apache.org/jira/browse/SPARK-7032?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14512951#comment-14512951 ] Lior Chaga commented on SPARK-7032: --- I noticed that if I use the same group by for both

[jira] [Updated] (SPARK-7032) SparkSQL incorrect results when using UNION/EXCEPT with GROUP BY clause

[ https://issues.apache.org/jira/browse/SPARK-7032?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lior Chaga updated SPARK-7032: -- Description: When using UNION/EXCEPT clause with GROUP BY clause in spark sql, results do not match

[jira] [Updated] (SPARK-7032) SparkSQL incorrect results when using UNION/EXCEPT with GROUP BY clause

[ https://issues.apache.org/jira/browse/SPARK-7032?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lior Chaga updated SPARK-7032: -- Description: When using UNION/EXCEPT clause with GROUP BY clause in spark sql, results do not

[jira] [Closed] (SPARK-7032) SparkSQL incorrect results when using UNION/EXCEPT with GROUP BY clause

[ https://issues.apache.org/jira/browse/SPARK-7032?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lior Chaga closed SPARK-7032. - Resolution: Not A Problem SparkSQL incorrect results when using UNION/EXCEPT with GROUP BY clause

[jira] [Commented] (SPARK-7032) SparkSQL incorrect results when using UNION/EXCEPT with GROUP BY clause

[ https://issues.apache.org/jira/browse/SPARK-7032?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14562641#comment-14562641 ] Lior Chaga commented on SPARK-7032: --- [~viirya], the following modification to the code

[jira] [Commented] (SPARK-7032) SparkSQL incorrect results when using UNION/EXCEPT with GROUP BY clause

[ https://issues.apache.org/jira/browse/SPARK-7032?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14562634#comment-14562634 ] Lior Chaga commented on SPARK-7032: --- You are right. The first example with the explicit

[jira] [Commented] (SPARK-7032) SparkSQL incorrect results when using UNION/EXCEPT with GROUP BY clause

[ https://issues.apache.org/jira/browse/SPARK-7032?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14558803#comment-14558803 ] Lior Chaga commented on SPARK-7032: --- [~viirya], the problem still occurs with latest

[jira] [Commented] (SPARK-7032) SparkSQL incorrect results when using UNION/EXCEPT with GROUP BY clause

[ https://issues.apache.org/jira/browse/SPARK-7032?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14558989#comment-14558989 ] Lior Chaga commented on SPARK-7032: --- Thanks, [~viirya]. I'm not sure this is the problem

[jira] [Commented] (SPARK-4300) Race condition during SparkWorker shutdown

[ https://issues.apache.org/jira/browse/SPARK-4300?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14680154#comment-14680154 ] Lior Chaga commented on SPARK-4300: --- Also exists in spark 1.4: {panel} 12:31:10.821

[jira] [Commented] (SPARK-9096) Unevenly distributed task loads after using JavaRDD.subtract()

[ https://issues.apache.org/jira/browse/SPARK-9096?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14732230#comment-14732230 ] Lior Chaga commented on SPARK-9096: --- I would suggest the following alternative to changing partitioner:

[jira] [Commented] (SPARK-14919) Spark Cannot be used with software that requires jackson-databind 2.6+: RDDOperationScope

[ https://issues.apache.org/jira/browse/SPARK-14919?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15307622#comment-15307622 ] Lior Chaga commented on SPARK-14919: [~srowen], I totally agree that thing get done by people. But

[jira] [Commented] (SPARK-14919) Spark Cannot be used with software that requires jackson-databind 2.6+: RDDOperationScope

[ https://issues.apache.org/jira/browse/SPARK-14919?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15307631#comment-15307631 ] Lior Chaga commented on SPARK-14919: Well, my experience with user classpath first is not so good. I

[jira] [Commented] (SPARK-14919) Spark Cannot be used with software that requires jackson-databind 2.6+: RDDOperationScope

[ https://issues.apache.org/jira/browse/SPARK-14919?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15307606#comment-15307606 ] Lior Chaga commented on SPARK-14919: I totally agree with [~fergjo00], if shading is so no un-common,

[jira] [Created] (SPARK-13567) Document spark.shuffle.reduceLocality.enabled configuration

Lior Chaga created SPARK-13567: -- Summary: Document spark.shuffle.reduceLocality.enabled configuration Key: SPARK-13567 URL: https://issues.apache.org/jira/browse/SPARK-13567 Project: Spark

[jira] [Commented] (SPARK-13567) Document spark.shuffle.reduceLocality.enabled configuration

[ https://issues.apache.org/jira/browse/SPARK-13567?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15171887#comment-15171887 ] Lior Chaga commented on SPARK-13567: Well, given my findings (see SPARK-10567), I think it should, at

[jira] [Commented] (SPARK-10567) Reducer locality follow-up for Spark 1.6

[ https://issues.apache.org/jira/browse/SPARK-10567?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15171879#comment-15171879 ] Lior Chaga commented on SPARK-10567: Hi, I have a spark job that does the following: Given a list of

[jira] [Commented] (SPARK-12836) spark enable both driver run executor & write to HDFS

[ https://issues.apache.org/jira/browse/SPARK-12836?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15197120#comment-15197120 ] Lior Chaga commented on SPARK-12836: I used the --no-switch_user mesos config, and it worked. Writing

[jira] [Commented] (SPARK-21795) Broadcast hint ignored when dataframe is cached

[ https://issues.apache.org/jira/browse/SPARK-21795?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16136891#comment-16136891 ] Lior Chaga commented on SPARK-21795: Hi, what I say is that if I use a hint to broadcast join on a

[jira] [Updated] (SPARK-21795) Broadcast hint ignored when dataframe is cached

[ https://issues.apache.org/jira/browse/SPARK-21795?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lior Chaga updated SPARK-21795: --- Description: Not sure if it's a bug or by design, but if a DF is cached, the broadcast hint is

[jira] [Created] (SPARK-21795) Broadcast hint ignored when dataframe is cached

Lior Chaga created SPARK-21795: -- Summary: Broadcast hint ignored when dataframe is cached Key: SPARK-21795 URL: https://issues.apache.org/jira/browse/SPARK-21795 Project: Spark Issue Type: Bug

[jira] [Created] (SPARK-22840) Incorrect results when using distinct on window

Lior Chaga created SPARK-22840: -- Summary: Incorrect results when using distinct on window Key: SPARK-22840 URL: https://issues.apache.org/jira/browse/SPARK-22840 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-22840) Incorrect results when using distinct on window

[ https://issues.apache.org/jira/browse/SPARK-22840?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lior Chaga updated SPARK-22840: --- Description: Given the following schema: {code} root |-- id: string (nullable = true) |--

[jira] [Updated] (SPARK-22840) Incorrect results when using distinct on window

[ https://issues.apache.org/jira/browse/SPARK-22840?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lior Chaga updated SPARK-22840: --- Attachment: sample.parquet.zip > Incorrect results when using distinct on window >

[jira] [Commented] (SPARK-22840) Incorrect results when using distinct on window

[ https://issues.apache.org/jira/browse/SPARK-22840?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16303284#comment-16303284 ] Lior Chaga commented on SPARK-22840: gotcha [~greenhat], thanks > Incorrect results when using

[jira] [Resolved] (SPARK-22840) Incorrect results when using distinct on window

[ https://issues.apache.org/jira/browse/SPARK-22840?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lior Chaga resolved SPARK-22840. Resolution: Not A Bug > Incorrect results when using distinct on window >

[jira] [Commented] (SPARK-24906) Adaptively set split size for columnar file to ensure the task read data size fit expectation

[ https://issues.apache.org/jira/browse/SPARK-24906?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17006345#comment-17006345 ] Lior Chaga commented on SPARK-24906: [~habren] The suggested approach of using a multiplier will be

[jira] [Commented] (SPARK-24906) Adaptively set split size for columnar file to ensure the task read data size fit expectation

[ https://issues.apache.org/jira/browse/SPARK-24906?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17006691#comment-17006691 ] Lior Chaga commented on SPARK-24906: Looking at the PR, I see only using configurable estimations

[jira] [Updated] (SPARK-35508) job group and description do not apply on broadcasts

[ https://issues.apache.org/jira/browse/SPARK-35508?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lior Chaga updated SPARK-35508: --- Description: Given the following code: {code:java} SparkContext context = new SparkContext("local",

[jira] [Created] (SPARK-35508) job group and description do not apply on broadcasts

Lior Chaga created SPARK-35508: -- Summary: job group and description do not apply on broadcasts Key: SPARK-35508 URL: https://issues.apache.org/jira/browse/SPARK-35508 Project: Spark Issue Type:

[jira] [Updated] (SPARK-35508) job group and description do not apply on broadcasts

[ https://issues.apache.org/jira/browse/SPARK-35508?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lior Chaga updated SPARK-35508: --- Attachment: spark2-image.png > job group and description do not apply on broadcasts >

[jira] [Updated] (SPARK-35508) job group and description do not apply on broadcasts

[ https://issues.apache.org/jira/browse/SPARK-35508?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lior Chaga updated SPARK-35508: --- Attachment: spark3-image.png > job group and description do not apply on broadcasts >

[jira] [Updated] (SPARK-35508) job group and description do not apply on broadcasts

[ https://issues.apache.org/jira/browse/SPARK-35508?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lior Chaga updated SPARK-35508: --- Description: Given the following code: {code:java} SparkContext context = new SparkContext("local",