[jira] [Commented] (SPARK-20112) SIGSEGV in GeneratedIterator.sort_addToSorter

2017-07-24 Thread Mitesh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20112?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16099030#comment-16099030 ] Mitesh commented on SPARK-20112: Still seeing this on 2.1.0, attached new err file > SIG

[jira] [Comment Edited] (SPARK-20112) SIGSEGV in GeneratedIterator.sort_addToSorter

2017-07-24 Thread Mitesh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20112?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16099030#comment-16099030 ] Mitesh edited comment on SPARK-20112 at 7/24/17 7:37 PM: - Still s

[jira] [Created] (SPARK-16398) Make cancelJob and cancelStage API public

2016-07-06 Thread Mitesh (JIRA)
Mitesh created SPARK-16398: -- Summary: Make cancelJob and cancelStage API public Key: SPARK-16398 URL: https://issues.apache.org/jira/browse/SPARK-16398 Project: Spark Issue Type: Improvement

[jira] [Updated] (SPARK-16398) Make cancelJob and cancelStage API public

2016-07-06 Thread Mitesh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16398?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mitesh updated SPARK-16398: --- Affects Version/s: 1.6.2 > Make cancelJob and cancelStage API public > --

[jira] [Updated] (SPARK-16398) Make cancelJob and cancelStage API public

2016-07-06 Thread Mitesh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16398?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mitesh updated SPARK-16398: --- Description: Make the {{SparkContext}} {{cancelJob}} and {{cancelStage}} APIs public. This allows application

[jira] [Updated] (SPARK-16398) Make cancelJob and cancelStage API public

2016-07-06 Thread Mitesh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16398?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mitesh updated SPARK-16398: --- Description: Make the SparkContext {{cancelJob}} and {{cancelStage}} APIs public. This allows applications to

[jira] [Updated] (SPARK-16398) Make cancelJob and cancelStage API public

2016-07-06 Thread Mitesh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16398?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mitesh updated SPARK-16398: --- Description: Make the SparkContext {{cancelJob}} and {{cancelStage}} APIs public. This allows applications to

[jira] [Updated] (SPARK-16419) EnsureRequirements adds extra Sort to already sorted cached table

2016-07-07 Thread Mitesh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16419?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mitesh updated SPARK-16419: --- Description: EnsureRequirements compares the required and given sort ordering, but uses Scala equals instead

[jira] [Created] (SPARK-16419) EnsureRequirements adds extra Sort to already sorted cached table

2016-07-07 Thread Mitesh (JIRA)
Mitesh created SPARK-16419: -- Summary: EnsureRequirements adds extra Sort to already sorted cached table Key: SPARK-16419 URL: https://issues.apache.org/jira/browse/SPARK-16419 Project: Spark Issue

[jira] [Comment Edited] (SPARK-13979) Killed executor is respawned without AWS keys in standalone spark cluster

2016-07-07 Thread Mitesh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13979?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15366459#comment-15366459 ] Mitesh edited comment on SPARK-13979 at 7/7/16 5:36 PM: Just to b

[jira] [Commented] (SPARK-13979) Killed executor is respawned without AWS keys in standalone spark cluster

2016-07-07 Thread Mitesh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13979?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15366459#comment-15366459 ] Mitesh commented on SPARK-13979: Just to be clear, you dont kill the worker process, you

[jira] [Commented] (SPARK-13979) Killed executor is respawned without AWS keys in standalone spark cluster

2016-07-23 Thread Mitesh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13979?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15390833#comment-15390833 ] Mitesh commented on SPARK-13979: Yeah that sounds like it. We worked around it by adding

[jira] [Commented] (SPARK-11033) Launcher: add support for monitoring standalone/cluster apps

2019-04-16 Thread Mitesh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11033?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16819178#comment-16819178 ] Mitesh commented on SPARK-11033: [~vanzin] Is there any plan to get this working? I'm on

[jira] [Comment Edited] (SPARK-11033) Launcher: add support for monitoring standalone/cluster apps

2019-04-16 Thread Mitesh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11033?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16819178#comment-16819178 ] Mitesh edited comment on SPARK-11033 at 4/16/19 3:58 PM: - [~vanz

[jira] [Comment Edited] (SPARK-11033) Launcher: add support for monitoring standalone/cluster apps

2019-04-16 Thread Mitesh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11033?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16819178#comment-16819178 ] Mitesh edited comment on SPARK-11033 at 4/16/19 3:58 PM: - [~vanz

[jira] [Comment Edited] (SPARK-11033) Launcher: add support for monitoring standalone/cluster apps

2019-04-16 Thread Mitesh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11033?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16819178#comment-16819178 ] Mitesh edited comment on SPARK-11033 at 4/16/19 4:00 PM: - [~vanz

[jira] [Commented] (SPARK-11033) Launcher: add support for monitoring standalone/cluster apps

2019-04-16 Thread Mitesh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11033?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16819270#comment-16819270 ] Mitesh commented on SPARK-11033: Thanks [~vanzin]! One quick question...is getting this

[jira] [Comment Edited] (SPARK-11033) Launcher: add support for monitoring standalone/cluster apps

2019-04-16 Thread Mitesh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11033?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16819270#comment-16819270 ] Mitesh edited comment on SPARK-11033 at 4/16/19 4:59 PM: - Thanks

[jira] [Commented] (SPARK-19468) Dataset slow because of unnecessary shuffles

2019-02-05 Thread Mitesh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16761207#comment-16761207 ] Mitesh commented on SPARK-19468: +1 I'm seeing the same behavior. It seems like any phys

[jira] [Comment Edited] (SPARK-19468) Dataset slow because of unnecessary shuffles

2019-02-05 Thread Mitesh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16761207#comment-16761207 ] Mitesh edited comment on SPARK-19468 at 2/5/19 8:59 PM: +1 I'm s

[jira] [Comment Edited] (SPARK-19468) Dataset slow because of unnecessary shuffles

2019-02-05 Thread Mitesh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16761207#comment-16761207 ] Mitesh edited comment on SPARK-19468 at 2/5/19 8:59 PM: +1 I'm s

[jira] [Commented] (SPARK-19468) Dataset slow because of unnecessary shuffles

2019-02-05 Thread Mitesh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16761262#comment-16761262 ] Mitesh commented on SPARK-19468: Also curious why in the fix for SPARK-19931, it was onl

[jira] [Commented] (SPARK-19468) Dataset slow because of unnecessary shuffles

2019-02-05 Thread Mitesh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16761309#comment-16761309 ] Mitesh commented on SPARK-19468: Also this may be a dupe of SPARK-19981 > Dataset slow

[jira] [Commented] (SPARK-17636) Parquet predicate pushdown for nested fields

2019-02-05 Thread Mitesh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17636?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16761311#comment-16761311 ] Mitesh commented on SPARK-17636: Should this be closed, as a duplicate of SPARK-4502? >

[jira] [Commented] (SPARK-19981) Sort-Merge join inserts shuffles when joining dataframes with aliased columns

2019-02-05 Thread Mitesh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19981?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16761535#comment-16761535 ] Mitesh commented on SPARK-19981: Ping any updates here? This still is an issue in 2.3.2.

[jira] [Comment Edited] (SPARK-19981) Sort-Merge join inserts shuffles when joining dataframes with aliased columns

2019-02-05 Thread Mitesh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19981?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16761535#comment-16761535 ] Mitesh edited comment on SPARK-19981 at 2/6/19 6:54 AM: Ping any

[jira] [Comment Edited] (SPARK-19981) Sort-Merge join inserts shuffles when joining dataframes with aliased columns

2019-02-06 Thread Mitesh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19981?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16761535#comment-16761535 ] Mitesh edited comment on SPARK-19981 at 2/6/19 7:12 PM: Ping [~m

[jira] [Comment Edited] (SPARK-19468) Dataset slow because of unnecessary shuffles

2019-02-06 Thread Mitesh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16761207#comment-16761207 ] Mitesh edited comment on SPARK-19468 at 2/6/19 7:13 PM: +1 I'm s

[jira] [Comment Edited] (SPARK-19468) Dataset slow because of unnecessary shuffles

2019-02-06 Thread Mitesh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16761207#comment-16761207 ] Mitesh edited comment on SPARK-19468 at 2/6/19 7:13 PM: I'm seei

[jira] [Created] (SPARK-17636) Parquet filter push down doesn't handle struct fields

2016-09-22 Thread Mitesh (JIRA)
Mitesh created SPARK-17636: -- Summary: Parquet filter push down doesn't handle struct fields Key: SPARK-17636 URL: https://issues.apache.org/jira/browse/SPARK-17636 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-17636) Parquet filter push down doesn't handle struct fields

2016-09-22 Thread Mitesh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17636?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mitesh updated SPARK-17636: --- Description: The filter gets pushed down for a simple numeric field, but not for a numeric field inside a st

[jira] [Updated] (SPARK-17636) Parquet filter push down doesn't handle struct fields

2016-09-22 Thread Mitesh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17636?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mitesh updated SPARK-17636: --- Description: The filter gets pushed down for a simple numeric field, but not for a numeric field inside a st

[jira] [Updated] (SPARK-17636) Parquet filter push down doesn't handle struct fields

2016-09-22 Thread Mitesh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17636?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mitesh updated SPARK-17636: --- Description: The filter gets pushed down for a simple numeric field, but not for a numeric field inside a st

[jira] [Updated] (SPARK-17636) Parquet filter push down doesn't handle struct fields

2016-09-22 Thread Mitesh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17636?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mitesh updated SPARK-17636: --- Description: Theres a *PushedFilters* for a simple numeric field, but not for a numeric field inside a struc

[jira] [Updated] (SPARK-17636) Parquet filter push down doesn't handle struct fields

2016-09-22 Thread Mitesh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17636?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mitesh updated SPARK-17636: --- Description: Theres a `PushedFilters` for a simple numeric field, but not for a numeric field inside a struc

[jira] [Updated] (SPARK-17636) Parquet filter push down doesn't handle struct fields

2016-09-22 Thread Mitesh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17636?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mitesh updated SPARK-17636: --- Priority: Minor (was: Major) > Parquet filter push down doesn't handle struct fields > -

[jira] [Updated] (SPARK-17636) Parquet filter push down doesn't handle struct fields

2016-09-23 Thread Mitesh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17636?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mitesh updated SPARK-17636: --- Affects Version/s: 1.6.3 > Parquet filter push down doesn't handle struct fields > --

[jira] [Commented] (SPARK-17636) Parquet filter push down doesn't handle struct fields

2016-10-10 Thread Mitesh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17636?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15562757#comment-15562757 ] Mitesh commented on SPARK-17636: [~liancheng] Could you or someone else familiar with the

[jira] [Commented] (SPARK-13979) Killed executor is respawned without AWS keys in standalone spark cluster

2016-03-19 Thread Mitesh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13979?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15200183#comment-15200183 ] Mitesh commented on SPARK-13979: I'm seeing this too. Its really annoying because I set t

[jira] [Comment Edited] (SPARK-13979) Killed executor is respawned without AWS keys in standalone spark cluster

2016-03-20 Thread Mitesh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13979?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15200183#comment-15200183 ] Mitesh edited comment on SPARK-13979 at 3/17/16 7:10 PM: - I'm see

[jira] [Comment Edited] (SPARK-13979) Killed executor is respawned without AWS keys in standalone spark cluster

2016-03-20 Thread Mitesh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13979?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15200183#comment-15200183 ] Mitesh edited comment on SPARK-13979 at 3/17/16 7:09 PM: - I'm see

[jira] [Commented] (SPARK-17636) Parquet predicate pushdown for nested fields

2020-03-31 Thread Mitesh (Jira)
[ https://issues.apache.org/jira/browse/SPARK-17636?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17072119#comment-17072119 ] Mitesh commented on SPARK-17636: [~cloud_fan] [~dbtsai] thanks for fixing! Is there any

[jira] [Commented] (SPARK-39441) Speed up DeduplicateRelations

2023-07-31 Thread Mitesh (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39441?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17749321#comment-17749321 ] Mitesh commented on SPARK-39441: After applying this fix to 3.3.2, I still see some slow

[jira] [Comment Edited] (SPARK-39441) Speed up DeduplicateRelations

2023-07-31 Thread Mitesh (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39441?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17749321#comment-17749321 ] Mitesh edited comment on SPARK-39441 at 7/31/23 7:40 PM: - After

[jira] [Comment Edited] (SPARK-39441) Speed up DeduplicateRelations

2023-07-31 Thread Mitesh (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39441?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17749321#comment-17749321 ] Mitesh edited comment on SPARK-39441 at 7/31/23 9:03 PM: - After

[jira] [Commented] (SPARK-23643) XORShiftRandom.hashSeed allocates unnecessary memory

2023-08-24 Thread Mitesh (Jira)
[ https://issues.apache.org/jira/browse/SPARK-23643?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17758770#comment-17758770 ] Mitesh commented on SPARK-23643: +1 can we please document this in the ML migration guid

[jira] [Comment Edited] (SPARK-23643) XORShiftRandom.hashSeed allocates unnecessary memory

2023-08-24 Thread Mitesh (Jira)
[ https://issues.apache.org/jira/browse/SPARK-23643?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17758770#comment-17758770 ] Mitesh edited comment on SPARK-23643 at 8/24/23 11:02 PM: -- +1 c

[jira] [Comment Edited] (SPARK-23643) XORShiftRandom.hashSeed allocates unnecessary memory

2023-08-24 Thread Mitesh (Jira)
[ https://issues.apache.org/jira/browse/SPARK-23643?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17758770#comment-17758770 ] Mitesh edited comment on SPARK-23643 at 8/24/23 11:17 PM: -- +1 c

[jira] [Commented] (SPARK-30602) SPIP: Support push-based shuffle to improve shuffle efficiency

2023-11-14 Thread Mitesh (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30602?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17785989#comment-17785989 ] Mitesh commented on SPARK-30602: Is there a plan to support push-based shuffle on Spark

[jira] [Comment Edited] (SPARK-30602) SPIP: Support push-based shuffle to improve shuffle efficiency

2023-11-14 Thread Mitesh (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30602?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17785989#comment-17785989 ] Mitesh edited comment on SPARK-30602 at 11/14/23 7:34 PM: -- Is t

[jira] [Comment Edited] (SPARK-39441) Speed up DeduplicateRelations

2024-02-05 Thread Mitesh (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39441?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17749321#comment-17749321 ] Mitesh edited comment on SPARK-39441 at 2/5/24 7:00 PM: After ap

[jira] [Comment Edited] (SPARK-39441) Speed up DeduplicateRelations

2024-02-05 Thread Mitesh (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39441?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17749321#comment-17749321 ] Mitesh edited comment on SPARK-39441 at 2/5/24 7:01 PM: After ap

[jira] [Comment Edited] (SPARK-39441) Speed up DeduplicateRelations

2024-02-05 Thread Mitesh (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39441?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17749321#comment-17749321 ] Mitesh edited comment on SPARK-39441 at 2/5/24 11:28 PM: - After

[jira] [Commented] (SPARK-10970) Executors overload Hive metastore by making massive connections at execution time

2022-08-03 Thread Mitesh (Jira)
[ https://issues.apache.org/jira/browse/SPARK-10970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17575014#comment-17575014 ] Mitesh commented on SPARK-10970: [~cheolsoo] I'm seeing this on Spark 2.4, here is my ca

[jira] [Updated] (SPARK-17636) Parquet filter push down doesn't handle struct fields

2016-12-02 Thread Mitesh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17636?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mitesh updated SPARK-17636: --- Affects Version/s: 2.0.2 > Parquet filter push down doesn't handle struct fields > --

[jira] [Commented] (SPARK-13979) Killed executor is respawned without AWS keys in standalone spark cluster

2016-05-31 Thread Mitesh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13979?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15308608#comment-15308608 ] Mitesh commented on SPARK-13979: [~gvernik] you can just `kill` the executor on the comma

[jira] [Created] (SPARK-20112) SIGSEGV in GeneratedIterator.sort_addToSorter

2017-03-27 Thread Mitesh (JIRA)
Mitesh created SPARK-20112: -- Summary: SIGSEGV in GeneratedIterator.sort_addToSorter Key: SPARK-20112 URL: https://issues.apache.org/jira/browse/SPARK-20112 Project: Spark Issue Type: Bug C

[jira] [Updated] (SPARK-20112) SIGSEGV in GeneratedIterator.sort_addToSorter

2017-03-27 Thread Mitesh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20112?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mitesh updated SPARK-20112: --- Attachment: hs_err_pid19271.log codegen_sorter_crash > SIGSEGV in GeneratedIterator.sort_addT

[jira] [Updated] (SPARK-20112) SIGSEGV in GeneratedIterator.sort_addToSorter

2017-03-27 Thread Mitesh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20112?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mitesh updated SPARK-20112: --- Attachment: codegen_sorter_crash.log > SIGSEGV in GeneratedIterator.sort_addToSorter > --

[jira] [Updated] (SPARK-20112) SIGSEGV in GeneratedIterator.sort_addToSorter

2017-03-27 Thread Mitesh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20112?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mitesh updated SPARK-20112: --- Attachment: (was: codegen_sorter_crash) > SIGSEGV in GeneratedIterator.sort_addToSorter > ---

[jira] [Updated] (SPARK-20112) SIGSEGV in GeneratedIterator.sort_addToSorter

2017-03-27 Thread Mitesh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20112?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mitesh updated SPARK-20112: --- Description: I'm seeing a very weird crash in {{GeneratedIterator.sort_addToSorter}}. The hs_err_pid and cod

[jira] [Updated] (SPARK-20112) SIGSEGV in GeneratedIterator.sort_addToSorter

2017-03-27 Thread Mitesh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20112?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mitesh updated SPARK-20112: --- Description: I'm seeing a very weird crash in {{GeneratedIterator.sort_addToSorter}}. The hs_err_pid and cod

[jira] [Updated] (SPARK-20112) SIGSEGV in GeneratedIterator.sort_addToSorter

2017-03-27 Thread Mitesh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20112?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mitesh updated SPARK-20112: --- Attachment: (was: codegen_sorter_crash.log) > SIGSEGV in GeneratedIterator.sort_addToSorter > ---

[jira] [Updated] (SPARK-20112) SIGSEGV in GeneratedIterator.sort_addToSorter

2017-03-27 Thread Mitesh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20112?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mitesh updated SPARK-20112: --- Attachment: codegen_sorter_crash.log > SIGSEGV in GeneratedIterator.sort_addToSorter > --

[jira] [Commented] (SPARK-20112) SIGSEGV in GeneratedIterator.sort_addToSorter

2017-03-27 Thread Mitesh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20112?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15944100#comment-15944100 ] Mitesh commented on SPARK-20112: This kind of looks like https://issues.apache.org/jira/b

[jira] [Issue Comment Deleted] (SPARK-20112) SIGSEGV in GeneratedIterator.sort_addToSorter

2017-03-27 Thread Mitesh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20112?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mitesh updated SPARK-20112: --- Comment: was deleted (was: This kind of looks like https://issues.apache.org/jira/browse/SPARK-15822, but th

[jira] [Updated] (SPARK-20112) SIGSEGV in GeneratedIterator.sort_addToSorter

2017-03-27 Thread Mitesh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20112?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mitesh updated SPARK-20112: --- Description: I'm seeing a very weird crash in {{GeneratedIterator.sort_addToSorter}}. The hs_err_pid and cod

[jira] [Updated] (SPARK-20112) SIGSEGV in GeneratedIterator.sort_addToSorter

2017-03-27 Thread Mitesh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20112?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mitesh updated SPARK-20112: --- Description: I'm seeing a very weird crash in {{GeneratedIterator.sort_addToSorter}}. The hs_err_pid and cod

[jira] [Commented] (SPARK-19981) Sort-Merge join inserts shuffles when joining dataframes with aliased columns

2017-03-28 Thread Mitesh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19981?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15945166#comment-15945166 ] Mitesh commented on SPARK-19981: As I mentioned on the PR, this seems like it should be h

[jira] [Comment Edited] (SPARK-19981) Sort-Merge join inserts shuffles when joining dataframes with aliased columns

2017-03-28 Thread Mitesh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19981?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15945166#comment-15945166 ] Mitesh edited comment on SPARK-19981 at 3/28/17 1:33 PM: - As I me

[jira] [Comment Edited] (SPARK-19981) Sort-Merge join inserts shuffles when joining dataframes with aliased columns

2017-03-28 Thread Mitesh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19981?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15945166#comment-15945166 ] Mitesh edited comment on SPARK-19981 at 3/28/17 1:34 PM: - As I me

[jira] [Comment Edited] (SPARK-19981) Sort-Merge join inserts shuffles when joining dataframes with aliased columns

2017-03-28 Thread Mitesh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19981?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15945166#comment-15945166 ] Mitesh edited comment on SPARK-19981 at 3/28/17 1:34 PM: - As I me

[jira] [Comment Edited] (SPARK-19981) Sort-Merge join inserts shuffles when joining dataframes with aliased columns

2017-03-28 Thread Mitesh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19981?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15945166#comment-15945166 ] Mitesh edited comment on SPARK-19981 at 3/28/17 1:34 PM: - As I me

[jira] [Comment Edited] (SPARK-19981) Sort-Merge join inserts shuffles when joining dataframes with aliased columns

2017-03-28 Thread Mitesh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19981?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15945166#comment-15945166 ] Mitesh edited comment on SPARK-19981 at 3/28/17 1:35 PM: - As I me

[jira] [Comment Edited] (SPARK-19981) Sort-Merge join inserts shuffles when joining dataframes with aliased columns

2017-03-28 Thread Mitesh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19981?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15945166#comment-15945166 ] Mitesh edited comment on SPARK-19981 at 3/28/17 3:21 PM: - As I me

[jira] [Updated] (SPARK-20112) SIGSEGV in GeneratedIterator.sort_addToSorter

2017-03-28 Thread Mitesh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20112?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mitesh updated SPARK-20112: --- Attachment: hs_err_pid22870.log > SIGSEGV in GeneratedIterator.sort_addToSorter > ---

[jira] [Commented] (SPARK-20112) SIGSEGV in GeneratedIterator.sort_addToSorter

2017-03-28 Thread Mitesh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20112?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15945399#comment-15945399 ] Mitesh commented on SPARK-20112: [~kiszk] I can try out spark 2.0.3+ or 2.1. Actually I d

[jira] [Comment Edited] (SPARK-20112) SIGSEGV in GeneratedIterator.sort_addToSorter

2017-03-28 Thread Mitesh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20112?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15945399#comment-15945399 ] Mitesh edited comment on SPARK-20112 at 3/28/17 3:40 PM: - [~kiszk

[jira] [Comment Edited] (SPARK-20112) SIGSEGV in GeneratedIterator.sort_addToSorter

2017-03-28 Thread Mitesh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20112?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15945399#comment-15945399 ] Mitesh edited comment on SPARK-20112 at 3/28/17 3:46 PM: - [~kiszk

[jira] [Comment Edited] (SPARK-20112) SIGSEGV in GeneratedIterator.sort_addToSorter

2017-03-28 Thread Mitesh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20112?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15945399#comment-15945399 ] Mitesh edited comment on SPARK-20112 at 3/28/17 3:46 PM: - [~kiszk

[jira] [Comment Edited] (SPARK-17867) Dataset.dropDuplicates (i.e. distinct) should consider the columns with same column name

2017-05-18 Thread Mitesh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17867?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16015772#comment-16015772 ] Mitesh edited comment on SPARK-17867 at 5/18/17 1:47 PM: - I'm see

[jira] [Commented] (SPARK-17867) Dataset.dropDuplicates (i.e. distinct) should consider the columns with same column name

2017-05-18 Thread Mitesh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17867?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16015772#comment-16015772 ] Mitesh commented on SPARK-17867: I'm seeing a regression from this change, the last filte

[jira] [Comment Edited] (SPARK-17867) Dataset.dropDuplicates (i.e. distinct) should consider the columns with same column name

2017-05-18 Thread Mitesh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17867?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16015772#comment-16015772 ] Mitesh edited comment on SPARK-17867 at 5/18/17 1:49 PM: - I'm see

[jira] [Comment Edited] (SPARK-17867) Dataset.dropDuplicates (i.e. distinct) should consider the columns with same column name

2017-05-18 Thread Mitesh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17867?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16015772#comment-16015772 ] Mitesh edited comment on SPARK-17867 at 5/18/17 1:48 PM: - I'm see

[jira] [Comment Edited] (SPARK-17867) Dataset.dropDuplicates (i.e. distinct) should consider the columns with same column name

2017-05-18 Thread Mitesh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17867?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16015772#comment-16015772 ] Mitesh edited comment on SPARK-17867 at 5/18/17 1:48 PM: - I'm see

[jira] [Comment Edited] (SPARK-17867) Dataset.dropDuplicates (i.e. distinct) should consider the columns with same column name

2017-05-18 Thread Mitesh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17867?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16015772#comment-16015772 ] Mitesh edited comment on SPARK-17867 at 5/18/17 1:48 PM: - I'm see

[jira] [Commented] (SPARK-17867) Dataset.dropDuplicates (i.e. distinct) should consider the columns with same column name

2017-05-18 Thread Mitesh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17867?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16016021#comment-16016021 ] Mitesh commented on SPARK-17867: Ah I see, thanks [~viirya]. The repartitionByColumns is