dongjoon-hyun commented on issue #24637: [SPARK-27707][SQL] Prune unnecessary
nested fields from Generate
URL: https://github.com/apache/spark/pull/24637#issuecomment-572144816
Could you file a JIRA for that, @cloud-fan ?
dongjoon-hyun commented on issue #24637: [SPARK-27707][SQL] Prune unnecessary
nested fields from Generate
URL: https://github.com/apache/spark/pull/24637#issuecomment-572144426
Thank you for reporting, @cloud-fan !
This is
dongjoon-hyun commented on issue #24637: [SPARK-27707][SQL] Prune unnecessary
nested fields from Generate
URL: https://github.com/apache/spark/pull/24637#issuecomment-512872770
Thank you for adding more. Then, please do a white list approach for those
four expressions as I mentioned here.
dongjoon-hyun commented on issue #24637: [SPARK-27707][SQL] Prune unnecessary
nested fields from Generate
URL: https://github.com/apache/spark/pull/24637#issuecomment-512358172
Thank you for adding a new test. Are you going to add more tests since
`Stack` is one of them? In fact, we need
dongjoon-hyun commented on issue #24637: [SPARK-27707][SQL] Prune unnecessary
nested fields from Generate
URL: https://github.com/apache/spark/pull/24637#issuecomment-510760456
Thank you so much, @viirya !
This is an
dongjoon-hyun commented on issue #24637: [SPARK-27707][SQL] Prune unnecessary
nested fields from Generate in explode
URL: https://github.com/apache/spark/pull/24637#issuecomment-510336289
I made a PR to you which reverts all the other benchmark result to the
master and adds the newly
dongjoon-hyun commented on issue #24637: [SPARK-27707][SQL] Prune unnecessary
nested fields from Generate in explode
URL: https://github.com/apache/spark/pull/24637#issuecomment-510330614
Weirdly. The benchmark suite failed with the exact same reason at the first
run. I'm running the
dongjoon-hyun commented on issue #24637: [SPARK-27707][SQL] Prune unnecessary
nested fields from Generate in explode
URL: https://github.com/apache/spark/pull/24637#issuecomment-510287567
Sure. I'll rerun and make a PR to you tonight.
dongjoon-hyun commented on issue #24637: [SPARK-27707][SQL] Prune unnecessary
nested fields from Generate in explode
URL: https://github.com/apache/spark/pull/24637#issuecomment-510266629
This is the result on this PR
-
dongjoon-hyun commented on issue #24637: [SPARK-27707][SQL] Prune unnecessary
nested fields from Generate in explode
URL: https://github.com/apache/spark/pull/24637#issuecomment-510253829
Oh, I faced the following during benchmark. It's weird. Let me rerun it.
```
[info] 22:18:13.279
dongjoon-hyun commented on issue #24637: [SPARK-27707][SQL] Prune unnecessary
nested fields from Generate to address performance issue in explode
URL: https://github.com/apache/spark/pull/24637#issuecomment-503380719
Retest this please.
dongjoon-hyun commented on issue #24637: [SPARK-27707][SQL] Prune unnecessary
nested fields from Generate to address performance issue in explode
URL: https://github.com/apache/spark/pull/24637#issuecomment-502417605
Retest this please.
dongjoon-hyun commented on issue #24637: [SPARK-27707][SQL] Prune unnecessary
nested fields from Generate to address performance issue in explode
URL: https://github.com/apache/spark/pull/24637#issuecomment-493838695
cc @dbtsai
13 matches
Mail list logo