Re: [SS] FlatMapGroupsWithStateExec with no commitTimeMs metric?

2018-11-25 Thread Jungtaek Lim
Just filed a new JIRA issue [1] as well as PR [2]. - Jungtaek Lim (HeartSaVioR) 1. https://issues.apache.org/jira/browse/SPARK-26170 2. https://github.com/apache/spark/pull/23142 2018년 11월 26일 (월) 오후 3:26, Jungtaek Lim 님이 작성: > Looks like just a kind of missing spot. Just crafted the patch

Re: [SS] FlatMapGroupsWithStateExec with no commitTimeMs metric?

2018-11-25 Thread Jungtaek Lim
Looks like just a kind of missing spot. Just crafted the patch and now in progress of testing. Once done with testing I'll file a new issue and submit a patch. Thanks, Jungtaek Lim (HeartSaVioR) 2018년 11월 26일 (월) 오전 4:49, Burak Yavuz 님이 작성: > Probably just oversight. Anyone is welcome to add

Re: [SS] FlatMapGroupsWithStateExec with no commitTimeMs metric?

2018-11-25 Thread Burak Yavuz
Probably just oversight. Anyone is welcome to add it :) On Sun, Nov 25, 2018 at 8:55 AM Jacek Laskowski wrote: > Hi, > > Why is FlatMapGroupsWithStateExec not measuring the time taken on state > commit [1](like StreamingDeduplicateExec [2] and StreamingGlobalLimitExec > [3])? Is this on

[SS] FlatMapGroupsWithStateExec with no commitTimeMs metric?

2018-11-25 Thread Jacek Laskowski
Hi, Why is FlatMapGroupsWithStateExec not measuring the time taken on state commit [1](like StreamingDeduplicateExec [2] and StreamingGlobalLimitExec [3])? Is this on purpose? Why? Thanks. [1]