Sorry for the late reply. Agree to split this into sub tasks. For RESTART EVENTPROCESSOR IN INVALIDATE MODE, what I'm thinking is falling back to the old behavior of EventProcessor that just invalidates the tables for ALTER_TABLE events, i.e. the behavior before IMPALA-9101. The current behavior can be considered as the RELOAD MODE that EventProcessor reloads metadata for loaded tables. Though we have tried to skip reloading unnecessary metadata (e.g. IMPALA-11534), EventProcessor could still lag behind when reloading large tables. Falling back to the INVALIDATE mode will let EventProcessor catch up quickly. After that, admins can switch back to the RELOAD mode.
On Tue, Feb 27, 2024 at 3:23 PM guo Maxwell <cclive1...@gmail.com> wrote: > Hi Quanlong， > I think the grammar for RESTART EVENTPROCESSOR is ok , > I want to discuss some details with you： > (1) RESTART EVENTPROCESSOR , means just restart the processor from the > current event id , not matter the event status is NEEDS_INVALIDATE or > STOPPED; > (2) RESTART EVENTPROCESSOR FROM evnet id , means restart the process or > from the specified event id . > But I am not clear with RESTART EVENTPROCESSOR IN INVALIDATE AND RELOAD > MODE means ? > > can you explain it to me? > Besides, I think the issue may need to be split into some sub tasks > together with IMPALA-12506 > <https://issues.apache.org/jira/browse/IMPALA-12506> , Do you agree with > me ? > > > guo Maxwell <cclive1...@gmail.com> 于2023年10月19日周四 11:12写道： > >> I noticed that we have a sql AdminFnStmt >> <https://github.com/apache/impala/blob/master/fe/src/main/cup/sql-parser.cup#L3115> >> ,and >> it seems this is just for admin commands through functions, even >> though only shutdown function is supported ,So what about extend this >> syntax ?WDYT? >> e.g. set_event_status (event_id, status). >> >> Of course, webUi must also be modified. >> >> But all users that can view the WebUI can perform the operations. We >>> currently have the same issue for canceling queries or adjusting the log >>> level in WebUI. >> >> This is a problem, but I feel that it may not have much to do with the >> function itself. It may be necessary to make an authentication design for >> weiUi. >> >> guo Maxwell <cclive1...@gmail.com> 于2023年10月19日周四 11:01写道： >> >>> Thanks quanlong, >>> yeah, It seems I missed.😄 >>> >>> Quanlong Huang <huangquanl...@gmail.com> 于2023年10月19日周四 10:16写道： >>> >>>> Hi, I had a reply but maybe you missed it: >>>> https://lists.apache.org/thread/5mw8jd5hgz7yycz9h2pxvqj101k0j47m >>>> Feel free to file JIRAs and welcome to work on these! >>>> >>>> Thanks, >>>> Quanlong >>>> >>>> On Wed, Oct 18, 2023 at 1:59 PM guo Maxwell <cclive1...@gmail.com> >>>> wrote: >>>> >>>>> Hello everyone, >>>>> It’s been a while since my last email. And any feedback? >>>>> >>>>> guo Maxwell <cclive1...@gmail.com> 于2023年9月28日周四 10:33写道： >>>>> >>>>> > *background : * >>>>> > Sometimes when the EventProcessorStatus of our impala changes to >>>>> > NEED_INVALIDATE, and we manually perform invalidate metadata >>>>> operations on >>>>> > certain tables(as we know the changed schema).But the >>>>> EventProcessorStatus >>>>> > may not be changed, as we can see the description in IMPALA-12468 >>>>> > <https://issues.apache.org/jira/browse/IMPALA-12468> . >>>>> > >>>>> > So what about support some impala operation and maintenance behavior >>>>> > through SQL. As I know some database has already exposed some >>>>> operation and >>>>> > maintenance operations through sql. >>>>> > >>>>> > *Action :* >>>>> > To fix this problem , I recommend providing Impala with a new SQL >>>>> named : >>>>> > SET EVENT ACTIVE. >>>>> > Initially we can only support set ACTIVE status when we found the >>>>> status >>>>> > is all right . Other states can be supported on demand. >>>>> > >>>>> > So what do you think? Thanks. >>>>> > >>>>> > >>>>> >>>>> -- >>>>> you are the apple of my eye ! >>>>> >>>>
