waynexia commented on issue #4809: URL: https://github.com/apache/arrow-datafusion/issues/4809#issuecomment-1371598729
>I wonder if you have any thoughts about adding such a feature to DataFusion? It looks good to have `time_bucket_gapfill` and `locf` natively in DataFusion to me. Selecting and postprocessing the selected data in TSDB is a bit complex compared to general DBMS. Prometheus/PromQL also have this kind of logic, which will try to align and lookback the "series": https://promlabs.com/blog/2020/07/02/selecting-data-in-promql#lookback-delta I build something similar in the very recent: https://github.com/GreptimeTeam/greptimedb/blob/develop/src/promql/src/extension_plan/instant_manipulate.rs#L423 And we plan to expose this functionality to SQL interface in some way, which will become something similar to this proposal I think. But my concern is, to provide a good use experience and functionality, we may need a bunch of "gap-filling" functions. Like filling it with null, filling it with the last value, filling it with the last value if the gap is less than 1 day otherwise left blank etc. I'm not sure if these "time-series functions" is also useful to other users of DataFusion. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
