Hi, I recently came across some limitations in expressing timestamp type with Substrait in the Acero substrait consumer and am curious to hear what people's thoughts are.
The particular issue that I have is when specifying timestamp type in substrait, the unit is "microseconds" and there is no way to change that. When integrating with Arrow, often we have timestamps in an internal system that is of another unit, e.g., a flight service that returns a timestamp in nanos. Also, interop with pandas, because pandas internally use nanoseconds, that is another gap. Currently as a result, we often need to convert from nanos <-> micro when a substrait plan is involved to specify timestamps. It feels to me as something missing in substrait but I wonder what other people think. (Sending this to Arrow mailing list because I know some people here are pretty involved with substrait and I am more familiar with the folks in the Arrow community. Therefore wanted to get some thoughts from the people here). Li