snleee edited a comment on issue #2756: URL: https://github.com/apache/incubator-pinot/issues/2756#issuecomment-618751277
I want to discuss about enforcing a uniform type & format and only allow users to pick the granularity for the `primary time column`, which is configured in the table config. - start with the primary time column (enforced to have `long, "1:milliseconds:epoch", user-defined-granularity`). - Users can initially try to use time conversion UDF for the time column. - If they need the better performance, they can add a new time column to `dateTimeFieldSpecs` with the specific time format & type that they want to materialize. I think that having a uniform type & format for the primary column can simplify a lot of components because we won't need to support all different type formats when we need to play with the time. One example can benefit from this is that segment roll-up config can be very simple - getting only time granularity. Another huge benefit will be an easier integration with external UIs such as Superset. I believe that Pinot-Superset connector have some extra logic to convert the time column values into a format that Superset understands. Since pinot's time column format/type will be different for each table, the external system now needs to know about Pinot's table configuration & schema (or separate config for each table on Superset) for correctly parsing the time column value. Since we make the interface change here to the time column, it's going to be great if we can consider this option while doing this work. @kishoreg @npawar @mayankshriv @Jackie-Jiang How do you guys think? ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
