clintropolis commented on pull request #11713: URL: https://github.com/apache/druid/pull/11713#issuecomment-932492779
I've been doing a lot of testing with a mix of historicals, brokers, and routers that do and do not have the changes of this PR to determine how queries will behaving during various upgrade scenarios. So far, the only issues I have run into are when you have updated brokers but old historicals (which is not the recommended upgrade order), and only for queries with subqueries which have array or complex types, e.g. something like: ``` SELECT dim1,dim2 FROM foo WHERE ARRAY_CONTAINS((SELECT ARRAY_AGG(DISTINCT dim1) FROM foo WHERE dim1 is not null), dim1) ``` If you have an updated broker but older historicals, you will observe an error of this form: ``` Error: Unknown exception Cannot construct instance of `org.apache.druid.segment.column.ValueType`, problem: No enum constant org.apache.druid.segment.column.ValueType.ARRAY<STRING> at [Source: (org.eclipse.jetty.server.HttpInputOverHTTP); line: -1, column: 128] (through reference chain: org.apache.druid.query.scan.ScanQuery["dataSource"]->org.apache.druid.query.JoinDataSource["right"]->org.apache.druid.query.InlineDataSource["columnTypes"]->java.util.ArrayList[1]) com.fasterxml.jackson.databind.exc.ValueInstantiationException ``` because the inline datasource which is pushed down to the historicals is not forgiving with type information. I'm not sure if this is a big deal, since we do not recommend updating in this order, and I haven't run into any query shapes that run into issues when following the recommended order, so I think we might be able to get away without having some configuration option to force serialization of arrays and complex types in the old way. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
