#general
@itay: @itay has joined the channel
@yoav: @yoav has joined the channel
@saumya2700: Hi All, is there any way that we can dump whole message from topic in one column, my use case is that we have very complex avro-schema , and to create schema and map every field is very tedious work. I have done some transformations using *tranformationConfigs* but for every topic and schema it is going to be very complex.
@mayanks: There is avro to Pinot schema utility in tools you can use to get a starting point, and then customize it. You can also programmatically generate it for your customizations
@g.kishore: You can also store complex fields as Json in Pinot and use json index
@mayanks: There is avro to Pinot schema utility in tools you can use to get a starting point, and then customize it. You can also programmatically generate it for your customizations
#random
@itay: @itay has joined the channel
@yoav: @yoav has joined the channel
#troubleshooting
@ryantle1028: hi now i try to create real time table consume kafka. it's created but status BAD. Please Help recommend this. error log is --> Mar 21 12:06:40 poc-pinot01 pinot-admin.sh: 2022/03/21 12:06:40.937 ERROR [MessageGenerationPhase] [HelixController-pipeline-default-pinot-uat-(1ad8bfc9_DEFAULT)] Event 1ad8bfc9_DEFAULT : Unable to find a next state for resource: user_stream_tykb_REALTIME partition: user_stream_tykb__2__0__20220321T0506Z from stateModelDefinitionclass org.apache.helix.model.StateModelDefinition from:ERROR to:CONSUMING Mar 21 12:06:40 poc-pinot01 pinot-admin.sh: 2022/03/21 12:06:40.937 ERROR [MessageGenerationPhase] [HelixController-pipeline-default-pinot-uat-(1ad8bfc9_DEFAULT)] Event 1ad8bfc9_DEFAULT : Unable to find a next state for resource: user_stream_tykb_REALTIME partition: user_stream_tykb__3__0__20220321T0506Z from stateModelDefinitionclass org.apache.helix.model.StateModelDefinition from:ERROR to:CONSUMING Mar 21 12:06:40 poc-pinot01 pinot-admin.sh: 2022/03/21 12:06:40.937 ERROR [MessageGenerationPhase] [HelixController-pipeline-default-pinot-uat-(1ad8bfc9_DEFAULT)] Event 1ad8bfc9_DEFAULT : Unable to find a next state for resource: user_stream_tykb_REALTIME partition: user_stream_tykb__4__0__20220321T0506Z from stateModelDefinitionclass org.apache.helix.model.StateModelDefinition from:ERROR to:CONSUMING Mar 21 12:06:40 poc-pinot01 pinot-admin.sh: 2022/03/21 12:06:40.937 ERROR [MessageGenerationPhase] [HelixController-pipeline-default-pinot-uat-(1ad8bfc9_DEFAULT)] Event 1ad8bfc9_DEFAULT : Unable to find a next state for resource: user_stream_tykb_REALTIME partition: user_stream_tykb__5__0__20220321T0506Z from stateModelDefinitionclass org.apache.helix.model.StateModelDefinition from:ERROR to:CONSUMING Mar 21 12:06:40 poc-pinot01 pinot-admin.sh: 2022/03/21 12:06:40.937 ERROR [MessageGenerationPhase] [HelixController-pipeline-default-pinot-uat-(1ad8bfc9_DEFAULT)] Event 1ad8bfc9_DEFAULT : Unable to find a next state for resource: user_stream_tykb_REALTIME partition: user_stream_tykb__6__0__20220321T0506Z from stateModelDefinitionclass org.apache.helix.model.StateModelDefinition from:ERROR to:CONSUMING Mar 21 12:06:40 poc-pinot01 pinot-admin.sh: 2022/03/21 12:06:40.937 ERROR [MessageGenerationPhase] [HelixController-pipeline-default-pinot-uat-(1ad8bfc9_DEFAULT)] Event 1ad8bfc9_DEFAULT : Unable to find a next state for resource: user_stream_tykb_REALTIME partition: user_stream_tykb__7__0__20220321T0506Z from stateModelDefinitionclass org.apache.helix.model.StateModelDefinition from:ERROR to:CONSUMING Mar 21 12:06:40 poc-pinot01 pinot-admin.sh: 2022/03/21 12:06:40.937 ERROR [MessageGenerationPhase] [HelixController-pipeline-default-pinot-uat-(1ad8bfc9_DEFAULT)] Event 1ad8bfc9_DEFAULT : Unable to find a next state for resource: user_stream_tykb_REALTIME partition: user_stream_tykb__8__0__20220321T0506Z from stateModelDefinitionclass org.apache.helix.model.StateModelDefinition from:ERROR to:CONSUMING Mar 21 12:06:40 poc-pinot01 pinot-admin.sh: 2022/03/21 12:06:40.938 ERROR [MessageGenerationPhase] [HelixController-pipeline-default-pinot-uat-(1ad8bfc9_DEFAULT)] Event 1ad8bfc9_DEFAULT : Unable to find a next state for resource: user_stream_tykb_REALTIME partition: user_stream_tykb__9__0__20220321T0506Z from stateModelDefinitionclass org.apache.helix.model.StateModelDefinition from:ERROR to:CONSUMING Mar 21 12:07:07 poc-pinot01 systemd-logind: Removed session 590.
@walterddr: can you share your table config and settings for your kafka topic?
@alihaydar.atil: Hi everyone, Do indexes also work for multi-valued columns?
@walterddr: which index method do you wan to use for mv columns?
@mayanks: Inv index certainly is supported for MV columns.
@itay: @itay has joined the channel
@yoav: @yoav has joined the channel
@luisfernandez: looking for some help here as the dead node hasn't been able to be deleted and the UI is still blocked on the realtime table, also I have questions about the new server and the rebalance operation as I don't see the data that I would expect there
#pinot-dev
@ashish: Thanks for taking a look. Will attach the profile/stacktrace soon. The retrieval of dictId is ok - it's the call to getInternal that does not seem to go through the cache and hence we see the same latency when the same query is repeated. Usually, second run of the query runs faster but not in the case we observed
@ashish: Will try it out. Not sure why retrieving dict values through getInternal shouldn't go through the data fetcher and cached - it would have done that if the same column was present in projection only. No?
@ashish: Yes, that makes sense.
#getting-started
@itay: @itay has joined the channel
@yoav: @yoav has joined the channel
--------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@pinot.apache.org For additional commands, e-mail: dev-h...@pinot.apache.org
