itschrispeck commented on PR #13776: URL: https://github.com/apache/pinot/pull/13776#issuecomment-2287205289
> Can we not use the `noForwardIndex` feature to get this behavior ? > > When we had added the `noRawDataForTextIndex` support, `noForwardIndex` feature did not exist and so it was just more of a workaround. I was thinking it would be good to support both eventually. Using the forward index disabled config seemed to require larger changes to handle/validate everything properly in the realtime segment commit path. The other factor was, for now disabling it would never be used except for a Lucene text indexed column since no other indexes can be reused (and it's debatable how much improvement reusing others could bring, since the Lucene based index is comparatively expensive to create). Is the complexity worth it? Since even though the feature is generic the implementation would entirely support Lucene text index only and it seems unlikely other indexes will benefit in near future. I don't have the answer, but since we haven't marked `noRawDataForTextIndex` as deprecated, to me this seemed like a simpler implementation that solved our immediate needs and brings an overall improvement -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
