[GitHub] [pinot] xiangfu0 closed issue #9992: Auto clean up the dead broker and server instances if possible

xiangfu0 closed issue #9992: Auto clean up the dead broker and server instances if possible URL: https://github.com/apache/pinot/issues/9992 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the speci

[GitHub] [pinot] xiangfu0 commented on issue #9992: Auto clean up the dead broker and server instances if possible

xiangfu0 commented on issue #9992: URL: https://github.com/apache/pinot/issues/9992#issuecomment-1374438667 https://github.com/apache/pinot/pull/10027 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [pinot] suddendust opened a new issue, #10079: Cannot Apply JSON Index on STRING Column

suddendust opened a new issue, #10079: URL: https://github.com/apache/pinot/issues/10079 Applied JSON index on a tags column with the following config (0.9.0): ``` "jsonIndexColumns": [ "tags" ], ``` Ran the query: `select span_id from spanEventView where JS

[GitHub] [pinot] richardstartin commented on pull request #10077: Add an option to reduce random access by caching fields of visited nodes when doing BFS over StarTree index

richardstartin commented on PR #10077: URL: https://github.com/apache/pinot/pull/10077#issuecomment-1374422662 If #8570 had negative impacts for certain disk setups, I’d suggest just reverting the startreeenode changes there, rather than making good performance opt in in what could be or (e

[GitHub] [pinot] richardstartin commented on a diff in pull request #10077: Add an option to reduce random access by caching fields of visited nodes when doing BFS over StarTree index

richardstartin commented on code in PR #10077: URL: https://github.com/apache/pinot/pull/10077#discussion_r1063979667 ## pinot-segment-local/src/main/java/org/apache/pinot/segment/local/startree/OffHeapStarTreeNode.java: ## @@ -39,11 +39,13 @@ public class OffHeapStarTreeNode im

[GitHub] [pinot] richardstartin commented on pull request #10077: Add an option to reduce random access by caching fields of visited nodes when doing BFS over StarTree index

richardstartin commented on PR #10077: URL: https://github.com/apache/pinot/pull/10077#issuecomment-1374421782 Out of interest how do you know this is more disk-friendly (presumably better for block prefetching and fewer page faults)? Have you looked at major faults with sar -B or something

[GitHub] [pinot] xiangfu0 merged pull request #10078: Fixing the issue that S3PinotFS cannot open a file > 2GB

xiangfu0 merged PR #10078: URL: https://github.com/apache/pinot/pull/10078 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot.apa

[GitHub] [pinot] elonazoulay commented on issue #10073: Segment build fails with null value for time column

elonazoulay commented on issue #10073: URL: https://github.com/apache/pinot/issues/10073#issuecomment-1374349611 Thanks for responding @Jackie-Jiang ! I tried that and create segment is fine but upload segment still invokes the check regardless. lmk what you think: would it be worth it to

[GitHub] [pinot] deemoliu commented on a diff in pull request #10052: Add an API to download validDocIdsSnapshots from peer servers

deemoliu commented on code in PR #10052: URL: https://github.com/apache/pinot/pull/10052#discussion_r1063933545 ## pinot-server/src/main/java/org/apache/pinot/server/api/resources/TablesResource.java: ## @@ -423,6 +414,83 @@ public Response downloadSegment( } } + /**

[GitHub] [pinot] deemoliu commented on a diff in pull request #10052: Add an API to download validDocIdsSnapshots from peer servers

deemoliu commented on code in PR #10052: URL: https://github.com/apache/pinot/pull/10052#discussion_r1063933578 ## pinot-server/src/main/java/org/apache/pinot/server/api/resources/TablesResource.java: ## @@ -423,6 +411,74 @@ public Response downloadSegment( } } + /**

[GitHub] [pinot] deemoliu commented on a diff in pull request #10052: Add an API to download validDocIdsSnapshots from peer servers

deemoliu commented on code in PR #10052: URL: https://github.com/apache/pinot/pull/10052#discussion_r1063933449 ## pinot-segment-local/src/main/java/org/apache/pinot/segment/local/indexsegment/immutable/ImmutableSegmentImpl.java: ## @@ -150,7 +150,7 @@ public void deleteValidDoc

[GitHub] [pinot] npawar commented on a diff in pull request #10071: fix(ui): show query result along with error message if both exist

npawar commented on code in PR #10071: URL: https://github.com/apache/pinot/pull/10071#discussion_r1063927607 ## pinot-controller/src/main/resources/app/utils/PinotMethodUtils.ts: ## @@ -272,15 +272,13 @@ const getQueryResults = (params) => { // if sql api throws error, han

[GitHub] [pinot] klsince commented on a diff in pull request #10077: Add an option to reduce random access by caching fields of visited nodes when doing BFS over StarTree index

klsince commented on code in PR #10077: URL: https://github.com/apache/pinot/pull/10077#discussion_r1063925350 ## pinot-spi/src/main/java/org/apache/pinot/spi/utils/CommonConstants.java: ## @@ -280,6 +280,7 @@ public static class QueryOptionKey { public static final Str

[GitHub] [pinot] klsince commented on a diff in pull request #10077: Add an option to reduce random access by caching fields of visited nodes when doing BFS over StarTree index

klsince commented on code in PR #10077: URL: https://github.com/apache/pinot/pull/10077#discussion_r1063925289 ## pinot-segment-local/src/main/java/org/apache/pinot/segment/local/startree/OffHeapStarTreeNode.java: ## @@ -39,11 +39,13 @@ public class OffHeapStarTreeNode implement

[GitHub] [pinot] Jackie-Jiang commented on issue #10073: Segment build fails with null value for time column

Jackie-Jiang commented on issue #10073: URL: https://github.com/apache/pinot/issues/10073#issuecomment-1374324677 One workaround would be to turn the validation off (there is a flag in table config). But in regular case people should not ingest records with timestamp in 1970. The underlying

[GitHub] [pinot] codecov-commenter commented on pull request #10078: Fixing the issue that S3PinotFS cannot open a file > 2GB

codecov-commenter commented on PR #10078: URL: https://github.com/apache/pinot/pull/10078#issuecomment-1374318148 # [Codecov](https://codecov.io/gh/apache/pinot/pull/10078?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Soft

[GitHub] [pinot] elonazoulay commented on issue #10073: Segment build fails with null value for time column

elonazoulay commented on issue #10073: URL: https://github.com/apache/pinot/issues/10073#issuecomment-1374312345 Ah ok, thanks! So how does someone insert values where the epoch milli is 0? It fails the validation. -- This is an automated message from the Apache Git Service. To respond to

[GitHub] [pinot] Jackie-Jiang commented on a diff in pull request #10074: add realtime consumption exception to LL consumer data manager metrics

Jackie-Jiang commented on code in PR #10074: URL: https://github.com/apache/pinot/pull/10074#discussion_r1063906583 ## pinot-core/src/main/java/org/apache/pinot/core/data/manager/realtime/LLRealtimeSegmentDataManager.java: ## @@ -420,6 +423,9 @@ protected boolean consumeLoop()

[GitHub] [pinot] Jackie-Jiang commented on issue #10073: Segment build fails with null value for time column

Jackie-Jiang commented on issue #10073: URL: https://github.com/apache/pinot/issues/10073#issuecomment-1374299266 It is intentional to prevent users from putting the wrong time unit. E.g. if the time value is `daysSinceEpoch`, but user mistakenly put `MILLISECONDS` as the time unit, this

[GitHub] [pinot] joshigaurava commented on a diff in pull request #10071: fix(ui): show query result along with error message if both exist

joshigaurava commented on code in PR #10071: URL: https://github.com/apache/pinot/pull/10071#discussion_r1063902631 ## pinot-controller/src/main/resources/app/utils/PinotMethodUtils.ts: ## @@ -272,15 +272,13 @@ const getQueryResults = (params) => { // if sql api throws erro

[GitHub] [pinot] joshigaurava commented on pull request #10071: fix(ui): show query result along with error message if both exist

joshigaurava commented on PR #10071: URL: https://github.com/apache/pinot/pull/10071#issuecomment-1374288227 Ok, so code wise exception is at the top (first thing handled) except that the logic seems to be display nothing else if there is an exception. Let's keep that structure and also ref

[GitHub] [pinot] joshigaurava commented on pull request #10071: fix(ui): show query result along with error message if both exist

joshigaurava commented on PR #10071: URL: https://github.com/apache/pinot/pull/10071#issuecomment-1374285557 @jayeshchoudhary what happens if there are multiple rows to be displayed in the result? 1) Is that a valid use case? 2. If yes, is there a chance that the exception gets pushed down

[GitHub] [pinot] mcvsubbu commented on a diff in pull request #9994: [feature] Add a Tracker class to support aggregate-worst case consumption delay…

mcvsubbu commented on code in PR #9994: URL: https://github.com/apache/pinot/pull/9994#discussion_r1063872683 ## pinot-core/src/main/java/org/apache/pinot/core/data/manager/realtime/IngestionDelayTracker.java: ## @@ -0,0 +1,371 @@ +/** + * Licensed to the Apache Software Foundat

[GitHub] [pinot] richardstartin commented on a diff in pull request #8764: Vectorize predicates over full col filter scans

richardstartin commented on code in PR #8764: URL: https://github.com/apache/pinot/pull/8764#discussion_r880647573 ## pinot-core/src/main/java/org/apache/pinot/core/operator/dociditerators/SVScanDocIdIterator.java: ## @@ -55,6 +58,33 @@ public SVScanDocIdIterator(PredicateEvalua

[GitHub] [pinot] klsince commented on a diff in pull request #10077: Add an option to cache fields of visited nodes while doing BFS over StarTree index

klsince commented on code in PR #10077: URL: https://github.com/apache/pinot/pull/10077#discussion_r1063863732 ## pinot-spi/src/main/java/org/apache/pinot/spi/utils/CommonConstants.java: ## @@ -280,6 +280,7 @@ public static class QueryOptionKey { public static final Str

[GitHub] [pinot] klsince commented on a diff in pull request #10077: Add an option to cache fields of visited nodes while doing BFS over StarTree index

klsince commented on code in PR #10077: URL: https://github.com/apache/pinot/pull/10077#discussion_r1063862763 ## pinot-segment-local/src/main/java/org/apache/pinot/segment/local/startree/OffHeapStarTreeNode.java: ## @@ -39,11 +39,13 @@ public class OffHeapStarTreeNode implement

[GitHub] [pinot] Jackie-Jiang commented on a diff in pull request #10047: [WIP] Add upsert ttl configs for Pinot upsert optimizations

Jackie-Jiang commented on code in PR #10047: URL: https://github.com/apache/pinot/pull/10047#discussion_r1063860563 ## pinot-segment-local/src/main/java/org/apache/pinot/segment/local/upsert/BasePartitionUpsertMetadataManager.java: ## @@ -217,6 +226,17 @@ public void replaceSegm

[GitHub] [pinot] codecov-commenter commented on pull request #10077: Add an option to cache fields of visited nodes while doing BFS over StarTree index

codecov-commenter commented on PR #10077: URL: https://github.com/apache/pinot/pull/10077#issuecomment-1374228184 # [Codecov](https://codecov.io/gh/apache/pinot/pull/10077?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Soft

[GitHub] [pinot] Jackie-Jiang commented on a diff in pull request #10052: Add an API to download validDocIdsSnapshots from peer servers

Jackie-Jiang commented on code in PR #10052: URL: https://github.com/apache/pinot/pull/10052#discussion_r1063848896 ## pinot-segment-local/src/main/java/org/apache/pinot/segment/local/indexsegment/immutable/ImmutableSegmentImpl.java: ## @@ -150,7 +150,7 @@ public void deleteVali

[GitHub] [pinot] Jackie-Jiang commented on a diff in pull request #10077: Add an option to cache fields of visited nodes while doing BFS over StarTree index

Jackie-Jiang commented on code in PR #10077: URL: https://github.com/apache/pinot/pull/10077#discussion_r1063842531 ## pinot-segment-local/src/main/java/org/apache/pinot/segment/local/startree/CachedStarTreeNode.java: ## @@ -0,0 +1,114 @@ +/** + * Licensed to the Apache Software

[GitHub] [pinot] klsince opened a new pull request, #10077: Add an option to cache fields of visited nodes while doing BFS over StarTree index

klsince opened a new pull request, #10077: URL: https://github.com/apache/pinot/pull/10077 Add a query option `scanStarTreeNodesOnHeap` to turn on/off whether to cache the tree node's fields in object fields when traversing the StarTree index. Caching node's fields can avoid probing the und

[GitHub] [pinot] mcvsubbu commented on a diff in pull request #9994: [feature] Add a Tracker class to support aggregate-worst case consumption delay…

mcvsubbu commented on code in PR #9994: URL: https://github.com/apache/pinot/pull/9994#discussion_r1063836862 ## pinot-core/src/main/java/org/apache/pinot/core/data/manager/realtime/ConsumptionDelayTracker.java: ## @@ -0,0 +1,361 @@ +/** + * Licensed to the Apache Software Found

[GitHub] [pinot] shwin commented on issue #7130: Add a partialResults flag in Response metadata

shwin commented on issue #7130: URL: https://github.com/apache/pinot/issues/7130#issuecomment-1374145632 I think this would be helpful for us too, assuming that (2) would also cover the case in which segments have been generated and pushed, but servers are still downloading (or whatever the

[GitHub] [pinot] shwin opened a new issue, #10076: Indicate Partial Response when Servers have not loaded all

shwin opened a new issue, #10076: URL: https://github.com/apache/pinot/issues/10076 We have observed the following behavior: 1. We have an OFFLINE table 2. We load 1000 segments via the SegmentGenerationAndPush minion-based task; they are about 500MB each 3. The minions finish all t

[GitHub] [pinot] Jackie-Jiang commented on issue #10069: Add a controller API to reflect whether force commit is finished

Jackie-Jiang commented on issue #10069: URL: https://github.com/apache/pinot/issues/10069#issuecomment-1374107384 @sajjad-moradi This is a good work around, and I think we should make this check automatic so that user doesn't need to implement this extra logic. This logic is very similar to

[GitHub] [pinot] snleee commented on issue #10070: [Question] Is it possible to insert thrift data into pinot through stream ingestion?

snleee commented on issue #10070: URL: https://github.com/apache/pinot/issues/10070#issuecomment-1374073928 @ruhz3 We currently have the message decoder implementation for CSV, AVRO, JSON, PROTOBUF. Is it an option for you to change your Kafka stream to be based on one of formats that we su

[GitHub] [pinot] Jackie-Jiang merged pull request #10075: [metric] adding rows with error to docker metrics config

Jackie-Jiang merged PR #10075: URL: https://github.com/apache/pinot/pull/10075 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot

[GitHub] [pinot] walterddr commented on a diff in pull request #10074: add realtime consumption exception to LL consumer data manager metrics

walterddr commented on code in PR #10074: URL: https://github.com/apache/pinot/pull/10074#discussion_r106371 ## pinot-core/src/main/java/org/apache/pinot/core/data/manager/realtime/LLRealtimeSegmentDataManager.java: ## @@ -489,6 +489,9 @@ protected boolean consumeLoop()

[GitHub] [pinot] Jackie-Jiang merged pull request #10072: upgrade to RoaringBitmap 0.9.38

Jackie-Jiang merged PR #10072: URL: https://github.com/apache/pinot/pull/10072 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot

[GitHub] [pinot] Jackie-Jiang commented on a diff in pull request #10074: add realtime consumption exception to LL consumer data manager metrics

Jackie-Jiang commented on code in PR #10074: URL: https://github.com/apache/pinot/pull/10074#discussion_r1063707998 ## pinot-core/src/main/java/org/apache/pinot/core/data/manager/realtime/LLRealtimeSegmentDataManager.java: ## @@ -489,6 +489,9 @@ protected boolean consumeLoop()

[GitHub] [pinot] codecov-commenter commented on pull request #10074: add realtime consumption exception to LL consumer data manager metrics

codecov-commenter commented on PR #10074: URL: https://github.com/apache/pinot/pull/10074#issuecomment-1374003582 # [Codecov](https://codecov.io/gh/apache/pinot/pull/10074?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Soft

[GitHub] [pinot] siddharthteotia merged pull request #10031: Optimize TableRebalancer to not always recompute target assignment when IS changes

siddharthteotia merged PR #10031: URL: https://github.com/apache/pinot/pull/10031 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pi

[GitHub] [pinot] siddharthteotia commented on a diff in pull request #10054: Add DistinctSum aggregation function

siddharthteotia commented on code in PR #10054: URL: https://github.com/apache/pinot/pull/10054#discussion_r1063680461 ## pinot-core/src/test/java/org/apache/pinot/queries/InterSegmentAggregationSingleValueQueriesTest.java: ## @@ -568,4 +568,32 @@ public void testNumGroupsLimit(

[GitHub] [pinot] elonazoulay opened a new issue, #10073: Segment build fails with null value for time column

elonazoulay opened a new issue, #10073: URL: https://github.com/apache/pinot/issues/10073 Happy New Year! We noticed that creating a segment fails due to TimeUtils.VALID_MIN_TIME_MILLIS being 1971/01/01 instead of 1970/01/01, i.e. 0 epoch millis. Is this intended? Context: The

[GitHub] [pinot] codecov-commenter commented on pull request #10072: upgrade to RoaringBitmap 0.9.38

codecov-commenter commented on PR #10072: URL: https://github.com/apache/pinot/pull/10072#issuecomment-1373940043 # [Codecov](https://codecov.io/gh/apache/pinot/pull/10072?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Soft

[GitHub] [pinot] richardstartin opened a new pull request, #10072: upgrade to RoaringBitmap 0.9.38

richardstartin opened a new pull request, #10072: URL: https://github.com/apache/pinot/pull/10072 Contains a bug fix for the timestamp index, see release notes https://github.com/RoaringBitmap/RoaringBitmap/releases/tag/0.9.38 -- This is an automated message from the Apache Git Service. T

[GitHub] [pinot] walterddr commented on a diff in pull request #10045: add checkpointing support to flink connector

walterddr commented on code in PR #10045: URL: https://github.com/apache/pinot/pull/10045#discussion_r1063605992 ## pinot-connectors/pinot-flink-connector/src/main/java/org/apache/pinot/connector/flink/sink/PinotSinkFunction.java: ## @@ -117,31 +123,22 @@ public void close()

[GitHub] [pinot] walterddr commented on a diff in pull request #10045: add checkpointing support to flink connector

walterddr commented on code in PR #10045: URL: https://github.com/apache/pinot/pull/10045#discussion_r1063605992 ## pinot-connectors/pinot-flink-connector/src/main/java/org/apache/pinot/connector/flink/sink/PinotSinkFunction.java: ## @@ -117,31 +123,22 @@ public void close()

[GitHub] [pinot] walterddr commented on a diff in pull request #10045: add checkpointing support to flink connector

walterddr commented on code in PR #10045: URL: https://github.com/apache/pinot/pull/10045#discussion_r1063604999 ## pinot-connectors/pinot-flink-connector/src/main/java/org/apache/pinot/connector/flink/sink/PinotSinkFunction.java: ## @@ -117,31 +123,22 @@ public void close()

[GitHub] [pinot] walterddr commented on a diff in pull request #10045: add checkpointing support to flink connector

walterddr commented on code in PR #10045: URL: https://github.com/apache/pinot/pull/10045#discussion_r1063598717 ## pinot-connectors/pinot-flink-connector/src/main/java/org/apache/pinot/connector/flink/sink/PinotSinkFunction.java: ## @@ -151,4 +148,18 @@ private void flush()

[GitHub] [pinot] walterddr commented on a diff in pull request #10045: add checkpointing support to flink connector

walterddr commented on code in PR #10045: URL: https://github.com/apache/pinot/pull/10045#discussion_r1063598717 ## pinot-connectors/pinot-flink-connector/src/main/java/org/apache/pinot/connector/flink/sink/PinotSinkFunction.java: ## @@ -151,4 +148,18 @@ private void flush()

[GitHub] [pinot] walterddr commented on a diff in pull request #10045: add checkpointing support to flink connector

walterddr commented on code in PR #10045: URL: https://github.com/apache/pinot/pull/10045#discussion_r1063598717 ## pinot-connectors/pinot-flink-connector/src/main/java/org/apache/pinot/connector/flink/sink/PinotSinkFunction.java: ## @@ -151,4 +148,18 @@ private void flush()

[GitHub] [pinot] snleee commented on a diff in pull request #10045: add checkpointing support to flink connector

snleee commented on code in PR #10045: URL: https://github.com/apache/pinot/pull/10045#discussion_r1063592772 ## pinot-connectors/pinot-flink-connector/src/main/java/org/apache/pinot/connector/flink/sink/PinotSinkFunction.java: ## @@ -151,4 +148,18 @@ private void flush()

[GitHub] [pinot] snleee merged pull request #10068: [Cleanup] Remove ConvertToRawIndexTask

snleee merged PR #10068: URL: https://github.com/apache/pinot/pull/10068 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot.apach

[GitHub] [pinot] codecov-commenter commented on pull request #10071: fix(ui): show query result along with error message if both exist

codecov-commenter commented on PR #10071: URL: https://github.com/apache/pinot/pull/10071#issuecomment-1373819500 # [Codecov](https://codecov.io/gh/apache/pinot/pull/10071?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Soft

[GitHub] [pinot] jayeshchoudhary opened a new pull request, #10071: fix(ui): show query result along with error message if both exist

jayeshchoudhary opened a new pull request, #10071: URL: https://github.com/apache/pinot/pull/10071 ### Issue - when running a sql query, if the api result contains both `exceptions` as well as `resultTable` then ideally UI should show both - instead UI is showing only exceptions -

[GitHub] [pinot] bobby-richard commented on a diff in pull request #10045: add checkpointing support to flink connector

bobby-richard commented on code in PR #10045: URL: https://github.com/apache/pinot/pull/10045#discussion_r1063467126 ## pinot-connectors/pinot-flink-connector/src/main/java/org/apache/pinot/connector/flink/sink/PinotSinkFunction.java: ## @@ -151,4 +148,18 @@ private void flush()

[GitHub] [pinot] bobby-richard commented on a diff in pull request #10045: add checkpointing support to flink connector

bobby-richard commented on code in PR #10045: URL: https://github.com/apache/pinot/pull/10045#discussion_r1063463950 ## pinot-connectors/pinot-flink-connector/src/main/java/org/apache/pinot/connector/flink/sink/PinotSinkFunction.java: ## @@ -117,31 +123,22 @@ public void close()

[GitHub] [pinot] bobby-richard commented on a diff in pull request #10045: add checkpointing support to flink connector

bobby-richard commented on code in PR #10045: URL: https://github.com/apache/pinot/pull/10045#discussion_r1063462740 ## pinot-connectors/pinot-flink-connector/src/main/java/org/apache/pinot/connector/flink/sink/PinotSinkFunction.java: ## @@ -117,31 +123,22 @@ public void close()

[GitHub] [pinot] richardstartin commented on a diff in pull request #10043: evaluate EQ queries against range index when available

richardstartin commented on code in PR #10043: URL: https://github.com/apache/pinot/pull/10043#discussion_r1063374698 ## pinot-core/src/main/java/org/apache/pinot/core/operator/filter/RangeIndexBasedFilterOperator.java: ## @@ -124,38 +137,102 @@ public List getChildOperators() {

[GitHub] [pinot] richardstartin commented on a diff in pull request #10043: evaluate EQ queries against range index when available

richardstartin commented on code in PR #10043: URL: https://github.com/apache/pinot/pull/10043#discussion_r1063363939 ## pinot-core/src/main/java/org/apache/pinot/core/operator/filter/RangeIndexBasedFilterOperator.java: ## @@ -131,27 +139,75 @@ public String toExplainString() {

[GitHub] [pinot] richardstartin commented on a diff in pull request #10043: evaluate EQ queries against range index when available

richardstartin commented on code in PR #10043: URL: https://github.com/apache/pinot/pull/10043#discussion_r1063353961 ## pinot-core/src/main/java/org/apache/pinot/core/operator/filter/RangeIndexBasedFilterOperator.java: ## @@ -131,27 +139,75 @@ public String toExplainString() {

[GitHub] [pinot] richardstartin commented on a diff in pull request #10043: evaluate EQ queries against range index when available

richardstartin commented on code in PR #10043: URL: https://github.com/apache/pinot/pull/10043#discussion_r1063272372 ## pinot-core/src/main/java/org/apache/pinot/core/operator/filter/RangeIndexBasedFilterOperator.java: ## @@ -131,27 +139,75 @@ public String toExplainString() {

[GitHub] [pinot] richardstartin commented on a diff in pull request #10043: evaluate EQ queries against range index when available

richardstartin commented on code in PR #10043: URL: https://github.com/apache/pinot/pull/10043#discussion_r1063271581 ## pinot-core/src/main/java/org/apache/pinot/core/operator/filter/RangeIndexBasedFilterOperator.java: ## @@ -131,27 +139,75 @@ public String toExplainString() {

[GitHub] [pinot] richardstartin commented on a diff in pull request #10043: evaluate EQ queries against range index when available

richardstartin commented on code in PR #10043: URL: https://github.com/apache/pinot/pull/10043#discussion_r1063269463 ## pinot-core/src/main/java/org/apache/pinot/core/operator/filter/RangeIndexBasedFilterOperator.java: ## @@ -131,27 +139,75 @@ public String toExplainString() {

[GitHub] [pinot] richardstartin commented on pull request #10044: push utf8 comparison down into ValueReader

richardstartin commented on PR #10044: URL: https://github.com/apache/pinot/pull/10044#issuecomment-1373383973 > > I think @klsince's question needs addressing - what happens when there are two invalid UTF8 sequences at the mismatch, can we return 0 spuriously, I'll address that before merg

[GitHub] [pinot] ruhz3 opened a new issue, #10070: [Question] Is it possible to insert thrift data into pinot through stream ingestion?

ruhz3 opened a new issue, #10070: URL: https://github.com/apache/pinot/issues/10070 Hi, I' am Hyungju. I'm trying to apply this great product on my service. Reading the documents, It seemed like thrift ingestion is only possible in batch ingestion. I tried with below con

[GitHub] [pinot] sajjad-moradi commented on issue #10060: RT segment size does NOT follow "realtime.segment.flush.desired.size" parameter

sajjad-moradi commented on issue #10060: URL: https://github.com/apache/pinot/issues/10060#issuecomment-1373205082 This is a false alarm. The segment size reported here are for "tar.gz" version of the segments, not the original ones. I just double checked one of our tables and un-tarred a s

[GitHub] [pinot] sajjad-moradi closed issue #10060: RT segment size does NOT follow "realtime.segment.flush.desired.size" parameter

sajjad-moradi closed issue #10060: RT segment size does NOT follow "realtime.segment.flush.desired.size" parameter URL: https://github.com/apache/pinot/issues/10060 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

[GitHub] [pinot] jugomezv commented on a diff in pull request #9994: [feature] Add a Tracker class to support aggregate-worst case consumption delay…

jugomezv commented on code in PR #9994: URL: https://github.com/apache/pinot/pull/9994#discussion_r1063091637 ## pinot-server/src/main/java/org/apache/pinot/server/starter/helix/IngestionBasedConsumptionStatusChecker.java: ## @@ -77,6 +77,9 @@ public int getNumConsumingSegments

[GitHub] [pinot] jugomezv commented on a diff in pull request #9994: [feature] Add a Tracker class to support aggregate-worst case consumption delay…

jugomezv commented on code in PR #9994: URL: https://github.com/apache/pinot/pull/9994#discussion_r1063091381 ## pinot-core/src/main/java/org/apache/pinot/core/data/manager/realtime/LLRealtimeSegmentDataManager.java: ## @@ -1594,4 +1616,9 @@ public String getSegmentName() { p

[GitHub] [pinot] jugomezv commented on a diff in pull request #9994: [feature] Add a Tracker class to support aggregate-worst case consumption delay…

jugomezv commented on code in PR #9994: URL: https://github.com/apache/pinot/pull/9994#discussion_r1063091118 ## pinot-core/src/main/java/org/apache/pinot/core/data/manager/realtime/LLRealtimeSegmentDataManager.java: ## @@ -1554,6 +1557,25 @@ private void createPartitionMetadata

[GitHub] [pinot] jugomezv commented on a diff in pull request #9994: [feature] Add a Tracker class to support aggregate-worst case consumption delay…

jugomezv commented on code in PR #9994: URL: https://github.com/apache/pinot/pull/9994#discussion_r1063090872 ## pinot-core/src/main/java/org/apache/pinot/core/data/manager/realtime/RealtimeSegmentDataManager.java: ## @@ -69,4 +69,13 @@ protected static PinotDataBufferMemoryMana

[GitHub] [pinot] sajjad-moradi commented on issue #10069: Add a controller API to reflect whether force commit is finished

sajjad-moradi commented on issue #10069: URL: https://github.com/apache/pinot/issues/10069#issuecomment-1373082267 Another option is to use pauseStatus endpoint on the client side: 1. Client calls /pauseStatus to get the list of existing consuming segments. 2. Client issues the force co

[GitHub] [pinot] jugomezv commented on a diff in pull request #9994: [feature] Add a Tracker class to support aggregate-worst case consumption delay…

jugomezv commented on code in PR #9994: URL: https://github.com/apache/pinot/pull/9994#discussion_r1063043991 ## pinot-core/src/test/java/org/apache/pinot/core/data/manager/realtime/ConsumptionDelayTrackerTest.java: ## @@ -0,0 +1,238 @@ +/** + * Licensed to the Apache Software F

[GitHub] [pinot] codecov-commenter commented on pull request #10068: [Cleanup] Remove ConvertToRawIndexTask

codecov-commenter commented on PR #10068: URL: https://github.com/apache/pinot/pull/10068#issuecomment-1373029360 # [Codecov](https://codecov.io/gh/apache/pinot/pull/10068?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Soft

[GitHub] [pinot] walterddr commented on issue #9655: Support multi-thread execution on multistage engine

walterddr commented on issue #9655: URL: https://github.com/apache/pinot/issues/9655#issuecomment-1373021032 CC @agavra -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

[GitHub] [pinot] walterddr commented on issue #9970: [multistage] default group limit unable to override

walterddr commented on issue #9970: URL: https://github.com/apache/pinot/issues/9970#issuecomment-1373020243 I think it is related to #9957 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the sp

[GitHub] [pinot] Jackie-Jiang opened a new issue, #10069: Add a controller API to reflect whether force commit is finished

Jackie-Jiang opened a new issue, #10069: URL: https://github.com/apache/pinot/issues/10069 Force commit is executed asynchronously leveraging the Helix messages. Currently there is no way to tell whether force commit is finished or not. We want to add a controller API to indicate whether al

[GitHub] [pinot] codecov-commenter commented on pull request #10067: Bump json5 from 1.0.1 to 1.0.2 in /pinot-controller/src/main/resources

codecov-commenter commented on PR #10067: URL: https://github.com/apache/pinot/pull/10067#issuecomment-1373008148 # [Codecov](https://codecov.io/gh/apache/pinot/pull/10067?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Soft

[GitHub] [pinot] Jackie-Jiang opened a new pull request, #10068: [Cleanup] Remove ConvertToRawIndexTask

Jackie-Jiang opened a new pull request, #10068: URL: https://github.com/apache/pinot/pull/10068 Remove the ConvertToRawIndex minion task for the following reasons: 1. Pinot supports on-the-fly index conversion now, so this minion task is no longer needed 2. The task requires v1 segment

[GitHub] [pinot] Jackie-Jiang merged pull request #10066: add query option allowing to scan startree index tree nodes

Jackie-Jiang merged PR #10066: URL: https://github.com/apache/pinot/pull/10066 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot

[GitHub] [pinot] dependabot[bot] opened a new pull request, #10067: Bump json5 from 1.0.1 to 1.0.2 in /pinot-controller/src/main/resources

dependabot[bot] opened a new pull request, #10067: URL: https://github.com/apache/pinot/pull/10067 Bumps [json5](https://github.com/json5/json5) from 1.0.1 to 1.0.2. Release notes Sourced from https://github.com/json5/json5/releases";>json5's releases. v1.0.2 Fix: Prop

[GitHub] [pinot] jadami10 commented on issue #10064: Deleting a table does not reset metric gauges

jadami10 commented on issue #10064: URL: https://github.com/apache/pinot/issues/10064#issuecomment-1372965127 wow... what timing -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comme

[GitHub] [pinot] jadami10 commented on issue #10064: Deleting a table does not reset metric gauges

jadami10 commented on issue #10064: URL: https://github.com/apache/pinot/issues/10064#issuecomment-1372965047 wow... what timing -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comme

[GitHub] [pinot] chenboat merged pull request #9942: [feature] Add pinot-clp-log plugin to encode user-specified fields with CLP during ingestion

chenboat merged PR #9942: URL: https://github.com/apache/pinot/pull/9942 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot.apach

[GitHub] [pinot] Jackie-Jiang merged pull request #10063: output explain plan when compatibility verifier rejects change in query execution

Jackie-Jiang merged PR #10063: URL: https://github.com/apache/pinot/pull/10063 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot

[GitHub] [pinot] ankitsultana commented on a diff in pull request #10005: [multistage] Add Worker SPI to Make Worker Assignment Configurable

ankitsultana commented on code in PR #10005: URL: https://github.com/apache/pinot/pull/10005#discussion_r1062990982 ## pinot-query-planner/src/main/java/org/apache/pinot/query/planner/logical/worker/WorkerAssignmentStrategy.java: ## @@ -0,0 +1,31 @@ +/** + * Licensed to the Apac

[GitHub] [pinot] Jackie-Jiang commented on a diff in pull request #10043: evaluate EQ queries against range index when available

Jackie-Jiang commented on code in PR #10043: URL: https://github.com/apache/pinot/pull/10043#discussion_r1062985665 ## pinot-core/src/main/java/org/apache/pinot/core/operator/filter/RangeIndexBasedFilterOperator.java: ## @@ -131,27 +139,75 @@ public String toExplainString() {

[GitHub] [pinot] Jackie-Jiang commented on issue #10064: Deleting a table does not reset metric gauges

Jackie-Jiang commented on issue #10064: URL: https://github.com/apache/pinot/issues/10064#issuecomment-1372918463 Fixed with #10062 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific c

[GitHub] [pinot] Jackie-Jiang closed issue #10064: Deleting a table does not reset metric gauges

Jackie-Jiang closed issue #10064: Deleting a table does not reset metric gauges URL: https://github.com/apache/pinot/issues/10064 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

[GitHub] [pinot] Jackie-Jiang commented on pull request #10044: push utf8 comparison down into ValueReader

Jackie-Jiang commented on PR #10044: URL: https://github.com/apache/pinot/pull/10044#issuecomment-1372916301 > I think @klsince's question needs addressing - what happens when there are two invalid UTF8 sequences at the mismatch, can we return 0 spuriously, I'll address that before merging.

[GitHub] [pinot] klsince opened a new pull request, #10066: add query option allowing to scan startree index tree nodes

klsince opened a new pull request, #10066: URL: https://github.com/apache/pinot/pull/10066 Add this option to test scaning vs. binary search while traversing the StarTree index easily. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [pinot] snleee commented on a diff in pull request #10045: add checkpointing support to flink connector

snleee commented on code in PR #10045: URL: https://github.com/apache/pinot/pull/10045#discussion_r1062938976 ## pinot-connectors/pinot-flink-connector/src/main/java/org/apache/pinot/connector/flink/sink/PinotSinkFunction.java: ## @@ -151,4 +148,18 @@ private void flush()

[GitHub] [pinot] snleee commented on a diff in pull request #10045: add checkpointing support to flink connector

snleee commented on code in PR #10045: URL: https://github.com/apache/pinot/pull/10045#discussion_r1062938976 ## pinot-connectors/pinot-flink-connector/src/main/java/org/apache/pinot/connector/flink/sink/PinotSinkFunction.java: ## @@ -151,4 +148,18 @@ private void flush()

[GitHub] [pinot] xiangfu0 merged pull request #10062: Remove table gauge metrics once table got deleted

xiangfu0 merged PR #10062: URL: https://github.com/apache/pinot/pull/10062 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot.apa

[GitHub] [pinot] snleee commented on a diff in pull request #10045: add checkpointing support to flink connector

snleee commented on code in PR #10045: URL: https://github.com/apache/pinot/pull/10045#discussion_r1062938976 ## pinot-connectors/pinot-flink-connector/src/main/java/org/apache/pinot/connector/flink/sink/PinotSinkFunction.java: ## @@ -151,4 +148,18 @@ private void flush()

[GitHub] [pinot] snleee commented on a diff in pull request #10045: add checkpointing support to flink connector

snleee commented on code in PR #10045: URL: https://github.com/apache/pinot/pull/10045#discussion_r1062938976 ## pinot-connectors/pinot-flink-connector/src/main/java/org/apache/pinot/connector/flink/sink/PinotSinkFunction.java: ## @@ -151,4 +148,18 @@ private void flush()

[GitHub] [pinot] walterddr merged pull request #10058: [refactor] move min/max combine out

walterddr merged PR #10058: URL: https://github.com/apache/pinot/pull/10058 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot.ap

[GitHub] [pinot] jugomezv commented on a diff in pull request #9994: [feature] Add a Tracker class to support aggregate-worst case consumption delay…

jugomezv commented on code in PR #9994: URL: https://github.com/apache/pinot/pull/9994#discussion_r1062886088 ## pinot-common/src/main/java/org/apache/pinot/common/metrics/ServerGauge.java: ## @@ -45,8 +45,10 @@ public enum ServerGauge implements AbstractMetrics.Gauge { // De

<    3   4   5   6   7   8   9   10   11   12   >