[GitHub] [druid] ivan-demchenkov edited a comment on issue #10735: Misleading info about tsColumn in the lookups-cached-global docs

ivan-demchenkov edited a comment on issue #10735: URL: https://github.com/apache/druid/issues/10735#issuecomment-757433050 I thought, and, according to docs, the whole point of having tsColumn is to avoid full reload. If use this to support deletions then it's possible to have a situation

[GitHub] [druid] ivan-demchenkov commented on issue #10735: Misleading info about tsColumn in the lookups-cached-global docs

ivan-demchenkov commented on issue #10735: URL: https://github.com/apache/druid/issues/10735#issuecomment-757433050 I thought, and, according to docs, the whole point of having tsColumn is to avoid full reload. Because then it's possible to have a situation when records were deleted, but

[GitHub] [druid] vogievetsky merged pull request #10710: fix web-console show json bug

vogievetsky merged pull request #10710: URL: https://github.com/apache/druid/pull/10710 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[GitHub] [druid] gianm commented on pull request #10738: Fix collision between #10689 and #10593.

gianm commented on pull request #10738: URL: https://github.com/apache/druid/pull/10738#issuecomment-756905866 Travis passed. I'll merge this. This is an automated message from the Apache Git Service. To respond to the

[GitHub] [druid] capistrant commented on pull request #10287: allow the LogUsedSegments duty to be skipped

capistrant commented on pull request #10287: URL: https://github.com/apache/druid/pull/10287#issuecomment-756913585 > I like this log message from LogUsedSegments and have found it very useful: > > ```java > log.info("Found [%,d] used segments.", params.getUsedSegments().size());

[GitHub] [druid] vogievetsky commented on pull request #10710: fix web-console show json bug

vogievetsky commented on pull request #10710: URL: https://github.com/apache/druid/pull/10710#issuecomment-757038204 I think this could be tested by adding a snapshot test to each of the changes to make sure the correct dom is rendered even when there are bigints in the JSON. That would

[GitHub] [druid] zhangyue19921010 commented on pull request #10737: Add missing unit tests for segment loading in historicals

zhangyue19921010 commented on pull request #10737: URL: https://github.com/apache/druid/pull/10737#issuecomment-757230249 Thanks for adding this UT. LTGM :) This is an automated message from the Apache Git Service. To

[GitHub] [druid] zhangyue19921010 commented on pull request #10688: Historical reloads damaged segments automatically when lazy on start.

zhangyue19921010 commented on pull request #10688: URL: https://github.com/apache/druid/pull/10688#issuecomment-757039352 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [druid] zhangyue19921010 closed pull request #10688: Historical reloads damaged segments automatically when lazy on start.

zhangyue19921010 closed pull request #10688: URL: https://github.com/apache/druid/pull/10688 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

[GitHub] [druid] gianm merged pull request #10738: Fix collision between #10689 and #10593.

gianm merged pull request #10738: URL: https://github.com/apache/druid/pull/10738 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [druid] gianm commented on pull request #10635: Add SQL functions to format numbers into human readable format

gianm commented on pull request #10635: URL: https://github.com/apache/druid/pull/10635#issuecomment-757031980 My 2¢ on the naming thing: I like having 3 separate functions, because I think a 1-function model works best if you have a good spec for format strings, which isn't what this

[GitHub] [druid] suneet-s commented on pull request #10710: fix web-console show json bug

suneet-s commented on pull request #10710: URL: https://github.com/apache/druid/pull/10710#issuecomment-756933916 > > @bananaaggle Thanks for this fix - can you please add a test that catches parsing these large numbers in the web-console so that future refactorings don't accidentally

[GitHub] [druid] zhangyouxun commented on issue #10720: Kafka index task is working, how can not found consumer groups

zhangyouxun commented on issue #10720: URL: https://github.com/apache/druid/issues/10720#issuecomment-757132078 > How do I set the consumer group name？ > > "spec": { > "ioConfig": { > "type": "kafka", > "consumerProperties": { > "bootstrap.servers": "127.0.0.1:9092", >

[GitHub] [druid] LeeMo2K10 commented on issue #10720: Kafka index task is working, how can not found consumer groups

LeeMo2K10 commented on issue #10720: URL: https://github.com/apache/druid/issues/10720#issuecomment-757129714 How do I set the consumer group name？ "spec": { "ioConfig": { "type": "kafka", "consumerProperties": { "bootstrap.servers":

[GitHub] [druid] gianm commented on pull request #10247: Remove CloseQuietly and migrate its usages to other methods.

gianm commented on pull request #10247: URL: https://github.com/apache/druid/pull/10247#issuecomment-756907541 Fixed up some merge conflicts. This is an automated message from the Apache Git Service. To respond to the

[GitHub] [druid] gianm commented on a change in pull request #10605: bitwise math function expressions

gianm commented on a change in pull request #10605: URL: https://github.com/apache/druid/pull/10605#discussion_r554224182 ## File path: docs/misc/math-expr.md ## @@ -119,6 +119,13 @@ See javadoc of java.lang.Math for detailed explanation for each function. |acos|acos(x)

[GitHub] [druid] jihoonson commented on a change in pull request #10677: Add URI based allowPrefexList and denyPrefixList

jihoonson commented on a change in pull request #10677: URL: https://github.com/apache/druid/pull/10677#discussion_r554239833 ## File path: indexing-service/src/main/java/org/apache/druid/indexing/common/task/CompactionTask.java ## @@ -153,6 +154,9 @@ @JsonIgnore

[GitHub] [druid] gianm commented on pull request #10730: Introduce KafkaRecordEntity to support Kafka headers in InputFormats

gianm commented on pull request #10730: URL: https://github.com/apache/druid/pull/10730#issuecomment-757000160 > looks like tests are failing due to insufficient diff coverage for some lines. Many of them appear to be where we introduced additional type parameters. @gianm what's the

[GitHub] [druid] zhangyue19921010 commented on a change in pull request #10737: Add missing unit tests for segment loading in historicals

zhangyue19921010 commented on a change in pull request #10737: URL: https://github.com/apache/druid/pull/10737#discussion_r554432481 ## File path: server/src/test/java/org/apache/druid/server/coordination/SegmentLoadDropHandlerTest.java ## @@ -29,22 +29,28 @@ import

[GitHub] [druid] jihoonson merged pull request #10734: add offsetFetchPeriod to kinesis ingestion doc

jihoonson merged pull request #10734: URL: https://github.com/apache/druid/pull/10734 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [druid] xvrl merged pull request #10730: Introduce KafkaRecordEntity to support Kafka headers in InputFormats

xvrl merged pull request #10730: URL: https://github.com/apache/druid/pull/10730 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [druid] jihoonson merged pull request #10670: [Doc-Fix] Remove extra comma in `druid/integration-tests/README.md`

jihoonson merged pull request #10670: URL: https://github.com/apache/druid/pull/10670 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [druid] gianm commented on a change in pull request #10635: Add SQL functions to format numbers into human readable format

gianm commented on a change in pull request #10635: URL: https://github.com/apache/druid/pull/10635#discussion_r554228178 ## File path: docs/querying/sql.md ## @@ -563,6 +563,9 @@ The [DataSketches extension](../development/extensions-core/datasketches-extensi

[GitHub] [druid] jihoonson commented on a change in pull request #10737: Add missing unit tests for segment loading in historicals

jihoonson commented on a change in pull request #10737: URL: https://github.com/apache/druid/pull/10737#discussion_r554469450 ## File path: server/src/test/java/org/apache/druid/server/coordination/SegmentLoadDropHandlerTest.java ## @@ -29,22 +29,28 @@ import

[GitHub] [druid] capistrant commented on a change in pull request #10287: allow the LogUsedSegments duty to be skipped

capistrant commented on a change in pull request #10287: URL: https://github.com/apache/druid/pull/10287#discussion_r554117678 ## File path: server/src/main/java/org/apache/druid/server/coordinator/DruidCoordinator.java ## @@ -669,21 +669,31 @@ private void stopBeingLeader()

[GitHub] [druid] FrankChen021 commented on issue #8715: "group.id" of ConsumerPropertie be replaced by "kafka-supervisor-%s" in kafka-indexing-service

FrankChen021 commented on issue #8715: URL: https://github.com/apache/druid/issues/8715#issuecomment-757150063 @Elvis0607 @kyostyle1 @mjain1994 @LeeMo2K10 Could you tell me how does your kafka platform restrict client access by a pre-assigned group id ? Kafka clients provides two

[GitHub] [druid] xvrl commented on pull request #10730: Introduce KafkaRecordEntity to support Kafka headers in InputFormats

xvrl commented on pull request #10730: URL: https://github.com/apache/druid/pull/10730#issuecomment-756995841 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

[GitHub] [druid] jihoonson merged pull request #10661: [Minor Doc Fix] Correct the default value of `druid.server.http.gracefulShutdownTimeout`

jihoonson merged pull request #10661: URL: https://github.com/apache/druid/pull/10661 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [druid] zhangyouxun edited a comment on issue #10720: Kafka index task is working, how can not found consumer groups

zhangyouxun edited a comment on issue #10720: URL: https://github.com/apache/druid/issues/10720#issuecomment-757132078 > How do I set the consumer group name？ > > "spec": { > "ioConfig": { > "type": "kafka", > "consumerProperties": { > "bootstrap.servers":

[GitHub] [druid] FrankChen021 commented on pull request #10635: Add SQL functions to format numbers into human readable format

FrankChen021 commented on pull request #10635: URL: https://github.com/apache/druid/pull/10635#issuecomment-757192914 > > * IMO the name `decimal_format` is misleading. It sounds like a function that will format a number to a specific amount of decimals. Maybe instead we could

[GitHub] [druid] jihoonson commented on pull request #10677: Add URI based allowPrefexList and denyPrefixList

jihoonson commented on pull request #10677: URL: https://github.com/apache/druid/pull/10677#issuecomment-757043116 @nishantmonu51 BTW, since we are about to cut a branch for 0.21.0 release (scheduled on the next Monday), do you think we can get this PR merged before the branch cut? I'm

[GitHub] [druid] abhishekagarwal87 commented on pull request #10635: Add SQL functions to format numbers into human readable format

abhishekagarwal87 commented on pull request #10635: URL: https://github.com/apache/druid/pull/10635#issuecomment-757354535 I, for one, don't have a strong opinion. `decimal_format` here is equivalent to

[GitHub] [druid] jihoonson closed issue #10719: Javascript filter return true when the expression result is false

jihoonson closed issue #10719: URL: https://github.com/apache/druid/issues/10719 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [druid] jihoonson commented on pull request #10661: [Minor Doc Fix] Correct the default value of `druid.server.http.gracefulShutdownTimeout`

jihoonson commented on pull request #10661: URL: https://github.com/apache/druid/pull/10661#issuecomment-757045393 > I guess if we indeed do want to support shutdown without waiting, we might need another config property to toggle

[GitHub] [druid] LeeMo2K10 commented on issue #8715: "group.id" of ConsumerPropertie be replaced by "kafka-supervisor-%s" in kafka-indexing-service

LeeMo2K10 commented on issue #8715: URL: https://github.com/apache/druid/issues/8715#issuecomment-757130246 I have this issue too. This is an automated message from the Apache Git Service. To respond to the message, please

[GitHub] [druid] jihoonson merged pull request #10721: javascript filter result convert to java boolean

jihoonson merged pull request #10721: URL: https://github.com/apache/druid/pull/10721 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [druid] gianm commented on pull request #10267: DruidInputSource: Fix issues in column projection, timestamp handling.

gianm commented on pull request #10267: URL: https://github.com/apache/druid/pull/10267#issuecomment-756633682 Fixed up some merge conflicts. This is an automated message from the Apache Git Service. To respond to the

[GitHub] [druid] ayushkul2910 commented on a change in pull request #10448: Added CronScheduler support as a proof to clock drift while emitting metrics

ayushkul2910 commented on a change in pull request #10448: URL: https://github.com/apache/druid/pull/10448#discussion_r553893369 ## File path: server/src/main/java/org/apache/druid/server/metrics/MetricsModule.java ## @@ -108,10 +108,10 @@ public MonitorScheduler

[GitHub] [druid] gianm merged pull request #9667: Thread-safety for ResponseContext.REGISTERED_KEYS

gianm merged pull request #9667: URL: https://github.com/apache/druid/pull/9667 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [druid] gianm commented on pull request #9667: Thread-safety for ResponseContext.REGISTERED_KEYS

gianm commented on pull request #9667: URL: https://github.com/apache/druid/pull/9667#issuecomment-756625948 Thanks @alex-plekhanov! This is an automated message from the Apache Git Service. To respond to the message, please

[GitHub] [druid] clintropolis commented on a change in pull request #10605: bitwise math function expressions

clintropolis commented on a change in pull request #10605: URL: https://github.com/apache/druid/pull/10605#discussion_r553810003 ## File path: core/src/main/java/org/apache/druid/math/expr/Evals.java ## @@ -71,4 +71,14 @@ public static boolean asBoolean(@Nullable String x)

[GitHub] [druid] clintropolis commented on a change in pull request #10613: expression filter support for vectorized query engines

clintropolis commented on a change in pull request #10613: URL: https://github.com/apache/druid/pull/10613#discussion_r553814648 ## File path: processing/src/main/java/org/apache/druid/segment/DimensionHandlerUtils.java ## @@ -305,17 +305,29 @@ private static

[GitHub] [druid] FrankChen021 edited a comment on issue #10735: Misleading info about tsColumn in the lookups-cached-global docs

FrankChen021 edited a comment on issue #10735: URL: https://github.com/apache/druid/issues/10735#issuecomment-756784327 I guess why `getLookupPairs` currently reloads the whole table is that if there're records deletions in database, such a full reload could drop those deleted records

[GitHub] [druid] gianm commented on a change in pull request #10613: expression filter support for vectorized query engines

gianm commented on a change in pull request #10613: URL: https://github.com/apache/druid/pull/10613#discussion_r553782719 ## File path: core/src/main/java/org/apache/druid/math/expr/BinaryLogicalOperatorExpr.java ## @@ -74,7 +74,7 @@ public ExprType

[GitHub] [druid] gianm commented on pull request #10043: Add support for REGEXP_REPLACE

gianm commented on pull request #10043: URL: https://github.com/apache/druid/pull/10043#issuecomment-756624149 > one more time please stalebot  two more times?  This is an automated message from the Apache Git

[GitHub] [druid] gianm closed issue #9106: Unbalanced synchronization of ResponseContext.REGISTERED_KEYS

gianm closed issue #9106: URL: https://github.com/apache/druid/issues/9106 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [druid] liran-funaro commented on pull request #10593: IncrementalIndex Tests and Benchmarks Parametrization

liran-funaro commented on pull request #10593: URL: https://github.com/apache/druid/pull/10593#issuecomment-756795145 Thanks! This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [druid] stale[bot] commented on pull request #10043: Add support for REGEXP_REPLACE

stale[bot] commented on pull request #10043: URL: https://github.com/apache/druid/pull/10043#issuecomment-756624168 This issue is no longer marked as stale. This is an automated message from the Apache Git Service. To

[GitHub] [druid] gianm commented on pull request #10287: allow the LogUsedSegments duty to be skipped

gianm commented on pull request #10287: URL: https://github.com/apache/druid/pull/10287#issuecomment-756630739 > circling back on this. my cluster with 161k segments takes ~20ms to execute `LogUsedSegments`. This duty should scale fairly linearly with number of segments so you'd have to

[GitHub] [druid] FrankChen021 commented on issue #10735: Misleading info about tsColumn in the lookups-cached-global docs

FrankChen021 commented on issue #10735: URL: https://github.com/apache/druid/issues/10735#issuecomment-756784327 I guess why `getLookupPairs` currently reload the whole table is that if there're records deletions in database, such a full reload could drop those deleted records from cache.

[GitHub] [druid] jihoonson commented on a change in pull request #10737: Add missing unit tests for segment loading in historicals

jihoonson commented on a change in pull request #10737: URL: https://github.com/apache/druid/pull/10737#discussion_r554469450 ## File path: server/src/test/java/org/apache/druid/server/coordination/SegmentLoadDropHandlerTest.java ## @@ -29,22 +29,28 @@ import

[GitHub] [druid] abhishekagarwal87 commented on pull request #10635: Add SQL functions to format numbers into human readable format

abhishekagarwal87 commented on pull request #10635: URL: https://github.com/apache/druid/pull/10635#issuecomment-757354535 I, for one, don't have a strong opinion. `decimal_format` here is equivalent to

[GitHub] [druid] zhangyue19921010 commented on a change in pull request #10737: Add missing unit tests for segment loading in historicals

zhangyue19921010 commented on a change in pull request #10737: URL: https://github.com/apache/druid/pull/10737#discussion_r554432481 ## File path: server/src/test/java/org/apache/druid/server/coordination/SegmentLoadDropHandlerTest.java ## @@ -29,22 +29,28 @@ import

[GitHub] [druid] zhangyue19921010 commented on a change in pull request #10737: Add missing unit tests for segment loading in historicals

zhangyue19921010 commented on a change in pull request #10737: URL: https://github.com/apache/druid/pull/10737#discussion_r554432481 ## File path: server/src/test/java/org/apache/druid/server/coordination/SegmentLoadDropHandlerTest.java ## @@ -29,22 +29,28 @@ import

[GitHub] [druid] zhangyue19921010 commented on pull request #10737: Add missing unit tests for segment loading in historicals

zhangyue19921010 commented on pull request #10737: URL: https://github.com/apache/druid/pull/10737#issuecomment-757230249 Thanks for adding this UT. LTGM :) This is an automated message from the Apache Git Service. To

[GitHub] [druid] zhangyue19921010 commented on pull request #10688: Historical reloads damaged segments automatically when lazy on start.

zhangyue19921010 commented on pull request #10688: URL: https://github.com/apache/druid/pull/10688#issuecomment-757223194 CI passed. PTAL :) This is an automated message from the Apache Git Service. To respond to the

[GitHub] [druid] FrankChen021 commented on pull request #10635: Add SQL functions to format numbers into human readable format

FrankChen021 commented on pull request #10635: URL: https://github.com/apache/druid/pull/10635#issuecomment-757192914 > > * IMO the name `decimal_format` is misleading. It sounds like a function that will format a number to a specific amount of decimals. Maybe instead we could

[GitHub] [druid] FrankChen021 commented on issue #8715: "group.id" of ConsumerPropertie be replaced by "kafka-supervisor-%s" in kafka-indexing-service

FrankChen021 commented on issue #8715: URL: https://github.com/apache/druid/issues/8715#issuecomment-757150063 @Elvis0607 @kyostyle1 @mjain1994 @LeeMo2K10 Could you tell me how does your kafka platform restrict client access by a pre-assigned group id ? Kafka clients provides two

[GitHub] [druid] zhangyouxun edited a comment on issue #10720: Kafka index task is working, how can not found consumer groups

zhangyouxun edited a comment on issue #10720: URL: https://github.com/apache/druid/issues/10720#issuecomment-757132078 > How do I set the consumer group name？ > > "spec": { > "ioConfig": { > "type": "kafka", > "consumerProperties": { > "bootstrap.servers":

[GitHub] [druid] zhangyouxun commented on issue #10720: Kafka index task is working, how can not found consumer groups

zhangyouxun commented on issue #10720: URL: https://github.com/apache/druid/issues/10720#issuecomment-757132078 > How do I set the consumer group name？ > > "spec": { > "ioConfig": { > "type": "kafka", > "consumerProperties": { > "bootstrap.servers": "127.0.0.1:9092", >

[GitHub] [druid] LeeMo2K10 commented on issue #8715: "group.id" of ConsumerPropertie be replaced by "kafka-supervisor-%s" in kafka-indexing-service

LeeMo2K10 commented on issue #8715: URL: https://github.com/apache/druid/issues/8715#issuecomment-757130246 I have this issue too. This is an automated message from the Apache Git Service. To respond to the message, please

[GitHub] [druid] LeeMo2K10 commented on issue #10720: Kafka index task is working, how can not found consumer groups

LeeMo2K10 commented on issue #10720: URL: https://github.com/apache/druid/issues/10720#issuecomment-757129714 How do I set the consumer group name？ "spec": { "ioConfig": { "type": "kafka", "consumerProperties": { "bootstrap.servers":

[GitHub] [druid] zhangyue19921010 commented on pull request #10688: Historical reloads damaged segments automatically when lazy on start.

zhangyue19921010 commented on pull request #10688: URL: https://github.com/apache/druid/pull/10688#issuecomment-757117820 Hi @kaijianding and @clintropolis. Sorry to bother you guys. Code coverage(job 14 to job 16) is passed in CI which means all the change requested is finished. Please