[GitHub] [druid] maytasm opened a new pull request #10740: Fix byte calculation for maxBytesInMemory to take into account of Sink/Hydrant Object overhead

2021-01-08 Thread GitBox
maytasm opened a new pull request #10740: URL: https://github.com/apache/druid/pull/10740 Fix byte calculation for maxBytesInMemory to take into account of Sink/Hydrant Object overhead ### Description Fix byte calculation for maxBytesInMemory to take into account of

[druid] branch master updated (2837a9b -> 118b501)

2021-01-08 Thread xvrl
This is an automated email from the ASF dual-hosted git repository. xvrl pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/druid.git. from 2837a9b [Minor Doc Fix] Correct the default value of `druid.server.http.gracefulShutdownTimeout` (#10661) add

[GitHub] [druid] xvrl merged pull request #10730: Introduce KafkaRecordEntity to support Kafka headers in InputFormats

2021-01-08 Thread GitBox
xvrl merged pull request #10730: URL: https://github.com/apache/druid/pull/10730 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [druid] xvrl commented on pull request #10730: Introduce KafkaRecordEntity to support Kafka headers in InputFormats

2021-01-08 Thread GitBox
xvrl commented on pull request #10730: URL: https://github.com/apache/druid/pull/10730#issuecomment-757055227 ok I will merge it as is then. Adding specific tests for the trace logging seems a little overkill. We can add it later if we feel strongly about it.

[druid] branch master updated (d519264 -> 2837a9b)

2021-01-08 Thread jihoonson
This is an automated email from the ASF dual-hosted git repository. jihoonson pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/druid.git. from d519264 remove extra comma (#10670) add 2837a9b [Minor Doc Fix] Correct the default value of

[GitHub] [druid] jihoonson merged pull request #10661: [Minor Doc Fix] Correct the default value of `druid.server.http.gracefulShutdownTimeout`

2021-01-08 Thread GitBox
jihoonson merged pull request #10661: URL: https://github.com/apache/druid/pull/10661 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [druid] jihoonson commented on pull request #10661: [Minor Doc Fix] Correct the default value of `druid.server.http.gracefulShutdownTimeout`

2021-01-08 Thread GitBox
jihoonson commented on pull request #10661: URL: https://github.com/apache/druid/pull/10661#issuecomment-757045393 > I guess if we indeed do want to support shutdown without waiting, we might need another config property to toggle

[druid] branch master updated (3624acb -> d519264)

2021-01-08 Thread jihoonson
This is an automated email from the ASF dual-hosted git repository. jihoonson pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/druid.git. from 3624acb fix web-console show json bug (#10710) add d519264 remove extra comma (#10670) No new revisions

[GitHub] [druid] jihoonson merged pull request #10670: [Doc-Fix] Remove extra comma in `druid/integration-tests/README.md`

2021-01-08 Thread GitBox
jihoonson merged pull request #10670: URL: https://github.com/apache/druid/pull/10670 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [druid] jihoonson commented on pull request #10677: Add URI based allowPrefexList and denyPrefixList

2021-01-08 Thread GitBox
jihoonson commented on pull request #10677: URL: https://github.com/apache/druid/pull/10677#issuecomment-757043116 @nishantmonu51 BTW, since we are about to cut a branch for 0.21.0 release (scheduled on the next Monday), do you think we can get this PR merged before the branch cut? I'm

[GitHub] [druid] jihoonson commented on a change in pull request #10677: Add URI based allowPrefexList and denyPrefixList

2021-01-08 Thread GitBox
jihoonson commented on a change in pull request #10677: URL: https://github.com/apache/druid/pull/10677#discussion_r554239833 ## File path: indexing-service/src/main/java/org/apache/druid/indexing/common/task/CompactionTask.java ## @@ -153,6 +154,9 @@ @JsonIgnore

[GitHub] [druid] zhangyue19921010 commented on pull request #10688: Historical reloads damaged segments automatically when lazy on start.

2021-01-08 Thread GitBox
zhangyue19921010 commented on pull request #10688: URL: https://github.com/apache/druid/pull/10688#issuecomment-757039352 close and reopen to run ci again This is an automated message from the Apache Git Service. To respond

[GitHub] [druid] zhangyue19921010 closed pull request #10688: Historical reloads damaged segments automatically when lazy on start.

2021-01-08 Thread GitBox
zhangyue19921010 closed pull request #10688: URL: https://github.com/apache/druid/pull/10688 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

[druid] branch master updated (c62b7c1 -> 3624acb)

2021-01-08 Thread vogievetsky
This is an automated email from the ASF dual-hosted git repository. vogievetsky pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/druid.git. from c62b7c1 javascript filter result convert to java boolean (#10721) add 3624acb fix web-console show json

[GitHub] [druid] vogievetsky merged pull request #10710: fix web-console show json bug

2021-01-08 Thread GitBox
vogievetsky merged pull request #10710: URL: https://github.com/apache/druid/pull/10710 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[GitHub] [druid] vogievetsky commented on pull request #10710: fix web-console show json bug

2021-01-08 Thread GitBox
vogievetsky commented on pull request #10710: URL: https://github.com/apache/druid/pull/10710#issuecomment-757038204 I think this could be tested by adding a snapshot test to each of the changes to make sure the correct dom is rendered even when there are bigints in the JSON. That would

[GitHub] [druid] gianm commented on pull request #10635: Add SQL functions to format numbers into human readable format

2021-01-08 Thread GitBox
gianm commented on pull request #10635: URL: https://github.com/apache/druid/pull/10635#issuecomment-757031980 My 2¢ on the naming thing: I like having 3 separate functions, because I think a 1-function model works best if you have a good spec for format strings, which isn't what this

[druid] branch master updated (f66fdbf -> c62b7c1)

2021-01-08 Thread jihoonson
This is an automated email from the ASF dual-hosted git repository. jihoonson pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/druid.git. from f66fdbf add offsetFetchPeriod to kinesis ingestion doc (#10734) add c62b7c1 javascript filter result

[GitHub] [druid] jihoonson closed issue #10719: Javascript filter return true when the expression result is false

2021-01-08 Thread GitBox
jihoonson closed issue #10719: URL: https://github.com/apache/druid/issues/10719 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [druid] jihoonson merged pull request #10721: javascript filter result convert to java boolean

2021-01-08 Thread GitBox
jihoonson merged pull request #10721: URL: https://github.com/apache/druid/pull/10721 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [druid] gianm commented on a change in pull request #10635: Add SQL functions to format numbers into human readable format

2021-01-08 Thread GitBox
gianm commented on a change in pull request #10635: URL: https://github.com/apache/druid/pull/10635#discussion_r554228178 ## File path: docs/querying/sql.md ## @@ -563,6 +563,9 @@ The [DataSketches extension](../development/extensions-core/datasketches-extensi

[GitHub] [druid] capistrant opened a new pull request #10739: Druid Server Guilds and Guild Aware Replication

2021-01-08 Thread GitBox
capistrant opened a new pull request #10739: URL: https://github.com/apache/druid/pull/10739 Fixes #9816 ### Description ***Note***: I use the term "Guild" throughout this proposal. It is an arbitrary name that I chose for the Historical grouping construct that I am

[druid] branch master updated (6eef0e4 -> f66fdbf)

2021-01-08 Thread jihoonson
This is an automated email from the ASF dual-hosted git repository. jihoonson pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/druid.git. from 6eef0e4 Fix collision between #10689 and #10593. (#10738) add f66fdbf add offsetFetchPeriod to kinesis

[GitHub] [druid] jihoonson merged pull request #10734: add offsetFetchPeriod to kinesis ingestion doc

2021-01-08 Thread GitBox
jihoonson merged pull request #10734: URL: https://github.com/apache/druid/pull/10734 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [druid] gianm commented on a change in pull request #10605: bitwise math function expressions

2021-01-08 Thread GitBox
gianm commented on a change in pull request #10605: URL: https://github.com/apache/druid/pull/10605#discussion_r554224361 ## File path: core/src/test/java/org/apache/druid/math/expr/FunctionTest.java ## @@ -519,6 +519,31 @@ public void testLeast() assertExpr("least(1,

[GitHub] [druid] gianm commented on a change in pull request #10605: bitwise math function expressions

2021-01-08 Thread GitBox
gianm commented on a change in pull request #10605: URL: https://github.com/apache/druid/pull/10605#discussion_r554224182 ## File path: docs/misc/math-expr.md ## @@ -119,6 +119,13 @@ See javadoc of java.lang.Math for detailed explanation for each function. |acos|acos(x)

[GitHub] [druid] clintropolis commented on a change in pull request #10605: bitwise math function expressions

2021-01-08 Thread GitBox
clintropolis commented on a change in pull request #10605: URL: https://github.com/apache/druid/pull/10605#discussion_r554206473 ## File path: docs/misc/math-expr.md ## @@ -119,6 +119,13 @@ See javadoc of java.lang.Math for detailed explanation for each function.

[GitHub] [druid] gianm commented on pull request #10730: Introduce KafkaRecordEntity to support Kafka headers in InputFormats

2021-01-08 Thread GitBox
gianm commented on pull request #10730: URL: https://github.com/apache/druid/pull/10730#issuecomment-757000160 > looks like tests are failing due to insufficient diff coverage for some lines. Many of them appear to be where we introduced additional type parameters. @gianm what's the

[GitHub] [druid] xvrl commented on pull request #10730: Introduce KafkaRecordEntity to support Kafka headers in InputFormats

2021-01-08 Thread GitBox
xvrl commented on pull request #10730: URL: https://github.com/apache/druid/pull/10730#issuecomment-756995841 looks like tests are failing due to insufficient diff coverage for some lines. Many of them appear to be where we introduced additional type parameters. @gianm what's the guidance

[GitHub] [druid] gianm merged pull request #10738: Fix collision between #10689 and #10593.

2021-01-08 Thread GitBox
gianm merged pull request #10738: URL: https://github.com/apache/druid/pull/10738 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [druid] suneet-s commented on pull request #10710: fix web-console show json bug

2021-01-08 Thread GitBox
suneet-s commented on pull request #10710: URL: https://github.com/apache/druid/pull/10710#issuecomment-756933916 > > @bananaaggle Thanks for this fix - can you please add a test that catches parsing these large numbers in the web-console so that future refactorings don't accidentally

[GitHub] [druid] capistrant commented on a change in pull request #10287: allow the LogUsedSegments duty to be skipped

2021-01-08 Thread GitBox
capistrant commented on a change in pull request #10287: URL: https://github.com/apache/druid/pull/10287#discussion_r554119706 ## File path: server/src/main/java/org/apache/druid/server/coordinator/DruidCoordinator.java ## @@ -669,21 +669,31 @@ private void stopBeingLeader()

[GitHub] [druid] capistrant commented on a change in pull request #10287: allow the LogUsedSegments duty to be skipped

2021-01-08 Thread GitBox
capistrant commented on a change in pull request #10287: URL: https://github.com/apache/druid/pull/10287#discussion_r554117678 ## File path: server/src/main/java/org/apache/druid/server/coordinator/DruidCoordinator.java ## @@ -669,21 +669,31 @@ private void stopBeingLeader()

[GitHub] [druid] capistrant commented on pull request #10287: allow the LogUsedSegments duty to be skipped

2021-01-08 Thread GitBox
capistrant commented on pull request #10287: URL: https://github.com/apache/druid/pull/10287#issuecomment-756913585 > I like this log message from LogUsedSegments and have found it very useful: > > ```java > log.info("Found [%,d] used segments.", params.getUsedSegments().size());

[GitHub] [druid] gianm commented on pull request #10247: Remove CloseQuietly and migrate its usages to other methods.

2021-01-08 Thread GitBox
gianm commented on pull request #10247: URL: https://github.com/apache/druid/pull/10247#issuecomment-756907541 Fixed up some merge conflicts. This is an automated message from the Apache Git Service. To respond to the

[druid] branch master updated (26bcd47 -> 6eef0e4)

2021-01-08 Thread gian
This is an automated email from the ASF dual-hosted git repository. gian pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/druid.git. from 26bcd47 Thread-safety for ResponseContext.REGISTERED_KEYS (#9667) add 6eef0e4 Fix collision between #10689 and

[GitHub] [druid] gianm commented on pull request #10738: Fix collision between #10689 and #10593.

2021-01-08 Thread GitBox
gianm commented on pull request #10738: URL: https://github.com/apache/druid/pull/10738#issuecomment-756905866 Travis passed. I'll merge this. This is an automated message from the Apache Git Service. To respond to the

[GitHub] [druid] liran-funaro commented on pull request #10593: IncrementalIndex Tests and Benchmarks Parametrization

2021-01-08 Thread GitBox
liran-funaro commented on pull request #10593: URL: https://github.com/apache/druid/pull/10593#issuecomment-756795145 Thanks! This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [druid] FrankChen021 edited a comment on issue #10735: Misleading info about tsColumn in the lookups-cached-global docs

2021-01-08 Thread GitBox
FrankChen021 edited a comment on issue #10735: URL: https://github.com/apache/druid/issues/10735#issuecomment-756784327 I guess why `getLookupPairs` currently reloads the whole table is that if there're records deletions in database, such a full reload could drop those deleted records

[GitHub] [druid] FrankChen021 commented on issue #10735: Misleading info about tsColumn in the lookups-cached-global docs

2021-01-08 Thread GitBox
FrankChen021 commented on issue #10735: URL: https://github.com/apache/druid/issues/10735#issuecomment-756784327 I guess why `getLookupPairs` currently reload the whole table is that if there're records deletions in database, such a full reload could drop those deleted records from cache.

[GitHub] [druid] ayushkul2910 commented on a change in pull request #10448: Added CronScheduler support as a proof to clock drift while emitting metrics

2021-01-08 Thread GitBox
ayushkul2910 commented on a change in pull request #10448: URL: https://github.com/apache/druid/pull/10448#discussion_r553893369 ## File path: server/src/main/java/org/apache/druid/server/metrics/MetricsModule.java ## @@ -108,10 +108,10 @@ public MonitorScheduler

[GitHub] [druid] clintropolis commented on a change in pull request #10613: expression filter support for vectorized query engines

2021-01-08 Thread GitBox
clintropolis commented on a change in pull request #10613: URL: https://github.com/apache/druid/pull/10613#discussion_r553847083 ## File path: core/src/main/java/org/apache/druid/math/expr/BinaryLogicalOperatorExpr.java ## @@ -74,7 +74,7 @@ public ExprType

[GitHub] [druid] clintropolis commented on a change in pull request #10613: expression filter support for vectorized query engines

2021-01-08 Thread GitBox
clintropolis commented on a change in pull request #10613: URL: https://github.com/apache/druid/pull/10613#discussion_r553845596 ## File path: processing/src/main/java/org/apache/druid/query/dimension/DimensionSpec.java ## @@ -70,6 +71,11 @@ default

[GitHub] [druid] clintropolis commented on a change in pull request #10613: expression filter support for vectorized query engines

2021-01-08 Thread GitBox
clintropolis commented on a change in pull request #10613: URL: https://github.com/apache/druid/pull/10613#discussion_r553840570 ## File path: processing/src/main/java/org/apache/druid/query/groupby/epinephelinae/vector/GroupByVectorColumnProcessorFactory.java ## @@ -64,7

[GitHub] [druid] clintropolis commented on a change in pull request #10613: expression filter support for vectorized query engines

2021-01-08 Thread GitBox
clintropolis commented on a change in pull request #10613: URL: https://github.com/apache/druid/pull/10613#discussion_r553833318 ## File path: processing/src/main/java/org/apache/druid/query/filter/DruidPredicateFactory.java ## @@ -27,6 +27,13 @@ { Predicate

[GitHub] [druid] clintropolis commented on a change in pull request #10613: expression filter support for vectorized query engines

2021-01-08 Thread GitBox
clintropolis commented on a change in pull request #10613: URL: https://github.com/apache/druid/pull/10613#discussion_r553833052 ## File path: processing/src/main/java/org/apache/druid/query/filter/DruidPredicateFactory.java ## @@ -27,6 +27,13 @@ { Predicate

[GitHub] [druid] clintropolis commented on a change in pull request #10613: expression filter support for vectorized query engines

2021-01-08 Thread GitBox
clintropolis commented on a change in pull request #10613: URL: https://github.com/apache/druid/pull/10613#discussion_r553820619 ## File path: processing/src/main/java/org/apache/druid/segment/vector/VectorColumnSelectorFactory.java ## @@ -57,6 +57,15 @@ default int

[GitHub] [druid] clintropolis commented on a change in pull request #10613: expression filter support for vectorized query engines

2021-01-08 Thread GitBox
clintropolis commented on a change in pull request #10613: URL: https://github.com/apache/druid/pull/10613#discussion_r553818153 ## File path: processing/src/main/java/org/apache/druid/segment/filter/ExpressionFilter.java ## @@ -55,6 +69,105 @@ public ExpressionFilter(final

[GitHub] [druid] gianm commented on pull request #10267: DruidInputSource: Fix issues in column projection, timestamp handling.

2021-01-08 Thread GitBox
gianm commented on pull request #10267: URL: https://github.com/apache/druid/pull/10267#issuecomment-756633682 Fixed up some merge conflicts. This is an automated message from the Apache Git Service. To respond to the

[GitHub] [druid] clintropolis commented on a change in pull request #10613: expression filter support for vectorized query engines

2021-01-08 Thread GitBox
clintropolis commented on a change in pull request #10613: URL: https://github.com/apache/druid/pull/10613#discussion_r553816206 ## File path: processing/src/main/java/org/apache/druid/segment/filter/ExpressionFilter.java ## @@ -55,6 +69,105 @@ public ExpressionFilter(final

[GitHub] [druid] clintropolis commented on a change in pull request #10613: expression filter support for vectorized query engines

2021-01-08 Thread GitBox
clintropolis commented on a change in pull request #10613: URL: https://github.com/apache/druid/pull/10613#discussion_r553814648 ## File path: processing/src/main/java/org/apache/druid/segment/DimensionHandlerUtils.java ## @@ -305,17 +305,29 @@ private static

[GitHub] [druid] gianm commented on pull request #10287: allow the LogUsedSegments duty to be skipped

2021-01-08 Thread GitBox
gianm commented on pull request #10287: URL: https://github.com/apache/druid/pull/10287#issuecomment-756630739 > circling back on this. my cluster with 161k segments takes ~20ms to execute `LogUsedSegments`. This duty should scale fairly linearly with number of segments so you'd have to

[GitHub] [druid] clintropolis commented on a change in pull request #10605: bitwise math function expressions

2021-01-08 Thread GitBox
clintropolis commented on a change in pull request #10605: URL: https://github.com/apache/druid/pull/10605#discussion_r553811715 ## File path: core/src/test/java/org/apache/druid/math/expr/FunctionTest.java ## @@ -519,6 +519,31 @@ public void testLeast()

[GitHub] [druid] clintropolis commented on a change in pull request #10605: bitwise math function expressions

2021-01-08 Thread GitBox
clintropolis commented on a change in pull request #10605: URL: https://github.com/apache/druid/pull/10605#discussion_r553810857 ## File path: docs/misc/math-expr.md ## @@ -119,6 +119,13 @@ See javadoc of java.lang.Math for detailed explanation for each function.

[GitHub] [druid] clintropolis commented on a change in pull request #10605: bitwise math function expressions

2021-01-08 Thread GitBox
clintropolis commented on a change in pull request #10605: URL: https://github.com/apache/druid/pull/10605#discussion_r553810678 ## File path: docs/misc/math-expr.md ## @@ -119,6 +119,13 @@ See javadoc of java.lang.Math for detailed explanation for each function.

[GitHub] [druid] clintropolis commented on a change in pull request #10605: bitwise math function expressions

2021-01-08 Thread GitBox
clintropolis commented on a change in pull request #10605: URL: https://github.com/apache/druid/pull/10605#discussion_r553810003 ## File path: core/src/main/java/org/apache/druid/math/expr/Evals.java ## @@ -71,4 +71,14 @@ public static boolean asBoolean(@Nullable String x)

[druid] branch master updated (08ab82f -> 26bcd47)

2021-01-08 Thread gian
This is an automated email from the ASF dual-hosted git repository. gian pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/druid.git. from 08ab82f IncrementalIndex Tests and Benchmarks Parametrization (#10593) add 26bcd47 Thread-safety for

[GitHub] [druid] gianm merged pull request #9667: Thread-safety for ResponseContext.REGISTERED_KEYS

2021-01-08 Thread GitBox
gianm merged pull request #9667: URL: https://github.com/apache/druid/pull/9667 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [druid] gianm closed issue #9106: Unbalanced synchronization of ResponseContext.REGISTERED_KEYS

2021-01-08 Thread GitBox
gianm closed issue #9106: URL: https://github.com/apache/druid/issues/9106 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [druid] gianm commented on pull request #9667: Thread-safety for ResponseContext.REGISTERED_KEYS

2021-01-08 Thread GitBox
gianm commented on pull request #9667: URL: https://github.com/apache/druid/pull/9667#issuecomment-756625948 Thanks @alex-plekhanov! This is an automated message from the Apache Git Service. To respond to the message, please

[GitHub] [druid] stale[bot] commented on pull request #10043: Add support for REGEXP_REPLACE

2021-01-08 Thread GitBox
stale[bot] commented on pull request #10043: URL: https://github.com/apache/druid/pull/10043#issuecomment-756624168 This issue is no longer marked as stale. This is an automated message from the Apache Git Service. To

[GitHub] [druid] gianm commented on pull request #10043: Add support for REGEXP_REPLACE

2021-01-08 Thread GitBox
gianm commented on pull request #10043: URL: https://github.com/apache/druid/pull/10043#issuecomment-756624149 > one more time please stalebot  two more times?  This is an automated message from the Apache Git

[GitHub] [druid] gianm commented on a change in pull request #10613: expression filter support for vectorized query engines

2021-01-08 Thread GitBox
gianm commented on a change in pull request #10613: URL: https://github.com/apache/druid/pull/10613#discussion_r553782719 ## File path: core/src/main/java/org/apache/druid/math/expr/BinaryLogicalOperatorExpr.java ## @@ -74,7 +74,7 @@ public ExprType