[GitHub] [druid] zachjsh commented on a diff in pull request #13165: Druid Catalog basics

2022-11-03 Thread GitBox
zachjsh commented on code in PR #13165: URL: https://github.com/apache/druid/pull/13165#discussion_r1013280671 ## server/src/main/java/org/apache/druid/catalog/model/Columns.java: ## @@ -0,0 +1,105 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more

[GitHub] [druid] churromorales commented on pull request #13305: Fixing the K8s task runner to work with MSQ

2022-11-03 Thread GitBox
churromorales commented on PR #13305: URL: https://github.com/apache/druid/pull/13305#issuecomment-1302534201 @gianm you were right, the bug was due to the worker task's pod being scheduled but it wasn't yet launched (so it had no podIP yet) thus, the task location was partially set. The f

[GitHub] [druid] zachjsh commented on a diff in pull request #13165: Druid Catalog basics

2022-11-03 Thread GitBox
zachjsh commented on code in PR #13165: URL: https://github.com/apache/druid/pull/13165#discussion_r1013289389 ## server/src/main/java/org/apache/druid/catalog/model/ModelProperties.java: ## @@ -0,0 +1,305 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + *

[GitHub] [druid] zachjsh commented on a diff in pull request #13165: Druid Catalog basics

2022-11-03 Thread GitBox
zachjsh commented on code in PR #13165: URL: https://github.com/apache/druid/pull/13165#discussion_r1013324724 ## server/src/main/java/org/apache/druid/catalog/model/ObjectDefn.java: ## @@ -0,0 +1,138 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or m

[GitHub] [druid] gianm commented on a diff in pull request #13280: Compaction: Fetch segments one at a time on main task; skip when possible.

2022-11-03 Thread GitBox
gianm commented on code in PR #13280: URL: https://github.com/apache/druid/pull/13280#discussion_r1013176074 ## indexing-service/src/main/java/org/apache/druid/indexing/common/task/CompactionTask.java: ## @@ -1014,6 +897,254 @@ static DimensionSchema createDimensionSchema(

[GitHub] [druid] gianm opened a new pull request, #13306: Update Curator in licenses.yaml.

2022-11-03 Thread GitBox
gianm opened a new pull request, #13306: URL: https://github.com/apache/druid/pull/13306 For some reason, Travis CI didn't run on #13302, so we didn't catch that the `licenses.yaml` update was missing. This patch fixes the omission. -- This is an automated message from the Apache Git Serv

[GitHub] [druid] zachjsh commented on a diff in pull request #13165: Druid Catalog basics

2022-11-03 Thread GitBox
zachjsh commented on code in PR #13165: URL: https://github.com/apache/druid/pull/13165#discussion_r1013357390 ## server/src/main/java/org/apache/druid/catalog/model/ParameterizedDefn.java: ## @@ -0,0 +1,62 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one +

[GitHub] [druid] zachjsh commented on a diff in pull request #13165: Druid Catalog basics

2022-11-03 Thread GitBox
zachjsh commented on code in PR #13165: URL: https://github.com/apache/druid/pull/13165#discussion_r1013359130 ## server/src/main/java/org/apache/druid/catalog/model/ParameterizedDefn.java: ## @@ -0,0 +1,62 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one +

[GitHub] [druid] zachjsh commented on a diff in pull request #13165: Druid Catalog basics

2022-11-03 Thread GitBox
zachjsh commented on code in PR #13165: URL: https://github.com/apache/druid/pull/13165#discussion_r1013359130 ## server/src/main/java/org/apache/druid/catalog/model/ParameterizedDefn.java: ## @@ -0,0 +1,62 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one +

[GitHub] [druid] techdocsmith opened a new issue, #13307: Improve Tuple sketches docs

2022-11-03 Thread GitBox
techdocsmith opened a new issue, #13307: URL: https://github.com/apache/druid/issues/13307 From @gianm : Think of a tuple sketch as a theta sketch plus an additional vector of numeric values associated with each item being added to the sketch. The items added to the sketch are counte

[GitHub] [druid] amaechler opened a new pull request, #13308: Add missing MSQ error code fields to docs

2022-11-03 Thread GitBox
amaechler opened a new pull request, #13308: URL: https://github.com/apache/druid/pull/13308 ### Description This PR attempts to add some missing information to the MSQ error codes section. It might be easiest to review the four (small) commits individually (to distinguish white-spac

[GitHub] [druid] techdocsmith commented on a diff in pull request #13308: Add missing MSQ error code fields to docs

2022-11-03 Thread GitBox
techdocsmith commented on code in PR #13308: URL: https://github.com/apache/druid/pull/13308#discussion_r1013393862 ## docs/multi-stage-query/reference.md: ## @@ -226,32 +226,33 @@ The following table lists query limits: The following table describes error codes you may encou

[GitHub] [druid] zachjsh commented on a diff in pull request #13165: Druid Catalog basics

2022-11-03 Thread GitBox
zachjsh commented on code in PR #13165: URL: https://github.com/apache/druid/pull/13165#discussion_r1013404973 ## server/src/main/java/org/apache/druid/catalog/model/TableDefn.java: ## @@ -0,0 +1,176 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or mo

[GitHub] [druid] zachjsh commented on a diff in pull request #13165: Druid Catalog basics

2022-11-03 Thread GitBox
zachjsh commented on code in PR #13165: URL: https://github.com/apache/druid/pull/13165#discussion_r1013406794 ## server/src/main/java/org/apache/druid/catalog/model/TableDefnRegistry.java: ## @@ -0,0 +1,87 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one +

[GitHub] [druid] zachjsh commented on a diff in pull request #13165: Druid Catalog basics

2022-11-03 Thread GitBox
zachjsh commented on code in PR #13165: URL: https://github.com/apache/druid/pull/13165#discussion_r1013407241 ## server/src/main/java/org/apache/druid/catalog/model/TableDefnRegistry.java: ## @@ -0,0 +1,87 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one +

[GitHub] [druid] zachjsh commented on a diff in pull request #13165: Druid Catalog basics

2022-11-03 Thread GitBox
zachjsh commented on code in PR #13165: URL: https://github.com/apache/druid/pull/13165#discussion_r1013408875 ## server/src/main/java/org/apache/druid/catalog/model/TableId.java: ## @@ -0,0 +1,113 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more

[GitHub] [druid] zachjsh commented on a diff in pull request #13165: Druid Catalog basics

2022-11-03 Thread GitBox
zachjsh commented on code in PR #13165: URL: https://github.com/apache/druid/pull/13165#discussion_r1013411076 ## server/src/main/java/org/apache/druid/catalog/model/TableMetadata.java: ## @@ -0,0 +1,300 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * o

[GitHub] [druid] zachjsh commented on a diff in pull request #13165: Druid Catalog basics

2022-11-03 Thread GitBox
zachjsh commented on code in PR #13165: URL: https://github.com/apache/druid/pull/13165#discussion_r1013411733 ## server/src/main/java/org/apache/druid/catalog/model/TableMetadata.java: ## @@ -0,0 +1,300 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * o

[GitHub] [druid] zachjsh commented on a diff in pull request #13165: Druid Catalog basics

2022-11-03 Thread GitBox
zachjsh commented on code in PR #13165: URL: https://github.com/apache/druid/pull/13165#discussion_r1013413235 ## server/src/main/java/org/apache/druid/catalog/model/TableSpec.java: ## @@ -0,0 +1,143 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or mo

[GitHub] [druid] amaechler commented on pull request #13308: Add missing MSQ error code fields to docs

2022-11-03 Thread GitBox
amaechler commented on PR #13308: URL: https://github.com/apache/druid/pull/13308#issuecomment-1302684825 @techdocsmith Thanks for the first round of review @techdocsmith. @317brian pointed out to me that we [previously](https://github.com/apache/druid/pull/12983/commits/bbdff97d72148a8c249

[GitHub] [druid] paul-rogers commented on issue #12643: Flaky Test: RemoteTaskRunnerTest

2022-11-03 Thread GitBox
paul-rogers commented on issue #12643: URL: https://github.com/apache/druid/issues/12643#issuecomment-1302686598 Failed again in [this build](https://app.travis-ci.com/github/apache/druid/jobs/587271323) (JDK 17): ```text [INFO] Running org.apache.druid.indexing.overlord.Remote

[GitHub] [druid] amaechler commented on a diff in pull request #13308: Add missing MSQ error code fields to docs

2022-11-03 Thread GitBox
amaechler commented on code in PR #13308: URL: https://github.com/apache/druid/pull/13308#discussion_r1013441822 ## docs/multi-stage-query/reference.md: ## @@ -226,32 +226,33 @@ The following table lists query limits: The following table describes error codes you may encounte

[GitHub] [druid] gianm merged pull request #13306: Update Curator in licenses.yaml.

2022-11-03 Thread GitBox
gianm merged PR #13306: URL: https://github.com/apache/druid/pull/13306 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@druid.apache

[GitHub] [druid] lgtm-com[bot] commented on pull request #13165: Druid Catalog basics

2022-11-03 Thread GitBox
lgtm-com[bot] commented on PR #13165: URL: https://github.com/apache/druid/pull/13165#issuecomment-1302761340 This pull request **introduces 1 alert** when merging 5de07ce9341ac324b8bf1acd1844fa8398e6a8a3 into c875f4bd0402d49c57d56068537b8babc52386d1 - [view on LGTM.com](https://lgtm.com/p

[GitHub] [druid] jon-wei merged pull request #13296: Make RecordSupplierInputSource respect sampler timeout when stream is empty

2022-11-03 Thread GitBox
jon-wei merged PR #13296: URL: https://github.com/apache/druid/pull/13296 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@druid.apac

[GitHub] [druid] amaechler commented on a diff in pull request #13308: Add missing MSQ error code fields to docs

2022-11-03 Thread GitBox
amaechler commented on code in PR #13308: URL: https://github.com/apache/druid/pull/13308#discussion_r1013466751 ## docs/multi-stage-query/reference.md: ## @@ -226,32 +226,33 @@ The following table lists query limits: The following table describes error codes you may encounte

[GitHub] [druid] paul-rogers commented on issue #12359: Simplify and Accelerate Integration Tests

2022-11-03 Thread GitBox
paul-rogers commented on issue #12359: URL: https://github.com/apache/druid/issues/12359#issuecomment-1302770563 I realized that this issue hadn't been updated in some time. The "new IT framework" has been merged into `master`. Several existing tests are converted. New test should use the n

[GitHub] [druid] clintropolis opened a new pull request, #13309: fix bug when front-coded index has only the null value

2022-11-03 Thread GitBox
clintropolis opened a new pull request, #13309: URL: https://github.com/apache/druid/pull/13309 ### Description This PR fixes a bug with `FrontCodedIndexed` when containing only a single null value. Because the null value is not actually stored in the "value buckets", it can result in a

[GitHub] [druid] cryptoe commented on a diff in pull request #13308: Add missing MSQ error code fields to docs

2022-11-03 Thread GitBox
cryptoe commented on code in PR #13308: URL: https://github.com/apache/druid/pull/13308#discussion_r1013537359 ## docs/multi-stage-query/reference.md: ## @@ -226,32 +226,32 @@ The following table lists query limits: The following table describes error codes you may encounter

[GitHub] [druid] AmatyaAvadhanula commented on a diff in pull request #13304: Skip ALL to finer granularity compaction

2022-11-03 Thread GitBox
AmatyaAvadhanula commented on code in PR #13304: URL: https://github.com/apache/druid/pull/13304#discussion_r1013553662 ## server/src/main/java/org/apache/druid/server/coordinator/duty/NewestSegmentFirstIterator.java: ## @@ -129,6 +131,12 @@ public class NewestSegmentFirstIterat

[GitHub] [druid] AmatyaAvadhanula commented on issue #13208: Coordinator crashes trying to compact a datasource ingested through MSQ with PARTITIONED BY ALL

2022-11-03 Thread GitBox
AmatyaAvadhanula commented on issue #13208: URL: https://github.com/apache/druid/issues/13208#issuecomment-1302886876 https://github.com/apache/druid/pull/13304 tries to skip compaction when segments of ALL granularity are being compacted to a finer granularity. While it prevents a coord

[GitHub] [druid] AmatyaAvadhanula commented on a diff in pull request #13304: Skip ALL to finer granularity compaction

2022-11-03 Thread GitBox
AmatyaAvadhanula commented on code in PR #13304: URL: https://github.com/apache/druid/pull/13304#discussion_r1013555688 ## server/src/main/java/org/apache/druid/server/coordinator/duty/NewestSegmentFirstIterator.java: ## @@ -129,6 +131,12 @@ public class NewestSegmentFirstIterat

[GitHub] [druid] AmatyaAvadhanula commented on pull request #13304: Skip ALL to finer granularity compaction

2022-11-03 Thread GitBox
AmatyaAvadhanula commented on PR #13304: URL: https://github.com/apache/druid/pull/13304#issuecomment-1302889867 @gianm, thank you for the review. I've addressed your feedback, and will also try to enable compaction from ALL granularity in subsequent PRs -- This is an automated mes

[GitHub] [druid] abhishekagarwal87 commented on pull request #13304: Skip ALL to finer granularity compaction

2022-11-03 Thread GitBox
abhishekagarwal87 commented on PR #13304: URL: https://github.com/apache/druid/pull/13304#issuecomment-1302958993 @AmatyaAvadhanula - Please add this gotcha to the docs as well. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

[GitHub] [druid] abhishekagarwal87 commented on a diff in pull request #13304: Skip ALL to finer granularity compaction

2022-11-03 Thread GitBox
abhishekagarwal87 commented on code in PR #13304: URL: https://github.com/apache/druid/pull/13304#discussion_r1013603403 ## server/src/main/java/org/apache/druid/server/coordinator/duty/NewestSegmentFirstIterator.java: ## @@ -129,6 +131,12 @@ public class NewestSegmentFirstItera

[GitHub] [druid] abhishekagarwal87 merged pull request #13301: Suppressing package-lock.json?d3-color vulnerability

2022-11-03 Thread GitBox
abhishekagarwal87 merged PR #13301: URL: https://github.com/apache/druid/pull/13301 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@

[GitHub] [druid] abhishekagarwal87 commented on pull request #13301: Suppressing package-lock.json?d3-color vulnerability

2022-11-03 Thread GitBox
abhishekagarwal87 commented on PR #13301: URL: https://github.com/apache/druid/pull/13301#issuecomment-1303025470 Thank you for your contribution, @abhagraw -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL abov

[GitHub] [druid] kfaraz opened a new pull request, #13310: Improve fetch of pending segments from metadata store

2022-11-03 Thread GitBox
kfaraz opened a new pull request, #13310: URL: https://github.com/apache/druid/pull/13310 The fetch of pending segments happens behind a lock and can cause others threads to remain stuck while deserializing the payload fetched from the metadata store. __Changes:__ - Deserialize th

[GitHub] [druid] kfaraz commented on a diff in pull request #13310: Improve fetch of pending segments from metadata store

2022-11-04 Thread GitBox
kfaraz commented on code in PR #13310: URL: https://github.com/apache/druid/pull/13310#discussion_r1013689539 ## server/src/main/java/org/apache/druid/metadata/IndexerSQLMetadataStorageCoordinator.java: ## @@ -943,7 +947,7 @@ private SegmentIdWithShardSpec createNewSegment(

[GitHub] [druid] abhishekagarwal87 commented on pull request #13305: Fixing the K8s task runner to work with MSQ

2022-11-04 Thread GitBox
abhishekagarwal87 commented on PR #13305: URL: https://github.com/apache/druid/pull/13305#issuecomment-1303101999 @churromorales - does this PR has all the changes pushed? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and u

[GitHub] [druid] tejaswini-imply opened a new pull request, #13311: Adds cluster level idleConfig setting for supervisor

2022-11-04 Thread GitBox
tejaswini-imply opened a new pull request, #13311: URL: https://github.com/apache/druid/pull/13311 This PR enables cluster admin to define default idle config for all supervisors. The new cluster-level settings and their defaults are: ``` druid.supervisor.idleConfig.enabled = false

[GitHub] [druid] rohangarg commented on pull request #13310: Improve fetch of pending segments from metadata store

2022-11-04 Thread GitBox
rohangarg commented on PR #13310: URL: https://github.com/apache/druid/pull/13310#issuecomment-1303178331 Regarding the suggested query improvement, I think `BETWEEN` is an inclusive operator on both sides so it might also bring the same results as the current query. Also, the performance i

[GitHub] [druid] AmatyaAvadhanula commented on a diff in pull request #13304: Skip ALL to finer granularity compaction

2022-11-04 Thread GitBox
AmatyaAvadhanula commented on code in PR #13304: URL: https://github.com/apache/druid/pull/13304#discussion_r1013809825 ## server/src/main/java/org/apache/druid/server/coordinator/duty/NewestSegmentFirstIterator.java: ## @@ -129,6 +131,12 @@ public class NewestSegmentFirstIterat

[GitHub] [druid] AmatyaAvadhanula commented on pull request #13304: Skip ALL to finer granularity compaction

2022-11-04 Thread GitBox
AmatyaAvadhanula commented on PR #13304: URL: https://github.com/apache/druid/pull/13304#issuecomment-1303189884 @abhishekagarwal87 thank you for the feedback. I've added a note in the auto-compaction docs -- This is an automated message from the Apache Git Service. To respond to the mess

[GitHub] [druid] kfaraz commented on pull request #13310: Improve fetch of pending segments from metadata store

2022-11-04 Thread GitBox
kfaraz commented on PR #13310: URL: https://github.com/apache/druid/pull/13310#issuecomment-1303258952 Thanks for the suggestion, @rohangarg ! So, I guess we can just remove the equals from both the operators. I should have looked into the `overlaps` closer, I just assumed it would

[GitHub] [druid] clintropolis opened a new issue, #13312: modernize parquet logical type conversion

2022-11-04 Thread GitBox
clintropolis opened a new issue, #13312: URL: https://github.com/apache/druid/issues/13312 ### Description The [Parquet converter](https://github.com/apache/druid/blob/master/extensions-core/parquet-extensions/src/main/java/org/apache/druid/data/input/parquet/simple/ParquetGroupConverter.

[GitHub] [druid] clintropolis merged pull request #13294: fix issue with parquet list conversion of nullable lists with complex nullable elements

2022-11-04 Thread GitBox
clintropolis merged PR #13294: URL: https://github.com/apache/druid/pull/13294 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@druid

[GitHub] [druid] clintropolis merged pull request #13309: fix bug when FrontCodedIndexed has only the null value

2022-11-04 Thread GitBox
clintropolis merged PR #13309: URL: https://github.com/apache/druid/pull/13309 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@druid

[GitHub] [druid] AmatyaAvadhanula commented on a diff in pull request #13238: Add segment handoff time metric

2022-11-04 Thread GitBox
AmatyaAvadhanula commented on code in PR #13238: URL: https://github.com/apache/druid/pull/13238#discussion_r1014066915 ## server/src/main/java/org/apache/druid/segment/realtime/appenderator/StreamAppenderatorDriver.java: ## @@ -332,6 +332,7 @@ public ListenableFuture registerH

[GitHub] [druid] AmatyaAvadhanula commented on a diff in pull request #13238: Add segment handoff time metric

2022-11-04 Thread GitBox
AmatyaAvadhanula commented on code in PR #13238: URL: https://github.com/apache/druid/pull/13238#discussion_r1014067262 ## server/src/main/java/org/apache/druid/segment/realtime/appenderator/StreamAppenderatorDriver.java: ## @@ -359,6 +360,7 @@ public void onSuccess(Object resul

[GitHub] [druid] AmatyaAvadhanula commented on a diff in pull request #13238: Add segment handoff time metric

2022-11-04 Thread GitBox
AmatyaAvadhanula commented on code in PR #13238: URL: https://github.com/apache/druid/pull/13238#discussion_r1014069062 ## server/src/main/java/org/apache/druid/segment/realtime/RealtimeMetricsMonitor.java: ## @@ -137,6 +114,12 @@ public boolean doMonitor(ServiceEmitter emitter)

[GitHub] [druid] AmatyaAvadhanula opened a new pull request, #13313: Improve task pause logging and metrics for streaming ingestion

2022-11-04 Thread GitBox
AmatyaAvadhanula opened a new pull request, #13313: URL: https://github.com/apache/druid/pull/13313 ### Description Improved logging and added a metric related to pause and resume of streaming ingestion tasks

[GitHub] [druid] churromorales commented on pull request #13305: Fixing the K8s task runner to work with MSQ

2022-11-04 Thread GitBox
churromorales commented on PR #13305: URL: https://github.com/apache/druid/pull/13305#issuecomment-1303806747 @abhishekagarwal87 it has even more, in our local fork i didn't write a test ;). Seriously though, here was the original issue: https://github.com/apache/druid/pull/13156#is

[GitHub] [druid] amaechler commented on a diff in pull request #13308: Add missing MSQ error code fields to docs

2022-11-04 Thread GitBox
amaechler commented on code in PR #13308: URL: https://github.com/apache/druid/pull/13308#discussion_r1014229196 ## docs/multi-stage-query/reference.md: ## @@ -226,32 +226,32 @@ The following table lists query limits: The following table describes error codes you may encounte

[GitHub] [druid] writer-jill opened a new pull request, #13314: Update nested columns doc

2022-11-04 Thread GitBox
writer-jill opened a new pull request, #13314: URL: https://github.com/apache/druid/pull/13314 Updated nested columns doc for 25.0. This PR has: - [x] been self-reviewed. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitH

[GitHub] [druid] AmatyaAvadhanula opened a new pull request, #13315: Fix flaky testResumeAllActivelyReadingTasks

2022-11-04 Thread GitBox
AmatyaAvadhanula opened a new pull request, #13315: URL: https://github.com/apache/druid/pull/13315 Fixes flaky testResumeAllActivelyReadingTasks ### Description This

[GitHub] [druid] paul-rogers commented on issue #12643: Flaky Test: RemoteTaskRunnerTest

2022-11-04 Thread GitBox
paul-rogers commented on issue #12643: URL: https://github.com/apache/druid/issues/12643#issuecomment-1303852665 An attempt to rerun also failed, but in a [different way](https://app.travis-ci.com/github/apache/druid/jobs/587271323). There were dozens of failures, starting with: ```t

[GitHub] [druid] ektravel opened a new pull request, #13316: Update Metrics doc

2022-11-04 Thread GitBox
ektravel opened a new pull request, #13316: URL: https://github.com/apache/druid/pull/13316 ### Description Updated Metrics doc as follows: - fixed the formatting for dimensions (code font) - removed unnecessary punctuation This PR has: - [x] been self-reviewed --

[GitHub] [druid] zachjsh commented on a diff in pull request #13165: Druid Catalog basics

2022-11-04 Thread GitBox
zachjsh commented on code in PR #13165: URL: https://github.com/apache/druid/pull/13165#discussion_r1014321311 ## server/src/main/java/org/apache/druid/catalog/model/table/AbstractDatasourceDefn.java: ## @@ -0,0 +1,144 @@ +/* + * Licensed to the Apache Software Foundation (ASF)

[GitHub] [druid] zachjsh commented on a diff in pull request #13165: Druid Catalog basics

2022-11-04 Thread GitBox
zachjsh commented on code in PR #13165: URL: https://github.com/apache/druid/pull/13165#discussion_r1014322689 ## server/src/main/java/org/apache/druid/catalog/model/table/AbstractDatasourceDefn.java: ## @@ -0,0 +1,144 @@ +/* + * Licensed to the Apache Software Foundation (ASF)

[GitHub] [druid] zachjsh commented on a diff in pull request #13165: Druid Catalog basics

2022-11-04 Thread GitBox
zachjsh commented on code in PR #13165: URL: https://github.com/apache/druid/pull/13165#discussion_r1014324137 ## server/src/main/java/org/apache/druid/catalog/model/table/ClusterKeySpec.java: ## @@ -0,0 +1,78 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one

[GitHub] [druid] zachjsh commented on a diff in pull request #13165: Druid Catalog basics

2022-11-04 Thread GitBox
zachjsh commented on code in PR #13165: URL: https://github.com/apache/druid/pull/13165#discussion_r1014325971 ## server/src/main/java/org/apache/druid/catalog/model/table/ClusterKeySpec.java: ## @@ -0,0 +1,83 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one

[GitHub] [druid] zachjsh commented on a diff in pull request #13165: Druid Catalog basics

2022-11-04 Thread GitBox
zachjsh commented on code in PR #13165: URL: https://github.com/apache/druid/pull/13165#discussion_r1014329446 ## server/src/main/java/org/apache/druid/catalog/model/table/ExternalTableDefn.java: ## @@ -0,0 +1,273 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under

[GitHub] [druid] zachjsh commented on a diff in pull request #13165: Druid Catalog basics

2022-11-04 Thread GitBox
zachjsh commented on code in PR #13165: URL: https://github.com/apache/druid/pull/13165#discussion_r1014341908 ## server/src/main/java/org/apache/druid/catalog/model/table/ExternalTableDefn.java: ## @@ -0,0 +1,273 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under

[GitHub] [druid] zachjsh commented on a diff in pull request #13165: Druid Catalog basics

2022-11-04 Thread GitBox
zachjsh commented on code in PR #13165: URL: https://github.com/apache/druid/pull/13165#discussion_r1014342105 ## server/src/main/java/org/apache/druid/catalog/model/table/ExternalTableDefn.java: ## @@ -0,0 +1,273 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under

[GitHub] [druid] zachjsh commented on a diff in pull request #13165: Druid Catalog basics

2022-11-04 Thread GitBox
zachjsh commented on code in PR #13165: URL: https://github.com/apache/druid/pull/13165#discussion_r1014347172 ## server/src/main/java/org/apache/druid/catalog/model/table/HttpTableDefn.java: ## @@ -0,0 +1,190 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one

[GitHub] [druid] zachjsh commented on a diff in pull request #13165: Druid Catalog basics

2022-11-04 Thread GitBox
zachjsh commented on code in PR #13165: URL: https://github.com/apache/druid/pull/13165#discussion_r1014348660 ## server/src/main/java/org/apache/druid/catalog/model/table/HttpTableDefn.java: ## @@ -0,0 +1,190 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one

[GitHub] [druid] zachjsh commented on a diff in pull request #13165: Druid Catalog basics

2022-11-04 Thread GitBox
zachjsh commented on code in PR #13165: URL: https://github.com/apache/druid/pull/13165#discussion_r1014348660 ## server/src/main/java/org/apache/druid/catalog/model/table/HttpTableDefn.java: ## @@ -0,0 +1,190 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one

[GitHub] [druid] zachjsh commented on a diff in pull request #13165: Druid Catalog basics

2022-11-04 Thread GitBox
zachjsh commented on code in PR #13165: URL: https://github.com/apache/druid/pull/13165#discussion_r1014362425 ## server/src/main/java/org/apache/druid/catalog/model/table/InputFormats.java: ## @@ -0,0 +1,301 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one

[GitHub] [druid] zachjsh commented on a diff in pull request #13165: Druid Catalog basics

2022-11-04 Thread GitBox
zachjsh commented on code in PR #13165: URL: https://github.com/apache/druid/pull/13165#discussion_r1014368674 ## server/src/main/java/org/apache/druid/catalog/model/table/LocalTableDefn.java: ## @@ -0,0 +1,123 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under on

[GitHub] [druid] zachjsh commented on a diff in pull request #13165: Druid Catalog basics

2022-11-04 Thread GitBox
zachjsh commented on code in PR #13165: URL: https://github.com/apache/druid/pull/13165#discussion_r1014370314 ## server/src/main/java/org/apache/druid/catalog/model/table/HttpTableDefn.java: ## @@ -0,0 +1,190 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one

[GitHub] [druid] zachjsh commented on a diff in pull request #13165: Druid Catalog basics

2022-11-04 Thread GitBox
zachjsh commented on code in PR #13165: URL: https://github.com/apache/druid/pull/13165#discussion_r1014375054 ## server/src/main/java/org/apache/druid/catalog/model/table/TableBuilder.java: ## @@ -0,0 +1,269 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one

[GitHub] [druid] zachjsh commented on a diff in pull request #13165: Druid Catalog basics

2022-11-04 Thread GitBox
zachjsh commented on code in PR #13165: URL: https://github.com/apache/druid/pull/13165#discussion_r1014375054 ## server/src/main/java/org/apache/druid/catalog/model/table/TableBuilder.java: ## @@ -0,0 +1,269 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one

[GitHub] [druid] zachjsh commented on a diff in pull request #13165: Druid Catalog basics

2022-11-04 Thread GitBox
zachjsh commented on code in PR #13165: URL: https://github.com/apache/druid/pull/13165#discussion_r1014376872 ## server/src/main/java/org/apache/druid/metadata/storage/derby/DerbyMetadataStorage.java: ## @@ -55,6 +54,26 @@ public void start() catch (Exception e) { t

[GitHub] [druid] techdocsmith commented on a diff in pull request #13316: Update Metrics doc

2022-11-04 Thread GitBox
techdocsmith commented on code in PR #13316: URL: https://github.com/apache/druid/pull/13316#discussion_r1014374393 ## docs/operations/metrics.md: ## @@ -118,53 +118,52 @@ Metrics may have additional dimensions beyond those listed above. Review Comment: In line 115 & 116,

[GitHub] [druid] zachjsh commented on a diff in pull request #13165: Druid Catalog basics

2022-11-04 Thread GitBox
zachjsh commented on code in PR #13165: URL: https://github.com/apache/druid/pull/13165#discussion_r1014383800 ## extensions-core/druid-catalog/src/main/java/org/apache/druid/catalog/sync/CachedMetadataCatalog.java: ## @@ -84,14 +97,14 @@ protected TableEntry(TableMetadata table

[GitHub] [druid] zachjsh commented on a diff in pull request #13165: Druid Catalog basics

2022-11-04 Thread GitBox
zachjsh commented on code in PR #13165: URL: https://github.com/apache/druid/pull/13165#discussion_r1014383800 ## extensions-core/druid-catalog/src/main/java/org/apache/druid/catalog/sync/CachedMetadataCatalog.java: ## @@ -84,14 +97,14 @@ protected TableEntry(TableMetadata table

[GitHub] [druid] techdocsmith commented on a diff in pull request #13308: Add missing MSQ error code fields to docs

2022-11-04 Thread GitBox
techdocsmith commented on code in PR #13308: URL: https://github.com/apache/druid/pull/13308#discussion_r1014407892 ## docs/multi-stage-query/reference.md: ## @@ -226,32 +226,32 @@ The following table lists query limits: The following table describes error codes you may encou

[GitHub] [druid] paul-rogers commented on a diff in pull request #13165: Druid Catalog basics

2022-11-04 Thread GitBox
paul-rogers commented on code in PR #13165: URL: https://github.com/apache/druid/pull/13165#discussion_r1014435696 ## server/src/main/java/org/apache/druid/catalog/model/CatalogUtils.java: ## @@ -0,0 +1,147 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one +

[GitHub] [druid] kfaraz commented on pull request #13315: Fix flaky testResumeAllActivelyReadingTasks

2022-11-04 Thread GitBox
kfaraz commented on PR #13315: URL: https://github.com/apache/druid/pull/13315#issuecomment-1304406700 Thanks for the fix, @AmatyaAvadhanula . I hope this run didn't require retries 🙂. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [druid] kfaraz commented on pull request #13310: Improve fetch of pending segments from metadata store

2022-11-04 Thread GitBox
kfaraz commented on PR #13310: URL: https://github.com/apache/druid/pull/13310#issuecomment-1304406916 With the suggested change to the query, I don't think we even need the other changes. What do you think @abhishekagarwal87 , @AmatyaAvadhanula ? -- This is an automated message from the

[GitHub] [druid] kfaraz merged pull request #13315: Fix flaky testResumeAllActivelyReadingTasks

2022-11-04 Thread GitBox
kfaraz merged PR #13315: URL: https://github.com/apache/druid/pull/13315 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@druid.apach

[GitHub] [druid] didip commented on pull request #13027: Use standard library to correctly glob and stop at the correct folder structure when filtering cloud objects

2022-11-04 Thread GitBox
didip commented on PR #13027: URL: https://github.com/apache/druid/pull/13027#issuecomment-1304412404 Fix documentation. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

[GitHub] [druid] kfaraz commented on a diff in pull request #13311: Adds cluster level idleConfig setting for supervisor

2022-11-04 Thread GitBox
kfaraz commented on code in PR #13311: URL: https://github.com/apache/druid/pull/13311#discussion_r1014574126 ## extensions-core/kinesis-indexing-service/src/main/java/org/apache/druid/indexing/kinesis/supervisor/KinesisSupervisorIOConfig.java: ## @@ -91,7 +92,7 @@ public Kinesi

[GitHub] [druid] abhishekagarwal87 commented on pull request #13310: Improve fetch of pending segments from metadata store

2022-11-04 Thread GitBox
abhishekagarwal87 commented on PR #13310: URL: https://github.com/apache/druid/pull/13310#issuecomment-1304420294 I feel that there might be some nuances being missed here. The comparison in java code, that @rohangarg is referring to, is done on long `millis` representation of start and end

[GitHub] [druid] lgtm-com[bot] commented on pull request #13027: Use standard library to correctly glob and stop at the correct folder structure when filtering cloud objects

2022-11-04 Thread GitBox
lgtm-com[bot] commented on PR #13027: URL: https://github.com/apache/druid/pull/13027#issuecomment-1304424989 This pull request **introduces 2 alerts** when merging c91807e52c19efd382d1822839086abac43cddcd into a17ffdfc5d82c8cf60dfb91015f69142e427d76f - [view on LGTM.com](https://lgtm.com/

[GitHub] [druid] FrankChen021 merged pull request #13299: Bump commons-text from 1.9 to 1.10.0 in /extensions-contrib/kubernetes-overlord-extensions

2022-11-05 Thread GitBox
FrankChen021 merged PR #13299: URL: https://github.com/apache/druid/pull/13299 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@druid

[GitHub] [druid] kfaraz commented on pull request #13310: Improve fetch of pending segments from metadata store

2022-11-05 Thread GitBox
kfaraz commented on PR #13310: URL: https://github.com/apache/druid/pull/13310#issuecomment-1304433982 That makes sense, @abhishekagarwal87 . Looking at the other linked PR though, it is possible that even the current query would fail for dates with years that have more or less than 4 digit

[GitHub] [druid] kfaraz commented on a diff in pull request #13304: Skip ALL to finer granularity compaction

2022-11-05 Thread GitBox
kfaraz commented on code in PR #13304: URL: https://github.com/apache/druid/pull/13304#discussion_r1014601675 ## server/src/test/java/org/apache/druid/server/coordinator/duty/NewestSegmentFirstPolicyTest.java: ## @@ -1584,6 +1585,37 @@ public void testIteratorDoesNotReturnSegme

[GitHub] [druid] AshishKapoor closed issue #10806: maven-assembly-plugin issue with mac M1 chip

2022-11-05 Thread GitBox
AshishKapoor closed issue #10806: maven-assembly-plugin issue with mac M1 chip URL: https://github.com/apache/druid/issues/10806 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

[GitHub] [druid] abhishekagarwal87 commented on a diff in pull request #13305: Fixing the K8s task runner to work with MSQ

2022-11-05 Thread GitBox
abhishekagarwal87 commented on code in PR #13305: URL: https://github.com/apache/druid/pull/13305#discussion_r1014605161 ## extensions-contrib/kubernetes-overlord-extensions/src/main/java/org/apache/druid/k8s/overlord/KubernetesTaskRunner.java: ## @@ -494,20 +505,28 @@ public Ru

[GitHub] [druid] AmatyaAvadhanula commented on a diff in pull request #13304: Skip ALL to finer granularity compaction

2022-11-05 Thread GitBox
AmatyaAvadhanula commented on code in PR #13304: URL: https://github.com/apache/druid/pull/13304#discussion_r1014609027 ## server/src/main/java/org/apache/druid/server/coordinator/duty/NewestSegmentFirstIterator.java: ## @@ -129,6 +131,12 @@ public class NewestSegmentFirstIterat

[GitHub] [druid] AmatyaAvadhanula commented on a diff in pull request #13304: Skip ALL to finer granularity compaction

2022-11-05 Thread GitBox
AmatyaAvadhanula commented on code in PR #13304: URL: https://github.com/apache/druid/pull/13304#discussion_r1014609027 ## server/src/main/java/org/apache/druid/server/coordinator/duty/NewestSegmentFirstIterator.java: ## @@ -129,6 +131,12 @@ public class NewestSegmentFirstIterat

[GitHub] [druid] 599166320 opened a new issue, #13317: Data imbalance

2022-11-05 Thread GitBox
599166320 opened a new issue, #13317: URL: https://github.com/apache/druid/issues/13317 ### Affected Version 24.0.0 ### Description When druid is upgraded from 0.22.0 to 24.0.0, a strange phenomenon occurs: it always allocates segments to a specific machine until the

[GitHub] [druid] AmatyaAvadhanula closed issue #13289: [Flaky ut] KafkaSupervisorTest.testResumeAllActivelyReadingTasks

2022-11-05 Thread GitBox
AmatyaAvadhanula closed issue #13289: [Flaky ut] KafkaSupervisorTest.testResumeAllActivelyReadingTasks URL: https://github.com/apache/druid/issues/13289 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [druid] AmatyaAvadhanula commented on issue #13289: [Flaky ut] KafkaSupervisorTest.testResumeAllActivelyReadingTasks

2022-11-05 Thread GitBox
AmatyaAvadhanula commented on issue #13289: URL: https://github.com/apache/druid/issues/13289#issuecomment-1304540025 Fixed by https://github.com/apache/druid/pull/13315 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use

[GitHub] [druid] AmatyaAvadhanula commented on issue #13317: Data imbalance

2022-11-05 Thread GitBox
AmatyaAvadhanula commented on issue #13317: URL: https://github.com/apache/druid/issues/13317#issuecomment-1304541693 Hi, were any runtime properties changed with the upgrade? Could you also please share the coordinator's dynamic config and cost strategy as well? -- This is an automate

[GitHub] [druid] 599166320 commented on issue #13317: Data imbalance

2022-11-05 Thread GitBox
599166320 commented on issue #13317: URL: https://github.com/apache/druid/issues/13317#issuecomment-1304545194 @AmatyaAvadhanula 1. dynamic config ``` { "millisToWaitBeforeDeleting": 90, "mergeBytesLimit": 524288000, "mergeSegmentsLimit": 1000, "maxSegmen

[GitHub] [druid] AmatyaAvadhanula commented on issue #13317: Data imbalance

2022-11-05 Thread GitBox
AmatyaAvadhanula commented on issue #13317: URL: https://github.com/apache/druid/issues/13317#issuecomment-1304548666 @599166320 1) Could you please check the metric `segment/pendingDelete` for the problematic historical to confirm if it is an issue with segment assignment or if it

[GitHub] [druid] 599166320 commented on issue #13317: Data imbalance

2022-11-05 Thread GitBox
599166320 commented on issue #13317: URL: https://github.com/apache/druid/issues/13317#issuecomment-1304551558 @AmatyaAvadhanula I think the `segment/pendingDelete` is normal. The problem may be caused by `druid.coordinator.balancer.strategy`. -- This is an automated message from the Apac

<    1   2   3   4   5   6   7   8   9   10   >