Re: [PR] A Replacing task must read segments created before it acquired its lock (druid)

AmatyaAvadhanula commented on PR #15085: URL: https://github.com/apache/druid/pull/15085#issuecomment-1751905942 > since the new action deals with replace locks, it must be fired using the SurrogateTaskActionClient. Thanks, @kfaraz. I'll make the change. -- This is an

Re: [PR] Optimize used segment fetching in Kill tasks (druid)

kfaraz commented on code in PR #15107: URL: https://github.com/apache/druid/pull/15107#discussion_r1349611592 ## indexing-service/src/main/java/org/apache/druid/indexing/common/task/KillUnusedSegmentsTask.java: ## @@ -225,7 +225,14 @@ public TaskStatus runTask(TaskToolbox

Re: [PR] Add support for concurrent batch Append and Replace (druid)

kfaraz commented on PR #14407: URL: https://github.com/apache/druid/pull/14407#issuecomment-1751898094 @abhishekagarwal87 , I have replied to your comments. The changes will be included in #15097 . -- This is an automated message from the Apache Git Service. To respond to the message,

Re: [PR] A Replacing task must read segments created before it acquired its lock (druid)

kfaraz commented on PR #15085: URL: https://github.com/apache/druid/pull/15085#issuecomment-1751897995 @AmatyaAvadhanula , since the new action deals with replace locks, it must be fired using the `SurrogateTaskActionClient`. -- This is an automated message from the Apache Git Service.

Re: [PR] Add support for concurrent batch Append and Replace (druid)

kfaraz commented on code in PR #14407: URL: https://github.com/apache/druid/pull/14407#discussion_r1349610978 ## server/src/main/java/org/apache/druid/metadata/IndexerSQLMetadataStorageCoordinator.java: ## @@ -977,6 +1045,175 @@ private void insertPendingSegmentIntoMetastore(

Re: [PR] Add support for concurrent batch Append and Replace (druid)

kfaraz commented on code in PR #14407: URL: https://github.com/apache/druid/pull/14407#discussion_r1349610904 ## indexing-service/src/main/java/org/apache/druid/indexing/common/task/AbstractBatchIndexTask.java: ## @@ -443,6 +466,42 @@ protected boolean

Re: [PR] Add support for concurrent batch Append and Replace (druid)

kfaraz commented on code in PR #14407: URL: https://github.com/apache/druid/pull/14407#discussion_r1349610732 ## indexing-service/src/main/java/org/apache/druid/indexing/common/actions/TaskLocks.java: ## @@ -110,6 +127,63 @@ public static boolean isLockCoversSegments( );

Re: [PR] Add support for concurrent batch Append and Replace (druid)

kfaraz commented on code in PR #14407: URL: https://github.com/apache/druid/pull/14407#discussion_r1349610299 ## indexing-service/src/main/java/org/apache/druid/indexing/common/task/KillUnusedSegmentsTask.java: ## @@ -218,7 +221,22 @@ public TaskStatus runTask(TaskToolbox

Re: [PR] Add support for concurrent batch Append and Replace (druid)

kfaraz commented on code in PR #14407: URL: https://github.com/apache/druid/pull/14407#discussion_r1349610092 ## indexing-service/src/main/java/org/apache/druid/indexing/overlord/TaskLockbox.java: ## @@ -889,6 +891,71 @@ public List findLocksForTask(final Task task) } }

Re: [PR] Add support for concurrent batch Append and Replace (druid)

kfaraz commented on code in PR #14407: URL: https://github.com/apache/druid/pull/14407#discussion_r1349610004 ## server/src/main/java/org/apache/druid/metadata/IndexerSQLMetadataStorageCoordinator.java: ## @@ -977,6 +1045,175 @@ private void insertPendingSegmentIntoMetastore(

Re: [PR] Add support for concurrent batch Append and Replace (druid)

kfaraz commented on code in PR #14407: URL: https://github.com/apache/druid/pull/14407#discussion_r1349609904 ## server/src/main/java/org/apache/druid/metadata/IndexerSQLMetadataStorageCoordinator.java: ## @@ -977,6 +1045,175 @@ private void insertPendingSegmentIntoMetastore(

Re: [PR] Add support for concurrent batch Append and Replace (druid)

kfaraz commented on code in PR #14407: URL: https://github.com/apache/druid/pull/14407#discussion_r1349609828 ## server/src/main/java/org/apache/druid/metadata/IndexerSQLMetadataStorageCoordinator.java: ## @@ -977,6 +1045,175 @@ private void insertPendingSegmentIntoMetastore(

Re: [PR] Add support for concurrent batch Append and Replace (druid)

kfaraz commented on code in PR #14407: URL: https://github.com/apache/druid/pull/14407#discussion_r1349609131 ## indexing-service/src/main/java/org/apache/druid/indexing/common/actions/SegmentTransactionalReplaceAction.java: ## @@ -0,0 +1,147 @@ +/* + * Licensed to the Apache

Re: [I] Idle timeout expired:300000/300000 ms (druid)

github-actions[bot] commented on issue #10903: URL: https://github.com/apache/druid/issues/10903#issuecomment-1751873392 This issue has been marked as stale due to 280 days of inactivity. It will be closed in 4 weeks if no further activity occurs. If this issue is still relevant,

Re: [I] PendingTaskBasedWorkerProvisioningStrategy does not scale nodes up if there are no workers (druid)

github-actions[bot] commented on issue #10918: URL: https://github.com/apache/druid/issues/10918#issuecomment-1751873411 This issue has been marked as stale due to 280 days of inactivity. It will be closed in 4 weeks if no further activity occurs. If this issue is still relevant,

Re: [PR] Spark Direct Readers and Writers for Druid. (druid)

github-actions[bot] commented on PR #10920: URL: https://github.com/apache/druid/pull/10920#issuecomment-1751873419 This pull request has been marked as stale due to 60 days of inactivity. It will be closed in 4 weeks if no further activity occurs. If you think that's incorrect or this

Re: [I] Segments with ALL granularity can be killed by coordinator despite having "end" > "now - durationToRetain" (druid)

github-actions[bot] commented on issue #10875: URL: https://github.com/apache/druid/issues/10875#issuecomment-1751873331 This issue has been marked as stale due to 280 days of inactivity. It will be closed in 4 weeks if no further activity occurs. If this issue is still relevant,

Re: [I] Using columns in Flatten Spec for Spatial indexing leads to null value. (druid)

github-actions[bot] commented on issue #10890: URL: https://github.com/apache/druid/issues/10890#issuecomment-1751873342 This issue has been marked as stale due to 280 days of inactivity. It will be closed in 4 weeks if no further activity occurs. If this issue is still relevant,

Re: [I] Add possibility to use a different S3 endpoint for ingestion (druid)

github-actions[bot] commented on issue #10893: URL: https://github.com/apache/druid/issues/10893#issuecomment-1751873370 This issue has been marked as stale due to 280 days of inactivity. It will be closed in 4 weeks if no further activity occurs. If this issue is still relevant,

Re: [I] Documentation for SQL ARRAY to use [] instead of () (druid)

github-actions[bot] commented on issue #10409: URL: https://github.com/apache/druid/issues/10409#issuecomment-1751873141 This issue has been closed due to lack of activity. If you think that is incorrect, or the issue requires additional review, you can revive the issue at any time.

Re: [PR] fix 'is not null' is invalid for array-type in druid-sql (druid)

github-actions[bot] closed pull request #10921: fix 'is not null' is invalid for array-type in druid-sql URL: https://github.com/apache/druid/pull/10921 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

Re: [PR] fix 'is not null' is invalid for array-type in druid-sql (druid)

github-actions[bot] commented on PR #10921: URL: https://github.com/apache/druid/pull/10921#issuecomment-1751873428 This pull request/issue has been closed due to lack of activity. If you think that is incorrect, or the pull request requires review, you can revive the PR at any time.

Re: [I] Ingestion Data Failing (druid)

github-actions[bot] commented on issue #10873: URL: https://github.com/apache/druid/issues/10873#issuecomment-1751873323 This issue has been marked as stale due to 280 days of inactivity. It will be closed in 4 weeks if no further activity occurs. If this issue is still relevant,

Re: [I] Update docs related to update query granularity changes (druid)

github-actions[bot] commented on issue #10897: URL: https://github.com/apache/druid/issues/10897#issuecomment-1751873376 This issue has been marked as stale due to 280 days of inactivity. It will be closed in 4 weeks if no further activity occurs. If this issue is still relevant,

Re: [I] Index Kafka job throws Interrupted Exception during shutdown. (druid)

github-actions[bot] commented on issue #10902: URL: https://github.com/apache/druid/issues/10902#issuecomment-1751873383 This issue has been marked as stale due to 280 days of inactivity. It will be closed in 4 weeks if no further activity occurs. If this issue is still relevant,

Re: [I] coordinator-overlord.log so large (druid)

github-actions[bot] commented on issue #10879: URL: https://github.com/apache/druid/issues/10879#issuecomment-1751873335 This issue has been marked as stale due to 280 days of inactivity. It will be closed in 4 weeks if no further activity occurs. If this issue is still relevant,

Re: [I] How can we delete an datasource(unused)? (druid)

github-actions[bot] commented on issue #10916: URL: https://github.com/apache/druid/issues/10916#issuecomment-1751873399 This issue has been marked as stale due to 280 days of inactivity. It will be closed in 4 weeks if no further activity occurs. If this issue is still relevant,

Re: [I] Druid is on yarn ? (druid)

github-actions[bot] commented on issue #10415: URL: https://github.com/apache/druid/issues/10415#issuecomment-1751873155 This issue has been closed due to lack of activity. If you think that is incorrect, or the issue requires additional review, you can revive the issue at any time.

Re: [I] StatsD emmiter just emit native queries count in druid/query/count metric and does not consider sql queries (druid)

github-actions[bot] commented on issue #10411: URL: https://github.com/apache/druid/issues/10411#issuecomment-1751873147 This issue has been closed due to lack of activity. If you think that is incorrect, or the issue requires additional review, you can revive the issue at any time.

Re: [I] Druid is on yarn ? (druid)

github-actions[bot] closed issue #10415: Druid is on yarn ? URL: https://github.com/apache/druid/issues/10415 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe,

Re: [I] StatsD emmiter just emit native queries count in druid/query/count metric and does not consider sql queries (druid)

github-actions[bot] closed issue #10411: StatsD emmiter just emit native queries count in druid/query/count metric and does not consider sql queries URL: https://github.com/apache/druid/issues/10411 -- This is an automated message from the Apache Git Service. To respond to the message,

Re: [I] Documentation for SQL ARRAY to use [] instead of () (druid)

github-actions[bot] closed issue #10409: Documentation for SQL ARRAY to use [] instead of () URL: https://github.com/apache/druid/issues/10409 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

Re: [PR] Add ability for MSQ tasks to query realtime tasks (druid)

adarshsanjeev commented on code in PR #15024: URL: https://github.com/apache/druid/pull/15024#discussion_r1349549352 ## extensions-core/multi-stage-query/src/main/java/org/apache/druid/msq/exec/LoadedSegmentDataProvider.java: ## @@ -0,0 +1,234 @@ +/* + * Licensed to the Apache

[druid-website-src] branch dependabot/npm_and_yarn/postcss-8.4.31 created (now f8f532e6)

This is an automated email from the ASF dual-hosted git repository. github-bot pushed a change to branch dependabot/npm_and_yarn/postcss-8.4.31 in repository https://gitbox.apache.org/repos/asf/druid-website-src.git at f8f532e6 Bump postcss from 8.4.21 to 8.4.31 No new revisions were

[PR] Bump postcss from 8.4.21 to 8.4.31 [druid-website-src]

dependabot[bot] opened a new pull request, #421: URL: https://github.com/apache/druid-website-src/pull/421 Bumps [postcss](https://github.com/postcss/postcss) from 8.4.21 to 8.4.31. Release notes Sourced from https://github.com/postcss/postcss/releases;>postcss's releases.

[druid] branch master updated: Change type of AVG aggregates to double (#15089)

This is an automated email from the ASF dual-hosted git repository. abhishek pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/druid.git The following commit(s) were added to refs/heads/master by this push: new 7b869fd37a9 Change type of AVG aggregates to

Re: [PR] Change type of AVG aggregates to double (druid)

abhishekagarwal87 merged PR #15089: URL: https://github.com/apache/druid/pull/15089 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [PR] Add support for concurrent batch Append and Replace (druid)

abhishekagarwal87 commented on code in PR #14407: URL: https://github.com/apache/druid/pull/14407#discussion_r1349505977 ## server/src/main/java/org/apache/druid/metadata/IndexerSQLMetadataStorageCoordinator.java: ## @@ -977,6 +1045,175 @@ private void

[PR] Optimize used segment fetching in Kill tasks (druid)

AmatyaAvadhanula opened a new pull request, #15107: URL: https://github.com/apache/druid/pull/15107 https://github.com/apache/druid/pull/14407 - introduced a change in the behaviour of kill tasks to fetch used specs to prevent used load specs from being killed. However there could

Re: [PR] Add ability for MSQ tasks to query realtime tasks (druid)

adarshsanjeev commented on code in PR #15024: URL: https://github.com/apache/druid/pull/15024#discussion_r1349498657 ## extensions-core/multi-stage-query/src/main/java/org/apache/druid/msq/exec/ControllerImpl.java: ## @@ -1163,14 +1167,67 @@ private QueryKit

Re: [PR] Add ability for MSQ tasks to query realtime tasks (druid)

adarshsanjeev commented on code in PR #15024: URL: https://github.com/apache/druid/pull/15024#discussion_r1349498398 ## server/src/main/java/org/apache/druid/discovery/DataServerResponseHandler.java: ## @@ -0,0 +1,111 @@ +/* + * Licensed to the Apache Software Foundation (ASF)

Re: [PR] Add ability for MSQ tasks to query realtime tasks (druid)

adarshsanjeev commented on code in PR #15024: URL: https://github.com/apache/druid/pull/15024#discussion_r1349498328 ## extensions-core/multi-stage-query/src/main/java/org/apache/druid/msq/exec/LoadedSegmentDataProvider.java: ## @@ -0,0 +1,234 @@ +/* + * Licensed to the Apache

Re: [PR] Add ability for MSQ tasks to query realtime tasks (druid)

adarshsanjeev commented on code in PR #15024: URL: https://github.com/apache/druid/pull/15024#discussion_r1349498003 ## extensions-core/multi-stage-query/src/main/java/org/apache/druid/msq/input/table/SegmentsInputSliceReader.java: ## @@ -94,6 +98,7 @@ private Iterator

Re: [PR] Add ability for MSQ tasks to query realtime tasks (druid)

adarshsanjeev commented on code in PR #15024: URL: https://github.com/apache/druid/pull/15024#discussion_r1349498072 ## extensions-core/multi-stage-query/src/main/java/org/apache/druid/msq/exec/SegmentSource.java: ## @@ -0,0 +1,49 @@ +/* + * Licensed to the Apache Software

Re: [PR] Add ability for MSQ tasks to query realtime tasks (druid)

adarshsanjeev commented on code in PR #15024: URL: https://github.com/apache/druid/pull/15024#discussion_r1349497879 ## server/src/main/java/org/apache/druid/server/coordination/DruidServerMetadata.java: ## @@ -42,7 +43,7 @@ public class DruidServerMetadata @JsonCreator

Re: [PR] Add ability for MSQ tasks to query realtime tasks (druid)

adarshsanjeev commented on code in PR #15024: URL: https://github.com/apache/druid/pull/15024#discussion_r1349497838 ## extensions-core/multi-stage-query/src/main/java/org/apache/druid/msq/exec/LoadedSegmentDataProvider.java: ## @@ -0,0 +1,234 @@ +/* + * Licensed to the Apache

Re: [PR] Add support for concurrent batch Append and Replace (druid)

abhishekagarwal87 commented on code in PR #14407: URL: https://github.com/apache/druid/pull/14407#discussion_r1349492533 ## indexing-service/src/main/java/org/apache/druid/indexing/common/actions/SegmentTransactionalReplaceAction.java: ## @@ -0,0 +1,147 @@ +/* + * Licensed to

Re: [PR] Add ability for MSQ tasks to query realtime tasks (druid)

adarshsanjeev commented on code in PR #15024: URL: https://github.com/apache/druid/pull/15024#discussion_r1349497802 ## docs/multi-stage-query/reference.md: ## @@ -247,6 +247,7 @@ The following table lists the context parameters for the MSQ task engine: | `faultTolerance` |

Re: [PR] Add ability for MSQ tasks to query realtime tasks (druid)

adarshsanjeev commented on code in PR #15024: URL: https://github.com/apache/druid/pull/15024#discussion_r1349497645 ## extensions-core/multi-stage-query/src/main/java/org/apache/druid/msq/input/table/RichSegmentDescriptor.java: ## @@ -97,23 +122,21 @@ public boolean

Re: [PR] Add ability for MSQ tasks to query realtime tasks (druid)

adarshsanjeev commented on code in PR #15024: URL: https://github.com/apache/druid/pull/15024#discussion_r1349497625 ## extensions-core/multi-stage-query/src/main/java/org/apache/druid/msq/input/table/SegmentWithDescriptor.java: ## @@ -65,6 +82,18 @@ public ResourceHolder