Re: [PR] MSQ: Validate that strings and string arrays are not mixed. (druid)

2024-03-13 Thread via GitHub
clintropolis commented on code in PR #15920: URL: https://github.com/apache/druid/pull/15920#discussion_r1522590886 ## docs/querying/arrays.md: ## @@ -71,19 +71,61 @@ The following shows an example `dimensionsSpec` for native ingestion of the data ### SQL-based ingestion

(druid) branch master updated (82fced571bb -> 795e342ba86)

2024-03-13 Thread gian
This is an automated email from the ASF dual-hosted git repository. gian pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/druid.git from 82fced571bb Remove deprecated UnknownSegmentIdsException (#16112) add 795e342ba86 fix sql results mixed array and

Re: [PR] Fix toString for SingleThreadSpecializable ConstantExprs (druid)

2024-03-13 Thread via GitHub
kgyrtkirk commented on code in PR #16084: URL: https://github.com/apache/druid/pull/16084#discussion_r1522708617 ## processing/src/test/java/org/apache/druid/math/expr/ConstantExprTest.java: ## @@ -19,20 +19,150 @@ package org.apache.druid.math.expr; +import

[PR] reset keySerde when closing groupers to clear out heap dictionaries (druid)

2024-03-13 Thread via GitHub
clintropolis opened a new pull request, #16114: URL: https://github.com/apache/druid/pull/16114 ### Description `ConcurrentGrouper` [kind of misuses `ThreadLocal` to hold a

Re: [PR] fix sql results mixed array and scalar values (druid)

2024-03-13 Thread via GitHub
gianm merged PR #16105: URL: https://github.com/apache/druid/pull/16105 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[PR] Add tests for concurrent replace with streaming append (druid)

2024-03-13 Thread via GitHub
AmatyaAvadhanula opened a new pull request, #16115: URL: https://github.com/apache/druid/pull/16115 Fixes #. ### Description Fixed the bug ... Renamed the class ... Added a forbidden-apis entry ...

Re: [PR] reset keySerde when closing groupers to clear out heap dictionaries (druid)

2024-03-13 Thread via GitHub
LakshSingla merged PR #16114: URL: https://github.com/apache/druid/pull/16114 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [PR] reset keySerde when closing groupers to clear out heap dictionaries (druid)

2024-03-13 Thread via GitHub
LakshSingla commented on PR #16114: URL: https://github.com/apache/druid/pull/16114#issuecomment-1993961679 Thanks for the explanation! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

(druid) branch master updated (795e342ba86 -> aa2959b2bd3)

2024-03-13 Thread lakshsingla
This is an automated email from the ASF dual-hosted git repository. lakshsingla pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/druid.git from 795e342ba86 fix sql results mixed array and scalar values (#16105) add aa2959b2bd3 reset keySerde when

Re: [PR] pacj4: add UserProfile attributes to AuthenticationResult context (druid)

2024-03-13 Thread via GitHub
abhishekagarwal87 commented on PR #16109: URL: https://github.com/apache/druid/pull/16109#issuecomment-1993984741 Can you describe the authorization flow that uses these attributes? There are also compilation failures in your code. -- This is an automated message from the Apache Git

Re: [PR] Web console: Make array ingest mode ux better (druid)

2024-03-13 Thread via GitHub
gianm commented on PR #15927: URL: https://github.com/apache/druid/pull/15927#issuecomment-1993653655 @vogievetsky -- sorry, by "that" I meant the "array ingest mode" dropdown. I see it now in the screenshot. I just updated with new docs for `arrayIngestMode` that will appear on

Re: [PR] MSQ: Validate that strings and string arrays are not mixed. (druid)

2024-03-13 Thread via GitHub
gianm commented on PR #15920: URL: https://github.com/apache/druid/pull/15920#issuecomment-1993640129 Taking this out of draft since I've added tests and docs. I have also added error message examples to the PR description. -- This is an automated message from the Apache Git Service. To

Re: [PR] reset keySerde when closing groupers to clear out heap dictionaries (druid)

2024-03-13 Thread via GitHub
abhishekagarwal87 commented on PR #16114: URL: https://github.com/apache/druid/pull/16114#issuecomment-1993672313 Nice find @clintropolis. Wouldn't JVM claim those weak references more aggressively under memory pressure? -- This is an automated message from the Apache Git Service. To

Re: [PR] reset keySerde when closing groupers to clear out heap dictionaries (druid)

2024-03-13 Thread via GitHub
clintropolis commented on PR #16114: URL: https://github.com/apache/druid/pull/16114#issuecomment-1993886651 >Can something like this be done: >1. ConcurrentGrouper passes a Closeable to the SpillingGrouper: () -> threadLocalGrouper.remove(); >2. This closer gets invoked in the

[PR] MSQ: Remove unnecessary snapshot deserialization code. (druid)

2024-03-13 Thread via GitHub
gianm opened a new pull request, #16116: URL: https://github.com/apache/druid/pull/16116 Since #13205, a special deserializer module has no longer been necessary to read key collector snapshots. This patch removes the unnecessary code. -- This is an automated message from the Apache Git

Re: [PR] add google as external storage for msq export (druid)

2024-03-13 Thread via GitHub
adarshsanjeev commented on code in PR #16051: URL: https://github.com/apache/druid/pull/16051#discussion_r1522634321 ## docs/multi-stage-query/reference.md: ## @@ -149,6 +149,39 @@ The following runtime parameters must be configured to export into an S3 destina |

Re: [PR] Fix toString for SingleThreadSpecializable ConstantExprs (druid)

2024-03-13 Thread via GitHub
clintropolis merged PR #16084: URL: https://github.com/apache/druid/pull/16084 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [PR] reset keySerde when closing groupers to clear out heap dictionaries (druid)

2024-03-13 Thread via GitHub
clintropolis commented on PR #16114: URL: https://github.com/apache/druid/pull/16114#issuecomment-1993685983 > Nice find @clintropolis. Wouldn't JVM claim those weak references more aggressively under memory pressure? So i think it often does get reclaimed or else this would probably

Re: [PR] Add tests for concurrent replace with streaming append (druid)

2024-03-13 Thread via GitHub
github-advanced-security[bot] commented on code in PR #16115: URL: https://github.com/apache/druid/pull/16115#discussion_r1522659529 ## indexing-service/src/test/java/org/apache/druid/indexing/common/task/concurrent/ActionsTestTask.java: ## @@ -78,16 +85,29 @@ ); } +

Re: [PR] reset keySerde when closing groupers to clear out heap dictionaries (druid)

2024-03-13 Thread via GitHub
LakshSingla commented on PR #16114: URL: https://github.com/apache/druid/pull/16114#issuecomment-1993865173 > so removing them from the ThreadLocalMap of the processing threads is a bit tricksy. Can something like this be done: 1. `ConcurrentGrouper` passes a `Closeable` to

(druid) branch master updated (aa2959b2bd3 -> 818cc9eedf7)

2024-03-13 Thread cwylie
This is an automated email from the ASF dual-hosted git repository. cwylie pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/druid.git from aa2959b2bd3 reset keySerde when closing groupers to clear out heap dictionaries (#16114) add 818cc9eedf7 Fix

[PR] Adding more logging for s3 RetryableS3OutputStream (druid)

2024-03-13 Thread via GitHub
cryptoe opened a new pull request, #16117: URL: https://github.com/apache/druid/pull/16117 Adding more logging for s3 RetryableS3OutputStream which would help us determine if chunk size needs to be adjusted . -- This is an automated message from the Apache Git Service. To respond to the

Re: [PR] add google as external storage for msq export (druid)

2024-03-13 Thread via GitHub
pjain1 commented on code in PR #16051: URL: https://github.com/apache/druid/pull/16051#discussion_r1523124198 ## extensions-core/google-extensions/src/main/java/org/apache/druid/storage/google/output/GoogleExportStorageProvider.java: ## @@ -0,0 +1,148 @@ +/* + * Licensed to the

Re: [PR] Adding more logging for s3 RetryableS3OutputStream (druid)

2024-03-13 Thread via GitHub
kfaraz commented on code in PR #16117: URL: https://github.com/apache/druid/pull/16117#discussion_r1523104573 ## extensions-core/s3-extensions/src/main/java/org/apache/druid/storage/s3/output/RetryableS3OutputStream.java: ## @@ -269,7 +269,12 @@ public void close() throws

Re: [PR] Adding more logging for s3 RetryableS3OutputStream (druid)

2024-03-13 Thread via GitHub
kfaraz commented on code in PR #16117: URL: https://github.com/apache/druid/pull/16117#discussion_r1523104573 ## extensions-core/s3-extensions/src/main/java/org/apache/druid/storage/s3/output/RetryableS3OutputStream.java: ## @@ -269,7 +269,12 @@ public void close() throws

Re: [PR] add google as external storage for msq export (druid)

2024-03-13 Thread via GitHub
pjain1 commented on code in PR #16051: URL: https://github.com/apache/druid/pull/16051#discussion_r1523115949 ## docs/multi-stage-query/reference.md: ## @@ -149,6 +149,39 @@ The following runtime parameters must be configured to export into an S3 destina |

Re: [PR] MSQ: Plan without implicit sorting. (druid)

2024-03-13 Thread via GitHub
gianm merged PR #16073: URL: https://github.com/apache/druid/pull/16073 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

(druid) branch master updated: MSQ: Plan without implicit sorting. (#16073)

2024-03-13 Thread gian
This is an automated email from the ASF dual-hosted git repository. gian pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/druid.git The following commit(s) were added to refs/heads/master by this push: new 910124d4deb MSQ: Plan without implicit sorting.

Re: [PR] Update Calcite*Test to use junit5 (druid)

2024-03-13 Thread via GitHub
gianm commented on PR #16106: URL: https://github.com/apache/druid/pull/16106#issuecomment-1994660293 What's the openrewrite stuff for? It looks like that is also currently associated with a failing static check. -- This is an automated message from the Apache Git Service. To respond to

[PR] Handling null operand in JSON_QUERY_ARRAY (druid)

2024-03-13 Thread via GitHub
sreemanamala opened a new pull request, #16118: URL: https://github.com/apache/druid/pull/16118 ### Description This aims to handle the null argument in JSON_QUERY_ARRAY Currently `select JSON_QUERY_ARRAY('nested_data', 'path') from 'table' where 'nested_data' is null` fails

Re: [PR] Update Calcite*Test to use junit5 (druid)

2024-03-13 Thread via GitHub
gianm commented on PR #16106: URL: https://github.com/apache/druid/pull/16106#issuecomment-1994869811 Ah, I see the `rewrite.yml` with the config. Seems neat. I see that the static failure is about too many `UnnecessaryThrows`, maybe we should remove that rule? -- This is an automated

Re: [PR] Update Calcite*Test to use junit5 (druid)

2024-03-13 Thread via GitHub
kgyrtkirk commented on PR #16106: URL: https://github.com/apache/druid/pull/16106#issuecomment-1994841386 Right now openrewrite is configured to upgrade the `Calcite*Test` tests to junit5; if we decide to move on upgrading more - we can first change its configuration - let it do the bulk

Re: [PR] MSQ: Validate that strings and string arrays are not mixed. (druid)

2024-03-13 Thread via GitHub
gianm commented on code in PR #15920: URL: https://github.com/apache/druid/pull/15920#discussion_r1523646204 ## extensions-core/multi-stage-query/src/main/java/org/apache/druid/msq/util/DimensionSchemaUtils.java: ## @@ -57,76 +57,112 @@ public static DimensionSchema

Re: [PR] MSQ: Validate that strings and string arrays are not mixed. (druid)

2024-03-13 Thread via GitHub
gianm commented on code in PR #15920: URL: https://github.com/apache/druid/pull/15920#discussion_r1523646204 ## extensions-core/multi-stage-query/src/main/java/org/apache/druid/msq/util/DimensionSchemaUtils.java: ## @@ -57,76 +57,112 @@ public static DimensionSchema

Re: [I] [Performance] will a decompressed bytebuffer cache help? (druid)

2024-03-13 Thread via GitHub
IgorBerman commented on issue #7921: URL: https://github.com/apache/druid/issues/7921#issuecomment-1994783103 @litao91 Hi can you tell what was your resolution? We see same issue (in my case using visualvm on historical process) Have you disabled compression to solve this bottleneck?

Re: [PR] Adding more logging for s3 RetryableS3OutputStream (druid)

2024-03-13 Thread via GitHub
cryptoe commented on code in PR #16117: URL: https://github.com/apache/druid/pull/16117#discussion_r1523143722 ## extensions-core/s3-extensions/src/main/java/org/apache/druid/storage/s3/output/RetryableS3OutputStream.java: ## @@ -269,7 +269,12 @@ public void close() throws

(druid) branch master updated (256160aba6b -> e9d2caccb61)

2024-03-13 Thread cwylie
This is an automated email from the ASF dual-hosted git repository. cwylie pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/druid.git from 256160aba6b MSQ: Validate that strings and string arrays are not mixed. (#15920) add e9d2caccb61 Handling null

Re: [PR] Handling null operand in JSON_QUERY_ARRAY (druid)

2024-03-13 Thread via GitHub
clintropolis merged PR #16118: URL: https://github.com/apache/druid/pull/16118 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [PR] Allow non literal rhs in MV_FILTER_ONLY and MV_FILTER_NONE (druid)

2024-03-13 Thread via GitHub
clintropolis commented on code in PR #16113: URL: https://github.com/apache/druid/pull/16113#discussion_r1524084705 ## processing/src/main/java/org/apache/druid/math/expr/ApplyFunction.java: ## @@ -476,6 +476,44 @@ private Stream filter(T[] array, LambdaExpr expr,

(druid) branch master updated: fix issues with array_contains and array_overlap with null left side arguments (#15974)

2024-03-13 Thread cwylie
This is an automated email from the ASF dual-hosted git repository. cwylie pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/druid.git The following commit(s) were added to refs/heads/master by this push: new dd9bc3749a1 fix issues with array_contains and

Re: [PR] fix issues with array_contains and array_overlap with null left side arguments (druid)

2024-03-13 Thread via GitHub
clintropolis merged PR #15974: URL: https://github.com/apache/druid/pull/15974 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

(druid) branch master updated (910124d4deb -> f7c0e425a93)

2024-03-13 Thread gian
This is an automated email from the ASF dual-hosted git repository. gian pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/druid.git from 910124d4deb MSQ: Plan without implicit sorting. (#16073) add f7c0e425a93 fix URI wording (#16111) No new revisions

Re: [PR] Web console: fix URI wording for lookups (druid)

2024-03-13 Thread via GitHub
gianm merged PR #16111: URL: https://github.com/apache/druid/pull/16111 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [PR] docs: clarify description of uri/uriprefix (druid)

2024-03-13 Thread via GitHub
techdocsmith merged PR #16110: URL: https://github.com/apache/druid/pull/16110 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

(druid) branch master updated: docs: clarify description of uri/uriprefix (#16110)

2024-03-13 Thread techdocsmith
This is an automated email from the ASF dual-hosted git repository. techdocsmith pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/druid.git The following commit(s) were added to refs/heads/master by this push: new 03c191f7017 docs: clarify description of

Re: [PR] add google as external storage for msq export (druid)

2024-03-13 Thread via GitHub
317brian commented on code in PR #16051: URL: https://github.com/apache/druid/pull/16051#discussion_r1523793377 ## docs/multi-stage-query/reference.md: ## @@ -149,6 +149,39 @@ The following runtime parameters must be configured to export into an S3 destina |

Re: [PR] Web console: Make array ingest mode ux better (druid)

2024-03-13 Thread via GitHub
vogievetsky commented on PR #15927: URL: https://github.com/apache/druid/pull/15927#issuecomment-1995458752 Thank you for the feedback, updated: ![image](https://github.com/apache/druid/assets/177816/c80136dc-5a10-49bb-aa55-4ca489ac7114) -- This is an automated message from

Re: [PR] MSQ: Validate that strings and string arrays are not mixed. (druid)

2024-03-13 Thread via GitHub
clintropolis commented on code in PR #15920: URL: https://github.com/apache/druid/pull/15920#discussion_r1523815715 ## extensions-core/multi-stage-query/src/main/java/org/apache/druid/msq/util/DimensionSchemaUtils.java: ## @@ -57,76 +57,112 @@ public static DimensionSchema

Re: [PR] INSERT/REPLACE dimension target column types are validated against source input expressions (druid)

2024-03-13 Thread via GitHub
zachjsh commented on code in PR #15962: URL: https://github.com/apache/druid/pull/15962#discussion_r1523820359 ## extensions-core/druid-catalog/src/test/java/org/apache/druid/catalog/sql/CatalogIngestionTest.java: ## @@ -171,4 +196,313 @@ public void

Re: [PR] Web console: Make array ingest mode ux better (druid)

2024-03-13 Thread via GitHub
vogievetsky merged PR #15927: URL: https://github.com/apache/druid/pull/15927 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

(druid) branch master updated (03c191f7017 -> ccae19a5468)

2024-03-13 Thread vogievetsky
This is an automated email from the ASF dual-hosted git repository. vogievetsky pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/druid.git from 03c191f7017 docs: clarify description of uri/uriprefix (#16110) add ccae19a5468 Web console: Make array

Re: [PR] add google as external storage for msq export (druid)

2024-03-13 Thread via GitHub
317brian commented on PR #16051: URL: https://github.com/apache/druid/pull/16051#issuecomment-1995447062 Some minor copyedit nits. The only absolutely must fix one is the incorrect cloud provider being listed in an intro sentence -- This is an automated message from the Apache Git

Re: [PR] add google as external storage for msq export (druid)

2024-03-13 Thread via GitHub
317brian commented on code in PR #16051: URL: https://github.com/apache/druid/pull/16051#discussion_r1523793377 ## docs/multi-stage-query/reference.md: ## @@ -149,6 +149,39 @@ The following runtime parameters must be configured to export into an S3 destina |

Re: [PR] Web console: Make array ingest mode ux better (druid)

2024-03-13 Thread via GitHub
vogievetsky commented on PR #15927: URL: https://github.com/apache/druid/pull/15927#issuecomment-1995490568 Thanks for the feedback I was going back and forth on that second point. I guess I manifested [my inner wishes that Druid did have a `mv-string`

Re: [PR] INSERT/REPLACE dimension target column types are validated against source input expressions (druid)

2024-03-13 Thread via GitHub
zachjsh commented on code in PR #15962: URL: https://github.com/apache/druid/pull/15962#discussion_r1523822702 ## sql/src/main/java/org/apache/druid/sql/calcite/planner/DruidSqlValidator.java: ## @@ -449,6 +480,94 @@ private RelDataType validateTargetType( return

Re: [PR] INSERT/REPLACE dimension target column types are validated against source input expressions (druid)

2024-03-13 Thread via GitHub
zachjsh commented on code in PR #15962: URL: https://github.com/apache/druid/pull/15962#discussion_r1523822415 ## sql/src/main/java/org/apache/druid/sql/calcite/planner/DruidSqlValidator.java: ## @@ -449,6 +480,94 @@ private RelDataType validateTargetType( return

Re: [I] The resources are sufficient, but the following errors are still reported (druid)

2024-03-13 Thread via GitHub
github-actions[bot] closed issue #14218: The resources are sufficient, but the following errors are still reported URL: https://github.com/apache/druid/issues/14218 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

Re: [I] opentelemetry-emitter fails on startup (all druid services) - Druid Version 25.0.0 (druid)

2024-03-13 Thread via GitHub
github-actions[bot] closed issue #13924: opentelemetry-emitter fails on startup (all druid services) - Druid Version 25.0.0 URL: https://github.com/apache/druid/issues/13924 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

Re: [I] Group By query on Theta sketches breaks in version 25. (druid)

2024-03-13 Thread via GitHub
github-actions[bot] commented on issue #14326: URL: https://github.com/apache/druid/issues/14326#issuecomment-1996170431 This issue has been marked as stale due to 280 days of inactivity. It will be closed in 4 weeks if no further activity occurs. If this issue is still relevant,

Re: [PR] [WIP] Migrate Druid HttpClient from Netty 3 to 4 (druid)

2024-03-13 Thread via GitHub
github-actions[bot] closed pull request #14479: [WIP] Migrate Druid HttpClient from Netty 3 to 4 URL: https://github.com/apache/druid/pull/14479 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

Re: [I] opentelemetry-emitter fails on startup (all druid services) - Druid Version 25.0.0 (druid)

2024-03-13 Thread via GitHub
github-actions[bot] commented on issue #13924: URL: https://github.com/apache/druid/issues/13924#issuecomment-1996170393 This issue has been closed due to lack of activity. If you think that is incorrect, or the issue requires additional review, you can revive the issue at any time.

Re: [I] The resources are sufficient, but the following errors are still reported (druid)

2024-03-13 Thread via GitHub
github-actions[bot] commented on issue #14218: URL: https://github.com/apache/druid/issues/14218#issuecomment-1996170412 This issue has been closed due to lack of activity. If you think that is incorrect, or the issue requires additional review, you can revive the issue at any time.

(druid) branch master updated: MSQ: Validate that strings and string arrays are not mixed. (#15920)

2024-03-13 Thread gian
This is an automated email from the ASF dual-hosted git repository. gian pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/druid.git The following commit(s) were added to refs/heads/master by this push: new 256160aba6b MSQ: Validate that strings and string

Re: [PR] MSQ: Validate that strings and string arrays are not mixed. (druid)

2024-03-13 Thread via GitHub
gianm merged PR #15920: URL: https://github.com/apache/druid/pull/15920 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [PR] Add tests for concurrent replace with streaming append (druid)

2024-03-13 Thread via GitHub
kfaraz commented on PR #16115: URL: https://github.com/apache/druid/pull/16115#issuecomment-1996566151 If the new test class is mostly a copy of the original one and uses all the same methods, then we should just update the original test class to run in a parameterized manner. 1st