[GitHub] [druid] clintropolis opened a new pull request, #13698: merge druid-core, extendedset, and druid-hll into druid-processing to simplify everything

clintropolis opened a new pull request, #13698: URL: https://github.com/apache/druid/pull/13698 ### Description This PR picks up where #6443 left off and merges `druid-core`, `extendedset`, and `druid-hll` into `druid-processing`. My hope is that this is minimally disruptive to al

[GitHub] [druid] rohangarg commented on a diff in pull request #13368: Allow using composed storage for SuperSorter intermediate data

rohangarg commented on code in PR #13368: URL: https://github.com/apache/druid/pull/13368#discussion_r1082340975 ## processing/src/main/java/org/apache/druid/frame/file/FrameFile.java: ## @@ -246,7 +246,8 @@ static FrameFile open( log.warn("Could not delete frame fi

[GitHub] [druid] github-code-scanning[bot] commented on a diff in pull request #13696: Query tests to new IT framework

github-code-scanning[bot] commented on code in PR #13696: URL: https://github.com/apache/druid/pull/13696#discussion_r1082214178 ## integration-tests-ex/cases/src/test/java/org/apache/druid/testsEx/query/ITJdbcQueryTest.java: ## @@ -0,0 +1,232 @@ +/* + * Licensed to the Apache S

[GitHub] [druid] tejaswini-imply closed pull request #13681: Migrate ITs from Travis to GHA

tejaswini-imply closed pull request #13681: Migrate ITs from Travis to GHA URL: https://github.com/apache/druid/pull/13681 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To un

[GitHub] [druid] cryptoe commented on a diff in pull request #13368: Allow using composed storage for SuperSorter intermediate data

cryptoe commented on code in PR #13368: URL: https://github.com/apache/druid/pull/13368#discussion_r1082148672 ## processing/src/main/java/org/apache/druid/frame/file/FrameFile.java: ## @@ -246,7 +246,8 @@ static FrameFile open( log.warn("Could not delete frame file

[GitHub] [druid] cryptoe commented on a diff in pull request #13368: Allow using composed storage for SuperSorter intermediate data

cryptoe commented on code in PR #13368: URL: https://github.com/apache/druid/pull/13368#discussion_r1082148672 ## processing/src/main/java/org/apache/druid/frame/file/FrameFile.java: ## @@ -246,7 +246,8 @@ static FrameFile open( log.warn("Could not delete frame file

[GitHub] [druid] somu-imply commented on a diff in pull request #13576: SQL version of unnest native druid function

somu-imply commented on code in PR #13576: URL: https://github.com/apache/druid/pull/13576#discussion_r1081646179 ## sql/src/main/java/org/apache/druid/sql/calcite/rel/DruidCorrelateUnnestRel.java: ## @@ -0,0 +1,300 @@ +/* + * Licensed to the Apache Software Foundation (ASF) und

[GitHub] [druid] somu-imply commented on a diff in pull request #13576: SQL version of unnest native druid function

somu-imply commented on code in PR #13576: URL: https://github.com/apache/druid/pull/13576#discussion_r1081630567 ## sql/src/main/java/org/apache/druid/sql/calcite/expression/Expressions.java: ## @@ -214,12 +217,43 @@ public static DruidExpression toDruidExpressionWithPostAggOp

[GitHub] [druid] somu-imply commented on a diff in pull request #13576: SQL version of unnest native druid function

somu-imply commented on code in PR #13576: URL: https://github.com/apache/druid/pull/13576#discussion_r1081628635 ## sql/src/main/java/org/apache/druid/sql/calcite/expression/Expressions.java: ## @@ -214,12 +217,43 @@ public static DruidExpression toDruidExpressionWithPostAggOp

[GitHub] [druid] somu-imply commented on a diff in pull request #13576: SQL version of unnest native druid function

somu-imply commented on code in PR #13576: URL: https://github.com/apache/druid/pull/13576#discussion_r1081627237 ## sql/src/main/java/org/apache/druid/sql/calcite/expression/Expressions.java: ## @@ -214,12 +217,43 @@ public static DruidExpression toDruidExpressionWithPostAggOp

[GitHub] [druid] somu-imply commented on a diff in pull request #13576: SQL version of unnest native druid function

somu-imply commented on code in PR #13576: URL: https://github.com/apache/druid/pull/13576#discussion_r1081617930 ## .idea/misc.xml: ## @@ -84,7 +87,10 @@ http://maven.apache.org/ASSEMBLY/2.0.0"; location="$PROJECT_DIR$/.idea/xml-schemas/assembly-2.0.0.xsd" /> http://

[GitHub] [druid] somu-imply commented on a diff in pull request #13576: SQL version of unnest native druid function

somu-imply commented on code in PR #13576: URL: https://github.com/apache/druid/pull/13576#discussion_r1081616136 ## sql/src/main/java/org/apache/druid/sql/calcite/expression/Expressions.java: ## @@ -214,12 +217,43 @@ public static DruidExpression toDruidExpressionWithPostAggOp

[GitHub] [druid] somu-imply commented on a diff in pull request #13576: SQL version of unnest native druid function

somu-imply commented on code in PR #13576: URL: https://github.com/apache/druid/pull/13576#discussion_r1081615896 ## processing/src/test/java/org/apache/druid/segment/UnnestColumnValueSelectorCursorTest.java: ## @@ -592,7 +590,25 @@ public void test_list_unnest_cursors_dimSelect

[GitHub] [druid] somu-imply commented on a diff in pull request #13576: SQL version of unnest native druid function

somu-imply commented on code in PR #13576: URL: https://github.com/apache/druid/pull/13576#discussion_r1081615401 ## processing/src/main/java/org/apache/druid/segment/UnnestDimensionCursor.java: ## @@ -409,7 +412,12 @@ public int size() @Override public int get(int idx

[GitHub] [druid] somu-imply commented on a diff in pull request #13576: SQL version of unnest native druid function

somu-imply commented on code in PR #13576: URL: https://github.com/apache/druid/pull/13576#discussion_r1081614665 ## processing/src/main/java/org/apache/druid/segment/UnnestDimensionCursor.java: ## @@ -181,7 +181,10 @@ public void inspectRuntimeShape(RuntimeShapeInspector inspe

[GitHub] [druid] somu-imply commented on a diff in pull request #13576: SQL version of unnest native druid function

somu-imply commented on code in PR #13576: URL: https://github.com/apache/druid/pull/13576#discussion_r1081613908 ## processing/src/main/java/org/apache/druid/query/InlineDataSource.java: ## @@ -125,7 +125,7 @@ private static boolean rowsEqual(final Iterable rowsA, final Iterab

[GitHub] [druid] paul-rogers merged pull request #13625: Port Calcite's tests to run with MSQ

paul-rogers merged PR #13625: URL: https://github.com/apache/druid/pull/13625 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@druid.

[GitHub] [druid] rohangarg opened a new pull request, #13697: Add configuration for max S3 connections

rohangarg opened a new pull request, #13697: URL: https://github.com/apache/druid/pull/13697 For large MSQ jobs with durable storage, the default number of maximum connections (50 as of now) is sometimes a bottleneck leading to job failures. This provides a configuration to bump the maximum

[GitHub] [druid] rohangarg commented on a diff in pull request #13368: Allow using composed storage for SuperSorter intermediate data

rohangarg commented on code in PR #13368: URL: https://github.com/apache/druid/pull/13368#discussion_r1081147729 ## extensions-core/multi-stage-query/src/main/java/org/apache/druid/msq/exec/ControllerImpl.java: ## @@ -562,13 +563,30 @@ private QueryDefinition initializeQueryDefA

[GitHub] [druid] cryptoe commented on a diff in pull request #13368: Allow using composed storage for SuperSorter intermediate data

cryptoe commented on code in PR #13368: URL: https://github.com/apache/druid/pull/13368#discussion_r1081143362 ## extensions-core/multi-stage-query/src/main/java/org/apache/druid/msq/exec/ControllerImpl.java: ## @@ -562,13 +563,30 @@ private QueryDefinition initializeQueryDefAnd

[GitHub] [druid] cryptoe commented on a diff in pull request #13368: Allow using composed storage for SuperSorter intermediate data

cryptoe commented on code in PR #13368: URL: https://github.com/apache/druid/pull/13368#discussion_r1081140648 ## processing/src/main/java/org/apache/druid/frame/processor/DurableStorageOutputChannelFactory.java: ## @@ -0,0 +1,285 @@ +/* + * Licensed to the Apache Software Found

[GitHub] [druid] rohangarg commented on a diff in pull request #13368: Allow using composed storage for SuperSorter intermediate data

rohangarg commented on code in PR #13368: URL: https://github.com/apache/druid/pull/13368#discussion_r1081137315 ## .travis.yml: ## @@ -200,7 +200,7 @@ jobs: ${MAVEN_SKIP} -Dremoteresources.skip=true -Ddruid.generic.useDefaultValueForNull=${DRUID_USE_DEFAULT_VALUE_FO

[GitHub] [druid] rohangarg commented on a diff in pull request #13368: Allow using composed storage for SuperSorter intermediate data

rohangarg commented on code in PR #13368: URL: https://github.com/apache/druid/pull/13368#discussion_r1081136510 ## extensions-core/s3-extensions/src/main/java/org/apache/druid/storage/s3/output/S3StorageConnector.java: ## @@ -89,6 +90,43 @@ public InputStream open(GetObjectRequ

[GitHub] [druid] rohangarg commented on a diff in pull request #13368: Allow using composed storage for SuperSorter intermediate data

rohangarg commented on code in PR #13368: URL: https://github.com/apache/druid/pull/13368#discussion_r1081135950 ## processing/src/main/java/org/apache/druid/frame/processor/OutputChannelFactory.java: ## @@ -31,6 +31,11 @@ */ OutputChannel openChannel(int partitionNumber)

[GitHub] [druid] rohangarg commented on a diff in pull request #13368: Allow using composed storage for SuperSorter intermediate data

rohangarg commented on code in PR #13368: URL: https://github.com/apache/druid/pull/13368#discussion_r1081130623 ## processing/src/main/java/org/apache/druid/frame/processor/PartitionedOutputChannel.java: ## @@ -0,0 +1,117 @@ +/* + * Licensed to the Apache Software Foundation (A

[GitHub] [druid] rohangarg commented on a diff in pull request #13368: Allow using composed storage for SuperSorter intermediate data

rohangarg commented on code in PR #13368: URL: https://github.com/apache/druid/pull/13368#discussion_r1081130351 ## extensions-core/multi-stage-query/src/main/java/org/apache/druid/msq/indexing/CountingOutputChannelFactory.java: ## @@ -50,7 +51,22 @@ public OutputChannel openCha

[GitHub] [druid] rohangarg commented on a diff in pull request #13368: Allow using composed storage for SuperSorter intermediate data

rohangarg commented on code in PR #13368: URL: https://github.com/apache/druid/pull/13368#discussion_r1081128967 ## processing/src/main/java/org/apache/druid/frame/file/FrameFileFooter.java: ## @@ -0,0 +1,126 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one

[GitHub] [druid] rohangarg commented on a diff in pull request #13368: Allow using composed storage for SuperSorter intermediate data

rohangarg commented on code in PR #13368: URL: https://github.com/apache/druid/pull/13368#discussion_r1081128669 ## processing/src/main/java/org/apache/druid/frame/processor/DurableStorageOutputChannelFactory.java: ## @@ -0,0 +1,285 @@ +/* + * Licensed to the Apache Software Fou

[GitHub] [druid] rohangarg commented on a diff in pull request #13368: Allow using composed storage for SuperSorter intermediate data

rohangarg commented on code in PR #13368: URL: https://github.com/apache/druid/pull/13368#discussion_r1081128163 ## extensions-core/multi-stage-query/src/main/java/org/apache/druid/msq/exec/ControllerImpl.java: ## @@ -562,13 +563,30 @@ private QueryDefinition initializeQueryDefA

[GitHub] [druid] rohangarg commented on a diff in pull request #13368: Allow using composed storage for SuperSorter intermediate data

rohangarg commented on code in PR #13368: URL: https://github.com/apache/druid/pull/13368#discussion_r1081125913 ## extensions-core/multi-stage-query/src/main/java/org/apache/druid/msq/util/MultiStageQueryContext.java: ## @@ -88,14 +94,30 @@ public static String getMSQMode(final

[GitHub] [druid] rohangarg commented on a diff in pull request #13368: Allow using composed storage for SuperSorter intermediate data

rohangarg commented on code in PR #13368: URL: https://github.com/apache/druid/pull/13368#discussion_r1081125379 ## processing/src/main/java/org/apache/druid/frame/channel/ByteTracker.java: ## @@ -0,0 +1,57 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one +

[GitHub] [druid] rohangarg commented on a diff in pull request #13368: Allow using composed storage for SuperSorter intermediate data

rohangarg commented on code in PR #13368: URL: https://github.com/apache/druid/pull/13368#discussion_r1081125163 ## processing/src/main/java/org/apache/druid/frame/channel/ByteTracker.java: ## @@ -0,0 +1,57 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one +

[GitHub] [druid] rohangarg commented on a diff in pull request #13368: Allow using composed storage for SuperSorter intermediate data

rohangarg commented on code in PR #13368: URL: https://github.com/apache/druid/pull/13368#discussion_r1081124937 ## processing/src/main/java/org/apache/druid/frame/channel/ByteTracker.java: ## @@ -0,0 +1,57 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one +

[GitHub] [druid] rohangarg commented on a diff in pull request #13368: Allow using composed storage for SuperSorter intermediate data

rohangarg commented on code in PR #13368: URL: https://github.com/apache/druid/pull/13368#discussion_r1081124120 ## processing/src/main/java/org/apache/druid/frame/channel/ComposingReadableFrameChannel.java: ## @@ -0,0 +1,147 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] [druid] rohangarg commented on a diff in pull request #13368: Allow using composed storage for SuperSorter intermediate data

rohangarg commented on code in PR #13368: URL: https://github.com/apache/druid/pull/13368#discussion_r1081123621 ## processing/src/main/java/org/apache/druid/frame/channel/ComposingReadableFrameChannel.java: ## @@ -0,0 +1,147 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] [druid] rohangarg commented on a diff in pull request #13368: Allow using composed storage for SuperSorter intermediate data

rohangarg commented on code in PR #13368: URL: https://github.com/apache/druid/pull/13368#discussion_r1081123338 ## processing/src/main/java/org/apache/druid/frame/channel/ComposingReadableFrameChannel.java: ## @@ -0,0 +1,147 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] [druid] rohangarg commented on a diff in pull request #13368: Allow using composed storage for SuperSorter intermediate data

rohangarg commented on code in PR #13368: URL: https://github.com/apache/druid/pull/13368#discussion_r1081122762 ## processing/src/main/java/org/apache/druid/frame/channel/ComposingReadableFrameChannel.java: ## @@ -0,0 +1,147 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] [druid] rohangarg commented on a diff in pull request #13368: Allow using composed storage for SuperSorter intermediate data

rohangarg commented on code in PR #13368: URL: https://github.com/apache/druid/pull/13368#discussion_r1081122205 ## processing/src/main/java/org/apache/druid/frame/channel/ComposingWritableFrameChannel.java: ## @@ -0,0 +1,99 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] [druid] rohangarg commented on a diff in pull request #13368: Allow using composed storage for SuperSorter intermediate data

rohangarg commented on code in PR #13368: URL: https://github.com/apache/druid/pull/13368#discussion_r1081120486 ## processing/src/main/java/org/apache/druid/frame/channel/ComposingWritableFrameChannel.java: ## @@ -0,0 +1,99 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] [druid] findingrish closed pull request #13661: Update exception message for ingestion parse errors

findingrish closed pull request #13661: Update exception message for ingestion parse errors URL: https://github.com/apache/druid/pull/13661 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the speci

[GitHub] [druid] abhagraw opened a new pull request, #13696: Query tests to new IT framework

abhagraw opened a new pull request, #13696: URL: https://github.com/apache/druid/pull/13696 Migrated the old query group tests to the new IT framework. This PR migrates the following tests - ITTwitterQueryTest ITBroadcastJoinQueryTest ITSqlCancelTest ITSystemTableQueryT

[GitHub] [druid] adarshsanjeev commented on a diff in pull request #13674: Add delete syntax

adarshsanjeev commented on code in PR #13674: URL: https://github.com/apache/druid/pull/13674#discussion_r1080913156 ## sql/src/main/codegen/includes/delete.ftl: ## @@ -0,0 +1,69 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor licens

[GitHub] [druid] AmatyaAvadhanula commented on issue #13692: Druid Version 0.22.1 - Long Coordinator handoff times

AmatyaAvadhanula commented on issue #13692: URL: https://github.com/apache/druid/issues/13692#issuecomment-1396563950 1. Could you also please share how long your coordinator runs take and which duty takes the most time as well? (`coordinator/time` metric) 2. Which cost balancer strategy

[GitHub] [druid] kfaraz commented on pull request #13683: doc: fix markdown spacing

kfaraz commented on PR #13683: URL: https://github.com/apache/druid/pull/13683#issuecomment-1396539515 The lines above seem misaligned too. Maybe we align these too in this PR? https://github.com/apache/druid/blob/fb26a1093d11f0574dbb905ac0d39c8eb3777312/docs/tutorials/tutorial-jupyte

[GitHub] [druid] abhishekagarwal87 commented on issue #13692: Druid Version 0.22.1 - Long Coordinator handoff times

abhishekagarwal87 commented on issue #13692: URL: https://github.com/apache/druid/issues/13692#issuecomment-1396533209 Also tagging @kfaraz and @AmatyaAvadhanula if they have some more suggestions. -- This is an automated message from the Apache Git Service. To respond to the message, pl

[GitHub] [druid] abhishekagarwal87 commented on issue #13692: Druid Version 0.22.1 - Long Coordinator handoff times

abhishekagarwal87 commented on issue #13692: URL: https://github.com/apache/druid/issues/13692#issuecomment-1396532889 Wow. that's a pretty impressive scale. I would definitely recommend upgrading to 25.0. However, that alone won't be sufficient. There are some config changes require

[GitHub] [druid] abhagraw commented on a diff in pull request #13669: Generate the IT docker-compose.yaml files

abhagraw commented on code in PR #13669: URL: https://github.com/apache/druid/pull/13669#discussion_r1080856667 ## integration-tests-ex/docs/compose.md: ## @@ -236,3 +236,56 @@ To define a test cluster, do the following: * If you need multiple instances of the same service, ext

[GitHub] [druid] kfaraz opened a new pull request, #13695: Fix value of lookup sync period in docs

kfaraz opened a new pull request, #13695: URL: https://github.com/apache/druid/pull/13695 # Changes - Update value of `druid.manager.lookups.period` in docs to `120_000`, which is the value used in `LookupCoordinatorManagerConfig` - Remove statements that are not pertinent anymore --

[GitHub] [druid] abhagraw commented on a diff in pull request #13669: Generate the IT docker-compose.yaml files

abhagraw commented on code in PR #13669: URL: https://github.com/apache/druid/pull/13669#discussion_r1080827405 ## integration-tests-ex/cases/cluster.sh: ## @@ -137,17 +142,34 @@ function build_shared_dir { chmod -R a+rwx $SHARED_DIR } -# Each test must have a default

[GitHub] [druid] abhagraw commented on a diff in pull request #13669: Generate the IT docker-compose.yaml files

abhagraw commented on code in PR #13669: URL: https://github.com/apache/druid/pull/13669#discussion_r1080856667 ## integration-tests-ex/docs/compose.md: ## @@ -236,3 +236,56 @@ To define a test cluster, do the following: * If you need multiple instances of the same service, ext

[GitHub] [druid] imply-cheddar commented on a diff in pull request #13672: discover nested columns when using nested column indexer for schemaless ingestion

imply-cheddar commented on code in PR #13672: URL: https://github.com/apache/druid/pull/13672#discussion_r1080772845 ## core/src/main/java/org/apache/druid/data/input/impl/DimensionsSpec.java: ## @@ -78,15 +80,16 @@ public static Builder builder() public DimensionsSpec(List

[GitHub] [druid] cryptoe commented on a diff in pull request #13670: msq path to emit unparseable exceptions

cryptoe commented on code in PR #13670: URL: https://github.com/apache/druid/pull/13670#discussion_r1080770286 ## extensions-core/multi-stage-query/src/main/java/org/apache/druid/msq/indexing/error/MSQWarningReportLimiterPublisher.java: ## @@ -108,6 +115,12 @@ public void publis

[GitHub] [druid] imply-cheddar commented on a diff in pull request #13694: Adjust Operators to be Pausable

imply-cheddar commented on code in PR #13694: URL: https://github.com/apache/druid/pull/13694#discussion_r1080766048 ## processing/src/main/java/org/apache/druid/query/rowsandcols/RearrangedRowsAndColumns.java: ## @@ -72,13 +85,14 @@ @Override public int numRows() { -

[GitHub] [druid] imply-cheddar commented on a diff in pull request #13576: SQL version of unnest native druid function

imply-cheddar commented on code in PR #13576: URL: https://github.com/apache/druid/pull/13576#discussion_r1080745376 ## processing/src/main/java/org/apache/druid/segment/UnnestDimensionCursor.java: ## @@ -181,7 +181,10 @@ public void inspectRuntimeShape(RuntimeShapeInspector in

[GitHub] [druid] github-code-scanning[bot] commented on a diff in pull request #13694: Adjust Operators to be Pausable

github-code-scanning[bot] commented on code in PR #13694: URL: https://github.com/apache/druid/pull/13694#discussion_r1080744790 ## processing/src/main/java/org/apache/druid/query/operator/join/SortedInnerJoinOperator.java: ## @@ -0,0 +1,550 @@ +/* + * Licensed to the Apache Sof

[GitHub] [druid] imply-cheddar opened a new pull request, #13694: Adjust Operators to be Pausable

imply-cheddar opened a new pull request, #13694: URL: https://github.com/apache/druid/pull/13694 This enables "merge" style operations that combine multiple streams. This change includes a naive implementation of one such merge operator just to provide concrete evidence that the refac

[GitHub] [druid] TSFenwick opened a new pull request, #13693: attempted :/

TSFenwick opened a new pull request, #13693: URL: https://github.com/apache/druid/pull/13693 Fixes #. ### Description Fixed the bug ... Renamed the class ... Added a forbidden-apis entry ...

[GitHub] [druid] clintropolis merged pull request #13672: discover nested columns when using nested column indexer for schemaless ingestion

clintropolis merged PR #13672: URL: https://github.com/apache/druid/pull/13672 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@druid

[GitHub] [druid] maytasm merged pull request #13689: Fix Parquet Reader for schema-less ingestion need to read all columns

maytasm merged PR #13689: URL: https://github.com/apache/druid/pull/13689 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@druid.apac

[GitHub] [druid] benkrug commented on a diff in pull request #13671: docs: clarify Java precision

benkrug commented on code in PR #13671: URL: https://github.com/apache/druid/pull/13671#discussion_r1074028863 ## docs/querying/sql-operators.md: ## @@ -37,6 +37,10 @@ sidebar_label: "Operators" Operators in [Druid SQL](./sql.md) typically operate on one or two values and re

[GitHub] [druid] benkrug commented on a diff in pull request #13671: docs: clarify Java precision

benkrug commented on code in PR #13671: URL: https://github.com/apache/druid/pull/13671#discussion_r1073979796 ## docs/querying/sql-operators.md: ## @@ -37,6 +37,10 @@ sidebar_label: "Operators" Operators in [Druid SQL](./sql.md) typically operate on one or two values and re

[GitHub] [druid] paul-rogers commented on a diff in pull request #13671: docs: clarify Java precision

paul-rogers commented on code in PR #13671: URL: https://github.com/apache/druid/pull/13671#discussion_r1073918523 ## docs/querying/sql-operators.md: ## @@ -37,6 +37,10 @@ sidebar_label: "Operators" Operators in [Druid SQL](./sql.md) typically operate on one or two values and

[GitHub] [druid] paul-rogers commented on a diff in pull request #13674: Add delete syntax

paul-rogers commented on code in PR #13674: URL: https://github.com/apache/druid/pull/13674#discussion_r1073897566 ## sql/src/main/codegen/includes/delete.ftl: ## @@ -0,0 +1,69 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license

[GitHub] [druid] paul-rogers commented on pull request #13674: Add delete syntax

paul-rogers commented on PR #13674: URL: https://github.com/apache/druid/pull/13674#issuecomment-1387496613 This is a change to SQL syntax. SQL syntax changes typically involve much discussion as we want to make the syntax a SQL-compatible as possible, but we usually need some Druid-specifi

[GitHub] [druid] paul-rogers commented on a diff in pull request #13681: Runs unit-tests on jdk 11, 17 in Github Workflow

paul-rogers commented on code in PR #13681: URL: https://github.com/apache/druid/pull/13681#discussion_r1073851602 ## .github/workflows/reusable-revised-its.yml: ## @@ -0,0 +1,107 @@ +# Licensed to the Apache Software Foundation (ASF) under one or more +# contributor license agr

[GitHub] [druid] acherla opened a new issue, #13692: Druid Version 0.22.1 Long Coordinator handoff times

acherla opened a new issue, #13692: URL: https://github.com/apache/druid/issues/13692 Hi Druid team, We manage a cluster that has over 3K+ peons (116 middle managers) and 200+ historical nodes (4-5 million segments) with two masters configured as both coordinator/overlord. We run close

[GitHub] [druid] paul-rogers commented on a diff in pull request #13691: Add more log statements for authorization

paul-rogers commented on code in PR #13691: URL: https://github.com/apache/druid/pull/13691#discussion_r1073840092 ## sql/src/main/java/org/apache/druid/sql/AbstractStatement.java: ## @@ -160,7 +160,12 @@ protected void authorize( // Authentication is done by the planner

[GitHub] [druid] findingrish commented on a diff in pull request #13682: `date_expand` function

findingrish commented on code in PR #13682: URL: https://github.com/apache/druid/pull/13682#discussion_r1073825727 ## core/src/main/java/org/apache/druid/math/expr/Function.java: ## @@ -3681,4 +3681,58 @@ protected HumanReadableBytes.UnitSystem getUnitSystem() return Huma

[GitHub] [druid] findingrish commented on a diff in pull request #13682: `date_expand` function

findingrish commented on code in PR #13682: URL: https://github.com/apache/druid/pull/13682#discussion_r1073825727 ## core/src/main/java/org/apache/druid/math/expr/Function.java: ## @@ -3681,4 +3681,58 @@ protected HumanReadableBytes.UnitSystem getUnitSystem() return Huma

[GitHub] [druid] findingrish closed pull request #13682: `date_expand` function

findingrish closed pull request #13682: `date_expand` function URL: https://github.com/apache/druid/pull/13682 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e

[GitHub] [druid] somu-imply commented on pull request #13682: `date_expand` function

somu-imply commented on PR #13682: URL: https://github.com/apache/druid/pull/13682#issuecomment-1387409421 Please add some testcases to `FunctionTest.java`. We can add the sql versions in the `CalciteQueryTest.java`. Additionally `sql.md` and `website/.spelling` should include the `date_exp

[GitHub] [druid] paul-rogers merged pull request #13684: Convert from DRUID_INTEGRATION_TEST_INDEXER to USE_INDEXER

paul-rogers merged PR #13684: URL: https://github.com/apache/druid/pull/13684 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@druid.

[GitHub] [druid] adarshsanjeev opened a new pull request, #13691: Add more log statements for authorization

adarshsanjeev opened a new pull request, #13691: URL: https://github.com/apache/druid/pull/13691 Adds logging and for the case that an exception is thrown during authorization This PR has: - [x] been self-reviewed. - [ ] using the [concurrency checklist](

[GitHub] [druid-website-src] petermarshallio opened a new pull request, #367: Update README.md

petermarshallio opened a new pull request, #367: URL: https://github.com/apache/druid-website-src/pull/367 Quick one - made the requirement for node 10 explicit. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

[GitHub] [druid] github-code-scanning[bot] commented on a diff in pull request #13368: Allow using durable storage for SuperSorter intermediate data

github-code-scanning[bot] commented on code in PR #13368: URL: https://github.com/apache/druid/pull/13368#discussion_r1073524878 ## processing/src/main/java/org/apache/druid/frame/channel/ByteTracker.java: ## @@ -0,0 +1,78 @@ +/* + * Licensed to the Apache Software Foundation (A

[GitHub] [druid] stale[bot] commented on pull request #11240: Add apache rocketmq based indexing service

stale[bot] commented on PR #11240: URL: https://github.com/apache/druid/pull/11240#issuecomment-1386854623 This pull request/issue is no longer marked as stale. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the U

[GitHub] [druid] abhishekagarwal87 merged pull request #13690: Install node for unit-tests Github Workflow runners

abhishekagarwal87 merged PR #13690: URL: https://github.com/apache/druid/pull/13690 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@

[GitHub] [druid] abhishekagarwal87 commented on pull request #13690: Install node for unit-tests Github Workflow runners

abhishekagarwal87 commented on PR #13690: URL: https://github.com/apache/druid/pull/13690#issuecomment-1386853354 @vogievetsky FYI. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific c

[GitHub] [druid] abhishekagarwal87 commented on pull request #13617: use AmazonEC2ClientBuilder to create aws ec2 client

abhishekagarwal87 commented on PR #13617: URL: https://github.com/apache/druid/pull/13617#issuecomment-1386728166 @toughrogrammer - can you look into CI-CD failures? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

[GitHub] [druid] abhishekagarwal87 merged pull request #13687: Fix broken links to Oracle JDK docs

abhishekagarwal87 merged PR #13687: URL: https://github.com/apache/druid/pull/13687 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@

[GitHub] [druid] tejaswini-imply opened a new pull request, #13690: Install node for unit-tests Github Workflow runners

tejaswini-imply opened a new pull request, #13690: URL: https://github.com/apache/druid/pull/13690 GitHub workflow `unit-tests` needs node installed. Although GitHub-provided runners have node installed by default, some of the self-hosted runners might not have the same installed. This PR i

[GitHub] [druid] maytasm commented on a diff in pull request #13689: Fix Parquet Reader for schema-less ingestion need to read all columns

maytasm commented on code in PR #13689: URL: https://github.com/apache/druid/pull/13689#discussion_r1073175927 ## server/src/main/java/org/apache/druid/segment/indexing/ReaderUtils.java: ## @@ -60,6 +60,12 @@ public static Set getColumnsRequiredForIngestion( // Find colum

[GitHub] [druid] clintropolis commented on a diff in pull request #13689: Fix Parquet Reader for schema-less ingestion need to read all columns

clintropolis commented on code in PR #13689: URL: https://github.com/apache/druid/pull/13689#discussion_r1073145521 ## server/src/main/java/org/apache/druid/segment/indexing/ReaderUtils.java: ## @@ -60,6 +60,12 @@ public static Set getColumnsRequiredForIngestion( // Find

[GitHub] [druid] maytasm opened a new pull request, #13689: Fix Parquet Reader for schema-less ingestion need to read all columns

maytasm opened a new pull request, #13689: URL: https://github.com/apache/druid/pull/13689 Fix Parquet Reader for schema-less ingestion need to read all columns ### Description When we have either schema-less ingestion without having a flattenSpec (which set useFieldDiscovery t

[GitHub] [druid] somu-imply commented on a diff in pull request #13682: `date_expand` function

somu-imply commented on code in PR #13682: URL: https://github.com/apache/druid/pull/13682#discussion_r1073115716 ## core/src/main/java/org/apache/druid/math/expr/Function.java: ## @@ -3681,4 +3681,58 @@ protected HumanReadableBytes.UnitSystem getUnitSystem() return Human

[GitHub] [druid] clintropolis commented on a diff in pull request #13682: `date_expand` function

clintropolis commented on code in PR #13682: URL: https://github.com/apache/druid/pull/13682#discussion_r1073105114 ## core/src/main/java/org/apache/druid/math/expr/Function.java: ## @@ -3681,4 +3681,58 @@ protected HumanReadableBytes.UnitSystem getUnitSystem() return Hum

[GitHub] [druid] abhishekagarwal87 commented on a diff in pull request #13682: `date_expand` function

abhishekagarwal87 commented on code in PR #13682: URL: https://github.com/apache/druid/pull/13682#discussion_r1073103928 ## core/src/main/java/org/apache/druid/math/expr/Function.java: ## @@ -3681,4 +3681,58 @@ protected HumanReadableBytes.UnitSystem getUnitSystem() retur

[GitHub] [druid] zachjsh commented on pull request #13627: Improved table functions

zachjsh commented on PR #13627: URL: https://github.com/apache/druid/pull/13627#issuecomment-1386467104 @paul-rogers , hmm it doesnt seem to give me the ability to approve the pr anymore 😢 . But I do approve of change. -- This is an automated message from the Apache Git Service. T

[GitHub] [druid] laglangyue closed issue #13688: How to use meta API?

laglangyue closed issue #13688: How to use meta API? URL: https://github.com/apache/druid/issues/13688 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: c

[GitHub] [druid] laglangyue opened a new issue, #13688: How to use meta API?

laglangyue opened a new issue, #13688: URL: https://github.com/apache/druid/issues/13688 /druid/coordinator/v1/metadata/segments?datasources=rollup-tutorial I want to use this api by invoke it to broker，but it return 404. how to use it ? our clusters don't have router,so

[GitHub] [druid] laglangyue closed issue #13658: 0.16.1 No known server / java.lang.RuntimeException

laglangyue closed issue #13658: 0.16.1 No known server / java.lang.RuntimeException URL: https://github.com/apache/druid/issues/13658 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific com

[GitHub] [druid] github-code-scanning[bot] commented on a diff in pull request #13686: Integrate the catalog with the Calcite planner

github-code-scanning[bot] commented on code in PR #13686: URL: https://github.com/apache/druid/pull/13686#discussion_r1073002223 ## sql/src/main/java/org/apache/druid/sql/calcite/table/DatasourceTable.java: ## @@ -181,9 +300,10 @@ public String toString() { // Don't i

[GitHub] [druid] yurmix opened a new pull request, #13687: Fix broken links to Oracle JDK docs

yurmix opened a new pull request, #13687: URL: https://github.com/apache/druid/pull/13687 This simple docs PR fix several broken links to docs.oracle.com. These links were broken due to missing .html (perhaps the server has stopped accepting the original links). -- This is an automated m

[GitHub] [druid] davidgmonical commented on issue #13685: JSON flattener should respect druid null handling config (druid.generic.useDefaultValueForNull) for arrays

davidgmonical commented on issue #13685: URL: https://github.com/apache/druid/issues/13685#issuecomment-1386316165 Nothing underway but happy to take a stab at it. Would love a review once I have it up thank you! -- This is an automated message from the Apache Git Service. To respond to t

[GitHub] [druid] paul-rogers opened a new pull request, #13686: Integrates the catalog with the Calcite planner

paul-rogers opened a new pull request, #13686: URL: https://github.com/apache/druid/pull/13686 This PR is currently a draft. Prior PRs added the catalog (table metadata) foundations, and an improved set of table functions. This PR brings it all together: * Validates the MSQ INS

[GitHub] [druid] benkrug commented on pull request #13671: docs: clarify Java precision

benkrug commented on PR #13671: URL: https://github.com/apache/druid/pull/13671#issuecomment-1386275064 As far as max/min bounds, this seems accurate? https://cs.fit.edu/~ryan/java/language/java-data.html (see the "Numeric" section.) So, for long, it's 2^63 (since one bit is for sign

[GitHub] [druid] paul-rogers commented on pull request #13627: Much improved table functions

paul-rogers commented on PR #13627: URL: https://github.com/apache/druid/pull/13627#issuecomment-1386246212 @maytasm, thanks for checking. @zachjsh had given his approval, but I see it wasn't actually recorded here. @zachjsh, can you do the honors? Thanks! -- This is an automated messag

[GitHub] [druid] clintropolis commented on a diff in pull request #13672: discover nested columns when using nested column indexer for schemaless ingestion

clintropolis commented on code in PR #13672: URL: https://github.com/apache/druid/pull/13672#discussion_r1072946806 ## core/src/main/java/org/apache/druid/java/util/common/parsers/JSONFlattenerMaker.java: ## @@ -57,15 +57,21 @@ public class JSONFlattenerMaker implements ObjectF

[GitHub] [druid] clintropolis commented on a diff in pull request #13672: discover nested columns when using nested column indexer for schemaless ingestion

clintropolis commented on code in PR #13672: URL: https://github.com/apache/druid/pull/13672#discussion_r1072898133 ## core/src/main/java/org/apache/druid/java/util/common/parsers/JSONFlattenerMaker.java: ## @@ -57,15 +57,21 @@ public class JSONFlattenerMaker implements ObjectF

[GitHub] [druid] clintropolis commented on a diff in pull request #13682: `date_expand` function

clintropolis commented on code in PR #13682: URL: https://github.com/apache/druid/pull/13682#discussion_r1072896301 ## core/src/main/java/org/apache/druid/math/expr/Function.java: ## @@ -3681,4 +3681,58 @@ protected HumanReadableBytes.UnitSystem getUnitSystem() return Hum

  1   2   3   4   5   6   7   8   9   10   >