This is an automated email from the ASF dual-hosted git repository.
dependabot[bot] pushed a change to branch
dependabot/npm_and_yarn/website/webpack-dev-server-5.2.6
in repository https://gitbox.apache.org/repos/asf/druid.git
discard 7c6c71aa219 build(deps): bump webpack-dev-server from 5.2.4 to 5.2.6
in /website
add 3ee535bbcd1 minor: [JDK upgrade] Update forking task runner to use
bin/run-java (#19709)
add 543256bd422 fix: loading pool callers decide when to permit (#19717)
add 1ebdf5ff7f9 feat: simd negate and abs (#19703)
add 4a4c8838da5 minor: Add QOS filtering for coordinator endpoints (#19271)
add 15e50434614 perf: add datasource filter pushdown for sys.segments
table (#19718)
add a85d64c4cbd refactor: Make shard spec collector an interface for
extensibility (#19689)
add 4c0f5240c1a fix: Fix undesireable load rule behavior for clustered
segments with conforming shape but no matched groups (#19728)
add dbe1d23ee03 fix: ensure supervisor worker threadpool is reapable
(#19738)
add 80ae6bc7d0f feat: support time ordered cursor for clustered segments
when possible (#19726)
add f46ee871953 feat: simd sqrt (#19742)
add 1e510ab555e feat: catalog + clustered segments (#19711)
add 004f8eb9f40 test: Fixes for flaky StreamIndexFaultToleranceTest tests.
(#19749)
add 13f2f583e2c feat: restrict virtual column usage in clustered segments,
allow remapping them at query time (#19659)
add 67aa25cef0d fix: Avoid sleep on stop in K8sDruidNodeDiscoveryProvider.
(#19748)
add be1863f0bfe feat: Always allow scaling during task rollover in
costBased auto-scaler (#19750)
add ff7584596d8 fix: Stop kill task if segment IDs that share load spec of
killable segment could not be determined (#19737)
add 957121248f3 feat: clustered segment time ordering for realtime queries
(#19753)
add 0ea17adccd4 fix: isMounted() should never be blocked by a lock (#19746)
add 80667e8b37b Bump org.assertj:assertj-core from 3.24.2 to 3.27.7
(#18956)
add d1c3399e40c build(deps): bump org.asynchttpclient:async-http-client
from 3.0.2 to 3.0.10 (#19476)
add a4ac032b714 build(deps): bump
org.apache.commons:commons-configuration2 from 2.10.1 to 2.15.0 (#19490)
add bac0310989a build(deps): bump io.vertx:vertx-core from 4.5.24 to
4.5.27 in /extensions-core/kubernetes-overlord-extensions (#19546)
add b9e5016941f build(deps): bump org.postgresql:postgresql from 42.7.11
to 42.7.12 (#19760)
add 134d4c81866 build(deps): bump org.apache.thrift:libthrift from 0.13.0
to 0.23.0 in /extensions-contrib/thrift-extensions (#19764)
add 5b4e3f955f4 build(deps-dev): bump org.apache.thrift:libthrift from
0.13.0 to 0.23.0 in /embedded-tests (#19763)
add 27652077034 build(deps): bump Jetty from 12.1.8 to 12.1.11 (#19757)
add 0841ec68bea build(deps): bump at.yawk.lz4:lz4-java from 1.10.2 to
1.11.1 (#19756)
add c49ac0ac375 Bump org.apache.maven:maven-artifact from 3.6.0 to 3.9.11
(#18271)
add db7347cecb0 build(deps-dev): bump
io.github.ascopes:protobuf-maven-plugin (#18714)
add 5604d22139e build(deps): upgrade Protobuf from 3.25.8 to 4.33.0 with
gencode compatibility (#18650)
add cbe902f6b62 build(deps): bump io.grpc:grpc-netty-shaded from 1.65.1 to
1.75.0 in /extensions-contrib/opentelemetry-emitter (#18588)
add 36a7d964391 minor: empty cursor holder time ordering (#19773)
add 4bb735c1d75 fix: return actionable errors for invalid native queries
(#19775)
add 6c3e0a98e7a fix: Improve reserved keyword SQL parse errors (#19719)
add ea690003d09 feat: add support for specifying DimensionSchema to
catalog clustered segment spec (#19770)
add f0cbad8f0f0 feat: support partial loads for tasks (#19739)
add 0ed37086bc5 build(deps-dev): bump http-proxy-middleware in
/web-console (#19762)
add ef54970e5b3 build(deps): bump ws in /web-console (#19761)
add c799396edab build(deps): bump immutable from 4.3.0 to 4.3.9 in
/web-console (#19733)
add d19c5343322 build(deps-dev): bump shell-quote from 1.8.1 to 1.10.0 in
/web-console (#19710)
add 2cb507ea663 build(deps-dev): bump websocket-driver in /web-console
(#19694)
add 8211c1efe59 build(deps): bump protobuf.version from 4.33.0 to 4.35.1
(#19786)
add aeb6658a974 build(deps): bump webpack-dev-server from 5.2.4 to 5.2.6
in /website
This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version. This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:
* -- * -- B -- O -- O -- O (7c6c71aa219)
\
N -- N -- N
refs/heads/dependabot/npm_and_yarn/website/webpack-dev-server-5.2.6
(aeb6658a974)
You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.
Any revisions marked "omit" are not gone; other references still
refer to them. Any revisions marked "discard" are gone forever.
No new revisions were added by this update.
Summary of changes:
benchmarks/pom.xml | 5 +
.../benchmark/query/SqlExpressionBenchmark.java | 22 +-
.../calcite/schema/SysSegmentsTableBenchmark.java | 286 ++++++++++
docs/configuration/index.md | 4 +-
docs/data-management/delete.md | 8 +-
docs/ingestion/kafka-ingestion.md | 4 +-
embedded-tests/pom.xml | 2 +-
.../catalog/CatalogIngestAndQueryTest.java | 137 ++++-
.../testing/embedded/catalog/CatalogTestBase.java | 5 +-
.../indexing/StreamIndexFaultToleranceTest.java | 11 +
.../embedded/indexing/StreamIndexTestBase.java | 43 ++
.../msq/EmbeddedMSQPartialDownloadsTest.java | 204 +++++++
.../embedded/tools/ThriftEventSerializer.java | 13 +-
extensions-contrib/grpc-query/pom.xml | 2 +-
.../apache/druid/grpc/server/ProtobufWriter.java | 8 +-
.../org/apache/druid/grpc/server/QueryDriver.java | 6 +-
extensions-contrib/opentelemetry-emitter/pom.xml | 4 +-
extensions-contrib/thrift-extensions/pom.xml | 2 +-
.../data/input/thrift/ThriftDeserialization.java | 29 +-
.../druid/catalog/sql/CatalogInsertTest.java | 4 +
.../druid/catalog/sql/CatalogReplaceTest.java | 4 +
.../apache/druid/catalog/storage/CatalogTests.java | 6 +-
.../kafka/KafkaIndexTaskTuningConfigTest.java | 64 ++-
.../EmbeddedDimensionValueSetShardSpecTest.java | 14 +-
.../discovery/K8sDruidNodeDiscoveryProvider.java | 18 +-
.../K8sDruidNodeDiscoveryProviderTest.java | 94 ++++
.../kubernetes-overlord-extensions/pom.xml | 2 +-
.../MultipleKubernetesTaskRunnerFactoryTest.java | 2 +-
extensions-core/protobuf-extensions/pom.xml | 2 +-
.../common/SegmentCacheManagerFactory.java | 49 +-
.../druid/indexing/common/TaskToolboxFactory.java | 12 +-
.../druid/indexing/common/config/TaskConfig.java | 27 +-
.../common/task/KillUnusedSegmentsTask.java | 162 +++---
.../apache/druid/indexing/common/task/Tasks.java | 1 +
.../druid/indexing/input/DruidInputSource.java | 2 +-
.../druid/indexing/overlord/ForkingTaskRunner.java | 30 +-
.../overlord/config/ForkingTaskRunnerConfig.java | 17 +-
.../overlord/duty/UnusedSegmentsKiller.java | 25 +-
.../seekablestream/DimensionValueSetCollector.java | 191 +++++++
...c.java => DimensionValueSetPartitionsSpec.java} | 51 +-
.../SeekableStreamIndexTaskRunner.java | 160 ++----
.../seekablestream/StreamingPartitionsSpec.java | 119 +----
.../StreamingShardSpecCollector.java | 75 +++
.../supervisor/SeekableStreamSupervisor.java | 12 +-
.../supervisor/autoscaler/CostBasedAutoScaler.java | 9 +-
.../common/SegmentCacheManagerFactoryTest.java | 116 +++-
.../druid/indexing/common/TaskToolboxTest.java | 4 +
.../indexing/common/actions/TaskActionTestKit.java | 36 ++
.../indexing/common/config/TaskConfigBuilder.java | 10 +-
...kStorageConfigTest.java => TaskConfigTest.java} | 26 +-
.../common/task/CompactionTaskRunBase.java | 53 +-
.../druid/indexing/common/task/IndexTaskTest.java | 13 +-
.../indexing/common/task/IngestionTestBase.java | 4 +-
.../common/task/KillUnusedSegmentsTaskTest.java | 150 +++++-
.../AbstractMultiPhaseParallelIndexingTest.java | 2 +-
.../indexing/overlord/ForkingTaskRunnerTest.java | 47 ++
.../druid/indexing/overlord/TaskLifecycleTest.java | 2 +-
.../overlord/duty/UnusedSegmentsKillerTest.java | 32 +-
.../sampler/InputSourceSamplerDiscoveryTest.java | 2 +-
.../SeekableStreamIndexTaskRunnerAuthTest.java | 1 +
.../SeekableStreamIndexTaskRunnerTest.java | 119 +++--
.../TestIndexerMetadataStorageCoordinator.java | 2 +-
licenses.yaml | 49 +-
.../druid/msq/indexing/IndexerWorkerContext.java | 8 +-
.../apache/druid/msq/sql/MSQTaskQueryMaker.java | 42 +-
.../druid/msq/util/MultiStageQueryContext.java | 15 +
.../org/apache/druid/msq/exec/MSQInsertTest.java | 170 ++++++
.../msq/indexing/MSQCompactionRunnerTest.java | 2 +-
.../msq/input/RegularLoadableSegmentTest.java | 17 +-
.../druid/msq/test/CalciteMSQTestsHelper.java | 2 +-
.../org/apache/druid/msq/test/MSQTestBase.java | 24 +-
.../druid/msq/test/MSQTestWorkerContext.java | 7 +-
.../druid/msq/util/MultiStageQueryContextTest.java | 24 +
pom.xml | 20 +-
...lusteredValueGroupsBaseTableProjectionSpec.java | 102 +++-
.../druid/data/input/impl/DimensionSchema.java | 7 +-
.../util/common/concurrent/ScheduledExecutors.java | 10 +-
...VectorMathUnivariateDoubleProcessorFactory.java | 29 +-
...SimpleVectorMathUnivariateProcessorFactory.java | 29 +-
.../math/expr/vector/VectorMathProcessors.java | 8 +-
.../expr/vector/simd/SimdDoubleAbsProcessor.java | 57 ++
.../expr/vector/simd/SimdDoubleNegProcessor.java | 57 ++
.../expr/vector/simd/SimdDoubleSqrtProcessor.java | 59 +++
...rocessor.java => SimdDoubleUnaryProcessor.java} | 46 +-
.../expr/vector/simd/SimdLongAbsProcessor.java | 57 ++
.../expr/vector/simd/SimdLongNegProcessor.java | 57 ++
.../vector/simd/SimdLongToDoubleSqrtProcessor.java | 63 +++
...or.java => SimdLongToDoubleUnaryProcessor.java} | 48 +-
...gProcessor.java => SimdLongUnaryProcessor.java} | 46 +-
.../math/expr/vector/simd/SimdProcessors.java | 47 +-
.../expr/vector/simd/SimdSupportedUnaryOp.java | 26 +-
.../apache/druid/query/BadJsonQueryException.java | 24 +
.../apache/druid/segment/ConcatenatingCursor.java | 10 +-
.../druid/segment/DefaultColumnFormatConfig.java | 2 +-
.../apache/druid/segment/EmptyCursorHolder.java | 32 +-
.../druid/segment/MergingClusterGroupCursor.java | 294 +++++++++++
.../druid/segment/NestedDataColumnSchema.java | 11 +-
.../PartialQueryableIndexCursorFactory.java | 2 +-
.../druid/segment/QueryableIndexCursorFactory.java | 352 ++++++++-----
...AwareIncrementalIndexColumnSelectorFactory.java | 112 ++++
.../incremental/IncrementalIndexCursorFactory.java | 69 ++-
.../segment/nested/NestedDataComplexTypeSerde.java | 2 +-
.../segment/projections/ClusterGroupQueryPlan.java | 101 +++-
.../ClusteringColumnSelectorFactory.java | 46 +-
.../ClusteringVectorColumnSelectorFactory.java | 43 +-
.../projections/MergingColumnSelectorFactory.java | 461 ++++++++++++++++
.../druid/segment/projections/Projections.java | 181 ++++++-
.../segment/vector/ConcatenatingVectorCursor.java | 10 +-
...eredValueGroupsBaseTableProjectionSpecTest.java | 217 ++++++++
.../druid/data/input/impl/DimensionSchemaTest.java | 47 ++
.../common/concurrent/ScheduledExecutorsTest.java | 28 +
.../druid/query/BadJsonQueryExceptionTest.java | 73 +++
.../ClusteredSegmentTimeOrderedQueryTest.java | 583 +++++++++++++++++++++
.../druid/segment/ConcatenatingCursorTest.java | 25 +-
.../druid/segment/EmptyCursorHolderTest.java | 91 ++++
.../druid/segment/IndexMergerV10ClusteredTest.java | 128 +++++
.../segment/MergingClusterGroupCursorTest.java | 349 ++++++++++++
.../druid/segment/NestedDataColumnSchemaTest.java | 2 +-
.../QueryableIndexCursorFactoryClusteredTest.java | 524 +++++++++++++++---
...IncrementalIndexCursorFactoryClusteredTest.java | 228 ++++++++
.../MergingColumnSelectorFactoryTest.java | 274 ++++++++++
.../druid/segment/projections/NoRewriteFilter.java | 62 +++
.../ProjectionsPlanClusterGroupQueryTest.java | 324 +++++++++++-
.../druid/segment/projections/ProjectionsTest.java | 46 ++
.../vector/ConcatenatingVectorCursorTest.java | 28 +-
.../ClusteredValueGroupsBaseTableMetadata.java | 304 +++++++++++
.../catalog/model/DatasourceBaseTableMetadata.java | 65 +++
.../catalog/model/facade/DatasourceFacade.java | 80 ++-
.../druid/catalog/model/table/DatasourceDefn.java | 47 +-
.../druid/catalog/model/table/TableBuilder.java | 6 +
.../org/apache/druid/guice/StorageNodeModule.java | 2 +-
.../IndexerMetadataStorageCoordinator.java | 5 +-
.../IndexerSQLMetadataStorageCoordinator.java | 2 +-
.../druid/metadata/SqlSegmentsMetadataQuery.java | 26 +-
.../loading/PartialSegmentBundleCacheEntry.java | 19 +-
.../loading/PartialSegmentMetadataCacheEntry.java | 31 +-
.../loading/PermitLimitedSegmentRangeReader.java | 86 +++
.../druid/segment/loading/SegmentLoaderConfig.java | 487 +++++++++++++----
.../segment/loading/SegmentLocalCacheManager.java | 52 +-
.../segment/loading/StorageLoadingThreadPool.java | 302 +++++------
.../StorageLocationVirtualStorageManager.java | 10 +-
.../metadata/AbstractSegmentMetadataCache.java | 17 +
.../org/apache/druid/server/QueryResource.java | 4 +
.../druid/server/coordinator/duty/RunRules.java | 109 +---
.../rules/ClusterGroupPartialLoadMatcher.java | 31 +-
.../coordinator/rules/PartialLoadMatcher.java | 19 +-
.../druid/server/initialization/ServerConfig.java | 12 +
.../jetty/CliIndexerServerModule.java | 8 +-
.../server/initialization/jetty/JettyBindings.java | 41 +-
.../jetty/PathExcludingQoSFilter.java | 82 +++
.../ClusteredValueGroupsBaseTableMetadataTest.java | 567 ++++++++++++++++++++
.../catalog/model/table/DatasourceTableTest.java | 144 +++++
.../apache/druid/client/DruidServerConfigTest.java | 13 +-
.../apache/druid/guice/StorageNodeModuleTest.java | 6 +-
.../IndexerSQLMetadataStorageCoordinatorTest.java | 9 +-
.../PermitBoundedListeningExecutorServiceTest.java | 106 ----
.../PermitLimitedSegmentRangeReaderTest.java | 81 +++
.../segment/loading/SegmentLoaderConfigTest.java | 68 ++-
.../SegmentLocalCacheManagerConcurrencyTest.java | 168 ++++--
...SegmentLocalCacheManagerPartialAcquireTest.java | 38 +-
.../SegmentLocalCacheManagerPartialDropTest.java | 2 +-
...egmentLocalCacheManagerPartialRuleLoadTest.java | 9 +-
.../loading/SegmentLocalCacheManagerTest.java | 246 ++-------
.../loading/StorageLoadingThreadPoolTest.java | 87 ++-
.../appenderator/StreamAppenderatorTester.java | 11 +-
.../org/apache/druid/server/QueryResourceTest.java | 18 +
.../apache/druid/server/SegmentManagerTest.java | 48 +-
.../apache/druid/server/StatusResourceTest.java | 6 +-
.../SegmentCacheBootstrapperCacheTest.java | 23 +-
.../coordination/SegmentCacheBootstrapperTest.java | 35 +-
.../coordination/SegmentLoadDropHandlerTest.java | 83 +--
...egmentManagerBroadcastJoinIndexedTableTest.java | 20 +-
.../SegmentManagerThreadSafetyTest.java | 20 +-
.../duty/RunRulesEmptyDeferringHandlerTest.java | 255 ---------
.../duty/RunRulesPartialLoadPlacementTest.java | 285 ++++++++++
...WildcardClusterGroupPartialLoadMatcherTest.java | 22 +-
.../jetty/PathExcludingQoSFilterTest.java | 154 ++++++
.../server/metrics/SegmentStatsMonitorTest.java | 2 +-
.../java/org/apache/druid/cli/CliCoordinator.java | 20 +
.../java/org/apache/druid/cli/CliOverlord.java | 32 +-
.../org/apache/druid/cli/CliCoordinatorTest.java | 121 +++++
.../java/org/apache/druid/cli/CliOverlordTest.java | 23 +-
.../druid/sql/calcite/parser/DruidSqlParser.java | 95 +++-
.../sql/calcite/planner/DruidSqlValidator.java | 13 +-
.../sql/calcite/schema/MetadataSegmentView.java | 22 +-
.../druid/sql/calcite/schema/SystemSchema.java | 34 +-
.../sql/calcite/schema/SystemSchemaFilters.java | 184 +++++++
.../schema/SystemServerPropertiesTable.java | 61 +--
.../calcite/CalciteCatalogIngestionDmlTest.java | 194 ++++++-
.../sql/calcite/parser/DruidSqlParserTest.java | 50 ++
.../calcite/schema/SystemSchemaFiltersTest.java | 274 ++++++++++
.../druid/sql/calcite/schema/SystemSchemaTest.java | 127 ++++-
web-console/package-lock.json | 88 ++--
website/.spelling | 1 +
194 files changed, 11655 insertions(+), 2532 deletions(-)
create mode 100644
benchmarks/src/test/java/org/apache/druid/sql/calcite/schema/SysSegmentsTableBenchmark.java
create mode 100644
embedded-tests/src/test/java/org/apache/druid/testing/embedded/msq/EmbeddedMSQPartialDownloadsTest.java
create mode 100644
indexing-service/src/main/java/org/apache/druid/indexing/seekablestream/DimensionValueSetCollector.java
copy
indexing-service/src/main/java/org/apache/druid/indexing/seekablestream/{StreamingPartitionsSpec.java
=> DimensionValueSetPartitionsSpec.java} (65%)
create mode 100644
indexing-service/src/main/java/org/apache/druid/indexing/seekablestream/StreamingShardSpecCollector.java
copy
indexing-service/src/test/java/org/apache/druid/indexing/common/config/{TaskStorageConfigTest.java
=> TaskConfigTest.java} (60%)
create mode 100644
processing/src/main/java/org/apache/druid/math/expr/vector/simd/SimdDoubleAbsProcessor.java
create mode 100644
processing/src/main/java/org/apache/druid/math/expr/vector/simd/SimdDoubleNegProcessor.java
create mode 100644
processing/src/main/java/org/apache/druid/math/expr/vector/simd/SimdDoubleSqrtProcessor.java
copy
processing/src/main/java/org/apache/druid/math/expr/vector/simd/{SimdDoubleDoubleProcessor.java
=> SimdDoubleUnaryProcessor.java} (59%)
create mode 100644
processing/src/main/java/org/apache/druid/math/expr/vector/simd/SimdLongAbsProcessor.java
create mode 100644
processing/src/main/java/org/apache/druid/math/expr/vector/simd/SimdLongNegProcessor.java
create mode 100644
processing/src/main/java/org/apache/druid/math/expr/vector/simd/SimdLongToDoubleSqrtProcessor.java
copy
processing/src/main/java/org/apache/druid/math/expr/vector/simd/{SimdLongDoubleProcessor.java
=> SimdLongToDoubleUnaryProcessor.java} (58%)
copy
processing/src/main/java/org/apache/druid/math/expr/vector/simd/{SimdLongLongProcessor.java
=> SimdLongUnaryProcessor.java} (60%)
copy
extensions-contrib/compressed-bigdecimal/src/main/java/org/apache/druid/compressedbigdecimal/ObjBiIntConsumer.java
=>
processing/src/main/java/org/apache/druid/math/expr/vector/simd/SimdSupportedUnaryOp.java
(55%)
create mode 100644
processing/src/main/java/org/apache/druid/segment/MergingClusterGroupCursor.java
create mode 100644
processing/src/main/java/org/apache/druid/segment/incremental/ClusteringAwareIncrementalIndexColumnSelectorFactory.java
create mode 100644
processing/src/main/java/org/apache/druid/segment/projections/MergingColumnSelectorFactory.java
create mode 100644
processing/src/test/java/org/apache/druid/query/BadJsonQueryExceptionTest.java
create mode 100644
processing/src/test/java/org/apache/druid/segment/ClusteredSegmentTimeOrderedQueryTest.java
create mode 100644
processing/src/test/java/org/apache/druid/segment/EmptyCursorHolderTest.java
create mode 100644
processing/src/test/java/org/apache/druid/segment/MergingClusterGroupCursorTest.java
create mode 100644
processing/src/test/java/org/apache/druid/segment/projections/MergingColumnSelectorFactoryTest.java
create mode 100644
processing/src/test/java/org/apache/druid/segment/projections/NoRewriteFilter.java
create mode 100644
server/src/main/java/org/apache/druid/catalog/model/ClusteredValueGroupsBaseTableMetadata.java
create mode 100644
server/src/main/java/org/apache/druid/catalog/model/DatasourceBaseTableMetadata.java
create mode 100644
server/src/main/java/org/apache/druid/segment/loading/PermitLimitedSegmentRangeReader.java
create mode 100644
server/src/main/java/org/apache/druid/server/initialization/jetty/PathExcludingQoSFilter.java
create mode 100644
server/src/test/java/org/apache/druid/catalog/model/ClusteredValueGroupsBaseTableMetadataTest.java
delete mode 100644
server/src/test/java/org/apache/druid/segment/loading/PermitBoundedListeningExecutorServiceTest.java
create mode 100644
server/src/test/java/org/apache/druid/segment/loading/PermitLimitedSegmentRangeReaderTest.java
delete mode 100644
server/src/test/java/org/apache/druid/server/coordinator/duty/RunRulesEmptyDeferringHandlerTest.java
create mode 100644
server/src/test/java/org/apache/druid/server/coordinator/duty/RunRulesPartialLoadPlacementTest.java
create mode 100644
server/src/test/java/org/apache/druid/server/initialization/jetty/PathExcludingQoSFilterTest.java
create mode 100644
services/src/test/java/org/apache/druid/cli/CliCoordinatorTest.java
create mode 100644
sql/src/main/java/org/apache/druid/sql/calcite/schema/SystemSchemaFilters.java
create mode 100644
sql/src/test/java/org/apache/druid/sql/calcite/schema/SystemSchemaFiltersTest.java
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
