This is an automated email from the ASF dual-hosted git repository.
reschke pushed a change to branch OAK-10274
in repository https://gitbox.apache.org/repos/asf/jackrabbit-oak.git
from 0d955ca531 OAK-10274: fix EOLs
add 2e6a9d9de8 OAK-10274: fix EOLs
add 4352bb31fe OAK-10317 : Additional tests for dynamic automembership
add 601352d820 OAK-10304: log registration of invalid namespace names
(#1000)
add 60cb9728ba OAK-5782: Test failure:
persistentCache.BroadcastTest.broadcastTCP
add 16edc2c0b7 Merge pull request #997 from mreutegg/OAK-5782
add a149bfc42f OAK-10316: Importer exceptions always should mention path
(#1001)
add 6f8cf0c7cf OAK-10307: re-add guava to several subprojects to re-enable
isolated compilation (will be removed later on)
add 8c970c0f24 OAK-10307: re-add guava to several subprojects to re-enable
isolated compilation (will be removed later on)
add 330f5cc0fe OAK-10307: re-add guava to several subprojects to re-enable
isolated compilation (will be removed later on)
add 99a28f8e2b OAK-10307: re-add guava to several subprojects to re-enable
isolated compilation (will be removed later on)
add 47ead8c04d OAK-10326: temporarily set baseline version explicitly (and
remove before release of 1.54.0)
add 05030fcba4 OAK-10307: re-add guava to several subprojects to re-enable
isolated compilation (will be removed later on)
add 03904787bc OAK-10324: IN queries on boolean fields should not fail
when one of the values cannot be parsed (#1002)
add 495f96a098 OAK-10313: Counter for DocumentStore check
add 0b6d60bfc3 Merge pull request #996 from mreutegg/OAK-10315
add 676535262b OAK-10093 : provided support for SSE Customer Keys for AWS
add be16e2e4f7 OAK-10093 : replaces if/else block with switch statement
add 133d4d3ab1 OAK-10093 : fixed issues with unit cases and provided steps
to create base64 encoded 32 bytes SSE_C keys
add c91b16e172 OAK-10093 : updated properties file with steps to generate
SSE_C key
add c4aa1b5fc8 OAK-10093 : fixed code smells suggested by SONAR
add 3a744e293a OAK-10093 : incorporated review comments
add d18125c839 OAK-10093 : added note that PresinedUrls won't work with
Direct Binary Access
add 83b30949f8 Merge pull request #960 from apache/OAK-10093
add 53ee3d9188 OAK-10330: oak-it-osgi: temporarily introduce Guava
dependency again
add 4c98e62760 Merge pull request #1009 from mreutegg/OAK-10330
add 4b1d15e5c5 OAK-10315: Counter for DocumentStore check
add 41e9d740ce Merge pull request #1008 from mreutegg/OAK-10315-1
add 667e2d234b OAK-10328 fix scope of "jackrabbit-jcr-tests" (#1005)
add 5aa3c1b4dd oak-search-elastic: field values should be deduplicated
(#1006)
add e50de3a317 OAK-10226: fix lucene->elastic conversion for WordDelimiter
(#1003)
add e50f432887 OAK-10313: Identify revisions created by late-write scenario
add 41e78a3a77 OAK-10313: Identify revisions created by late-write scenario
add e2bbbebf49 Merge branch 'trunk' into OAK-10313
add c5fe2b73cd OAK-10313: Identify revisions created by late-write scenario
add 3669b31f39 Merge branch 'trunk' into OAK-10313
add 56366607a4 Merge pull request #995 from mreutegg/OAK-10313
add cbfacdb90f Initial commit of pipelined download strategy.
add c505124a35 Minor fixes
add 03e3be8992 WIP
add 58a36a1bd2 Merge remote-tracking branch 'upstream/trunk' into
GRANITE-45911
add f805fdab28 Add recovery from broken MongoDB connections to downloader.
add 9bfb9fc54a Merge remote-tracking branch 'upstream/trunk' into
GRANITE-45911
add 45d0eaf291 Shutdown gracefully if one of the dump stages fails with an
exception.
add 9d1382ccb0 Add documentation. Change the configuration for the retrial
mechanism for MongoDB connection failures: instead of number of retrials, use
the amount of time to keep trying before giving up.
add db824bd6dc Merge remote-tracking branch 'upstream/trunk' into OAK-10294
add 832074fe64 Merge remote-tracking branch 'upstream/trunk' into OAK-10294
add 7384304e73 Add support for auto-tuning working set memory based on
total available memory to the indexer. Switch to using system properties to
configure the pipelined strategy (instead of env variables).
add 981e673e6c Always use read preference secondaryPreferred for
downloading from Mongo.
add 0e10939319 Address reviews comments.
add 8633be3e60 Use 0 and Long.MAX_VALUE as boundaries for download range,
no need to query Mongo to determine the earliest and latest values of _modified.
add 5edda975aa Merge remote-tracking branch 'upstream/trunk' into OAK-10294
add c0b80afe51 Add a new stage to merge sorted files. Log how much time
the download and transform threads spend waiting to enqueue their outputs in
the out queues (this indicates that the stages after them are too slow).
Collect and log metrics describing the download and transform stages, like
mongo documents downloaded, node state entries extracted, filtered, mongo
documents that do not match a node state (garbage) and a few others. Use only
FileUtils.byteCountToDisplaySize() to p [...]
add ea7ce8a70f Merge remote-tracking branch 'upstream/trunk' into OAK-10294
add 2b84a30d44 Improve collection and logging of metrics.
add c68fad0dcc Add unit test for merge-sort task.
add 93f9eacd85 Merge remote-tracking branch 'upstream/trunk' into OAK-10294
add 8c5dc547e8 Add more tests, including a first draft of an integration
test. Refactor code.
add 787a4d575c Fix
add 62697ca44e Merge remote-tracking branch 'upstream/trunk' into OAK-10294
add 66d176bb90 Add an integration test
add 6387d4dc5b Add license header.
add a9dd47546f Add license header.
add ab27625a83 Rename test files.
add fabea754fd Do not filter on _modified != null when doing
non-recoverable download.
add 1cf44cf70c Merge remote-tracking branch 'upstream/trunk' into OAK-10294
add a7f48a0370 Fixes based on review comments.
add 5590997ded Bound the size of the histograms.
add ab06697edb Add missing license files.
add 2839cc9d45 Improve logging and code clean of bounded histogram.
add ef2182e4aa Display human readable byte counts with 2 decimal places
instead of abbreviating to closest integer value.
add 27f9b710ef Improved logging.
add 4f92e897d9 Refactoring
add 4c33e932b4 Remove changes unrelated to PR.
add ec2b12fd8e Merge remote-tracking branch 'upstream/trunk' into OAK-10294
add a4b7bf1016 Add tests for error handling when configuration properties
have invalid values. Minor refactoring.
add 11f308c86f Add test for when the path predicate does not match any
existing path in the document store. Address other review comments.
add 81eaeead4a Update
oak-run-commons/src/main/java/org/apache/jackrabbit/oak/index/indexer/document/flatfile/pipelined/PipelinedMergeSortTask.java
add 560dd56d5d Update
oak-run-commons/src/main/java/org/apache/jackrabbit/oak/index/indexer/document/flatfile/pipelined/PipelinedMongoDownloadTask.java
add 00923a38c0 Merge remote-tracking branch 'origin/OAK-10294' into
OAK-10294
add 074e4af804 Add unit test to test recovery from broken mongo
connections. Fix: when a connection to Mongo is lost, the documents that were
collected in a block but not yet enqueued were being lost.
add 70e6b89a86 Add missing license.
add 25c01b8176 Merge pull request #979 from nfsantos/OAK-10294
add 8677db8d98 OAK-10331 add caller information to different warnings
(#1010)
add 965d6703b5 OAK-10337: mvn jetty:run fails to start oak-web (#1016)
add db3bf04066 OAK-10307: re-add guava to several subprojects to re-enable
isolated compilation (will be removed later on) - fix import range
add e1d3e3e8db OAK-10338 - Do not use platform-dependent newlines in for
files generated when creating a FlatFileStore (#1018)
add 6b1686326e OAK-10307: re-add guava to several subprojects to re-enable
isolated compilation (will be removed later on) - fix import range
add 288e81cacb OAK-10342: Update Apache Parent POM to version 30
add 1a6acc0a8b OAK-10338: Change test to tolerate different line endings
add 9ea124cfbb OAK-10327 Embedded dependencies should have "provided"
scope (#1004)
add 53bc40653e OAK-10326: remove baseline comparison version (back to
default)
add 4d7cf4b58d Revert "OAK-10342: Update Apache Parent POM to version 30"
add 03b703e91f OAK-10332 Allow reusing FilterProviderImpl outside OSGi
context (#1014)
add 5e7c1f6bc8 OAK-10349: Update Tomcat dependency to 9.0.78 (#1022)
add 9e56920b14 OAK-10345 - Add debug log message if larger blob is
compared byte-wise. (#1024)
add 8ef02635be OAK-10352: Update (shaded) Guava to 32.1.1 (#1028)
add 1ae7fa3f64 Merge remote-tracking branch 'origin/trunk' into OAK-10274
No new revisions were added by this update.
Summary of changes:
oak-auth-external/pom.xml | 7 +
.../external/impl/DynamicAutomembershipTest.java | 208 +++++-
oak-authorization-cug/pom.xml | 7 +
oak-authorization-principalbased/pom.xml | 7 +
.../principalbased/impl/FilterProviderImpl.java | 23 +-
.../impl/AbstractPrincipalBasedTest.java | 4 +-
oak-blob-cloud-azure/pom.xml | 4 +
.../jackrabbit/oak/blob/cloud/s3/S3Backend.java | 41 +-
.../jackrabbit/oak/blob/cloud/s3/S3Constants.java | 12 +
.../oak/blob/cloud/s3/S3RequestDecorator.java | 86 ++-
...ithKey.java => TestS3DSWithSSECustomerKey.java} | 28 +-
.../jackrabbit/oak/blob/cloud/s3/TestS3Ds.java | 63 +-
oak-blob-cloud/src/test/resources/aws.properties | 4 +
.../org/apache/jackrabbit/oak/commons/IOUtils.java | 21 +
.../jackrabbit/oak/commons/sort/ExternalSort.java | 4 +-
.../jackrabbit/oak/commons/jmx/package-info.java | 40 +-
.../plugins/name/ReadWriteNamespaceRegistry.java | 16 +
.../org/apache/jackrabbit/oak/query/QueryImpl.java | 7 +-
.../site/markdown/features/direct-binary-access.md | 2 +
oak-examples/webapp/pom.xml | 21 +-
oak-it-osgi/pom.xml | 5 +
.../name/ReadWriteNamespaceRegistryTest.java | 48 +-
oak-jcr/pom.xml | 8 +
.../jackrabbit/oak/jcr/xml/ImporterImpl.java | 4 +-
oak-lucene/pom.xml | 7 +
oak-run-commons/pom.xml | 7 +
.../indexer/document/DocumentStoreIndexerBase.java | 63 +-
.../indexer/document/NodeStateEntryTraverser.java | 7 +-
.../document/NodeStateEntryTraverserFactory.java | 5 +-
.../flatfile/FlatFileNodeStoreBuilder.java | 39 +-
.../document/flatfile/FlatFileStoreUtils.java | 8 +-
.../MultithreadedTraverseWithSortStrategy.java | 10 +-
.../document/flatfile/NodeStateEntryWriter.java | 26 +-
.../document/flatfile/StoreAndSortStrategy.java | 11 +-
.../document/flatfile/TraverseAndSortTask.java | 10 +-
.../flatfile/TraverseWithSortStrategy.java | 19 +-
.../flatfile/pipelined/BoundedHistogram.java | 89 +++
.../document/flatfile/pipelined/ConfigHelper.java} | 44 +-
.../document/flatfile/pipelined/DownloadRange.java | 74 +++
.../flatfile/pipelined/NodeStateEntryBatch.java | 96 +++
.../NodeStateHolder.java} | 39 +-
.../{ => pipelined}/PathElementComparator.java | 32 +-
.../flatfile/pipelined/PipelinedMergeSortTask.java | 140 ++++
.../pipelined/PipelinedMongoDownloadTask.java | 327 ++++++++++
.../flatfile/pipelined/PipelinedSortBatchTask.java | 155 +++++
.../flatfile/pipelined/PipelinedStrategy.java | 433 +++++++++++++
.../flatfile/pipelined/PipelinedTransformTask.java | 246 +++++++
.../document/flatfile/pipelined/SortKey.java | 89 +++
.../pipelined/TransformStageStatistics.java | 171 +++++
.../document/mongo/MongoDocumentStoreHelper.java | 9 +
.../document/mongo/MongoDocumentTraverser.java | 65 +-
.../plugins/document/mongo/TraversingRange.java | 81 +++
.../document/flatfile/FlatFileStoreTest.java | 8 +-
.../MultithreadedTraverseWithSortStrategyTest.java | 4 +-
.../document/flatfile/TraverseAndSortTaskTest.java | 2 +-
.../flatfile/pipelined/BoundedHistogramTest.java | 56 ++
.../pipelined/NodeStateEntryBatchTest.java | 88 +++
.../document/flatfile/pipelined/PipelinedIT.java | 214 +++++++
.../pipelined/PipelinedMergeSortTaskTest.java | 121 ++++
.../pipelined/PipelinedMongoDownloadTaskTest.java | 104 +++
.../pipelined/PipelinedSortBatchTaskTest.java | 186 ++++++
.../test/resources/pipelined/merge-expected.json | 6 +
.../test/resources/pipelined/merge-stage-1.json | 3 +
.../test/resources/pipelined/merge-stage-2.json | 3 +
...rphanedNodeCheck.java => ConsistencyCheck.java} | 80 +--
.../plugins/document/check/DocumentStoreCheck.java | 34 +-
.../oak/plugins/document/check/NodeCounter.java | 77 +++
.../oak/run/DocumentStoreCheckCommand.java | 18 +
.../document/mongo/DocumentTraverserTest.java | 2 +-
.../oak/run/DocumentStoreCheckCommandTest.java | 5 +
.../index/ElasticCustomAnalyzerMappings.java | 43 ++
.../index/elastic/index/ElasticDocument.java | 10 +-
.../index/elastic/query/ElasticRequestHandler.java | 50 +-
.../index/elastic/ElasticAbstractQueryTest.java | 15 +
.../plugins/index/elastic/ElasticContentTest.java | 24 +
oak-search-mt/pom.xml | 1 +
oak-search/pom.xml | 6 +-
.../plugins/index/FullTextAnalyzerCommonTest.java | 37 ++
.../oak/plugins/index/PropertyIndexCommonTest.java | 24 +
oak-segment-azure/pom.xml | 1 +
oak-segment-tar/pom.xml | 21 +-
oak-shaded-guava/pom.xml | 4 +-
oak-store-document/pom.xml | 8 +
.../oak/plugins/document/Consistency.java | 138 ++++
.../oak/plugins/document/FailingDocumentStore.java | 37 +-
.../plugins/document/WriteAfterRecoveryTest.java | 707 +++++++++++++++++++++
.../document/persistentCache/BroadcastTest.java | 1 +
oak-store-spi/pom.xml | 5 +
.../oak/plugins/memory/AbstractBlob.java | 21 +
89 files changed, 4688 insertions(+), 448 deletions(-)
copy
oak-blob-cloud/src/test/java/org/apache/jackrabbit/oak/blob/cloud/s3/{TestS3DSWithSSEKMSwithKey.java
=> TestS3DSWithSSECustomerKey.java} (59%)
mode change 100755 => 100644
oak-core-spi/src/main/java/org/apache/jackrabbit/oak/commons/jmx/package-info.java
create mode 100644
oak-run-commons/src/main/java/org/apache/jackrabbit/oak/index/indexer/document/flatfile/pipelined/BoundedHistogram.java
copy
oak-run-commons/src/{test/java/org/apache/jackrabbit/oak/index/indexer/document/flatfile/CountingIterable.java
=>
main/java/org/apache/jackrabbit/oak/index/indexer/document/flatfile/pipelined/ConfigHelper.java}
(53%)
create mode 100644
oak-run-commons/src/main/java/org/apache/jackrabbit/oak/index/indexer/document/flatfile/pipelined/DownloadRange.java
create mode 100644
oak-run-commons/src/main/java/org/apache/jackrabbit/oak/index/indexer/document/flatfile/pipelined/NodeStateEntryBatch.java
copy
oak-run-commons/src/main/java/org/apache/jackrabbit/oak/index/indexer/document/flatfile/{SimpleNodeStateHolder.java
=> pipelined/NodeStateHolder.java} (55%)
copy
oak-run-commons/src/main/java/org/apache/jackrabbit/oak/index/indexer/document/flatfile/{
=> pipelined}/PathElementComparator.java (73%)
create mode 100644
oak-run-commons/src/main/java/org/apache/jackrabbit/oak/index/indexer/document/flatfile/pipelined/PipelinedMergeSortTask.java
create mode 100644
oak-run-commons/src/main/java/org/apache/jackrabbit/oak/index/indexer/document/flatfile/pipelined/PipelinedMongoDownloadTask.java
create mode 100644
oak-run-commons/src/main/java/org/apache/jackrabbit/oak/index/indexer/document/flatfile/pipelined/PipelinedSortBatchTask.java
create mode 100644
oak-run-commons/src/main/java/org/apache/jackrabbit/oak/index/indexer/document/flatfile/pipelined/PipelinedStrategy.java
create mode 100644
oak-run-commons/src/main/java/org/apache/jackrabbit/oak/index/indexer/document/flatfile/pipelined/PipelinedTransformTask.java
create mode 100644
oak-run-commons/src/main/java/org/apache/jackrabbit/oak/index/indexer/document/flatfile/pipelined/SortKey.java
create mode 100644
oak-run-commons/src/main/java/org/apache/jackrabbit/oak/index/indexer/document/flatfile/pipelined/TransformStageStatistics.java
create mode 100644
oak-run-commons/src/main/java/org/apache/jackrabbit/oak/plugins/document/mongo/TraversingRange.java
create mode 100644
oak-run-commons/src/test/java/org/apache/jackrabbit/oak/index/indexer/document/flatfile/pipelined/BoundedHistogramTest.java
create mode 100644
oak-run-commons/src/test/java/org/apache/jackrabbit/oak/index/indexer/document/flatfile/pipelined/NodeStateEntryBatchTest.java
create mode 100644
oak-run-commons/src/test/java/org/apache/jackrabbit/oak/index/indexer/document/flatfile/pipelined/PipelinedIT.java
create mode 100644
oak-run-commons/src/test/java/org/apache/jackrabbit/oak/index/indexer/document/flatfile/pipelined/PipelinedMergeSortTaskTest.java
create mode 100644
oak-run-commons/src/test/java/org/apache/jackrabbit/oak/index/indexer/document/flatfile/pipelined/PipelinedMongoDownloadTaskTest.java
create mode 100644
oak-run-commons/src/test/java/org/apache/jackrabbit/oak/index/indexer/document/flatfile/pipelined/PipelinedSortBatchTaskTest.java
create mode 100644
oak-run-commons/src/test/resources/pipelined/merge-expected.json
create mode 100644
oak-run-commons/src/test/resources/pipelined/merge-stage-1.json
create mode 100644
oak-run-commons/src/test/resources/pipelined/merge-stage-2.json
copy
oak-run/src/main/java/org/apache/jackrabbit/oak/plugins/document/check/{OrphanedNodeCheck.java
=> ConsistencyCheck.java} (53%)
create mode 100644
oak-run/src/main/java/org/apache/jackrabbit/oak/plugins/document/check/NodeCounter.java
create mode 100644
oak-store-document/src/main/java/org/apache/jackrabbit/oak/plugins/document/Consistency.java
create mode 100644
oak-store-document/src/test/java/org/apache/jackrabbit/oak/plugins/document/WriteAfterRecoveryTest.java
