[jira] [Resolved] (OAK-7092) Build Jackrabbit Oak #1115 failed
[ https://issues.apache.org/jira/browse/OAK-7092?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcel Reutegger resolved OAK-7092. --- Resolution: Duplicate > Build Jackrabbit Oak #1115 failed > - > > Key: OAK-7092 > URL: https://issues.apache.org/jira/browse/OAK-7092 > Project: Jackrabbit Oak > Issue Type: Bug > Components: continuous integration >Reporter: Hudson > > No description is provided > The build Jackrabbit Oak #1115 has failed. > First failed run: [Jackrabbit Oak > #1115|https://builds.apache.org/job/Jackrabbit%20Oak/1115/] [console > log|https://builds.apache.org/job/Jackrabbit%20Oak/1115/console] -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Closed] (OAK-7092) Build Jackrabbit Oak #1115 failed
[ https://issues.apache.org/jira/browse/OAK-7092?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcel Reutegger closed OAK-7092. - > Build Jackrabbit Oak #1115 failed > - > > Key: OAK-7092 > URL: https://issues.apache.org/jira/browse/OAK-7092 > Project: Jackrabbit Oak > Issue Type: Bug > Components: continuous integration >Reporter: Hudson > > No description is provided > The build Jackrabbit Oak #1115 has failed. > First failed run: [Jackrabbit Oak > #1115|https://builds.apache.org/job/Jackrabbit%20Oak/1115/] [console > log|https://builds.apache.org/job/Jackrabbit%20Oak/1115/console] -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (OAK-7092) Build Jackrabbit Oak #1115 failed
[ https://issues.apache.org/jira/browse/OAK-7092?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16297928#comment-16297928 ] Hudson commented on OAK-7092: - Previously failing build now is OK. Passed run: [Jackrabbit Oak #1116|https://builds.apache.org/job/Jackrabbit%20Oak/1116/] [console log|https://builds.apache.org/job/Jackrabbit%20Oak/1116/console] > Build Jackrabbit Oak #1115 failed > - > > Key: OAK-7092 > URL: https://issues.apache.org/jira/browse/OAK-7092 > Project: Jackrabbit Oak > Issue Type: Bug > Components: continuous integration >Reporter: Hudson > > No description is provided > The build Jackrabbit Oak #1115 has failed. > First failed run: [Jackrabbit Oak > #1115|https://builds.apache.org/job/Jackrabbit%20Oak/1115/] [console > log|https://builds.apache.org/job/Jackrabbit%20Oak/1115/console] -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (OAK-6353) Use Document order traversal for reindexing performed on DocumentNodeStore setups
[
https://issues.apache.org/jira/browse/OAK-6353?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16297924#comment-16297924
]
Chetan Mehrotra commented on OAK-6353:
--
Some performance numbers for reindexing done for repo having 255M Mongo Docs,
66M nodes under /content and having 4.2M assets
# Normal NodeStore traversal - 13.66 h
*Document Traversal*
A - Default setup
# Total time - 3.469 h
## Time in dumping - 2.405 h
## Time in sorting - 39.87 min
### Batch sorting - 19.13 min
### Merging - 20.17
## Indexing 24 mins
# Space consumed
#* dumped json - 43.6 GB
#* chunked files - 43.6 GB
#* index size - 2.5 GB
{noformat}
2017-12-15 16:48:34 Proceeding to index [/oak:index/damAssetLucene2] upto
checkpoint head {}
2017-12-15 19:12:55 Dumped 65472172 nodestates in json format in 2.405 h
2017-12-15 19:12:55 Compression enabled while sorting : false
(oak.indexer.useZip)
2017-12-15 19:12:55 Delete original dump from traversal : true
(oak.indexer.deleteOriginal)
2017-12-15 19:12:55 Max heap memory (GB) to be used for merge sort : 3
(oak.indexer.maxSortMemoryInGB)
2017-12-15 19:12:57 Sorting with memory 3.2 GB (estimated 12.6 GB)
2017-12-15 19:32:05 Batch sorting done in 19.13 min with 29 files of size 43.6
GB to merge
2017-12-15 19:32:05 Removing the original file temp/flat-file-store/store.json
2017-12-15 19:52:50 Merging of sorted files completed in 20.71 min
2017-12-15 19:52:50 Sorting completed in 39.87 min
2017-12-15 19:52:50 Estimated node count to be traversed for reindexing under /
is [65472172]
2017-12-15 20:16:35 Indexing report
- /oak:index/damAssetLucene2*(4407265)
2017-12-15 20:16:43 Indexing completed for indexes [/oak:index/damAssetLucene2]
in 3.469 h (12488171 ms)
{noformat}
B - Compression enabled in sorting
# Total time - 3.811 h
## Time in dumping - 2.929 h
## Time in sorting - 29.56 min
### Batch sorting - 17.67 min
### Merging - 11.87 min
## Indexing 24 mins
# Space consumed
#* dumped json - 43.6 GB
#* chunked files - 5.5 GB
#* index size - 2.5 GB
{noformat}
2017-12-19 10:56:00 Proceeding to index [/oak:index/damAssetLucene2] upto
checkpoint head {}
2017-12-19 13:51:50 oreBuilder - Dumped 65469575 nodestates in json format in
2.929 h (43.6 GB)
2017-12-19 13:51:50 oreBuilder - Compression enabled while sorting : true
(oak.indexer.useZip)
2017-12-19 13:51:50 oreBuilder - Delete original dump from traversal : true
(oak.indexer.deleteOriginal)
2017-12-19 13:51:50 oreBuilder - Max heap memory (GB) to be used for merge sort
: 3 (oak.indexer.maxSortMemoryInGB)
2017-12-19 13:51:52 Sorter - Sorting with memory 3.2 GB (estimated 12.6 GB)
2017-12-19 14:09:32 Sorter - Batch sorting done in 17.67 min with 29 files of
size 5.5 GB to merge
2017-12-19 14:09:32 Sorter - Removing the original file
temp/flat-file-store/store.json
2017-12-19 14:21:25 Sorter - Merging of sorted files completed in 11.87 min
2017-12-19 14:21:25 Sorter - Sorting completed in 29.56 min
2017-12-19 14:21:26 Estimated node count to be traversed for reindexing under /
is [65469575]
2017-12-19 14:44:30 Indexing report
- /oak:index/damAssetLucene2*(4407265)
2017-12-19 14:44:30 Reindexing completed
2017-12-19 14:44:30 Switched the async lane for indexes at
[/oak:index/damAssetLucene2] back to there original lanes
2017-12-19 14:44:39 Indexing completed for indexes [/oak:index/damAssetLucene2]
in 3.811 h (13718589 ms)
{noformat}
> Use Document order traversal for reindexing performed on DocumentNodeStore
> setups
> -
>
> Key: OAK-6353
> URL: https://issues.apache.org/jira/browse/OAK-6353
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: run
>Reporter: Chetan Mehrotra
>Assignee: Chetan Mehrotra
> Fix For: 1.7.13, 1.8
>
> Attachments: OAK-6353-v1.patch, OAK-6353-v2.patch
>
>
> [~tmueller] suggested
> [here|https://issues.apache.org/jira/browse/OAK-6246?focusedCommentId=16034442=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16034442]
> that document order traversal can be faster compared to current mode of path
> based traversal. Initial test indicate that such a traversal can be order of
> magnitude faster.
> So this task is meant to implement such an approach and see if it can be a
> viable indexing mode used for DocumentNodeStore based setups
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Resolved] (OAK-7093) ActiveDelete synchronization with BlobTracker leaves temp files
[ https://issues.apache.org/jira/browse/OAK-7093?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Amit Jain resolved OAK-7093. Resolution: Fixed Fixed with r1818737 > ActiveDelete synchronization with BlobTracker leaves temp files > --- > > Key: OAK-7093 > URL: https://issues.apache.org/jira/browse/OAK-7093 > Project: Jackrabbit Oak > Issue Type: Bug > Components: blob-plugins >Reporter: Amit Jain >Assignee: Amit Jain > Fix For: 1.8 > > > When synchronizing active deleted files with blob tracker a temp file is > created and not removed. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (OAK-7093) ActiveDelete synchronization with BlobTracker leaves temp files
Amit Jain created OAK-7093: -- Summary: ActiveDelete synchronization with BlobTracker leaves temp files Key: OAK-7093 URL: https://issues.apache.org/jira/browse/OAK-7093 Project: Jackrabbit Oak Issue Type: Bug Components: blob-plugins Reporter: Amit Jain Assignee: Amit Jain Fix For: 1.8 When synchronizing active deleted files with blob tracker a temp file is created and not removed. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (OAK-7092) Build Jackrabbit Oak #1115 failed
Hudson created OAK-7092: --- Summary: Build Jackrabbit Oak #1115 failed Key: OAK-7092 URL: https://issues.apache.org/jira/browse/OAK-7092 Project: Jackrabbit Oak Issue Type: Bug Components: continuous integration Reporter: Hudson No description is provided The build Jackrabbit Oak #1115 has failed. First failed run: [Jackrabbit Oak #1115|https://builds.apache.org/job/Jackrabbit%20Oak/1115/] [console log|https://builds.apache.org/job/Jackrabbit%20Oak/1115/console] -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (OAK-7085) Build Jackrabbit Oak #1113 failed
[ https://issues.apache.org/jira/browse/OAK-7085?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16297318#comment-16297318 ] Hudson commented on OAK-7085: - Previously failing build now is OK. Passed run: [Jackrabbit Oak #1114|https://builds.apache.org/job/Jackrabbit%20Oak/1114/] [console log|https://builds.apache.org/job/Jackrabbit%20Oak/1114/console] > Build Jackrabbit Oak #1113 failed > - > > Key: OAK-7085 > URL: https://issues.apache.org/jira/browse/OAK-7085 > Project: Jackrabbit Oak > Issue Type: Bug > Components: continuous integration >Reporter: Hudson > > No description is provided > The build Jackrabbit Oak #1113 has failed. > First failed run: [Jackrabbit Oak > #1113|https://builds.apache.org/job/Jackrabbit%20Oak/1113/] [console > log|https://builds.apache.org/job/Jackrabbit%20Oak/1113/console] -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (OAK-6606) Move BulkTransferBenchmark to oak-benchmarks module
[
https://issues.apache.org/jira/browse/OAK-6606?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16297300#comment-16297300
]
Andrei Dulceanu commented on OAK-6606:
--
Added missing license header for {{ScalabilityStandbySuite}} at r1818706.
> Move BulkTransferBenchmark to oak-benchmarks module
> ---
>
> Key: OAK-6606
> URL: https://issues.apache.org/jira/browse/OAK-6606
> Project: Jackrabbit Oak
> Issue Type: Task
> Components: segment-tar
>Reporter: Andrei Dulceanu
>Assignee: Andrei Dulceanu
>Priority: Minor
> Labels: cold-standby
> Fix For: 1.8
>
>
> {{BulkTransferBenchmark}} should be moved from {{oak-segment-tar}} to
> {{oak-benchmarks}} to allow standard run of this cold standby related
> benchmark.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Created] (OAK-7091) Avoid streaming data twice in composite data store
Matt Ryan created OAK-7091: -- Summary: Avoid streaming data twice in composite data store Key: OAK-7091 URL: https://issues.apache.org/jira/browse/OAK-7091 Project: Jackrabbit Oak Issue Type: Technical task Reporter: Matt Ryan Assignee: Matt Ryan When adding a new record to an Oak instance that is using composite data store, the blob stream will be read twice before it is stored - once by the composite data store (to determine the blob ID) and again by the delegate. We could add a method to the CompositeDataStoreAware interface wherein the data store can be told which blob ID to use (from the composite) so that it doesn't have to process the stream again. Then the composite data store, after having read the stream to a temporary file, can pass an input stream from the temporary file to the delegate along with the computed blob ID, to avoid reading the stream twice. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Assigned] (OAK-7089) Populate composite data store blob ID table at startup
[ https://issues.apache.org/jira/browse/OAK-7089?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt Ryan reassigned OAK-7089: -- Assignee: (was: Matt Ryan) > Populate composite data store blob ID table at startup > -- > > Key: OAK-7089 > URL: https://issues.apache.org/jira/browse/OAK-7089 > Project: Jackrabbit Oak > Issue Type: Technical task > Components: blob, blob-cloud, blob-cloud-azure, blob-plugins >Reporter: Matt Ryan > > The composite data store blob ID mapping table is used to quickly find the > delegate that is handling this blob ID. At startup we need to fill the table > with appropriate mappings in order for it to be useful. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (OAK-7090) Use Bloom filters for composite data store blob ID lookup table
Matt Ryan created OAK-7090: -- Summary: Use Bloom filters for composite data store blob ID lookup table Key: OAK-7090 URL: https://issues.apache.org/jira/browse/OAK-7090 Project: Jackrabbit Oak Issue Type: Technical task Reporter: Matt Ryan The composite data store attempts to keep a mapping of blob ids to delegates where that blob id should be found. We should use Bloom filters to make this mapping more efficient. There are a couple of challenges with implementing Bloom filters for this purpose. # Determining the appropriate size of the Bloom filter. Assuming OAK-7089 is completed before this one, we should have a reasonable guess as to the number of blob IDs at startup time, but this may change over time. This may require a task to rebuild the table for a more appropriate size once the table becomes too full (too many false positives). # Handling deletions. Once a record has been deleted, the corresponding blob ID may also need to be removed (similar algorithm to data store GC). Bloom filters don't typically handle deletions though. This may require something like e.g. [Invertible Bloom Filter|http://www.i-programmer.info/programming/theory/4641-the-invertible-bloom-filter.html], or this may be as simple as using data store GC time to rebuild the Bloom filter appropriately. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (OAK-7089) Populate composite data store blob ID table at startup
Matt Ryan created OAK-7089: -- Summary: Populate composite data store blob ID table at startup Key: OAK-7089 URL: https://issues.apache.org/jira/browse/OAK-7089 Project: Jackrabbit Oak Issue Type: Technical task Reporter: Matt Ryan Assignee: Matt Ryan The composite data store blob ID mapping table is used to quickly find the delegate that is handling this blob ID. At startup we need to fill the table with appropriate mappings in order for it to be useful. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (OAK-7088) Add AzureDataStoreFactory
Matt Ryan created OAK-7088: -- Summary: Add AzureDataStoreFactory Key: OAK-7088 URL: https://issues.apache.org/jira/browse/OAK-7088 Project: Jackrabbit Oak Issue Type: Technical task Reporter: Matt Ryan Assignee: Matt Ryan Support using AzureDataStore as delegates for CompositeDataStore. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (OAK-7087) Add S3DataStoreFactory
[ https://issues.apache.org/jira/browse/OAK-7087?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16297242#comment-16297242 ] Matt Ryan commented on OAK-7087: Available for review in the composite data store pull request: https://github.com/apache/jackrabbit-oak/pull/74 > Add S3DataStoreFactory > -- > > Key: OAK-7087 > URL: https://issues.apache.org/jira/browse/OAK-7087 > Project: Jackrabbit Oak > Issue Type: Technical task > Components: blob, blob-cloud, blob-cloud-azure, blob-plugins >Reporter: Matt Ryan >Assignee: Matt Ryan > > Support using S3DataStore as delegates for CompositeDataStore. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (OAK-7087) Add S3DataStoreFactory
Matt Ryan created OAK-7087: -- Summary: Add S3DataStoreFactory Key: OAK-7087 URL: https://issues.apache.org/jira/browse/OAK-7087 Project: Jackrabbit Oak Issue Type: Technical task Reporter: Matt Ryan Assignee: Matt Ryan Support using S3DataStore as delegates for CompositeDataStore. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (OAK-7086) Add FileDataStoreFactory
[ https://issues.apache.org/jira/browse/OAK-7086?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16297222#comment-16297222 ] Matt Ryan commented on OAK-7086: The original implementation for this was submitted for review in this pull request: https://github.com/apache/jackrabbit-oak/pull/71 Since that time it has been merged into the primary pull request for composite data store, which is open for review: https://github.com/apache/jackrabbit-oak/pull/74 > Add FileDataStoreFactory > > > Key: OAK-7086 > URL: https://issues.apache.org/jira/browse/OAK-7086 > Project: Jackrabbit Oak > Issue Type: Technical task > Components: blob, blob-cloud, blob-cloud-azure, blob-plugins >Reporter: Matt Ryan >Assignee: Matt Ryan > > This allows FileDataStore to be used as a delegate for CompositeDataStore. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (OAK-7086) Add FileDataStoreFactory
Matt Ryan created OAK-7086: -- Summary: Add FileDataStoreFactory Key: OAK-7086 URL: https://issues.apache.org/jira/browse/OAK-7086 Project: Jackrabbit Oak Issue Type: Technical task Reporter: Matt Ryan Assignee: Matt Ryan This allows FileDataStore to be used as a delegate for CompositeDataStore. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (OAK-7084) Implement CompositeDataStore and CompositeDataStoreService
[ https://issues.apache.org/jira/browse/OAK-7084?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16297219#comment-16297219 ] Matt Ryan commented on OAK-7084: The composite data store and delegates are configured as follows: Delegates use a factory to be configured. They must specify a role name (any string) and any other configuration specific to that delegate (e.g. readOnly=true). The composite configuration simply takes a "roles" property with a value containing a comma-delimited list of roles that composite uses, for example "roles=production,staging". > Implement CompositeDataStore and CompositeDataStoreService > -- > > Key: OAK-7084 > URL: https://issues.apache.org/jira/browse/OAK-7084 > Project: Jackrabbit Oak > Issue Type: Technical task > Components: blob, blob-cloud, blob-cloud-azure, blob-plugins >Reporter: Matt Ryan >Assignee: Matt Ryan > > This task is to wire up the composite data store and the corresponding > service, respond to OSGi events when delegates are created, register the > service appropriately, implement the data store portions, etc. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (OAK-7085) Build Jackrabbit Oak #1113 failed
Hudson created OAK-7085: --- Summary: Build Jackrabbit Oak #1113 failed Key: OAK-7085 URL: https://issues.apache.org/jira/browse/OAK-7085 Project: Jackrabbit Oak Issue Type: Bug Components: continuous integration Reporter: Hudson No description is provided The build Jackrabbit Oak #1113 has failed. First failed run: [Jackrabbit Oak #1113|https://builds.apache.org/job/Jackrabbit%20Oak/1113/] [console log|https://builds.apache.org/job/Jackrabbit%20Oak/1113/console] -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (OAK-7084) Implement CompositeDataStore and CompositeDataStoreService
[ https://issues.apache.org/jira/browse/OAK-7084?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16297199#comment-16297199 ] Matt Ryan commented on OAK-7084: There is a pull request in for review at https://github.com/apache/jackrabbit-oak/pull/74, open for review. > Implement CompositeDataStore and CompositeDataStoreService > -- > > Key: OAK-7084 > URL: https://issues.apache.org/jira/browse/OAK-7084 > Project: Jackrabbit Oak > Issue Type: Technical task > Components: blob, blob-cloud, blob-cloud-azure, blob-plugins >Reporter: Matt Ryan >Assignee: Matt Ryan > > This task is to wire up the composite data store and the corresponding > service, respond to OSGi events when delegates are created, register the > service appropriately, implement the data store portions, etc. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (OAK-7084) Implement CompositeDataStore and CompositeDataStoreService
Matt Ryan created OAK-7084: -- Summary: Implement CompositeDataStore and CompositeDataStoreService Key: OAK-7084 URL: https://issues.apache.org/jira/browse/OAK-7084 Project: Jackrabbit Oak Issue Type: Technical task Reporter: Matt Ryan Assignee: Matt Ryan This task is to wire up the composite data store and the corresponding service, respond to OSGi events when delegates are created, register the service appropriately, implement the data store portions, etc. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Assigned] (OAK-5960) Multi blob store support
[ https://issues.apache.org/jira/browse/OAK-5960?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt Ryan reassigned OAK-5960: -- Assignee: (was: Matt Ryan) > Multi blob store support > > > Key: OAK-5960 > URL: https://issues.apache.org/jira/browse/OAK-5960 > Project: Jackrabbit Oak > Issue Type: Epic > Components: blob >Reporter: Amit Jain > > Epic to collect issues and discussion for support for multi blob store which > could potentially support the following scenarios: > * A primary writable blob store and read only secondary blob stores - useful > for cloud cross-geo deployments > * Typed blob store where based on type passed the blobs are written and read > from corresponding blob stores. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (OAK-7083) CompositeDataStore - ReadOnly/ReadWrite Delegate Support
Matt Ryan created OAK-7083: -- Summary: CompositeDataStore - ReadOnly/ReadWrite Delegate Support Key: OAK-7083 URL: https://issues.apache.org/jira/browse/OAK-7083 Project: Jackrabbit Oak Issue Type: New Feature Components: blob, blob-cloud, blob-cloud-azure, blob-plugins Reporter: Matt Ryan Assignee: Matt Ryan Support a specific composite data store use case, which is the following: * One instance uses no composite data store, but instead is using a single standard Oak data store (e.g. FileDataStore) * Another instance is created by snapshotting the first instance node store, and then uses a composite data store to refer to the first instance's data store read-only, and refers to a second data store as a writable data store One way this can be used is in creating a test or staging instance from a production instance. At creation, the test instance will look like production, but any changes made to the test instance do not affect production. The test instance can be quickly created from production by cloning only the node store, and not requiring a copy of all the data in the data store. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Assigned] (OAK-5960) Multi blob store support
[ https://issues.apache.org/jira/browse/OAK-5960?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt Ryan reassigned OAK-5960: -- Assignee: Matt Ryan > Multi blob store support > > > Key: OAK-5960 > URL: https://issues.apache.org/jira/browse/OAK-5960 > Project: Jackrabbit Oak > Issue Type: Epic > Components: blob >Reporter: Amit Jain >Assignee: Matt Ryan > > Epic to collect issues and discussion for support for multi blob store which > could potentially support the following scenarios: > * A primary writable blob store and read only secondary blob stores - useful > for cloud cross-geo deployments > * Typed blob store where based on type passed the blobs are written and read > from corresponding blob stores. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Resolved] (OAK-7082) ArrayIndexOutOfBoundsException when upgrading from Oak 1.6
[
https://issues.apache.org/jira/browse/OAK-7082?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Dürig resolved OAK-7082.
Resolution: Fixed
Fixed at http://svn.apache.org/viewvc?rev=1818690=rev
> ArrayIndexOutOfBoundsException when upgrading from Oak 1.6
> --
>
> Key: OAK-7082
> URL: https://issues.apache.org/jira/browse/OAK-7082
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: segment-tar
>Reporter: Michael Dürig
>Assignee: Michael Dürig
>Priority: Critical
> Labels: migration
> Fix For: 1.8
>
>
> When starting on an Oak 1.6 repository and the {{gc.log}} file is present the
> TarMK fails with:
> {noformat}
> java.lang.ArrayIndexOutOfBoundsException: 5
> at
> org.apache.jackrabbit.oak.segment.file.GCJournal$GCJournalEntry.parseString(GCJournal.java:217)
> at
> org.apache.jackrabbit.oak.segment.file.GCJournal$GCJournalEntry.fromString(GCJournal.java:204)
> at
> org.apache.jackrabbit.oak.segment.file.GCJournal.read(GCJournal.java:115)
> at
> org.apache.jackrabbit.oak.segment.file.FileStore$GarbageCollector.compact(FileStore.java:750)
> at
> org.apache.jackrabbit.oak.segment.file.FileStore$GarbageCollector.compactFull(FileStore.java:731)
> at
> org.apache.jackrabbit.oak.segment.file.FileStore.compactFull(FileStore.java:385)
> at
> org.apache.jackrabbit.oak.segment.tool.Compact.run(Compact.java:273)
> at
> org.apache.jackrabbit.oak.run.CompactCommand.execute(CompactCommand.java:72)
> at org.apache.jackrabbit.oak.run.Main.main(Main.java:49)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (OAK-7082) ArrayIndexOutOfBoundsException when upgrading from Oak 1.6
[
https://issues.apache.org/jira/browse/OAK-7082?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16297066#comment-16297066
]
Michael Dürig commented on OAK-7082:
At http://svn.apache.org/viewvc?rev=1818689=rev I fixed the test
expectations of the respective UTs. Those tests are failing now with
{{ArrayIndexOutOfBoundsException}} so I ignored the until we have a fix.
> ArrayIndexOutOfBoundsException when upgrading from Oak 1.6
> --
>
> Key: OAK-7082
> URL: https://issues.apache.org/jira/browse/OAK-7082
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: segment-tar
>Reporter: Michael Dürig
>Assignee: Michael Dürig
>Priority: Critical
> Labels: migration
> Fix For: 1.8
>
>
> When starting on an Oak 1.6 repository and the {{gc.log}} file is present the
> TarMK fails with:
> {noformat}
> java.lang.ArrayIndexOutOfBoundsException: 5
> at
> org.apache.jackrabbit.oak.segment.file.GCJournal$GCJournalEntry.parseString(GCJournal.java:217)
> at
> org.apache.jackrabbit.oak.segment.file.GCJournal$GCJournalEntry.fromString(GCJournal.java:204)
> at
> org.apache.jackrabbit.oak.segment.file.GCJournal.read(GCJournal.java:115)
> at
> org.apache.jackrabbit.oak.segment.file.FileStore$GarbageCollector.compact(FileStore.java:750)
> at
> org.apache.jackrabbit.oak.segment.file.FileStore$GarbageCollector.compactFull(FileStore.java:731)
> at
> org.apache.jackrabbit.oak.segment.file.FileStore.compactFull(FileStore.java:385)
> at
> org.apache.jackrabbit.oak.segment.tool.Compact.run(Compact.java:273)
> at
> org.apache.jackrabbit.oak.run.CompactCommand.execute(CompactCommand.java:72)
> at org.apache.jackrabbit.oak.run.Main.main(Main.java:49)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (OAK-7081) Build Jackrabbit Oak #1109 failed
[ https://issues.apache.org/jira/browse/OAK-7081?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16297064#comment-16297064 ] Hudson commented on OAK-7081: - Previously failing build now is OK. Passed run: [Jackrabbit Oak #1112|https://builds.apache.org/job/Jackrabbit%20Oak/1112/] [console log|https://builds.apache.org/job/Jackrabbit%20Oak/1112/console] > Build Jackrabbit Oak #1109 failed > - > > Key: OAK-7081 > URL: https://issues.apache.org/jira/browse/OAK-7081 > Project: Jackrabbit Oak > Issue Type: Bug > Components: continuous integration >Reporter: Hudson > > No description is provided > The build Jackrabbit Oak #1109 has failed. > First failed run: [Jackrabbit Oak > #1109|https://builds.apache.org/job/Jackrabbit%20Oak/1109/] [console > log|https://builds.apache.org/job/Jackrabbit%20Oak/1109/console] -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (OAK-7082) ArrayIndexOutOfBoundsException when upgrading from Oak 1.6
[
https://issues.apache.org/jira/browse/OAK-7082?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16297057#comment-16297057
]
Michael Dürig commented on OAK-7082:
This is caused by {{GCJournal}} only taking care of the {{gc full generation}}
field that was added to the {{gc.log}} in 1.8 but forgets to also take care of
{{root id}}, which was also added with Oak 1.8.
> ArrayIndexOutOfBoundsException when upgrading from Oak 1.6
> --
>
> Key: OAK-7082
> URL: https://issues.apache.org/jira/browse/OAK-7082
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: segment-tar
>Reporter: Michael Dürig
>Assignee: Michael Dürig
>Priority: Critical
> Labels: migration
> Fix For: 1.8
>
>
> When starting on an Oak 1.6 repository and the {{gc.log}} file is present the
> TarMK fails with:
> {noformat}
> java.lang.ArrayIndexOutOfBoundsException: 5
> at
> org.apache.jackrabbit.oak.segment.file.GCJournal$GCJournalEntry.parseString(GCJournal.java:217)
> at
> org.apache.jackrabbit.oak.segment.file.GCJournal$GCJournalEntry.fromString(GCJournal.java:204)
> at
> org.apache.jackrabbit.oak.segment.file.GCJournal.read(GCJournal.java:115)
> at
> org.apache.jackrabbit.oak.segment.file.FileStore$GarbageCollector.compact(FileStore.java:750)
> at
> org.apache.jackrabbit.oak.segment.file.FileStore$GarbageCollector.compactFull(FileStore.java:731)
> at
> org.apache.jackrabbit.oak.segment.file.FileStore.compactFull(FileStore.java:385)
> at
> org.apache.jackrabbit.oak.segment.tool.Compact.run(Compact.java:273)
> at
> org.apache.jackrabbit.oak.run.CompactCommand.execute(CompactCommand.java:72)
> at org.apache.jackrabbit.oak.run.Main.main(Main.java:49)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Created] (OAK-7082) ArrayIndexOutOfBoundsException when upgrading from Oak 1.6
Michael Dürig created OAK-7082:
--
Summary: ArrayIndexOutOfBoundsException when upgrading from Oak 1.6
Key: OAK-7082
URL: https://issues.apache.org/jira/browse/OAK-7082
Project: Jackrabbit Oak
Issue Type: Bug
Components: segment-tar
Reporter: Michael Dürig
Assignee: Michael Dürig
Priority: Critical
Fix For: 1.8
When starting on an Oak 1.6 repository and the {{gc.log}} file is present the
TarMK fails with:
{noformat}
java.lang.ArrayIndexOutOfBoundsException: 5
at
org.apache.jackrabbit.oak.segment.file.GCJournal$GCJournalEntry.parseString(GCJournal.java:217)
at
org.apache.jackrabbit.oak.segment.file.GCJournal$GCJournalEntry.fromString(GCJournal.java:204)
at
org.apache.jackrabbit.oak.segment.file.GCJournal.read(GCJournal.java:115)
at
org.apache.jackrabbit.oak.segment.file.FileStore$GarbageCollector.compact(FileStore.java:750)
at
org.apache.jackrabbit.oak.segment.file.FileStore$GarbageCollector.compactFull(FileStore.java:731)
at
org.apache.jackrabbit.oak.segment.file.FileStore.compactFull(FileStore.java:385)
at org.apache.jackrabbit.oak.segment.tool.Compact.run(Compact.java:273)
at
org.apache.jackrabbit.oak.run.CompactCommand.execute(CompactCommand.java:72)
at org.apache.jackrabbit.oak.run.Main.main(Main.java:49)
{noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (OAK-7081) Build Jackrabbit Oak #1109 failed
[ https://issues.apache.org/jira/browse/OAK-7081?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16296983#comment-16296983 ] Hudson commented on OAK-7081: - Previously failing build now is OK. Passed run: [Jackrabbit Oak #|https://builds.apache.org/job/Jackrabbit%20Oak//] [console log|https://builds.apache.org/job/Jackrabbit%20Oak//console] > Build Jackrabbit Oak #1109 failed > - > > Key: OAK-7081 > URL: https://issues.apache.org/jira/browse/OAK-7081 > Project: Jackrabbit Oak > Issue Type: Bug > Components: continuous integration >Reporter: Hudson > > No description is provided > The build Jackrabbit Oak #1109 has failed. > First failed run: [Jackrabbit Oak > #1109|https://builds.apache.org/job/Jackrabbit%20Oak/1109/] [console > log|https://builds.apache.org/job/Jackrabbit%20Oak/1109/console] -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Resolved] (OAK-6606) Move BulkTransferBenchmark to oak-benchmarks module
[
https://issues.apache.org/jira/browse/OAK-6606?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrei Dulceanu resolved OAK-6606.
--
Resolution: Fixed
Refactored {{BulkTransferBenchmark}} to adhere to constraints imposed by
{{ScalabilityBenchmark}} and created {{ScalabilityStandbySuite}} which takes
care of setting up cold standby server and client.
Fixed at r1818682.
> Move BulkTransferBenchmark to oak-benchmarks module
> ---
>
> Key: OAK-6606
> URL: https://issues.apache.org/jira/browse/OAK-6606
> Project: Jackrabbit Oak
> Issue Type: Task
> Components: segment-tar
>Reporter: Andrei Dulceanu
>Assignee: Andrei Dulceanu
>Priority: Minor
> Labels: cold-standby
> Fix For: 1.8
>
>
> {{BulkTransferBenchmark}} should be moved from {{oak-segment-tar}} to
> {{oak-benchmarks}} to allow standard run of this cold standby related
> benchmark.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Resolved] (OAK-7077) Documentation for RDBDocumentStore statistics
[ https://issues.apache.org/jira/browse/OAK-7077?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Julian Reschke resolved OAK-7077. - Resolution: Fixed Fix Version/s: 1.7.13 trunk: [r1818678|http://svn.apache.org/r1818678] > Documentation for RDBDocumentStore statistics > - > > Key: OAK-7077 > URL: https://issues.apache.org/jira/browse/OAK-7077 > Project: Jackrabbit Oak > Issue Type: Task >Reporter: Julian Reschke >Assignee: Julian Reschke >Priority: Minor > Fix For: 1.7.13, 1.8 > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (OAK-7081) Build Jackrabbit Oak #1109 failed
[ https://issues.apache.org/jira/browse/OAK-7081?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16296880#comment-16296880 ] Hudson commented on OAK-7081: - Previously failing build now is OK. Passed run: [Jackrabbit Oak #1110|https://builds.apache.org/job/Jackrabbit%20Oak/1110/] [console log|https://builds.apache.org/job/Jackrabbit%20Oak/1110/console] > Build Jackrabbit Oak #1109 failed > - > > Key: OAK-7081 > URL: https://issues.apache.org/jira/browse/OAK-7081 > Project: Jackrabbit Oak > Issue Type: Bug > Components: continuous integration >Reporter: Hudson > > No description is provided > The build Jackrabbit Oak #1109 has failed. > First failed run: [Jackrabbit Oak > #1109|https://builds.apache.org/job/Jackrabbit%20Oak/1109/] [console > log|https://builds.apache.org/job/Jackrabbit%20Oak/1109/console] -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Comment Edited] (OAK-6965) RDBDocumentStore: allow schema evolution part 5: add rows for performant VGC
[ https://issues.apache.org/jira/browse/OAK-6965?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16280583#comment-16280583 ] Julian Reschke edited comment on OAK-6965 at 12/19/17 2:11 PM: --- trunk: [r1818672|http://svn.apache.org/r1818672] [r1817802|http://svn.apache.org/r1817802] [r1817311|http://svn.apache.org/r1817311] was (Author: reschke): trunk: [r1817802|http://svn.apache.org/r1817802] [r1817311|http://svn.apache.org/r1817311] > RDBDocumentStore: allow schema evolution part 5: add rows for performant VGC > > > Key: OAK-6965 > URL: https://issues.apache.org/jira/browse/OAK-6965 > Project: Jackrabbit Oak > Issue Type: Task > Components: rdbmk >Reporter: Julian Reschke >Assignee: Julian Reschke > Fix For: 1.7.13, 1.8 > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (OAK-7077) Documentation for RDBDocumentStore statistics
[ https://issues.apache.org/jira/browse/OAK-7077?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Julian Reschke updated OAK-7077: Summary: Documentation for RDBDocumentStore statistics (was: Document RDBDocumentStore statistics) > Documentation for RDBDocumentStore statistics > - > > Key: OAK-7077 > URL: https://issues.apache.org/jira/browse/OAK-7077 > Project: Jackrabbit Oak > Issue Type: Task >Reporter: Julian Reschke >Assignee: Julian Reschke >Priority: Minor > Fix For: 1.8 > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (OAK-7081) Build Jackrabbit Oak #1109 failed
Hudson created OAK-7081: --- Summary: Build Jackrabbit Oak #1109 failed Key: OAK-7081 URL: https://issues.apache.org/jira/browse/OAK-7081 Project: Jackrabbit Oak Issue Type: Bug Components: continuous integration Reporter: Hudson No description is provided The build Jackrabbit Oak #1109 has failed. First failed run: [Jackrabbit Oak #1109|https://builds.apache.org/job/Jackrabbit%20Oak/1109/] [console log|https://builds.apache.org/job/Jackrabbit%20Oak/1109/console] -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (OAK-7080) Segment-Tar-Cold fixture should have options for secure communication and one shot runs
[
https://issues.apache.org/jira/browse/OAK-7080?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrei Dulceanu updated OAK-7080:
-
Fix Version/s: 1.7.13
> Segment-Tar-Cold fixture should have options for secure communication and one
> shot runs
> ---
>
> Key: OAK-7080
> URL: https://issues.apache.org/jira/browse/OAK-7080
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: benchmarks, segment-tar, tarmk-standby
>Reporter: Andrei Dulceanu
>Assignee: Andrei Dulceanu
>Priority: Minor
> Labels: cold-standby
> Fix For: 1.7.13, 1.8.
>
> Attachments: OAK-7080.patch
>
>
> The newly introduced {{Segment-Tar-Cold}} fixture should support secure
> communication between primary and standby via a {{--secure}} option.
> Moreover, the current implementation allows only for continuous sync between
> primary and standby. It should be possible to allow a "one-shot run" of the
> sync to easily measure and compare specific metrics ({{--oneShotRun}} option).
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (OAK-6606) Move BulkTransferBenchmark to oak-benchmarks module
[
https://issues.apache.org/jira/browse/OAK-6606?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16296692#comment-16296692
]
Andrei Dulceanu commented on OAK-6606:
--
Moved {{BulkTransferBenchmark}} to {{oak-benchmarks}} and moved around existing
scalability search-related benchmarks at r1818664.
> Move BulkTransferBenchmark to oak-benchmarks module
> ---
>
> Key: OAK-6606
> URL: https://issues.apache.org/jira/browse/OAK-6606
> Project: Jackrabbit Oak
> Issue Type: Task
> Components: segment-tar
>Reporter: Andrei Dulceanu
>Assignee: Andrei Dulceanu
>Priority: Minor
> Labels: cold-standby
> Fix For: 1.8
>
>
> {{BulkTransferBenchmark}} should be moved from {{oak-segment-tar}} to
> {{oak-benchmarks}} to allow standard run of this cold standby related
> benchmark.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (OAK-7070) rep:excerpt selector broken as regression of OAK-6750
[
https://issues.apache.org/jira/browse/OAK-7070?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vikas Saurabh updated OAK-7070:
---
Fix Version/s: 1.6.8
1.4.19
> rep:excerpt selector broken as regression of OAK-6750
> -
>
> Key: OAK-7070
> URL: https://issues.apache.org/jira/browse/OAK-7070
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: lucene
>Affects Versions: 1.6.7, 1.8
>Reporter: Dirk Rudolph
>Assignee: Vikas Saurabh
> Labels: excerpt
> Fix For: 1.7.13, 1.4.19, 1.6.8, 1.8
>
>
> The change made here:
> https://github.com/apache/jackrabbit-oak/commit/00c94b71293abcae6d76bb162c3f55c7d09b702e#diff-d4bdf443c61f24b634f33aab607e2114
> breaks the logic in line 676:
> {{else if (oakPropertyName.equals(QueryConstants.REP_EXCERPT + "("))}}
> This statement doesn't make much sense considering a query like {{select
> \[rep:excerpt] from \[test:Page] as page where contains(\*, 'term\*')}} or
> even {{select \[rep:excerpt(text)] from \[test:Page] as page where
> contains(page.\[text], 'term\*')}}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Comment Edited] (OAK-7070) rep:excerpt selector broken as regression of OAK-6750
[
https://issues.apache.org/jira/browse/OAK-7070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16296590#comment-16296590
]
Vikas Saurabh edited comment on OAK-7070 at 12/19/17 11:42 AM:
---
Fixed in trunk at [r1818645|https://svn.apache.org/r1818645], 1.6 at
[r1818652|https://svn.apache.org/r1818652] and on 1.4 at
[r1818657|https://svn.apache.org/r1818657].
was (Author: catholicon):
Fixed in trunk at [r1818645|https://svn.apache.org/r1818645]. Backports to 1.6
and 1.4 in a bit.
> rep:excerpt selector broken as regression of OAK-6750
> -
>
> Key: OAK-7070
> URL: https://issues.apache.org/jira/browse/OAK-7070
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: lucene
>Affects Versions: 1.6.7, 1.8
>Reporter: Dirk Rudolph
>Assignee: Vikas Saurabh
> Labels: excerpt
> Fix For: 1.7.13, 1.4.19, 1.6.8, 1.8
>
>
> The change made here:
> https://github.com/apache/jackrabbit-oak/commit/00c94b71293abcae6d76bb162c3f55c7d09b702e#diff-d4bdf443c61f24b634f33aab607e2114
> breaks the logic in line 676:
> {{else if (oakPropertyName.equals(QueryConstants.REP_EXCERPT + "("))}}
> This statement doesn't make much sense considering a query like {{select
> \[rep:excerpt] from \[test:Page] as page where contains(\*, 'term\*')}} or
> even {{select \[rep:excerpt(text)] from \[test:Page] as page where
> contains(page.\[text], 'term\*')}}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Resolved] (OAK-7080) Segment-Tar-Cold fixture should have options for secure communication and one shot runs
[
https://issues.apache.org/jira/browse/OAK-7080?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrei Dulceanu resolved OAK-7080.
--
Resolution: Fixed
Fixed at r1818653.
> Segment-Tar-Cold fixture should have options for secure communication and one
> shot runs
> ---
>
> Key: OAK-7080
> URL: https://issues.apache.org/jira/browse/OAK-7080
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: benchmarks, segment-tar, tarmk-standby
>Reporter: Andrei Dulceanu
>Assignee: Andrei Dulceanu
>Priority: Minor
> Labels: cold-standby
> Fix For: 1.8.
>
> Attachments: OAK-7080.patch
>
>
> The newly introduced {{Segment-Tar-Cold}} fixture should support secure
> communication between primary and standby via a {{--secure}} option.
> Moreover, the current implementation allows only for continuous sync between
> primary and standby. It should be possible to allow a "one-shot run" of the
> sync to easily measure and compare specific metrics ({{--oneShotRun}} option).
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (OAK-7080) Segment-Tar-Cold fixture should have options for secure communication and one shot runs
[
https://issues.apache.org/jira/browse/OAK-7080?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrei Dulceanu updated OAK-7080:
-
Attachment: OAK-7080.patch
> Segment-Tar-Cold fixture should have options for secure communication and one
> shot runs
> ---
>
> Key: OAK-7080
> URL: https://issues.apache.org/jira/browse/OAK-7080
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: benchmarks, segment-tar, tarmk-standby
>Reporter: Andrei Dulceanu
>Assignee: Andrei Dulceanu
>Priority: Minor
> Labels: cold-standby
> Fix For: 1.8.
>
> Attachments: OAK-7080.patch
>
>
> The newly introduced {{Segment-Tar-Cold}} fixture should support secure
> communication between primary and standby via a {{--secure}} option.
> Moreover, the current implementation allows only for continuous sync between
> primary and standby. It should be possible to allow a "one-shot run" of the
> sync to easily measure and compare specific metrics ({{--oneShotRun}} option).
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Resolved] (OAK-7070) rep:excerpt selector broken as regression of OAK-6750
[
https://issues.apache.org/jira/browse/OAK-7070?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vikas Saurabh resolved OAK-7070.
Resolution: Fixed
Fix Version/s: 1.8
1.7.13
Fixed in trunk at [r1818645|https://svn.apache.org/r1818645]. Backports to 1.6
and 1.4 in a bit.
> rep:excerpt selector broken as regression of OAK-6750
> -
>
> Key: OAK-7070
> URL: https://issues.apache.org/jira/browse/OAK-7070
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: lucene
>Affects Versions: 1.6.7, 1.8
>Reporter: Dirk Rudolph
>Assignee: Vikas Saurabh
> Labels: excerpt
> Fix For: 1.7.13, 1.8
>
>
> The change made here:
> https://github.com/apache/jackrabbit-oak/commit/00c94b71293abcae6d76bb162c3f55c7d09b702e#diff-d4bdf443c61f24b634f33aab607e2114
> breaks the logic in line 676:
> {{else if (oakPropertyName.equals(QueryConstants.REP_EXCERPT + "("))}}
> This statement doesn't make much sense considering a query like {{select
> \[rep:excerpt] from \[test:Page] as page where contains(\*, 'term\*')}} or
> even {{select \[rep:excerpt(text)] from \[test:Page] as page where
> contains(page.\[text], 'term\*')}}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Comment Edited] (OAK-7070) rep:excerpt selector broken as regression of OAK-6750
[
https://issues.apache.org/jira/browse/OAK-7070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16296560#comment-16296560
]
Dirk Rudolph edited comment on OAK-7070 at 12/19/17 9:55 AM:
-
Thanks [~catholicon] Your understanding is correct. And yes, I will move the
comment to OAK-6597.
To give a bit more context: I'm working currently on an AEM 6.3 project
implementing fulltext search and we recently upgraded to 1.6.7 to make use of
the changes in OAK-6750. Though our requirements also ask for excerpts and
that's why I investigated in OAK-6597 as well and asked there for a backport it
to 1.6. As this is blocking OAK-6597 and if we agree on making OAK-6597
available in 1.6 I would still like to backport it. The risk should be minimal
and afaik I applied those changes to my for of 1.6 without any problems. Don't
doing so opens the risk for us to use an unoffical port of oak for our project
- or rejecting some of the customers requirements. This also comes together
with OAK-7078 and OAK-7071.
was (Author: diru):
Thanks [~catholicon] Your understanding is correct. And yes, I will move the
comment to OAK-6597.
To give a bit more context: I'm working currently on an AEM 6.3 project
implementing fulltext search and we recently upgraded to 1.6.7 to make use of
the changes in OAK-6750. Though our requirements also ask for excerpts and
that's why I investigated in OAK-6597 as well and ask there for a backport to
1.6 too. As this is blocking OAK-6597 and if we agree on making OAK-6597
available in 1.6 I would still like to backport it. The risk should be minimal
and afaik I applied those changes to my for of 1.6 without any problems. Don't
doing so opens the risk for us to use an unoffical port of oak for our project
- or rejecting some of the customers requirements. This also comes together
with OAK-7078 and OAK-7071.
> rep:excerpt selector broken as regression of OAK-6750
> -
>
> Key: OAK-7070
> URL: https://issues.apache.org/jira/browse/OAK-7070
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: lucene
>Affects Versions: 1.6.7, 1.8
>Reporter: Dirk Rudolph
>Assignee: Vikas Saurabh
> Labels: excerpt
>
> The change made here:
> https://github.com/apache/jackrabbit-oak/commit/00c94b71293abcae6d76bb162c3f55c7d09b702e#diff-d4bdf443c61f24b634f33aab607e2114
> breaks the logic in line 676:
> {{else if (oakPropertyName.equals(QueryConstants.REP_EXCERPT + "("))}}
> This statement doesn't make much sense considering a query like {{select
> \[rep:excerpt] from \[test:Page] as page where contains(\*, 'term\*')}} or
> even {{select \[rep:excerpt(text)] from \[test:Page] as page where
> contains(page.\[text], 'term\*')}}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Issue Comment Deleted] (OAK-7070) rep:excerpt selector broken as regression of OAK-6750
[
https://issues.apache.org/jira/browse/OAK-7070?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dirk Rudolph updated OAK-7070:
--
Comment: was deleted
(was: There is still the risk, that duplication appear in the excerpt because
there is a highlighting hit in {{:fulltext}} and one for example in
{{full:bar}}. To prevent that, it probably makes sense to first do the
highlighting on {{:fulltext}} fields when analyzeFulltext is enabled and only
if that hasn't been success full we fallback to the logic of highlighting
{{full:}} fields. wdyt?)
> rep:excerpt selector broken as regression of OAK-6750
> -
>
> Key: OAK-7070
> URL: https://issues.apache.org/jira/browse/OAK-7070
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: lucene
>Affects Versions: 1.6.7, 1.8
>Reporter: Dirk Rudolph
>Assignee: Vikas Saurabh
> Labels: excerpt
>
> The change made here:
> https://github.com/apache/jackrabbit-oak/commit/00c94b71293abcae6d76bb162c3f55c7d09b702e#diff-d4bdf443c61f24b634f33aab607e2114
> breaks the logic in line 676:
> {{else if (oakPropertyName.equals(QueryConstants.REP_EXCERPT + "("))}}
> This statement doesn't make much sense considering a query like {{select
> \[rep:excerpt] from \[test:Page] as page where contains(\*, 'term\*')}} or
> even {{select \[rep:excerpt(text)] from \[test:Page] as page where
> contains(page.\[text], 'term\*')}}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (OAK-6597) rep:excerpt not working for content indexed by aggregation in lucene
[
https://issues.apache.org/jira/browse/OAK-6597?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16296561#comment-16296561
]
Dirk Rudolph commented on OAK-6597:
---
There is still the risk, that duplication appear in the excerpt because there
is a highlighting hit in :fulltext and one for example in full:bar. To prevent
that, it probably makes sense to first do the highlighting on :fulltext fields
when analyzeFulltext is enabled and only if that hasn't been success full we
fallback to the logic of highlighting full: fields. wdyt?
> rep:excerpt not working for content indexed by aggregation in lucene
>
>
> Key: OAK-6597
> URL: https://issues.apache.org/jira/browse/OAK-6597
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: lucene
>Affects Versions: 1.6.1, 1.7.6, 1.8
>Reporter: Dirk Rudolph
>Assignee: Chetan Mehrotra
> Labels: excerpt
> Fix For: 1.10
>
> Attachments: excerpt-with-aggregation-test.patch
>
>
> I mentioned that properties that got indexed due to an aggregation are not
> considered for excerpts (highlighting) as they are not indexed as stored
> fields.
> See the attached patch that implements a test for excerpts in
> {{LuceneIndexAggregationTest2}}.
> It creates the following structure:
> {code}
> /content/foo [test:Page]
> + bar (String)
> - jcr:content [test:PageContent]
> + bar (String)
> {code}
> where both strings (the _bar_ property at _foo_ and the _bar_ property at
> _jcr:content_) contain different text.
> Afterwards it queries for 2 terms ("tinc*" and "aliq*") that either exist in
> _/content/foo/bar_ or _/content/foo/jcr:content/bar_ but not in both. For the
> former one the excerpt is properly provided for the later one it isn't.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Comment Edited] (OAK-6597) rep:excerpt not working for content indexed by aggregation in lucene
[
https://issues.apache.org/jira/browse/OAK-6597?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16296561#comment-16296561
]
Dirk Rudolph edited comment on OAK-6597 at 12/19/17 9:51 AM:
-
There is still the risk, that duplications appear in the excerpt because there
is a highlighting hit in :fulltext and one for example in full:bar. To prevent
that, it probably makes sense to first do the highlighting on :fulltext fields
when analyzeFulltext is enabled and only if that hasn't been successful we
fallback to the logic of highlighting full: fields. wdyt?
was (Author: diru):
There is still the risk, that duplications appear in the excerpt because there
is a highlighting hit in :fulltext and one for example in full:bar. To prevent
that, it probably makes sense to first do the highlighting on :fulltext fields
when analyzeFulltext is enabled and only if that hasn't been success full we
fallback to the logic of highlighting full: fields. wdyt?
> rep:excerpt not working for content indexed by aggregation in lucene
>
>
> Key: OAK-6597
> URL: https://issues.apache.org/jira/browse/OAK-6597
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: lucene
>Affects Versions: 1.6.1, 1.7.6, 1.8
>Reporter: Dirk Rudolph
>Assignee: Chetan Mehrotra
> Labels: excerpt
> Fix For: 1.10
>
> Attachments: excerpt-with-aggregation-test.patch
>
>
> I mentioned that properties that got indexed due to an aggregation are not
> considered for excerpts (highlighting) as they are not indexed as stored
> fields.
> See the attached patch that implements a test for excerpts in
> {{LuceneIndexAggregationTest2}}.
> It creates the following structure:
> {code}
> /content/foo [test:Page]
> + bar (String)
> - jcr:content [test:PageContent]
> + bar (String)
> {code}
> where both strings (the _bar_ property at _foo_ and the _bar_ property at
> _jcr:content_) contain different text.
> Afterwards it queries for 2 terms ("tinc*" and "aliq*") that either exist in
> _/content/foo/bar_ or _/content/foo/jcr:content/bar_ but not in both. For the
> former one the excerpt is properly provided for the later one it isn't.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Comment Edited] (OAK-6597) rep:excerpt not working for content indexed by aggregation in lucene
[
https://issues.apache.org/jira/browse/OAK-6597?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16296561#comment-16296561
]
Dirk Rudolph edited comment on OAK-6597 at 12/19/17 9:51 AM:
-
There is still the risk, that duplications appear in the excerpt because there
is a highlighting hit in :fulltext and one for example in full:bar. To prevent
that, it probably makes sense to first do the highlighting on :fulltext fields
when analyzeFulltext is enabled and only if that hasn't been success full we
fallback to the logic of highlighting full: fields. wdyt?
was (Author: diru):
There is still the risk, that duplication appear in the excerpt because there
is a highlighting hit in :fulltext and one for example in full:bar. To prevent
that, it probably makes sense to first do the highlighting on :fulltext fields
when analyzeFulltext is enabled and only if that hasn't been success full we
fallback to the logic of highlighting full: fields. wdyt?
> rep:excerpt not working for content indexed by aggregation in lucene
>
>
> Key: OAK-6597
> URL: https://issues.apache.org/jira/browse/OAK-6597
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: lucene
>Affects Versions: 1.6.1, 1.7.6, 1.8
>Reporter: Dirk Rudolph
>Assignee: Chetan Mehrotra
> Labels: excerpt
> Fix For: 1.10
>
> Attachments: excerpt-with-aggregation-test.patch
>
>
> I mentioned that properties that got indexed due to an aggregation are not
> considered for excerpts (highlighting) as they are not indexed as stored
> fields.
> See the attached patch that implements a test for excerpts in
> {{LuceneIndexAggregationTest2}}.
> It creates the following structure:
> {code}
> /content/foo [test:Page]
> + bar (String)
> - jcr:content [test:PageContent]
> + bar (String)
> {code}
> where both strings (the _bar_ property at _foo_ and the _bar_ property at
> _jcr:content_) contain different text.
> Afterwards it queries for 2 terms ("tinc*" and "aliq*") that either exist in
> _/content/foo/bar_ or _/content/foo/jcr:content/bar_ but not in both. For the
> former one the excerpt is properly provided for the later one it isn't.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (OAK-7070) rep:excerpt selector broken as regression of OAK-6750
[
https://issues.apache.org/jira/browse/OAK-7070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16296560#comment-16296560
]
Dirk Rudolph commented on OAK-7070:
---
Thanks [~catholicon] Your understanding is correct. And yes, I will move the
comment to OAK-6597.
To give a bit more context: I'm working currently on an AEM 6.3 project
implementing fulltext search and we recently upgraded to 1.6.7 to make use of
the changes in OAK-6750. Though our requirements also ask for excerpts and
that's why I investigated in OAK-6597 as well and ask there for a backport to
1.6 too. As this is blocking OAK-6597 and if we agree on making OAK-6597
available in 1.6 I would still like to backport it. The risk should be minimal
and afaik I applied those changes to my for of 1.6 without any problems. Don't
doing so opens the risk for us to use an unoffical port of oak for our project
- or rejecting some of the customers requirements. This also comes together
with OAK-7078 and OAK-7071.
> rep:excerpt selector broken as regression of OAK-6750
> -
>
> Key: OAK-7070
> URL: https://issues.apache.org/jira/browse/OAK-7070
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: lucene
>Affects Versions: 1.6.7, 1.8
>Reporter: Dirk Rudolph
>Assignee: Vikas Saurabh
> Labels: excerpt
>
> The change made here:
> https://github.com/apache/jackrabbit-oak/commit/00c94b71293abcae6d76bb162c3f55c7d09b702e#diff-d4bdf443c61f24b634f33aab607e2114
> breaks the logic in line 676:
> {{else if (oakPropertyName.equals(QueryConstants.REP_EXCERPT + "("))}}
> This statement doesn't make much sense considering a query like {{select
> \[rep:excerpt] from \[test:Page] as page where contains(\*, 'term\*')}} or
> even {{select \[rep:excerpt(text)] from \[test:Page] as page where
> contains(page.\[text], 'term\*')}}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (OAK-7070) rep:excerpt selector broken as regression of OAK-6750
[
https://issues.apache.org/jira/browse/OAK-7070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16296541#comment-16296541
]
Vikas Saurabh commented on OAK-7070:
[~diru], afaiu, the reason you opened the issue is that for a query like -
{{SELECT [jcr:path],[rep:excerpt] from [nt:base] WHERE CONTAINS(\*, 'ipsum')}},
doing {{resultRow.getValue("rep:excerpt")}} doesn't work anymore (while
{{getValue("rep:ecerpt(.)")}} and {{getValue("rep:excerpt(bar)")}} still works.
Is that understanding correct? I'd apply you patch to trunk in a bit... but,
I'm not sure about the bug being "Major" and if we should backport it.
{quote}
There is still the risk, that duplication appear in the excerpt because there
is a highlighting hit in :fulltext and one for example in full:bar. To prevent
that, it probably makes sense to first do the highlighting on :fulltext fields
when analyzeFulltext is enabled and only if that hasn't been success full we
fallback to the logic of highlighting full: fields. wdyt?
{quote}
I'm guessing you meant to put that comment on OAK-6597 instead!?
> rep:excerpt selector broken as regression of OAK-6750
> -
>
> Key: OAK-7070
> URL: https://issues.apache.org/jira/browse/OAK-7070
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: lucene
>Affects Versions: 1.6.7, 1.8
>Reporter: Dirk Rudolph
>Assignee: Vikas Saurabh
> Labels: excerpt
>
> The change made here:
> https://github.com/apache/jackrabbit-oak/commit/00c94b71293abcae6d76bb162c3f55c7d09b702e#diff-d4bdf443c61f24b634f33aab607e2114
> breaks the logic in line 676:
> {{else if (oakPropertyName.equals(QueryConstants.REP_EXCERPT + "("))}}
> This statement doesn't make much sense considering a query like {{select
> \[rep:excerpt] from \[test:Page] as page where contains(\*, 'term\*')}} or
> even {{select \[rep:excerpt(text)] from \[test:Page] as page where
> contains(page.\[text], 'term\*')}}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (OAK-7074) Ensure that all Documents are read with document order traversal indexing
[ https://issues.apache.org/jira/browse/OAK-7074?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16296527#comment-16296527 ] Vikas Saurabh commented on OAK-7074: Also, in the same offline discussion - for problem#2 in description (document _may_ not show up at all), it seemed that we could also run another query with _modified > checkpointTime. Since we're now removing the dups - this should still be ok. (note, a document might move because of a reverted commit - so, we must have all the documents that existed at checkpoint). The question about some document getting missed due to mongo internal process is, imo, would require some more guarantees from mongo (as [~chetanm] indicated in #2 in his last comment). > Ensure that all Documents are read with document order traversal indexing > - > > Key: OAK-7074 > URL: https://issues.apache.org/jira/browse/OAK-7074 > Project: Jackrabbit Oak > Issue Type: Improvement > Components: mongomk, run >Reporter: Chetan Mehrotra >Assignee: Chetan Mehrotra > Fix For: 1.8 > > > With OAK-6353 support was added for document order traversal indexing. In > this mode we open a DB cursor and try to read all documents from it using > document order traversal. Such a cursor may remain open for long time (2-4 > hrs) and its possible that document may get reordered by the Mongo storage > engine. This would result in 2 aspects to be thought about > # Duplicate documents - Same document may appear more than once in result set > # Possibly missed document - It may be a possibility that a document got > moved and missed becoming part of cursor. > Both these aspects would need to be handled -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (OAK-7080) Segment-Tar-Cold fixture should have options for secure communication and one shot runs
[
https://issues.apache.org/jira/browse/OAK-7080?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrei Dulceanu updated OAK-7080:
-
Description: The newly introduced {{Segment-Tar-Cold}} fixture should
support secure communication between primary and standby via a {{--secure}}
option. Moreover, the current implementation allows only for continuous sync
between primary and standby. It should be possible to allow a "one-shot run" of
the sync to easily measure and compare specific metrics ({{--oneShotRun}}
option). (was: The newly introduced {{Segment-Tar-Cold}} fixture should
support secure communication between primary and standby via a {{--secure}}
option. Moreover, the current implementation allows only for continuous sync
between primary and standby. It should be possible to allow a "one-shot run" of
the sync to easily measure and compare specific metrics.)
> Segment-Tar-Cold fixture should have options for secure communication and one
> shot runs
> ---
>
> Key: OAK-7080
> URL: https://issues.apache.org/jira/browse/OAK-7080
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: benchmarks, segment-tar, tarmk-standby
>Reporter: Andrei Dulceanu
>Assignee: Andrei Dulceanu
>Priority: Minor
> Labels: cold-standby
> Fix For: 1.8.
>
>
> The newly introduced {{Segment-Tar-Cold}} fixture should support secure
> communication between primary and standby via a {{--secure}} option.
> Moreover, the current implementation allows only for continuous sync between
> primary and standby. It should be possible to allow a "one-shot run" of the
> sync to easily measure and compare specific metrics ({{--oneShotRun}} option).
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (OAK-7080) Segment-Tar-Cold fixture should have options for secure communication and one shot runs
[
https://issues.apache.org/jira/browse/OAK-7080?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrei Dulceanu updated OAK-7080:
-
Description: The newly introduced {{Segment-Tar-Cold}} fixture should
support secure communication between primary and standby via a {{--secure}}
option. Moreover, the current implementation allows only for continuous sync
between primary and standby. It should be possible to allow a "one-shot run" of
the sync to easily measure and compare specific metrics. (was: The newly
introduced {{Segment-Tar-Cold}} fixture should support secure communication
between primary and standby via a {{--secure}} option. Moreover, the current
implementation allows only for continuous sync between primary and standby. It
should be possible to allow a "one-shot run" of the sync to measure easily
measure and compare specific metrics.)
> Segment-Tar-Cold fixture should have options for secure communication and one
> shot runs
> ---
>
> Key: OAK-7080
> URL: https://issues.apache.org/jira/browse/OAK-7080
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: benchmarks, segment-tar, tarmk-standby
>Reporter: Andrei Dulceanu
>Assignee: Andrei Dulceanu
>Priority: Minor
> Labels: cold-standby
> Fix For: 1.8.
>
>
> The newly introduced {{Segment-Tar-Cold}} fixture should support secure
> communication between primary and standby via a {{--secure}} option.
> Moreover, the current implementation allows only for continuous sync between
> primary and standby. It should be possible to allow a "one-shot run" of the
> sync to easily measure and compare specific metrics.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Created] (OAK-7080) Segment-Tar-Cold fixture should have options for secure communication and one shot runs
Andrei Dulceanu created OAK-7080:
Summary: Segment-Tar-Cold fixture should have options for secure
communication and one shot runs
Key: OAK-7080
URL: https://issues.apache.org/jira/browse/OAK-7080
Project: Jackrabbit Oak
Issue Type: Improvement
Components: benchmarks, segment-tar, tarmk-standby
Reporter: Andrei Dulceanu
Assignee: Andrei Dulceanu
Priority: Minor
Fix For: 1.8.
The newly introduced {{Segment-Tar-Cold}} fixture should support secure
communication between primary and standby via a {{--secure}} option. Moreover,
the current implementation allows only for continuous sync between primary and
standby. It should be possible to allow a "one-shot run" of the sync to measure
easily measure and compare specific metrics.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
