Re: OSGi configuration lookup
On Tue, Jun 30, 2015 at 5:52 AM, Chetan Mehrotra chetan.mehro...@gmail.com wrote: ...if we need to do that for Segment also then we should use a similar namespacing... And that should IMO be done with a utility class instead of duplicating code. -Bertrand
Re: OSGi configuration lookup
I attached a patch in OAK-3048. Can someone review? 2015-06-30 9:02 GMT+02:00 Francesco Mari mari.france...@gmail.com: I created OAK-3048 to track the implementation of the framework-first-component-next strategy. I will attach a patch to this issue shortly. I also opened OAK-3049 as a reminder to address the uniformity issue when reading configuration. 2015-06-30 8:35 GMT+02:00 Chetan Mehrotra chetan.mehro...@gmail.com: On Tue, Jun 30, 2015 at 12:00 PM, Francesco Mari mari.france...@gmail.com wrote: I suggest to fix OAK-3022 maintaining exactly the same behaviour, and without changing the SegmentNodeStoreService Makes sense. They are two different issue Chetan Mehrotra
RE: S3DataStore leverage Cross-Region Replication
Yes Michael. It would be in OAK's BlobStore Thanks, -shashank -Original Message- From: Michael Marth [mailto:mma...@adobe.com] Sent: Tuesday, June 30, 2015 4:50 PM To: oak-dev@jackrabbit.apache.org Subject: Re: S3DataStore leverage Cross-Region Replication Shashank, In case we think it’s needed to implement multiple chained S3 DSs then I think we should model it after Jackrabbit’s Multidatastore which allows arbitrary DS implementations to be chained: http://jackrabbit.510166.n4.nabble.com/MultiDataStore-td4655772.html Michael On 30/06/15 12:11, Shashank Gupta shgu...@adobe.com wrote: Hi Tim, There is no time bound SLA provided by AWS when a given binary would be successfully replicated to destination S3 bucket. There would be cases of missing binaries if mongo nodes sync faster than S3 replication. Also S3 replication works between a given pair of buckets. So one S3 bucket can replicate to a single S3 destination bucket. I think we can implement a tiered S3Datastore which writes/reads to/from multiple S3 buckets. The tiered S3DS first tries to read from same-region bucket and if not found than fallback to cross-geo buckets. Has this been tested already ? Generally, wdyt ? No. I suggest to first test cross geo mongo deployment with single S3 bucket. There shouldn't be functional issue in using single S3 bucket. Few customers use single shared S3 bucket between non-clustered cross-geo jackrabbit2 repositories in production. Thanks, -shashank -Original Message- From: maret.timot...@gmail.com [mailto:maret.timot...@gmail.com] On Behalf Of Timothée Maret Sent: Monday, June 29, 2015 4:05 PM To: oak-dev@jackrabbit.apache.org Subject: S3DataStore leverage Cross-Region Replication Hi, In a cross region setup using the S3 data store, it may make sense to leverage the Cross-Region auto replication of S3 buckets [0,1]. In order to avoid data replication issues it would make sense IMO to allow configuring the S3DataStore with two S3 buckets, one for writing and one for reading. The writing bucket would be shared among all instance (from all regions) while the reading bucket would be in each region (thus decreasing the latency). The writing bucket would auto replicate to the reading buckets. Has this been tested already ? Generally, wdyt ? Regards, Timothee [0] https://aws.amazon.com/blogs/aws/new-cross-region-replication-for-amazo n-s3/ [1] https://docs.aws.amazon.com/AmazonS3/latest/dev/crr.html
Re: S3DataStore leverage Cross-Region Replication
Hi Shashank, Thanks for this. 2015-06-30 12:11 GMT+02:00 Shashank Gupta shgu...@adobe.com: Hi Tim, There is no time bound SLA provided by AWS when a given binary would be successfully replicated to destination S3 bucket. There would be cases of missing binaries if mongo nodes sync faster than S3 replication. Yes, this would be expected, until the buckets replicate. Also S3 replication works between a given pair of buckets. So one S3 bucket can replicate to a single S3 destination bucket. Yes, the setup would be limited to two regions. I think we can implement a tiered S3Datastore which writes/reads to/from multiple S3 buckets. The tiered S3DS first tries to read from same-region bucket and if not found than fallback to cross-geo buckets. Great, although I see it would be valuable in a limited set of use cases (only two regions involved). Has this been tested already ? Generally, wdyt ? No. I suggest to first test cross geo mongo deployment with single S3 bucket. There shouldn't be functional issue in using single S3 bucket. Few customers use single shared S3 bucket between non-clustered cross-geo jackrabbit2 repositories in production. Sure, adding more complexity only make sense if we can demonstrate this is a bottleneck. Regards, Timothee Thanks, -shashank -Original Message- From: maret.timot...@gmail.com [mailto:maret.timot...@gmail.com] On Behalf Of Timothée Maret Sent: Monday, June 29, 2015 4:05 PM To: oak-dev@jackrabbit.apache.org Subject: S3DataStore leverage Cross-Region Replication Hi, In a cross region setup using the S3 data store, it may make sense to leverage the Cross-Region auto replication of S3 buckets [0,1]. In order to avoid data replication issues it would make sense IMO to allow configuring the S3DataStore with two S3 buckets, one for writing and one for reading. The writing bucket would be shared among all instance (from all regions) while the reading bucket would be in each region (thus decreasing the latency). The writing bucket would auto replicate to the reading buckets. Has this been tested already ? Generally, wdyt ? Regards, Timothee [0] https://aws.amazon.com/blogs/aws/new-cross-region-replication-for-amazon-s3/ [1] https://docs.aws.amazon.com/AmazonS3/latest/dev/crr.html
jackrabbit-oak build #5897: Errored
Build Update for apache/jackrabbit-oak - Build: #5897 Status: Errored Duration: 3003 seconds Commit: 86acd0eb2a76ee07d1b07b025c0d7235e1be9c80 (trunk) Author: Chetan Mehrotra Message: OAK-3053 - Locking issues seen with CopyOnWrite mode enabled Adding disabled testcase reproducing the issue git-svn-id: https://svn.apache.org/repos/asf/jackrabbit/oak/trunk@1688421 13f79535-47bb-0310-9956-ffa450edef68 View the changeset: https://github.com/apache/jackrabbit-oak/compare/e0ebc75950d4...86acd0eb2a76 View the full build log and details: https://travis-ci.org/apache/jackrabbit-oak/builds/68961965 -- sent by Jukka's Travis notification gateway
RE: S3DataStore leverage Cross-Region Replication
Hi Tim, There is no time bound SLA provided by AWS when a given binary would be successfully replicated to destination S3 bucket. There would be cases of missing binaries if mongo nodes sync faster than S3 replication. Also S3 replication works between a given pair of buckets. So one S3 bucket can replicate to a single S3 destination bucket. I think we can implement a tiered S3Datastore which writes/reads to/from multiple S3 buckets. The tiered S3DS first tries to read from same-region bucket and if not found than fallback to cross-geo buckets. Has this been tested already ? Generally, wdyt ? No. I suggest to first test cross geo mongo deployment with single S3 bucket. There shouldn't be functional issue in using single S3 bucket. Few customers use single shared S3 bucket between non-clustered cross-geo jackrabbit2 repositories in production. Thanks, -shashank -Original Message- From: maret.timot...@gmail.com [mailto:maret.timot...@gmail.com] On Behalf Of Timothée Maret Sent: Monday, June 29, 2015 4:05 PM To: oak-dev@jackrabbit.apache.org Subject: S3DataStore leverage Cross-Region Replication Hi, In a cross region setup using the S3 data store, it may make sense to leverage the Cross-Region auto replication of S3 buckets [0,1]. In order to avoid data replication issues it would make sense IMO to allow configuring the S3DataStore with two S3 buckets, one for writing and one for reading. The writing bucket would be shared among all instance (from all regions) while the reading bucket would be in each region (thus decreasing the latency). The writing bucket would auto replicate to the reading buckets. Has this been tested already ? Generally, wdyt ? Regards, Timothee [0] https://aws.amazon.com/blogs/aws/new-cross-region-replication-for-amazon-s3/ [1] https://docs.aws.amazon.com/AmazonS3/latest/dev/crr.html
jackrabbit-oak build #5901: Passed
Build Update for apache/jackrabbit-oak - Build: #5901 Status: Passed Duration: 2545 seconds Commit: 75df12f6e56b4f8038cf7e76ae3fa617475ca9c0 (trunk) Author: Marcel Reutegger Message: OAK-3023: Long running MongoDB query may block other threads git-svn-id: https://svn.apache.org/repos/asf/jackrabbit/oak/trunk@1688453 13f79535-47bb-0310-9956-ffa450edef68 View the changeset: https://github.com/apache/jackrabbit-oak/compare/7552a10bd3f8...75df12f6e56b View the full build log and details: https://travis-ci.org/apache/jackrabbit-oak/builds/68988272 -- sent by Jukka's Travis notification gateway
jackrabbit-oak build #5902: Fixed
Build Update for apache/jackrabbit-oak - Build: #5902 Status: Fixed Duration: 1532 seconds Commit: c8f52e93065c8077f3bbbd2f6d83a246172c9512 (1.2) Author: Marcel Reutegger Message: OAK-3023: Long running MongoDB query may block other threads Merged revision 1688453 from trunk git-svn-id: https://svn.apache.org/repos/asf/jackrabbit/oak/branches/1.2@1688469 13f79535-47bb-0310-9956-ffa450edef68 View the changeset: https://github.com/apache/jackrabbit-oak/compare/00cfd6c4bf0e...c8f52e93065c View the full build log and details: https://travis-ci.org/apache/jackrabbit-oak/builds/68998448 -- sent by Jukka's Travis notification gateway
Re: OSGi configuration lookup
To keep things simple, I suggest to fix OAK-3022 maintaining exactly the same behaviour, and without changing the SegmentNodeStoreService. What do you think if we address the issue about different behaviours separately? 2015-06-30 5:52 GMT+02:00 Chetan Mehrotra chetan.mehro...@gmail.com: That can be done but some more details! When properties are read from framework properties then property names are prefixed with 'oak.documentstore.' kind of like namespaced as framework properties are flat and global. So if we need to do that for Segment also then we should use a similar namespaceing. For e.g. if the property name is 'cache' then when reading from fwk then 'oak.documentstore.cache' would be used oak.mongo.uri and oak.mongo.db are spacial cased though and not follow this rule. Chetan Mehrotra On Tue, Jun 30, 2015 at 2:55 AM, Francesco Mari mari.france...@gmail.com wrote: So we should probably adopt this strategy instead. This means that SegmentNodeStoreService is the one that should be modified. 2015-06-29 17:15 GMT+02:00 Chetan Mehrotra chetan.mehro...@gmail.com: Looking at code flow now yes it differs. The thought behind reading from framework property first was to provide a simple way to override the config which might be packaged by default. For e.g. while launching Oak via Sling one can provide the framework property at command line (using -Doak.mongo.uri) which would supercede the one packaged by default. This simplifies the testing. Chetan Mehrotra On Mon, Jun 29, 2015 at 7:01 PM, Davide Giannella dav...@apache.org wrote: On 29/06/2015 10:22, Francesco Mari wrote: ... Is it possible - or does it make sense - to make this behaviour uniform across components? I think it's a good idea to uniform this aspect. Maybe we could put it down as a guideline by setting up a new page on the doc site: code-conventions.md. Somewhere beside: http://jackrabbit.apache.org/oak/docs/dev_getting_started.html Personally I'd go for component first and bundle then, but I'm not too religious about it :) Anyone against it? Davide
Re: OSGi configuration lookup
On Tue, Jun 30, 2015 at 12:00 PM, Francesco Mari mari.france...@gmail.com wrote: I suggest to fix OAK-3022 maintaining exactly the same behaviour, and without changing the SegmentNodeStoreService Makes sense. They are two different issue Chetan Mehrotra
[ANNOUNCE] Apache Jackrabbit Oak 1.0.16 released
The Apache Jackrabbit community is pleased to announce the release of Apache Jackrabbit Oak 1.0.16. The release is available for download at: http://jackrabbit.apache.org/downloads.html See the full release notes below for details about this release. Release Notes -- Apache Jackrabbit Oak -- Version 1.0.16 Introduction Jackrabbit Oak is a scalable, high-performance hierarchical content repository designed for use as the foundation of modern world-class web sites and other demanding content applications. Apache Jackrabbit Oak 1.0.16 is a patch release that contains fixes and improvements over Oak 1.0. Jackrabbit Oak 1.0.x releases are considered stable and targeted for production use. The Oak effort is a part of the Apache Jackrabbit project. Apache Jackrabbit is a project of the Apache Software Foundation. New configuration options since Oak 1.0.15 -- The DocumentNodeStore has a new system property, which controls the time a commit tries to acquire the merge lock: -Doak.maxLockTryTimeMultiplier=30 The default value is 30 and roughly translates to 60 seconds. See OAK-2762 and OAK-2823 for more details. LuceneIndexEditor now supports CopyOnWrite mode (OAK-2247) for faster indexing. Refer to http://jackrabbit.apache.org/oak/docs/query/lucene.html#CopyOnWrite for more details. Changes in Oak 1.0.16 - Bugs [OAK-3000] - SimpleExcerptProvider causes OOM for some wildcard expressions [OAK-3019] - VersionablePathHook must not process hidden nodes [OAK-3021] - UserValidator and AccessControlValidator must not process hidden nodes [OAK-3028] - Hierarchy conflict detection broken [OAK-3029] - EmbeddedSolrServerProvider should check if core is / can be loaded Improvements [OAK-3004] - OSGI wrapper service for Jackrabbit CachingFDS [OAK-3017] - Log message when a branch is created New Features [OAK-2980] - Fast result size estimate in Solr index Sub-tasks [OAK-2410] - [sonar]Some statements not being closed in RDBDocumentStore [OAK-2747] - Admin cannot create versions on a locked page by itself [OAK-2982] - BasicDocumentStoreTest: separate actual unit tests from performance tests [OAK-2985] - RDBDocumentStore: more diagnostics for long-running queries [OAK-2987] - RDBDocumentStore: try PreparedStatement batching [OAK-2995] - RDB*Store: check transaction isolation level [OAK-3009] - RDBDocumentStore: add support for optional additional index [OAK-3010] - RDBDocumentStore: remove hardwired id-is-binary flag In addition to the above-mentioned changes, this release contains all changes included in previous Apache Jackrabbit Oak 1.0.x releases. Please note, the backported RDB support for the DocumentNodeStore is considered experimental at this point and is not yet ready for production use. Feel free to try it out and report any issues you may see to the Oak developers. For more detailed information about all the changes in this and other Oak releases, please see the Oak issue tracker at https://issues.apache.org/jira/browse/OAK Release Contents This release consists of a single source archive packaged as a zip file. The archive can be unpacked with the jar tool from your JDK installation. See the README.md file for instructions on how to build this release. The source archive is accompanied by SHA1 and MD5 checksums and a PGP signature that you can use to verify the authenticity of your download. The public key used for the PGP signature can be found at http://www.apache.org/dist/jackrabbit/KEYS. About Apache Jackrabbit Oak --- Jackrabbit Oak is a scalable, high-performance hierarchical content repository designed for use as the foundation of modern world-class web sites and other demanding content applications. The Oak effort is a part of the Apache Jackrabbit project. Apache Jackrabbit is a project of the Apache Software Foundation. For more information, visit http://jackrabbit.apache.org/oak About The Apache Software Foundation Established in 1999, The Apache Software Foundation provides organizational, legal, and financial support for more than 140 freely-available, collaboratively-developed Open Source projects. The pragmatic Apache License enables individual and commercial users to easily deploy Apache software; the Foundation's intellectual property framework limits the legal exposure of its 3,800+ contributors. For more information, visit http://www.apache.org/
Re: S3DataStore leverage Cross-Region Replication
Shashank, In case we think it’s needed to implement multiple chained S3 DSs then I think we should model it after Jackrabbit’s Multidatastore which allows arbitrary DS implementations to be chained: http://jackrabbit.510166.n4.nabble.com/MultiDataStore-td4655772.html Michael On 30/06/15 12:11, Shashank Gupta shgu...@adobe.com wrote: Hi Tim, There is no time bound SLA provided by AWS when a given binary would be successfully replicated to destination S3 bucket. There would be cases of missing binaries if mongo nodes sync faster than S3 replication. Also S3 replication works between a given pair of buckets. So one S3 bucket can replicate to a single S3 destination bucket. I think we can implement a tiered S3Datastore which writes/reads to/from multiple S3 buckets. The tiered S3DS first tries to read from same-region bucket and if not found than fallback to cross-geo buckets. Has this been tested already ? Generally, wdyt ? No. I suggest to first test cross geo mongo deployment with single S3 bucket. There shouldn't be functional issue in using single S3 bucket. Few customers use single shared S3 bucket between non-clustered cross-geo jackrabbit2 repositories in production. Thanks, -shashank -Original Message- From: maret.timot...@gmail.com [mailto:maret.timot...@gmail.com] On Behalf Of Timothée Maret Sent: Monday, June 29, 2015 4:05 PM To: oak-dev@jackrabbit.apache.org Subject: S3DataStore leverage Cross-Region Replication Hi, In a cross region setup using the S3 data store, it may make sense to leverage the Cross-Region auto replication of S3 buckets [0,1]. In order to avoid data replication issues it would make sense IMO to allow configuring the S3DataStore with two S3 buckets, one for writing and one for reading. The writing bucket would be shared among all instance (from all regions) while the reading bucket would be in each region (thus decreasing the latency). The writing bucket would auto replicate to the reading buckets. Has this been tested already ? Generally, wdyt ? Regards, Timothee [0] https://aws.amazon.com/blogs/aws/new-cross-region-replication-for-amazon-s3/ [1] https://docs.aws.amazon.com/AmazonS3/latest/dev/crr.html
