[CompositeDataStore] Configuration format for delegate data stores
Hi, I’d be interested in ideas on how we should define configuration for delegate data stores for the CompositeDataStore. One idea was proposed on-list earlier in [0]. I like this idea, but it is a unified node-store/data-store configuration concept that might take a bit longer to support than we want to handle right now, and may be a bit more invasive than we want to tackle at this stage in the release. For my POC I did a really simple configuration format, like this: datastore.1=dataStoreName:FileDataStore,className:org.apache.jackrabbit.oak.plugins.blob.datastore.OakCachingFDS,bundleName:org.apache.jackrabbit.oak-core,path:/Users/maryan/POCs/content_backflow_poc/ds_testing datastore.2=dataStoreName:FileDataStore,className:org.apache.jackrabbit.oak.plugins.blob.datastore.OakCachingFDS,bundleName:org.apache.jackrabbit.oak-core,path:/Users/maryan/POCs/content_backflow_poc/ds_production,readOnly:true I admit this is kinda ugly. Delegates are identified by starting with “datastore” - the rest of this part of the line is not super relevant other than to make each key different. Everything on the right of the “=“ is the value which gets passed to the service. The service or the data store itself has to parse the rest of the string. The rest of the string is key/value pairs, each pair separated by “,", each key separated from the value by “:”. Mostly I’m just indicating the delegate type, class path (so it can be created by reflection), and bundle name (so when the bundle changes state we know to take some action). Note that this relates to the earlier thread [1] regarding how delegates get created. The remainder gets passed to the delegate as configuration options. This approach isn’t super flexible and is hard to edit and read. Open to other ideas. Thoughts? [0] - http://oak.markmail.org/thread/zhnntcdxe5jj2rg2 [1] - http://oak.markmail.org/thread/4ouvpitv5zrypiqg -MR
Re: [CompositeDataStore] How to properly create delegate data stores?
Hi Tomek, Thanks for the feedback. I hadn’t thought about it this way, so I’ll consider it further. Some concerns listed below. On October 3, 2017 at 12:35:32 AM, Tomek Rekawek (reka...@adobe.com.invalid) wrote: Hello Matt, I don’t think we should rely on the bundle activation / deactivation, but rather on the service registration / reregistration. OSGi allows to use MANDATORY_MULTIPLE cardinality for a @Reference - in this case, the service consumer will be informed every time there is a new service implementing given interface. Then, if the CompositeDataStoreService thinks that all the required partial data stores are there, it can register itself as a BlobStore, using BundleContext#registerService. Also, we probably need some kind of differentiation between “partial” and the “final” datastore (so the node store won’t pick the first one). For the node stores, we introduced a new property “role”. If the node store service is configured with this property in place, it registers a NodeStoreProvider rather than NodeStore (so we are sure the partial node store is not used directly). So, my idea is as follows: 1. Create new BlobStoreProvider interface, with just one method: getBlobStore(). 2. Modify all the existing blob store services adding them an optional “role” property (any string). One concern I have with this approach is that if we want a data store to be usable as a CompositeDataStore delegate, that data store has to make specific provisions to do this. My thinking was that it would be preferable to have the CompositeDataStore contain as much of the logic as possible. Ideally a data store should work as a delegate without having to make any changes to the data store itself. (Not sure if we can achieve this, but…) 3. If the data store service is configured with this role, it should register the BlobStoreProvider service rather than a normal BlobStore. 4. The CompositeDataStoreService should be configured with a list of blob store roles it should wait for. 5. The CompositeDataStoreService has a MANDATORY_MULTIPLE @Reference of type BlobStoreProvider. 6. Once (a) the CompositeDataStoreService is activated and (b) all the blob store providers are there, it’ll register a BlobStore service, which will be picked up by the node store. I have concerns about this part also. Which blob store providers should the CompositeDataStoreService wait for? For example, should it wait for S3DataStore? If yes, and if the installation doesn’t use the S3 connector, that provider will never show up, and therefore the CompositeDataStoreService would never get registered. If it doesn’t wait for S3DataStore but the installation does use S3DataStore, what happens if that bundle is unloaded? Wouldn’t this approach require that every possible data store that can be a blob store provider for the composite be included in each installation that wants to use the CompositeDataStore? Regards, -MR
Re: No documentation available for Oak MBeans?
Hi Jörg, I don't know all the Oak documentation, but I can start the list with a page I'm aware of: http://jackrabbit.apache.org/oak/docs/nodestore/segment/overview.html#monitoring-via-jmx HTH, Valentin On Thu, Sep 28, 2017 at 12:21 PM Jörg Hoh wrote: > Hi, > > I see that with Oak 1.6 there are a lot of MBeans around which will likely > help me to monitor an OAK repository. But at the moment I can only try to > guess what the MBean properties describe. And therefor it's hard to > interprete the values and define thresholds for them. > > I've checked the Oak site but I did not find any documentation covering the > MBeans, their properties and methods? > > Jörg > > -- > Cheers, > Jörg Hoh, > > http://cqdump.wordpress.com > Twitter: @joerghoh >
Re: Cold standby for oak-benchmarks: option vs fixture
Fixture looks better option here! Chetan Mehrotra On Mon, Oct 2, 2017 at 8:09 AM, Andrei Dulceanu wrote: > Hi, > > With OAK-6615 [0] we'd like to lay the foundation for including cold > standby among features which could be included in a benchmark. This means > that we'd like to have a cold standby that will sync with the primary every > N seconds while the benchmarks only run on the primary. Initially the plan > was to have this as an option, --cold-standby [N] for Oak-Segment-Tar* > fixtures. > > Looking more carefully through the code in oak-benchmarks and > oak-run-commons, I was thinking about implementing this as a new fixture. > This could work similarly to Oak-Segment-Tar-DS and have dedicated options > like --no-data-store, --private-data-store or --shared-data-store. It would > allow us to better setup primary and standby instances (will also cover the > shared data store use case, left uncovered by --cold-standby option). > > What do you think about this? Should we go with an option or a fixture? > > Thanks, > Andrei > > [0] https://issues.apache.org/jira/browse/OAK-6615
