Re: Azure Segment Store

Hi, I misread the documentation in the patch. Thank you for pointing out my mistake. Best Regards Ian On 6 March 2018 at 09:53, Tomek Rękawek <tom...@apache.org> wrote: > Hi Ian, > > > On 5 Mar 2018, at 17:47, Ian Boston <i...@tfd.co.uk> wrote: > > > > I assu

Re: Azure Segment Store

t.com/en-us/azure/azure-subscription-service-limits > > Michael > > On 2 March 2018 at 09:45, Ian Boston <i...@tfd.co.uk> wrote: > > Hi Tomek, > > Thank you for the pointers and the description in OAK-6922. It all makes > > sense and seems like a reasonable ap

Re: Azure Segment Store

Hi Tomek, Thank you for the pointers and the description in OAK-6922. It all makes sense and seems like a reasonable approach. I assume the description is upto date. How does it perform compared to TarMK a) when the entire repo doesn't fit into RAM allocated to the container ? b) when the working

Re: Oak 1.7.10 release plan

Hi, Amit has resolved the issue. 1.7.10 is no longer blocked. Best Regards Ian On 20 October 2017 at 09:57, Davide Giannella <dav...@apache.org> wrote: > On 19/10/2017 16:57, Ian Boston wrote: > > Are you able to revert the 3 commits listed in the issue ? > > If thats

Re: Oak 1.7.10 release plan

Hi Davide, Are you able to revert the 3 commits listed in the issue ? If thats done, you can resolve the issue. Nothing else depends on these changes at this time, and should not, hence the blocker. Best Regards Ian On 19 October 2017 at 14:30, Davide Giannella wrote: > Hello

Re: Oak modularisation unstable roadmap ?

hat Jackrabbit API was not an option > because I don't recall that we had that option discussed in the lengthy > thread that preceded the contribution to oak-api. > > Kind regards > Angela > > > From: Ian Boston <i...@tfd.co.uk> > Date: Tuesday 17 October 2017 19:3

Re: Oak modularisation unstable roadmap ?

Regards Ian On 17 October 2017 at 18:37, Ian Boston <i...@tfd.co.uk> wrote: > Hi, > > I dont really care where the API is, however IIUC > > Oak 1.6 needs JR API 2.14.2 (based on Slings oak.txt provisioning model) > Oak 1.7 nees JR API 2.15.5 (based on my patch to make S

Re: Oak modularisation unstable roadmap ?

e JCR specification). From a > Sling point of view the Oak API is an implementation detail of a particular > JCR implementation (namely Jackrabbit Oak). > > Kind regards > Angela > > From: Ian Boston <i...@tfd.co.uk> > Date: Friday 13 October 2017 18:32 > To: Ange

Re: Oak modularisation unstable roadmap ?

xtension of JCR, where as Oak API is on a > different layer in the stack and from a Sling PoV a implementation detail > of a JCR implementation. > > So, I would opt for taking following up on Marcels suggestion. > > Kind regards > Angela > > On 13/10/17 17:22, "Julian Resc

Re: Oak modularisation unstable roadmap ?

l send a separate email to get the > discussion started. > > kind regards > angela > > On 13/10/17 15:48, "Ian Boston" <i...@tfd.co.uk> wrote: > > >Hi, > >I have ported Sling to depend on Oak 1.7.8, and now that 1.7.9 is released > >I am upda

Oak modularisation unstable roadmap ?

Hi, I have ported Sling to depend on Oak 1.7.8, and now that 1.7.9 is released I am updating the patch to port, finding 2 new bundles that were not required to make Sling build with Oak 1.7.8. oak-store-document oak-security-spi Is it too soon for Sling to depend on Oak 1.7.x ? Is there more

Re: Oak Bundle versioning.

becomes easier to integrate various Oak changes, > especially for consumers only depending on stable APIs. > > On Thu, 2017-10-05 at 13:33 +0100, Ian Boston wrote: > > Obviously bundles remain the release unit, and the build must include > > OSGi > > based integration te

Oak Bundle versioning.

Hi, Currently the whole Oak source tree is synchronously versioned. Which is great for Oak and its releases, but problematic downstream. I assume this has been discussed before, and hope it can be discussed again ? -- If it can, here is some supporting evidence, using Sling as an example.If

Re: OAK-6575 - A word of caution

stable release. If anyone feels consensus has not been reached, please continue this thread. Best Regards Ian On 15 September 2017 at 07:52, Angela Schreiber <anch...@adobe.com.invalid> wrote: > Hi Ian > > On 13/09/17 23:34, "Ian Boston" <i...@tfd.co.uk> wrot

Re: OAK-6575 - A word of caution

lementation will not be suitable for a Workflow to use, due to network topology, although that was out so scope for the requirements in OAK-6575. There was some suggestions r adding a getPrivateURI to the URIProvider interface. btw, thanks for your comments in Jira, I have updated the patch.

Re: OAK-6575 - A word of caution

be a great step forwards. On the other hand, if you are saying that the Oak Threat model has to be developed and agreed, before this patch can be added, then I am concerned that will take too long. Doing justice to an Oak Treat model will require resource. Best `Regards Ian > > Kind regards >

Re: OAK-6575 - A word of caution

On 7 September 2017 at 14:41, Francesco Mari <mari.france...@gmail.com> wrote: > On Thu, Sep 7, 2017 at 11:05 AM, Ian Boston <i...@tfd.co.uk> wrote: > > On 7 September 2017 at 07:22, Ian Boston <i...@tfd.co.uk> wrote: > > > >> Hi, > >> >

Re: OAK-6575 - A word of caution

On 7 September 2017 at 07:22, Ian Boston <i...@tfd.co.uk> wrote: > Hi, > > On 6 September 2017 at 22:43, Michael Dürig <mdue...@apache.org> wrote: > >> >> >> On 06.09.17 23:08, Michael Dürig wrote: >> >>> >>> Hi, >>&

Re: OAK-6575 - A word of caution

Hi, On 6 September 2017 at 22:43, Michael Dürig <mdue...@apache.org> wrote: > > > On 06.09.17 23:08, Michael Dürig wrote: > >> >> Hi, >> >> On 05.09.17 14:09, Ian Boston wrote: >> >>> Repeating the comment to on OAK-6575 here for fur

Re: OAK-6575 - A word of caution

Oak API version changes. Best Regards Ian > > Finally, I suspect that in the second patch there was too much of an > aggressive rename refactoring. "types" was renamed to "customtypes" in > a lot of unrelated places. I would definitely double-check that. &g

Re: OAK-6575 - A word of caution

and multi datastore code used for 0DT in the process. An Oak committer with global knowledge will probably be able to do better. On 5 September 2017 at 08:19, Ian Boston <i...@tfd.co.uk> wrote: > Hi, > > On 5 September 2017 at 07:55, Francesco Mari <mari.france...@gmail.com> > w

Re: OAK-6575 - A word of caution

Hi, On 5 September 2017 at 07:55, Francesco Mari <mari.france...@gmail.com> wrote: > On Mon, Sep 4, 2017 at 6:18 PM, Ian Boston <i...@tfd.co.uk> wrote: > > Do you mean: > > keep the OakConversionService but put all the logic to convert from a > > Value to a URI

Re: OAK-6575 - A word of caution

Hi, On 4 September 2017 at 16:43, Francesco Mari <mari.france...@gmail.com> wrote: > On Mon, Sep 4, 2017 at 4:57 PM, Ian Boston <i...@tfd.co.uk> wrote: > > Hi, > > IIUC There are 2 patterns: > > > > 1 Emitting a short lived signed URL as per the AWS Cloud

Re: OAK-6575 - A word of caution

at 4:18 PM, Bertrand Delacretaz > <bdelacre...@apache.org> wrote: > > On Mon, Sep 4, 2017 at 3:44 PM, Ian Boston <i...@tfd.co.uk> wrote: > >> ...I feel > >> that Oak is weaker without the ability to offload bulk data streaming to > >> infrastructure

Re: OAK-6575 - A word of caution

575 and I will delete the GitHub branch. (I am not a committer, so have nothing binding here, other than a desire to improve Oak.) Best Regards Ian On 4 September 2017 at 14:44, Ian Boston <i...@tfd.co.uk> wrote: > Hi Francesco and Tommaso, > I was not aware of the previ

Re: OAK-6575 - A word of caution

Hi Francesco and Tommaso, I was not aware of the previous discussions and will now read those threads and issues. I submitted the issue, patch and thread in good faith, not having a detailed knowledge of everything that has been discussed on oak-dev. I was not trying to ignore or circumvent any

Re: OAK-6575 - Provide a secure external URL to a DataStore binary.

t; [1]. But that module would still require some API in Oak to provide >>>> such an adaptor >>>> >>>> The adaptor proposal here is for enabling layers within Oak to allow >>>> conversion of JCR Binary instance to SignedBinary. Now how this is >>>

Re: OAK-6575 - Provide a secure external URL to a DataStore binary.

ary.class); > } > Within Sling > > > Have an AdapterManager implemented in Sling JCR Base [1] which uses > above approach > > Chetan Mehrotra > [1] https://github.com/apache/sling/tree/trunk/bundles/jcr/base > > > On Thu, Aug 24, 2017 at 4:

Re: OAK-6575 - Provide a secure external URL to a DataStore binary.

>From the javadoc in [1] "The adaptable object may be any non-null object and is not required to implement the Adaptable interface." On 24 August 2017 at 12:54, Ian Boston <i...@tfd.co.uk> wrote: > Hi, > That would require javax.jcr.Binary to implement Adaptable, wh

Re: OAK-6575 - Provide a secure external URL to a DataStore binary.

Hi, That would require javax.jcr.Binary to implement Adaptable, which it cant. (OakBinary could but it doesnt need to). Using Sling AdapterFactory/AdapterManger javadoc (to be replaced with Oaks internal version of the same) What is needed is an AdapterFactory for javax.jcr.Binary to

Re: OAK-6575 - Provide a secure external URL to a DataStore binary.

Hi, On 24 August 2017 at 10:20, Julian Sedding <jsedd...@gmail.com> wrote: > Hi > > On Thu, Aug 24, 2017 at 9:27 AM, Ian Boston <i...@tfd.co.uk> wrote: > > On 24 August 2017 at 08:18, Michael Dürig <mdue...@apache.org> wrote: > > > >> >

Re: OAK-6575 - Provide a secure external URL to a DataStore binary.

On 24 August 2017 at 09:16, Michael Dürig <mdue...@apache.org> wrote: > > > On 24.08.17 09:27, Ian Boston wrote: > >> On 24 August 2017 at 08:18, Michael Dürig <mdue...@apache.org> wrote: >> >> >>> >>> URI uri = ((OakValueFactory) v

Re: OAK-6575 - Provide a secure external URL to a DataStore binary.

The datastore should understand how to go from Blob -> URI. In the case of S3 it does and uses Blob.getContentId(). If the datastore doesnt know how to do it, then its not supported by the datastore. You might need a DataStore.getSignedURI(Blob b) method. On 24 August 2017 at 08:27, Chetan

Re: OAK-6575 - Provide a secure external URL to a DataStore binary.

On 24 August 2017 at 08:18, Michael Dürig wrote: > > > URI uri = ((OakValueFactory) valueFactory).getSignedURI(binProp); > > +1 One point Users in Sling dont know abou Oak, they know about JCR. URI uri = ((OakValueFactory)

Re: Oak Metrics.

Hi, On 15 June 2017 at 11:42, Chetan Mehrotra <chetan.mehro...@gmail.com> wrote: > On Thu, Jun 15, 2017 at 2:55 PM, Ian Boston <i...@tfd.co.uk> wrote: > > Is are the new Oak Metrics documented somewhere ? > > No because so far no one else asked for it and only I wa

Oak Metrics.

Hi, Is are the new Oak Metrics documented somewhere ? I see that [1] exists. Google returns 2 hits for NRT_REFRESH_TIME, oddly neither relevant. (now there will be 1 relevant hit, this thread) Is there 1 page where I can look for a description of all metrics in Oak. Best Regards Ian 1

Re: minimize the impact when creating a new index (or re-indexing)

Hi, Assuming the MongoDB instance is performing well and does not show any slow queries in the mongodb logs, running the index operation on many cores, each core handling one index writer should parallelise the operation. IIRC this is theoretically possible, and might have been implemented in the

Re: codahale metrics Jmx reporter

On 30 May 2017 at 07:35, Chetan Mehrotra wrote: > On Tue, May 30, 2017 at 11:53 AM, Andrei Kalfas > wrote: > > Looks to me that there is a dependency on oak functionality. > > Ian can confirm but I believe thats not required now (the

Re: codahale metrics Jmx reporter

Hi, Here are some reporters that work with Sling/Oak/AEM. [1]. They all look for components registered as implementing MetricsRegistry and then aggregate the data pumping it out to a reporter. They are all implemented as independent bundles. TBH I would avoid pumping the metrics into JMX as JMX

Re: MongoMK failover behaviour.

Hi, On 4 May 2017 at 15:19, Marcel Reutegger <mreut...@adobe.com> wrote: > Hi, > > On 04/05/17 14:57, Ian Boston wrote: > >> Before 120 seconds, should the MongoDB Java driver route read queries to a >> secondary and use the new primary without any action by O

Re: MongoMK failover behaviour.

Hi, On 4 May 2017 at 11:26, Marcel Reutegger <mreut...@adobe.com> wrote: > Hi, > > On 04/05/17 12:02, Ian Boston wrote: > >> What is the expected behaviour when a Oak MongoMK experiences a MongoDB >> primary failure. >> >> I am looking at an in

MongoMK failover behaviour.

Hi, What is the expected behaviour when a Oak MongoMK experiences a MongoDB primary failure. I am looking at an instance that appears to try and retry reads repeatedly from the MongoDB primary and after 60s or more reports the Oak Discovery lease has been lost, resulting in many minutes of

Re: Cassandra as NodeStore

Hi Juanjo, For Cassandra to work as a DocumentStore for Oak it needs to be configured with a Quorum high enough to ensure that all writes are sequentially eventually consistent. That might kill Cassandras write performance. RDB backends have this behaviour because they are single instance. MongoDB

Re: Timeout on Oak Queries ?

I have created https://issues.apache.org/jira/browse/OAK-5978 to track. Best Regards Ian On 23 March 2017 at 14:03, Davide Giannella <dav...@apache.org> wrote: > On 23/03/2017 11:12, Ian Boston wrote: > > Hi, > > Is it possible to configure a maximum execution

Timeout on Oak Queries ?

Hi, Is it possible to configure a maximum execution time for Oak queries ? Other "database" systems often monitor the time a query is taking and kill if if it exceeds a time limit to avoid long running queries causing outages. Best Regards Ian

Re: Oak 1.0.29 vs 1.4.10 memory mapping.

changes in this > > area 1.0 vs. 1.4 branches. > > If no one has better ideas, I'll create an oak issue and investigate > this a > > bit further. > > > > thanks, > > alex > > > > > > On Wed, Mar 22, 2017 at 4:28 PM, Ian Boston <i...@tfd

Re: Metrics support in Oak

Hi, IIRC (a) is doable and the prefered way of naming metrics. In other systems that use Metrics they typically use the package or class name, sometimes an API classname, in the same way that loggers do. This makes it much easier to process and report on blocks of functionality at the reporting

Oak 1.0.29 vs 1.4.10 memory mapping.

Hi, I am looking at Oak-run and I see 2x the mapped memory between 1.0.29 and 1.4.10. It looks like in 1.0.29 each segment file is mapped into memory once, but in 1.4.10 its mapped into memory 2x. Is this expected ? Its not great for page faults. Best Regards Ian

Re: Mongo timeouts

Hi Arek, Have you checked the mongodb logs to determine what is taking too long ? Generally the default timeouts used by Oak on the MongoDB connections are correct to run Oak on MongoMK successfully. The stack trace you shared looks like a slow query in MongoDB rather than a protocol timeout. As a

Hybrid indexing and soft commits.

Hi, IIUC the Hybrid indexing on the master operates in parallel with the master index writer, performing the same task but repeatedly throwing its work away when the master provides an update. IIUC, it effectively performs many soft commits to achieve NRT behaviour. I wonder if there is an

Re: IndexEditorProvider behaviour question.

ng > case and hence invoked only on leader node every 5 sec. So performance > aspects here were not that critical. However with recent work on > Hybrid indexes they would be used in critical path and hence such > aspects are important > > On Wed, Sep 14, 2016 at 3:10 PM, Ian Bo

IndexEditorProvider behaviour question.

Hi, The behaviour of calls to the IndexEditorProvider appears to be suboptimal. Has this area been looked at before? I am working from a complete lack of historical knowledge about the area, so probably don't know the full picture. Based on logging the calls into

Re: CommitHooks as OSGi Components.

On 12 September 2016 at 10:45, Chetan Mehrotra <chetan.mehro...@gmail.com> wrote: > On Mon, Sep 12, 2016 at 3:12 PM, Ian Boston <i...@tfd.co.uk> wrote: > > but if the information that connect a sessionID/userID to the > > paths that are modified is available throug

Re: CommitHooks as OSGi Components.

Hi, On 12 September 2016 at 09:43, Chetan Mehrotra <chetan.mehro...@gmail.com> wrote: > On Mon, Sep 12, 2016 at 2:08 PM, Ian Boston <i...@tfd.co.uk> wrote: > > Unfortunately the IndexProvider route doesn't appear give me the > > information I am after (CommitIn

Re: CommitHooks as OSGi Components.

been considered > way to dangerous and a breach of modularity. > > What has been done in the past for use cases requiring commit hook > functionality on one hand and some part of dynamicity on the other, was to > to specialise the use case. Index editors are one example here. > > Micha

CommitHooks as OSGi Components.

Hi, Is it possible write a CommitHook as an OSGI Component/Service and for Oak to pick it up ? The Component starts and gets registered as a service, but Oak doesn't appear to pick it up. If its not possible to add a CommitHook in this way, what is the best way of doing it from outside the

Re: Seekable access to a Binary

Hi, On 6 September 2016 at 11:34, Bertrand Delacretaz wrote: > Hi, > > On Tue, Sep 6, 2016 at 9:49 AM, Marcel Reutegger > wrote: > > ...we'd still have to add > > Jackrabbit API to support it. E.g. something like: > > > >

Seekable access to a Binary

Hi, Is it possible to write to an Oak Binary via the JCR API at an offset ? I am asking because I am working on the Sling Upload mechanism to make it streamable in an attempt to eliminate the many duplicate IO operations. A whole body upload works, and depending on the DS being used shows good

Re: Oak Indexing. Was Re: Property index replacement / evolution

Hi, On 11 August 2016 at 13:03, Chetan Mehrotra <chetan.mehro...@gmail.com> wrote: > On Thu, Aug 11, 2016 at 5:19 PM, Ian Boston <i...@tfd.co.uk> wrote: > > correct. > > Documents are shared by ID so all updates hit the same shard. > > That may result in network

Re: Oak Indexing. Was Re: Property index replacement / evolution

On 11 August 2016 at 11:10, Chetan Mehrotra <chetan.mehro...@gmail.com> wrote: > On Thu, Aug 11, 2016 at 3:03 PM, Ian Boston <i...@tfd.co.uk> wrote: > > Both Solr Cloud and ES address this by sharding and > > replicating the indexes, so that all commits are soft

Re: Oak Indexing. Was Re: Property index replacement / evolution

; > For deployments having very large repository Solr or ES based indexes > would be preferable and there oak-solr can be used (some day oak-es!) > > So in brief Oak should be self sufficient for smaller deployment and > still allow plugging in Solr/ES for large deployment and there also &g

Re: Property index replacement / evolution

Hi, On 8 August 2016 at 15:39, Vikas Saurabh <vikas.saur...@gmail.com> wrote: > Hi Ian, > > On Mon, Aug 8, 2016 at 3:41 PM, Ian Boston <i...@tfd.co.uk> wrote: > > > > If every successful commit writes the root node, due to every update > > updating a sync p

Re: Property index replacement / evolution

Hi Vikas, On 8 August 2016 at 14:13, Vikas Saurabh <vikas.saur...@gmail.com> wrote: > Hi Ian, > > On Sun, Aug 7, 2016 at 10:01 AM, Ian Boston <i...@tfd.co.uk> wrote: > > Also, IIRC, the root document is not persisted on every commit, but > > synchronized periodi

Re: Property index replacement / evolution

Hi, For TarMK, none of this is an issue as TarMK is all in memory on 1 JVM with local disk. Scaling up by throwing RAM and IO at the problem is a viable option, as far as it's safe/sensible to do so. But TarMK doesn't cluster, and if it did cluster, this would probably be an issue. I think, but

Re: Does Oak core check the repository version ?

Hi, On 4 July 2016 at 09:53, Marcel Reutegger <mreut...@adobe.com> wrote: > Hi, > > On 30/06/16 13:10, "ianbos...@gmail.com on behalf of Ian Boston" wrote: > >I have heard reports of a case of the wrong version of oak-run causing > >problems in

Does Oak core check the repository version ?

Hi, Does Oak core check the persisted repository version to make certain it fits in a range that is compatible with the code being run ? If it doesn't already, I think it should to avoid something like the wrong version of oak-run being used potentially damaging the repository. Best Regards Ian

Re: API proposal for - Expose URL for Blob source (OAK-1963)

Hi, On 11 May 2016 at 14:21, Marius Petria wrote: > Hi, > > I would add another use case in the same area, even if it is more > problematic from the point of view of security. To better support load > spikes an application could return 302 redirects to (signed) S3 urls such

Re: API proposal for - Expose URL for Blob source (OAK-1963)

Hi Angela, On 10 May 2016 at 17:19, Angela Schreiber wrote: > Hi Ian > > >Fair enough, provided there is a solution that addresses the issue Chetan > >is trying to address. > > That's what we are all looking for :) > > >The alternative, for some applications, seems to store

Re: API proposal for - Expose URL for Blob source (OAK-1963)

On 10 May 2016 at 15:02, Angela Schreiber <anch...@adobe.com> wrote: > Hi Ian > > On 04/05/16 18:37, "Ian Boston" <i...@tfd.co.uk> wrote: > >[...] The locations will certainly probably leak > >outside the context of an Oak session so the API contract

Re: API proposal for - Expose URL for Blob source (OAK-1963)

hro...@gmail.com> wrote: > On Mon, May 9, 2016 at 8:27 PM, Ian Boston <i...@tfd.co.uk> wrote: > > > I thought the consumers of this api want things like the absolute path of > > the File in the BlobStore, or the bucket and key of the S3 Object, so > that > > they coul

Re: API proposal for - Expose URL for Blob source (OAK-1963)

; up with an API that allows the blobs to stay under control of Oak? If > >>> not, > >>> this is probably an indication that those blobs shouldn't go into Oak > but > >>> just references to it as Francesco already proposed. Anything else is > >>> w

Re: API proposal for - Expose URL for Blob source (OAK-1963)

Hi, If the File or URL is writable, will writing to the location cause issues for Oak ? IIRC some Oak DS implementations use a digest of the content to determine the location in the DS, so changing the content via Oak will change the location, but changing the content via the File or URL wont. If

Re: DocumentStore question.

that the underlying storage is immediately consistent in nature ? Best Regards Ian On 16 February 2016 at 10:36, Marcel Reutegger <mreut...@adobe.com> wrote: > Hi, > > On 16/02/16 09:56, "ianbos...@gmail.com<mailto:ianbos...@gmail.com> on > behalf of Ian Bost

Re: DocumentStore question.

On 15 February 2016 at 14:49, Marcel Reutegger <mreut...@adobe.com> wrote: > Hi, > > On 12/02/16 17:11, "ianbos...@gmail.com on behalf of Ian Boston" wrote: > >Is there an assumption that the revisions listed in _revisions are > >ordered ? > > There i

DocumentStore question.

Hi, I am looking at [1], and probably confused. Is there an assumption that the revisions listed in _revisions are ordered ? If not, then how is the order of the revisions be determined, given that the clocks on each node in a cluster will have different offsets ? Best Regards Ian 1

cache and index backup and restore ?

Hi, Having done a cold backup of a MongoMK instance with a FS Datastore, is there any advantage in also backing up the local disk copy of the lucene index (normally in repository/index/** ) and persistent cache file (repository/cache/**) so that it can be restored on more than one Oak instance in

Re: OAK-3884 for UDBBroadcaster

he BROADCAST > option configured. > > All of that could also be solved differently, e.g. with a virtual network. > I am not sure if that exists for unit testing. > > Cheers, > > Philipp > > > On 14/01/16 19:41, "ianbos...@gmail.com on behalf of Ian Boston&

Re: OAK-3884 for UDBBroadcaster

Hi, Does the patch work ? According to the answer in the serverfault article referenced in OAK-3884 it should not I tried the pattern referenced on OSX using nc and it doesn't work. The original poster seems to think it works, but those answering disagree and the posted wasn't able to tell them

Re: Multiplexing Document Store

On 3 December 2015 at 04:15, Chetan Mehrotra <chetan.mehro...@gmail.com> wrote: > Hi Ian, > > On Wed, Dec 2, 2015 at 9:24 PM, Ian Boston <i...@tfd.co.uk> wrote: > > Hence all MutableTrees get their > > NodeBuilder from the root DocumentNodeState that is the Docu

Re: Multiplexing Document Store

Hi Robert, On 5 November 2015 at 22:58, Robert Munteanu <romb...@apache.org> wrote: > Hi Ian, > > On Fri, 2015-10-30 at 15:38 +0000, Ian Boston wrote: > > Hi, > > I am trying to enhance a multiplexing document store written > > initially by > > Robert Munt

Re: Lucene auto-tune of cost

On 4 November 2015 at 00:45, Davide Giannella wrote: > Hello Team, > > Lucene index is always asynchronous and the async index could lag behind > by definition. > > Sometimes we could have the same query better served by a property > index, or traversing for example. In case

Re: Lucene auto-tune of cost

a full-text index, which may need to > do (slow) text-extraction. However, updates on a Lucene-based property > index are usually very fast. So it is not obvious to me why they > should not be synchronous. > > Thanks for any enlightening replies! > > Regards > Julian > >

Multiplexing Document Store

Hi, I am trying to enhance a multiplexing document store written initially by Robert Munteanu to support multiplexing of content under /jcr:system/** in particular the version store and the permissions store. I have a scheme that should theoretically work, encoding the target store in the entry

Multiplexing Document Store

Hi, I am trying to enhance a multiplexing document store written initially by Robert Munteanu to support multiplexing of content under /jcr:system/** in particular the version store and the permissions store. I have a scheme that should theoretically work, encoding the target store in the entry

Re: [discuss] Near real time search to account for latency in background indexing

Hi Chetan, The overall approach looks ok. Some questions about indexing. How will you deal with JVM failure ? and related. How frequently will commits to the persisted index be performed ? I assume that switching to use ElasticSearch, which delivers NRT reliably in the 0.1s range has been

Re: [discuss] Near real time search to account for latency in background indexing

0.1s latency) normally needs a write ahead log to work in production and avoid data loss and/or high hard commit volumes killing latency and creating merge/too many files issues as the number of segments grows. Best Regards Ian Chetan Mehrotra On Fri, Jul 24, 2015 at 1:01 PM, Ian Boston i

Re: [discuss] Near real time search to account for latency in background indexing

Hi Tommaso, My knowledge of Solr is not anything like as deep as yours. I would like to check what I know is correct, to avoid sharing the wrong information. In [1] the first test does not commit and is backed by a RAMDirectory shared by the reader and writer. Does that mean that Lucene natively

Re: [discuss] Near real time search to account for latency in background indexing

Hi, On 24 July 2015 at 11:11, Tommaso Teofili tommaso.teof...@gmail.com wrote: Hi Ian, 2015-07-24 11:11 GMT+02:00 Ian Boston i...@tfd.co.uk: Hi Tommaso, My knowledge of Solr is not anything like as deep as yours. I would like to check what I know is correct, to avoid sharing

Re: /oak:index (DocumentNodeStore)

indexes can be asynchronous, because there was no hard requirement for synchronous index updates. Regards Marcel On 08/07/15 18:18, ianbos...@gmail.com on behalf of Ian Boston wrote: Hi, I am confused at how /oak:index works and why it is needed in a MongoDB setting which has native

Re: /oak:index (DocumentNodeStore)

On 9 July 2015 at 09:16, Chetan Mehrotra chetan.mehro...@gmail.com wrote: On Thu, Jul 9, 2015 at 12:45 PM, Marcel Reutegger mreut...@adobe.com wrote: - Data in Oak is multi-versioned. It must be possible to query nodes at a specific revision of the tree. To add - That also makes it

Re: /oak:index (DocumentNodeStore)

@content_products_a = true, @content_products_b = true Regards, Thomas On 08/07/15 18:18, Ian Boston i...@tfd.co.uk wrote: Hi, I am confused at how /oak:index works and why it is needed in a MongoDB setting which has native database indexes that appear to cover the same functionality. Could the Oak

/oak:index (DocumentNodeStore)

Hi, I am confused at how /oak:index works and why it is needed in a MongoDB setting which has native database indexes that appear to cover the same functionality. Could the Oak Query engine use DB indexes directly for all indexes that are built into Oak, and Lucene indexes for all custom indexes ?

Re: Observation: External vs local - Load distribution

Hi, +1 for unbounded, let the GC take care of it and log periodically when its size becomes significant so that anyone wondering why their JVM is consuming so much GC time gets a clue as to the cause, without having to perform heap dumps, thread dumps or jvm probes. (but ideally all queues would

MongoDB collections in MongoDocumentStore

Hi, Is there a fundamental reason why data stored in MongoDB for MongoDocumentStore cant be stored in more than the 3 MondoDB collections currently used ? I am thinking that the collection name is a fn(key). What problems would that cause elsewhere ? Best Regards Ian

Re: MongoDB collections in MongoDocumentStore

throughput of the system. N. On Fri, Jun 12, 2015 at 4:15 PM, Chetan Mehrotra chetan.mehro...@gmail.com wrote: On Fri, Jun 12, 2015 at 7:32 PM, Ian Boston i...@tfd.co.uk wrote: Initially I was thinking about the locking behaviour but I realises 2.6.* is still locking at a database

Re: MongoDB collections in MongoDocumentStore

On 12 June 2015 at 14:13, Chetan Mehrotra chetan.mehro...@gmail.com wrote: On Fri, Jun 12, 2015 at 5:20 PM, Ian Boston i...@tfd.co.uk wrote: Are all queries expected to query all keys within a collection as it is now, or is there some logical structure to the querying ? Not sure if I get

Re: MongoDB collections in MongoDocumentStore

implementation, going from database level to collection. N. On Fri, Jun 12, 2015 at 7:31 PM, Ian Boston i...@tfd.co.uk wrote: H Norberto, Thank you for the feedback on the questions. I see you work for as an Evangelist for MongoDB, so will probably know the answers, and can save me

Re: Semantics of the document key at the DocumentStore level

Hi, On 10 June 2015 at 09:41, Robert Munteanu romb...@apache.org wrote: On Tue, 2015-06-09 at 17:01 +0200, Julian Reschke wrote: On 2015-06-09 16:41, Ian Boston wrote: Hi, Should the opaque String key be abstracted into a DocumentKey interface so that how the key is interpreted

Re: ImmutableTreeTest.testHiddenExists , confusion.

on the Oak API s.str. This one *must never* expose the hidden items. hope that helps angela On 03/06/15 12:50, Ian Boston i...@tfd.co.uk wrote: Hi, I am confused how hidden trees work in MutableTree and ImmutableTree. The ImmutableTreeTesttestHiddenExists asserts that a hidden node from

ImmutableTreeTest.testHiddenExists , confusion.

Hi, I am confused how hidden trees work in MutableTree and ImmutableTree. The ImmutableTreeTesttestHiddenExists asserts that a hidden node from an ImmutableTree will return true() to exists, and yet the same node from a MuttableTree is hard coded to return false to exists(). Is this correct ?

  1   2   >