Re: [discuss] Near real time search to account for latency in background indexing

2015-07-24 Thread Michael Marth
Hi Chetan, Question about the indexing step: From your description I am not sure how the indexing would be triggered for local changes. Probably not through the Async Indexer (this would not gain us much, right?). Would this be a Commit Hook? Michael On 23/07/15 13:48, Chetan Mehrotra

Re: [discuss] Near real time search to account for latency in background indexing

2015-07-24 Thread Chetan Mehrotra
On Fri, Jul 24, 2015 at 12:15 PM, Michael Marth mma...@adobe.com wrote: From your description I am not sure how the indexing would be triggered for local changes. Probably not through the Async Indexer (this would not gain us much, right?). Would this be a Commit Hook? My thought was to use

Re: [discuss] Near real time search to account for latency in background indexing

2015-07-24 Thread Michael Marth
Ah OK - makes sense! One more thing I pondered when thinking about the use case: Would the indexer need to be Lucene-based? It seems what is needed is that nodes can be found quickly based on certain properties, but not the additional features Lucene provides. So, an alternative for the “fast

Re: [discuss] Near real time search to account for latency in background indexing

2015-07-24 Thread Ian Boston
Hi Chetan, The overall approach looks ok. Some questions about indexing. How will you deal with JVM failure ? and related. How frequently will commits to the persisted index be performed ? I assume that switching to use ElasticSearch, which delivers NRT reliably in the 0.1s range has been

Re: [discuss] Near real time search to account for latency in background indexing

2015-07-24 Thread Tommaso Teofili
I think the proposal makes sense; in the end NRT is something that is inherently supported by Lucene (see an example [1]) and, as Ian mentioned, something that has been similarly implemented in ES and Solr. I think it'd be possible though to make use of Lucene's NRT capability by changing a bit

Re: [discuss] Near real time search to account for latency in background indexing

2015-07-24 Thread Chetan Mehrotra
On Fri, Jul 24, 2015 at 2:40 PM, Amit Jain am...@ieee.org wrote: Well that would work for a single node deployment when TarMK is used but would still have a lag as based on frequency of AsyncIndexer which we are seeing is having delays of upto 10-20 sec and may vary. For cluster where indexing

Re: [discuss] Near real time search to account for latency in background indexing

2015-07-24 Thread Chetan Mehrotra
Hi Ian, To be clear the in memory index is purely ephemeral and is not meant to be persisted. It just compliments the persistent index to allow access to recently added/modified entries. So now to your queries How will you deal with JVM failure ? Do nothing. The index as explained is transient.

Re: [discuss] Near real time search to account for latency in background indexing

2015-07-24 Thread Ian Boston
Hi, On 24 July 2015 at 09:06, Chetan Mehrotra chetan.mehro...@gmail.com wrote: Hi Ian, To be clear the in memory index is purely ephemeral and is not meant to be persisted. It just compliments the persistent index to allow access to recently added/modified entries. So now to your queries

Re: [discuss] Near real time search to account for latency in background indexing

2015-07-24 Thread Chetan Mehrotra
On Fri, Jul 24, 2015 at 2:19 PM, Tommaso Teofili tommaso.teof...@gmail.com wrote: I think it'd be possible though to make use of Lucene's NRT capability by changing a bit the code that creates an IndexReader [2] to use DirectoryReader#open(IndexWriter,boolean) [3]. Well that would work for a

Re: [discuss] Near real time search to account for latency in background indexing

2015-07-24 Thread Amit Jain
On Fri, Jul 24, 2015 at 2:35 PM, Chetan Mehrotra chetan.mehro...@gmail.com wrote: Well that would work for a single node deployment when TarMK is used but would still have a lag as based on frequency of AsyncIndexer which we are seeing is having delays of upto 10-20 sec and may vary. For

Re: [discuss] Near real time search to account for latency in background indexing

2015-07-24 Thread Chetan Mehrotra
On Fri, Jul 24, 2015 at 12:56 PM, Michael Marth mma...@adobe.com wrote: Would the indexer need to be Lucene-based? Need not be. The reason I preferred using Lucene is that current property index only support single condition evaluation. Having a Lucene based impl would allow most of current JCR

[ANNOUNCE] Apache Jackrabbit Oak 1.3.3 released

2015-07-24 Thread Davide Giannella
The Apache Jackrabbit community is pleased to announce the release of Apache Jackrabbit Oak ${version}. The release is available for download at: http://jackrabbit.apache.org/downloads.html See the full release notes below for details about this release. Release Notes -- Apache Jackrabbit Oak

Re: [discuss] Near real time search to account for latency in background indexing

2015-07-24 Thread Amit Jain
On Fri, Jul 24, 2015 at 2:46 PM, Chetan Mehrotra chetan.mehro...@gmail.com wrote: The in memory index would be managed on each cluster node and has visibility to *local* changes happening on that cluster node. So it would certainly not provide a cluster wide real time search. But it help in

Re: [discuss] Near real time search to account for latency in background indexing

2015-07-24 Thread Michael Marth
The reason I preferred using Lucene is that current property index only support single condition evaluation. I did not know this. That’s a strong argument in favour of using Lucene.

Re: [discuss] Near real time search to account for latency in background indexing

2015-07-24 Thread Ian Boston
Hi Tommaso, My knowledge of Solr is not anything like as deep as yours. I would like to check what I know is correct, to avoid sharing the wrong information. In [1] the first test does not commit and is backed by a RAMDirectory shared by the reader and writer. Does that mean that Lucene natively

Re: [discuss] Near real time search to account for latency in background indexing

2015-07-24 Thread Tommaso Teofili
Hi Ian, 2015-07-24 11:11 GMT+02:00 Ian Boston i...@tfd.co.uk: Hi Tommaso, My knowledge of Solr is not anything like as deep as yours. I would like to check what I know is correct, to avoid sharing the wrong information. In [1] the first test does not commit and is backed by a RAMDirectory

jackrabbit-oak build #6065: Fixed

2015-07-24 Thread Travis CI
Build Update for apache/jackrabbit-oak - Build: #6065 Status: Fixed Duration: 2291 seconds Commit: 9f2a6c04cea3d36ba0de6052fe1ee4cb9995ea3e (1.0) Author: Tommaso Teofili Message: OAK-3142 - using solr's fqs for desc and pt restrictions (branch 1.0)

jackrabbit-oak build #6064: Errored

2015-07-24 Thread Travis CI
Build Update for apache/jackrabbit-oak - Build: #6064 Status: Errored Duration: 2496 seconds Commit: ff2f1a0371fd6c353c5c17a72223edc1e5604610 (trunk) Author: Tommaso Teofili Message: OAK-3142 - using solr's fqs for desc and pt restrictions git-svn-id:

Re: [discuss] Near real time search to account for latency in background indexing

2015-07-24 Thread Ian Boston
Hi, On 24 July 2015 at 11:11, Tommaso Teofili tommaso.teof...@gmail.com wrote: Hi Ian, 2015-07-24 11:11 GMT+02:00 Ian Boston i...@tfd.co.uk: Hi Tommaso, My knowledge of Solr is not anything like as deep as yours. I would like to check what I know is correct, to avoid sharing the

jackrabbit-oak build #6067: Passed

2015-07-24 Thread Travis CI
Build Update for apache/jackrabbit-oak - Build: #6067 Status: Passed Duration: 2137 seconds Commit: 82cb301f276468025610be29fdcea17668b4001d (trunk) Author: Tommaso Teofili Message: OAK-2175, OAK-2176 - fixed another typo in the doc git-svn-id:

[ANNOUNCE] Apache Jackrabbit Oak 1.2.3 released

2015-07-24 Thread Davide Giannella
The Apache Jackrabbit community is pleased to announce the release of Apache Jackrabbit Oak 1.2.3. The release is available for download at: http://jackrabbit.apache.org/downloads.html See the full release notes below for details about this release. Release Notes -- Apache Jackrabbit Oak --

[RESULT][VOTE] Release Apache Jackrabbit Oak 1.2.3

2015-07-24 Thread Davide Giannella
Hello Team, the vote passes as follows: +1 Alex Parvulescu +1 Amit Jain +1 Davide Giannella +1 Marcel Reutegger Thanks for voting. I'll push the release out. -- Davide

[Oak origin/1.2] Apache Jackrabbit Oak matrix - Build # 290 - Failure

2015-07-24 Thread Apache Jenkins Server
The Apache Jenkins build system has built Apache Jackrabbit Oak matrix (build #290) Status: Failure Check console output at https://builds.apache.org/job/Apache%20Jackrabbit%20Oak%20matrix/290/ to view the results.

[Oak origin/1.0] Apache Jackrabbit Oak matrix - Build # 291 - Still Failing

2015-07-24 Thread Apache Jenkins Server
The Apache Jenkins build system has built Apache Jackrabbit Oak matrix (build #291) Status: Still Failing Check console output at https://builds.apache.org/job/Apache%20Jackrabbit%20Oak%20matrix/291/ to view the results.