Re: New Jackrabbit Committer: Mohit Kataria

2019-08-21 Thread Chetan Mehrotra
Welcome Mohit!
Chetan Mehrotra

On Thu, Aug 15, 2019 at 7:26 PM Matt Ryan  wrote:
>
> Welcome Mohit!
>
>
> -MR
>
> On Thu, Aug 15, 2019 at 6:51 AM Julian Sedding  wrote:
>
> > Welcome Mohit!
> >
> > Regards
> > Julian
> >
> > On Wed, Aug 14, 2019 at 3:25 PM Woonsan Ko  wrote:
> > >
> > > Welcome, Mohit!
> > >
> > > Cheers,
> > >
> > > Woonsan
> > >
> > > On Wed, Aug 14, 2019 at 2:31 AM Tommaso Teofili
> > >  wrote:
> > > >
> > > > Welcome to the team Mohit!
> > > >
> > > > Regards,
> > > > Tommaso
> > > >
> > > > On Thu, 8 Aug 2019 at 08:33, Marcel Reutegger 
> > wrote:
> > > >>
> > > >> Hi,
> > > >>
> > > >> Please welcome Mohit Kataria as a new committer and PMC member of
> > > >> the Apache Jackrabbit project. The Jackrabbit PMC recently decided to
> > > >> offer Mohit committership based on his contributions. I'm happy to
> > > >> announce that he accepted the offer and that all the related
> > > >> administrative work has now been taken care of.
> > > >>
> > > >> Welcome to the team, Mohit!
> > > >>
> > > >> Regards
> > > >>  Marcel
> > > >>
> >


Re: New Jackrabbit Committer: Nitin Gupta

2019-08-21 Thread Chetan Mehrotra
Welcome Nitin!

Chetan Mehrotra

On Thu, Aug 15, 2019 at 7:26 PM Matt Ryan  wrote:
>
> Welcome Nitin!
>
>
> -MR
>
> On Thu, Aug 15, 2019 at 6:51 AM Julian Sedding  wrote:
>>
>> Welcome Nitin!
>>
>> Regards
>> Julian
>>
>> On Wed, Aug 14, 2019 at 3:24 PM Woonsan Ko  wrote:
>> >
>> > Welcome, Nitin!
>> >
>> > Cheers,
>> >
>> > Woonsan
>> >
>> > On Wed, Aug 14, 2019 at 2:30 AM Tommaso Teofili
>> >  wrote:
>> > >
>> > > Welcome to the team Nitin!
>> > >
>> > > Regards,
>> > > Tommaso
>> > >
>> > > On Thu, 8 Aug 2019 at 08:31, Marcel Reutegger  wrote:
>> > >>
>> > >> Hi,
>> > >>
>> > >> Please welcome Nitin Gupta as a new committer and PMC member of
>> > >> the Apache Jackrabbit project. The Jackrabbit PMC recently decided to
>> > >> offer Nitin committership based on his contributions. I'm happy to
>> > >> announce that he accepted the offer and that all the related
>> > >> administrative work has now been taken care of.
>> > >>
>> > >> Welcome to the team, Nitin!
>> > >>
>> > >> Regards
>> > >>  Marcel
>> > >>


Re: [DISCUSS] Enabling CI for Oak cloud-based features

2018-07-31 Thread Chetan Mehrotra
For S3 there are quite a few options and one suggested by Amit should
be good for our usecase. For Azure specific services we would need to
get an account created and use that from our Jenkins server.

Chetan Mehrotra

On Tue, Jul 31, 2018 at 1:45 PM Amit Jain  wrote:
>
> https://jclouds.apache.org/start/blobstore/
>
> On Tue, Jul 31, 2018 at 1:43 PM Amit Jain  wrote:
>
> > I don't have too much details but jclouds has some modules to persist to
> > filesystem and memory with the same API they have for other cloud backends.
> >
> >
> > On Tue, Jul 31, 2018 at 1:38 PM Michael Dürig  wrote:
> >
> >>
> >> I wonder how other communities address such issues. I.e. Apache jclouds
> >> would need to be tested against a rather broad variety of different
> >> cloud vendors. Maybe its worth to do some research on their approach.
> >>
> >> Michael
> >>
> >>
> >> On 31.07.18 10:01, Amit Jain wrote:
> >> > There's one provided by Adobe as well - https://github.com/adobe/S3Mock
> >> > But these would have to be enhanced to support the upload/download urls
> >> > which I don't think these support. Also, I am not aware of any similar
> >> > utility for Azure.
> >> >
> >> > Thanks
> >> > Amit
> >> >
> >> > On Tue, Jul 31, 2018 at 1:24 PM Julian Reschke 
> >> > wrote:
> >> >
> >> >> On 2018-07-30 19:26, Matt Ryan wrote:
> >> >>> Hi,
> >> >>>
> >> >>> Oak now has a fair few cloud-based modules - meaning, modules that
> >> enable
> >> >>> Oak to make use of cloud service provider capabilities in order for
> >> the
> >> >>> feature to work - among them being oak-blob-cloud,
> >> oak-blob-cloud-azure,
> >> >>> and oak-segment-azure.
> >> >>>
> >> >>> I’m not as familiar with oak-segment-azure, but I do know for
> >> >>> oak-blob-cloud and oak-blob-cloud-azure you need an environment set
> >> up to
> >> >>> run the tests including credentials for the corresponding cloud
> >> service
> >> >>> provider.  The consequence of this is that there is no regular CI
> >> testing
> >> >>> run on these modules, IIUC.
> >> >>>
> >> >>> I wanted to kick off a discussion to see what everyone else thinks.  I
> >> >>> think coming up with some form of mock for the cloud objects might be
> >> >> nice,
> >> >>> or even better to use existing Apache-license-friendly ones if there
> >> are
> >> >>> some, but maybe others have already gone down this road further or
> >> have
> >> >>> better ideas?
> >> >>> ...
> >> >>
> >> >> FWIW; this has been concerning me as well for quite some time.
> >> >>
> >> >> Maybe it's worth trying out <https://github.com/findify/s3mock>?
> >> >>
> >> >> Best regards, Julian
> >> >>
> >> >
> >>
> >


Re: oak-search module

2018-04-04 Thread Chetan Mehrotra
+1. In addition we should also include common set of test case which
can be used to validate the SPI implementations. Also we can leave
oak-lucene as is for now and just create new module and implement
oak-lucene-v2 based on that. Once it reaches feature parity we can
remove oak-lucene bundle
Chetan Mehrotra


On Wed, Apr 4, 2018 at 5:43 PM, Thomas Mueller
<muel...@adobe.com.invalid> wrote:
> +1
>
> On 04.04.18, 10:23, "Tommaso Teofili" <tommaso.teof...@gmail.com> wrote:
>
> Hi all,
>
> In the context of creating an (abstract) implementation for Oak full text
> indexes [1], I'd like to create a new module called _oak-search_.
> Such module will contain:
> - implementation agnostic utilities for full text search (e.g. aggregation
> utilities)
> - implementation agnostic SPIs to be extended by implementors (currently 
> we
> expose SPIs in oak-lucene whose signatures include Lucene specific APIs)
> - abstract full text editor / query index implementations
> - text extraction utilities
>
> Please share your feedback / opinions / concerns.
>
> Regards,
> Tommaso
>
> [1] : https://issues.apache.org/jira/browse/OAK-3336
>
>


Re: Jackrabbit oak: Want these two (OAK-7304 and OAK-7265) fixes in 1.8.3 release.

2018-03-22 Thread Chetan Mehrotra
OAK-7304 is now backported. As Marcel and Torgeir suggested for now
you can build from source untill 1.8.3 is  released
Chetan Mehrotra


On Thu, Mar 22, 2018 at 1:46 PM, Marcel Reutegger <mreut...@adobe.com> wrote:
> Hi,
>
> On 21.03.18 13:43, rajesh wrote:
>>
>> I am using jackrabbit oak (1.8.2 ) in my spring boot application. Even
>> though we are not using OSGI environment, we are using oak-pojosr packages
>> which uses the OSGI kind of environment  to configure the FileDataStore,
>> S3DataStore and AzureDataStore etc. (Since documentation and examples are
>> using OSGI kind of things only.)  I need the following two fixes
>> immediately
>> in the next realease ( 1.8.3 ) so that i can proceed further.
>>
>> https://issues.apache.org/jira/browse/OAK-7304
>
>
> This issue is about deploying the release to the maven repository. You can
> build and deploy this module from the source release yourself.
>
>> https://issues.apache.org/jira/browse/OAK-7265.
>
>
> Please note, this is only an example and not meant to be used as is. Feel
> free to copy the example and adjust to your needs.
>
> Regards
>  Marcel


Re: [SegmentStore] Blobs under 16 KB always inlined in tar files?

2018-02-15 Thread Chetan Mehrotra
See OAK-6911
Chetan Mehrotra


On Fri, Feb 16, 2018 at 2:36 AM, Alexander Klimetschek
<aklim...@adobe.com.invalid> wrote:
> Hi,
>
> it seems the segment store will inline any binary blob up to ~16KB in the tar 
> files and not store them in the BlobStore [1]. The 16 KB limit 
> (Segment.MEDIUM_LIMIT) is hardcoded and not configurable.
>
> I can see this in action when debugging and when looking at an S3 datastore 
> of a full Oak segment + s3 ds installation, where the smallest binaries in S3 
> are 16 + something KB.
>
> As Ian pointed out:
>
>> This could bloat Tar files, impact memory mapping, and may be a major 
>> consumer of RAM for TarMK mmap mode, but I don't know TarMK well enough to 
>> know the logic behind doing that. The OS Disk cache is the correct place to 
>> deal with any file over 1 block in size, especially if its accessed 
>> sporadically.
>
> I would agree on first sight. However, there might be good reasons for the 
> current design and these concerns would not be true in practice. The same 
> setting is essentially used for both STRING and BINARY properties - maybe it 
> makes a lot of sense for Strings, but not so much for immutable binaries?
>
> Could someone shed some light?
>
> IIUC, it also makes the minRecordLength config [3] of the datastore(s) have 
> no effect, since that should probably be rather low (default is 100 bytes), 
> given it encodes the binary in the blob id itself. But since only binaries 
> larger than 16KB will ever reach the blob store (for a segment store setup), 
> all binaries will effectively always be larger than minRecordLength.
>
> [1] 
> https://github.com/apache/jackrabbit-oak/blob/58fdaf0dc0786f4cc9e39e7d26684fda04b32e78/oak-segment-tar/src/main/java/org/apache/jackrabbit/oak/segment/DefaultSegmentWriter.java#L648
> [2] 
> https://github.com/apache/jackrabbit-oak/blob/trunk/oak-segment-tar/src/main/java/org/apache/jackrabbit/oak/segment/Segment.java#L111-L118
> [3] https://jackrabbit.apache.org/oak/docs/osgi_config.html
>
> Cheers,
> Alex


Intent to backport to 1.8: OAK-7147

2018-01-11 Thread Chetan Mehrotra
Need to backport OAK-7147 - Oak run LuceneIndexer indexes excluded parent nodes

regards
Chetan Mehrotra


Re: Intent to backport to 1.8: OAK-7060

2018-01-11 Thread Chetan Mehrotra
+1
Chetan Mehrotra


On Thu, Jan 11, 2018 at 6:18 PM, Julian Reschke <julian.resc...@gmx.de> wrote:
> <https://issues.apache.org/jira/browse/OAK-7060>:
> "RDBDocumentStore.getStats() for SQLServer"
>
> (Due to other priorities this had to wait until today, but I'd like 1.8 to
> ship with that detail)
>
> Best regards, Julian


Re: [windows] Jenkins test failure - o.a.j.o.index.indexer.document.flatfile.FlatFileStoreTest.basicTest

2017-12-22 Thread Chetan Mehrotra
Thanks Vikas for the fix!
Chetan Mehrotra


On Fri, Dec 22, 2017 at 2:14 PM, Robert Munteanu <romb...@apache.org> wrote:
> On Fri, 2017-12-22 at 06:07 +0530, Vikas Saurabh wrote:
>> Hi Robert,
>>
>> > I am not able to qualify whether this is a valid failure or not,
>> > therefore I'm bringing it to the mailing list.
>> >
>>
>> That should be independent of OS... for me, following failed:
>> MAVEN_OPTS='-Xmx512m -Xms512m' mvn test -pl :oak-run
>> -Dtest=FlatFileStoreTest#basicTest
>>
>> Opened OAK-7108 and fixed in r1818991.
>
>
> Thanks! Windows build is now passing again.
>
> Robert


Re: Intent to backport OAK-5317

2017-12-21 Thread Chetan Mehrotra
+1
Chetan Mehrotra


On Thu, Dec 21, 2017 at 4:57 PM, Marcel Reutegger
<mreut...@adobe.com.invalid> wrote:
> Hi,
>
> I'd like to backport OAK-5317 to the maintenance branches. It is an 
> improvement, but would allow users to run a DocumentNodeStore on more recent 
> MongoDB version. I consider the change low risk, because the index is there 
> anyway.
>
> Regards
>  Marcel
>


Re: [m12n] Module oak-spi-core defines no export versions

2017-11-15 Thread Chetan Mehrotra
> And I definitely hope that we after that can stop making incompatible
changes as that separations allows us to stop exporting things that are
meant to be internal

There are few packages which I think are exported mostly to allow
other modules in Oak to work and as such are not like api to be used
by end users (like spi.mount, spi.gc and spi.stats). Enforcing full
backward compatibility for all spi package would hinder further
evolution of design going forward.

So would suggest to reconsider this and only version those which are
meant to be used by users outside of Oak modules
Chetan Mehrotra


On Tue, Nov 14, 2017 at 11:00 PM, Angela Schreiber
<anch...@adobe.com.invalid> wrote:
> Hi Chetan
>
> Well... I would have excepted that one goal of the m12n was to get a clear
> separation between public API and internals and everything that we target
> as API/SPI should be public IMO.
>
> And I definitely hope that we after that can stop making incompatible
> changes as that separations allows us to stop exporting things that are
> meant to be internal...
>
> Angela
>
> On 14/11/17 18:10, "Chetan Mehrotra" <chetan.mehro...@gmail.com> wrote:
>
>>Do we want to have explicit version for all packages in oak-core-spi
>>or should we only do it for packages which we expect code outside of
>>Oak codebase would be using? As once we version it we cannot change in
>>backward incompatible way easily
>>Chetan Mehrotra
>>
>>
>>On Tue, Nov 14, 2017 at 10:05 PM, Angela Schreiber
>><anch...@adobe.com.invalid> wrote:
>>> Hi Robert
>>>
>>> Ok... I will add 1.0.0 and go ahead tomorrow unless someone objects.
>>>
>>> Kind regards
>>> Angela
>>>
>>> On 14/11/17 17:23, "Robert Munteanu" <romb...@apache.org> wrote:
>>>
>>>>On Tue, 2017-11-14 at 13:51 +, Angela Schreiber wrote:
>>>>> Any preference wrt the initial version number?
>>>>
>>>>The initial version number should be 1.0.0 IMO.
>>>>
>>>>Robert
>>>
>


Re: [m12n] Module oak-spi-core defines no export versions

2017-11-14 Thread Chetan Mehrotra
Do we want to have explicit version for all packages in oak-core-spi
or should we only do it for packages which we expect code outside of
Oak codebase would be using? As once we version it we cannot change in
backward incompatible way easily
Chetan Mehrotra


On Tue, Nov 14, 2017 at 10:05 PM, Angela Schreiber
<anch...@adobe.com.invalid> wrote:
> Hi Robert
>
> Ok... I will add 1.0.0 and go ahead tomorrow unless someone objects.
>
> Kind regards
> Angela
>
> On 14/11/17 17:23, "Robert Munteanu" <romb...@apache.org> wrote:
>
>>On Tue, 2017-11-14 at 13:51 +, Angela Schreiber wrote:
>>> Any preference wrt the initial version number?
>>
>>The initial version number should be 1.0.0 IMO.
>>
>>Robert
>


Re: Intent to backport OAK-6750 and OAK-6792 to 1.6 and 1.4

2017-11-01 Thread Chetan Mehrotra
+1
Chetan Mehrotra


On Thu, Nov 2, 2017 at 9:26 AM, Vikas Saurabh <vikas.saur...@gmail.com> wrote:
> Hi,
>
> Oak's facet implementation had these two issues [0], [1]. I would like
> to backport those to 1.6 and 1.4 branch.
>
> Thanks,
> Vikas
>
>
> [0]: OAK-6750 - Facet for relative properties not working
> [1]: OAK-6792 - xpath support for facet queries


Re: BUILD FAILURE: Jackrabbit Oak - Build # 919 - Failure

2017-10-27 Thread Chetan Mehrotra
Now build should pass post 1813545
Chetan Mehrotra


On Fri, Oct 27, 2017 at 8:22 PM, Chetan Mehrotra
<chetan.mehro...@gmail.com> wrote:
> Oops ... Looks like this impacts whole build. Would look into that
> Chetan Mehrotra
>
>
> On Fri, Oct 27, 2017 at 7:54 PM, Christian Schneider
> <ch...@die-schneider.net> wrote:
>> Hi Chetan,
>>
>> there are more similar failures. The build then breaks for me
>> at oak-upgrade.
>>
>> .e.g:
>>
>> [*INFO*] Running org.apache.jackrabbit.oak.upgrade.cli.*Jcr2ToSegmentTest*
>>
>> [*ERROR*] *Tests **run: 2*, Failures: 0, *Errors: 2*, Skipped: 0, Time
>> elapsed: 1.657 s* <<< FAILURE!* - in org.apache.jackrabbit.oak.upgrade.cli.
>> *Jcr2ToSegmentTest*
>>
>> [*ERROR*]
>> validateMigration(org.apache.jackrabbit.oak.upgrade.cli.Jcr2ToSegmentTest)
>>  Time
>> elapsed: 1.657 s  <<< ERROR!
>>
>> java.lang.RuntimeException: javax.jcr.RepositoryException: Failed to copy
>> content
>>
>> at
>> org.apache.jackrabbit.oak.upgrade.cli.Jcr2ToSegmentTest.prepare(Jcr2ToSegmentTest.java:56)
>>
>> Caused by: javax.jcr.RepositoryException: Failed to copy content
>>
>> at
>> org.apache.jackrabbit.oak.upgrade.cli.Jcr2ToSegmentTest.prepare(Jcr2ToSegmentTest.java:56)
>>
>> Caused by: java.lang.NullPointerException: QueryIndexProvider yet not
>> initialized
>>
>> at
>> org.apache.jackrabbit.oak.upgrade.cli.Jcr2ToSegmentTest.prepare(Jcr2ToSegmentTest.java:56)
>>
>> Christian
>>
>> 2017-10-27 15:42 GMT+02:00 Chetan Mehrotra <chetan.mehro...@gmail.com>:
>>
>>> Fixed the test now with 1813536
>>> Chetan Mehrotra
>>>
>>>
>>> On Fri, Oct 27, 2017 at 6:37 PM, Apache Jenkins Server
>>> <jenk...@builds.apache.org> wrote:
>>> > The Apache Jenkins build system has built Jackrabbit Oak (build #919)
>>> >
>>> > Status: Failure
>>> >
>>> > Check console output at https://builds.apache.org/job/
>>> Jackrabbit%20Oak/919/ to view the results.
>>> >
>>> > Changes:
>>> > [frm] OAK-6829 - Make the configuration of NetworkErrorProxy immutable
>>> >
>>> > [frm] OAK-6829 - Let the primary and the standby close the connection if
>>> an exception is caught
>>> >
>>> > [chetanm] OAK-6873 - UserInitializer should not use hard coded
>>> QueryIndexProvider
>>> >
>>> >
>>> >
>>> > Test results:
>>> > 1 tests failed.
>>> > FAILED:  org.apache.jackrabbit.oak.segment.InitializerTest.
>>> testInitializerSegment
>>> >
>>> > Error Message:
>>> > QueryIndexProvider yet not initialized
>>> >
>>> > Stack Trace:
>>> > java.lang.NullPointerException: QueryIndexProvider yet not initialized
>>> > at org.apache.jackrabbit.oak.segment.InitializerTest.
>>> testInitializerSegment(InitializerTest.java:51)
>>>
>>
>>
>>
>> --
>> --
>> Christian Schneider
>> http://www.liquid-reality.de
>> <https://owa.talend.com/owa/redir.aspx?C=3aa4083e0c744ae1ba52bd062c5a7e46=http%3a%2f%2fwww.liquid-reality.de>
>>
>> Computer Scientist
>> http://www.adobe.com


Re: BUILD FAILURE: Jackrabbit Oak - Build # 919 - Failure

2017-10-27 Thread Chetan Mehrotra
Oops ... Looks like this impacts whole build. Would look into that
Chetan Mehrotra


On Fri, Oct 27, 2017 at 7:54 PM, Christian Schneider
<ch...@die-schneider.net> wrote:
> Hi Chetan,
>
> there are more similar failures. The build then breaks for me
> at oak-upgrade.
>
> .e.g:
>
> [*INFO*] Running org.apache.jackrabbit.oak.upgrade.cli.*Jcr2ToSegmentTest*
>
> [*ERROR*] *Tests **run: 2*, Failures: 0, *Errors: 2*, Skipped: 0, Time
> elapsed: 1.657 s* <<< FAILURE!* - in org.apache.jackrabbit.oak.upgrade.cli.
> *Jcr2ToSegmentTest*
>
> [*ERROR*]
> validateMigration(org.apache.jackrabbit.oak.upgrade.cli.Jcr2ToSegmentTest)
>  Time
> elapsed: 1.657 s  <<< ERROR!
>
> java.lang.RuntimeException: javax.jcr.RepositoryException: Failed to copy
> content
>
> at
> org.apache.jackrabbit.oak.upgrade.cli.Jcr2ToSegmentTest.prepare(Jcr2ToSegmentTest.java:56)
>
> Caused by: javax.jcr.RepositoryException: Failed to copy content
>
> at
> org.apache.jackrabbit.oak.upgrade.cli.Jcr2ToSegmentTest.prepare(Jcr2ToSegmentTest.java:56)
>
> Caused by: java.lang.NullPointerException: QueryIndexProvider yet not
> initialized
>
> at
> org.apache.jackrabbit.oak.upgrade.cli.Jcr2ToSegmentTest.prepare(Jcr2ToSegmentTest.java:56)
>
> Christian
>
> 2017-10-27 15:42 GMT+02:00 Chetan Mehrotra <chetan.mehro...@gmail.com>:
>
>> Fixed the test now with 1813536
>> Chetan Mehrotra
>>
>>
>> On Fri, Oct 27, 2017 at 6:37 PM, Apache Jenkins Server
>> <jenk...@builds.apache.org> wrote:
>> > The Apache Jenkins build system has built Jackrabbit Oak (build #919)
>> >
>> > Status: Failure
>> >
>> > Check console output at https://builds.apache.org/job/
>> Jackrabbit%20Oak/919/ to view the results.
>> >
>> > Changes:
>> > [frm] OAK-6829 - Make the configuration of NetworkErrorProxy immutable
>> >
>> > [frm] OAK-6829 - Let the primary and the standby close the connection if
>> an exception is caught
>> >
>> > [chetanm] OAK-6873 - UserInitializer should not use hard coded
>> QueryIndexProvider
>> >
>> >
>> >
>> > Test results:
>> > 1 tests failed.
>> > FAILED:  org.apache.jackrabbit.oak.segment.InitializerTest.
>> testInitializerSegment
>> >
>> > Error Message:
>> > QueryIndexProvider yet not initialized
>> >
>> > Stack Trace:
>> > java.lang.NullPointerException: QueryIndexProvider yet not initialized
>> > at org.apache.jackrabbit.oak.segment.InitializerTest.
>> testInitializerSegment(InitializerTest.java:51)
>>
>
>
>
> --
> --
> Christian Schneider
> http://www.liquid-reality.de
> <https://owa.talend.com/owa/redir.aspx?C=3aa4083e0c744ae1ba52bd062c5a7e46=http%3a%2f%2fwww.liquid-reality.de>
>
> Computer Scientist
> http://www.adobe.com


Re: BUILD FAILURE: Jackrabbit Oak - Build # 919 - Failure

2017-10-27 Thread Chetan Mehrotra
Fixed the test now with 1813536
Chetan Mehrotra


On Fri, Oct 27, 2017 at 6:37 PM, Apache Jenkins Server
<jenk...@builds.apache.org> wrote:
> The Apache Jenkins build system has built Jackrabbit Oak (build #919)
>
> Status: Failure
>
> Check console output at https://builds.apache.org/job/Jackrabbit%20Oak/919/ 
> to view the results.
>
> Changes:
> [frm] OAK-6829 - Make the configuration of NetworkErrorProxy immutable
>
> [frm] OAK-6829 - Let the primary and the standby close the connection if an 
> exception is caught
>
> [chetanm] OAK-6873 - UserInitializer should not use hard coded 
> QueryIndexProvider
>
>
>
> Test results:
> 1 tests failed.
> FAILED:  
> org.apache.jackrabbit.oak.segment.InitializerTest.testInitializerSegment
>
> Error Message:
> QueryIndexProvider yet not initialized
>
> Stack Trace:
> java.lang.NullPointerException: QueryIndexProvider yet not initialized
> at 
> org.apache.jackrabbit.oak.segment.InitializerTest.testInitializerSegment(InitializerTest.java:51)


Re: svn commit: r1813193 - /jackrabbit/oak/trunk/oak-lucene/src/main/java/org/apache/jackrabbit/oak/plugins/index/lucene/IndexDefinition.java

2017-10-25 Thread Chetan Mehrotra
> +String mmp = System.getProperty("cmmp");

Better to use system property like "oak.lucene.cmmp"
Chetan Mehrotra


On Tue, Oct 24, 2017 at 9:56 PM,  <tomm...@apache.org> wrote:
> Author: tommaso
> Date: Tue Oct 24 16:26:05 2017
> New Revision: 1813193
>
> URL: http://svn.apache.org/viewvc?rev=1813193=rev
> Log:
> OAK-6710 - possibly set mitigating merce policy via vm options
>
> Modified:
> 
> jackrabbit/oak/trunk/oak-lucene/src/main/java/org/apache/jackrabbit/oak/plugins/index/lucene/IndexDefinition.java
>
> Modified: 
> jackrabbit/oak/trunk/oak-lucene/src/main/java/org/apache/jackrabbit/oak/plugins/index/lucene/IndexDefinition.java
> URL: 
> http://svn.apache.org/viewvc/jackrabbit/oak/trunk/oak-lucene/src/main/java/org/apache/jackrabbit/oak/plugins/index/lucene/IndexDefinition.java?rev=1813193=1813192=1813193=diff
> ==
> --- 
> jackrabbit/oak/trunk/oak-lucene/src/main/java/org/apache/jackrabbit/oak/plugins/index/lucene/IndexDefinition.java
>  (original)
> +++ 
> jackrabbit/oak/trunk/oak-lucene/src/main/java/org/apache/jackrabbit/oak/plugins/index/lucene/IndexDefinition.java
>  Tue Oct 24 16:26:05 2017
> @@ -1577,6 +1577,11 @@ public final class IndexDefinition imple
>  }
>
>  private MergePolicy createMergePolicy() {
> +String mmp = System.getProperty("cmmp");
> +if (mmp != null) {
> +return new CommitMitigatingTieredMergePolicy();
> +}
> +
>  String mergePolicyName = getOptionalValue(definition, 
> LuceneIndexConstants.MERGE_POLICY_NAME, null);
>  MergePolicy mergePolicy = null;
>  if (mergePolicyName != null) {
>
>


Re: Intent to backport to 1.4: OAK-5878

2017-10-16 Thread Chetan Mehrotra
+1 given its already in 1.6.2
Chetan Mehrotra


On Mon, Oct 16, 2017 at 9:47 PM, Julian Reschke <julian.resc...@gmx.de> wrote:
> https://issues.apache.org/jira/browse/OAK-5878
>
> - already in 1.6.2
> - affects only users with RDB persistence
> - avoids one unnecessary repo scans for VersionGC
>
> Best regards, Julian


Re: Intent to backport OAK-6218

2017-10-10 Thread Chetan Mehrotra
+1
Chetan Mehrotra


On Tue, Oct 10, 2017 at 8:22 AM, Julian Reschke <julian.resc...@gmx.de> wrote:
> On 2017-10-10 17:16, Marcel Reutegger wrote:
>>
>> Hi,
>>
>> I'd like to backport OAK-6218 to the maintenance branches. It adds
>> additional information to a DocumentStoreException about IDs of affected
>> documents. As such it is not a bug fix, but very useful when analyzing what
>> went wrong. I consider it a low risk change.
>
>
> +1
>


Re: Cold standby for oak-benchmarks: option vs fixture

2017-10-03 Thread Chetan Mehrotra
Fixture looks better option here!
Chetan Mehrotra


On Mon, Oct 2, 2017 at 8:09 AM, Andrei Dulceanu
<andrei.dulce...@gmail.com> wrote:
> Hi,
>
> With OAK-6615 [0] we'd like to lay the foundation for including cold
> standby among features which could be included in a benchmark. This means
> that we'd like to have a cold standby that will sync with the primary every
> N seconds while the benchmarks only run on the primary. Initially the plan
> was to have this as an option, --cold-standby [N] for Oak-Segment-Tar*
> fixtures.
>
> Looking more carefully through the code in oak-benchmarks and
> oak-run-commons, I was thinking about implementing this as a new fixture.
> This could work similarly to Oak-Segment-Tar-DS and have dedicated options
> like --no-data-store, --private-data-store or --shared-data-store. It would
> allow us to better setup primary and standby instances (will also cover the
> shared data store use case, left uncovered by --cold-standby option).
>
> What do you think about this? Should we go with an option or a fixture?
>
> Thanks,
> Andrei
>
> [0] https://issues.apache.org/jira/browse/OAK-6615


Re: Debugging oak-pojosr test failures

2017-09-27 Thread Chetan Mehrotra
You need to do two changes

1. Set ds.loglevel=4 as you have done
2. Configure logger corresponding to bundle symbolic name to trace
level. Do it for the bundles you are interested in



> BTW, did you ever try to set up the web console? I would give it a shot.

Not here but the standalone example do configure the Felix WebConsole
[1]. You can try if standalone comes up with your changes to see it
works for basic case

Chetan Mehrotra
[1] 
https://github.com/apache/jackrabbit-oak/blob/trunk/oak-examples/standalone/src/main/java/org/apache/jackrabbit/oak/standalone/WebConsoleSupport.java


On Wed, Sep 27, 2017 at 5:18 PM, Robert Munteanu <romb...@apache.org> wrote:
> Hi Chetan,
>
> On Wed, 2017-09-27 at 15:48 +0530, Chetan Mehrotra wrote:
>> Hi Robert,
>>
>> The unit-test.log should give details about which bundle is starting
>> and which components are activated like [1]. For the error you are
>> seeing it means RepositoryManager is not getting activated for some
>> reason (may be some missing dependency).
>>
>> So we would need to analyze from OSGi side to see which component has
>> dependency missing. This is bit tricky in absence of Felix
>> WebConsole.
>> Probably enabling SCR logs (ds.loglevel=3?) would give some insight
>> here
>
> How would I do that? I tried the following change but got no extra
> logs:
>
> diff --git a/oak-
> pojosr/src/test/groovy/org/apache/jackrabbit/oak/run/osgi/AbstractRepos
> itoryFactoryTest.groovy b/oak-
> pojosr/src/test/groovy/org/apache/jackrabbit/oak/run/osgi/AbstractRepos
> itoryFactoryTest.groovy
> index e0d08b5025..91f39bc277 100644
> --- a/oak-
> pojosr/src/test/groovy/org/apache/jackrabbit/oak/run/osgi/AbstractRepos
> itoryFactoryTest.groovy
> +++ b/oak-
> pojosr/src/test/groovy/org/apache/jackrabbit/oak/run/osgi/AbstractRepos
> itoryFactoryTest.groovy
> @@ -57,6 +57,7 @@ abstract class AbstractRepositoryFactoryTest{
>  config = [
>  (REPOSITORY_HOME): workDir.absolutePath,
>  (REPOSITORY_TIMEOUT_IN_SECS) : 60,
> +    "ds.loglevel" : 3
>  ]
>  }
>
> BTW, did you ever try to set up the web console? I would give it a
> shot.
>
> Thanks,
>
> Robert
>
>>
>> Chetan Mehrotra
>> [1]
>> 27.09.2017 15:43:06.603 *INFO* [main]
>> org.apache.jackrabbit.oak-query-spi BundleEvent STARTED
>> 27.09.2017 15:43:06.603 *INFO* [main] org.apache.jackrabbit.oak-core
>> BundleEvent STARTING
>> 27.09.2017 15:43:06.761 *INFO* [main] org.apache.jackrabbit.oak-core
>> Service
>> [org.apache.jackrabbit.oak.spi.security.user.action.DefaultAuthorizab
>> leActionProvider,17,
>> [org.apache.jackrabbit.oak.spi.security.user.action.AuthorizableActio
>> nProvider]]
>> ServiceEvent REGISTERED
>> 27.09.2017 15:43:06.762 *INFO* [main] org.apache.jackrabbit.oak-core
>> Service
>> [org.apache.jackrabbit.oak.security.authorization.restriction.Restric
>> tionProviderImpl,18,
>> [org.apache.jackrabbit.oak.spi.security.authorization.restriction.Res
>> trictionProvider]]
>> ServiceEvent REGISTERED
>> 27.09.2017 15:43:06.770 *INFO* [main]
>> org.apache.jackrabbit.oak.plugins.metric.StatisticsProviderFactory
>> Using MetricsStatisticsProvider
>> 27.09.2017 15:43:06.804 *INFO* [main] org.apache.jackrabbit.oak-core
>> Service [19, [com.codahale.metrics.MetricRegistry]] ServiceEvent
>> REGISTERED
>> 27.09.2017 15:43:06.825 *INFO* [main]
>> org.apache.jackrabbit.oak.segment.file.FileStore Creating file store
>> FileStoreBuilder{version=1.8-SNAPSHOT,
>> directory=/home/chetanm/git/apache/jackrabbit-oak/oak-
>> pojosr/target/junit5333968922356600629/repository/segmentstore,
>> blobStore=null, maxFileSize=256, segmentCacheSize=256,
>> stringCacheSize=256, templateCacheSize=64,
>> stringDeduplicationCacheSize=15000,
>> templateDeduplicationCacheSize=3000,
>> nodeDeduplicationCacheSize=1048576, memoryMapping=true,
>> gcOptions=SegmentGCOptions{paused=false, estimationDisabled=false,
>> gcSizeDeltaEstimation=1073741824, retryCount=5, forceTimeout=60,
>> retainedGenerations=2, gcType=FULL}}
>> 27.09.2017 15:43:07.070 *INFO* [main]
>> org.apache.jackrabbit.oak.segment.file.FileStore TarMK opened:
>> /home/chetanm/git/apache/jackrabbit-oak/oak-
>> pojosr/target/junit5333968922356600629/repository/segmentstore
>> (mmap=true)
>> 27.09.2017 15:43:07.135 *INFO* [main]
>> org.apache.jackrabbit.oak-segment-tar Service [21,
>> [org.apache.jackrabbit.oak.api.jmx.CacheStatsMBean]] ServiceEvent
>> REGISTERED
>> 27.09.2017 15:43:07.137 *INFO* [main]
>> org.apache.jackrabbit.o

Re: Debugging oak-pojosr test failures

2017-09-27 Thread Chetan Mehrotra
Hi Robert,

The unit-test.log should give details about which bundle is starting
and which components are activated like [1]. For the error you are
seeing it means RepositoryManager is not getting activated for some
reason (may be some missing dependency).

So we would need to analyze from OSGi side to see which component has
dependency missing. This is bit tricky in absence of Felix WebConsole.
Probably enabling SCR logs (ds.loglevel=3?) would give some insight
here

Chetan Mehrotra
[1]
27.09.2017 15:43:06.603 *INFO* [main]
org.apache.jackrabbit.oak-query-spi BundleEvent STARTED
27.09.2017 15:43:06.603 *INFO* [main] org.apache.jackrabbit.oak-core
BundleEvent STARTING
27.09.2017 15:43:06.761 *INFO* [main] org.apache.jackrabbit.oak-core
Service 
[org.apache.jackrabbit.oak.spi.security.user.action.DefaultAuthorizableActionProvider,17,
[org.apache.jackrabbit.oak.spi.security.user.action.AuthorizableActionProvider]]
ServiceEvent REGISTERED
27.09.2017 15:43:06.762 *INFO* [main] org.apache.jackrabbit.oak-core
Service 
[org.apache.jackrabbit.oak.security.authorization.restriction.RestrictionProviderImpl,18,
[org.apache.jackrabbit.oak.spi.security.authorization.restriction.RestrictionProvider]]
ServiceEvent REGISTERED
27.09.2017 15:43:06.770 *INFO* [main]
org.apache.jackrabbit.oak.plugins.metric.StatisticsProviderFactory
Using MetricsStatisticsProvider
27.09.2017 15:43:06.804 *INFO* [main] org.apache.jackrabbit.oak-core
Service [19, [com.codahale.metrics.MetricRegistry]] ServiceEvent
REGISTERED
27.09.2017 15:43:06.825 *INFO* [main]
org.apache.jackrabbit.oak.segment.file.FileStore Creating file store
FileStoreBuilder{version=1.8-SNAPSHOT,
directory=/home/chetanm/git/apache/jackrabbit-oak/oak-pojosr/target/junit5333968922356600629/repository/segmentstore,
blobStore=null, maxFileSize=256, segmentCacheSize=256,
stringCacheSize=256, templateCacheSize=64,
stringDeduplicationCacheSize=15000,
templateDeduplicationCacheSize=3000,
nodeDeduplicationCacheSize=1048576, memoryMapping=true,
gcOptions=SegmentGCOptions{paused=false, estimationDisabled=false,
gcSizeDeltaEstimation=1073741824, retryCount=5, forceTimeout=60,
retainedGenerations=2, gcType=FULL}}
27.09.2017 15:43:07.070 *INFO* [main]
org.apache.jackrabbit.oak.segment.file.FileStore TarMK opened:
/home/chetanm/git/apache/jackrabbit-oak/oak-pojosr/target/junit5333968922356600629/repository/segmentstore
(mmap=true)
27.09.2017 15:43:07.135 *INFO* [main]
org.apache.jackrabbit.oak-segment-tar Service [21,
[org.apache.jackrabbit.oak.api.jmx.CacheStatsMBean]] ServiceEvent
REGISTERED
27.09.2017 15:43:07.137 *INFO* [main]
org.apache.jackrabbit.oak-segment-tar Service [22,
[org.apache.jackrabbit.oak.api.jmx.CacheStatsMBean]] ServiceEvent
REGISTERED
27.09.2017 15:43:07.138 *INFO* [main]
org.apache.jackrabbit.oak-segment-tar Service [23,
[org.apache.jackrabbit.oak.api.jmx.CacheStatsMBean]] ServiceEvent
REGISTERED
27.09.2017 15:43:07.140 *INFO* [main]
org.apache.jackrabbit.oak-segment-tar Service [24,
[org.apache.jackrabbit.oak.api.jmx.CacheStatsMBean]] ServiceEvent
REGISTERED
27.09.2017 15:43:07.141 *INFO* [main]
org.apache.jackrabbit.oak-segment-tar Service [25,
[org.apache.jackrabbit.oak.api.jmx.CacheStatsMBean]] ServiceEvent
REGISTERED
27.09.2017 15:43:07.143 *INFO* [main]
org.apache.jackrabbit.oak-segment-tar Service [26,
[org.apache.jackrabbit.oak.api.jmx.CacheStatsMBean]] ServiceEvent
REGISTERED
27.09.2017 15:43:07.145 *INFO* [main]
org.apache.jackrabbit.oak-segment-tar Service [27,
[org.apache.jackrabbit.oak.spi.gc.GCMonitor]] ServiceEvent REGISTERED
27.09.2017 15:43:07.148 *INFO* [main]
org.apache.jackrabbit.oak-segment-tar Service [28,
[org.apache.jackrabbit.oak.segment.compaction.SegmentRevisionGC]]
ServiceEvent REGISTERED
27.09.2017 15:43:07.153 *INFO* [main]
org.apache.jackrabbit.oak-segment-tar Service [29,
[org.apache.jackrabbit.oak.spi.state.RevisionGCMBean]] ServiceEvent
REGISTERED
27.09.2017 15:43:07.153 *INFO* [main]
org.apache.jackrabbit.oak-segment-tar Service [30,
[org.apache.jackrabbit.oak.segment.file.FileStoreStatsMBean]]
ServiceEvent REGISTERED
27.09.2017 15:43:07.155 *INFO* [main]
org.apache.jackrabbit.oak.segment.SegmentNodeStore$SegmentNodeStoreBuilder
Creating segment node store SegmentNodeStoreBuilder{blobStore=inline}
27.09.2017 15:43:07.161 *INFO* [main]
org.apache.jackrabbit.oak.segment.scheduler.LockBasedScheduler
Initializing SegmentNodeStore with the commitFairLock option enabled.
27.09.2017 15:43:07.170 *INFO* [main]
org.apache.jackrabbit.oak-segment-tar Service [31,
[org.apache.jackrabbit.oak.api.jmx.CheckpointMBean]] ServiceEvent
REGISTERED
27.09.2017 15:43:07.183 *INFO* [main]
org.apache.jackrabbit.oak.spi.cluster.ClusterRepositoryInfo
getOrCreateId: created a new
clusterId=c696b381-f3e7-4826-ac5e-552f3ac4fecc
27.09.2017 15:43:07.184 *INFO* [main]
org.apache.jackrabbit.oak-segment-tar Service [32,
[org.apache.jackrabbit.oak.api.Descriptors]] ServiceEvent REGISTERED


Re: Oak Session save behavior

2017-09-26 Thread Chetan Mehrotra
> Now the problem we are running into is, after save operation is performed,
> there is no good way for us to know if save was done (Since save in oak is
> async). So after save we refresh the page, the user sees old content.

Saves in Oak are not async. On a single cluster node if session S1
commits then any session S2 created after S1 (on same cluster node)
would see its change or any existing session can invoke refresh and
see latest repository state (per that cluster node)

For a cluster setup yes changes done by a session on one cluster node
would not be immediately seen on other cluster node. For such setups
you should setup sticky connections for your server


Commit history missing for files moved to new modules in git-svn/github

2017-09-26 Thread Chetan Mehrotra
Looks like with svn move/copy the history in git is coming out to be
blank [2]. In svn it looks fine though [1].

Would it be possible to get git history back (via some other tooling
anyone aware of) as it greatly helps in understanding code evolution
while analyzing any issue quickly. Otherwise would need to work with
svn copy where checking history is slow.

For now following command shows history in git. But looks like
Intellij does not support this so its not possible to annotate [3]
file in IDE

git log -C --follow
oak-store-document/src/main/java/org/apache/jackrabbit/oak/plugins/document/DocumentNodeStore.java


Chetan Mehrotra

[1] 
https://svn.apache.org/viewvc/jackrabbit/oak/trunk/oak-store-document/src/main/java/org/apache/jackrabbit/oak/plugins/document/DocumentNodeStore.java?view=log
[2] 
https://github.com/apache/jackrabbit-oak/commits/trunk/oak-store-document/src/main/java/org/apache/jackrabbit/oak/plugins/document/DocumentNodeStore.java
[3] 
https://stackoverflow.com/questions/27797965/showing-graphically-the-equivalent-of-git-log-follow-in-intellij


Re: Access to NodeStore instance within benchmark

2017-09-26 Thread Chetan Mehrotra
Its bit buried in layers. I saw few other benchmark use reflection to
access the NodeStore. So for now using that to move forward


Chetan Mehrotra


On Tue, Sep 26, 2017 at 3:32 PM, Michael Dürig <mdue...@apache.org> wrote:
>
> I tend to agree with Davide. Especially since the use case is for
> benchmarking only. Isn't there an alternative way where the node store could
> be exposed via the the benchmark's fixture?
>
> Michael
>
>
> On 25.09.17 11:00, Davide Giannella wrote:
>>
>> On 25/09/2017 07:05, Chetan Mehrotra wrote:
>>>
>>> One way can be to expose NodeStore instance from the Oak class. Would
>>> that be ok to do?
>>
>>
>> I don't like it very much as it would make it too easy to access the
>> NodeStore from a consumer pov. `Oak` itself is already low level
>> therefore I'm not strongly objecting for adding such feature. However as
>> the Oak object accept the NodeStore already as part of the constructor I
>> would see if it is feasible to keep a reference to the NodeStore in the
>> AbstractTest itself and use it during repository construction.
>>
>> HTH
>> Davide
>>
>>
>


Access to NodeStore instance within benchmark

2017-09-25 Thread Chetan Mehrotra
For a new benchmark I need access to NodeStore associated with current
Oak instance to configure a cleaner instance [1].

Any idea on how can it be done? So far not able to find a way to
access the NodeStore instance in the benchmark [2].

One way can be to expose NodeStore instance from the Oak class. Would
that be ok to do?

Chetan Mehrotra
[1] 
https://github.com/chetanmeh/jackrabbit-oak/blob/OAK-6535/oak-lucene/src/main/java/org/apache/jackrabbit/oak/plugins/index/lucene/property/PropertyIndexCleaner.java#L64

[2] 
https://github.com/chetanmeh/jackrabbit-oak/blob/OAK-6535/oak-benchmarks/src/main/java/org/apache/jackrabbit/oak/benchmark/HybridIndexTest.java#L151


Re: Oak 1.7.8 release plan

2017-09-22 Thread Chetan Mehrotra
OAK-6635 is now resolved.


Chetan Mehrotra


On Fri, Sep 22, 2017 at 3:25 PM, Davide Giannella <dav...@apache.org> wrote:
> Hello team,
>
> I'm planning to cut Oak on Monday 25th.
>
> There's however a blocker which will block the release if not resolved
> or re-scheduled: https://issues.apache.org/jira/browse/OAK-6635
>
> If there are any objections please let me know. Otherwise I will
> re-schedule any non-resolved and non-blocking issue for the next iteration.
>
> Thanks
> Davide
>
>


Intent to backport OAK-6637

2017-09-20 Thread Chetan Mehrotra
I would like to backport OAK-6637 which improves the resilience of
locking and ensures any exception should not cause the lock to be lost
preventing further index updates

Chetan Mehrotra


Re: Intent to backport OAK-6656 to 1.6 and 1.4 branch

2017-09-13 Thread Chetan Mehrotra
Its was +0 ;)
Chetan Mehrotra


On Wed, Sep 13, 2017 at 2:15 PM, Vikas Saurabh <vikas.saur...@gmail.com> wrote:
> Hi Chetan,
>
> Was your concern a -1 or a +/- 0?
>
> Thanks,
> Vikas


Re: [IP CLEARANCE][VOTE] FileVault Package Maven Plugin Contribution

2017-09-11 Thread Chetan Mehrotra
+1 Accept FileVault Package Maven Plugin into Apache Jackrabbit

Chetan Mehrotra


On Tue, Sep 12, 2017 at 8:18 AM, Tobias Bocanegra <tri...@adobe.com> wrote:
> [ ] +1 Accept FileVault Package Maven Plugin into Apache Jackrabbit
>
>
> Regards, Toby


Re: Using JCR APIs in a Python script

2017-09-06 Thread Chetan Mehrotra
Hi Vaibhav,

JCR API are only supported on JVM. So you cannot use them from python.

If you have Sling based app then you can use the Sling REST support to
access repository content via HTTP. Or you can use Jackrabbit WebDAV
support to access same content via web dav calls over HTTP
Chetan Mehrotra


On Wed, Sep 6, 2017 at 9:32 PM, Vaibhav Kumar Sharma <vaish...@adobe.com> wrote:
> Hi Team,
>
>
>
> I was just curious to know how can I use the JCR API in my python script to
> perform operations on my repo?
>
>
>
> Is that possible? Or Is there an implementation for JCR API in python?
>
>
>
> Thanks in advance!!
>
>
> Regards,
>
> Vaibhav


Re: OAK-6575 - A word of caution

2017-09-04 Thread Chetan Mehrotra
> Why couldn't we discuss this implementation in the light of the OAK-1963?
> Why the old concerns are not taken in to consideration once again? We already 
> talked about all that.

I closed OAK-1963 because it was specially related to exposing the
File instance url while objective of OAK-6575 is to provide a secure
url. My reading of discussion done in OAK-1963 that people had concern
around exposing the File instance location managed by Oak. Adaptable
pattern in itself was not much discussed at that time. Hence it made
sense to close OAK-1963 and have a fresh discussion on OAK-6575

As for OAK-6575 the approach taken there has been discussed in detail
at [1] and in the end there was consensus on the proposed approach by
Ian.

Chetan Mehrotra
[1] http://markmail.org/thread/lhljp2ksegw7dcod

On Mon, Sep 4, 2017 at 5:25 PM, Francesco Mari <mari.france...@gmail.com> wrote:
> The POC for OAK-6575 aims at introducing the AdapterManager and
> AdapterFactory API in Oak. This is just a step away from introducing a
> full-fledged implementation of Sling-like adapters in Oak. In fact,
> the OakConversionService is just an Adapter for a single, very
> specific conversion.
>
> We had a very similar conversation for OAK-1963 in a thread [1] in
> this mailing list. In that thread many people raised that concerns
> about adapters. Many of us, me included, see adapters as a glorified
> way of breaking data encapsulation and are not in favour of
> introducing them.
>
> After some quiet time in the thread and in OAK-1963, a new thread [2]
> was created, a new issue [3] was opened and a POC [4] was written. The
> content of the POC is little more than a permutation of what was
> already discussed in OAK-1963. Why a new thread and issue were needed?
> Why couldn't we discuss this implementation in the light of the
> OAK-1963? Why the old concerns are not taken into consideration once
> again? We already talked about all that.
>
> If the majority of the people here is alright in going forward with
> the aforementioned POC, so be it. But, at least, let's avoid closing
> threads and issues and create new ones that are just carbon copies.
> This might be misinterpreted as a way to clear concerns out of our
> way.
>
> [1]: http://markmail.org/thread/enmibgfwedypjnos
> [2]: 
> https://lists.apache.org/thread.html/2efb1da82b9b93a3a394c3abccbac960175ffc7a13facaac79c9181a@%3Coak-dev.jackrabbit.apache.org%3E
> [3]: https://issues.apache.org/jira/browse/OAK-6575
> [4]: 
> https://github.com/apache/jackrabbit-oak/compare/trunk...ieb:OAK-6575?expand=1


Re: Name node length limit in Oak 1.6 backed by MongoDB

2017-09-01 Thread Chetan Mehrotra
You can find it
https://jackrabbit.apache.org/oak/docs/differences.html#Node_Name_Length_Limit
Chetan Mehrotra


On Fri, Sep 1, 2017 at 2:55 PM, mouli ch <mouli.ocpjp...@gmail.com> wrote:
> Team,
>
> What is the node name length limit which is backed by MongoDB.
>
> Thanks
> Mouli Chintakunta


Intent to backport OAK-6339

2017-08-31 Thread Chetan Mehrotra
I would like to backport OAK-6339 to 1.6 branch. This change would be
backported to both segment implementation as its impacting collection
of repo stats for large repo via oakRepoStats script

Chetan Mehrotra


Re: svn commit: r1806602 - in /jackrabbit/oak/trunk/oak-jcr/src: main/java/org/apache/jackrabbit/oak/jcr/repository/ main/java/org/apache/jackrabbit/oak/jcr/session/ test/java/org/apache/jackrabbit/oa

2017-08-29 Thread Chetan Mehrotra
+
+Tracker tracker =
whiteboard.track(MountInfoProvider.class);
+List services = tracker.getServices();
+tracker.stop();
+
+if ( services.isEmpty() )
+this.mountInfoProvider = null;
+else if ( services.size() == 1 )
+this.mountInfoProvider = services.get(0);
+else
+throw new IllegalArgumentException("Found " +
services.size() + " MountInfoProvider references, expected at most
1.");

You can also use WhiteboardUtils#getService here which hides this stuff
Chetan Mehrotra


On Tue, Aug 29, 2017 at 7:48 AM,  <romb...@apache.org> wrote:
> Author: rombert
> Date: Tue Aug 29 14:48:56 2017
> New Revision: 1806602
>
> URL: http://svn.apache.org/viewvc?rev=1806602=rev
> Log:
> OAK-6563 - Session.hasCapability(...) should reflect read-only status of 
> mounts
>
> Add optional support for checking the Mount status in 
> SessionImpl.hasCapability.
>
> Added:
> 
> jackrabbit/oak/trunk/oak-jcr/src/test/java/org/apache/jackrabbit/oak/jcr/session/SessionImplCapabilityWithMountInfoProviderTest.java
> Modified:
> 
> jackrabbit/oak/trunk/oak-jcr/src/main/java/org/apache/jackrabbit/oak/jcr/repository/RepositoryImpl.java
> 
> jackrabbit/oak/trunk/oak-jcr/src/main/java/org/apache/jackrabbit/oak/jcr/session/SessionContext.java
> 
> jackrabbit/oak/trunk/oak-jcr/src/main/java/org/apache/jackrabbit/oak/jcr/session/SessionImpl.java
>
> Modified: 
> jackrabbit/oak/trunk/oak-jcr/src/main/java/org/apache/jackrabbit/oak/jcr/repository/RepositoryImpl.java
> URL: 
> http://svn.apache.org/viewvc/jackrabbit/oak/trunk/oak-jcr/src/main/java/org/apache/jackrabbit/oak/jcr/repository/RepositoryImpl.java?rev=1806602=1806601=1806602=diff
> ==
> --- 
> jackrabbit/oak/trunk/oak-jcr/src/main/java/org/apache/jackrabbit/oak/jcr/repository/RepositoryImpl.java
>  (original)
> +++ 
> jackrabbit/oak/trunk/oak-jcr/src/main/java/org/apache/jackrabbit/oak/jcr/repository/RepositoryImpl.java
>  Tue Aug 29 14:48:56 2017
> @@ -22,6 +22,7 @@ import static java.util.Collections.sing
>  import static 
> org.apache.jackrabbit.oak.spi.whiteboard.WhiteboardUtils.registerMBean;
>
>  import java.io.Closeable;
> +import java.util.List;
>  import java.util.Map;
>  import java.util.concurrent.Callable;
>  import java.util.concurrent.ScheduledExecutorService;
> @@ -61,8 +62,10 @@ import org.apache.jackrabbit.oak.jcr.ses
>  import org.apache.jackrabbit.oak.plugins.observation.CommitRateLimiter;
>  import org.apache.jackrabbit.oak.spi.gc.DelegatingGCMonitor;
>  import org.apache.jackrabbit.oak.spi.gc.GCMonitor;
> +import org.apache.jackrabbit.oak.spi.mount.MountInfoProvider;
>  import org.apache.jackrabbit.oak.spi.security.SecurityProvider;
>  import org.apache.jackrabbit.oak.spi.whiteboard.Registration;
> +import org.apache.jackrabbit.oak.spi.whiteboard.Tracker;
>  import org.apache.jackrabbit.oak.spi.whiteboard.Whiteboard;
>  import org.apache.jackrabbit.oak.stats.Clock;
>  import org.apache.jackrabbit.oak.stats.StatisticManager;
> @@ -105,6 +108,7 @@ public class RepositoryImpl implements J
>  private final Clock.Fast clock;
>  private final DelegatingGCMonitor gcMonitor = new DelegatingGCMonitor();
>  private final Registration gcMonitorRegistration;
> +private final MountInfoProvider mountInfoProvider;
>
>  /**
>   * {@link ThreadLocal} counter that keeps track of the save operations
> @@ -152,6 +156,17 @@ public class RepositoryImpl implements J
>  this.clock = new Clock.Fast(scheduledExecutor);
>  this.gcMonitorRegistration = whiteboard.register(GCMonitor.class, 
> gcMonitor, emptyMap());
>  this.fastQueryResultSize = fastQueryResultSize;
> +
> +Tracker tracker = 
> whiteboard.track(MountInfoProvider.class);
> +List services = tracker.getServices();
> +tracker.stop();
> +
> +if ( services.isEmpty() )
> +this.mountInfoProvider = null;
> +else if ( services.size() == 1 )
> +this.mountInfoProvider = services.get(0);
> +else
> +throw new IllegalArgumentException("Found " + services.size() + 
> " MountInfoProvider references, expected at most 1.");
>  }
>
>  //-< Repository 
> >---
> @@ -343,7 +358,7 @@ public class RepositoryImpl implements J
>  Map<String, Object> attributes, SessionDelegate delegate, int 
> observationQueueLength,
>  CommitRateLimiter commitRateLimiter) {
>  return new SessionContext(this, statisticManager, securityProvider, 
> w

Re: reading the checkpoint metadata in the SegmentMK

2017-08-28 Thread Chetan Mehrotra
OAK-6227 was meant to address similar requirement. May be we add this
to CheckpointMBean
Chetan Mehrotra


On Mon, Aug 28, 2017 at 9:40 PM, Michael Dürig <mdue...@apache.org> wrote:
>
> Hi,
>
> I would prefer to not make the
> org.apache.jackrabbit.oak.segment.SegmentNodeStore#getCheckpoints method
> public as this would bind us to a contract how to store the meta data. Maybe
> we could add some API that is agnostic to the storage format?
>
> Michael
>
>
> On 28.08.17 12:41, Tomek Rekawek wrote:
>>
>> Hello,
>>
>> the migration code requires access to the checkpoint metadata: the
>> creation and expiry timestamps. They can be read by accessing the
>> checkpoints root node (using the method mentioned in the subject). However,
>> the method is package-scoped. Can we make it public, so the other modules
>> can use it as well?
>>
>> Alternatively, we may introduce some general way to read that data for all
>> NodeStore implementations. Maybe some extra properties in the checkpoint
>> properties?
>>
>> Regards,
>> Tomek
>>
>>
>


Re: OAK-4348 - intent to add oak-lucene-mt

2017-08-25 Thread Chetan Mehrotra
This looks interesting!.

Is this bound to Lucene or can possibly be used for Solr also (if we
generalize FulltextQueryTermsProvider concept). If yes then we can
name the module as oak-search-mt
Chetan Mehrotra


On Fri, Aug 25, 2017 at 1:14 PM, Tommaso Teofili
<tommaso.teof...@gmail.com> wrote:
> Hi all,
>
> as part of OAK-4348 [1] some time ago I had developed an extension in
> oak-lucene to expand search terms using machine translated term, in order
> to provide cross language search capabilities to Oak (via
> FulltextQueryTermsProvider API [2]).
> In the context of a better modularized Oak, I was thinking to provide such
> an extension as a separate module (e.g. called oak-lucene-mt), which would
> be optional.
> In order to use it a language pack would need to be downloaded, for example
> from [3].
> Please let me know if there's any concern or feedback.
>
> Regards,
> Tommaso
>
> [1] : https://issues.apache.org/jira/browse/OAK-4348
> [2] :
> https://github.com/apache/jackrabbit-oak/blob/trunk/oak-lucene/src/main/java/org/apache/jackrabbit/oak/plugins/index/lucene/spi/FulltextQueryTermsProvider.java
> [3] : https://cwiki.apache.org/confluence/display/JOSHUA/Language+Packs


Re: OAK-6575 - Provide a secure external URL to a DataStore binary.

2017-08-24 Thread Chetan Mehrotra
> Inside Oak it would have its own version of an AdapterManager,
> AdapterFactory. the DataStore would implement an AdapterFactory and
> register it with the AdapterManager. The OakConversionService
> implementation would then use the AdapterManager to perform the conversion.
> If no AdapterFactory to adapt from JCR Binary to URI existed, then null
> would be returned from the OakConversionService.
>
> Thats no API changes to Blob, binary or anything. No complex transformation
> through multiple layers. No instanceof required and no difference between
> Sling and non Sling usage.
> It does require an Oak version of the AdapterManager and AdapterFactory
> concepts, but does not require anything to implement Adaptable.

Thanks for those details. Much clear now. So with this we need not add
adaptTo to all stuff. Instead provide an OakConversionService which
converts the Binary to provided type and then have DataStores provide
the AdapterFactory.

This would indeed avoid any new methods in existing objects and
provide a single entry point.

+1 for this approach

Chetan Mehrotra


On Thu, Aug 24, 2017 at 6:16 AM, Ian Boston <i...@tfd.co.uk> wrote:
> Hi,
> I am probably not helping as here as there are several layers and I think
> they are getting confused between what I am thinking and what you are
> thinking.
>
> I was thinking Oak exposed a service to convert along the lines of the OSCi
> converter service or the OakConversionService suggested earlier. Both Sling
> and other uses of Oak would use it.
>
> Inside Oak it would have its own version of an AdapterManager,
> AdapterFactory. the DataStore would implement an AdapterFactory and
> register it with the AdapterManager. The OakConversionService
> implementation would then use the AdapterManager to perform the conversion.
> If no AdapterFactory to adapt from JCR Binary to URI existed, then null
> would be returned from the OakConversionService.
>
> Thats no API changes to Blob, binary or anything. No complex transformation
> through multiple layers. No instanceof required and no difference between
> Sling and non Sling usage.
> It does require an Oak version of the AdapterManager and AdapterFactory
> concepts, but does not require anything to implement Adaptable.
>
> As I showed in the PoC, all the S3 specific implementation fits inside the
> S3DataStore which already does everything required to perform the
> conversion. It already goes from Binary -> Blob -> ContentIdentifier -> S3
> Key -> S3 URL by virtue of
> ValueImpl.getBlob((Value)jcrBinary).getContentIdentifier() -> convert to
> S3key and then signed URL.
>
> If it would help, I can do a patch to show how it works.
> Best Regards
> Ian
>
> On 24 August 2017 at 13:05, Chetan Mehrotra <chetan.mehro...@gmail.com>
> wrote:
>
>> > No API changes to any existing Oak APIs,
>>
>> Some API needs to be exposed. Note again Oak does not depend on Sling
>> API. Any such integration code is implemented in Sling Base module
>> [1]. But that module would still require some API in Oak to provide
>> such an adaptor
>>
>> The adaptor proposal here is for enabling layers within Oak to allow
>> conversion of JCR Binary instance to SignedBinary. Now how this is
>> exposed to end user depends on usage context
>>
>> Outside Sling
>> --
>>
>> Check if binary instanceof Oak Adaptable. If yes then cast it and adapt it
>>
>> import org.apache.jackrabbit.oak.api.Adaptable
>>
>> Binary b = ...
>> SignedBinary sb  = null
>> if (b instanceof Adaptable) {
>>sb = ((Adaptable)b).adaptTo(SignedBinary.class);
>> }
>
>
>> Within Sling
>> 
>>
>> Have an AdapterManager implemented in Sling JCR Base [1] which uses
>> above approach
>>
>> Chetan Mehrotra
>> [1] https://github.com/apache/sling/tree/trunk/bundles/jcr/base
>>
>>
>> On Thu, Aug 24, 2017 at 4:55 AM, Ian Boston <i...@tfd.co.uk> wrote:
>> > From the javadoc in [1]
>> >
>> > "The adaptable object may be any non-null object and is not required to
>> > implement the Adaptable interface."
>> >
>> >
>> > On 24 August 2017 at 12:54, Ian Boston <i...@tfd.co.uk> wrote:
>> >
>> >> Hi,
>> >> That would require javax.jcr.Binary to implement Adaptable, which it
>> cant.
>> >> (OakBinary could but it doesnt need to).
>> >>
>> >> Using Sling AdapterFactory/AdapterManger javadoc (to be replaced with
>> Oaks
>> >> internal version of the same)
>> >>
>> >> What is needed is an A

Re: OAK-6575 - Provide a secure external URL to a DataStore binary.

2017-08-24 Thread Chetan Mehrotra
> Which circles back to my initial concern: "According to YAGNI we should stick 
> with instance of checks unless we already have a somewhat clear picture of 
> future extensions."

I thought that with all those discussion around JCR Usecases for past
some time we have an agreement for such cases (specially UC3 and UC4).
Hence the push for this approach to enable further work on them going
forward.


Chetan Mehrotra


On Thu, Aug 24, 2017 at 5:41 AM, Michael Dürig <mdue...@apache.org> wrote:
>
>
> On 24.08.17 14:32, Chetan Mehrotra wrote:
>>>
>>> Why not just add a method Blob.getSignedURI()? This would be inline with
>>> getReference() and what we have done with ReferenceBinary.
>>
>>
>> Can be done. But later if we decide to support adapting to say
>> FileChannel [1] then would we be adding that to Blob. Though it may
>> not be related to different Blob types.
>>
>> Having adaptable support allows to extend this later with minimal changes.
>
>
> Which circles back to my initial concern: "According to YAGNI we should
> stick with instance of checks unless we already have a somewhat clear
> picture of future extensions."
>
> Michael
>
>
>>
>> Chetan Mehrotra
>> [1] https://wiki.apache.org/jackrabbit/JCR%20Binary%20Usecase#UC4
>>
>>
>> On Thu, Aug 24, 2017 at 5:25 AM, Michael Dürig <mdue...@apache.org> wrote:
>>>
>>>
>>>
>>> On 24.08.17 13:38, Chetan Mehrotra wrote:
>>>>>
>>>>>
>>>>> various layers involved. The bit I don't understand is how the
>>>>> adaptable
>>>>> pattern would make those go away. To me that pattern is just another
>>>>> way
>>>>> to
>>>>> implement this but it would also need to deal with all those layers.
>>>>
>>>>
>>>>
>>>> Yes this adapter support would need to be implement at all layers.
>>>>
>>>> So call to
>>>> 1. binary.adaptTo(SignedBinary.class) //binary is JCR Binary
>>>> 2. results in blob.adaptTo(SignedBinary.class) //blob is Oak Blob.
>>>> Blob interface would extend adaptable
>>>
>>>
>>>
>>> Why not just add a method Blob.getSignedURI()? This would be inline with
>>> getReference() and what we have done with ReferenceBinary.
>>>
>>> Michael
>>>
>>>
>>>> 3. results in SegmentBlob delegating to BlobStoreBlob which
>>>> 4. delegates to BlobStore // Here just passing the BlobId
>>>> 5. which delegates to DataStoreBlobStore
>>>> 6. which delegates to S3DataStore
>>>> 7. which returns the SignedBinary implementation
>>>>
>>>> However adapter support would allow us to make this instance of check
>>>> extensible. Otherwise we would be hardcoding instance of check to
>>>> SignedBinary at each of the above place though those layers need not
>>>> be aware of SignedBinary support (its specific to S3 impl)
>>>
>>>
>>>
>>>
>


Re: OAK-6575 - Provide a secure external URL to a DataStore binary.

2017-08-24 Thread Chetan Mehrotra
> No API changes to any existing Oak APIs,

Some API needs to be exposed. Note again Oak does not depend on Sling
API. Any such integration code is implemented in Sling Base module
[1]. But that module would still require some API in Oak to provide
such an adaptor

The adaptor proposal here is for enabling layers within Oak to allow
conversion of JCR Binary instance to SignedBinary. Now how this is
exposed to end user depends on usage context

Outside Sling
--

Check if binary instanceof Oak Adaptable. If yes then cast it and adapt it

import org.apache.jackrabbit.oak.api.Adaptable

Binary b = ...
SignedBinary sb  = null
if (b instanceof Adaptable) {
   sb = ((Adaptable)b).adaptTo(SignedBinary.class);
}

Within Sling


Have an AdapterManager implemented in Sling JCR Base [1] which uses
above approach

Chetan Mehrotra
[1] https://github.com/apache/sling/tree/trunk/bundles/jcr/base


On Thu, Aug 24, 2017 at 4:55 AM, Ian Boston <i...@tfd.co.uk> wrote:
> From the javadoc in [1]
>
> "The adaptable object may be any non-null object and is not required to
> implement the Adaptable interface."
>
>
> On 24 August 2017 at 12:54, Ian Boston <i...@tfd.co.uk> wrote:
>
>> Hi,
>> That would require javax.jcr.Binary to implement Adaptable, which it cant.
>> (OakBinary could but it doesnt need to).
>>
>> Using Sling AdapterFactory/AdapterManger javadoc (to be replaced with Oaks
>> internal version of the same)
>>
>> What is needed is an AdapterFactory for javax.jcr.Binary to SignedBinary
>> provided by the S3DataStore itself.
>>
>> Since javax.jcr.Binary cant extend Adaptable, its not possible to call
>> binary.adaptTo(SignedBinary.class) without a cast, hence,
>> the call is done via the AdapterManager[1]
>>
>> SignedBinary signedBinary =  adapterManager.getAdapter(binary,
>> SignedBinary.class);
>>
>> ---
>> You could just jump to
>> URI uri =  adapterManager.getAdapter(binary, URI.class);
>>
>> No API changes to any existing Oak APIs,
>>
>> Best Regards
>> Ian
>>
>>
>> 1 https://sling.apache.org/apidocs/sling5/org/apache/sling/api/adapter/
>> AdapterManager.html
>>
>>
>>
>> On 24 August 2017 at 12:38, Chetan Mehrotra <chetan.mehro...@gmail.com>
>> wrote:
>>
>>> > various layers involved. The bit I don't understand is how the adaptable
>>> > pattern would make those go away. To me that pattern is just another
>>> way to
>>> > implement this but it would also need to deal with all those layers.
>>>
>>> Yes this adapter support would need to be implement at all layers.
>>>
>>> So call to
>>> 1. binary.adaptTo(SignedBinary.class) //binary is JCR Binary
>>> 2. results in blob.adaptTo(SignedBinary.class) //blob is Oak Blob.
>>> Blob interface would extend adaptable
>>> 3. results in SegmentBlob delegating to BlobStoreBlob which
>>> 4. delegates to BlobStore // Here just passing the BlobId
>>> 5. which delegates to DataStoreBlobStore
>>> 6. which delegates to S3DataStore
>>> 7. which returns the SignedBinary implementation
>>>
>>> However adapter support would allow us to make this instance of check
>>> extensible. Otherwise we would be hardcoding instance of check to
>>> SignedBinary at each of the above place though those layers need not
>>> be aware of SignedBinary support (its specific to S3 impl)
>>>
>>> Chetan Mehrotra
>>>
>>
>>


Re: OAK-6575 - Provide a secure external URL to a DataStore binary.

2017-08-24 Thread Chetan Mehrotra
> 1. Figure out how to surface a signed URL from the DataStore to the
> level of the JCR (or Oak) API.
> 2. Provide OSGi glue inside Sling, possibly exposing the signed URL it
> via adaptTo().

Thats sums up the requirement well. Most of proposal here is for #1.
Once we have that implemented #2 can be done in Sling side
Chetan Mehrotra


On Thu, Aug 24, 2017 at 3:06 AM, Ian Boston <i...@tfd.co.uk> wrote:
> Hi,
>
> On 24 August 2017 at 10:20, Julian Sedding <jsedd...@gmail.com> wrote:
>
>> Hi
>>
>> On Thu, Aug 24, 2017 at 9:27 AM, Ian Boston <i...@tfd.co.uk> wrote:
>> > On 24 August 2017 at 08:18, Michael Dürig <mdue...@apache.org> wrote:
>> >
>> >>
>> >>
>> >> URI uri = ((OakValueFactory) valueFactory).getSignedURI(binProp);
>> >>
>> >>
>> > +1
>> >
>> > One point
>> > Users in Sling dont know abou Oak, they know about JCR.
>>
>> I think this issue should be solved in two steps:
>>
>> 1. Figure out how to surface a signed URL from the DataStore to the
>> level of the JCR (or Oak) API.
>> 2. Provide OSGi glue inside Sling, possibly exposing the signed URL it
>> via adaptTo().
>>
>> >
>> > URI uri = ((OakValueFactory)
>> > valueFactory).getSignedURI(jcrNode.getProperty("jcr:data"));
>> >
>> > No new APIs, let OakValueFactory work it out and return null if it cant
>> do
>> > it. It should also handle a null parameter.
>> > (I assume OakValueFactory already exists)
>> >
>> > If you want to make it extensible
>> >
>> >  T convertTo(Object source, Class target);
>> >
>> > used as
>> >
>> > URI uri = ((OakValueFactory)
>> > valueFactory). convertTo(jcrNode.getProperty("jcr:data"), URI.class);
>>
>> There is an upcoming OSGi Spec for a Converter service (RFC 215 Object
>> Conversion, also usable outside of OSGI)[0]. It has an implementation
>> in Felix, but afaik no releases so far.
>>
>> A generic Converter would certainly help with decoupling. Basically
>> the S3-DataStore could register an appropriate conversion, hiding all
>> implementation details.
>>
>
> Sounds like a good fit.
> +1
>
> Best Regards
> Ian
>
>
>>
>> Regards
>> Julian
>>
>> [0] https://github.com/osgi/design/blob/05cd5cf03d4b6f8a512886eae472a6
>> b6fde594b0/rfcs/rfc0215/rfc-0215-object-conversion.pdf
>>
>> >
>> > The user doesnt know or need to know the URI is signed, it needs a URI
>> that
>> > can be resolved.
>> > Oak wants it to be signed.
>> >
>> > Best Regards
>> > Ian
>> >
>> >
>> >
>> >> Michael
>> >>
>> >>
>> >>
>> >>
>> >>> A rough sketch of any alternative proposal would be helpful to decide
>> >>> how to move forward
>> >>>
>> >>> Chetan Mehrotra
>> >>>
>> >>>
>>


Re: OAK-6575 - Provide a secure external URL to a DataStore binary.

2017-08-24 Thread Chetan Mehrotra
> various layers involved. The bit I don't understand is how the adaptable
> pattern would make those go away. To me that pattern is just another way to
> implement this but it would also need to deal with all those layers.

Yes this adapter support would need to be implement at all layers.

So call to
1. binary.adaptTo(SignedBinary.class) //binary is JCR Binary
2. results in blob.adaptTo(SignedBinary.class) //blob is Oak Blob.
Blob interface would extend adaptable
3. results in SegmentBlob delegating to BlobStoreBlob which
4. delegates to BlobStore // Here just passing the BlobId
5. which delegates to DataStoreBlobStore
6. which delegates to S3DataStore
7. which returns the SignedBinary implementation

However adapter support would allow us to make this instance of check
extensible. Otherwise we would be hardcoding instance of check to
SignedBinary at each of the above place though those layers need not
be aware of SignedBinary support (its specific to S3 impl)

Chetan Mehrotra


Re: OAK-6575 - Provide a secure external URL to a DataStore binary.

2017-08-24 Thread Chetan Mehrotra
As explained in previous mail adaptable pattern requirement is to
enable such a support within Oak itself. due to multiple layers
involved.

> If it doesnt exist then perhaps Oak could add a Service interface that
> deals with conversions, rather than expose a second adaptable pattern in
> Sling, or require type casting and instanceof.

We can expose such a service also if that helps. That service
implementation internally would anyway have to use adaptable pattern.

public class OakConversionService{
 AdapterType adaptTo(Binary b, Class type) {
if (b instanceof Adaptable) {
return (type)b.adaptTo(type);
}
return null
}
}

So just another level of abstraction.

Chetan Mehrotra


Re: OAK-6575 - Provide a secure external URL to a DataStore binary.

2017-08-24 Thread Chetan Mehrotra
> Fair point. So this is more about dynamic adaptability than future
> extendibility. But AFIU this could still be achieved without the full
> adaptable machinery:
>
> if (binProp instanceOf SignableBin) {
>   URI uri = ((SignableBin) binProp).getSignedURI();
>   if (uri != null) {
> // resolve URI etc.
>   }
> }

This would be tricky. The current logic is like below.

1. Oak JCR BinaryImpl holds a ValueImpl
2. ValueImpl -> PropertyState -> Blob
3. From Blob following paths are possible
   - Blob -> SegmentBlob -> BlobStoreBlob -> DataRecord -> S3DataRecord
   - Blob -> ArrayBasedBlob
   - Blob ... MongoBlob

So at JCR level where we have a PropertyState we cannot determine if
the Blob provided by it can provide a signed binary without adding
such instance of check at each place. Hence the adaptor based proposal

Chetan Mehrotra


Re: OAK-6575 - Provide a secure external URL to a DataStore binary.

2017-08-24 Thread Chetan Mehrotra
> I think the discussion about the adapter pattern is orthogonal to the binary

For me its tied to how you are going to implement this support.
Adaptable patterns is one way based on my current understand of Oak
design.

At level of ValueFactory.getBinary we do not know if the Blob can
provide a signed url. Its deep down in the layer JCR Binary -> Oak
Blob -> DataStoreBlob -> S3DataStore DataRecord. So each Blob cannot
provided a signed url and it depends on backing DataStore. This can be
easily supported via adaptor pattern where JCR layer tries to adapt
and then final backing BlobStore impl decides to provide the adaption
implementation.

I do not see how instance of checks can be expressed across all these layers

A rough sketch of any alternative proposal would be helpful to decide
how to move forward

Chetan Mehrotra


Re: OAK-6575 - Provide a secure external URL to a DataStore binary.

2017-08-23 Thread Chetan Mehrotra
Below is one possible sketch of the proposed api. We introduce a new
AdaptableBinary which allows adapting a Binary to some other form.

API
===

public interface AdaptableBinary {

/**
 * Adapts the binary to another type
 *
 * @param  The generic type to which this binary is adapted
 *to
 * @param type The Class object of the target type
 * @return The adapter target or null if the binary cannot
 * adapt to the requested type
 */
 AdapterType adaptTo(Class type);
}



Usage
=

Binary binProp = node.getProperty("jcr:data").getBinary();

//Check if Binary is of type AdaptableBinary
if (binProp instanceof AdaptableBinary){
AdaptableBinary adaptableBinary = (AdaptableBinary) binProp;
SignedBinary url = adaptableBinary.adaptTo(SignedBinary.class);
}

Where SignedBinary is one of the supported adaptables.

public interface SignedBinary {

URL getUrl(int ttl, TimeUnit unit)
}

The user can specify ttl. The implementation may enforce an upper
bound on the allowed ttl.

This proposal is meant to provide base. If we agree on the general
approach then we can decide further details like

1. Under which package to expose AdaptableBinary

Proposal 'org.apache.jackrabbit.oak.jcr.binary'. We would also later
possibly need an AdaptableBlob for Oak layer

2. Under which package to expose SignedBinary

Proposal 'org.apache.jackrabbit.oak.api.blob' in oak-api

Thoughts?
Chetan Mehrotra


On Wed, Aug 23, 2017 at 4:25 AM, Chetan Mehrotra
<chetan.mehro...@gmail.com> wrote:
> Recently we had internal discussion for Ian's requirement in OAK-6575.
> See issue for complete details. In brief
>
> 1. Need a way to provide a signed url [1] for Blobs stored in Oak if
> they are stored in S3
> 2. The url would only be created if the user can access the Binary.
> 3.  The url would only be valid for certain time
>
> To meet this requirement various approaches were suggested like using
> Adaptable pattern in Sling, or having a new api in Binary object.
>
> Would follow up with a sketch for such an API
>
> Chetan Mehrotra
> [1] 
> http://docs.aws.amazon.com/AmazonS3/latest/dev/ShareObjectPreSignedURL.html


OAK-6575 - Provide a secure external URL to a DataStore binary.

2017-08-23 Thread Chetan Mehrotra
Recently we had internal discussion for Ian's requirement in OAK-6575.
See issue for complete details. In brief

1. Need a way to provide a signed url [1] for Blobs stored in Oak if
they are stored in S3
2. The url would only be created if the user can access the Binary.
3.  The url would only be valid for certain time

To meet this requirement various approaches were suggested like using
Adaptable pattern in Sling, or having a new api in Binary object.

Would follow up with a sketch for such an API

Chetan Mehrotra
[1] http://docs.aws.amazon.com/AmazonS3/latest/dev/ShareObjectPreSignedURL.html


Re: svn commit: r1805608 - /jackrabbit/oak/trunk/oak-segment-tar/pom.xml

2017-08-21 Thread Chetan Mehrotra
> +org.slf4j.*;resolution:=optional,

This should probably not be optional as its used by code
Chetan Mehrotra


On Mon, Aug 21, 2017 at 2:09 AM,  <adulce...@apache.org> wrote:
> Author: adulceanu
> Date: Mon Aug 21 09:09:05 2017
> New Revision: 1805608
>
> URL: http://svn.apache.org/viewvc?rev=1805608=rev
> Log:
> OAK-6567 - Fix OSGi wiring after netty update to 4.1.x
> Added missing packages to import-package section from oak-segment-tar pom
>
> Modified:
> jackrabbit/oak/trunk/oak-segment-tar/pom.xml
>
> Modified: jackrabbit/oak/trunk/oak-segment-tar/pom.xml
> URL: 
> http://svn.apache.org/viewvc/jackrabbit/oak/trunk/oak-segment-tar/pom.xml?rev=1805608=1805607=1805608=diff
> ==
> --- jackrabbit/oak/trunk/oak-segment-tar/pom.xml (original)
> +++ jackrabbit/oak/trunk/oak-segment-tar/pom.xml Mon Aug 21 09:09:05 2017
> @@ -50,9 +50,16 @@
>  
>  com.google.protobuf.*;resolution:=optional,
>  com.jcraft.jzlib.*;resolution:=optional,
> +com.ning.compress.*;resolution:=optional,
> +
> io.netty.internal.tcnative.*;resolution:=optional,
> +io.netty.resolver.*;resolution:=optional,
> +javax.security.cert.*;resolution:=optional,
>  javassist.*;resolution:=optional,
> +lzma.sdk.*;resolution:=optional,
> +net.jpountz.*;resolution:=optional,
>  org.apache.tomcat.*;resolution:=optional,
>  org.bouncycastle.*;resolution:=optional,
> +org.conscrypt.*;resolution:=optional,
>  org.eclipse.jetty.alpn.*;resolution:=optional,
>  org.eclipse.jetty.npn.*;resolution:=optional,
>  org.jboss.marshalling.*;resolution:=optional,
> @@ -60,8 +67,10 @@
>  sun.nio.ch.*;resolution:=optional,
>  sun.security.util.*;resolution:=optional,
>  sun.security.x509.*;resolution:=optional,
> +org.apache.commons.logging;resolution:=optional,
>  org.apache.logging.log4j.*;resolution:=optional,
>  org.apache.log4j.*;resolution:=optional,
> +org.slf4j.*;resolution:=optional,
>  com.codahale.metrics*;version="[3.1, 4)",
>  *
>  
>
>


Intent to backport OAK-5238 to 1.4 and 1.2 branch

2017-08-18 Thread Chetan Mehrotra
At time index corruption is being seen which can be attributed to
issue fixed in OAK-5238. To address them I need to backport this to
1.4 and 1.2 branch.

This change is slightly bigger so need to be done carefully

Chetan Mehrotra


Support for dumping repository content as json along with blobs via oak-run (OAK-6545)

2017-08-13 Thread Chetan Mehrotra
To simplify troubleshooting repository issue where we need to access
hidden content I have added new command "export" in oak-run to dump
repository content under any given path as json along with blobs [1]

java -jar oak-run-*.jar export -p /path/in/repo /path/of/segmentstore
-o /path/of/output/dir

Hope this would be useful in other cases also

Chetan Mehrotra
[1] https://github.com/apache/jackrabbit-oak/tree/trunk/oak-run#export


Re: frozen node modifications

2017-08-08 Thread Chetan Mehrotra
> From my point of view, blocking the modification of the history even for
> the system that uses Oak as a library, makes its usage far more complicated
> in terms of upgrade without any true benefit in terms of security.

Note that for one time upgrade work you can still modify the content
in version store using the NodeStore API directly. But then you would
need to be careful and understand how versioned data is stored

Chetan Mehrotra


Review current setup of CommitHooks

2017-08-02 Thread Chetan Mehrotra
Hi Team,

In a default Oak setup currently following CommitHooks are configured
to be run for each commit. Most of these hooks would perform separate
diff.

Would it make sense to review the current setup and see if some of
these hooks can be combined to avoid repetitive diff

 - CompositeHook
   - ResetCommitAttributeHook
   - CompositeHook
 - VersionHook
   - VersionEditorProvider
   - VersionableCollector
   - OrphanedVersionCleaner
 - ConflictHook
 - EditorHook
   - ItemSaveValidatorProvider
   - NameValidatorProvider
   - NamespaceEditorProvider
   - TypeEditorProvider
   - ConflictValidatorProvider
   - ChangeCollectorProvider
 - RepoStateCheckHook
 - EditorHook
   - IndexUpdateProvider
   - VersionablePathHook
   - EditorHook
 - PermissionStoreValidatorProvider
 - PermissionValidatorProvider
 - AccessControlValidatorProvider
 - PrivilegeValidatorProvider
 - UserValidatorProvider
 - CacheValidatorProvider
 - TokenValidatorProvider
   - PermissionHook
   - JcrAllCommitHook


Chetan Mehrotra


Re: frozen node modifications

2017-08-02 Thread Chetan Mehrotra
> it is preferable to not create custom node types, because typically the 
> business requirements are not understood enough

Nodetypes are helpful as "content annotation" and allow repository to
provide better performance by relying on them for indexing,
observation, bundling etc. So may be better to say avoid nodetypes
with restrictive definition. However do create new nodetype (or
mixins) to mark the domain objects in your application content.

Chetan Mehrotra


Intent to backport OAK-6333

2017-08-01 Thread Chetan Mehrotra
I want to backport OAK-6333. This is a change in logic which would be
locked behind a feature flag. This flag would allow users to enable
true cost for query calculations and ensure that expected index gets
picked up

Chetan Mehrotra


Intent to backport OAK-6493

2017-07-28 Thread Chetan Mehrotra
Hi,

OAK-6493 fixes an issue around disabling of NRT Indexing feature.

Chetan Mehrotra


Re: Intent to backport OAK-6495

2017-07-26 Thread Chetan Mehrotra
+1
Chetan Mehrotra


On Wed, Jul 26, 2017 at 2:22 PM, Marcel Reutegger
<mreut...@adobe.com.invalid> wrote:
> Hi,
>
> OAK-6495 is a minor improvement that falls back to the classic node state 
> comparison in DocumentNodeStore when the journal is broken. I consider the 
> changes low risk while they e.g. guarantee progress for the async index 
> update when the journal cannot be used for comparing node states.
>
> Regards
>  Marcel
>


Re: Intent to backport OAK-6462

2017-07-18 Thread Chetan Mehrotra
+1
Chetan Mehrotra


On Tue, Jul 18, 2017 at 5:54 PM, Marcel Reutegger
<mreut...@adobe.com.invalid> wrote:
> Hi,
>
> I'd like to backport OAK-6462 to the 1.6 branch sometime after the next 1.7.x 
> release. The issue can cause OOME in DocumentNodeStore based deployments with 
> usage patterns that involve many nt:file nodes. The fix is quite simple and I 
> consider it low risk.
>
> Regards
>  Marcel
>


Re: [VOTE] Release Apache Jackrabbit 2.4.8

2017-07-18 Thread Chetan Mehrotra
+1
Chetan Mehrotra


On Mon, Jul 17, 2017 at 4:24 PM, Julian Reschke <julian.resc...@gmx.de> wrote:
> On 2017-07-17 12:01, Marcel Reutegger wrote:
>>
>> On 17/07/17 10:50, "Julian Reschke" wrote:
>>>
>>> Please vote on releasing this package as Apache Jackrabbit 2.4.8. The
>>> vote is open for the next 72 hours and passes if a majority of at
>>> least three +1 Jackrabbit PMC votes are cast.
>>
>>
>> All checks OK.
>>   +1 Release this package as Apache Jackrabbit 2.4.8
>>
>> Wasn't the plan to EOL the 2.4 branch with this release? I was expecting
>> to see sentence in the release notes regarding this. It may be good to add a
>> note to the announcement email about this decision.
>
>
> Yes, that's the plan -- see separate mail thread.
>
> My plan though was to make the release now, and then to decide about EOL
> later this year, that's why I don't mention it in the release notes...
>
> Best regards, Julian


Intent to backport OAK-5899

2017-07-11 Thread Chetan Mehrotra
OAK-5899 - PropertyDefinitions should allow for some tweakability to
declare usefulness

Chetan Mehrotra


Re: [DiSCUSS] - highly vs rarely used data

2017-07-11 Thread Chetan Mehrotra
I would prefer a 2 phase implementation here

A - CompositeBlobStore
-

Have support for multiple BlobStores plugged within an Oak setup and
provide an API for layer above to select which BlobStore should be
used. This forms the lower most layer in stack. Such a feature should
support

1. Selecting which store a binary should be written to
2. How binary gets read
3. Support Blob GC

B - BinaryStorage Support


Once we have A implemented then layer above can implement some logic
to manage where binaries are stored without requiring major changes in
core. For example Oak can extend the current extension point in
BlobStatsCollector to allow plugging in custom stats collector. This
can be then used by application to build logic to move content based
on various heuristics

1. Path Based
2. Access Based

Application can then use std api to "copy/move" binary from one store
to another.

We can also provide some out of box implementation but key thing here
is that it should be built on top of Oak Core and hence plug-gable.

Given that we have been discussing enhancements in Binary area for
long time now [1] it would be better to get #A implemented now with an
eye for requirements of #B. So that we make some progress here

Chetan Mehrotra
[1] https://wiki.apache.org/jackrabbit/JCR%20Binary%20Usecase


Re: Percentile implementation

2017-07-06 Thread Chetan Mehrotra
Instead of commons-math can we use Metric Histogram  (which I also
suggested earlier in the thread). This would avoid downstream Oak
users to include another dependency as Oak is already using Metrics in
other places.

Can we reconsider this decision?
Chetan Mehrotra


On Tue, Jul 4, 2017 at 4:45 PM, Julian Sedding <jsedd...@gmail.com> wrote:
> Maybe it is not necessary to embed *all* of commons-math3. The bnd
> tool (used by maven-bundle-plugin) can intelligently embed classes
> from specified java packages, but only if they are referenced.
> Depending on how well commons-math3 is modularized, that could allow
> for much less embedded classes. Neil Bartlett wrote a good blog post
> about this feature[0].
>
> Regards
> Julian
>
> [0] http://njbartlett.name/2014/05/26/static-linking.html
>
>
> On Tue, Jul 4, 2017 at 12:20 PM, Andrei Dulceanu
> <andrei.dulce...@gmail.com> wrote:
>> I'll add the dependency.
>>
>> Thanks,
>> Andrei
>>
>> 2017-07-04 13:10 GMT+03:00 Michael Dürig <mdue...@apache.org>:
>>
>>>
>>>
>>> On 04.07.17 11:15, Francesco Mari wrote:
>>>
>>>> 2017-07-04 10:52 GMT+02:00 Andrei Dulceanu <andrei.dulce...@gmail.com>:
>>>>
>>>>> Now my question is this: do we have a simple percentile implementation in
>>>>> Oak (I didn't find one)?
>>>>>
>>>>
>>>> I'm not aware of a percentile implementation in Oak.
>>>>
>>>> If not, would you recommend writing my own or adapting/extracting an
>>>>> existing one in a utility class?
>>>>>
>>>>
>>>> In the past we copied and pasted source code from other projects in
>>>> Oak. As long as the license allows it and proper attribution is given,
>>>> it shouldn't be a problem. That said, I'm not a big fan of either
>>>> rewriting an implementation from scratch or copying and pasting source
>>>> code from other projects. Is exposing a percentile really necessary?
>>>> If yes, how big of a problem is embedding of commons-math3?
>>>>
>>>>
>>> We should avoid copy paste as we might miss important fixes in later
>>> releases. I only did this once for some code where we needed a fix that
>>> wasn't yet released. It was a hassle.
>>> I would just add a dependency to commons-math3. Its a library exposing the
>>> functionality we require, so let's use it.
>>>
>>> Michael
>>>


Re: Oak 1.6.3 release plan

2017-07-06 Thread Chetan Mehrotra
I would like to have OAK-5899 fixed for 1.6. Have marked it as a blocker.
Chetan Mehrotra


On Thu, Jul 6, 2017 at 2:52 PM, Davide Giannella <dav...@apache.org> wrote:
> Hello team,
>
> I'm planning to cut Oak on Monday 10th July.
>
> If there are any objections please let me know. Otherwise I will
> re-schedule any non-resolved issue for the next iteration.
>
> Thanks
> Davide
>
>


Re: Lots of debug output in oak-remote test

2017-07-04 Thread Chetan Mehrotra
Fixed that with OAK-6417
Chetan Mehrotra


On Tue, Jul 4, 2017 at 3:04 PM, Chetan Mehrotra
<chetan.mehro...@gmail.com> wrote:
> While running below in oak-remote module I am seeing lots of debug
> output on console.
>
> mvn clean install -PintegrationTesting
>
> Is anyone else also observing that? Not sure why all debug logs are
> getting enabled
>
> Chetan Mehrotra


Lots of debug output in oak-remote test

2017-07-04 Thread Chetan Mehrotra
While running below in oak-remote module I am seeing lots of debug
output on console.

mvn clean install -PintegrationTesting

Is anyone else also observing that? Not sure why all debug logs are
getting enabled

Chetan Mehrotra


Re: Percentile implementation

2017-07-04 Thread Chetan Mehrotra
Oak has an optional dependency on dropwizard metric library which has
a Histogram implementation [1] which can be used.

So possibly we can use that. So far its optional and hidden behind the
statistics api. Recently I also had to make use of that for rate
estimation in indexing so used it in a way that Metrics based logic
gets used if avialable otherwise fallback to a simple mean based
implementation [2].

May be we can make it a required dependency?

Chetan Mehrotra
[1] 
http://metrics.dropwizard.io/3.1.0/apidocs/com/codahale/metrics/Histogram.html
[2] 
https://github.com/apache/jackrabbit-oak/blob/trunk/oak-core/src/main/java/org/apache/jackrabbit/oak/plugins/index/progress/MetricRateEstimator.java

On Tue, Jul 4, 2017 at 2:22 PM, Andrei Dulceanu
<andrei.dulce...@gmail.com> wrote:
> Hi all,
>
> I was working on an issue for which I needed to use *only* a percentile for
> some recorded stats. Initially I used SynchronizedDescriptiveStatistics [0]
> from commons-math3 [1], but then I thought that adding this dependency
> would be too much for such a simple use case.
>
> Now my question is this: do we have a simple percentile implementation in
> Oak (I didn't find one)? If not, would you recommend writing my own or
> adapting/extracting an existing one in a utility class?
>
> Regards,
> Andrei
>
> [0]
> http://commons.apache.org/proper/commons-math/javadocs/api-3.3/org/apache/commons/math3/stat/descriptive/SynchronizedDescriptiveStatistics.html
> [1] http://commons.apache.org/proper/commons-math/


Re: Trunk doesn't compile

2017-07-04 Thread Chetan Mehrotra
Reverted the commit with 1800742. The build should now pass
Chetan Mehrotra


On Tue, Jul 4, 2017 at 1:51 PM, Francesco Mari <mari.france...@gmail.com> wrote:
> Thanks for taking care of this.
>
> 2017-07-04 10:17 GMT+02:00 Chetan Mehrotra <chetan.mehro...@gmail.com>:
>
>> My fault. Looks like code used API from Tika 1.15 which I am yet
>> testing. Would fix it now
>> Chetan Mehrotra
>>
>>
>> On Tue, Jul 4, 2017 at 1:34 PM, Francesco Mari <mari.france...@gmail.com>
>> wrote:
>> > When compiling trunk (r1800739) I get the following error in the
>> oak-lucene
>> > module.
>> >
>> > [ERROR] Failed to execute goal
>> > org.apache.maven.plugins:maven-compiler-plugin:3.5.1:compile
>> > (default-compile) on project oak-lucene: Compilation failure
>> > [ERROR]
>> > /Users/mari/src/svn/oak/trunk/oak-lucene/src/main/java/org/
>> apache/jackrabbit/oak/plugins/index/lucene/binary/
>> TikaParserConfig.java:[94,34]
>> > cannot find symbol
>> > [ERROR]   symbol:   method getDocumentBuilder()
>> > [ERROR]   location: class org.apache.tika.parser.ParseContext
>> >
>> > Can someone have a look at it?
>>


Re: Trunk doesn't compile

2017-07-04 Thread Chetan Mehrotra
My fault. Looks like code used API from Tika 1.15 which I am yet
testing. Would fix it now
Chetan Mehrotra


On Tue, Jul 4, 2017 at 1:34 PM, Francesco Mari <mari.france...@gmail.com> wrote:
> When compiling trunk (r1800739) I get the following error in the oak-lucene
> module.
>
> [ERROR] Failed to execute goal
> org.apache.maven.plugins:maven-compiler-plugin:3.5.1:compile
> (default-compile) on project oak-lucene: Compilation failure
> [ERROR]
> /Users/mari/src/svn/oak/trunk/oak-lucene/src/main/java/org/apache/jackrabbit/oak/plugins/index/lucene/binary/TikaParserConfig.java:[94,34]
> cannot find symbol
> [ERROR]   symbol:   method getDocumentBuilder()
> [ERROR]   location: class org.apache.tika.parser.ParseContext
>
> Can someone have a look at it?


Re: lucene analyzers

2017-06-27 Thread Chetan Mehrotra
As of now yes. This would require support for per field analyzer and
also separate analyzer for index time and query time. Current design
was done to support such a usecase eventually but so far its not
implemented
Chetan Mehrotra


On Mon, Jun 26, 2017 at 1:19 PM, Alvaro Cabrerizo <topor...@gmail.com> wrote:
> Hello,
>
> Reviewing the oak documentation
> <https://jackrabbit.apache.org/oak/docs/query/lucene.html#analyzers> I
> read:
>
> *"Note that currently only one analyzer can be configured per index. Its
> not possible to specify separate analyzer for query and index time
> currently."*
> According to that statement (please correct me if I'm wrong):
>
>- I can not have and analyzer per field. For example imagine that I am
>indexing a node that contains a description field in Spanish and other
>description field in English and I want them to be analyzed in a different
>way.
>- I can not make use of native queries over *:path *and *:ancestors*
>fields as these fields are analyzed using a different analyzer (seems to be
>the KeywordTokenizerFactory) from the rest of fields (those defined as
>propertyIndex and also the :fulltext field that are analyzed using the
>StandardAnalyzer or the one defined under analyzers within the index).
>
> Regards.


Re: native clauses, index selection and sort clauses

2017-06-23 Thread Chetan Mehrotra
> When using a native query clause the index plan, of the selected index,
> does not include sorting capabilities although the index can. I'm not sure
> if this is the expected result.

Yes this is as designed. The native query support in Lucene was
implemented with the assumption that no other constraints are provided
and all constraints are part of the query itself. So its not possible
to mix native query with other JCR constraints currently

> take advantage of it. Furthermore, what I can see executing the query is
> that after retrieving a batch of documents from the index, they are
> validated one by one, against all the query conditions, which is an
> incredible amount of effort and waste of time as the lucene index was
> defined to do all this tasks.

This is done for following reasons

1. The index may not be handling all the expressed constraints
2. The index data may be out of sync with current session revision
state. For e.g. if a property /a/b/@foo = bar was indexed at revision
R1 and then it got changed to /a/b/@foo = baz in R2 (which is yet not
indexed) then doing a query with session at R2 should not allow that
path as part of final result set

Chetan Mehrotra


On Thu, Jun 22, 2017 at 7:12 PM, Alvaro Cabrerizo <topor...@gmail.com> wrote:
> Hello,
>
> When using a native query clause the index plan, of the selected index,
> does not include sorting capabilities although the index can. I'm not sure
> if this is the expected result.
>
> For example, using the next query:
>
> /jcr:root/content/dam/store//element(*, dam:Asset)[*rep:native('myIndex',
> 'myProp:myValue')*] order by jcr:content/metadata/@sortableProp
>
> The plan I get is:
>
> { costPerExecution : 1.0, costPerEntry : 50.0, estimatedEntryCount : 1,
> filter : Filter(query=select [jcr:path], [jcr:score], * from [dam:Asset] as
> a where native(a, 'myIndex', '*:*') ...order by
> [jcr:content/metadata/sortableProp]
> /..., isDelayed : true, isFulltextIndex : true, includesNodeData :
> false, *sortOrder
> : null*, definition : null, propertyRestriction : null, pathPrefix :  }
>
>
> Keeping in mind plan that the index defines the sortableProp as sortable:
>
> oak:index
> |_myIndex (functionName=myIndex)
>|_indexRules
>   |_dam:Asset
>  |_properties
> |_sortableProp* (ordered = true)*
>
>
> Basically it is not clear why (the rationale behind)
> org.apache.jackrabbit.oak.plugins.index.lucene.IndexPlanner.java returns
> the planner when the query contains a function that matches and index
> funtionName before adding the rest of the index capabilities to the plan.
> Thus, the plan does not describe all the index capabilities (e.g. sort) and
> when the query is performed through the LucenePropertyIndex I can't not
> take advantage of it. Furthermore, what I can see executing the query is
> that after retrieving a batch of documents from the index, they are
> validated one by one, against all the query conditions, which is an
> incredible amount of effort and waste of time as the lucene index was
> defined to do all this tasks.
>  I guess that rewriting the query should avoid this situation, but not sure
> how.
>
> Regards


Re: Oak Metrics.

2017-06-15 Thread Chetan Mehrotra
On Thu, Jun 15, 2017 at 2:55 PM, Ian Boston <i...@tfd.co.uk> wrote:
> Is are the new Oak Metrics documented somewhere ?

No because so far no one else asked for it and only I was making use
of them! Now that you asked would try to document it

> NRT_REFRESH_TIME

That measures the time to refresh the index reader (see NRTIndex
class). Probably we should have a way to add description at time of
Metric creation.

Chetan Mehrotra


Set baseline plugin comparison version to 1.6.0 (OAK-6346)

2017-06-14 Thread Chetan Mehrotra
Hi Team,

Please have a look at proposal at OAK-6346 to set base
comparisonVersion of baseline plugin to current stable release of
1.6.0.

This is done to ensure that any in progress work in trunk which get
release perioduially does not require bump of major version for new
features

Chetan Mehrotra


Re: svn commit: r1798453 - in /jackrabbit/oak/trunk: oak-api/src/main/java/org/apache/jackrabbit/oak/api/jmx/package-info.java oak-store-composite/src/main/java/org/apache/jackrabbit/oak/composite/pac

2017-06-12 Thread Chetan Mehrotra
On Mon, Jun 12, 2017 at 6:13 PM,  <catholi...@apache.org> wrote:
> -@Version("4.6.0")
> +@Version("5.0.0")
>  @Export(optional = "provide:=true")
>  package org.apache.jackrabbit.oak.api.jmx;

This would cause potential issue for the clients as this package is
meant to be used by user of Oak as an API. I know that newly removed
method is not yet part of stable release but then we had a unstable
release with new method. I see 2 options

1. Deprecate the method instead of removing it
2. Or better specify comparisonVersion to 1.6 via [1]

Chetan Mehrotra
[1] 
http://felix.apache.org/components/bundle-plugin/baseline-mojo.html#comparisonVersion


Re: index selection when 0 or 1 property clause (or sort clause) matches

2017-06-09 Thread Chetan Mehrotra
On Wed, May 17, 2017 at 1:40 PM, Alvaro Cabrerizo <topor...@gmail.com> wrote:
> The main issue of delegating in entryCount, is that if the index contains
> more than 1000 docs and the query does not contain fulltext clauses the
> index planner will use the number *1000 *as the entryCount, ovewriting the
> actual size of the index [Math.min(definition.getEntryCount(), getReader().
> *numDocs())*].

A very late reply here but I have opened OAK-6333 to remove this
artificial limit of 1000 in cost estimates. Once we have consensus on
the approach then I would fix and backport it to older branched

Chetan Mehrotra


Re: minimize the impact when creating a new index (or re-indexing)

2017-06-09 Thread Chetan Mehrotra
> indexing time would be fine (it was previously reduced from 7 to 2 days),
> but I think that traversing more than 1 nodes is a pretty tough

Yes that would take time (i.e. indexing 100M+ nodes). This is an area
where work is in progress (OAK-6246) to get much shorter indexing
time.

> related to indexing optimization or any advice on how to design the repo
> (e.g. use different paths to isolate different groups of assets, use
> different nodetypes to differentiate content type, create different
> repositories [is that possible?] for different groups of uses...) is

Hard to give a generic advice here. It all depends on type of query,
index definition and content structure. So would need such details to
provide any suggestion.

Chetan Mehrotra


Re: minimize the impact when creating a new index (or re-indexing)

2017-06-08 Thread Chetan Mehrotra
On Thu, Jun 8, 2017 at 4:04 PM, Alvaro Cabrerizo <topor...@gmail.com> wrote:
> It is a DocumentNodeStore based instance. We don't extract data from binary
> files, just indexing metadata stored on nodes.

In that case 48 hrs is a long time. Can you share some details around
how many nodes are being indexed as part of that index and the repo
size in terms of Mongo stats if possible?

Chetan Mehrotra


Re: any suggestion!!!

2017-06-06 Thread Chetan Mehrotra
Hi Austin,

I believe this query was asked earlier also. See thread at
https://lists.apache.org/thread.html/bb48b79ae9031c7fc3deb3992a32ac8e3ab53205e9fd362318812e30@%3Coak-dev.jackrabbit.apache.org%3E
Chetan Mehrotra


On Wed, Jun 7, 2017 at 1:25 AM, Austin Zhang <austinyizh...@gmail.com> wrote:
> Hi,
>
> I’m learning jackrabbit oak, when trying  the standalone server example,
> after the server starting up, from my java code I want to connect to the
> server and get a handle(reference) of the repository back.
>
> We know jackrabbit 2 can do this kind of thing by RMI, JNDI, Webdav or some
> other ways, but I could not find anyway that jackrabbit oak does.
>
> My question is how can our code to connect to a remote jackrabbit oak
> standalone server and get a repository reference back, so we can do all our
> operations on it?
>
> Thanks for your time and help!
>
> Austin


Re: minimize the impact when creating a new index (or re-indexing)

2017-06-06 Thread Chetan Mehrotra
> I'm not sure how to minimize the impact of performing a re-index (or new
> index creation), that will take 48 hours (using oak 1.4). I mean, I don't
> want to block other indexes update.

Is this a SegmentNodeStore based setup or DocumentNodeStore based?

The reindexing log would have some stats around time spent in indexing
and time spent in text extraction. Can you check whats the part which
takes most time. If its text extraction then you can reduce the time
spent in that via using Pre-Extraction support [1]. This allow
extracting text before hand and then using that at time of actual
indexing

Changing the "indexing lane" should help but is tricky to get right
and something we are improving currently OAK-6246 and OAK-5553

> indexes won't be updated. On the other hand, it seems that using the
> *reindex-async* flag (see OAK-1456
> <https://issues.apache.org/jira/browse/OAK-1456>) could do the trick. I

This mode is useful for property index as in the end it removes the
async flag and makes the index synchronous which would cause issues
for lucene based index

Chetan Mehrotra
[1] https://jackrabbit.apache.org/oak/docs/query/lucene.html#text-extraction


On Tue, Jun 6, 2017 at 9:02 PM, Alvaro Cabrerizo <topor...@gmail.com> wrote:
> Hello,
>
> I'm not sure how to minimize the impact of performing a re-index (or new
> index creation), that will take 48 hours (using oak 1.4). I mean, I don't
> want to block other indexes update.
>
> First, we have set the value of async as *fulltext-async* for the new
> index. I guess, that at least, all the indexes managed by the *async* lane
> <http://jackrabbit.apache.org/oak/docs/query/indexing.html#indexing-lane>
> will not be affected (please, confirm if I'm right). Then we try to
> minimize the impact on the fulltext-async lane. According to OAK-5553
> <https://issues.apache.org/jira/browse/OAK-5553> there isn't much we can do
> while the indexing process is active for the new index, as the rest of
> indexes won't be updated. On the other hand, it seems that using the
> *reindex-async* flag (see OAK-1456
> <https://issues.apache.org/jira/browse/OAK-1456>) could do the trick. I
> mean, setting reindex-async=true to the new index will allow other indexes
> (in the same lane) being updated while it is being populated? If that is
> true, we could create the index with that flag and then remove it.
>
> Regards.


Intent to backport OAK-5961

2017-06-02 Thread Chetan Mehrotra
A minor fix in oak-run tooling https://issues.apache.org/jira/browse/OAK-5961

Chetan Mehrotra


Re: How can java code connect to remote jackrabbit oak standalone server and get a repository reference back?

2017-06-01 Thread Chetan Mehrotra
You should be able to use same approach as used for Jackrabbit with
Oak. The standalone example application server would listen at port
8080 and support remote access via DavEx (at `/server`)  and WebDAV
(at `/repository/default`).
Chetan Mehrotra


On Fri, Jun 2, 2017 at 12:23 AM, Austin Zhang <austinyizh...@gmail.com> wrote:
> Hi,
>
> I’m learning jackrabbit oak, when trying  the standalone server example, 
> after the server starting up, from my java code I want to connect to the 
> server and get a handle(reference) of the repository back.
>
> We know jackrabbit 2 can do this kind of thing by RMI, JNDI, Webdav or some 
> other ways, but I could not find anyway that jackrabbit oak does.
>
> My question is how can our code to connect to a remote jackrabbit oak 
> standalone server and get a repository reference back, so we can do all our 
> operations on it?


Intent to backport to 1.6: OAK-6267

2017-05-31 Thread Chetan Mehrotra
https://issues.apache.org/jira/browse/OAK-6267

This is required to fix a bug for setups where bundling is enabled. It
affects DocumentNodeStore based setups

Chetan Mehrotra


Re: [VOTE] Release Apache Jackrabbit 2.14.1

2017-05-30 Thread Chetan Mehrotra
+1
Chetan Mehrotra


On Tue, May 30, 2017 at 2:21 PM, KÖLL Claus <c.ko...@tirol.gv.at> wrote:
> +1
>
> greets
> claus


Re: codahale metrics Jmx reporter

2017-05-30 Thread Chetan Mehrotra
On Tue, May 30, 2017 at 11:53 AM, Andrei Kalfas
<akal...@adobe.com.invalid> wrote:
> Looks to me that there is a dependency on oak functionality.

Ian can confirm but I believe thats not required now (the component
does not get activated) and was only a temporary workaround. Oak
publishes the MetricRegistry instance in OSGi service registry and
then any component can look up that service and configure a reporter
for it

Chetan Mehrotra


Re: codahale metrics Jmx reporter

2017-05-30 Thread Chetan Mehrotra
On Tue, May 30, 2017 at 11:30 AM, Andrei Kalfas
<akal...@adobe.com.invalid> wrote:
> I thought that oak-core is the best position to say what metrics are needed, 
> and the way that we want to extend the behavior is by picking whatever 
> reporter we say fits the environment - this is why I started the thread, I 
> wanted to understand whats the intended design.

Oak is only concerned with publishing metrics to a MetricRegistry. How
the metrics are consumed and reported are outside of Oak scope

Various reporters that Ian referred are generic and can support any
MetricRegistry and have no dependency on Oak. Hence no needs for them
to be in Oak

Chetan Mehrotra


Re: svn commit: r1796624 - /jackrabbit/oak/trunk/oak-store-spi/src/main/java/org/apache/jackrabbit/oak/spi/state/AbstractNodeState.java

2017-05-29 Thread Chetan Mehrotra
On Mon, May 29, 2017 at 6:50 PM,  <mdue...@apache.org> wrote:
> +private static final int CHILDREN_CAP = getInteger("children.cap", 100);

Better to have system property prefix with 'oak' i.e. 'oak.children.cap'

Chetan Mehrotra


Re: codahale metrics Jmx reporter

2017-05-29 Thread Chetan Mehrotra
On Mon, May 29, 2017 at 5:18 PM, Andrei Kalfas
<akal...@adobe.com.invalid> wrote:
> have a different component peeks into the metrics registry or change oak-core 
> to deal with multiple reporters

I would prefer to let Oak focus on basic reporting and some other
component deal with integration with different types of reporters

Chetan Mehrotra


Re: codahale metrics Jmx reporter

2017-05-29 Thread Chetan Mehrotra
On Mon, May 29, 2017 at 5:02 PM, Andrei Kalfas
<akal...@adobe.com.invalid> wrote:
> Is this the only intended way of reporting these metrics?

By default this is the only mode. In addition the MetricRegistry is
registered in OSGi service registry. So some other component can look
it up and use it with different reporter. This is used in Sling to
report the metrics to Felix WebConsole [1] and [2]

Chetan Mehrotra

[1] 
https://github.com/apache/sling/blob/trunk/bundles/commons/metrics/src/main/java/org/apache/sling/commons/metrics/internal/MetricWebConsolePlugin.java#L106
[2] 
https://sling.apache.org/documentation/bundles/metrics.html#webconsole-plugin


Intent to backport to 1.4, 1.2 : OAK-4668

2017-05-26 Thread Chetan Mehrotra
Need to backport OAK-4668 [1] to older branches. This is required to
prevent loss of checkpoint in unstable topology

Chetan Mehrotra
[1] https://issues.apache.org/jira/browse/OAK-4668


Re: Intent to backport to 1.4: OAK-4863

2017-05-23 Thread Chetan Mehrotra
+1.
Chetan Mehrotra


On Tue, May 23, 2017 at 12:30 PM, Julian Reschke <julian.resc...@gmx.de> wrote:
> https://issues.apache.org/jira/browse/OAK-4863


Re: Intent to backport to 1.6 and 1.4: OAK-5571

2017-05-19 Thread Chetan Mehrotra
As mentioned in previous mail can we wait for 1.7.0 release such that
all new changes in RevisionGC get some longevity testing. And after
that we do the backports.
Chetan Mehrotra


On Fri, May 19, 2017 at 12:29 PM, Julian Reschke <julian.resc...@gmx.de> wrote:
> https://issues.apache.org/jira/browse/OAK-5571


Re: Intent to backport to 1.6 and 1.4: OAK-5741

2017-05-19 Thread Chetan Mehrotra
On Thu, May 18, 2017 at 5:51 PM, Julian Reschke <julian.resc...@gmx.de> wrote:
> (needed for a subsequent backport of OAK-5704)

Probably we should wait for OAK-5704 for sometime after 1.7.0 is cut
and that build is subjected some longevity testing before backporting
it to branched

Chetan Mehrotra


Re: Possible memory leak

2017-05-17 Thread Chetan Mehrotra
On Tue, May 16, 2017 at 7:04 PM, Barry d'Hoine
<barry.dho...@amplexor.com> wrote:
> org.apache.jackrabbit.oak.cache.CacheLIRS

The CacheLIRS instance is a cache and hence would be referring to a
big chunk of heap. Probably thats why MAT flags it. This alone does
not confirm its a memory leak. The reachable size via this object
should be closer to the cache size you have configured. If its much
higher than configured cache size then that would need to be looked
into

What type of setup is this Document/Mongo or Tar?



Chetan Mehrotra


Re: index selection when 0 or 1 property clause (or sort clause) matches

2017-05-17 Thread Chetan Mehrotra
> Using the entryCount was our first option, but we decided to modify
> costPerEntry instead. Basically these are the reasons:

What I meant was not the use of "entryCount" property but just that
the sub index /nodeA/nodeB/includedC being a subset of /nodeA/nodeB/
it would have lower value for numDocs and hence any query having path
restriction of /nodeA/nodeB/includedC would get to use that index (by
virtue of numDocs being less than parent). So you would not need to do
any tweaking

> response we expect. It seems that in OAK, indexes are used to speed up
> queries, but not as a complete information entity that must answer every
> query clause. Well, I'm still figuring out the big picture of querying

Yes the index may only be partially covering the query. So index job
is to provide a super set of possible result paths and Query Engine
would then evaluate the restriction for each path and at the
repository state per current JCR Session (except fulltext constraints)

Chetan Mehrotra


Re: index selection when 0 or 1 property clause (or sort clause) matches

2017-05-16 Thread Chetan Mehrotra
Having same asset indexed twice would add overhead in terms of async
indexing speed and space consumption by index. So if possible avoid
such a setup

> We could assume that we always add a path restriction, but I'm not sure how
> index movement can help. I mean, both indexes contains documents under
> /nodeA/nodeB/nodeC  so any query under this path is satisfied by both

One way it would help is that entryCount for second index would be
smaller compared to first and hence queries with path restriction for
second index would have lesser cost

Chetan Mehrotra


Re: index selection when 0 or 1 property clause (or sort clause) matches

2017-05-16 Thread Chetan Mehrotra
>  - Rebuilding Assets index takes several days

Is the time spent in text extraction?

Would the code always specify path restriction in the queries for
my:Asset? If yes then you can just move the index definition under
respective paths

Would that be an option
Chetan Mehrotra


On Tue, May 16, 2017 at 8:20 PM, Alvaro Cabrerizo <topor...@gmail.com> wrote:
> Hello,
>
> Yes, there are some reasons:
>
>- One team is working with all the assets under /nodeA/nodeB
>- Other team only works with an small subset, only assets under
>/nodeA/node/nodeC
>- Both teams have different searching requirements
>- Rebuilding Assets index takes several days
>- Rebuilding deeperAsset takes a couple of hours. Thus merging both will
>penalize the size and modification agility of deeperAsset
>
> One option we are evaluating is to cheat the deepAsset cost (using
> costPerEntry property) and make that team work with lucene native queries
> (any recommendation is welcome).  Anyway, it is not clear which index is
> selected in case of tie.
>
> Regards.
>
>
> On Tue, May 16, 2017 at 4:36 PM, Chetan Mehrotra <chetan.mehro...@gmail.com>
> wrote:
>
>> Any reason for having separate definitions for same nodetype?
>> Chetan Mehrotra
>>
>>
>> On Tue, May 16, 2017 at 7:52 PM, Alvaro Cabrerizo <topor...@gmail.com>
>> wrote:
>> > Hello,
>> >
>> > Actually, it is OAK-5449. Sorry, I hadn't seen it.
>> >
>> > On the other hand, having these two definitions under oak:index (just a
>> > sketch):
>> >
>> >
>> >- Asset
>> >- evaluatePathRestrictions="true"
>> >   - type="lucene"
>> >   - includedPath="/nodeA/nodeB"
>> >   - indexRules
>> >   - my:Asset
>> >  - properties
>> >- name="my:title"
>> > - deeperAsset
>> >   - evaluatePathRestrictions="true"
>> >   - type="lucene"
>> >   - includedPath="/nodeA/nodeB/includedC"
>> >   - ndexRules
>> >   - my:Asset
>> >  - properties
>> >- name="my:description"
>> >
>> > Made the system to return a cost of 1001 (for both indexes) when
>> performing
>> > these kind of queries:
>> >
>> >- SELECT * FROM [my:Asset] AS s WHERE ISDESCENDANTNODE(s,'/nodeA/
>> nodeB')
>> >- SELECT * FROM [my:Asset] AS s WHERE ISDESCENDANTNODE(s,'/nodeA/
>> nodeB')
>> >AND s.[my:title]='title'
>> >- SELECT * FROM [my:Asset] AS s WHERE ISDESCENDANTNODE(s,'/nodeA/
>> nodeB')
>> >AND s.[my:description]='description'
>> >
>> > Once cost is assigned (equal for both indexes), it is not clear which
>> index
>> > will be selected.
>> >
>> > Regards.
>> >
>> > On Tue, May 16, 2017 at 3:50 PM, Chetan Mehrotra <
>> chetan.mehro...@gmail.com>
>> > wrote:
>> >
>> >> This looks similar to OAK-5449 (not yet fixed). Can you give a sample
>> >> index definition there and some usecase details which is leading to
>> >> ambiguity in index selection.
>> >>
>> >> In general index selection should not have multiple competing index
>> >> definitions hence interested in knowing setup details
>> >> Chetan Mehrotra
>> >>
>> >>
>> >> On Tue, May 16, 2017 at 1:53 PM, Alvaro Cabrerizo <topor...@gmail.com>
>> >> wrote:
>> >> > Hello,
>> >> >
>> >> > I've been checking the code of the IndexPlanner (apache OAK 1.4.1)
>> and I
>> >> > was surprised because the costPerEntryFactor remains 1 in both cases:
>> >> >
>> >> >- when no property indexed or sorted match any property clause or
>> sort
>> >> >clause from the query
>> >> >- when only an indexed or sorted property matches a property
>> clause or
>> >> >sort clause from the query
>> >> >
>> >> > Although this piece of code avoids a division by zero (see
>> >> > org.apache.jackrabbit.oak.plugins.index.lucene.IndexPlanner lines
>> >> 201-203)
>> >> >
>> >> > if (costPerEntryFactor == 0){
>> >> > costPerEntryFactor = 1;
>> >> > }
>> >> >
>> >> > It also avoids the boosting of indexes that match 1 query clause (or
>> in
>> >> > other words, it doesn't penalize indexes that don't match any clause).
>> >> I'm
>> >> > thinking about opening an issue. Although it is for the long-tern,
>> >> actually
>> >> > I would like to know which index is selected in case that more than
>> one
>> >> had
>> >> > the same cost.
>> >> >
>> >> > Regards.
>> >>
>>


Re: index selection when 0 or 1 property clause (or sort clause) matches

2017-05-16 Thread Chetan Mehrotra
Any reason for having separate definitions for same nodetype?
Chetan Mehrotra


On Tue, May 16, 2017 at 7:52 PM, Alvaro Cabrerizo <topor...@gmail.com> wrote:
> Hello,
>
> Actually, it is OAK-5449. Sorry, I hadn't seen it.
>
> On the other hand, having these two definitions under oak:index (just a
> sketch):
>
>
>- Asset
>- evaluatePathRestrictions="true"
>   - type="lucene"
>   - includedPath="/nodeA/nodeB"
>   - indexRules
>   - my:Asset
>  - properties
>- name="my:title"
> - deeperAsset
>   - evaluatePathRestrictions="true"
>   - type="lucene"
>   - includedPath="/nodeA/nodeB/includedC"
>   - ndexRules
>   - my:Asset
>  - properties
>- name="my:description"
>
> Made the system to return a cost of 1001 (for both indexes) when performing
> these kind of queries:
>
>- SELECT * FROM [my:Asset] AS s WHERE ISDESCENDANTNODE(s,'/nodeA/nodeB')
>- SELECT * FROM [my:Asset] AS s WHERE ISDESCENDANTNODE(s,'/nodeA/nodeB')
>AND s.[my:title]='title'
>- SELECT * FROM [my:Asset] AS s WHERE ISDESCENDANTNODE(s,'/nodeA/nodeB')
>AND s.[my:description]='description'
>
> Once cost is assigned (equal for both indexes), it is not clear which index
> will be selected.
>
> Regards.
>
> On Tue, May 16, 2017 at 3:50 PM, Chetan Mehrotra <chetan.mehro...@gmail.com>
> wrote:
>
>> This looks similar to OAK-5449 (not yet fixed). Can you give a sample
>> index definition there and some usecase details which is leading to
>> ambiguity in index selection.
>>
>> In general index selection should not have multiple competing index
>> definitions hence interested in knowing setup details
>> Chetan Mehrotra
>>
>>
>> On Tue, May 16, 2017 at 1:53 PM, Alvaro Cabrerizo <topor...@gmail.com>
>> wrote:
>> > Hello,
>> >
>> > I've been checking the code of the IndexPlanner (apache OAK 1.4.1) and I
>> > was surprised because the costPerEntryFactor remains 1 in both cases:
>> >
>> >- when no property indexed or sorted match any property clause or sort
>> >clause from the query
>> >- when only an indexed or sorted property matches a property clause or
>> >sort clause from the query
>> >
>> > Although this piece of code avoids a division by zero (see
>> > org.apache.jackrabbit.oak.plugins.index.lucene.IndexPlanner lines
>> 201-203)
>> >
>> > if (costPerEntryFactor == 0){
>> > costPerEntryFactor = 1;
>> > }
>> >
>> > It also avoids the boosting of indexes that match 1 query clause (or in
>> > other words, it doesn't penalize indexes that don't match any clause).
>> I'm
>> > thinking about opening an issue. Although it is for the long-tern,
>> actually
>> > I would like to know which index is selected in case that more than one
>> had
>> > the same cost.
>> >
>> > Regards.
>>


Re: index selection when 0 or 1 property clause (or sort clause) matches

2017-05-16 Thread Chetan Mehrotra
This looks similar to OAK-5449 (not yet fixed). Can you give a sample
index definition there and some usecase details which is leading to
ambiguity in index selection.

In general index selection should not have multiple competing index
definitions hence interested in knowing setup details
Chetan Mehrotra


On Tue, May 16, 2017 at 1:53 PM, Alvaro Cabrerizo <topor...@gmail.com> wrote:
> Hello,
>
> I've been checking the code of the IndexPlanner (apache OAK 1.4.1) and I
> was surprised because the costPerEntryFactor remains 1 in both cases:
>
>- when no property indexed or sorted match any property clause or sort
>clause from the query
>- when only an indexed or sorted property matches a property clause or
>sort clause from the query
>
> Although this piece of code avoids a division by zero (see
> org.apache.jackrabbit.oak.plugins.index.lucene.IndexPlanner lines 201-203)
>
> if (costPerEntryFactor == 0){
> costPerEntryFactor = 1;
> }
>
> It also avoids the boosting of indexes that match 1 query clause (or in
> other words, it doesn't penalize indexes that don't match any clause). I'm
> thinking about opening an issue. Although it is for the long-tern, actually
> I would like to know which index is selected in case that more than one had
> the same cost.
>
> Regards.


Provide a consistent and extensible way to handle oak-run options while creating NodeStore (OAK-6210)

2017-05-11 Thread Chetan Mehrotra
Hi Team,

Please have a look at OAK-6210. It aims to simplify and unify the way
NodeStore and BlobStore are created in various command in oak-run and
how various options are handled

Please provide your feedback!

If this looks ok I would like commit it by May-15 and then would try
to move some of the existing commands to new approach

Chetan Mehrotra


Re: svn commit: r1794332 - in /jackrabbit/oak/trunk/oak-core/src: main/java/org/apache/jackrabbit/oak/plugins/document/mongo/MongoMissingLastRevSeeker.java test/java/org/apache/jackrabbit/oak/plugins/

2017-05-08 Thread Chetan Mehrotra
On Mon, May 8, 2017 at 5:01 PM,  <mreut...@apache.org> wrote:
> -DBObject sortFields = new 
> BasicDBObject(NodeDocument.MODIFIED_IN_SECS, -1);
> +DBObject sortFields = new 
> BasicDBObject(NodeDocument.MODIFIED_IN_SECS, 1);

May be we skip sorting altogether. In that case iteration would be on
id which should be stable order.

Chetan Mehrotra


Re: Sort order in MongoMissingLastRevSeeker

2017-05-04 Thread Chetan Mehrotra
On Thu, May 4, 2017 at 3:15 PM, Marcel Reutegger <mreut...@adobe.com> wrote:
> A can't really think of a reason why the sort order must be descending in
> MongoMissingLastRevSeeker.

I also do not remember. I think it should be fine to drop the sort criteria.

Chetan Mehrotra


Re: [VOTE] Release Apache Jackrabbit 2.15.2

2017-05-02 Thread Chetan Mehrotra
+1 Release this package as Apache Jackrabbit 2.15.2
Chetan Mehrotra


On Tue, May 2, 2017 at 3:39 PM, Julian Reschke <julian.resc...@gmx.de> wrote:
> On 2017-05-02 12:02, Julian Reschke wrote:
>>
>> On 2017-05-02 11:35, Marcel Reutegger wrote:
>>>
>>> On 01/05/17 17:05, Julian Reschke wrote:
>>>>
>>>> Please vote on releasing this package as Apache Jackrabbit 2.15.2.
>>>> The vote is open for the next 72 hours and passes if a majority of at
>>>> least three +1 Jackrabbit PMC votes are cast.
>>>
>>>
>>> All checks OK.
>>>
>>> +1 Release this package as Apache Jackrabbit 2.15.2
>>>
>>> Just one minor comment, the README.txt needs an update. It still says,
>>> the minimum Java version is 6.
>>>
>>> Regards
>>>  Marcel
>>
>>
>> Oh. Will do (for the next release). Thanks.
>
>
> -> https://issues.apache.org/jira/browse/JCR-4134
>
>


  1   2   3   4   5   6   7   8   >