Re: Oak Jenkins build takes up too many resources

[Robert Munteanu]

Hi Marcel,
Hi Konrad,

Thanks for looking into this, some good improvements are on the way,
and it seems like the problem severity was inflated by the way Jenkins
reports build step duration.

Thanks,
Robert

On Tue, 2022-11-15 at 10:56 +0100, Konrad Windszus wrote:
> Probably it would also be fair to not parallelise all modules (i.e.
> reserve only handful number of slots instead of almost all
> available).
> Currently we trigger 35 builds at the same time:
> https://github.com/apache/jackrabbit-oak/blob/6c04951be723c4574d3a7c1bcc4ec04e9f9e7dd0/Jenkinsfile#L23
>  <
> https://github.com/apache/jackrabbit-oak/blob/6c04951be723c4574d3a7c1bcc4ec04e9f9e7dd0/Jenkinsfile#L23
> > while ASF Jenkins has only about the same amount of slots.
> I think we should block at most a handful of nodes in parallel
> (although this may lead to slightly slower build times).
> 
> Konrad
> 
> > On 14. Nov 2022, at 16:29, Konrad Windszus  wrote:
> > 
> > Seems this is pretty easy:
> > https://stackoverflow.com/a/70375236 <https://stackoverflow.com/a/70375236
> > >
> > Can you come up with a PR?
> > Thanks,
> > Konrad
> > 
> > > On 14. Nov 2022, at 16:16, Marcel Reutegger
> > >  wrote:
> > > 
> > > Hi,
> > > 
> > > On 11.11.22, 15:11, "Robert Munteanu"  wrote:
> > > > I think it would be worthile to check if the build can be
> > > > optimised,
> > > > otherwise Oak builds are blocking many execution slots of the
> > > > Jenkins
> > > > ASF instance.
> > > 
> > > I did notice one thing. Our PRs may schedule a build each time
> > > commits
> > > are pushed to the branch. Is there a way to cancel an already
> > > running
> > > build when there are new changes coming in from a PR?
> > > 
> > > Regards
> > > Marcel
> > 
> 

Oak Jenkins build takes up too many resources

[Robert Munteanu]

Hi,

I was waiting for a small PR of mine in Sling to be built by Jenkins
and I saw that it was not picked for some time. Looking at the
executing jobs I noticed that there were many, different, Oak stages
being ran.

A sample PR check [1] shows that there are many stages (one per
module?) but each of them takes at least 20 minutes, usually over 30.

I think it would be worthile to check if the build can be optimised,
otherwise Oak builds are blocking many execution slots of the Jenkins
ASF instance.

Thanks,
Robert

[1]:
https://ci-builds.apache.org/job/Jackrabbit/job/oak-trunk-pr/job/snfe_during_recovery/

Re: Oak-it-osgi module

[Robert Munteanu]

On Wed, 2022-01-19 at 13:08 +, Carlo Jelmini wrote:
> Hi Robert,
> 
> > Well, it's probably much quicker if you use the Sling Starter.
> 
> Thanks to the pointer to Sling Starter. Looks interesting. I’ll
> definitively try to use it for manual tests instead of Quickstart.
> 
> While exploring the Sling documentation, I have found that Sling
> already provides support for PaxExam [0] with the
> org.apache.sling.testing.paxexam library [1].
> Are there any objections if I import this library in an Oak module to
> implement my Integration Tests?

(not from me, but as noted I'm not that active in Oak anymore)

> 
> > On that note, maybe Oak itself can be packaged as an OSGi
> > application
> > using the feature model and that can then be used as a basis for
> > integration tests.
> 
> I’m not familiar yet with the feature model and I’m not sure I
> understand what packaging Oak as an OSGi application entails in this
> context?

The feature model [2] allows you to describe an OSGi application:

- bundles
- configurations
- framework properties

etc in a standard format. What is being worked on in Sling is the basis
for the OSGi features specification.

Packaging Oak as a feature model application means listing the needed
bundles and configurations and launching it via the feature launcher.
It will be very similar to what is done in the Sling Starter, but
without a lot of the things available to Sling.

Whether that is actually useful for your particular scenario is another
discussion :-)

Thanks,
Robert

[2]:
https://sling.apache.org/documentation/development/feature-model.html

> 
> Thanks,
> 
> Carlo
> 
> [0]:
> https://sling.apache.org/documentation/development/testing-paxexam.html
> [1]: https://github.com/apache/sling-org-apache-sling-testing-paxexam
> 
> 
> 
> From: Robert Munteanu 
> Date: Tuesday, 18 January 2022 at 17:44
> To: oak-dev@jackrabbit.apache.org 
> Subject: Re: Oak-it-osgi module
> On Tue, 2022-01-18 at 16:38 +, Carlo Jelmini wrote:
> > Up until now, I have used AEM Quickstart to manually test Oak
> > changes, but the roundtrip is very slow, cumbersome, and
> > frustrating...
> 
> Well, it's probably much quicker if you use the Sling Starter.
> 
> On that note, maybe Oak itself can be packaged as an OSGi application
> using the feature model and that can then be used as a basis for
> integration tests.
> 
> That, of course, means that ITs will need to run from outside the
> application, but on the other hand a proper project setup will
> eliminate many runtime errors, like unsatisfied imports or missing
> components.
> 
> Thanks,
> Robert
> 
> [1]:
> https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fsling-org-apache-sling-starterdata=04%7C01%7Cjelmini%40adobe.com%7C30f2c130a7e04035576708d9daa1bded%7Cfa7b1b5a7b34438794aed2c178decee1%7C0%7C0%7C637781210483210866%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000sdata=KamBGKCeJxbK9uVloOxoFIwqp3WynYn5TzdkrPfm5jI%3Dreserved=0

Re: Oak-it-osgi module

[Robert Munteanu]

On Tue, 2022-01-18 at 16:38 +, Carlo Jelmini wrote:
> Up until now, I have used AEM Quickstart to manually test Oak
> changes, but the roundtrip is very slow, cumbersome, and
> frustrating...

Well, it's probably much quicker if you use the Sling Starter.

On that note, maybe Oak itself can be packaged as an OSGi application
using the feature model and that can then be used as a basis for
integration tests.

That, of course, means that ITs will need to run from outside the
application, but on the other hand a proper project setup will
eliminate many runtime errors, like unsatisfied imports or missing
components.

Thanks,
Robert

[1]: https://github.com/apache/sling-org-apache-sling-starter

Re: Oak-it-osgi module

[Robert Munteanu]

Hi,

On Tue, 2022-01-18 at 15:04 +, Angela Schreiber wrote:
> Hi Carlo
> 
> In the various security related modules I used
> org.apache.sling.testing.osgi-mock to test the OSGi setup.
> I never used oak-it-osgi as a base for tests.
> 

In my understanding these are slightly different.

The OSGi-mock library helps write (unit/fast) tests that use
OSGi/Declarative Services components. A fully fledged OSGi framework
validates bundle package imports and requirements and also exposes
components to a less forgiving environment, so it's geared more towards
integration/end-to-end testing.

There is value in both, and OSGi-mock tests probably should IMO be the
majority.

But, of course, this requires someone to writes and maintains these
tests :-)

Thanks,
Robert

Re: [VOTE] Release Apache Jackrabbit Oak 1.40.0

[Robert Munteanu]

On Fri, 2021-05-28 at 19:49 +0200, Miroslav Smiljanic wrote:
> Please vote on releasing this package as Apache Jackrabbit Oak 1.40.0.

+1, with

[INFO] 
[INFO] Apache Maven 3.8.1 (SUSE 3.8.1-2.1)
[INFO] OS name: "linux", version: "5.12.4-1-default", arch: "amd64", family: 
"unix"
[INFO] Java version: 11.0.11, vendor: Oracle Corporation, runtime: 
/usr/lib64/jvm/java-11-openjdk-11
[INFO] MAVEN_OPTS: 
[INFO] 
[INFO] ALL CHECKS OK   
[INFO] 

Thanks,
Robert

Re: [VOTE] Migrate to Git

[Robert Munteanu]

Hi,

On Wed, 2021-05-19 at 09:24 +0200, Konrad Windszus wrote:
> After the successful migration of FileVault to Git we should tackle
> Oak as well.
> I am volunteering to trigger the actual work (mostly on ASF INFRA
> side). I hereby propose to
> 
> 1. migrate the SVN repository at 
> https://svn.apache.org/repos/asf/jackrabbit/oak/ to a Git repository
> named "jackrabbit-oak"
> 2. migrate GitHub SVN mirror at 
> https://github.com/apache/jackrabbit-oak to mirror the new native Git
> repo (at Gitbox)
> 3. change the main branch name from "trunk" to "main"
> 4. make the SVN repository read only
> 
> Vote is open for 72 hours.
> [ ] +1, migrate Oak to Git
> [ ] 0
> [ ] -1, because...

I am not contributing enough to Oak to feel that I currently have a say
in the migration, but I think overall that this is a good idea and from
the Sling point of view experiences have been almost entirely positive.

Thanks,
Robert

Re: Getting commit 'origin' in a CommitHook/Editor

[Robert Munteanu]

Hi Marcel,

On Fri, 2020-09-11 at 06:55 +, Marcel Reutegger wrote:
> Hi Robert,
> 
> On 10.09.20, 19:01, "Robert Munteanu"  wrote:
> > I am trying to get commit origin information in an Editor ( well,
> > Validator actually ). The scenario is that I want to log writes to
> > paths
> > that should belong to the immutable mount (/libs, /apps) in a
> > composite
> > setup when preparing. The intercept is done simply to analyse who
> > writes
> > to /libs and /apps and report unexpected writes.
> > 
> > What I am doing right now is create a ValidatorProvider and record
> > all
> > the childNodeXXX operations. In the 'leave' method of the root
> > validator
> > I am doing the logging and walking up the current thread's stack
> > trace
> > to eliminate 'permitted' callers, like repoinit and the FileVault
> > package installer.
> > 
> > Is this approach sound at a high-level? Is there a better way of
> > getting
> > the commit 'origin'?
> 
> It depends how you define 'origin'. Alternatively, if 'origin' is
> more about
> _who_ writes, then your validator could use CommitInfo passed to
> ValidatorProvider.getRootValidator(). E.g. the CommitInfo contains
> the
> userId of the session that performs the write operation.
> 
> But I guess you are more interested in the code that invokes the
> write
> operation.

Yes, I'm interested in the subsystem/component, not the originating
user, so I opted to look at the stack trace.

Thanks,
Robert

Getting commit 'origin' in a CommitHook/Editor

[Robert Munteanu]

Hi,

I am trying to get commit origin information in an Editor ( well,
Validator actually ). The scenario is that I want to log writes to
paths that should belong to the immutable mount (/libs, /apps) in a
composite setup when preparing. The intercept is done simply to analyse
who writes to /libs and /apps and report unexpected writes.

What I am doing right now is create a ValidatorProvider and record all
the childNodeXXX operations. In the 'leave' method of the root
validator I am doing the logging and walking up the current thread's
stack trace to eliminate 'permitted' callers, like repoinit and the
FileVault package installer.

Is this approach sound at a high-level? Is there a better way of
getting the commit 'origin'?

Thanks,
Robert

Re: [filevault] Installing users and access control entries in the same package

[Robert Munteanu]

Hi Angela,

On Tue, 2020-08-25 at 07:39 +, Angela Schreiber wrote:
> hi robert
> 
> without having a closer look, i would suspect that your repository
> comes with default import-behavior configuration that strictly
> follows JCR specification which mandates a given principal to exist
> when dealing with access control management. oak comes with a variety
> of import-behavior options and if you configure it to be
> 'besteffort', it will allow to create ACEs for principals, which do
> not yet exist.

That does help, thank you. I've swiched the Sling Starter to use this
configuration from now on.

Best regards,

Robert

> 
> hope that helps
> angela
> ________
> From: Robert Munteanu 
> Sent: Friday, August 21, 2020 6:47 PM
> To: oak-dev@jackrabbit.apache.org 
> Subject: [filevault] Installing users and access control entries in
> the same package
> 
> Hi,
> 
> I am trying to install a content package that includes:
> 
> - two users under /home/users/slingshot
> - content with ACEs that reference the two users under
> /content/slingshot
> 
> When installing the content packages I see that the /content entry is
> processed first, leading to errors like
> 
>   E  /content/slingshot/users/slingshot1/rep:policy
>   ! org.xml.sax.SAXException:
> javax.jcr.security.AccessControlException: Unknown principal
> slingshot1
> javax.jcr.security.AccessControlException: Unknown principal
> slingshot1
> 
> and only later on is the user created
> 
>   -  /home
>   -  /home/users
>   A  /home/users/slingshot
>   A  /home/users/slingshot/slingshot1
> 
> Obviously, reinstalling the content package fixes the problem but I'm
> looking for a more error-safe way of installing the content package.
> 
> How can I install this content package with users and ACEs from
> without
> errors?
> 
> Thanks,
> Robert
> 

Re: [filevault] Installing users and access control entries in the same package

[Robert Munteanu]

Hi Kondrad,

On Fri, 2020-08-21 at 23:10 +0200, Konrad Windszus wrote:
> I would suggest to split it up into two packages and make the ACL
> package depend on the user package. You can even put those two
> packages into a common container package. The dependencies will
> enforce the right installation order.

Thanks for the suggestion, but I would like to avoid that duplication
if possible. I am currently splitting this application from a single
bundle to a bundle + two content packages ( sample content + apps ).
Splitting it further would make even more complicated to understand
IMO.

I am not sure if principal-based authentication would work here, but I
could give it a try.

Robert

[filevault] Installing users and access control entries in the same package

[Robert Munteanu]

Hi,

I am trying to install a content package that includes:

- two users under /home/users/slingshot
- content with ACEs that reference the two users under
/content/slingshot

When installing the content packages I see that the /content entry is
processed first, leading to errors like

  E  /content/slingshot/users/slingshot1/rep:policy
  ! org.xml.sax.SAXException:
javax.jcr.security.AccessControlException: Unknown principal slingshot1
javax.jcr.security.AccessControlException: Unknown principal slingshot1

and only later on is the user created

  -  /home
  -  /home/users
  A  /home/users/slingshot
  A  /home/users/slingshot/slingshot1

Obviously, reinstalling the content package fixes the problem but I'm
looking for a more error-safe way of installing the content package.

How can I install this content package with users and ACEs from without
errors?

Thanks,
Robert

Re: [filevault-package-maven-plugin] Packaging additional files in META-INF

[Robert Munteanu]

Hi Konrad,

On Fri, 2020-08-21 at 12:05 +0200, Konrad Windszus wrote:
> Hi Robert,
> Currently the filevault-package-maven-plugin creates packages from
> mainly two different source directories:
> 
> https://jackrabbit.apache.org/filevault-package-maven-plugin/package-mojo.html#jcrRootSourceDirectory
>  plugin/package-mojo.html#jcrRootSourceDirectory> -> ends up in the
> installed package
> https://jackrabbit.apache.org/filevault-package-maven-plugin/package-mojo.html#metaInfVaultDirectory
>  plugin/package-mojo.html#metaInfVaultDirectory> -> ends up in META-
> INF/vault
> 
> Probably we need an optional third source to put META-INF somewhere
> outside the vault directory.
> Feel free to raise a JIRA.

Thank you for the reply. I filed

  https://issues.apache.org/jira/browse/JCRVLT-471

Thanks,
Robert

[filevault-package-maven-plugin] Packaging additional files in META-INF

[Robert Munteanu]

Hi,

I am trying to use the filevault-package-maven-plugin with a Sling
project. By default, artifacts are checked to see if they have legal
files: META-INF/{LICENSE,NOTICE}.

When building a content package these are not included.  I've
configured the plug-in with:


org.apache.jackrabbit
filevault-package-maven-plugin
1.1.4
true

${project.build.outputDirectory}



The legal files from target/classes are understandably skipped since
they don't match the filter defintions, so that's not a good place for
them. What I would like is to have then copied over to META-INF in the
resulting content package, but I did not find an easy way to do this.

I worked around it with an additional resources plug-in execution:


maven-resources-plugin


copy-legal-files
prepare-package

copy-resources



${project.build.directory}/vault-work/META-INF



${project.build.directory}/maven-shared-archive-resources/META-INF/







Is there a simpler way to copy files to META-INF/ using the plug-in?

Thanks,
Robert

Re: [RT] Limited write support for non-default CompositeNodeStore mounts

[Robert Munteanu]

On Thu, 2020-07-09 at 15:35 +, Marcel Reutegger wrote:
> Hi,
> 
> On 09.07.20, 15:39, "Robert Munteanu"  wrote:
> > The details are rough and my memory of the implementation is not
> > that
> > good
> 
> Same here.
> 
> Besides the problem with a potential two phase commit, I think the
> biggest concern was impact of a change in a mounted subtree on
> repository wide data structures. E.g. what happens when an access
> control entry is added to such a subree? How are indexes updated for
> nodes changed in the subtree?

Well, it depends. The current check when merging changes is simple -
only allow them to the default (read-write) mount.

What I propose is allow writes to all mounts under given circumstances.
For instance (all examples inside a non-default mount):

- if there is an observation listener that should be triggered, fail
the commit
- if an index should be updated and we know we can't do that, fail the
commit

The downside is that it is less predictable that a 'plain' read-only
situation and can be confusing for users of the repository. Or that,
due to the way the repository is configured, effectively no changes can
be persisted ( e.g. observation listener for all of /apps ).

> 
> I'm also wondering whether and where the current implementation
> assumes
> mounted subtrees are read-only and takes short cuts. These would need
> to
> be changed as well.

Yes, those implementations need to be revisited as well. I am a bit
optimistic here since for instance indexing can write to a separate
location; this is one of the steps of preparing a composite node store.

Thanks,
Robert

[RT] Limited write support for non-default CompositeNodeStore mounts

[Robert Munteanu]

Hi,

I've been thinking about the read-only aspect of the CNS, and whether
it would be useful to relax this constraint. The primary driver (in the
Sling world) is that while content packages and repoinit allow easy
inspection of the payload so we can know ahead of time if anyone would
write to a non-default mount ( e.g. /apps ) there are two wildcards:

- bundle activators and OSGi components
- vault hooks

We cannot be sure that none of them will write in the repository in a
non-default mount, and for the sake of compatibility we could allow
writing.

There are two major constraints listed in the CNS documentation [1]:

- atomic commits across node stores
- oak subsystems that are not composite-aware

I would propose that we 'solve' these problems in the following ways:

1. Allow commits only over individual node stores, never commits that
span multiple stores
2. Validate that subsystems that are not composite-aware are not
affected. For instance, that no observation listeners are bound to
paths under non-default mounts.

The details are rough and my memory of the implementation is not that
good, but I would like to see what others think before committing to a
POC.

Thanks!
Robert

[1]: https://jackrabbit.apache.org/oak/docs/nodestore/compositens.html

Re: Request of (paid) support

[Robert Munteanu]

Hello Eugenio,

On Fri, 2020-06-05 at 14:14 +, Beato Eugenio wrote:
> Dear Team,
> I'm Eugenio Beato, Manager of the software factory of Siav (
> www.siav.it;).
> We're using OAK in our product and we're facing some issues
> (performances, indexing on SolR and others).
> Would you be available to helps us with a technical consultancy? We
> urgently need someone with architectural background of OAK. (Maybe
> you know a company that offer this service?)

It may help if you'd share the concrete problems you're facing, as
someone might've already run into them.

Thanks,
Robert

Re: Jackrabbit Composite Nodestore Example / Docu

[Robert Munteanu]

On Mon, 2020-03-30 at 08:36 -0700, Andreas Schaefer wrote:
> 
> I was wondering if there is some further documentation or examples
> available than just 
> https://jackrabbit.apache.org/oak/docs/nodestore/compositens.html  tps://jackrabbit.apache.org/oak/docs/nodestore/compositens.html> ?

For reference, I have a working demo of the Sling Starter running with
the CompositeNodeStore at

  
https://github.com/apache/sling-org-apache-sling-starter/tree/feature/composite-node-store

See README-composite.md for the details.

Thanks,
Robert

Re: Oak doesn't reindex lucene indexes - AsyncIndexUpdate is not ran

[Robert Munteanu]

Hi David,

On Tue, 2020-01-28 at 14:44 +0100, Dawid Cokan wrote:
> Hi all,
> 
> We built application using OSGi framework and we use the jackrabbit
> oak
> 1.6.17 to store some content. We have just enabled lucene indexes for
> full
> text search. We configured them accordingly to documentation and
> search
> works correctly. When I add new content the lucene index is updated
> as
> commit hook and new items can be seen in search results. However
> after
> restarting the server search is returning no results. I noticed that
> after
> restart my index is marked to be reindexed by OAK but it seems it
> doesn't
> happen - property reindex is never set back to false. While debugging
> the
> code I can see that instance of AsyncIndexUpdate which performs async
> reindexing is properly registered to ServiceRegistry but gets never
> invoked.
> 
> I analysed the code and the only code that possibly could execute
> that
> service is a service tracker -
> org.apache.jackrabbit.oak.run.osgi.RunnableJobTracker - which comes
> with
> oak-pojosr. Indeed when adding a oak-pojosr to our dependencies and
> starting that service tracker issue is solved. However I don't
> understand
> why should I add oak-pojosr to our environment since oak-pojosr is
> intended
> to mimic OSGi behaviour and as per my understanding should be used in
> case
> of running in non OSGi environment which is not our case.

You're right, PojoSR should not be required and is only needed when
running outside of OSGi environments.

I don't have an answer to your question, but maybe looking on how we
use Oak in Apache Sling can point you in the right direction.

You will find all bundles/configs listed under [1] and the repository
is set up using the bundle from [2]

Hope this helps,
Robert


[1]: 
https://github.com/apache/sling-org-apache-sling-starter/tree/master/src/main/provisioning
[2]: https://github.com/apache/sling-org-apache-sling-jcr-oak-server

Re: [Jenkins] Oak Windows build

[Robert Munteanu]

On Thu, 2019-09-12 at 13:52 +0200, Julian Reschke wrote:
> On 12.09.2019 09:47, Robert Munteanu wrote:
> > Hi,
> > 
> > Some time ago I set up a windows build for Oak
> > 
> >https://builds.apache.org/job/Jackrabbit-Oak-Windows/
> > 
> > I noticed that it's still running, passing most of time. The build
> > is
> > relatively stable and I was wondering if it would be useful to make
> > it
> > an 'official' build and enable notifications for it.
> > 
> > I think as an experiment it has run its course and we either
> > 'promote'
> > it or shut it down.
> > 
> > Thoughts?
> 
> Keep it!
> 

Works for me :-) I've directed notifications to oak-dev in the email
config and added a comment for it at [1].

Thanks,
Robert

[1]: https://issues.apache.org/jira/browse/OAK-8325

[Jenkins] Oak Windows build

[Robert Munteanu]

Hi,

Some time ago I set up a windows build for Oak

  https://builds.apache.org/job/Jackrabbit-Oak-Windows/

I noticed that it's still running, passing most of time. The build is
relatively stable and I was wondering if it would be useful to make it
an 'official' build and enable notifications for it.

I think as an experiment it has run its course and we either 'promote'
it or shut it down.

Thoughts?

Robert

Configuring Oak for direct binary access

[Robert Munteanu]

Hi,

I'd like to configure Oak for direct binary access (with Azure ideally)
to take it for a spin.

I've look at 

  https://wiki.apache.org/jackrabbit/Direct%20Binary%20Access

and

  https://jira.apache.org/jira/browse/OAK-7569


but it's not clear to me how to actually configure Oak to work this way
:-), ideally with Sling.

Suggestions welcome.

Thanks,

Robert

Re: [DISCUSS] Branching and release: version numbers

[Robert Munteanu]

On Mon, 2019-03-04 at 13:24 +, Davide Giannella wrote:
> On 04/03/2019 12:08, Robert Munteanu wrote:
> > It all sounds reasonable to me. One comment regarding your example
> > -
> > branching does not necessarily force to increase the major
> > component of
> > the version.
> > 
> > > From your initial email, one use case for having maintenance
> > > branches
> > is to support incompatible changes in the JVM.
> > 
> > Asumming that with 1.26 we want to end Java 19 support, then the
> > next
> > Oak release could be 1.28, as we did not previously treat such
> > changes
> > as 'breaking'.
> 
> hmmm. We will add/change JVM even without branching if it will not
> require us to change API in a non-compatible way. By the definition I
> gave before a Major change is for:
> 
> > > MAJOR version when you make incompatible API changes.
> 
> so while in the past we didn't branch for JVM itself I think it
> happened
> either along with a new branch anyhow or it didn't require us to
> change
> our API in a non-compatible way.

I was referring to your previous email [1] which listed

- incompatible changes in the JVM which we may have or want to use

My understanding is that this would be something like "make use of new
API available only from Java 11".  And that was listed as a reason for
branching.

As you mentioned, we don't need to increase the major version whenever
we branch. I just wanted to clarify that since in this email thread
branching seems to be conflated with major version increases and that
IMO not correct (and your reply seems to support that).

Thanks,

Robert

[1]: 
https://lists.apache.org/thread.html/768807eed3379dc08921a1510264136ffe4a7a1230d9ca7881cc0a59@%3Coak-dev.jackrabbit.apache.org%3E

Re: [DISCUSS] Branching and release: version numbers

[Robert Munteanu]

Hi Davide,

On Fri, 2019-03-01 at 09:48 +, Davide Giannella wrote:
> Now let's say we have to branch at 1.26 time we would create a branch
> `svn/.../branches/1.26` that will produce thereafter 1.26.0, 1.26.1,
> etc.
> 
> Trunk will become: 2.0, 2.2, 2.4, 2.6, etc.

It all sounds reasonable to me. One comment regarding your example -
branching does not necessarily force to increase the major component of
the version.

>From your initial email, one use case for having maintenance branches
is to support incompatible changes in the JVM.

Asumming that with 1.26 we want to end Java 19 support, then the next
Oak release could be 1.28, as we did not previously treat such changes
as 'breaking'.

Thanks,

Robert

Re: [DISCUSS] branching and release strategies

[Robert Munteanu]

On Wed, 2019-02-20 at 14:54 +, Marcel Reutegger wrote:
> Hi,
> 
> On 20.02.19, 13:21, "Robert Munteanu"  wrote:
> > On Wed, 2019-02-20 at 10:45 +, Davide Giannella wrote:
> > > - Any previous oak release will be automatically deprecated. What
> > > has
> > > been already branched and released still stays there. This
> > > applies
> > > only
> > > to future releases.
> > 
> > So say we release 1.12 from trunk and then 1.12.1 from trunk. Does
> > that
> > mean that 1.12 becomes instantly deprecated?
> 
> I wouldn't call it deprecated, but 1.12.1 will be the new recommended
> version. This specific situation is very similar to how we do
> maintenance
> releases. The most recent release from a maintenance branch is the
> one
> we recommend.
> 
> > Do we even plan to make minor releases such as 1.12.1 or will 1.13
> > immediately follow after 1.12?
> 
> That's a good question. In my view we could make micro releases based
> on what has changed since the last minor release. If it's just bug
> fixes,
> release as 1.12.1 otherwise move to 1.14.
> 
> Note the even minor version. To avoid confusion, we could still say
> only
> even minor versions are considered stable release. Even though  we
> wouldn't create releases anymore with uneven minor version numbers.

Ack, it all makes sense.

There is an argument to be made for also making unstable releases now
that we're changing things anyway, but it's not something that I view
as terribly important. In Sling we always release even versions for
technical reasons and I don't remember anyone complaining about it.

Thanks,

Robert

Re: [DISCUSS] branching and release strategies

[Robert Munteanu]

Hi,

On Wed, 2019-02-20 at 10:45 +, Davide Giannella wrote:
> - Any previous oak release will be automatically deprecated. What has
> been already branched and released still stays there. This applies
> only
> to future releases.

So say we release 1.12 from trunk and then 1.12.1 from trunk. Does that
mean that 1.12 becomes instantly deprecated?

Do we even plan to make minor releases such as 1.12.1 or will 1.13
immediately follow after 1.12?

Thanks,

Robert

Re: [Monitoring] - Provisioning StatisticsProvider in Oak

[Robert Munteanu]

On Wed, 2018-11-28 at 11:34 +0100, Francesco Mari wrote:
> On Tue, 27 Nov 2018 at 11:45, Tommaso Teofili <
> tommaso.teof...@gmail.com>
> wrote:
> 
> > After having dig into the code for a while I can see following
> options:
> > - have a dedicated OSGi service on the query side that can be then
> > polled in a static way from QueryEngineImpl in order to obtain such
> a
> > reference (see patch from OAK-7904 [1])
> >
> 
> I suggest you don't pursue this option. I find that every service
> introduces additional complexity and hinders the testability of the
> code.
> In oak-segment-tar we pushed all the OSGi bits to a couple of
> services
> (read below) and everything else is just plain Java code that
> explicitly
> declares its dependencies. The API is clearer and the code is much
> more
> testable. You don't want to resort on oak-it-osgi to test your stuff
> ;)

Note that with OSGi R7 you can use constructor injection, and that
makes it really easy to construct your object graphs manually for
testing. So no need to resort to ITs for testing OSGi-aware components.

Not really arguing for one option or the other, but wanted to make sure
everyone is aware of the most recent changes.

Thanks,

Robert

Re: Oak Composite Store

[Robert Munteanu]

Hi Rory,

On Fri, 2018-11-02 at 10:47 +1100, Rory Elrick wrote:
> Hi,
> 
> I am interested in using the Oak Composite Store but am having a hard
> time
> finding any detailed information or examples of usage.
> 
> I have built release 1.8.9 of the oak-store-composite module and read
> the
> corresponding Java doc but still not sure how to proceed.
> 
> The use case is blue/green deployments as described here:
> 
> https://adapt.to/content/dam/adaptto/production/presentations/2017/adaptTo2017-Zero-downtime-deployments-for-Sling-based-apps-Tomek-Rekawek-Robert-Munteanu.pdf

At the time there is not much documentation and tooling around on how
to use the composite node store, sorry. I assume you're using Sling or
AEM as you pasted the adaptTo link, you may need to adapt your steps
otherwise.

As a general outline regarding its usage, I would suggest investigating
the following generic actions:

1. You need to generate a repository that can be mounted under /libs
and /apps. Simplest way is to start up Sling/AEM and then shut it down.
Then you have your repository.

2. Move that repository to another place on disk, for instance
'repository-libs'

3. Start up Sling/AEM again, this time configuring the
CompositeNodeStore with two SegmentNodeStoreFactory instances, one
pointing to the 'global' repository - which is ok to not exist, and
another pointing to the 'libs' repository, which is the one you
generated in step 1, and set to read-only. 

There are improvements to be made ( e.g. in step 1 you could make sure
to only store /libs and /apps ) but this is the general direction.

Hope this proves useful in your investigations,

Robert

Re: [VOTE] Release Apache Jackrabbit Oak 1.8.8

[Robert Munteanu]

On Tue, 2018-09-25 at 11:33 +0100, Davide Giannella wrote:
> Please vote on releasing this package as Apache Jackrabbit Oak 1.8.8.

+1, with

[INFO] 
[INFO] Apache Maven 3.5.4 (1edded0938998edf8bf061f1ceb3cfdeccf443fe; 
2018-06-17T20:33:14+02:00)
[INFO] OS name: "linux", version: "4.18.8-1-default", arch: "amd64", family: 
"unix"
[INFO] Java version: 1.8.0_171, vendor: Oracle Corporation, runtime: 
/usr/lib64/jvm/java-1.8.0-openjdk-1.8.0/jre
[INFO] 

Thanks,

Robert


signature.asc
Description: This is a digitally signed message part

[m12n] Modularised module releases in Oak

[Robert Munteanu]

Hi,

During the Oakathon we've been discussing the possibility of switching
to individual module releases.

I've gathered the notes at [1] and would like to gather input from the
wider oak-dev audience.

So please go through the wiki page and reply to the list with any
comments/criticisms and hopefully raising issues we did not see during
the discussion.

Note that we are still planning to be able to change our mind and roll
back any modularised release work in case we decide it was a bad idea.

We also have not yet decided which module should be the guinea pig so
input is welcome on that as well.

Thanks,

Robert


[1]: 
https://wiki.apache.org/jackrabbit/Oakathon%20September%202018#Modularization

Re: [VOTE] Release Apache Jackrabbit Oak 1.8.6

[Robert Munteanu]

On Tue, 2018-07-31 at 17:47 +0200, Manfred Baedke wrote:
> Please vote on releasing this package as Apache Jackrabbit Oak 1.8.6.

+1, with

[INFO] 
[INFO] Apache Maven 3.5.4 (1edded0938998edf8bf061f1ceb3cfdeccf443fe; 
2018-06-17T20:33:14+02:00)
[INFO] OS name: "linux", version: "4.17.9-1-default", arch: "amd64", family: 
"unix"
[INFO] Java version: 1.8.0_171, vendor: Oracle Corporation, runtime: 
/usr/lib64/jvm/java-1.8.0-openjdk-1.8.0/jre
[INFO] 

Thanks,

Robert


signature.asc
Description: This is a digitally signed message part

Re: [DISCUSS] Enabling CI for Oak cloud-based features

[Robert Munteanu]

On Tue, 2018-07-31 at 13:49 +0530, Chetan Mehrotra wrote:
> For Azure specific services we would need to
> get an account created and use that from our Jenkins server.

I know Tomek has been using azurite [1] with some success ( although
not 100% perfect ).

Robert

[1]: https://github.com/azure/azurite

Re: [VOTE] Release Apache Jackrabbit Oak 1.9.5

[Robert Munteanu]

On Tue, 2018-07-03 at 13:20 +0100, Davide Giannella wrote:
> Please vote on releasing this package as Apache Jackrabbit Oak 1.9.5.

+1


where

[INFO] 
[INFO] Apache Maven 3.5.3 (3383c37e1f9e9b3bc3df5050c29c8aff9f295297; 
2018-02-24T21:49:05+02:00)
[INFO] OS name: "linux", version: "4.17.1-1-default", arch: "amd64", family: 
"unix"
[INFO] Java version: 1.8.0_171, vendor: Oracle Corporation
[INFO] 

Thanks,

Robert

signature.asc
Description: This is a digitally signed message part

Re: Jenkins failures

[Robert Munteanu]

On Wed, 2018-04-11 at 09:39 +0200, Julian Reschke wrote:
> Hi,
> 
> seems that many of our builds fail on "H20", see 
> .
> 
> > [ERROR] Failed to execute goal org.codehaus.mojo:animal-sniffer-
> > maven-plugin:1.16:check (animal-sniffer) on project oak-parent:
> > Execution animal-sniffer of goal org.codehaus.mojo:animal-sniffer-
> > maven-plugin:1.16:check failed: Plugin org.codehaus.mojo:animal-
> > sniffer-maven-plugin:1.16 or one of its dependencies could not be
> > resolved: Failed to collect dependencies at
> > org.codehaus.mojo:animal-sniffer-maven-plugin:jar:1.16 ->
> > org.codehaus.mojo:animal-sniffer:jar:1.16 -> org.ow2.asm:asm-
> > all:jar:5.2: Failed to read artifact descriptor for
> > org.ow2.asm:asm-all:jar:5.2: Could not transfer artifact
> > org.ow2.asm:asm-all:pom:5.2 from/to central (https://repo.maven.apa
> > che.org/maven2): /home/jenkins/.m2/repository/org/ow2/asm/asm-
> > all/5.2/asm-all-5.2.pom.part.lock (Permission denied) -> [Help 1]
> > [ERROR] 
> 
> Does anybody know how to get that machine fixed, or how to configure
> our 
> builds not to use it?

File an INFRA bug, Jenkins slaves failures get sorted out decently
fast.

Robert

Re: Recommending Oak over Jackrabbit 2.x

[Robert Munteanu]

On Tue, 2018-02-27 at 12:21 -0500, Matt Ryan wrote:
> Are there use cases where users should prefer Jackrabbit over Oak? 
> Or is
> Oak considered a full replacement for Jackrabbit in every case?

My understanding is that Jackrabbit is a reference implementation of
the spec, with all bells and whistles, while Oak does not implement the
parts that are tricky to get done in a performant way, e.g.
EventJournal or multiple workspaces.

Robert

Recommending Oak over Jackrabbit 2.x

[Robert Munteanu]

Hi,

Recent questions to the jackrabbit user's list lead me to believe that
users are still picking Jackrabbit 2.x as a first implementation
choice. I believe that we should be pointing them to Oak instead, as
that's (IMO) the better documented, supported and implemented version
of a JCR repository.

I haven't sketched out what needs to be changed in the docs, as I'd
like to get your thoughts on this first.

Thanks,

Robert

JDK 11 EA builds

[Robert Munteanu]

Hi,

JDK 11 builds are our, in case someone wants to do some early testing.

  http://jdk.java.net/11/

Robert

Re: OAK-7203 - Where should MountInfoProviderService reside?

[Robert Munteanu]

On Tue, 2018-02-13 at 15:29 +0100, Oliver Lietz wrote:
> On Tuesday 13 February 2018 14:37:29 Alex Deparvu wrote:
> > Hi,
> 
> Hi,
> 
> > I would not move it to oak-core, it would be (I think) a step in
> > the wrong
> > direction wrt. the modularization effort.
> 
> seriously, which direction is it? oak-core now depends on oak-store-
> composite 
> (which provides optional features) and oak-lucene depends on oak-
> store-
> document which is otherwise not required when running Oak with
> Segment Tar 
> store. What's the point of this m12n effort?

oak-lucene depending on oak-store-document is not a very good thing
IMO, looks wrong. But that's an implementation problem, not a design
fault in modularizing Pak. Filed [1] for it.

The reasoning behind this is documented in OAK-6069 - Modularisation of
Oak [2], and the main driver is stop releasing all modules at a time.
For instance, a bugfix in oak-store-segment should not require
_everything_ to be released.

[1]: https://issues.apache.org/jira/browse/OAK-7263
[2]: https://issues.apache.org/jira/browse/OAK-6069

Re: OAK-7203 - Where should MountInfoProviderService reside?

[Robert Munteanu]

On Tue, 2018-02-13 at 14:37 +0100, Alex Deparvu wrote:
> Hi,
> 
> I would not move it to oak-core, it would be (I think) a step in the
> wrong
> direction wrt. the modularization effort.
> 
> Re. OAK-7203, I think we should make that specific dependency
> optional, but
> I'm not convinced you won't have another bundle pulling in the
> composite
> dependency anyway.

Making the dependency optional would complicate the component. There
are a number of components that perform initialization work and depend
on the MountInfoProvider to know whether they should write data in a
composite-friendly setup. AN example are the index-related classes.

Initialising those without a MountInfoProvider or with a 'default'
MountInfoProvider opens up the door to data being incorrectly handled
for a short window of time.

I think we should keep the dependency required.

Thanks,

Robert

> 
> 
> best,
> alex
> 
> 
> 
> 
> On Tue, Feb 13, 2018 at 1:46 PM, Robert Munteanu <romb...@apache.org>
> wrote:
> 
> > On Tue, 2018-02-13 at 13:04 +0100, Oliver Lietz wrote:
> > > On Tuesday 13 February 2018 13:10:23 Robert Munteanu wrote:
> > > > On Tue, 2018-02-13 at 11:51 +0100, Oliver Lietz wrote:
> > > > > > 1. Move the service to oak-core.
> > > > > > 2. Require oak-store-composite for deployments
> > > > > > 
> > > > > > If we go with 1, we have simpler deployments ( one less
> > > > > > bundle
> > > > > > ).
> > > > > > If we
> > > > > > go with 2, we split the logic from the oak-store-composite
> > > > > > bundle
> > > > > > and
> > > > > > add more stuff on top of oak-core.
> > > > > 
> > > > > 1 means simpler deployment and more stuff in oak-core, but
> > > > > the
> > > > > MountInfoProvider is required for composite and non-composite
> > > > > stores.
> > > > > Having it in (experimental) module oak-store-composite feels
> > > > > strange.
> > > > 
> > > > Why do you consider oak-store-composite experimental? It's not
> > > > documented very well unfortunately, but it's as well-tested as
> > > > any
> > > > other component in Oak from my point of view.
> > > 
> > > AFAIR Oak's documentation says it's "experimental". The composite
> > > ns
> > > page says
> > > it's work-in-progress – outdated documentation?
> > 
> > That's a good point. I've updated the docs to remove the
> > information
> > about the compositens being work-in-progress, just the
> > documentation is
> > :-)
> > 
> > Robert
> > 

Re: OAK-7203 - Where should MountInfoProviderService reside?

[Robert Munteanu]

On Tue, 2018-02-13 at 13:04 +0100, Oliver Lietz wrote:
> On Tuesday 13 February 2018 13:10:23 Robert Munteanu wrote:
> > On Tue, 2018-02-13 at 11:51 +0100, Oliver Lietz wrote:
> > > > 1. Move the service to oak-core.
> > > > 2. Require oak-store-composite for deployments
> > > > 
> > > > If we go with 1, we have simpler deployments ( one less bundle
> > > > ).
> > > > If we
> > > > go with 2, we split the logic from the oak-store-composite
> > > > bundle
> > > > and
> > > > add more stuff on top of oak-core.
> > > 
> > > 1 means simpler deployment and more stuff in oak-core, but the
> > > MountInfoProvider is required for composite and non-composite
> > > stores.
> > > Having it in (experimental) module oak-store-composite feels
> > > strange.
> > 
> > Why do you consider oak-store-composite experimental? It's not
> > documented very well unfortunately, but it's as well-tested as any
> > other component in Oak from my point of view.
> 
> AFAIR Oak's documentation says it's "experimental". The composite ns
> page says 
> it's work-in-progress – outdated documentation?

That's a good point. I've updated the docs to remove the information
about the compositens being work-in-progress, just the documentation is
:-)

Robert

Re: OAK-7203 - Where should MountInfoProviderService reside?

[Robert Munteanu]

On Tue, 2018-02-13 at 11:51 +0100, Oliver Lietz wrote:
> > 1. Move the service to oak-core.
> > 2. Require oak-store-composite for deployments
> > 
> > If we go with 1, we have simpler deployments ( one less bundle ).
> > If we
> > go with 2, we split the logic from the oak-store-composite bundle
> > and
> > add more stuff on top of oak-core.
> 
> 1 means simpler deployment and more stuff in oak-core, but the 
> MountInfoProvider is required for composite and non-composite stores.
> Having it in (experimental) module oak-store-composite feels strange.

Why do you consider oak-store-composite experimental? It's not
documented very well unfortunately, but it's as well-tested as any
other component in Oak from my point of view.

Robert

OAK-7203 - Where should MountInfoProviderService reside?

[Robert Munteanu]

Hi,

In OAK-7203 [1] we're discussing the best location for the
MountInfoProviderService. The context is that due to the addition of
the mounts concept a MountInfoProvider implementation is required and
for OSGi deployment we have to add oak-store-composite.

There are two options here:

1. Move the service to oak-core. 
2. Require oak-store-composite for deployments

If we go with 1, we have simpler deployments ( one less bundle ). If we
go with 2, we split the logic from the oak-store-composite bundle and
add more stuff on top of oak-core.

Thoughts?

Thanks,

Robert


[1]: https://issues.apache.org/jira/browse/OAK-7203

[Fwd: Jenkins build is still unstable: Jackrabbit-Oak-Windows #110]

[Robert Munteanu]

Hi,

We have gathered some failures on Windows ( 11 tests failing now ). 


org.apache.jackrabbit.oak.run.osgi.DocumentNodeStoreConfigTest.testRDBDocumentStoreRestart

org.apache.jackrabbit.oak.run.osgi.RepositoryShutdownTest.multipleShutdownAndWait

org.apache.jackrabbit.oak.run.osgi.DocumentNodeStoreConfigTest.testRDBDocumentStoreLateDataSource

org.apache.jackrabbit.oak.run.osgi.DocumentNodeStoreConfigTest.testRDBDocumentStoreLateDataSource

org.apache.jackrabbit.oak.segment.standby.BrokenNetworkIT.testProxySSLSkippedBytes

org.apache.jackrabbit.oak.segment.standby.BrokenNetworkIT.testProxyFlippedStartByte

org.apache.jackrabbit.oak.segment.standby.BrokenNetworkIT.testProxySSLSkippedBytesIntermediateChange

org.apache.jackrabbit.oak.segment.standby.ExternalPrivateStoreIT.testResilientSync

org.apache.jackrabbit.oak.segment.standby.ExternalPrivateStoreIT.testSyncUpdatedBinaryProperty
org.apache.jackrabbit.oak.segment.standby.ExternalPrivateStoreIT.testSync

org.apache.jackrabbit.oak.segment.standby.ExternalPrivateStoreIT.testSyncBigBlob

Would be great to have them reviewed by those that touched those areas of code 
recently.

Thanks,

Robert--- Begin Message ---
See 


--- End Message ---

Intent to backport OAK-7136 to 1.8

[Robert Munteanu]

Hi,

I intent to backport OAK-7136 to the 1.8 stable branch. The fixes are
basically moving the metatype.properties files in the right directory,
so I expect zero issues.

Thanks,

Robert

Re: [VOTE] Release Apache Jackrabbit Oak 1.8.0

[Robert Munteanu]

On Tue, 2018-01-09 at 11:53 +, Davide Giannella wrote:
> Please vote on releasing this package as Apache Jackrabbit Oak 1.8.0.

+1

> [INFO] 
> 
> [INFO] Apache Maven 3.5.2 (138edd61fd100ec658bfa2d307c43b76940a5d7d; 
> 2017-10-18T10:58:13+03:00)
> [INFO] OS name: "linux", version: "4.13.11-0-default", arch: "amd64", family: 
> "unix"
> [INFO] Java version: 1.8.0_151, vendor: Oracle Corporation
> [INFO] 
> 

Thanks,

Robert

signature.asc
Description: This is a digitally signed message part

Re: [windows] Jenkins test failure - o.a.j.o.index.indexer.document.flatfile.FlatFileStoreTest.basicTest

[Robert Munteanu]

On Fri, 2017-12-22 at 06:07 +0530, Vikas Saurabh wrote:
> Hi Robert,
> 
> > I am not able to qualify whether this is a valid failure or not,
> > therefore I'm bringing it to the mailing list.
> > 
> 
> That should be independent of OS... for me, following failed:
> MAVEN_OPTS='-Xmx512m -Xms512m' mvn test -pl :oak-run
> -Dtest=FlatFileStoreTest#basicTest
> 
> Opened OAK-7108 and fixed in r1818991.


Thanks! Windows build is now passing again.

Robert

[windows] Jenkins test failure - o.a.j.o.index.indexer.document.flatfile.FlatFileStoreTest.basicTest

[Robert Munteanu]

Hi,

The
org.apache.jackrabbit.oak.index.indexer.document.flatfile.FlatFileStore
Test.basicTest has failed for the past 2 runs on the Windows build job:

https://builds.apache.org/job/Jackrabbit-Oak-Windows/lastBuild/org.apac
he.jackrabbit$oak-
run/testReport/org.apache.jackrabbit.oak.index.indexer.document.flatfil
e/FlatFileStoreTest/basicTest/

The error message is: java.lang.IllegalArgumentException: Invalid
threshold: 2147483648 > max (358088704).

I am not able to qualify whether this is a valid failure or not,
therefore I'm bringing it to the mailing list.

Thanks,

Robert

Re: Experimental build for Oak on Windows

[Robert Munteanu]

On Wed, 2017-12-06 at 13:48 +0200, Robert Munteanu wrote:
> I'll keep it alive for a couple of weeks to assess its stability, and
> then we can discuss whether we want to promote it to a 'proper' job
> that we actually pay attention to and that sends notifications.

The builds have started failing due to Jenkins issues, so it seems the
stability is not how we need it.

I've reported it at [1], hopefully we can eliminate the root cause of
such problems.

Robert

[1]: https://lists.apache.org/thread.html/3c614a08275ff7484c51bfe025d8b
b983aaf58a407dde53f7bf354e7@%3Cbuilds.apache.org%3E

Re: Build failure due to out of heap in oak-solr

[Robert Munteanu]

On Fri, 2017-12-15 at 12:52 +0100, Michael Dürig wrote:
> We shouldn't add any memory setting in the Jenins jobs but rely on
> the 
> test.opts.memory property of the parent pom.xml. This is
> intentionally 
> set to 512M so we find out about potential regressions early.

That's interesting. So if the memory setup is the same, why does the
Windows build not fail? My only guess is for windows the JVM (or OS
alltogether) is 32-bit.

Robert

Re: Build failure due to out of heap in oak-solr

[Robert Munteanu]

On Fri, 2017-12-15 at 15:18 +0530, Chetan Mehrotra wrote:
> > Caused by: java.lang.OutOfMemoryError: Java heap space
> 
> The build is failing due to OOM in oak-solr-core
> https://builds.apache.org/job/Jackrabbit%20Oak/1090/

FWIW, the windows build does not fail ( yay?) 

  https://builds.apache.org/job/Jackrabbit-Oak-Windows/

Granted, I have not set any -Xmx flag for this job so I don't know how
much memory it's taking. I see the 'main' job uses -Xmx2g, so maybe we
can bump it up to -Xmx3g, just as a test?

Robert

Re: Supported JVMs

[Robert Munteanu]

Hi Davide,

On Thu, 2017-12-14 at 11:27 +, Davide Giannella wrote:
> Good morning,
> 
> while looking at http://jackrabbit.apache.org/jcr/downloads.html I
> can
> see that we have a sort-of matrix for supported jvms.
> 
> However stating something like _Java 7 and later_ implies that we run
> test coverage on java9 as well; which is not true.
> 
> I think we should amend this page stating explicitly what JVMs we are
> testing and what we're expecting to work. Or maybe create an apposite
> page. I personal prefer a dedicated page.

For the record, this is what Tomcat does

  http://tomcat.apache.org/whichversion.html

Robert

Re: Experimental build for Oak on Windows

[Robert Munteanu]

On Tue, 2017-12-12 at 23:00 +0100, Julian Reschke wrote:
> > java.nio.file.InvalidPathException: Illegal char <:> at index 2:
> > /F:/jenkins/jenkins-slave/workspace/Jackrabbit-Oak-Windows/oak-
> > solr-
> > core/target/test-classes/solr
> > 
> > Not sure who can look into this, but it would be good to fix.
> > 
> > Thanks,
> > 
> > Robert
> 
> -> 

Thanks Julian, I missed those. So that's at least confirmation that the
build is still working :-)

Robert

Re: Experimental build for Oak on Windows

[Robert Munteanu]

On Wed, 2017-12-06 at 13:48 +0200, Robert Munteanu wrote:
> Hi,
> 
> I set up yesterday an experimental build for Oak on Windows
> 
>   https://builds.apache.org/job/Jackrabbit-Oak-Windows/
> 

I've switched from Git to SVN since Git sync seems to be problematic
lately.

With the latest build I get 4 failures in oak-solr-core

https://builds.apache.org/job/Jackrabbit-Oak-Windows/11/testReport/

that seem to be mostly test setup issues:

java.nio.file.InvalidPathException: Illegal char <:> at index 2:
/F:/jenkins/jenkins-slave/workspace/Jackrabbit-Oak-Windows/oak-solr-
core/target/test-classes/solr

Not sure who can look into this, but it would be good to fix.

Thanks,

Robert

Experimental build for Oak on Windows

[Robert Munteanu]

Hi,

I set up yesterday an experimental build for Oak on Windows

  https://builds.apache.org/job/Jackrabbit-Oak-Windows/

It _seems_ to be working fine, but I've marked it as experimental given
the historical stability issues with ASF Windows bots. Feel free to
double-check with it in case you have doubts regarding the status of
the build on Windows.

I'll keep it alive for a couple of weeks to assess its stability, and
then we can discuss whether we want to promote it to a 'proper' job
that we actually pay attention to and that sends notifications.

Thanks,

Robert

Re: [VOTE] Release Apache Jackrabbit Oak 1.6.7

[Robert Munteanu]

On Tue, 2017-11-28 at 10:15 +0100, Davide Giannella wrote:
> Please vote on releasing this package as Apache Jackrabbit Oak 1.6.7.

[INFO] 

[INFO] Apache Maven 3.5.2 (138edd61fd100ec658bfa2d307c43b76940a5d7d;
2017-10-18T10:58:13+03:00)
[INFO] OS name: "linux", version: "4.13.11-0-default", arch: "amd64",
family: "unix"
[INFO] Java version: 1.8.0_151, vendor: Oracle Corporation
[INFO] 


+1

Robert

signature.asc
Description: This is a digitally signed message part

Re: [m12n] Module oak-spi-core defines no export versions

[Robert Munteanu]

On Tue, 2017-11-14 at 13:51 +, Angela Schreiber wrote:
> Any preference wrt the initial version number?

The initial version number should be 1.0.0 IMO.

Robert

Re: Naming convention for unstable releases

[Robert Munteanu]

On Tue, 2017-10-17 at 07:28 +0200, Julian Reschke wrote:
> Should we consider to label them accordingly in the future? Such as
> 
>1.9.0-EXPERIMENTAL

I'm +0 on such a change as I would expect users to actually read the
fine web pages at [1].

If we go through with this, I'd to with something less scary that
experimental. Maybe '-dev' or '-unstable'?

Robert


[1]: http://jackrabbit.apache.org/jcr/downloads.html

Re: Oak Bundle versioning.

[Robert Munteanu]

On Fri, 2017-10-06 at 15:06 +0100, Ian Boston wrote:
> Hi,
> 
> On 6 October 2017 at 15:02, Robert Munteanu <romb...@apache.org>
> wrote:
> 
> > Hi Ian,
> > 
> > Thanks for starting the discussion. I think this can be one of the
> > big
> > benefits of modularising Oak and I am interested in seeing this
> > being
> > done.
> > 
> > As you mentioned, it becomes easier to integrate various Oak
> > changes,
> > especially for consumers only depending on stable APIs.
> > 
> > On Thu, 2017-10-05 at 13:33 +0100, Ian Boston wrote:
> > > Obviously bundles remain the release unit, and the build must
> > > include
> > > OSGi
> > > based integration tests that validate a release is viable.
> > 
> > This brings about a question - what is an Oak release? If doing
> > per-
> > module releases, will we also do a "product" releases?
> > 
> > A product release in my view is - similar to the Sling Launchpad -
> > a
> > way of saying 'these module versions are guaranteed to work
> > together
> > beyond compilation and semantic versioning constraints'.
> > 
> > Also of interest is how/if we want to address the issue of
> > supporting
> > various module versions combinations. So if we have ( for instance
> > )
> > 
> > - oak-api 1.7.9
> > - oak-core 1.7.12
> > - oak-segment-tar 1.8.0
> > 
> > will these work together? Furthermore, which versions of oak-
> > upgrade
> > and oak-run are compatible with this combination?
> > 
> 
> 
> Perhaps, there needs to be a Oak Quickstart jar to define a
> combination of
> jars that work.
> Perhaps that is oak-run ?

Yes, that is one option.

The question is how do we actually expect consumers to define their
usage of Oak. 

One clear case is OSGi via Sling - and that can be defined via a Sling
feature.

Another is the one you mentioned - a runnable jar and that is oak-run.

Yeat another one can be embedding Oak in their project via
Maven/Gradle/etc dependencies . This is currently done in the oak-
examples module. Here we can consider defining an 'uber-pom' that is
used to pull in dependencies of a given Oak release.

Robert

Re: Oak Bundle versioning.

[Robert Munteanu]

Hi Ian,

Thanks for starting the discussion. I think this can be one of the big
benefits of modularising Oak and I am interested in seeing this being
done.

As you mentioned, it becomes easier to integrate various Oak changes,
especially for consumers only depending on stable APIs.

On Thu, 2017-10-05 at 13:33 +0100, Ian Boston wrote:
> Obviously bundles remain the release unit, and the build must include
> OSGi
> based integration tests that validate a release is viable.

This brings about a question - what is an Oak release? If doing per-
module releases, will we also do a "product" releases? 

A product release in my view is - similar to the Sling Launchpad - a
way of saying 'these module versions are guaranteed to work together
beyond compilation and semantic versioning constraints'.

Also of interest is how/if we want to address the issue of supporting
various module versions combinations. So if we have ( for instance )

- oak-api 1.7.9
- oak-core 1.7.12
- oak-segment-tar 1.8.0

will these work together? Furthermore, which versions of oak-upgrade
and oak-run are compatible with this combination?

We should have these discussion first, and then (hopefully) switch to a
more modular release process.

Thanks,

Robert

Waiting for an async index to be updated

[Robert Munteanu]

Hi,

In Sling we have some tests which validate that full text search is
working. Occasionaly this test times out because the full-text lucene
index is not updated and a traversal query is used. More details at
[1].

We should probably add a way of waiting for the index to be updated, so
my question is - what would be the way to do that? Ideally we would do
this from outside the Oak/Sling process, but can also deploy an OSGi
bundle if needed.

Thanks,

Robert


[1]: https://issues.apache.org/jira/browse/SLING-7169

Switch to official OSGi component and metatype annotations

[Robert Munteanu]

Hi,

You might've noticed that we've begun migrating to the OSGi R6
annotations. The biggest reason for this switch - besides moving to the
latest and greatest and from proprietary to standard - is using the
declarative services spec 1.3, see [1] for highlights.

There's a good write-up about the conversion at [2] and also you can
see how this went for oak-core and oak-store-document at [3].

So next time you work on your module it would be great if you also see
if it's one of the open sub-tasks of OAK-6471 [4] and convert it :-)

Thanks,

Robert


[1]: http://njbartlett.name/2015/08/17/osgir6-declarative-services.html
[2]: https://blog.osoco.de/2016/05/migrating-from-the-apache-felix-scr-
annotations-to-the-osgi-declarative-services-annotations/
[3]: https://svn.apache.org/r1810002
[4]: https://issues.apache.org/jira/browse/OAK-6741

Re: Debugging oak-pojosr test failures

[Robert Munteanu]

Eventually managed to understand the root cause using the Felix
ScrService and printing out component information.

See https://issues.apache.org/jira/browse/OAK-6736 .

Thanks,

Robert


On Wed, 2017-09-27 at 19:27 +0530, Chetan Mehrotra wrote:
> You need to do two changes
> 
> 1. Set ds.loglevel=4 as you have done
> 2. Configure logger corresponding to bundle symbolic name to trace
> level. Do it for the bundles you are interested in
> 
> 
> 
> > BTW, did you ever try to set up the web console? I would give it a
> > shot.
> 
> Not here but the standalone example do configure the Felix WebConsole
> [1]. You can try if standalone comes up with your changes to see it
> works for basic case
> 
> Chetan Mehrotra
> [1] https://github.com/apache/jackrabbit-oak/blob/trunk/oak-examples/
> standalone/src/main/java/org/apache/jackrabbit/oak/standalone/WebCons
> oleSupport.java
> 
> 
> On Wed, Sep 27, 2017 at 5:18 PM, Robert Munteanu <romb...@apache.org>
> wrote:
> > Hi Chetan,
> > 
> > On Wed, 2017-09-27 at 15:48 +0530, Chetan Mehrotra wrote:
> > > Hi Robert,
> > > 
> > > The unit-test.log should give details about which bundle is
> > > starting
> > > and which components are activated like [1]. For the error you
> > > are
> > > seeing it means RepositoryManager is not getting activated for
> > > some
> > > reason (may be some missing dependency).
> > > 
> > > So we would need to analyze from OSGi side to see which component
> > > has
> > > dependency missing. This is bit tricky in absence of Felix
> > > WebConsole.
> > > Probably enabling SCR logs (ds.loglevel=3?) would give some
> > > insight
> > > here
> > 
> > How would I do that? I tried the following change but got no extra
> > logs:
> > 
> > diff --git a/oak-
> > pojosr/src/test/groovy/org/apache/jackrabbit/oak/run/osgi/AbstractR
> > epos
> > itoryFactoryTest.groovy b/oak-
> > pojosr/src/test/groovy/org/apache/jackrabbit/oak/run/osgi/AbstractR
> > epos
> > itoryFactoryTest.groovy
> > index e0d08b5025..91f39bc277 100644
> > --- a/oak-
> > pojosr/src/test/groovy/org/apache/jackrabbit/oak/run/osgi/AbstractR
> > epos
> > itoryFactoryTest.groovy
> > +++ b/oak-
> > pojosr/src/test/groovy/org/apache/jackrabbit/oak/run/osgi/AbstractR
> > epos
> > itoryFactoryTest.groovy
> > @@ -57,6 +57,7 @@ abstract class AbstractRepositoryFactoryTest{
> >  config = [
> >  (REPOSITORY_HOME): workDir.absolutePath,
> >  (REPOSITORY_TIMEOUT_IN_SECS) : 60,
> > +"ds.loglevel" : 3
> >  ]
> >  }
> > 
> > BTW, did you ever try to set up the web console? I would give it a
> > shot.
> > 
> > Thanks,
> > 
> > Robert
> > 
> > > 
> > > Chetan Mehrotra
> > > [1]
> > > 27.09.2017 15:43:06.603 *INFO* [main]
> > > org.apache.jackrabbit.oak-query-spi BundleEvent STARTED
> > > 27.09.2017 15:43:06.603 *INFO* [main] org.apache.jackrabbit.oak-
> > > core
> > > BundleEvent STARTING
> > > 27.09.2017 15:43:06.761 *INFO* [main] org.apache.jackrabbit.oak-
> > > core
> > > Service
> > > [org.apache.jackrabbit.oak.spi.security.user.action.DefaultAuthor
> > > izab
> > > leActionProvider,17,
> > > [org.apache.jackrabbit.oak.spi.security.user.action.AuthorizableA
> > > ctio
> > > nProvider]]
> > > ServiceEvent REGISTERED
> > > 27.09.2017 15:43:06.762 *INFO* [main] org.apache.jackrabbit.oak-
> > > core
> > > Service
> > > [org.apache.jackrabbit.oak.security.authorization.restriction.Res
> > > tric
> > > tionProviderImpl,18,
> > > [org.apache.jackrabbit.oak.spi.security.authorization.restriction
> > > .Res
> > > trictionProvider]]
> > > ServiceEvent REGISTERED
> > > 27.09.2017 15:43:06.770 *INFO* [main]
> > > org.apache.jackrabbit.oak.plugins.metric.StatisticsProviderFactor
> > > y
> > > Using MetricsStatisticsProvider
> > > 27.09.2017 15:43:06.804 *INFO* [main] org.apache.jackrabbit.oak-
> > > core
> > > Service [19, [com.codahale.metrics.MetricRegistry]] ServiceEvent
> > > REGISTERED
> > > 27.09.2017 15:43:06.825 *INFO* [main]
> > > org.apache.jackrabbit.oak.segment.file.FileStore Creating file
> > > store
> > > FileStoreBuilder{version=1.8-SNAPSHOT,
> > > directory=/home/chetanm/git/apache/jackrabbit-oak/oak-
> 

Debugging oak-pojosr test failures

[Robert Munteanu]

Hi,

I am trying out some changes to the OSGi annotations based on Julian's
work and I'm getting test failures in oak-pojosr, for instance

[INFO] Running org.apache.jackrabbit.oak.run.osgi.RepositoryClosedTest
[ERROR] Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed:
61.084 s <<< FAILURE! - in
org.apache.jackrabbit.oak.run.osgi.RepositoryClosedTest
[ERROR]
sessionUsePostClose(org.apache.jackrabbit.oak.run.osgi.RepositoryClosed
Test)  Time elapsed: 60.638 s  <<< ERROR!
javax.jcr.RepositoryException: Repository could not be started in 60
seconds
at
org.apache.jackrabbit.oak.run.osgi.RepositoryClosedTest.sessionUsePostC
lose(RepositoryClosedTest.groovy:47)
Caused by: java.util.concurrent.TimeoutException: Timeout waiting for
task.
at
org.apache.jackrabbit.oak.run.osgi.RepositoryClosedTest.sessionUsePostC
lose(RepositoryClosedTest.groovy:47)

Unfortunately I get no useful output in the target/unit-tests.log file,
only some warnings from fileinstall:

3659:27.09.2017 12:33:58.435 *WARN* [main] org.apache.felix.fileinstall
/home/robert/Documents/sources/apache/jackrabbit-oak/oak-
pojosr/target/junit7412715572491972111/config does not exist, please
create it.
3660:27.09.2017 12:33:58.435 *WARN* [main] org.apache.felix.fileinstall
Root path does not exist:
/home/robert/Documents/sources/apache/jackrabbit-oak/oak-
pojosr/target/junit7412715572491972111/config

How can I get more debug information about what is going on with the
oak-pojosr setup?

Thanks,

Robert

Re: [VOTE] Release Apache Jackrabbit Oak 1.7.8

[Robert Munteanu]

On Mon, 2017-09-25 at 21:15 +0100, Davide Giannella wrote:
> Please vote on releasing this package as Apache Jackrabbit Oak 1.7.8.

+1

Robert

signature.asc
Description: This is a digitally signed message part

Re: chroot-like content segregation?

[Robert Munteanu]

Hi Tomek,

On Fri, 2017-09-22 at 05:03 +, Tomek Rekawek wrote:
> Hello Bertrand,
> 
> this seems like an opposite of the composite node store - rather than
> combining multiple repositories together, we’re trying to split one
> repository into many jails. Maybe I’m too optimistic, but I think the
> implementation should be quite easy if done on the node store level.

(snip)

But that would require support in the various Oak subsystems, like
indexing, node types, etc right? Either that, or we create full self-
contained 'jails' with node type definitions, indexes, principals, etc.

Robert

Re: [VOTE] Release Apache Jackrabbit Oak 1.7.7

[Robert Munteanu]

On Mon, 2017-09-11 at 14:12 +0100, Davide Giannella wrote:
> Please vote on releasing this package as Apache Jackrabbit Oak 1.7.7.

+1

Robert

signature.asc
Description: This is a digitally signed message part

Re: Test-only utility for dumping the repository?

[Robert Munteanu]

Hi Alex,

On Thu, 2017-08-31 at 16:50 +0200, Alex Deparvu wrote:
> Hi Robert,
> 
> Have you tried NodeStateUtils#toString?
> 


That looks like exactly what I was looking for, thanks!

Robert

> 
> 
> 
> 
> On Thu, Aug 31, 2017 at 4:37 PM, Robert Munteanu <romb...@apache.org>
> wrote:
> 
> > Hi,
> > 
> > When I have a test issue that relies on inspecting the repository I
> > usually write a small method that prints the contents of the
> > repository.
> > 
> > Today, as I wrote that method for the third time, I was wondering
> > whether we should add it to test-only utility class.
> > 
> > The problem with that is that we don't have something like oak-
> > test-
> > support which contains only testing classes. Would that make sense,
> > and
> > also maybe start breaking out 'test-jar' dependencies between
> > modules?
> > 
> > Thanks,
> > 
> > Robert
> > 

Test-only utility for dumping the repository?

[Robert Munteanu]

Hi,

When I have a test issue that relies on inspecting the repository I
usually write a small method that prints the contents of the
repository.

Today, as I wrote that method for the third time, I was wondering
whether we should add it to test-only utility class.

The problem with that is that we don't have something like oak-test-
support which contains only testing classes. Would that make sense, and
also maybe start breaking out 'test-jar' dependencies between modules?

Thanks,

Robert

Re: [VOTE] Release Apache Jackrabbit Oak 1.4.18

[Robert Munteanu]

On Wed, 2017-08-30 at 11:23 +0200, Davide Giannella wrote:
> Please vote on releasing this package as Apache Jackrabbit Oak
> 1.4.18.

+1

Robert

signature.asc
Description: This is a digitally signed message part

Re: svn commit: r1806602 - in /jackrabbit/oak/trunk/oak-jcr/src: main/java/org/apache/jackrabbit/oak/jcr/repository/ main/java/org/apache/jackrabbit/oak/jcr/session/ test/java/org/apache/jackrabbit/oa

[Robert Munteanu]

On Tue, 2017-08-29 at 21:03 -0700, Chetan Mehrotra wrote:
> +
> +Tracker tracker =
> whiteboard.track(MountInfoProvider.class);
> +List services = tracker.getServices();
> +tracker.stop();
> +
> +if ( services.isEmpty() )
> +this.mountInfoProvider = null;
> +else if ( services.size() == 1 )
> +this.mountInfoProvider = services.get(0);
> +else
> +throw new IllegalArgumentException("Found " +
> services.size() + " MountInfoProvider references, expected at most
> 1.");
> 
> You can also use WhiteboardUtils#getService here which hides this
> stuff
> Chetan Mehrotra

Good catch, thanks Chetan. Applied in r1806682.

Robert

Re: [VOTE] Release Apache Jackrabbit Oak 1.7.6

[Robert Munteanu]

On Tue, 2017-08-22 at 12:22 +0530, Amit Jain wrote:
> Please vote on releasing this package as Apache Jackrabbit Oak 1.7.6.

+1

Robert

signature.asc
Description: This is a digitally signed message part

Re: [VOTE] Release Apache Jackrabbit Oak 1.0.39

[Robert Munteanu]

On Tue, 2017-08-15 at 20:25 +0200, Manfred Baedke wrote:
> Please vote on releasing this package as Apache Jackrabbit Oak
> 1.0.39.


+1

Robert

signature.asc
Description: This is a digitally signed message part

Re: [VOTE] Release Apache Jackrabbit Oak 1.6.4

[Robert Munteanu]

On Wed, 2017-08-16 at 10:15 +0530, Amit Jain wrote:
> Please vote on releasing this package as Apache Jackrabbit Oak 1.6.4.

+1

Robert

signature.asc
Description: This is a digitally signed message part

Re: BUILD FAILURE: Jackrabbit Oak - Build # 615 - Still Failing

[Robert Munteanu]

This is issue https://issues.apache.org/jira/browse/INFRA-14804 . I
have disabled the Jira post-build step, just to get the builds going.

Just to make sure I don't forget, the following settings will be
reapplied once the INFRA issue is sorted out:

- Jira project key: OAK
- Description of test: Jenkins CI failure: https://builds.apache.org/vi
ew/J/job/Jackrabbit%20Oak/
- Component name: continuous integration

Robert

On Tue, 2017-08-08 at 17:13 +, Apache Jenkins Server wrote:
> The Apache Jenkins build system has built Jackrabbit Oak (build #615)
> 
> Status: Still Failing
> 
> Check console output at https://builds.apache.org/job/Jackrabbit%20Oa
> k/615/ to view the results.
> 
> Changes:
> [rombert] OAK-6533 - Adjust test classpath order to reduce number of
> error markers in Eclipse
> 
> [rombert] OAK-6532 - Minimise usage of junit-addons
> 
> - remove dependency where it is not used
> - add dependency exclusion where needed ( fixes Eclipse-only oak-
> lucene
>   compilation error )
> 
> [mduerig] OAK-6520: Improve tail compactions resilience when base
> state cannot be determined
> Fall back to full compaction if the base state is not accessible
> because it points to a missing segment
> 
>  
> 
> Test results:
> All tests passed

Re: BUILD FAILURE: Jackrabbit Oak - Build # 602 - Still Failing

[Robert Munteanu]

Jenkins was restarted recently and plugins upgraded.

  https://issues.apache.org/jira/browse/INFRA-14798

You might want to file an INFRA issue if this persists.

robert

On Mon, 2017-08-07 at 10:55 +0200, Alex Deparvu wrote:
> looks infra related:
> 
> [INFO] BUILD SUCCESS
> 
> channel stopped
> Archiving artifacts
> ERROR: Step ?JIRA: Create issue? aborted due to exception:
> java.lang.ClassNotFoundException: com.atlassian.fugue.Effect
> at
> jenkins.util.AntClassLoader.findClassInComponents(AntClassLoader.java
> :1374)
> at
> jenkins.util.AntClassLoader.findClass(AntClassLoader.java:1327)
> at
> jenkins.util.AntClassLoader.loadClass(AntClassLoader.java:1080)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
> Caused: java.lang.NoClassDefFoundError: com/atlassian/fugue/Effect
> at
> com.atlassian.httpclient.apache.httpcomponents.DefaultHttpClientFacto
> ry.doCreate(DefaultHttpClientFactory.java:68)
> at
> com.atlassian.httpclient.apache.httpcomponents.DefaultHttpClientFacto
> ry.create(DefaultHttpClientFactory.java:35)
> at
> com.atlassian.jira.rest.client.internal.async.AsynchronousHttpClientF
> actory.createClient(AsynchronousHttpClientFactory.java:63)
> at
> com.atlassian.jira.rest.client.internal.async.AsynchronousJiraRestCli
> entFactory.create(AsynchronousJiraRestClientFactory.java:35)
> at
> com.atlassian.jira.rest.client.internal.async.AsynchronousJiraRestCli
> entFactory.createWithBasicHttpAuthentication(AsynchronousJiraRestClie
> ntFactory.java:42)
> at hudson.plugins.jira.JiraSite.createSession(JiraSite.java:280)

[composite] Detecting values present in multiple indexes

[Robert Munteanu]

Hi,

I working on OAK-6505 [0] which will check, in a composite setup,
whether a unique index provided by multiple mounts is consistent. That
is, if the entries are unique for all index 'fragments' provided by
mounts, e.g.


/oak:index
  /uuid [ default mount index ]
  /oak:mount-private
/uuid [ index provided by 'private' mount ]

Looking at the code, it looks like my best approach would be to use
obtain the respective IndexStoreStrategy instances and then iterate
through the values and checking for values appearing in both entries.

Is there a better way of achieving this?

Thanks,

Robert


[0]: https://issues.apache.org/jira/browse/OAK-6506

Re: [VOTE] Release Apache Jackrabbit Oak 1.4.17

[Robert Munteanu]

On Mon, 2017-07-17 at 12:00 +0100, Davide Giannella wrote:
> Please vote on releasing this package as Apache Jackrabbit Oak
> 1.4.17.

+1

Robert

signature.asc
Description: This is a digitally signed message part

Re: Pedantic build failure - oak-jcr/derby.log license check

[Robert Munteanu]

Hi Alex,

On Fri, 2017-07-14 at 11:02 +0200, Alex Deparvu wrote:
> Hi Robert,
> 
> I tested this and I don't see the same error. I do see a derby.log
> file being created in target/ which is fine from the license check
> pov.
> Where are you running the command from exactly (project root, oak-
> jcr)?
> 
> What I usually do is a fresh checkout in a different location, and
> try
> again, just to make sure this is not caused by something else.

Right, it's a false problem. The file was last modified on 2016-09-21,
and therefore not created by the build.

Thanks and sorry for the noise,

Robert

Pedantic build failure - oak-jcr/derby.log license check

[Robert Munteanu]

Hi,

Does anyone else get build failures when running

  $  mvn clean install -Ppedantic

? There is a derby.log file in oak-jcr which does not have a license
header ( obviously ). I can add a rat check exclusion but I'm curious
to see if anyone else gets this.

Thanks,

Robert

Re: [VOTE] Release Apache Jackrabbit Oak 1.6.2

[Robert Munteanu]

On Mon, 2017-06-12 at 15:32 +0100, Davide Giannella wrote:
> Please vote on releasing this package as Apache Jackrabbit Oak 1.6.2.

+1

Robert

signature.asc
Description: This is a digitally signed message part

Re: [VOTE] Release Apache Jackrabbit Oak 1.7.1

[Robert Munteanu]

On Tue, 2017-06-06 at 13:27 +0100, Davide Giannella wrote:
> Please vote on releasing this package as Apache Jackrabbit Oak 1.7.1.

+1

Robert

signature.asc
Description: This is a digitally signed message part

Re: Running javadoc as part of standard build.

[Robert Munteanu]

Hi Davide,

On Mon, 2017-06-05 at 12:18 +0100, Davide Giannella wrote:
> Hello team,
> 
> I'm currently trying to release 1.7.1 and it's failing on javadocs
> (again).
> 
> I'm trying to find a way to run the javadoc at least at the same time
> of
> the IT, otherwise we won't catch any failures up to the release and
> it
> will move the burden the person who's doing the release. It will be
> easier if each one of us fix a non-working javadoc before committing
> it.

What speaks against building the javadocs as part of the regular build?
Seems to be much simpler this way.

Robert

Re: Oakathon: Oak Hackathon

[Robert Munteanu]

On Wed, 2017-05-24 at 21:47 +0200, Michael Dürig wrote:
> > 
> > The page is marked as ImmutablePage to me so I can't edit it.
> > Username
> > is RobertMunteanu.
> 
> I added you contributor: 
> https://wiki.apache.org/jackrabbit/ContributorsGroup. Should work
> now.

It does, thank you.

Robert

API changes in unstable branches (was: [VOTE] Release Apache Jackrabbit Oak 1.7.0)

[Robert Munteanu]

Hi,

On Thu, 2017-05-25 at 12:08 +0100, Davide Giannella wrote:
> On 24/05/2017 19:33, Robert Munteanu wrote:
> > Is there a chance to delay this release and include
> > 
> >   https://issues.apache.org/jira/browse/OAK-6265
> 
> Nope. We already cut and it's under votes. You may add the -1 but I
> think it's trivial for the following reasons:
> 
> - it's an "unstable" release. Which means we expect API changes as
> well
> if needed

I'd like to better understand what our policy on backwards
compatibility at the package level is. For OAK-6265 I would remove two
already exported methods which means that the exported version for
org.apache.jackrabbit.oak.spi.mount would move from 3.0.0 to 4.0.0 .

Is that acceptable for an unstable release? Or should I rather
deprecate those and keep backwards compatibility?

Thanks,

Robert

Re: [VOTE] Release Apache Jackrabbit Oak 1.7.0

[Robert Munteanu]

On Thu, 2017-05-25 at 12:08 +0100, Davide Giannella wrote:
> On 24/05/2017 19:33, Robert Munteanu wrote:
> > Is there a chance to delay this release and include
> > 
> >   https://issues.apache.org/jira/browse/OAK-6265
> 
> Nope. We already cut and it's under votes. You may add the -1 but I
> think it's trivial for the following reasons:
> 
> - it's an "unstable" release. Which means we expect API changes as
> well
> if needed
> - the next one will be in two weeks.

Fair enough.

Thanks,

Robert

Re: Oakathon: Oak Hackathon

[Robert Munteanu]

Hi,

On Wed, 2017-05-24 at 11:17 +0200, Michael Dürig wrote:
> Hi,
> 
> We are going to hold an Oak Hackathon in Basel, Switzerland from
> August 
> 21st to August 25th. This is a developers workshop with the goal to 
> advance the overall state of Oak by fixing bugs, implementing new 
> features or coming up with interesting POCs.
> 
> Please indicate your attendance on the Oakathon Wiki page [1]. Feel
> free 
> to propose topics and discussion items in the respective table or 
> subscribe for already proposed topics by putting your name in the
> box 
> next to it.

The page is marked as ImmutablePage to me so I can't edit it. Username
is RobertMunteanu.

Robert

Re: [VOTE] Release Apache Jackrabbit Oak 1.7.0

[Robert Munteanu]

Hi,

Is there a chance to delay this release and include

  https://issues.apache.org/jira/browse/OAK-6265

?

I would like to remove two methods introduced in
o.a.j.o.spi.mount.Mounts, which can be safely done until we cut the
first release.

If it's not possible I'll leave them as deprecated, but ideally we
would not expose them as API.

Thanks,

Robert

Re: upgrade repository structure with backward-incompatible changes

[Robert Munteanu]

On Wed, 2017-05-17 at 09:37 +, Michael Marth wrote:
> Hi Marco,
> 
> Maybe I don’t understand correctly your use case, but would it be
> easier to simply write a script using the JCR API to do the changes
> in the repo?
> 
> Michael


For bulk operations like that Sling Pipes is also an options ( if
you're using Sling )

https://sling.apache.org/documentation/bundles/sling-pipes.html
https://adapt.to/2016/en/schedule/introduction-to-sling-pipes.html

Robert

> 
> 
> 
> 
> On 16/05/17 18:33, "Marco Piovesana"  wrote:
> 
> > Hi Tomek,
> > yes I'm trying to upgrade within the same repository type but I can
> > decide
> > weather to migrate the repository or not based on what makes the
> > upgrade
> > easier.
> > The CommitHooks can only be used inside an upgrade to a new
> > repository?
> > What is the suggested way to apply backward-incompatible changes if
> > i don't
> > want to migrate the data from one repository to another but I want
> > to apply
> > the modifications to the original one?
> > 
> > Marco.
> > 
> > On Tue, May 16, 2017 at 4:04 PM, Tomek Rekawek  > nvalid>
> > wrote:
> > 
> > > Hi Marco,
> > > 
> > > the main purpose of the oak-upgrade is to migrate a Jackrabbit 2
> > > / CRX2
> > > repository into Oak or to migrate one Oak node store (eg.
> > > segment) to
> > > another (like Mongo). On the other hand, it’s not a good choice
> > > to use it
> > > for the application upgrades within the same repository type. You
> > > didn’t
> > > mention if your upgrade involves the repository migration (in
> > > this case
> > > choosing oak-upgrade would be justified) or not.
> > > 
> > > If you still want to use oak-upgrade, it allows to use custom
> > > CommitHooks
> > > [1] during the migration. They should be included in the class
> > > path with
> > > the ServiceLoader mechanism [2].
> > > 
> > > Regards,
> > > Tomek
> > > 
> > > [1] http://jackrabbit.apache.org/oak/docs/architecture/
> > > nodestate.html#The_commit_hook_mechanism
> > > [2] https://docs.oracle.com/javase/tutorial/sound/SPI-intro.html
> > > 
> > > --
> > > Tomek Rękawek | Adobe Research | www.adobe.com
> > > reka...@adobe.com
> > > 
> > > > On 14 May 2017, at 12:20, Marco Piovesana  > > > > wrote:
> > > > 
> > > > Hi all,
> > > > I'm trying to deal with backward-incompatible changes on my
> > > > repository
> > > > structure. I was looking at the oak-upgrade module but, as far
> > > > as I could
> > > > understand, I can't really make modifications that require some
> > > > logic
> > > 
> > > (e.g.
> > > > remove a property and add a new mandatory property with a value
> > > > based on
> > > > the removed one).
> > > > I saw that one of the options might be the "namespace
> > > > migration":
> > > > - remap the current namespace to a different prefix;
> > > > - create a new namespace with original prefix;
> > > > - port all nodes from old namespace to new namespace applying
> > > > the
> > > 
> > > required
> > > > modifications.
> > > > 
> > > > I couldn't find much documentation on the topic, so my question
> > > > is: is
> > > 
> > > this
> > > > the right way to do it? There are other suggested approaches to
> > > > the
> > > > problem? There's already a tool that can be used to define how
> > > > to map a
> > > > source CND definition into a destination CND definition and
> > > > then apply
> > > 
> > > the
> > > > modifications to a repository?
> > > > 
> > > > Marco.

Re: new name for the multiplexing node store

[Robert Munteanu]

Compositing, Aggregating, Unifying, Consolidating, Coalescing.

(Courtesy of an online thesaurus )

But I agree that the concept behind composition is the right one. All
in all, the MultiplexingNodeStore is a pretty standard implementation
of the Composite design pattern.

Robert

On Wed, 2017-05-10 at 13:20 +0200, Dominik Süß wrote:
> Naming discussions - love it (where is my popcorn? ;) )
> 
> I would think that something with Compositing might be suitable as
> this is
> about composition of something that works as as final result but the
> artifacts might not be useful on their own.
> 
> Cheers
> Dominik
> 
> Am 05.05.2017 20:40 schrieb "Robert Munteanu" <romb...@apache.org>:
> 
> Hi,
> 
> On Fri, 2017-05-05 at 07:18 -0600, Matt Ryan wrote:
> > I was wondering about this also WRT federated data store.  If the
> > intent
> > and effect of both are the same ("both" meaning what is currently
> > called
> > the "multiplexing node store" and the proposed (and in-progress)
> > "federated
> > data store"), it seems they should use a similar naming convention
> > at
> > least.
> > 
> > WDYT?  Does that make it more confusing or less confusing?
> 
> I think the high-level intent is the same for both - compose a single
> {Data,Node}Store out of multiple sub-stores.
> 
> The mechanisms might be different though, as the the NodeStore is
> hierarchical in nature, while the BlobStore blob ids are opaque.
> 
> Also I still maintain :-) that federated blob stores will work well
> individually as they have no overall hierarchy to respect, while the
> multiplexed node stores will have to be composed to create a
> meaningful
> image.
> 
> Robert
> 
> > 
> > -MR
> > 
> > On Fri, May 5, 2017 at 6:10 AM, Julian Sedding <jsedd...@gmail.com>
> > wrote:
> > 
> > > Hi Tomek
> > > 
> > > In all related discussions the term "mount" appears a lot. So why
> > > not
> > > Mounting NodeStore? The module could be "oak-store-mount".
> > > 
> > > Regards
> > > Julian
> > > 
> > > 
> > > On Fri, May 5, 2017 at 1:39 PM, Tomek Rekawek <reka...@adobe.com>
> > > wrote:
> > > > Hello oak-dev,
> > > > 
> > > > the multiplexing node store has been recently extracted from
> > > > the
> > > 
> > > oak-core into a separate module and I’ve used it as an
> > > opportunity
> > > to
> > > rename the thing. The name I suggested is Federated Node Store.
> > > Robert
> > > doesn’t agree it’s the right name, mostly because the “partial”
> > > node
> > > stores, creating the combined (multiplexing / federated) one, are
> > > not
> > > usable on their own and stores only a part of the overall
> > > repository
> > > content.
> > > > 
> > > > Our arguments in their full lengths can be found in the OAK-
> > > > 6136
> > > > (last
> > > 
> > > 3-4 comments), so there’s no need to repeat them here. We wanted
> > > to
> > > ask you
> > > for opinion about the name. We kind of agree that the
> > > “multiplexing” is not
> > > the best choice - can you suggest something else or maybe you
> > > think
> > > that
> > > “federated” is good enough?
> > > > 
> > > > Thanks for the feedback.
> > > > 
> > > > Regards,
> > > > Tomek
> > > > 
> > > > --
> > > > Tomek Rękawek | Adobe Research | www.adobe.com
> > > > reka...@adobe.com
> > > > 

Re: BUILD FAILURE: Jackrabbit Oak - Build # 261 - Failure

[Robert Munteanu]

On Mon, 2017-05-08 at 13:12 +0200, Alex Parvulescu wrote:
> Hi,
> 
> This looks odd [0], builds takes 49 mins, doesn't fail, but it still
> times-out at 60.
> 
> alex
> 
> 
> [0]
> [INFO] BUILD SUCCESS
> [INFO]
> ---
> -
> [INFO] Total time: 49:02 min
> [INFO] Finished at: 2017-05-08T10:40:03+00:00
> [INFO] Final Memory: 192M/1065M
> [INFO]
> ---
> -
> Waiting for Jenkins to finish collecting data
> Build timed out (after 60 minutes). Marking the build as failed.

I guess it was waiting for Jenkins to finish collection data and that
took too long.

Robert

> Build was aborted
> Aborted
> channel stopped
> Build failed, created JIRA issue OAK-6182
> 
> 
> 
> On Mon, May 8, 2017 at 1:06 PM, Apache Jenkins Server <
> jenk...@builds.apache.org> wrote:
> 
> > The Apache Jenkins build system has built Jackrabbit Oak (build
> > #261)
> > 
> > Status: Failure
> > 
> > Check console output at https://builds.apache.org/job/
> > Jackrabbit%20Oak/261/ to view the results.
> > 
> > Changes:
> > [alexparvulescu] OAK-6145 Move MembershipWriter#setMembers() to
> > oak-upgrade
> > 
> > 
> > 
> > Test results:
> > All tests passed

Re: new name for the multiplexing node store

[Robert Munteanu]

Hi,

On Fri, 2017-05-05 at 07:18 -0600, Matt Ryan wrote:
> I was wondering about this also WRT federated data store.  If the
> intent
> and effect of both are the same ("both" meaning what is currently
> called
> the "multiplexing node store" and the proposed (and in-progress)
> "federated
> data store"), it seems they should use a similar naming convention at
> least.
> 
> WDYT?  Does that make it more confusing or less confusing?

I think the high-level intent is the same for both - compose a single
{Data,Node}Store out of multiple sub-stores.

The mechanisms might be different though, as the the NodeStore is
hierarchical in nature, while the BlobStore blob ids are opaque.

Also I still maintain :-) that federated blob stores will work well
individually as they have no overall hierarchy to respect, while the
multiplexed node stores will have to be composed to create a meaningful
image.

Robert

> 
> -MR
> 
> On Fri, May 5, 2017 at 6:10 AM, Julian Sedding 
> wrote:
> 
> > Hi Tomek
> > 
> > In all related discussions the term "mount" appears a lot. So why
> > not
> > Mounting NodeStore? The module could be "oak-store-mount".
> > 
> > Regards
> > Julian
> > 
> > 
> > On Fri, May 5, 2017 at 1:39 PM, Tomek Rekawek 
> > wrote:
> > > Hello oak-dev,
> > > 
> > > the multiplexing node store has been recently extracted from the
> > 
> > oak-core into a separate module and I’ve used it as an opportunity
> > to
> > rename the thing. The name I suggested is Federated Node Store.
> > Robert
> > doesn’t agree it’s the right name, mostly because the “partial”
> > node
> > stores, creating the combined (multiplexing / federated) one, are
> > not
> > usable on their own and stores only a part of the overall
> > repository
> > content.
> > > 
> > > Our arguments in their full lengths can be found in the OAK-6136
> > > (last
> > 
> > 3-4 comments), so there’s no need to repeat them here. We wanted to
> > ask you
> > for opinion about the name. We kind of agree that the
> > “multiplexing” is not
> > the best choice - can you suggest something else or maybe you think
> > that
> > “federated” is good enough?
> > > 
> > > Thanks for the feedback.
> > > 
> > > Regards,
> > > Tomek
> > > 
> > > --
> > > Tomek Rękawek | Adobe Research | www.adobe.com
> > > reka...@adobe.com
> > > 

OAK-6050 - Unify used version of osgi-mock

[Robert Munteanu]

Hi,

I have a patch attached to OAK-6050 [1]

  https://issues.apache.org/jira/browse/OAK-6050

Nothing earth-shattering, but it adds a bit more consistency and makes
Eclipse happier.

It would be great if someone could take a look and review.

Robert

Re: OAK-6089 (maven plugin updates) and OAK-5664 (require Java 8), also: move to Maven 3.3.9?

[Robert Munteanu]

Hi Julian,

On Thu, 2017-04-20 at 15:43 +0200, Julian Reschke wrote:
> Hi there,
> 
> in preparation of the switch to Java 8, I have updated lots of maven 
> plugin dependencies and also explicitly required the use of Maven
> 3.2.1 
> (this is the version we use in CI, and was apparently was also de
> facto 
> required already).
> 
> I ran integration tests after each of these changes, but can't
> promise 
> that I didn't break something not covered by the tests.
> 
> There are a few recommended plugin updates still to be done:
> 
> > [INFO]   org.apache.felix:maven-scr-plugin ..
> > 1.21.0 -> 1.24.0
> > [INFO]   org.springframework.boot:spring-boot-maven-
> > plugin  1.3.8.RELEASE -> 1.5.2.RELEASE
> 
> ...but these require more changes, and I'll open separate tickets for
> these.
> 
> With this done, we can switch to Java 8 (as required language
> level); 
> I'll do that later today.
> 
> That said, while at it, we could go even further and require Maven
> 3.3.9 
> - I've been building past branches of Jackrabbit and Oak with it,
> and 
> haven't noticed any problems. Feedback on this would be appreciated.

I am not aware of any breaking changes in Maven 3.3.9 and I have been
using it since it's been released.

In my opinion we can be a bit more relaxed about Maven version
requirements since they are involved in building Oak, not running Oak.

It's usually pain-free to update to a newer version of Maven or, if
really needed, to install an additional one next to the old one.

Robert

Re: [m12n] Location of InitialContent

[Robert Munteanu]

On Thu, 2017-04-20 at 08:07 +, Angela Schreiber wrote:
> hi
> 
> the original intention of the 'InitialContent' was just to registers
> built-in JCR node types, which explains it's location in
> org.apache.jackrabbit.oak.plugins.nodetype.write
> 
> in the mean time it has a evolved to container for all kind of
> initial
> content required for a JCR repository: mandatory structure, version
> storage, uuid-index and most recently document ns specific
> configuration
> (see also OAK-5656 ).
> 
> to me the location in the
> org.apache.jackrabbit.oak.plugins.nodetype.write
> package no longer makes sense and i would suggest to move it to the
> org.apache.jackrabbit.oak package along with the Oak, OakInitializer,
> OakVersion and other utilities used to create an JCR/Oak repository.
> 
> wdyt?

+1, would fit in well there.

Robert

Re: [m12n] A new home for ApproximateCounter, NodeUtil and TreeUtil?

[Robert Munteanu]

On Wed, 2017-04-19 at 15:53 +, Angela Schreiber wrote:
> hi robert
> 
> i created 2 separate tasks in the OAK-6069 m12n epic to track
> progress
> with ApproximateCounter and NodeUtil|TreeUtil respectively.
> moving the ApproximateCounter should be doable with limited effort,
> we
> just need to get a better understanding if that is/will/shouldbe used
> more
> widely than just for the indexing.

I've commented on OAK-6093 - I think we should start with getting the
naming right.

Robert

> 
> the other one probably needs some more effort.
> angela 
> 
> 
> On 19/04/17 17:45, "Robert Munteanu" <romb...@apache.org> wrote:
> 
> > On Tue, 2017-04-18 at 06:21 +, Angela Schreiber wrote:
> > > Hi Robert
> > > 
> > > While NodeUtil and TreeUtil would naturally fit to plugins.tree,
> > > I am
> > > not
> > > convinced that ApproximateCounter really belongs there. Afaik it
> > > is
> > > only
> > > used for query index strategy and counting. I would rather move
> > > 'ApproximateCounter' to 'plugins.index'.
> > > 
> > > Regarding moving 'NodeUtil' and 'TreeUtil': IMHO we have here 2
> > > utility
> > > classes providing almost the same functionality. I would prefer
> > > to
> > > decide
> > > on the redundancy (and potentially clean it up) before moving it
> > > to a
> > > package that already has semantic versioning enabled (in contrast
> > > to
> > > the
> > > util package where they currently are located).
> > > 
> > > wdyt?
> > 
> > (Sorry, missed this somehow)
> > 
> > Yes, holding on for now sounds good to me.
> > 
> > Thanks,
> > 
> > Robert
> > 
> > > 
> > > Kind regards
> > > Angela
> > > 
> > > 
> > > On 14/04/17 12:47, "Robert Munteanu" <romb...@apache.org> wrote:
> > > 
> > > > I created a final PR for this as I have somewhat mixed
> > > > feelings.
> > > > One
> > > > one had, it finally nukes the util package. On the other hand,
> > > > it
> > > > looks
> > > > like a lot of noise for 3 classes.
> > > > 
> > > >  
> > > > https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%
> > > > 2Fgi
> > > > thub.co
> > > > m%2Fmreutegg%2Fjackrabbit-
> > > > oak%2Fpull%2F6=02%7C01%7C%7C287ecd3d735246c
> > > > cbc8308d48323b62e%7Cfa7b1b5a7b34438794aed2c178decee1%7C0%7C0%7C
> > > > 6362
> > > > 7763678
> > > > 7796080=HSDWauCHB%2Bb4OaX90CfEWsA7487EP3FvrSicZNgKD8Q%3D&
> > > > rese
> > > > rved=0
> > > > 
> > > > Robert
> > > > 
> > > > On Thu, 2017-04-06 at 14:49 +, Angela Schreiber wrote:
> > > > > Hi Robert
> > > > > 
> > > > > plugins.tree would feel natural to me.
> > > > > regarding the export: not sure about that either... the
> > > > > plugins.tree
> > > > > has
> > > > > some unfortunate dependencies e.g. to oak.core. so probably
> > > > > more
> > > > > work
> > > > > ahead in that area.
> > > > > 
> > > > > kind regards
> > > > > angela
> > > > > 
> > > > > On 06/04/17 16:41, "Robert Munteanu" <romb...@apache.org>
> > > > > wrote:
> > > > > 
> > > > > > Hi,
> > > > > > 
> > > > > > Working in the m12n branch [1] I'm trying to get rid of the
> > > > > > o.a.j.oak.util package and the last surviving members are
> > > > > > ApproximateCounter, NodeUtil and TreeUtil.
> > > > > > 
> > > > > > As I see it these classes are essentially helpers built on
> > > > > > top
> > > > > > of
> > > > > > the
> > > > > > Tree and NodeState APIs. Those would make them candidates
> > > > > > on
> > > > > > for
> > > > > > either
> > > > > > oak-store-spi or (if we manage to trim down the
> > > > > > dependencies)
> > > > > > oak-
> > > > > > base.
> > > > > > 
> > > > > > However I am having trouble naming the package which will
> > > > > > hold
> > > > > > them.
> > > > > > They're not part of the spi, so I can't put them in
> > > > > > spi.state .
> > > > > > 
> > > > > > Maybe they belong in oak-core in plugins.tree, but I'm not
> > > > > > sure
> > > > > > if
> > > > > > we
> > > > > > want to keep that as a package which is exported outside
> > > > > > oak-
> > > > > > core.
> > > > > > 
> > > > > > Thoughts?
> > > > > > 
> > > > > > Robert
> > > > > > 
> > > > > > [1]: 
> > > > > > https://na01.safelinks.protection.outlook.com/?url=https%3A
> > > > > > %2F%
> > > > > > 2Fgi
> > > > > > thub.co
> > > > > > m%2Fmreutegg%2Fjackrabbit-
> > > > > > oak%2Ftree%2Fm12n=02%7C01%7C%7Cbfc1feb5ff4a
> > > > > > 4866c79c08d47cfafe6d%7Cfa7b1b5a7b34438794aed2c178decee1%7C0
> > > > > > %7C0
> > > > > > %7C6
> > > > > > 3627086
> > > > > > 4841267177=CWwq4ifTZIU1gW9UEd2STRLm%2B1svSP0kvlkLMksm
> > > > > > WcM%
> > > > > > 3D
> > > > > > eserved
> > > > > > =0
> > > > > 
> > > > > 
> > > 
> > > 
> 
> 

Re: [m12n] A new home for ApproximateCounter, NodeUtil and TreeUtil?

[Robert Munteanu]

On Tue, 2017-04-18 at 06:21 +, Angela Schreiber wrote:
> Hi Robert
> 
> While NodeUtil and TreeUtil would naturally fit to plugins.tree, I am
> not
> convinced that ApproximateCounter really belongs there. Afaik it is
> only
> used for query index strategy and counting. I would rather move
> 'ApproximateCounter' to 'plugins.index'.
> 
> Regarding moving 'NodeUtil' and 'TreeUtil': IMHO we have here 2
> utility
> classes providing almost the same functionality. I would prefer to
> decide
> on the redundancy (and potentially clean it up) before moving it to a
> package that already has semantic versioning enabled (in contrast to
> the
> util package where they currently are located).
> 
> wdyt?

(Sorry, missed this somehow)

Yes, holding on for now sounds good to me.

Thanks,

Robert

> 
> Kind regards
> Angela
> 
> 
> On 14/04/17 12:47, "Robert Munteanu" <romb...@apache.org> wrote:
> 
> > I created a final PR for this as I have somewhat mixed feelings.
> > One
> > one had, it finally nukes the util package. On the other hand, it
> > looks
> > like a lot of noise for 3 classes.
> > 
> >  
> > https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgi
> > thub.co
> > m%2Fmreutegg%2Fjackrabbit-
> > oak%2Fpull%2F6=02%7C01%7C%7C287ecd3d735246c
> > cbc8308d48323b62e%7Cfa7b1b5a7b34438794aed2c178decee1%7C0%7C0%7C6362
> > 7763678
> > 7796080=HSDWauCHB%2Bb4OaX90CfEWsA7487EP3FvrSicZNgKD8Q%3D
> > rved=0
> > 
> > Robert
> > 
> > On Thu, 2017-04-06 at 14:49 +, Angela Schreiber wrote:
> > > Hi Robert
> > > 
> > > plugins.tree would feel natural to me.
> > > regarding the export: not sure about that either... the
> > > plugins.tree
> > > has
> > > some unfortunate dependencies e.g. to oak.core. so probably more
> > > work
> > > ahead in that area.
> > > 
> > > kind regards
> > > angela
> > > 
> > > On 06/04/17 16:41, "Robert Munteanu" <romb...@apache.org> wrote:
> > > 
> > > > Hi,
> > > > 
> > > > Working in the m12n branch [1] I'm trying to get rid of the
> > > > o.a.j.oak.util package and the last surviving members are
> > > > ApproximateCounter, NodeUtil and TreeUtil.
> > > > 
> > > > As I see it these classes are essentially helpers built on top
> > > > of
> > > > the
> > > > Tree and NodeState APIs. Those would make them candidates on
> > > > for
> > > > either
> > > > oak-store-spi or (if we manage to trim down the dependencies)
> > > > oak-
> > > > base.
> > > > 
> > > > However I am having trouble naming the package which will hold
> > > > them.
> > > > They're not part of the spi, so I can't put them in spi.state .
> > > > 
> > > > Maybe they belong in oak-core in plugins.tree, but I'm not sure
> > > > if
> > > > we
> > > > want to keep that as a package which is exported outside oak-
> > > > core.
> > > > 
> > > > Thoughts?
> > > > 
> > > > Robert
> > > > 
> > > > [1]: 
> > > > https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%
> > > > 2Fgi
> > > > thub.co
> > > > m%2Fmreutegg%2Fjackrabbit-
> > > > oak%2Ftree%2Fm12n=02%7C01%7C%7Cbfc1feb5ff4a
> > > > 4866c79c08d47cfafe6d%7Cfa7b1b5a7b34438794aed2c178decee1%7C0%7C0
> > > > %7C6
> > > > 3627086
> > > > 4841267177=CWwq4ifTZIU1gW9UEd2STRLm%2B1svSP0kvlkLMksmWcM%
> > > > 3D
> > > > eserved
> > > > =0
> > > 
> > > 
> 
> 

Re: [m12n] A new home for ApproximateCounter, NodeUtil and TreeUtil?

[Robert Munteanu]

I created a final PR for this as I have somewhat mixed feelings. One
one had, it finally nukes the util package. On the other hand, it looks
like a lot of noise for 3 classes.

  https://github.com/mreutegg/jackrabbit-oak/pull/6

Robert

On Thu, 2017-04-06 at 14:49 +, Angela Schreiber wrote:
> Hi Robert
> 
> plugins.tree would feel natural to me.
> regarding the export: not sure about that either... the plugins.tree
> has
> some unfortunate dependencies e.g. to oak.core. so probably more work
> ahead in that area.
> 
> kind regards
> angela
> 
> On 06/04/17 16:41, "Robert Munteanu" <romb...@apache.org> wrote:
> 
> > Hi,
> > 
> > Working in the m12n branch [1] I'm trying to get rid of the
> > o.a.j.oak.util package and the last surviving members are
> > ApproximateCounter, NodeUtil and TreeUtil.
> > 
> > As I see it these classes are essentially helpers built on top of
> > the
> > Tree and NodeState APIs. Those would make them candidates on for
> > either
> > oak-store-spi or (if we manage to trim down the dependencies) oak-
> > base.
> > 
> > However I am having trouble naming the package which will hold
> > them.
> > They're not part of the spi, so I can't put them in spi.state .
> > 
> > Maybe they belong in oak-core in plugins.tree, but I'm not sure if
> > we
> > want to keep that as a package which is exported outside oak-core.
> > 
> > Thoughts?
> > 
> > Robert
> > 
> > [1]: 
> > https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgi
> > thub.co
> > m%2Fmreutegg%2Fjackrabbit-
> > oak%2Ftree%2Fm12n=02%7C01%7C%7Cbfc1feb5ff4a
> > 4866c79c08d47cfafe6d%7Cfa7b1b5a7b34438794aed2c178decee1%7C0%7C0%7C6
> > 3627086
> > 4841267177=CWwq4ifTZIU1gW9UEd2STRLm%2B1svSP0kvlkLMksmWcM%3D
> > eserved
> > =0
> 
> 

Re: [m12] Effort to Improve Modularisation of Oak

[Robert Munteanu]

Hi Matt,

On Thu, 2017-04-13 at 09:07 -0600, Matt Ryan wrote:
> Great work so far to everyone involved in this effort.
> 
> I'm under the impression that this refactoring will constitute a
> change in
> the public API contract of Oak.  In reading the links it seems to
> hint at
> this but whether or not this will actually result in a public API
> change
> isn't explicitly called out.
> 
> So my first question:  Is my impression accurate here?

It is accurate to a certain degree. There is no change for applications
consuming Oak via the JCR API ( obviously ) or via the Oak API (
ContentRepository / ContentSession / Tree etc ).

There can be changes to applications hooking into Oak's SPIs, for
instance Sling which sets up an Oak content repository instance via
OSGi.

So in my view most consumers will not be affected.

> 
> If so, the follow-up question is, do we have plans to retain backward
> API
> compatibility between 1.6 and 1.8 such that any current code relying
> on 1.6
> API will continue to work with the 1.8 release?
> 
> If backward compatibility is in the plan let's make sure it is
> identified
> in the plan; so far I haven't noticed this but surely I could have
> missed
> it.

When we first approached the m12n task we decided to first make it work
as it should. I am not sure with the scale of changes that we
can/should offer backwards compatibility. I'd rather adapt a 'wait and
see' approach, and handle each case individually. There may be just a
small number of backwards compatible shims we need provide, rather than
making everything backwards compatible.

Instead, I view this as an opportunity to really make things backwards
compatible from now on :-)

> 
> If backward compatibility is not in the plan, the next question I ask
> is
> whether the next release should actually be Oak 2.0 instead of Oak
> 1.8?  If
> semantic versioning matters it seems to me this change would qualify
> for a
> major release bump.

In my view the Oak version is a 'marketing' version, denoting stability
and feature increments, rather than backwards compatibility to
implementors.

I think that packages should be semantically versioned and - even more
- that modules should be independently versioned. With that, the Oak
version would become even less relevant to these changes.

Thanks,

Robert

> 
> 
> My intent isn't as much to lobby for strict adherence to semantic
> versioning so much as to make sure we are mindful of what appears to
> be a
> change in the public API contract and that we have a plan to manage
> it.
> 
> 
> -MR
> 
> 
> On Thu, Apr 13, 2017 at 7:52 AM, Angela Schreiber 
> wrote:
> 
> > Hi Michael and Oak-Devs
> > 
> > Just a quick update: I added more details to OAK-6073 included a
> > summary
> > of the effects on oak-core and oak-commons as well as listing
> > changes to
> > non-test dependencies of most modules.
> > 
> > I would like to encourage you to look at the fork and the summary
> > and
> > provide feedback if you spot any problems or concerns.
> > 
> > Regarding
> > 
> > {quote}
> > I would suggest to go with a naming scheme that reflects how
> > modules
> > would be grouped together in a hierarchical structure as much as
> > possible for now. E.g. rename oak-commons-run to oak-run-commons.
> > {quote}
> > 
> > I would like to address this separately as it would further expand
> > the
> > scope of OAK-6073, which will be open for review over the weekend.
> > After
> > that I would suggest that we incorporate the refactoring into oak-
> > trunk.
> > 
> > Kind regards
> > Angela
> > 
> > 
> > 
> > On 13/04/17 12:17, "Michael Dürig"  wrote:
> > 
> > > 
> > > > I try to describe the changes proposed by the PoC in
> > > > 
> > > > https://na01.safelinks.protection.outlook.com/?url=
> > 
> > https%3A%2F%2Fissues.a
> > > > pache.org%2Fjira%2Fbrowse%2FOAK-6073%3FfocusedCommentId%
> > 
> > 3D15965623%26pa
> > > > ata=02%7C01%7C%7Cd062470b1185427e793608d48256
> > 
> > 5535%7Cfa7b1b5a7b34438794aed
> > > > 2c178decee1%7C0%7C0%7C636276754686224956=
> > 
> > W7sTOFt4hmoew%2BMR7K%2F45I
> > > > PvOAOSQPsaGKhWfMUWOuI%3D=0
> > > > 
> > > > ge=com.atlassian.jira.plugin.system.issuetabpanels:
> > 
> > comment-tabpanel#comme
> > > > nt
> > > > -15965623.
> > > > Additionally added some step-by-step instruction on how we
> > > > proceeded.
> > > > 
> > > 
> > > Thanks, this is very valuable!
> > > 
> > > 
> > > > When we first looked at it on Tuesday last week, I thought that
> > > > we would
> > > > end the exercise with a "tried hard but failed" summary. So, I
> > > > am quite
> > > > pleased that actually ended up with a working PoC.
> > > 
> > > So am I, I'm impressed by the progress here!
> > > 
> > > 
> > > > Looking back I thing the biggest issues are
> > > > 
> > > > - putting everything in oak-core was obviously convenient but
> > > > it turned
> > > > out to be impossible to protect against boundary violations
> > > > - packages sometimes contain classes that not 

Re: Module to host oak-run console scripts

[Robert Munteanu]

On Thu, 2017-04-13 at 15:03 +0530, Chetan Mehrotra wrote:
> On Thu, Apr 13, 2017 at 2:51 PM, Marcel Reutegger  > wrote:
> > one concern I have is maintainability. Scripts tend to go out of
> > date and my
> > question would be how we can detect this when it happens.
> 
> Yes that would be the case. The scripts come with lesser testing and
> can go stale
> 
> > I'm wondering why this is
> > implemented as a script instead of plain Java. I'd say if we
> > consider this
> > an important functionality that should be added to Oak, then it is
> > probably
> > better to integrate it better, with tests, documentation, etc.
> 
> There is OAK-5558 for that but that would take time and then also it
> might not be present in old branches.
> 
> Major benefit of scripts are they are faster to implement and can be
> used in older branches without impacting runtime. Commonly used
> script
> can then later be supported in a proper way (MBean, oak-run command
> etc). For now I have quite a bit of them in my github (and few other
> devs also maintain such scripts). So idea here was to consolidate
> such
> common scripts at one place.

Maybe a good compromise to place then in a non-releasable module - or
just a directory is fine. This way they are not released as other
modules, and are clearly defined as 'your mileage may vary'.

One option is oak-run/contrib/scripts . Or we can even have a 'contrib'
area in the top-level oak directory and place the scripts there, i.e.
/contrib/scripts or /contrib/oak-run-scripts , to make it clear what
kind of scripts these are.

Robert

Re: possible to graft a remote non-JCR source to a path?

[Robert Munteanu]

Hi,

On Wed, 2017-04-12 at 15:31 +, Beaudet, David wrote:
> Thanks Roy, yes that's the other suggestion Adobe had identified and
> we're looking at it as well.  In your opinion is integration of DAM
> Assets at the JCR level with Oak just a bad idea even if it is
> possible?

With the disclaimer that this is sliding out of oak-dev into AEM-
specific stuff and should be moved to an Adobe-specific discussion
channel ...

Make sure to validate that the DAM UI works at the Sling resource level
and not at the JCR level. If I plug in a remote ResourceProvider and
the AEM DAM works at the JCR level, it won't be able to see the data
you grafted.

Robert

> 
> From: Roy Teeuwen [mailto:r...@teeuwen.be]
> Sent: Wednesday, April 12, 2017 11:22 AM
> To: oak-dev@jackrabbit.apache.org
> Subject: Re: possible to graft a remote non-JCR source to a path?
> 
> Hey David,
> 
> Seeing as you are using AEM, thus Sling, have you looked at the
> ResourceProvider interface? With this you can bind resources coming
> from another place than the JCR repo to the content tree.
> You can look at these examples, the first one fetches content from a
> couchbase, the second one from a mongodb:
> 
>  - https://github.com/apache/sling/tree/trunk/contrib/nosql/couchbase
> -resourceprovider
>  - https://github.com/apache/sling/tree/trunk/contrib/nosql/mongodb-r
> esourceprovider
>  - https://github.com/Adobe-Consulting-Services/acs-aem-samples/blob/
> master/bundle/src/main/java/com/adobe/acs/samples/resourceproviders/i
> mpl/SampleResourceProvider.java
> 
> Greets,
> Roy
> On 12 Apr 2017, at 16:34, Beaudet, David  beau...@nga.gov>> wrote:
> 
> 
> Greetings,
> 
> We are researching the best way to expose digital assets that are
> managed in a non-JCR based enterprise DAM system via Adobe's native
> DAM UI and are wondering if it's possible to, for example, bind a
> path like /content/dam/edam/... to a remote data source.  Adobe had
> released an alpha or beta version of a package for JackRabbit that
> enabled a remote RDBMS source dataset to be grafted into the JCR tree
> at a configured path. Content was mapped from the database and
> exposed via the JCR APIs.  Is it still possible to implement such a
> feature with Oak and if so, can someone point me to the interfaces
> that would have to be implemented or a how-to guide if one exists?
> 
> Thanks,
> 
> Dave Beaudet
> National Gallery of Art
>