Re: Javadoc API outdated again

[Bertrand Delacretaz]

On Thu, Oct 12, 2023 at 11:22 AM Julian Reschke  wrote:
> ...Updating the site is not trivial as it requires a plugin that does not
> exist for Windows...

FWIW, if you can run Docker you can build the site using

  docker run -it --rm --name maven -v "$(pwd)":/usr/src/maven -w
/usr/src/maven maven:3.3-jdk-8 mvn clean

-Bertrand

Re: FYI, created a JCR API wrappers module in the Sling whiteboard

[Bertrand Delacretaz]

Hi Francesco,

On Mon, Jan 28, 2019 at 5:36 PM Francesco Mari  wrote:
> ...I would be more skeptical if I saw this code used in production overriding 
> or
> enriching the behaviour of the API. Moreover, the JCR API encourages a very
> fine-grained interaction with the underlying storage system and, because of
> this reason, it's often on the critical path...

Fully agreed - for now our wrapper module is in the Sling Whiteboard,
which clearly labels it as experimental.

I have added a warning about that to the README at
https://github.com/apache/sling-whiteboard/tree/master/jcr-wrappers -
thanks for your comments!

> ...- Use new and better abstractions to hide the JCR API

I don't know if it's "better" but the Sling Resource Provider
mechanism indeed allows one to plug various persistence mechanisms in
a fairly simple way. But then of course it doesn't provide the same
level of functionality than JCR, it's just focused on CRUD operations.

-Bertrand

FYI, created a JCR API wrappers module in the Sling whiteboard

[Bertrand Delacretaz]

Hi,

FYI we have created together with Karl Pauls a library of wrappers for
the JCR API, that in theory allows one to intercept JCR calls for
various purposes.

I'm saying "in theory" as it's an early version that hasn't been
tested much so far, but worked in simple use cases like pre-loading
and simulating JCR content.

The code is at 
https://github.com/apache/sling-whiteboard/tree/master/jcr-wrappers
in the Sling whiteboard, if there's interest we might move it to
Jackrabbit at some point (it has no Sling dependencies except a minor
test library) but as we're both Sling committers it's easier for us to
play with it there for now.

-Bertrand

How to find out if similarity search is active - without doing a search

[Bertrand Delacretaz]

Hi,

I need to find out whether the Oak similarity search functionality is
active. I talked to Tommaso and he recommended doing a search under
/oak:index [1].

That works fine [2] but I need to use a service user to do that, which
is suboptimal.

Would it be possible for Oak to provide this capability information in
a different way that does not require a JCR Session? I suppose the
functionality is available if a specific version of the oak-lucene
bundle is installed, so the following options come to mind:

a) Adding an OSGi Provide-Capability header to that bundle, that's a
trivial change to that bundle's build

b) Providing that information in the JCR Repository object's Descriptors

Both would work fine for my use case - if a) is acceptable I'm happy
to provide a patch, for b) I would need more research.

-Bertrand

[1] /jcr:root/oak:index//* [@useInSimilarity = true]
[2] 
https://github.com/apache/sling-org-apache-sling-capabilities-jcr/blob/master/src/main/java/org/apache/sling/capabilities/jcr/SearchSource.java

[jira] [Commented] (JCR-4378) MD5 and SHA1 should no longer be provided on download pages

[Bertrand Delacretaz (JIRA)]

[ 
https://issues.apache.org/jira/browse/JCR-4378?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16633785#comment-16633785
 ] 

Bertrand Delacretaz commented on JCR-4378:
--

FYI we've also be looking at this for the Sling project and it looks like Maven 
and/or repository.apache.org get in the way depending on your release process.

We're following up at SLING-7534, just wanted to mention this to avoid 
duplication of efforts.

Note also that http://www.apache.org/dev/release-distribution#sigs-and-sums 
initially said that we MUST NOT include md5 digests anymore, but that was later 
changed to SHOULD NOT, due to those tooling issues IIUC.

> MD5  and SHA1 should no longer be provided on download pages
> 
>
> Key: JCR-4378
> URL: https://issues.apache.org/jira/browse/JCR-4378
> Project: Jackrabbit Content Repository
>  Issue Type: Bug
>Reporter: Sebb
>Assignee: Julian Reschke
>Priority: Major
>
> As the subject says: SHA1 is deprecated and should no longer be linked from 
> download pages.
> New releases should have sha256 and/or sha512 hashes instead.
> If a historic release only has a SHA1 hash, that can be retained, but ideally 
> it can be replaced with a better sha256/512 one.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Re: Anyone in ApacheCon NA, Montreal?

[Bertrand Delacretaz]

Hi,

On Sun, Sep 23, 2018 at 7:53 PM Woonsan Ko  wrote:
> ...Is there anyone joining ApacheCon Montreal this week?...

I'm not a Jackrabbit committer but close to that "family" - I'm here,
actually on stage at 9am today for the State of the Feater. Hoping to
meet you!

-Bertrand

Re: Oak Direct Binary Access pull request

[Bertrand Delacretaz]

Hi Matt,

On Wed, Jun 27, 2018 at 6:11 PM Matt Ryan  wrote:
> ...Of course Sling could take the return value from the call to initiate the
> upload and turn it into a JSON document that the client can then consume.
> As you say the client will need to have some knowledge of the new API to do
> this...

Ok, agreed, it looks like we'll need to make Sling clients aware of
various upload mechanisms if we want to go that route.

Thanks for the clarifications (and thanks Julian for the note on
relative links, makes sense!).

-Bertrand

Re: Oak Direct Binary Access pull request

[Bertrand Delacretaz]

Hi Matt,

On Thu, Jun 21, 2018 at 6:25 AM Julian Reschke  wrote:
> ...it would be helpful if you could link to example client code taking
> advantage of this extension...

+1, ideally as readable test code that one can use as examples.

>From the Sling clients perspective I'm interested in making this
somewhat transparent, maybe something like:

For downloads, a client requests
http://my.sling.instance/somebinary.jpg and is redirected to
https://somecloudprovider/23874623748623746234782634273846237846723864.jpg

For uploads, it's a bit more complicated - maybe the client POSTing to
Sling receives a 307 status with a JSON document that describes
where/how to upload. In this case the client requires some knowledge
of this new API, unless someone has a better idea.

Do you see any obstacles in implementing something like this on top of
your suggested API?

-Bertrand

Re: Supporting direct binary access - PR submitted

[Bertrand Delacretaz]

On Wed, Jun 27, 2018 at 9:43 AM Julian Reschke  wrote:
> ...Is the feature really tied to HTTP(s)? I don't think so. And if a future
> platform used a different protocol, the API wouldn't really need to change...

+1, this is probably more about URIs than HTTP URLs.

-Bertrand

Re: [ANNOUNCE] Apache Jackrabbit 2.6 retired

[Bertrand Delacretaz]

Hi,

On Fri, Jun 22, 2018 at 9:20 AM Julian Reschke  wrote:
> ...Previous branch, tags and releases will still be available
> for future references, but will not show up on the download page anymore

It might be good to indicate that in the branch itself, in the README
or by adding a THIS-BRANCH-IS-RETIRED file at
http://svn.apache.org/repos/asf/jackrabbit/branches/2.6/ to avoid any
confusion?

-Bertrand

[jira] [Commented] (JCR-4301) get rid of JSR 305 dependency

[Bertrand Delacretaz (JIRA)]

[ 
https://issues.apache.org/jira/browse/JCR-4301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16487078#comment-16487078
 ] 

Bertrand Delacretaz commented on JCR-4301:
--

EPL is Category B as per https://www.apache.org/legal/resolved.html , such 
dependencies are ok if included in binary form only. The Oak releases won't 
even include them, just have them as dependencies so I think you're fine.

> get rid of JSR 305 dependency
> -
>
> Key: JCR-4301
> URL: https://issues.apache.org/jira/browse/JCR-4301
> Project: Jackrabbit Content Repository
>  Issue Type: Task
>  Components: jackrabbit-api
>Reporter: Julian Reschke
>Assignee: Julian Reschke
>Priority: Major
> Attachments: JCR-4301-2.diff, JCR-4301.diff
>
>
> We should consider getting rid of the JSR 305 dependency (see 
> <https://blog.codefx.org/java/jsr-305-java-9/>). Note that we only used the 
> annotations in a few recently changed APIs.
> Further note that we could use spotbugs as replacement for findbugs to detect 
> problems.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (JCR-4301) get rid of JSR 305 dependency

[Bertrand Delacretaz (JIRA)]

[ 
https://issues.apache.org/jira/browse/JCR-4301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16487023#comment-16487023
 ] 

Bertrand Delacretaz commented on JCR-4301:
--

I don't think it's ok to use an LGPL annotations library, it's not like running 
an external GPLed tool at build time - that tool does not "pollute" our code 
but annotations in code and classes do.

I don't have an opinion on annotations and Java 9, haven't looked at that so 
far.

> get rid of JSR 305 dependency
> -
>
> Key: JCR-4301
> URL: https://issues.apache.org/jira/browse/JCR-4301
> Project: Jackrabbit Content Repository
>  Issue Type: Task
>  Components: jackrabbit-api
>Reporter: Julian Reschke
>Assignee: Julian Reschke
>Priority: Major
> Attachments: JCR-4301-2.diff, JCR-4301.diff
>
>
> We should consider getting rid of the JSR 305 dependency (see 
> <https://blog.codefx.org/java/jsr-305-java-9/>). Note that we only used the 
> annotations in a few recently changed APIs.
> Further note that we could use spotbugs as replacement for findbugs to detect 
> problems.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Re: identify abandoned oak modules

[Bertrand Delacretaz]

Hi,

On Tue, Nov 21, 2017 at 4:58 PM, Michael Dürig  wrote:
> ...Not exactly retiring but what about moving...

FWIW, in Sling we are using an "attic" folder for such retired
modules, see http://sling.apache.org/project-information.html#attic

-Bertrand

Re: chroot-like content segregation?

[Bertrand Delacretaz]

On Fri, Sep 22, 2017 at 9:14 AM, Tomek Rekawek
 wrote:
> ...Bertrand - could you clarify if the jails should share some data with the 
> “main” repository (which would
> make it more difficult, as Robert and Michael wrote) or the self-contained 
> approach is ok?...

They don't need to share anything, jails being completely isolated
from each other is fine.

Which makes me think that this could also be handled at the Sling
level by switching repositories with one SlingRepository service per
jail. There are some caches that would need to be made aware of that,
at least in Sling, so that probably also leaves a few "details" to
address.

-Bertrand

chroot-like content segregation?

[Bertrand Delacretaz]

Hi,

I'm presenting next week at https://adapt.to on creating multi-tenant
HTTP request processing / rendering farms with Sling, showing a mix of
Sling-based experiments and theoretical considerations on what would
help creating such farms.

Having chroot-style [1] user segregation at the repository level would
help: after opening a session as a member of the jail group "foo",
/jails/foo becomes my new root, blocking me from accessing anything
above that and transparently mapping my repository root to /jails/foo.

Access control can of course help implementing this, but having the
path mapping to transparently jail the user or group in their own
subtree makes things much easier at the application level.

Has anyone already played with something like this?
Any prototypes or experiments worth mentioning?

-Bertrand

[1] https://linux.die.net/man/2/chroot

Re: OAK-6575 - A word of caution

[Bertrand Delacretaz]

On Mon, Sep 4, 2017 at 3:44 PM, Ian Boston  wrote:
> ...I feel
> that Oak is weaker without the ability to offload bulk data streaming to
> infrastructure designed for that purpose

FWIW as an Oak user I share that feeling, IMO the use cases described
at https://wiki.apache.org/jackrabbit/JCR%20Binary%20Usecase are
becoming more and more important.

Not being a committer I don't really care about the internals, but
please do not "throw the baby out with the bath water" if the
internals need to change.

-Bertrand

Re: [DiSCUSS] - highly vs rarely used data

[Bertrand Delacretaz]

Hi Thomas,

On Tue, Jul 11, 2017 at 3:14 PM, Thomas Mueller
 wrote:
> ...if it's in cold storage, throw an exception saying so, and load the binary 
> into hot storage.
> A few minutes later, re-reading will not throw an exception as it's in hot 
> storage

Ok great, sorry that I missed that earlier.

Note that the exception should not prevent the client from getting the
rest of the data (other properties) of the same Node - I suppose
that's natural if the exception is thrown when calling
Binary.getStream().

-Bertrand

Re: [DiSCUSS] - highly vs rarely used data

[Bertrand Delacretaz]

On Tue, Jul 4, 2017 at 12:48 PM, Julian Sedding  wrote:
> ...I suggest to come up with a pluggable "strategy" interface and
> provide a sensible default implementation...

Big +1 to that, requirements can vary widely IMO, also depending on
the characteristics of whatever cold storage is used.

> ...A much more important and difficult question to answer IMHO is how to
> deal with the slow retrieval of archived content

Throw an exception maybe? BinaryNotAvailableAtThisTime, including an
ETA for availability. The application can then decide how to handle
that.

-Bertrand

Re: [DiSCUSS] - highly vs rarely used data

[Bertrand Delacretaz]

On Fri, Jun 30, 2017 at 10:44 AM, Thomas Mueller
 wrote:
> ...About deciding which binaries to move to the slow storage: It would be 
> good if that's automatic...

>From my perspective as an Oak user I would like to have control on that.

It would be nice for Oak to make *suggestions* about moving things to
cold storage, but there might be application constraints that need to
be accounted for.

-Bertrand

Re: Enforcing minimal test coverage

[Bertrand Delacretaz]

Hi Angela,

On Thu, May 11, 2017 at 4:41 PM, Angela Schreiber
 wrote:
> ...As far as I am concerned I only run the ITs if I know (or
> suspect) a change to have a huge impact otherwise it just takes too
> long...

Ok! I just meant to expose our solution in case it's useful otherwise.
Our setup also works if you skip integration tests, the "unit" and
"merged" jacoco reports are then identical (which makes tons of sense
;-)

-Bertrand

Re: Enforcing minimal test coverage

[Bertrand Delacretaz]

On Thu, May 11, 2017 at 2:46 PM, Michael Dürig  wrote:
> ...I guess this only includes the coverage from running unit tests and does 
> not
> include integration tests...

FWIW, In Sling we have a setup that computes the aggregate coverage of
unit and integration tests that run in a single Maven module.

That's defined in the jacoco-report profile in the parent pom at [1].

-Bertrand

https://svn.apache.org/repos/asf/sling/trunk/parent/pom.xml

Re: new name for the multiplexing node store

[Bertrand Delacretaz]

On Thu, May 11, 2017 at 9:33 AM, Robert Munteanu  wrote:
> ...MultiplexingNodeStore is a pretty standard implementation
> of the Composite design pattern...

So CompositeNodeStore maybe? I like it.

-Bertrand

Expiring the META/repository-* markers used by MarkSweepGarbageCollector ?

[Bertrand Delacretaz]

Hi,

When a client connects only temporarily to a SharedS3DataStore (for
example) and then goes away, the META/repository-* marker created by
SegmentNodeStoreService is not removed.

This causes MarkSweepGarbageCollector to abort with a "not all
repositories have marked references available" message.

Do people see an issue with adding an expiration time to those
META/repository-* markers?

MarkSweepGarbageCollector can then ignore expired markers, considering
them to belong to long gone clients.

I suppose that the expiration time can be stored as data in the marker
blob, and it would have to be refreshed periodically by then client,
unless configured to never expire.

I can provide a patch for that but wanted to first check for any
issues that I overlooked, as I'm not familiar with that code.

What do people think?

-Bertrand

Re: the broken JCR locking

[Bertrand Delacretaz]

hi Angela,

On Wed, Apr 5, 2017 at 3:18 PM, Angela Schreiber  wrote:
> ...Does no reply mean that everyone agrees with the proposed steps? :-)...

I missed it but as an Oak user +1 to the below suggestion, it makes
things clear ;-)

> On 22/03/17 10:35, "Angela Schreiber"  wrote:
>>...That would mean that we change the corresponding repository descriptor
>>OPTION_LOCKING_SUPPORTED to return false and adjust oak-jcr to throw
>>UnsupportedRepositoryOperation upon all lock calls defined by the JCR 2.0...

-Bertrand

Re: [FileVault][discuss] performance improvement proposal

[Bertrand Delacretaz]

Hi,

On Tue, Mar 7, 2017 at 2:28 PM, Timothée Maret  wrote:
> ...IMO we should still allow to tweak between best performance and best
> compression though, in order to accommodate different use cases...

Best compression is probably just "compress everything regardless", right?
If yes that's a simple configuration switch.

-Bertrand

Re: svn commit: r1781064 - /jackrabbit/oak/trunk/oak-doc/src/site/markdown/query/index-management.md

[Bertrand Delacretaz]

On Wed, Feb 1, 2017 at 5:29 AM, Chetan Mehrotra
 wrote:
> ...Scripting in JSON looks interesting!...

arghh...not if a human has to write it!

I've done lots of XSLT in the past, and I love that language's data
model, power and expressiveness.

But I think using an XML-based syntax killed it in many cases.

-Bertrand

Re: RIP Apache Jenkins!?

[Bertrand Delacretaz]

Hi,

On Tue, Nov 29, 2016 at 11:15 AM, Thomas Mueller  wrote:
...
> https://issues.apache.org/jira/browse/INFRA-9709
>
> ...This is just crazy. Either issues get resolved, or they don't, in which
> case we should get a notification that they don't...

I agree. The above issue is about Sonar which I suspect is low
priority compared to Jenkins, and the ASF infra team was understaffed
for several months this year AFAIK so I suspect priorities have gotten
in the way.

I haven't worked much on Jenkins lately but I know Robert got great
results for Sling by adapting our build better to how the ASF's
Jenkins works, so maybe there's hope in that direction?

OTOH if people have concrete requests for how
https://builds.apache.org/ can be more useful I'm happy to relay those
concerns to ASF infra. The more concrete the better.

-Bertrand

Re: RIP Apache Jenkins!?

[Bertrand Delacretaz]

On Tue, Nov 29, 2016 at 10:13 AM, Robert Munteanu  wrote:
> ...There's also option #3, which is even more interesting :-)...

And option #4 - donate some computing capacity to run some dedicated
Jenkins slaves for Oak.

-Bertrand

Re: RIP Apache Jenkins!?

[Bertrand Delacretaz]

On Thu, Nov 17, 2016 at 9:27 AM, Michael Dürig  wrote:
> ...I was mostly thinking of using some internal resources as so far the public
> options didn't work out for us...

Internal to what?

-Bertrand

Re: On adding new APIs

[Bertrand Delacretaz]

On Thu, Oct 20, 2016 at 2:02 PM, Michael Marth  wrote:
> ...So I have a proposal: when a new public API is added the developer should 
> drop an
> email with subject tag [REVIEW] onto the dev list, so that others are aware 
> and can
> chime in if needed...

The Oak team can also just decide that new APIs are to be handled in
R-T-C mode instead of C-T-R.

It's kind of the same thing but using standard Apache terminology ;-)

-Bertrand

Re: Possibility of making nt:resource unreferenceable

[Bertrand Delacretaz]

On Wed, Oct 12, 2016 at 11:18 AM, Julian Sedding  wrote:
> ...As a remedy for implementations that rely on the current referencable
> nature, we could provide tooling that automatically adds the
> "mix:referencable" mixin to existing nt:resource nodes...

Good idea, I suppose this can be done with a commit hook in a non-intrusive way?

-Bertrand

Re: Seekable access to a Binary

[Bertrand Delacretaz]

Hi,

On Tue, Sep 6, 2016 at 1:25 PM, Ian Boston  wrote:
> On 6 September 2016 at 12:14, Marcel Reutegger  wrote:
>>... This can easily be prevented if the 'in progress' binary is
>> uploaded to a temporary location first and then copied over
>> to the correct location once complete

> Is that exposed as an API that can be used by Sling ?...

I think Marcel just means uploading to a node that's not visible to
clients first, and then moving that node to its final destination, so
that the binary is not visible too early there.

-Bertrand

Re: Seekable access to a Binary

[Bertrand Delacretaz]

Hi,

On Tue, Sep 6, 2016 at 9:49 AM, Marcel Reutegger  wrote:
> ...we'd still have to add
> Jackrabbit API to support it. E.g. something like:
>
> valueFactory.createBinary(existingBinary, appendThisInputStream); ...

And maybe a way to mark the binary as "in progress" to avoid
applications using half-uploaded binaries?

Maybe just a boolean property convention that application developers
are supposed to take into account, as I don't think JCR Sessions work
in that use case.

-Bertrand

Re: Requirements for multiple Oak clients on the same backend (was: [suggestion] introduce oak compatibility levels)

[Bertrand Delacretaz]

Hi Michael,

On Thu, Jul 28, 2016 at 11:03 AM, Michael Marth  wrote:
> ...to my knowledge there is no formal way to find out which values of X and Y 
> are safe - at least so far

Ok, thanks for the info!

That might be something to keep in mind when designing new features -
detecting that you're running on an incompatible version of the
storage/indexes/whatever and loudly failing if the conversion cannot
happen on the fly.

(which is similar to Stefan's question about compatibility levels)

-Bertrand

Requirements for multiple Oak clients on the same backend (was: [suggestion] introduce oak compatibility levels)

[Bertrand Delacretaz]

Hi,

On Thu, Jul 28, 2016 at 10:23 AM, Stefan Egli  wrote:
>...we could introduce a concept of
> 'compatibility levels' which are a set of features/behaviours that a
> particular oak version has and that application code relies upon

Good timing, I have a related question about multiple client apps
connecting to the same Oak backend.

Say I have to Java apps A and B which use the same Oak/Mongo/BlobStore
configuration, are there defined requirements as to the Oak library
versions or other settings that A and B use?

Do they need to use the exact same versions of the Oak bundles, and
are violations to that or to other compatibility requirements
detected?

-Bertrand

Re: Usecases around Binary handling in Oak

[Bertrand Delacretaz]

Hi Chetan,

On Wed, Jun 1, 2016 at 9:30 AM, Chetan Mehrotra
 wrote:
> ...To move forward on that I have tried to collect the various usecases at [2]
> which I have seen in the past

I've thought about adding an "adopt-a-binary" feature to Sling
recently, to allow it to serve existing (disk or cloud) binaries along
with those stored in Oak.

I think this might help for your use cases. I'm not sure about
implementing it at the Sling level or in a custom BlobStore so far,
but that's not important at this point.

Here's how I envision this:

1. Client prepares a set of binaries and metadata on disk or on cloud
storage. This happens without interacting with Sling, to make it
easier to farm out the costly binary preparation / metadata extraction
etc.

2. Client POSTs those binaries and their metadata to Sling, but
instead of including the actual binaries it uploads small references
in a specific binary format that's conceptually like
SLINGREF:mystore:myfile.mp4: a constant prefix + binary URI.

4. When serving such a binary, Sling recognizes the SLINGREF: prefix
(which needs to be made robust/unique) and dereferences it to get an
InputStream

Of course this means fully delegating the management of binaries to
external tools, though they can be replaced transparently with actual
non-adopted ones.

Also, if fine-grained access control is needed it needs to be
implemented by the URI resolvers that provide the actual binaries.

But in some cases I think using Sling/Oak as "just" a metadata
decorator for existing binaries might make a lot of sense.

Although this is a Sling idea and only in the back of my mind so far,
I wanted to mention it here as there are parallels with your use
cases.

-Bertrand




> [2] https://wiki.apache.org/jackrabbit/JCR%20Binary%20Usecase

Why is nt:resource referencable?

[Bertrand Delacretaz]

Hi,

OAK-4567 introduces a new oak:Resource nodetype as an alternative to
nt:resource which has mix:referenceable.

I understand the idea of making that node non-referencable for
performance reasons, but why is nt:resource referenceable in the first
place?

builtin_nodetypes.cnd [1] has this definition

  [nt:resource] > mix:mimeType, mix:lastModified, mix:referenceable
primaryitem jcr:data
- jcr:data (BINARY) mandatory

but the JCR spec (JSR 283 10 August 2009) only has

  [nt:resource] > mix:mimeType, mix:lastModified
primaryitem jcr:data
- jcr:data (BINARY) mandatory

Introducing a new nodetype leaks into applications, removing the
default mix:referenceable mixin on nt:resource would be much simpler
(at the cost of a migration tool probably), and people can always
re-add the mixin if they really need it.

WDYT?

-Bertrand

[1] 
http://svn.apache.org/repos/asf/jackrabbit/oak/trunk/oak-core/src/main/resources/org/apache/jackrabbit/oak/plugins/nodetype/write/builtin_nodetypes.cnd

Re: Clarification on Oak vs. Jackrabbit

[Bertrand Delacretaz]

On Tue, Jul 19, 2016 at 11:59 AM, Michael Dürig  wrote:
> ...From this POV I think we can dismiss the board's concerns as Oak is an
> evolution of Jackrabbit and will eventually replace it in one way or
> another. They are clearly not disjoint

To clarify, from the board's point of view what's important is the community.

If Oak and Jackrabbit are clearly managed by the same community it's
fine to have both under a single PMC.

On the other hand if a community emerges to continue to work on
Jackrabbit in the longer term (which I assume the people currently
working on Oak are doing less and less) it might make sense to split
the PMCs.

-Bertrand

Re: [OAK] - Problem to version Node when adding versionable mixin

[Bertrand Delacretaz]

Hi,

On Mon, Jun 20, 2016 at 5:25 PM, Ciro Santos  wrote:
> ...I don't know if i need to add something to the OSGI configuration. I also
> cannot find any proper documentation on how to set up oak bundles using
> OSGI

Sling is a good example that uses Oak in an OSGi enviromment.

If you want to try it [1] has build instructions and you can then
execute the runnable jar built by the launchpad/builder module.

The main bundle that relates to Oak is under bundles/jcr/oak-server,
you might be able to run it in your OSGi environment but it will
require some Sling bundles as dependencies so it's probably quicker to
test it using the Sling launchpad jar.

HTH,
-Bertrand

[1] 
http://sling.apache.org/documentation/development/getting-and-building-sling.html

Re: JIRA issue not showing associated commits

[Bertrand Delacretaz]

Hi,

On Mon, Jan 4, 2016 at 5:26 AM, Chetan Mehrotra
 wrote:
>... Any idea on how to get that integration back? Probably when we moved to
> Epic based model it got lost...

I suspect (but I'm not sure) that feature was turned off by ASF
infrastructure for performance reasons. If you want to ask best is
probably to create an INFRA jira ticket so that the answer gets a
permanent URL.

-Bertrand

[jira] [Commented] (JCRVLT-61) Allow AccessControllHandling be defined per filter root

[Bertrand Delacretaz (JIRA)]

[ 
https://issues.apache.org/jira/browse/JCRVLT-61?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15089037#comment-15089037
 ] 

Bertrand Delacretaz commented on JCRVLT-61:
---

FWIW I'm working on something similar in SLING-5355. We might move the parser 
and core code to Oak later if you guys want it.

> Allow AccessControllHandling be defined per filter root
> ---
>
> Key: JCRVLT-61
> URL: https://issues.apache.org/jira/browse/JCRVLT-61
> Project: Jackrabbit FileVault
>  Issue Type: Improvement
>  Components: Packaging
>Reporter: Tobias Bocanegra
>
> the current packaging only allows to specify the AccessControllHandling per 
> package. if one requires different behaviors for different content trees, the 
> only workaround today is to create sub-packages.
> it would be nice if the ac-control handling can be defined by workspace 
> filter root, similar to the iImportMode - or even tie the 
> AccessControllHandling to the ImportMode. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: DocumentRDB builds time out on Jenkins

[Bertrand Delacretaz]

On Thu, Dec 10, 2015 at 11:27 AM, Michael Dürig  wrote:
> On 10.12.15 11:02 , Tomek Rekawek wrote:
>> ...Maybe we can change the configuration, so the
>> target/unittest.log file will be archived if the job is failed?...
>
> This would be useful anyway but I think so far no one has figured out how to
> do it

On Jenkins, "Archive the artifacts" in "Post-build Actions" should do it.

-Bertrand

Re: Semantic version in Oak

[Bertrand Delacretaz]

Hi,

On Tue, Dec 8, 2015 at 12:38 PM, Michael Dürig  wrote:
> ...One idea that crossed my mind was to remove semantic versioning for the
> stuff we only export for consumption by ourselves...

>...The "real public APIs" are much more stable and we
> should be able do proper semantic versioning for those

I suppose moving those "real public APIs" to a distinct module (or set
of modules) with its own release cycle, and using semantic versioning
for that would be a big help, if that's possible.

-Bertrand

Re: v1.3.9 - http state of play?

[Bertrand Delacretaz]

On Tue, Nov 10, 2015 at 2:54 PM, Lukas Kahwe Smith  wrote:
> ...its not so easy to find a PHP dev willing and capable to contribute on the
> code level of a Java project. for now we can just do our part in
> giving feedback and testing

Understood. Feedback + tests are great already.

-Bertrand

Re: Content from multiple systems

[Bertrand Delacretaz]

Hi,

On Thu, Sep 24, 2015 at 4:49 AM, toramanan  wrote:
> ...I have seen Apache ManifoldCF and exploring
> the tool. Is there any other good alternatives to explore?...

Apache Sling, which runs on top of Jackrabbit, provides a
ResourceProvider mechanism that allows you to integrate content from
various sources into the Sling URL space fairly easily, especially if
it's for read-only purposes.

Docs are at
https://sling.apache.org/documentation/the-sling-engine/resources.html

and a very simple ResourceProvider example at

http://svn.apache.org/repos/asf/sling/trunk/launchpad/test-services/src/main/java/org/apache/sling/launchpad/testservices/resourceprovider/

-Bertrand

Re: oak-run upgrade improvements

[Bertrand Delacretaz]

On Thu, Sep 3, 2015 at 10:01 AM, Robert Munteanu  wrote:
> ... _If_ we decide OSGi, it must be embedded and controlled by the oak
> -run instance, and not a general-use OSGi application like e.g. Karaf
> with some bundles added...

Note that Sling's slingstart-maven-plugin makes it easy to create a
runnable jar that embeds the OSGi framework, dependencies, default
configs etc.

As a user of Oak in OSGi environments I'm very much in favor of using
OSGi for oak-run, as this would help make sure Oak stays OSGi-friendly
while not "polluting" the core Oak modules with OSGi stuff.

-Bertrand

Re: Is the hashing of long paths still needed?

[Bertrand Delacretaz]

On Fri, Aug 21, 2015 at 1:46 PM, Davide Giannella dav...@apache.org wrote:
...
 mvn clean install -PintegrationTesting -Dnsfixtures=

Ah ok, the LongPathTest in oak-jcr does fail with my patch when using
-Dnsfixtures=

Sorry for the noise - we'll have to find a way to cope with this for
the MultiplexingDocumentStore.

-Bertrand

Is the hashing of long paths still needed?

[Bertrand Delacretaz]

Hi,

Continuing to play with Robert's MultiplexingDocumentStore [1] I got a
failure in the oak-jcr module's LongPathTest.

That's due to the conversion of long paths to their hashed variants -
those cannot be used to locate the appropriate DocumentStore when
multiplexed, as that decision is based on the real path. In some cases
like UpdateOp the real path is saved alongside the hashed one, but not
always.

Removing the path hashing as shown in the below patch removes that
problem, and all oak-core tests pass with this change.

Is that hashing still needed, or was that created due to backend
limitations which are gone now?

If still needed, could it be made configurable so that users can make
their own tradeoff (MultiplexingDocumentStore vs. supporting backends
that require that), or are there other constraints?

-Bertrand

[1] https://github.com/bdelacretaz/jackrabbit-oak/tree/bertrand-multiplex



diff --git 
a/oak-core/src/main/java/org/apache/jackrabbit/oak/plugins/document/util/Utils.java
b/oak-core/src/main/java/org/apache/jackrabbit/oak/plugins/document/util/Utils.java
index 4577c4b..d97ae76 100644
--- 
a/oak-core/src/main/java/org/apache/jackrabbit/oak/plugins/document/util/Utils.java
+++ 
b/oak-core/src/main/java/org/apache/jackrabbit/oak/plugins/document/util/Utils.java
@@ -254,6 +254,7 @@ public class Utils {
 }

 public static String getIdFromPath(String path) {
+/*
 if (isLongPath(path)) {
 MessageDigest digest;
 try {
@@ -267,6 +268,7 @@ public class Utils {
 String name = PathUtils.getName(path);
 return depth + :h + Hex.encodeHexString(hash) + / + name;
 }
+*/
 int depth = Utils.pathDepth(path);
 return depth + : + path;
 }
diff --git 
a/oak-core/src/test/java/org/apache/jackrabbit/oak/plugins/document/util/UtilsTest.java
b/oak-core/src/test/java/org/apache/jackrabbit/oak/plugins/document/util/UtilsTest.java
index b9f619e..5ed1a95 100644
--- 
a/oak-core/src/test/java/org/apache/jackrabbit/oak/plugins/document/util/UtilsTest.java
+++ 
b/oak-core/src/test/java/org/apache/jackrabbit/oak/plugins/document/util/UtilsTest.java
@@ -32,6 +32,7 @@ import org.junit.Test;

 import static org.junit.Assert.assertEquals;
 import static org.junit.Assert.assertNull;
+import static org.junit.Assert.assertNotNull;
 import static org.junit.Assert.assertSame;
 import static org.junit.Assert.assertTrue;

@@ -62,7 +63,8 @@ public class UtilsTest {
 String longPath = PathUtils.concat(/+Strings.repeat(p,
Utils.PATH_LONG + 1), foo);
 assertTrue(Utils.isLongPath(longPath));

-assertNull(Utils.getParentId(Utils.getIdFromPath(longPath)));
+// updated to match the changes to Utils.getIdFromPath
+assertNotNull(Utils.getParentId(Utils.getIdFromPath(longPath)));

 assertNull(Utils.getParentId(Utils.getIdFromPath(/)));
 assertEquals(1:/foo,Utils.getParentId(2:/foo/bar));

Re: Is the hashing of long paths still needed?

[Bertrand Delacretaz]

On Fri, Aug 21, 2015 at 12:11 PM, Julian Reschke
julian.resc...@greenbytes.de wrote:

 ...It is still needed, as both persistence implementations (Mongo, RDB) are
 restricted to IDs of 512 bytes length (yes, bytes, not characters)

Ok, thanks for the info!

Interestingly, no oak-core or oak-jcr tests fail when I remove this
feature and test with Mongo, I'll dig deeper.

-Bertrand

Multiplexing DocumentStore, NodeStoreTest fails even without mounts (was: A multiplexing implementation of the DocumentStore)

[Bertrand Delacretaz]

Hi Robert,

On Wed, Jul 8, 2015 at 3:27 PM, Robert Munteanu romb...@apache.org wrote:
 ...I am working on a prototype to multiplex multiple DocumentStore
 instances behind a single DocumentStore...

I played with this stuff and added a fixture to run the standard
oak-core tests on a MultiplexingDocumentStore.

My latest finding is that NodeStoreTests fails in that case, even if
the MultiplexingDocumentStore has no mounts, just a root store, which
is surprising.
I'll investigate more, but maybe that's useful info for you.

Here's what I get at my commit
https://github.com/bdelacretaz/jackrabbit-oak/commit/66f8de76ece6f32b0151deaca47d63821f9908bc

$ mvn clean test -Dnsfixtures=MEMORY_MULTI_NS -Dtest=NodeStoreTest
...
Failed tests:
beforeCommitHook[0](org.apache.jackrabbit.oak.spi.state.NodeStoreTest)
  
compareAgainstBaseState20[0](org.apache.jackrabbit.oak.spi.state.NodeStoreTest):
expected:21 but was:20
  
compareAgainstBaseState100[0](org.apache.jackrabbit.oak.spi.state.NodeStoreTest):
expected:101 but was:80
  rebaseWithFailedMerge[0](org.apache.jackrabbit.oak.spi.state.NodeStoreTest):
must throw CommitFailedException

Feel free to merge the MEMORY_MULTI_NS fixture stuff if that's useful
to you, of course.

-Bertrand

Re: Multiplexing DocumentStore, NodeStoreTest fails even without mounts (was: A multiplexing implementation of the DocumentStore)

[Bertrand Delacretaz]

On Wed, Aug 19, 2015 at 5:09 PM, Michael Dürig mdue...@apache.org wrote:
 ...AFAIKs the problem is with the test setup where state is kept between test
 executions. Namely the SpyingMemoryNodeStore field...

Good catch, this indeed fixes the oak-core tests on my branch, thanks!

-Bertrand

Re: System.exit()???? , was: svn commit: r1696202 - in /jackrabbit/oak/trunk/oak-core/src/main/java/org/apache/jackrabbit/oak/plugins/document: ClusterNodeInfo.java DocumentMK.java DocumentNodeStore.j

[Bertrand Delacretaz]

Hi,

On Tue, Aug 18, 2015 at 10:57 AM, Julian Reschke julian.resc...@gmx.de wrote:
 I'm a bit concerned (and that's an understatement) that OAK is now calling
 System.exit. Detecting a serious problem - good. Stopping the content
 repository - probably good, at least for write operations? But stopping the
 whole VM, no matter what else it runs? Seriously?..

+1

-Bertrand

Re: Recipe for using Oak in standalone environments

[Bertrand Delacretaz]

On Tue, Aug 18, 2015 at 2:22 PM, Chetan Mehrotra
chetan.mehro...@gmail.com wrote:
 ...Q - Should we make oak-pojosr based setup as one of the
 recommended/supported approach for configuring Oak in non OSGi env...

From the peanuts gallery, I like the idea.

However, I cannot find any docs on Felix Connect which is AFAIK where
pojosr lives now, it's not even listed at
http://felix.apache.org/downloads.cgi - I'll ask on the Felix dev
list.

-Bertrand

Re: Modularization

[Bertrand Delacretaz]

Hi,

On Mon, Aug 10, 2015 at 8:50 AM, Philipp Suter su...@adobe.com wrote:
 ...I don’t think that it makes sense to have multiple separated maven 
 projects for OAK...

FWIW, note that in Sling we do have a few independent Maven project trees:

-The main pom builds the core modules and their integration tests
-A separate contrib tree and main pom
-Another tree and main pom for tools (Maven plugins etc.)

I have found it useful to have a clear separation between the sacred
core modules and other less critical things. There's no excuse for
breaking the core build but we use a smaller trout to slap whoever
breaks the other ones.

-Bertrand

Re: Modularization

[Bertrand Delacretaz]

On Thu, Aug 6, 2015 at 11:50 AM, Thomas Mueller muel...@adobe.com wrote:
 ...I very much prefer if things that belong together stay together, and are
 not taken appart into *-api, *-common, *-impl projects. Because taking
 things apart makes maintenance very hard

I agree, besides doing just one thing a module should be as complete
as possible. What you're showing is a typical example of too small
modules.

OTOH if one module does storage and indexing and access control and
monitoring for example it's too much IMO, that should ideally be four
modules.


 Testing is always a challenge, and testing many possible combinations of
 bundles in different version is just not possible

Also agreed, in Sling we are only testing very few combinations,
actually just one combination per release.

-Bertrand

Re: Modularization

[Bertrand Delacretaz]

On Thu, Aug 6, 2015 at 12:04 PM, Michael Dürig mdue...@apache.org wrote:
 ...This is not about having many Maven projects but reasonable
 module boundaries. The definition of which is low cohesion between modules
 and hight cohesion within them

Which is exactly the reason for Sling moving to smaller modules in
many places - create modules that Do One Thing And One Thing Well.

(and I'll end the Sling-based sidetracking now ;-)

-Bertrand

Re: Modularization

[Bertrand Delacretaz]

On Thu, Aug 6, 2015 at 12:25 PM, Thomas Mueller muel...@adobe.com wrote:
 Bertrand wrote:
...in Sling we are only testing very few combinations,
actually just one combination per release.

 Good to know! I still don't quite understand why you have many projects
 then... what is the driver to create smaller and smaller projects?...

To give two concrete examples:

https://issues.apache.org/jira/browse/SLING-3529 - the Sling launcher
contains code to detect if the current instance is a fresh one or an
upgraded one. There's currently no way to use just that startup mode
detection feature without using the full launcher, so we need to
extract it to make both things (launcher and startup mode detection)
reusable independently, when running Sling with other launchers like
Karaf for example.

https://issues.apache.org/jira/browse/SLING-4694 - Sling has a module
to detect media types based on filenames, and someone needed an
extension to also take the file contents into account. Creating that
extension in a separate module means that no changes were done to the
existing commons.mime module (which is a good indication of its
stability), and we don't bloat Sling with stuff that people might not
need - they can just decide to use the new bundle or not. Clean
separation between what's stable and what's in flux.

When I started with OSGi I was kind of afraid of having too many small
modules, as it felt harder to manage. We still ended up with more than
150 different modules released at
http://sling.apache.org/downloads.cgi, so having 100, 200 or 300 or
them does not matter much, and the benefits are obvious to in terms of
clearly differentiating what's in flux and what's stable.

I'm not saying Oak should have 300 modules...but maybe more than one ;-)

On Thu, Aug 6, 2015 at 1:37 PM, Davide Giannella dav...@apache.org wrote:
 ...I always struggle navigating through the Sling codebase but indeed I
 never really dedicated that much time

In a highly modular system you might need someone (or a debugger) to
help you find out where to look for something, but once you find that
the scope is much much smaller and it's much easier to make sure you
don't break things that you don't understand. IMO for a long-lived
project that's a huge benefit - and I always take
http://httpd.apache.org/ as a great example for that - very small
core, everything is a module.

-Bertrand

Re: Modularization

[Bertrand Delacretaz]

On Thu, Aug 6, 2015 at 10:51 AM, Francesco Mari
mari.france...@gmail.com wrote:
 ...What is your take on modularisation?..

From the peanuts gallery I'll just observe that since we started Sling
seven years ago we are constantly moving towards smaller and more
specific modules (OSGi bundles in that case), even though we already
have many of them.

Grady Booch's rule that a class should do one thing and one thing
well applies very well to modules/bundles IMO, being specific makes
many things easier and many modules become trivial and change rarely
which helps focus efforts in the right places.

You do need strong integration tests to verify that a given
combination of modules works then, but that also helps make your
project better. And you don't need to bless many combinations, maybe
just define a release cadence for a known good combination of modules
that becomes the next release.

-Bertrand

Re: JCR + MongoDb + Lucene Holy Grail Finally Found

[Bertrand Delacretaz]

On Tue, Aug 4, 2015 at 5:08 PM, Clay Ferguson wcl...@gmail.com wrote:
 ...I think *the* primary reason OSGi has a place in the world, is because it
 can make completely incompatible set of things be able to run together...

That's one aspect of OSGi, but our experience with Sling is that OSGi
is also a fantastic way of assembling very modular and dynamic
systems. The tooling and examples might not be as sexy as other
similar frameworks, but for the kind of industrial strength stuff that
Sling is aimed at OSGi works extremely well for us.

And on top of that, as you say, when one needs to integrate
legacy/ugly code OSGi can be a lifesaver.

-Bertrand

Re: OSGi configuration lookup

[Bertrand Delacretaz]

On Tue, Jun 30, 2015 at 5:52 AM, Chetan Mehrotra
chetan.mehro...@gmail.com wrote:
 ...if we need to do that for
 Segment also then we should use a similar namespacing...

And that should IMO be done with a utility class instead of duplicating code.

-Bertrand

Re: Initial work for the specification of a remote API

[Bertrand Delacretaz]

On Mon, Jan 26, 2015 at 12:38 PM, Francesco Mari
mari.france...@gmail.com wrote:
 ...Other people already applied this concept successfully with
 the creation of the JSON Patch standard [1]

I wasn't aware of that upcoming standard, looks interesting indeed!

-Bertrand

 [1]: https://tools.ietf.org/html/rfc6902

Re: Initial work for the specification of a remote API

[Bertrand Delacretaz]

On Mon, Jan 26, 2015 at 10:28 AM, Francesco Mari
mari.france...@gmail.com wrote:
 At the beginning I wanted to expose a more granular interface for node
 operations, mapping every node to a fully REST resource

BTW, what happened to JSOP?
http://slideshare.net/uncled/jsop

-Bertrand

Re: Initial work for the specification of a remote API

[Bertrand Delacretaz]

On Mon, Jan 26, 2015 at 9:55 AM, Francesco Mari
mari.france...@gmail.com wrote:
 ...I think a multi-tree read request could be a good improvement to the
 API...

In general, considering that you send a bunch of commands to the
server and get a bunch of corresponding responses back might help a
lot in reducing traffic. You might need to repeat part of the command
in the response (or at least its ID) to keep things simple on the
client side.

-Bertrand

[jira] [Commented] (JCRSITE-47) Site is completely broken

[Bertrand Delacretaz (JIRA)]

[ 
https://issues.apache.org/jira/browse/JCRSITE-47?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14201858#comment-14201858
 ] 

Bertrand Delacretaz commented on JCRSITE-47:


Both http and https work for me now:

{code}
$ curl -s https://jackrabbit.apache.org/ | grep Welcome
TITLEWelcome to Apache Jackrabbit/TITLE
LIA href= title=Welcome to Apache 
JackrabbitWelcome/A/LI
  H1Welcome to Apache Jackrabbit/H1
H2A name=WelcometoApacheJackrabbit-ApacheJackrabbitNews/AApache 
Jackrabbit News/H2

$ curl -s http://jackrabbit.apache.org/ | grep Welcome
TITLEWelcome to Apache Jackrabbit/TITLE
LIA href= title=Welcome to Apache 
JackrabbitWelcome/A/LI
  H1Welcome to Apache Jackrabbit/H1
H2A name=WelcometoApacheJackrabbit-ApacheJackrabbitNews/AApache 
Jackrabbit News/H2
{code}

 Site is completely broken
 -

 Key: JCRSITE-47
 URL: https://issues.apache.org/jira/browse/JCRSITE-47
 Project: Jackrabbit Site
  Issue Type: Bug
  Components: site
Reporter: Rick Herrick
Priority: Blocker
 Attachments: Screen Shot 2014-11-05 at 7.27.14 AM.png


 The [Jackrabbit site|http://jackrabbit.apache.org] is completely broken and 
 unusable. It has been for at least a couple of days. We're trying to decide 
 if we should be using Jackrabbit as a back-end storage system. Does this 
 indicate we shouldn't be using Jackrabbit as it's not being maintained? 
 Should we be looking to Oak instead?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (JCRSITE-47) Site is completely broken

[Bertrand Delacretaz (JIRA)]

[ 
https://issues.apache.org/jira/browse/JCRSITE-47?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14198377#comment-14198377
 ] 

Bertrand Delacretaz commented on JCRSITE-47:


It looks like the 
https://svn.apache.org/repos/asf/jackrabbit/site/live/.htaccess file is not 
active on the live website, as a result the content is under JCR, like 
http://jackrabbit.apache.org/JCR/documentation.html instead of 
http://jackrabbit.apache.org/documentation.html

 Site is completely broken
 -

 Key: JCRSITE-47
 URL: https://issues.apache.org/jira/browse/JCRSITE-47
 Project: Jackrabbit Site
  Issue Type: Bug
  Components: site
Reporter: Rick Herrick
Priority: Blocker
 Attachments: Screen Shot 2014-11-05 at 7.27.14 AM.png


 The [Jackrabbit site|http://jackrabbit.apache.org] is completely broken and 
 unusable. It has been for at least a couple of days. We're trying to decide 
 if we should be using Jackrabbit as a back-end storage system. Does this 
 indicate we shouldn't be using Jackrabbit as it's not being maintained? 
 Should we be looking to Oak instead?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: [SITE] Updating Jackrabbit Site

[Bertrand Delacretaz]

Hi Jukka,

On Wed, Oct 29, 2014 at 6:07 PM, Jukka Zitting jukka.zitt...@gmail.com wrote:
 ...it's just a static HTML site published directly with svnpubsub.
 http://markmail.org/message/etbl5hcoupwhioft ..

ah ok - it's confusing then that there is content at
https://cms.apache.org/jackrabbit/

-Bertrand

Re: [SITE] Updating Jackrabbit Site

[Bertrand Delacretaz]

Hi,

On Tue, Oct 28, 2014 at 9:08 PM, Tobias Bocanegra tri...@apache.org wrote:
 ...does anyone know why the Jackrabbit site isn't updating?...

Did you publish your changes?

https://cms.apache.org/jackrabbit/publish?diff=1 should show them if
not published (but that URL doesn't work for me right now).

-Bertrand

10-second delay in observation?

[Bertrand Delacretaz]

Hi,

I'm sometimes seeing a 10-second delay in delivering observation
events, in Sling integration tests running on Oak 1.07 with TarMK.
Details at SLING-4058.

At this point I'm not sure if Oak or Sling is the culprit as I'm
listening for the OSGi events that Sling generates, so not directly to
Oak events - still investigating.

But if someone has pointers as to what might be wrong, please let me know!

-Bertrand

Re: questions

[Bertrand Delacretaz]

Hi,

On Mon, Oct 13, 2014 at 9:55 AM, TALHAOUI Mohamed m.talha...@rsd.com wrote:
 ...I found Jackrabbit 
 OCMhttp://jackrabbit.apache.org/5-with-jackrabbit-ocm.html but
 nothing in Oak.
 Is there something in the pipe  ? ...

As Michael says, probably not in Oak.

The Sling Models [1] are IMO a nice way of handling JCR-to-object
mappings, but that requires using Sling as your applications layer on
top of Oak.

Depending on the current state of your project, using Sling might be
useful as it provides lots of out-of-the-box functionality when
working with JCR storage.

-Bertrand (Sling committer, so heavily biased ;-)

[1] http://sling.apache.org/documentation/bundles/models.html

Re: [DISCUSS] supporting faceting in Oak query engine

[Bertrand Delacretaz]

On Sat, Aug 30, 2014 at 4:25 AM, Alexander Klimetschek
aklim...@adobe.com wrote:
 ...you can leverage some kind of caching though. In practice, if you have a 
 public site
 with content that does not change permanently, the facet values are pretty 
 much
 stable, and authorization shouldn't cost much

Yes, I think it's very rare to require facets to be immediately up to
date after content changes, updating them (or the related caches)
asynchronously with low priority should be good enough for the large
majority of cases.

So maybe the facet indexes and caches can be handled differently than
primary queries, with more lenient update latency requirements.

-Bertrand

Re: svn commit: r1617510 - /jackrabbit/oak/trunk/oak-jcr/src/test/java/org/apache/jackrabbit/oak/jcr/MiscTest.java

[Bertrand Delacretaz]

On Wed, Aug 13, 2014 at 11:20 AM, Davide Giannella dav...@apache.org wrote:
...
 +public class MiscTest extends AbstractJCRTest {
...

FWIW, after writing tons of tests for Sling I've moved away from using
inheritance - composition and utility classes is much more convenient
IMO.

-Bertrand

A few billion nodes in Oak, how does that sound?

[Bertrand Delacretaz]

Hi,

What kind of limitations, if any, do people see in growing an
Oak/Mongo repository to a few billion nodes?

IIRC people were doing tests with a few hundred million nodes in
Jackrabbit, so given Oak's scalable design I suppose that would work -
but do any obvious bottlenecks come to mind?

Also, do you have an estimate of the Oak/Mongo overhead in terms of
storage size, assuming tons of small nodes in the 10kb range?

-Bertrand

HBase backend, is the OAK-784 still valid?

[Bertrand Delacretaz]

Hi,

I recently talked to someone who might be interested in reviving the
OAK-784 HBase microkernel that was contributed in April 2013.

That's based on the MicroKernel interface, is that still how you'd do
it today? I've lost track of the various ways of creating storage
backends.

-Bertrand

Re: HBase backend, is the OAK-784 still valid?

[Bertrand Delacretaz]

Hi Marcel,

On Thu, Jul 3, 2014 at 10:12 AM, Marcel Reutegger mreut...@adobe.com wrote:
 ...it is still possible to hook in a MicroKernel implementation into Oak,
 but the two primary implementations we have use the NodeStore API...

Thanks! I have added a reference to this to OAK-784.

-Bertrand

Re: My repository is not indexing PDFs, what am I missing?

[Bertrand Delacretaz]

Hi Chetan,

On Thu, May 22, 2014 at 6:52 AM, Chetan Mehrotra
chetan.mehro...@gmail.com wrote:
 ...This might be due to OAK-1462. We had to disable the
 LuceneIndexProvider form getting registered as OSGi service...

Would that mean that the LuceneIndexEditor is still called, but the
result isn't used?

I'm asking because when adding a PDF, LuceneIndexEditor.addOrUpdate
does call context.getWriter().updateDocument with a Document that does
contain the PDF's full text in a field named :fulltext, so the text
extraction is working (thanks Alex for the tika-parsers hint).

But the query mentioned earlier in this thread still finds only .txt
documents, not .pdf.

Adding a .txt also causes LuceneIndexEditor.addOrUpdate to call
context.getWriter().updateDocument, but maybe the text is also indexed
in another way?

-Bertrand

My repository is not indexing PDFs, what am I missing?

[Bertrand Delacretaz]

Hi,

I'm upgrading the OakSlingRepositoryManager used for Sling tests to
Oak 1.0, and it's not indexing PDFs anymore - it used to with oak 0.8.

After uploading a text file to /tmp, the
/jcr:root/foo//*[jcr:contains(.,'some word')] query finds it, but the
same doesn't work with a PDF.

My repository setup is in the OakSlingRepositoryManager [1] - am I
missing something in there?

-Bertrand

[1] 
https://svn.apache.org/repos/asf/sling/trunk/bundles/jcr/oak-server/src/main/java/org/apache/sling/oak/server/OakSlingRepositoryManager.java

Re: Oak CI notifications not comming

[Bertrand Delacretaz]

Hi Chetan,

On Thu, Apr 24, 2014 at 12:52 PM, Chetan Mehrotra
chetan.mehro...@gmail.com wrote:
 ...I was checking the CI status for Oak trunk and it seems build are not
 getting built at [1] and [2]...

I don't know about travis but ci.apache.org is currently not working,
AFAIK there was a multiple hardware failure that takes time to fix.

-Bertrand

Re: [trivia] unit test class length

[Bertrand Delacretaz]

On Fri, Apr 4, 2014 at 3:35 PM, Davide Giannella
giannella.dav...@gmail.com wrote:
 ...it could be that I'm paranoid and test every single bit ;)...

I don't remember how many times I thought I was being paranoid with
tests, and later saw my paranoid tests expose very stupid bugs ;-)

-Bertrand

[jira] [Commented] (JCR-3743) failing test if aws extensions

[Bertrand Delacretaz (JIRA)]

[ 
https://issues.apache.org/jira/browse/JCR-3743?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13939070#comment-13939070
 ] 

Bertrand Delacretaz commented on JCR-3743:
--

[~julian.resc...@gmx.de] I'm curious, why 20ms specifically? 

I'm aware of the granularity problem, so in general terms the only sure things 
is that System.currentTimeMillis() increases from time to time...why 20ms?

 failing test if aws extensions
 --

 Key: JCR-3743
 URL: https://issues.apache.org/jira/browse/JCR-3743
 Project: Jackrabbit Content Repository
  Issue Type: Bug
Reporter: Julian Reschke
Assignee: Dominique Pfister
Priority: Minor

 On Win7/64bit/corei7:
 Failed tests:
   testDeleteAllOlderThan(org.apache.jackrabbit.aws.ext.ds.TestInMemDs)
   testDeleteAllOlderThan(org.apache.jackrabbit.aws.ext.ds.TestInMemDsCacheOff)
 Likely because of incorrect assumptions about System.currentTimeMillis()



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Re: oak-console tool

[Bertrand Delacretaz]

On Thu, Feb 20, 2014 at 5:13 PM, Alexander Klimetschek
aklim...@adobe.com wrote:
 ...The focus should still be on standalone runs - when the system is 
 running, there are many jcr browsers already available to do the same...

Yet, once you have a standalone tool it shouldn't be too hard to make
it optionally available in the webconsole as well.

-Bertrand

Re: oak-console tool

[Bertrand Delacretaz]

Hi,

On Mon, Feb 17, 2014 at 2:43 PM, Axel Hanikel ahani...@adobe.com wrote:
 ...The scala console works pretty well, too...

And there's also
https://github.com/chetanmeh/c/wiki/Sling-Script-Console -
consolidating that stuff might make sense at some point.

-Bertrand

Re: oak-console tool

[Bertrand Delacretaz]

Hi,

On Thu, Feb 20, 2014 at 4:44 PM, Davide Giannella
giannella.dav...@gmail.com wrote:
 On 20/02/2014 13:58, Bertrand Delacretaz wrote:
 And there's also
 https://github.com/chetanmeh/c/wiki/Sling-Script-Console
...
 ...The purpose of the one I created was
 for being able to run it at shutdown repository in order to allow
 investigations. It uses NodeStore vs JCR API. And we don't want to
 expose the NS at OSGi level...

Probably not, but if your tool can be made available in the webconsole
when running on Sling, that would make it more integrated than if it's
standalone.

-Bertrand

Re: System time resolution

[Bertrand Delacretaz]

Hi,

On Wed, Feb 19, 2014 at 9:38 AM, Julian Reschke julian.resc...@gmx.de
wrote:
 ...(I just checked and nanoTime is recommended only for calculating time
 differences...)

Yes, quoting [1] This method provides nanosecond precision, but not
necessarily nanosecond resolution (that is, how frequently the value
changes) - no guarantees are made except that the resolution is at least as
good as that of currentTimeMillis() - so nanoTime is not better
than currentTimeMillis for generating unique values.

-Bertrand

[1]
http://docs.oracle.com/javase/7/docs/api/java/lang/System.html#nanoTime()

Re: Workspaces, once more

[Bertrand Delacretaz]

Hi,

On Wed, Feb 19, 2014 at 10:09 AM, Jukka Zitting jukka.zitt...@gmail.com wrote:
 ...We won't support workspaces in the full JCR sense (shared jcr:system,
 cross-workspace operations, etc.). However, we do allow a repository
 to have more than one workspace, each workspace being it's own
 mini-repository with it's own user accounts, node types, version
 histories, etc...

Sounds good to me. Would that also allow for tuning various parts of
the content tree according to their usage, like for example saying
that /tmp and /var have no indexing, no versioning and no observation?
I haven't checked if other features of Oak allow that, but that would
be useful in large content repositories.

Each partition having its own user accounts is a bit worrying though,
won't that be an operations/admin problem? If you compare that with
unix filesystem partitions, all partitions share a common security
layer that's implemented on top of them.

-Bertrand

Re: Roadmap for Jackrabbit 2.x and 3.0

[Bertrand Delacretaz]

On Wed, Jan 15, 2014 at 7:35 PM, Jukka Zitting jukka.zitt...@gmail.com wrote:
 g) ...Or as a last resort, abandon the idea of a joint deployment
 package. Jackrabbit Classic and Oak would be shipped in separate
 deployment artifacts

Does this have impact on how people can migrate existing Jackrabbit
repositories to Oak, or is migration a separate concern and plan?

-Bertrand

Dynamic ACLs in Oak?

[Bertrand Delacretaz]

Hi,

Does Oak have an extension point where I can plugin my own dynamic ACL logic?

A typical use case is hiding a content subtree to some of the JCR
Sessions that are created, based on a decision done in my code at
session creation time, without having to change any actual ACLs.

To avoid security issues, such a dynamic ACL should only be able to
deny permissions on top of what Oak grants, but not grant any by
itself.

For now my goal is just to experiment with this, even if it's
inefficient or incomplete that would be useful.

-Bertrand

Re: Dynamic ACLs in Oak?

[Bertrand Delacretaz]

Hi,

On Mon, Dec 9, 2013 at 5:34 PM, Jukka Zitting jukka.zitt...@gmail.com wrote:
 ...Assuming a working JAAS setup, you can configure a custom optional
 LoginModule that adds extra principals to the current subject based on
 whatever criteria you want (source IP, HTTP header, phase of the moon,
 etc.)

Ok, thanks! phase of the moon, as in any arbitrary external value
is indeed the kind of use case we're looking at.

This looks like another reason to use the Felix Jaas stuff [1] which
is good as this will be useful for Sling as well.

-Bertrand

[1] http://felix.apache.org/documentation/subprojects/apache-felix-jaas.html

Re: Question about MVCC with MongoMK.

[Bertrand Delacretaz]

Hi,

On Fri, Dec 6, 2013 at 11:12 AM, Jukka Zitting jukka.zitt...@gmail.com wrote:
 ...The SegmentMK (with the MongoDB backend) can do this using the
 hierarchical journal feature The SegmentMK maintains one or more
 journals that each track the evolution of a particular branch of the
 repository. These branches would normally be automatically merged back
 to the root journal, but a particular deployment could easily
 disable automatic merging for a particular branch and use it for a
 purpose like the one you described

Is there a way to tell the repository to start operating on such a
branch forever for a given client?

IIUC Ian's scenario, an application instance would tell Oak create a
BEFORE_UPGRADE branch and start working on that from now on so that
the content can be upgraded in the background and tested on other
application instances, before eventually merging the BEFORE_UPGRADE
branch back.

Is that possible today, or reasonably simple to implement? If yes that
would enable such a scenario with minimal application changes, which
sounds extremely useful.

-Bertrand

Re: Question about MVCC with MongoMK.

[Bertrand Delacretaz]

Hi,

On Fri, Dec 6, 2013 at 11:36 AM, Jukka Zitting jukka.zitt...@gmail.com wrote:
 ...The main question here is whether we want to go down that path, as the
 feature is only available with the SegmentMK (at least for now) and
 we've generally wanted to avoid exposing such implementation-specific
 features to higher level code...

Agreed, OTOH the scenario that we're discussing here looks extremely
useful in clustered environments, where managing upgrades and
minimizing downtime is hard. I suspect Ian will agree that having this
in Oak would be very valuable, even if that requires using a specific
microkernel.

-Bertrand

Re: writing a new DocumentStore implementation

[Bertrand Delacretaz]

On Mon, Dec 2, 2013 at 3:42 PM, Marcel Reutegger mreut...@adobe.com wrote:
 ...it may make sense to rename it to something more generic, since
 the majority of code is agnostic of the underlying storage

I'm curious now, is DocumentStore yet another way of creating a custom
storage backend for Oak, besides NodeStore and MicroKernel?

-Bertrand

[jira] [Created] (JCR-3692) MoveAtRootTest fails and is not included in test suite

[Bertrand Delacretaz (JIRA)]

Bertrand Delacretaz created JCR-3692:


 Summary: MoveAtRootTest fails and is not included in test suite
 Key: JCR-3692
 URL: https://issues.apache.org/jira/browse/JCR-3692
 Project: Jackrabbit Content Repository
  Issue Type: Bug
  Components: jackrabbit-core
Reporter: Bertrand Delacretaz
Priority: Minor


The MoveAtRootTest introduced by JCR-2680 fails when executed against the 
current jackrabbit-core trunk (mvn clean test -Dtest=MoveAtRootTest), with 
javax.jcr.RepositoryException: Attempt to remove/move the admin user.

The operation that fails is Session.move(/MoveAtRootTest_A, 
/MoveAtRootTest_B)

AFAICS this is caused by the JCR-3686 changes. The same test passes on the 
http://svn.apache.org/repos/asf/jackrabbit/tags/2.6.4 revision.

I'll attach a patch that includes the test in the core test suite. If there's a 
good reason to forbid such a move, it should be documented and the test changed 
to reflect the expected behavior.



--
This message was sent by Atlassian JIRA
(v6.1#6144)

[jira] [Updated] (JCR-3692) MoveAtRootTest fails and is not included in test suite

[Bertrand Delacretaz (JIRA)]

 [ 
https://issues.apache.org/jira/browse/JCR-3692?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Bertrand Delacretaz updated JCR-3692:
-

Attachment: activate-moveatroot.patch

 MoveAtRootTest fails and is not included in test suite
 --

 Key: JCR-3692
 URL: https://issues.apache.org/jira/browse/JCR-3692
 Project: Jackrabbit Content Repository
  Issue Type: Bug
  Components: jackrabbit-core
Reporter: Bertrand Delacretaz
Assignee: angela
Priority: Minor
 Attachments: activate-moveatroot.patch


 The MoveAtRootTest introduced by JCR-2680 fails when executed against the 
 current jackrabbit-core trunk (mvn clean test -Dtest=MoveAtRootTest), with 
 javax.jcr.RepositoryException: Attempt to remove/move the admin user.
 The operation that fails is Session.move(/MoveAtRootTest_A, 
 /MoveAtRootTest_B)
 AFAICS this is caused by the JCR-3686 changes. The same test passes on the 
 http://svn.apache.org/repos/asf/jackrabbit/tags/2.6.4 revision.
 I'll attach a patch that includes the test in the core test suite. If there's 
 a good reason to forbid such a move, it should be documented and the test 
 changed to reflect the expected behavior.



--
This message was sent by Atlassian JIRA
(v6.1#6144)

[jira] [Commented] (JCR-3692) MoveAtRootTest fails and is not included in test suite

[Bertrand Delacretaz (JIRA)]

[ 
https://issues.apache.org/jira/browse/JCR-3692?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13819067#comment-13819067
 ] 

Bertrand Delacretaz commented on JCR-3692:
--

FWIW, I forgot to mention explicitly in this issue's description that 
MoveAtRootTest is currently not executed in builds, and was not for 2.6.4 
either, as it's not included in o.a.j.core/TestAll.java - the attached patch 
fixes this.

 MoveAtRootTest fails and is not included in test suite
 --

 Key: JCR-3692
 URL: https://issues.apache.org/jira/browse/JCR-3692
 Project: Jackrabbit Content Repository
  Issue Type: Bug
  Components: jackrabbit-core
Reporter: Bertrand Delacretaz
Assignee: angela
Priority: Minor
 Attachments: activate-moveatroot.patch


 The MoveAtRootTest introduced by JCR-2680 fails when executed against the 
 current jackrabbit-core trunk (mvn clean test -Dtest=MoveAtRootTest), with 
 javax.jcr.RepositoryException: Attempt to remove/move the admin user.
 The operation that fails is Session.move(/MoveAtRootTest_A, 
 /MoveAtRootTest_B)
 AFAICS this is caused by the JCR-3686 changes. The same test passes on the 
 http://svn.apache.org/repos/asf/jackrabbit/tags/2.6.4 revision.
 I'll attach a patch that includes the test in the core test suite. If there's 
 a good reason to forbid such a move, it should be documented and the test 
 changed to reflect the expected behavior.



--
This message was sent by Atlassian JIRA
(v6.1#6144)

[jira] [Commented] (JCR-3692) MoveAtRootTest fails and is not included in test suite

[Bertrand Delacretaz (JIRA)]

[ 
https://issues.apache.org/jira/browse/JCR-3692?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13819086#comment-13819086
 ] 

Bertrand Delacretaz commented on JCR-3692:
--

yes, just wanted to make sure that was clear ;-)

 MoveAtRootTest fails and is not included in test suite
 --

 Key: JCR-3692
 URL: https://issues.apache.org/jira/browse/JCR-3692
 Project: Jackrabbit Content Repository
  Issue Type: Bug
  Components: jackrabbit-core
Reporter: Bertrand Delacretaz
Assignee: angela
Priority: Minor
 Attachments: activate-moveatroot.patch


 The MoveAtRootTest introduced by JCR-2680 fails when executed against the 
 current jackrabbit-core trunk (mvn clean test -Dtest=MoveAtRootTest), with 
 javax.jcr.RepositoryException: Attempt to remove/move the admin user.
 The operation that fails is Session.move(/MoveAtRootTest_A, 
 /MoveAtRootTest_B)
 AFAICS this is caused by the JCR-3686 changes. The same test passes on the 
 http://svn.apache.org/repos/asf/jackrabbit/tags/2.6.4 revision.
 I'll attach a patch that includes the test in the core test suite. If there's 
 a good reason to forbid such a move, it should be documented and the test 
 changed to reflect the expected behavior.



--
This message was sent by Atlassian JIRA
(v6.1#6144)

Re: writing a new MK, guidelines

[Bertrand Delacretaz]

Hi Tommaso,

On Wed, Nov 6, 2013 at 11:31 AM, Tommaso Teofili
tommaso.teof...@gmail.com wrote:
 ...out of curiosity, I was wondering if there's any guideline other than
 following the API on writing a new MK implementation

FWIW, the OAK-784 HBase microkernel code might provide some useful
pointers, but note that it was created early this year, so some things
might have changed.

-Bertrand

Re: Can I use LoginModulePlugins in Oak?

[Bertrand Delacretaz]

Hi Angela,

On Mon, Nov 4, 2013 at 9:53 AM, Angela Schreiber anch...@adobe.com wrote:
 ...if i was you i would just deprecate that unfortunate part of sling
 that intended to make the login module pluggable at runtime and
 drop it asap

Ok, but how do you suggest implementing the use case that's covered by
Sling's current FormLoginModulePlugin?

http://jackrabbit.apache.org/jackrabbit-configuration.html says if
JAAS authentication is not available or (as is often the case) too
complex to set up, Jackrabbit allows you to specify a
repository-specific JAAS LoginModule that is then used for
authenticating repository users.

I'm basically looking for the equivalent in Oak, something that a
Sling instance can setup by default, without requiring users to do any
configuration.

-Bertrand

Can I use LoginModulePlugins in Oak?

[Bertrand Delacretaz]

Hi,

Some Sling integration tests (SLING-3221) are failing, due to a form
login mechanism that's supported by Sling's FormLoginModulePlugin [1]
when running on Jackrabbit, and is not present on our Oak setup.

Can I use plugins that implement
org.apache.sling.jcr.jackrabbit.server.security.LoginModulePlugin in
Oak (and how), or is there a similar mechanism?

-Bertrand

[1] 
https://svn.apache.org/repos/asf/sling/trunk/bundles/auth/form/src/main/java/org/apache/sling/auth/form/impl/FormLoginModulePlugin.java

Re: Oak JCR Observation scalability aspects and concerns

[Bertrand Delacretaz]

Hi,

On Fri, Oct 25, 2013 at 8:53 AM, Angela Schreiber anch...@adobe.com wrote:
 ...the biggest challenge i see in terms of backwards compatibility is
 that the diff-mechanism in OAK doesn't allow a 1:1 translation
 to JCR events as they used to be generated in jackrabbit-core...

The OSGi events that Sling rebroadcasts are less granular than JCR
events, so this might not be a problem for that case.

If at some point we can say low-level JCR events behave slightly
differently between Oak and Jackrabbit, but the higher-level OSGi
events that Sling provides are the same that might be a realistic
compromise.

 ...we keep getting mixed signals
 (be fully compatible vs just make it scalable and fast) and need
 get a more reliable feedback to in...

So maybe being fully compatible (*) at the Sling level, and scalable
and fast at the JCR level, is a good way forward.

-Bertrand

(*) might require some Sling client code to indicate more precisely
which events it wants to listen to, but that's a reasonable price to
pay IMO

Re: Oak JCR Observation scalability aspects and concerns

[Bertrand Delacretaz]

Hi Angela,

On Fri, Oct 25, 2013 at 11:21 AM, Angela Schreiber anch...@adobe.com wrote:
 Bertrand wrote:
The OSGi events that Sling rebroadcasts are less granular than JCR
events, so this might not be a problem for that case.

 do you know it or are you guessing?...

I'm making an educated guess.

To get hard facts, as you rightly ask for, we need a test suite that
compares those OSGi events that Sling rebroadcasts, when doing various
things on Jackrabbit and Oak. I don't think we'll ever get 100%
identical observation behavior between Jackrabbit and Oak, so we'll
need to define which differences are acceptable. To me the only sane
way to do that is via a test suite.

This can be added to the Sling it-jackrabbit-oak module [1] which
makes it easy to run the exact same tests against Jackrabbit and Oak
in an OSGi environment, but someone will need to write those tests. We
might want to grant write access to both Sling + Oak committers to
that module to make things easier.

Thanks for the examples that you mention. I see the referenceable node
issue at http://jackrabbit.apache.org/oak/docs/differences.html, but
not the setPrimaryType one, could it be added?

Ciao,
-Bertrand

[1] http://svn.apache.org/repos/asf/sling/trunk/bundles/jcr/it-jackrabbit-oak/

Re: Oak JCR Observation scalability aspects and concerns

[Bertrand Delacretaz]

On Tue, Oct 22, 2013 at 4:19 PM, Thomas Mueller muel...@adobe.com wrote:
This is the contract we have to maintain in Sling.
 Well, we can't maintain this contract, because it blocks scalability

I tend to agree, and OTOH not everybody will need the kind of
scalability that we're discussing here.

Many of the usage patterns described at [1] can be solved without
observation, in more scalable ways.

Providing a way for users who actually need high scalability to move
away from the catch all events patterns, using Jukka's whiteboard
suggestion for example, sounds good to me. That might require work
from such users, but at some point we have to admit that there's no
magic.

-Bertrand

[1] 
https://cwiki.apache.org/confluence/display/SLING/Observation+usage+patterns#