[jira] [Updated] (OAK-75) specify format and semantics of 'filter' parameter in MicroKernel API
[
https://issues.apache.org/jira/browse/OAK-75?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Stefan Guggisberg updated OAK-75:
-
Attachment: OAK-83.patch
proposed MicroKernel API change/clarification
specify format and semantics of 'filter' parameter in MicroKernel API
-
Key: OAK-75
URL: https://issues.apache.org/jira/browse/OAK-75
Project: Jackrabbit Oak
Issue Type: New Feature
Components: mk
Reporter: Stefan Guggisberg
Attachments: OAK-83.patch
the following MicroKernel methods contain a 'filter' string parameter:
- getJournal
- diff
- getNodes
through the filter an API client could e.g. specify:
- special 'meta' properties to be included (e.g. :hash)
- glob patterns on the names of properties/child nodes to be included/excluded
- path filter (for getJournal and diff)
format/detailed semantics TBD, here's an initial proposal (json):
{code}
{
path : /some/path,
incl : [ :hash, * ],
excl : [ tmp* ]
}
{code}
name filter patterns should ideally be the same
format as specified for JCR Node.getNodes/getProperties.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (OAK-75) specify format and semantics of 'filter' parameter in MicroKernel API
[
https://issues.apache.org/jira/browse/OAK-75?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13268385#comment-13268385
]
Stefan Guggisberg commented on OAK-75:
--
Would that also be the case if we only add the (path-) filter at
getRevisionHistory, and keep waitForCommit and getHeadRevision as they are
now?
i've attached a new proposal (see OAK-83.patch):
- changed the 'filter' parameter on getJournal and diff to 'path'
- added a 'path' parameter to getRevisionHistory
- getNodes is the only method providing a 'filter' parameter
open questions:
- globbing syntax for filter needs to be specified. '*' is legal name
character...
- should we allow for node name filtering as well?
- if yes, do we need to specify separate filters for properties and node names?
- the implicit default filter needs to specified (e.g. { incl: [ *,
:childNodeCount ], excl: [ :hash ] }
specify format and semantics of 'filter' parameter in MicroKernel API
-
Key: OAK-75
URL: https://issues.apache.org/jira/browse/OAK-75
Project: Jackrabbit Oak
Issue Type: New Feature
Components: mk
Reporter: Stefan Guggisberg
Attachments: OAK-83.patch
the following MicroKernel methods contain a 'filter' string parameter:
- getJournal
- diff
- getNodes
through the filter an API client could e.g. specify:
- special 'meta' properties to be included (e.g. :hash)
- glob patterns on the names of properties/child nodes to be included/excluded
- path filter (for getJournal and diff)
format/detailed semantics TBD, here's an initial proposal (json):
{code}
{
path : /some/path,
incl : [ :hash, * ],
excl : [ tmp* ]
}
{code}
name filter patterns should ideally be the same
format as specified for JCR Node.getNodes/getProperties.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Issue Comment Edited] (OAK-75) specify format and semantics of 'filter' parameter in MicroKernel API
[
https://issues.apache.org/jira/browse/OAK-75?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13268385#comment-13268385
]
Stefan Guggisberg edited comment on OAK-75 at 5/4/12 2:05 PM:
--
Would that also be the case if we only add the (path-) filter at
getRevisionHistory, and keep waitForCommit and getHeadRevision as they are
now?
i've attached a new proposal (see OAK-75.patch):
- changed the 'filter' parameter on getJournal and diff to 'path'
- added a 'path' parameter to getRevisionHistory
- getNodes is the only method providing a 'filter' parameter
open questions:
- globbing syntax for filter needs to be specified. '*' is a legal name
character...
- should we allow for node name filtering as well?
- if yes, do we need to specify separate filters for properties and node names?
- the implicit default filter needs to specified (e.g. { incl: [ *,
:childNodeCount ], excl: [ :hash ] }
was (Author: stefan@jira):
Would that also be the case if we only add the (path-) filter at
getRevisionHistory, and keep waitForCommit and getHeadRevision as they are now?
i've attached a new proposal (see OAK-83.patch):
- changed the 'filter' parameter on getJournal and diff to 'path'
- added a 'path' parameter to getRevisionHistory
- getNodes is the only method providing a 'filter' parameter
open questions:
- globbing syntax for filter needs to be specified. '*' is legal name
character...
- should we allow for node name filtering as well?
- if yes, do we need to specify separate filters for properties and node names?
- the implicit default filter needs to specified (e.g. { incl: [ *,
:childNodeCount ], excl: [ :hash ] }
specify format and semantics of 'filter' parameter in MicroKernel API
-
Key: OAK-75
URL: https://issues.apache.org/jira/browse/OAK-75
Project: Jackrabbit Oak
Issue Type: New Feature
Components: mk
Reporter: Stefan Guggisberg
Attachments: OAK-83.patch
the following MicroKernel methods contain a 'filter' string parameter:
- getJournal
- diff
- getNodes
through the filter an API client could e.g. specify:
- special 'meta' properties to be included (e.g. :hash)
- glob patterns on the names of properties/child nodes to be included/excluded
- path filter (for getJournal and diff)
format/detailed semantics TBD, here's an initial proposal (json):
{code}
{
path : /some/path,
incl : [ :hash, * ],
excl : [ tmp* ]
}
{code}
name filter patterns should ideally be the same
format as specified for JCR Node.getNodes/getProperties.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (OAK-85) NPE and wrong result on copy operation
[
https://issues.apache.org/jira/browse/OAK-85?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Stefan Guggisberg reassigned OAK-85:
Assignee: Stefan Guggisberg
NPE and wrong result on copy operation
--
Key: OAK-85
URL: https://issues.apache.org/jira/browse/OAK-85
Project: Jackrabbit Oak
Issue Type: Bug
Components: mk
Affects Versions: 0.1
Reporter: Michael Dürig
Assignee: Stefan Guggisberg
{code}
mk.commit(, +\/root\:{}, null, null);
mk.commit(, +\/root/N4\:{} *\/root/N4\:\/root/N4/N5\, null, null);
{code}
results in a NPE. Doing all operations separately instead gives the wrong
result:
{code}
mk.commit(, +\/root\:{}, null, null);
mk.commit(, +\/root/N4\:{}, null, null);
mk.commit(, *\/root/N4\:\/root/N4/N5\, null, null);
{code}
results in (relative to /root)
{code}
{
:childNodeCount: 1,
N4: {
:childNodeCount: 1,
N5: {
:childNodeCount: 1,
N5: {
:childNodeCount: 0
}
}
}
}
{code}
instead of
{code}
{
:childNodeCount: 1,
N4: {
:childNodeCount: 0,
N5: {
:childNodeCount: 0
}
}
}
}
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (OAK-85) NPE and wrong result on copy operation
[
https://issues.apache.org/jira/browse/OAK-85?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Stefan Guggisberg resolved OAK-85.
--
Resolution: Fixed
Fix Version/s: 0.3
fixed in svn r1334040
NPE and wrong result on copy operation
--
Key: OAK-85
URL: https://issues.apache.org/jira/browse/OAK-85
Project: Jackrabbit Oak
Issue Type: Bug
Components: mk
Affects Versions: 0.1
Reporter: Michael Dürig
Assignee: Stefan Guggisberg
Fix For: 0.3
{code}
mk.commit(, +\/root\:{}, null, null);
mk.commit(, +\/root/N4\:{} *\/root/N4\:\/root/N4/N5\, null, null);
{code}
results in a NPE. Doing all operations separately instead gives the wrong
result:
{code}
mk.commit(, +\/root\:{}, null, null);
mk.commit(, +\/root/N4\:{}, null, null);
mk.commit(, *\/root/N4\:\/root/N4/N5\, null, null);
{code}
results in (relative to /root)
{code}
{
:childNodeCount: 1,
N4: {
:childNodeCount: 1,
N5: {
:childNodeCount: 1,
N5: {
:childNodeCount: 0
}
}
}
}
{code}
instead of
{code}
{
:childNodeCount: 1,
N4: {
:childNodeCount: 0,
N5: {
:childNodeCount: 0
}
}
}
}
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (OAK-75) specify format and semantics of 'filter' parameter in MicroKernel API
[
https://issues.apache.org/jira/browse/OAK-75?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13267469#comment-13267469
]
Stefan Guggisberg commented on OAK-75:
--
For the one getNodes method, the path is a parameter and could be set in the
filter as well. Could that lead to confusion?
my intention was to specify the 'path' filter on getJournal() and diff() only.
I noticed getJournal supports a filter, but the related getRevisionHistory
not yet. Would it make sense to add the filter to getRevisionHistory as well?
the original idea was to support 'filtering' the 'tree data' (either json or
diff) returned by getJournal, diff and getNodes. since getRevisionHistory
doesn't return tree data i didn't add a filter parameter.
If we support path filtering in getJournal (I hope we do), should we support
it in waitForCommit and getHeadRevision as well?
same rationale as for not supporting filters on getRevisionHistory.
but i see your point. i didn't consider supporting path-based filtering on
waitForCommit and getHeadRevision. i wanted to keep the proposal simple and i
am afraid that it will add considerable complexity to the API semantics.
OTOH if there's a real use case in oak-core for such a functionality which
justifies the added complexity of the MK API i am open to extending the current
API.
specify format and semantics of 'filter' parameter in MicroKernel API
-
Key: OAK-75
URL: https://issues.apache.org/jira/browse/OAK-75
Project: Jackrabbit Oak
Issue Type: New Feature
Components: mk
Reporter: Stefan Guggisberg
the following MicroKernel methods contain a 'filter' string parameter:
- getJournal
- diff
- getNodes
through the filter an API client could e.g. specify:
- special 'meta' properties to be included (e.g. :hash)
- glob patterns on the names of properties/child nodes to be included/excluded
- path filter (for getJournal and diff)
format/detailed semantics TBD, here's an initial proposal (json):
{code}
{
path : /some/path,
incl : [ :hash, * ],
excl : [ tmp* ]
}
{code}
name filter patterns should ideally be the same
format as specified for JCR Node.getNodes/getProperties.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (OAK-83) Copy operation would recurse indefinitely if memory permitted
[
https://issues.apache.org/jira/browse/OAK-83?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Stefan Guggisberg reassigned OAK-83:
Assignee: Stefan Guggisberg
Copy operation would recurse indefinitely if memory permitted
-
Key: OAK-83
URL: https://issues.apache.org/jira/browse/OAK-83
Project: Jackrabbit Oak
Issue Type: Bug
Components: mk
Affects Versions: 0.1
Reporter: Michael Dürig
Assignee: Stefan Guggisberg
{code}
microKernel.commit(, *\/a\:\/a/b\, null, null);
{code}
causes
{code}
java.lang.OutOfMemoryError: Java heap space
at java.util.Arrays.copyOfRange(Arrays.java:3209)
at java.lang.String.init(String.java:215)
at java.lang.StringBuilder.toString(StringBuilder.java:430)
at
org.apache.jackrabbit.oak.commons.PathUtils.concat(PathUtils.java:320)
at
org.apache.jackrabbit.mk.model.CommitBuilder.copyStagedNodes(CommitBuilder.java:293)
at
org.apache.jackrabbit.mk.model.CommitBuilder.copyStagedNodes(CommitBuilder.java:293)
at
org.apache.jackrabbit.mk.model.CommitBuilder.copyStagedNodes(CommitBuilder.java:293)
at
org.apache.jackrabbit.mk.model.CommitBuilder.copyStagedNodes(CommitBuilder.java:293)
at
org.apache.jackrabbit.mk.model.CommitBuilder.copyStagedNodes(CommitBuilder.java:293)
at
org.apache.jackrabbit.mk.model.CommitBuilder.copyStagedNodes(CommitBuilder.java:293)
at
org.apache.jackrabbit.mk.model.CommitBuilder.copyStagedNodes(CommitBuilder.java:293)
at
org.apache.jackrabbit.mk.model.CommitBuilder.copyStagedNodes(CommitBuilder.java:293)
at
org.apache.jackrabbit.mk.model.CommitBuilder.copyStagedNodes(CommitBuilder.java:293)
at
org.apache.jackrabbit.mk.model.CommitBuilder.copyStagedNodes(CommitBuilder.java:293)
at
org.apache.jackrabbit.mk.model.CommitBuilder.copyStagedNodes(CommitBuilder.java:293)
at
org.apache.jackrabbit.mk.model.CommitBuilder.copyStagedNodes(CommitBuilder.java:293)
at
org.apache.jackrabbit.mk.model.CommitBuilder.copyStagedNodes(CommitBuilder.java:293)
at
org.apache.jackrabbit.mk.model.CommitBuilder.copyStagedNodes(CommitBuilder.java:293)
at
org.apache.jackrabbit.mk.model.CommitBuilder.copyStagedNodes(CommitBuilder.java:293)
at
org.apache.jackrabbit.mk.model.CommitBuilder.copyStagedNodes(CommitBuilder.java:293)
at
org.apache.jackrabbit.mk.model.CommitBuilder.copyStagedNodes(CommitBuilder.java:293)
at
org.apache.jackrabbit.mk.model.CommitBuilder.copyStagedNodes(CommitBuilder.java:293)
at
org.apache.jackrabbit.mk.model.CommitBuilder.copyStagedNodes(CommitBuilder.java:293)
at
org.apache.jackrabbit.mk.model.CommitBuilder.copyStagedNodes(CommitBuilder.java:293)
at
org.apache.jackrabbit.mk.model.CommitBuilder.copyStagedNodes(CommitBuilder.java:293)
at
org.apache.jackrabbit.mk.model.CommitBuilder.copyStagedNodes(CommitBuilder.java:293)
at
org.apache.jackrabbit.mk.model.CommitBuilder.copyStagedNodes(CommitBuilder.java:293)
at
org.apache.jackrabbit.mk.model.CommitBuilder.copyStagedNodes(CommitBuilder.java:293)
at
org.apache.jackrabbit.mk.model.CommitBuilder.copyStagedNodes(CommitBuilder.java:293)
at
org.apache.jackrabbit.mk.model.CommitBuilder.copyStagedNodes(CommitBuilder.java:293)
at
org.apache.jackrabbit.mk.model.CommitBuilder.copyStagedNodes(CommitBuilder.java:293)
at
org.apache.jackrabbit.mk.model.CommitBuilder.copyStagedNodes(CommitBuilder.java:293)
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (OAK-80) Implement batched writing for KernelNodeStore
[ https://issues.apache.org/jira/browse/OAK-80?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13266538#comment-13266538 ] Stefan Guggisberg commented on OAK-80: -- FWIW: Personally I'd rather see such batching happening under the MicroKernel API where the implementation actually knows about things like whether network roundtrips are involved and what kind of batch sizes are most useful. i imagine that the implementation in the MK would be quite complicated (due to the MVCC contract) whereas supporting batched writes in the upper layer is probably not that difficult. Implement batched writing for KernelNodeStore - Key: OAK-80 URL: https://issues.apache.org/jira/browse/OAK-80 Project: Jackrabbit Oak Issue Type: Improvement Components: core Affects Versions: 0.1 Reporter: Michael Dürig Assignee: Michael Dürig Currently KernelNodeStore and KernelNodeStateBuilder directly apply every operation on the content tree to the private branch of the Microkernel. There have been some concerns re. performance hits due to network latency in the case where the Microkernel is not co-located. I suggest to add batching capabilities such that operations are only written through to the Microkernel on certain limits. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (OAK-79) Copy operation misses some child nodes
[
https://issues.apache.org/jira/browse/OAK-79?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Stefan Guggisberg reassigned OAK-79:
Assignee: Stefan Guggisberg
Copy operation misses some child nodes
--
Key: OAK-79
URL: https://issues.apache.org/jira/browse/OAK-79
Project: Jackrabbit Oak
Issue Type: Bug
Components: mk
Affects Versions: 0.1
Reporter: Michael Dürig
Assignee: Stefan Guggisberg
{code}
microKernel.commit(/, +\y\:{}+\x\:{}, null, null);
microKernel.commit(/, +\x/miss-me\:{} *\x\:\y/copy-of-x\, null,
null);
{code}
results in
{code}
{
:childNodeCount: 2,
y: {
:childNodeCount: 1,
copy-of-x: {
:childNodeCount: 0
}
},
x: {
:childNodeCount: 1,
miss-me: {
:childNodeCount: 0
}
}
}
{code}
instead of
{code}
{
:childNodeCount: 2,
y: {
:childNodeCount: 1,
copy-of-x: {
:childNodeCount: 1,
miss-me: {
:childNodeCount: 0
}
}
},
x: {
:childNodeCount: 1,
miss-me: {
:childNodeCount: 0
}
}
}
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (OAK-75) specify format and semantics of 'filter' parameter in MicroKernel API
Stefan Guggisberg created OAK-75:
Summary: specify format and semantics of 'filter' parameter in
MicroKernel API
Key: OAK-75
URL: https://issues.apache.org/jira/browse/OAK-75
Project: Jackrabbit Oak
Issue Type: New Feature
Components: mk
Reporter: Stefan Guggisberg
the following MicroKernel methods contain a 'filter' string parameter:
- getJournal
- diff
- getNodes
through the filter an API client could e.g. specify:
- special 'meta' properties to be included (e.g. :hash)
- glob patterns on the names of properties/child nodes to be included/excluded
- path filter (for getJournal and diff)
format/detailed semantics TBD, here's an initial proposal (json):
{
path : /some/path,
incl : [ :hash, * ],
excl : [ tmp* ]
}
name filter patterns should ideally be the same
format as specified for JCR Node.getNodes/getProperties.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (OAK-75) specify format and semantics of 'filter' parameter in MicroKernel API
[
https://issues.apache.org/jira/browse/OAK-75?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Stefan Guggisberg updated OAK-75:
-
Description:
the following MicroKernel methods contain a 'filter' string parameter:
- getJournal
- diff
- getNodes
through the filter an API client could e.g. specify:
- special 'meta' properties to be included (e.g. :hash)
- glob patterns on the names of properties/child nodes to be included/excluded
- path filter (for getJournal and diff)
format/detailed semantics TBD, here's an initial proposal (json):
{code}
{
path : /some/path,
incl : [ :hash, * ],
excl : [ tmp* ]
}
{code}
name filter patterns should ideally be the same
format as specified for JCR Node.getNodes/getProperties.
was:
the following MicroKernel methods contain a 'filter' string parameter:
- getJournal
- diff
- getNodes
through the filter an API client could e.g. specify:
- special 'meta' properties to be included (e.g. :hash)
- glob patterns on the names of properties/child nodes to be included/excluded
- path filter (for getJournal and diff)
format/detailed semantics TBD, here's an initial proposal (json):
pre
{
path : /some/path,
incl : [ :hash, * ],
excl : [ tmp* ]
}
/pre
name filter patterns should ideally be the same
format as specified for JCR Node.getNodes/getProperties.
specify format and semantics of 'filter' parameter in MicroKernel API
-
Key: OAK-75
URL: https://issues.apache.org/jira/browse/OAK-75
Project: Jackrabbit Oak
Issue Type: New Feature
Components: mk
Reporter: Stefan Guggisberg
the following MicroKernel methods contain a 'filter' string parameter:
- getJournal
- diff
- getNodes
through the filter an API client could e.g. specify:
- special 'meta' properties to be included (e.g. :hash)
- glob patterns on the names of properties/child nodes to be included/excluded
- path filter (for getJournal and diff)
format/detailed semantics TBD, here's an initial proposal (json):
{code}
{
path : /some/path,
incl : [ :hash, * ],
excl : [ tmp* ]
}
{code}
name filter patterns should ideally be the same
format as specified for JCR Node.getNodes/getProperties.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (OAK-45) Add support for branching and merging of private copies to MicroKernel
[
https://issues.apache.org/jira/browse/OAK-45?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13262564#comment-13262564
]
Stefan Guggisberg commented on OAK-45:
--
the OOME is not specific to the branch/merge support;
the tests are using the in-mem mk; to avoid the OOME
you can increase the JVM heap size or reduce the number
of commits.
Add support for branching and merging of private copies to MicroKernel
--
Key: OAK-45
URL: https://issues.apache.org/jira/browse/OAK-45
Project: Jackrabbit Oak
Issue Type: New Feature
Components: mk
Reporter: Michael Dürig
Attachments: OAK-45__OOME.patch
As discussed on the dev list [1] we should add support to the Microkernel for
branching of a private working copy which can be merged back later:
{code}
String addLotsOfData(MicroKernel mk) {
String baseRevision = mk.getHeadRevision();
String branchRevision = mk.branch(baseRevision);
for (int i = 0; i 100; i++) {
branchRevision = mk.commit(/, +\node + i + \:{},
branchRevision, null);
}
return mk.merge(branchRevision, baseRevision); }
{code}
[1] http://markmail.org/message/jbbut6vzvmmjqonr
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (OAK-45) Add support for branching and merging of private copies to MicroKernel
[
https://issues.apache.org/jira/browse/OAK-45?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13261780#comment-13261780
]
Stefan Guggisberg commented on OAK-45:
--
thanks, will investigate
Add support for branching and merging of private copies to MicroKernel
--
Key: OAK-45
URL: https://issues.apache.org/jira/browse/OAK-45
Project: Jackrabbit Oak
Issue Type: New Feature
Components: mk
Reporter: Michael Dürig
Attachments: OAK-45__OOME.patch
As discussed on the dev list [1] we should add support to the Microkernel for
branching of a private working copy which can be merged back later:
{code}
String addLotsOfData(MicroKernel mk) {
String baseRevision = mk.getHeadRevision();
String branchRevision = mk.branch(baseRevision);
for (int i = 0; i 100; i++) {
branchRevision = mk.commit(/, +\node + i + \:{},
branchRevision, null);
}
return mk.merge(branchRevision, baseRevision); }
{code}
[1] http://markmail.org/message/jbbut6vzvmmjqonr
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (OAK-45) Add support for branching and merging of private copies to MicroKernel
[
https://issues.apache.org/jira/browse/OAK-45?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13258308#comment-13258308
]
Stefan Guggisberg commented on OAK-45:
--
implemented initial support for branch merge in oak-mk incl. a basic
integration test in svn r1328397.
the integration test currently fails with UnsupportedOperationException when
using the non-default mk impl
in oak-core.
Add support for branching and merging of private copies to MicroKernel
--
Key: OAK-45
URL: https://issues.apache.org/jira/browse/OAK-45
Project: Jackrabbit Oak
Issue Type: New Feature
Components: mk
Reporter: Michael Dürig
Assignee: Stefan Guggisberg
As discussed on the dev list [1] we should add support to the Microkernel for
branching of a private working copy which can be merged back later:
{code}
String addLotsOfData(MicroKernel mk) {
String baseRevision = mk.getHeadRevision();
String branchRevision = mk.branch(baseRevision);
for (int i = 0; i 100; i++) {
branchRevision = mk.commit(/, +\node + i + \:{},
branchRevision, null);
}
return mk.merge(branchRevision, baseRevision); }
{code}
[1] http://markmail.org/message/jbbut6vzvmmjqonr
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (OAK-46) Efficient diffing of large child node lists
[ https://issues.apache.org/jira/browse/OAK-46?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stefan Guggisberg resolved OAK-46. -- Resolution: Fixed logic restored in svn r1325697 Efficient diffing of large child node lists --- Key: OAK-46 URL: https://issues.apache.org/jira/browse/OAK-46 Project: Jackrabbit Oak Issue Type: Improvement Components: mk Affects Versions: 0.1 Reporter: Jukka Zitting Assignee: Jukka Zitting Labels: performance Fix For: 0.2 As noted on oak-dev@ revision 1298366 broke the earlier ability to efficiently compare node states with large lists of child nodes. We should restore that feature to avoid introducing a major performance bottleneck for such cases. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (OAK-48) MicroKernel.getNodes() should return null for not existing nodes instead of throwing an exception
[ https://issues.apache.org/jira/browse/OAK-48?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stefan Guggisberg reassigned OAK-48: Assignee: Stefan Guggisberg MicroKernel.getNodes() should return null for not existing nodes instead of throwing an exception - Key: OAK-48 URL: https://issues.apache.org/jira/browse/OAK-48 Project: Jackrabbit Oak Issue Type: Improvement Components: mk Reporter: Michael Dürig Assignee: Stefan Guggisberg Attachments: OAK-48.patch As discussed on the list [1, 2] exceptions should only be used for exceptional cases. Requesting a not existing node through Microkernel.getNodes() should therefore rather return null instead of throwing an exception. [1] http://markmail.org/thread/agibgcdjv756m53u [2] http://markmail.org/message/gfbmogwr6mrhe2pm -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (OAK-48) MicroKernel.getNodes() should return null for not existing nodes instead of throwing an exception
[ https://issues.apache.org/jira/browse/OAK-48?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13253723#comment-13253723 ] Stefan Guggisberg commented on OAK-48: -- Also, it seems there is not test case yet. yes, there is: http://svn.apache.org/viewvc/jackrabbit/oak/trunk/oak-it/mk/src/main/java/org/apache/jackrabbit/mk/test/MicroKernelIT.java?r1=1325827r2=1325826view=diffpathrev=1325827 MicroKernel.getNodes() should return null for not existing nodes instead of throwing an exception - Key: OAK-48 URL: https://issues.apache.org/jira/browse/OAK-48 Project: Jackrabbit Oak Issue Type: Improvement Components: mk Reporter: Michael Dürig Assignee: Thomas Mueller Fix For: 0.2 Attachments: OAK-48.patch As discussed on the list [1, 2] exceptions should only be used for exceptional cases. Requesting a not existing node through Microkernel.getNodes() should therefore rather return null instead of throwing an exception. [1] http://markmail.org/thread/agibgcdjv756m53u [2] http://markmail.org/message/gfbmogwr6mrhe2pm -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (OAK-53) exclude longer running tests in the default maven profile
exclude longer running tests in the default maven profile - Key: OAK-53 URL: https://issues.apache.org/jira/browse/OAK-53 Project: Jackrabbit Oak Issue Type: Test Reporter: Stefan Guggisberg the current maven build using the default profile takes ~3min on my machine which is IMO way too long. 'mvn clean install' should IMO allow quick turn around cycles during development and finish within a couple of seconds. requiring the developers to use a custom maven profile dring development (as suggested in OAK-52) is IMO not acceptable. custom profiles are hard to remember when working on different projects. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (OAK-53) exclude longer running tests in the default maven profile
[ https://issues.apache.org/jira/browse/OAK-53?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13250532#comment-13250532 ] Stefan Guggisberg commented on OAK-53: -- if 'mvn clean install' takes too long i guess i'll be using 'mvn clean install -DskipTests' instead. exclude longer running tests in the default maven profile - Key: OAK-53 URL: https://issues.apache.org/jira/browse/OAK-53 Project: Jackrabbit Oak Issue Type: Test Reporter: Stefan Guggisberg the current maven build using the default profile takes ~3min on my machine which is IMO way too long. 'mvn clean install' should IMO allow quick turn around cycles during development and finish within a couple of seconds. requiring the developers to use a custom maven profile dring development (as suggested in OAK-52) is IMO not acceptable. custom profiles are hard to remember when working on different projects. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (OAK-43) Incomplete journal when move and copy operations are involved
[
https://issues.apache.org/jira/browse/OAK-43?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Stefan Guggisberg resolved OAK-43.
--
Resolution: Fixed
fixed in svn r1311085
Incomplete journal when move and copy operations are involved
-
Key: OAK-43
URL: https://issues.apache.org/jira/browse/OAK-43
Project: Jackrabbit Oak
Issue Type: Bug
Components: mk
Reporter: Michael Dürig
Assignee: Stefan Guggisberg
Given a node at /source:
{code}
head = mk.commit(/,
\source\ : \moved\ +
*\moved\ : \copy\,
head, );
{code}
results in the following journal:
{code}
/source:/copy
{code}
where the freshly created node at /moved is missing.
Similarly
{code}
head = mk.commit(/,
*\source\ : \copy\ +
\copy\ : \moved\,
head, );
{code}
results in
{code}
+/moved:{}
{code}
where moved away node at /source is missing.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
Re: [Jackrabbit Wiki] Update of Jsop by JulianReschke
On Sat, Mar 31, 2012 at 9:34 AM, Julian Reschke julian.resc...@gmx.de wrote: On 2012-03-27 11:47, Stefan Guggisberg wrote: On Fri, Mar 23, 2012 at 6:04 PM, Julian Reschkejulian.resc...@gmx.de wrote: I have updated the Wiki page with an IETF ABNF variant of the JSOP Diff syntax. Open questions are: - JSON Patch (the IETF spec) allows set operations with objects; JSOP Diff didn't need it yet (as it's for overwriting properties); we probably should allow it in order to remove the mismatch (same for test) agreed - are pointers escaped the same way as in JSON Pointers or not? As we do not need / in names, we probably can get away without escaping, but then JSOP-Diff wouldn't be able to express all JSON-Patch documents would that be a problem? if possible i'd rather keep it simple and not support the '^' escaping. We just need to be aware of the issue/mismatch and document it. I'm ok with doing it this way. OTOH if it proofs to be a real iterop issue it might perhaps be worth the effort. The question here is what needs to interop with that. If this format is used purely inside Oak, we don't have to deal with it. If this is positioned as a generic patch format for JSON, we do have to. It would be good to finally decide on where we want to go with this format. agreed. while i like the idea of JSOP diff becoming a generic patch format for JSON i don't think that it's worth the extra headaches. on second thought i am now convinced that we should not support JSON Pointer (i.e. the escaping of forward slashes). i imagine such paths could/will be used in URLs. paths like /foo/bar/yes^/no would certainly cause problems. WDYT? cheers stefan - extensibility / metadata are not addressed yet As a next step we should document the mapping between these two formats (while noting the remaining differences). That way we can define the semantics of a JSOP-Diff instance in terms of JSON Patch, we should be on the IETF Standards Track in the not-to distant future. +1, excellent! cheers stefan Best regards, Julian
[jira] [Resolved] (OAK-47) Wrong results and NPE with copy operation
[
https://issues.apache.org/jira/browse/OAK-47?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Stefan Guggisberg resolved OAK-47.
--
Resolution: Fixed
fixed in svn r1310044
Wrong results and NPE with copy operation
-
Key: OAK-47
URL: https://issues.apache.org/jira/browse/OAK-47
Project: Jackrabbit Oak
Issue Type: Bug
Components: mk
Reporter: Michael Dürig
The following code either results in an NPE or in a wrong result depending on
which Microkernel instance is used.
{code}
mk.commit(, +\/root\:{}, mk.getHeadRevision(), );
mk.commit(,
+\/root/N0\:{}*\/root/N0\:\/root/N1\+\/root/N0/N4\:{},
mk.getHeadRevision(), );
{code}
The wrong result is
{code}
{
:childNodeCount: 2,
N0: {
:childNodeCount: 1,
N4: {
:childNodeCount: 0
}
},
N1: {
:childNodeCount: 1,
N4: {
:childNodeCount: 0
}
}
}
{code}
The expected result is
{code}
{
:childNodeCount: 2,
N0: {
:childNodeCount: 1,
N4: {
:childNodeCount: 0
}
},
N1: {
:childNodeCount: 0
}
}
{code}
simple:fs:target/temp: wrong result
fs:{homeDir}/target: NPE
http-bridge:fs:{homeDir}/target: NPE
simple: wrong result
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
Re: svn commit: r1298366 - in /jackrabbit/oak/trunk/oak-core/src/main/java/org/apache/jackrabbit/mk: ./ model/ store/
i just realized (sorry, a bit late, i know) that this commit
breaks some of the optimizations for large child node lists.
comparison of nodes had previously been delegated
to the Node/ChildNodeEntries implementation by calling
Node.diff(Node other). this allowed for specialized handling of
large child node collections (see e.g. ChildNodeEnttriesTree.getAdded()).
now, with diffing implemented on top of the abstract tree model,
those optimizations are unfortunately lost and diffing large
collections of child node entries will be highly inefficient
and will significantly affect the performance of e.g. MK.getJournal()...
cheers
stefan
On Thu, Mar 8, 2012 at 12:35 PM, ju...@apache.org wrote:
Author: jukka
Date: Thu Mar 8 11:35:35 2012
New Revision: 1298366
URL: http://svn.apache.org/viewvc?rev=1298366view=rev
Log:
OAK-3: Internal tree model
Commit proposed patch to implement diffing and related operations on top of
the abstract tree model.
Added:
jackrabbit/oak/trunk/oak-core/src/main/java/org/apache/jackrabbit/mk/model/NodeStateDiff.java
(with props)
Modified:
jackrabbit/oak/trunk/oak-core/src/main/java/org/apache/jackrabbit/mk/MicroKernelImpl.java
jackrabbit/oak/trunk/oak-core/src/main/java/org/apache/jackrabbit/mk/model/AbstractNode.java
jackrabbit/oak/trunk/oak-core/src/main/java/org/apache/jackrabbit/mk/model/CommitBuilder.java
jackrabbit/oak/trunk/oak-core/src/main/java/org/apache/jackrabbit/mk/model/Node.java
jackrabbit/oak/trunk/oak-core/src/main/java/org/apache/jackrabbit/mk/model/NodeDelta.java
jackrabbit/oak/trunk/oak-core/src/main/java/org/apache/jackrabbit/mk/model/NodeDiffHandler.java
jackrabbit/oak/trunk/oak-core/src/main/java/org/apache/jackrabbit/mk/model/TraversingNodeDiffHandler.java
jackrabbit/oak/trunk/oak-core/src/main/java/org/apache/jackrabbit/mk/store/CopyingGC.java
jackrabbit/oak/trunk/oak-core/src/main/java/org/apache/jackrabbit/mk/store/DefaultRevisionStore.java
jackrabbit/oak/trunk/oak-core/src/main/java/org/apache/jackrabbit/mk/store/RevisionProvider.java
jackrabbit/oak/trunk/oak-core/src/main/java/org/apache/jackrabbit/mk/store/StoredNodeAsState.java
Modified:
jackrabbit/oak/trunk/oak-core/src/main/java/org/apache/jackrabbit/mk/MicroKernelImpl.java
URL:
http://svn.apache.org/viewvc/jackrabbit/oak/trunk/oak-core/src/main/java/org/apache/jackrabbit/mk/MicroKernelImpl.java?rev=1298366r1=1298365r2=1298366view=diff
==
---
jackrabbit/oak/trunk/oak-core/src/main/java/org/apache/jackrabbit/mk/MicroKernelImpl.java
(original)
+++
jackrabbit/oak/trunk/oak-core/src/main/java/org/apache/jackrabbit/mk/MicroKernelImpl.java
Thu Mar 8 11:35:35 2012
@@ -20,14 +20,13 @@ import org.apache.jackrabbit.mk.api.Micr
import org.apache.jackrabbit.mk.api.MicroKernelException;
import org.apache.jackrabbit.mk.json.JsopBuilder;
import org.apache.jackrabbit.mk.json.JsopTokenizer;
-import org.apache.jackrabbit.mk.model.ChildNodeEntry;
import org.apache.jackrabbit.mk.model.Commit;
import org.apache.jackrabbit.mk.model.CommitBuilder;
import org.apache.jackrabbit.mk.model.Id;
import org.apache.jackrabbit.mk.model.StoredCommit;
-import org.apache.jackrabbit.mk.model.StoredNode;
import org.apache.jackrabbit.mk.model.TraversingNodeDiffHandler;
import org.apache.jackrabbit.mk.store.NotFoundException;
+import org.apache.jackrabbit.mk.store.RevisionProvider;
import org.apache.jackrabbit.mk.util.CommitGate;
import org.apache.jackrabbit.mk.util.PathUtils;
import org.apache.jackrabbit.mk.util.SimpleLRUCache;
@@ -207,19 +206,20 @@ public class MicroKernelImpl implements
try {
final JsopBuilder buff = new JsopBuilder();
+ final RevisionProvider rp = rep.getRevisionStore();
// maps (key: id of target node, value: path/to/target)
// for tracking added/removed nodes; this allows us
// to detect 'move' operations
final HashMapId, String addedNodes = new HashMapId, String();
final HashMapId, String removedNodes = new HashMapId,
String();
- StoredNode node1, node2;
+ NodeState node1, node2;
try {
- node1 = rep.getNode(fromRevisionId, path);
+ node1 = rep.getNodeState(fromRevisionId, path);
} catch (NotFoundException e) {
node1 = null;
}
try {
- node2 = rep.getNode(toRevisionId, path);
+ node2 = rep.getNodeState(toRevisionId, path);
} catch (NotFoundException e) {
node2 = null;
}
@@ -227,8 +227,7 @@ public class MicroKernelImpl implements
if (node1 == null) {
if (node2 != null) {
buff.tag('+').key(path).object();
- toJson(buff,
Re: Lifetime of revision identifiers
On Sat, Mar 31, 2012 at 12:39 PM, Jukka Zitting jukka.zitt...@gmail.com wrote: Hi, The revision identifiers returned by methods like MicroKernel.getHeadRevision() are plain strings so in theory I could write one down on a piece of paper, lock it in a safe, and come back ten years later expecting the identifier to give me access to the repository content as it existed a decade ago. Currently there's nothing in the documented MicroKernel contract that prevents me from expecting that the above use case would work. This is troublesome as it means that *no* past state of the repository should ever be automatically cleaned out as garbage. right, there's currently no mention of lifetime of revisions in the javadoc. i agree that this needs to be clearly specified. To allow automatic garbage collection without unexpectedly breaking client expectations, we should define some rules on the expected lifetime of revision identifiers. Without rules like that a client can't even do the following without worrying about potential interference from the garbage collector: String revision = mk.getHeadRevision(); String root = mk.getNodes(revision, /); Since the revision identifiers are plain strings, we can't leverage the standard garbage collector of the JVM and simply declare that all revisions identifiers will remain valid for at least as long as they are being referenced by some client. Thus a lease mechanism like revision identifiers remain valid for at least N minutes since last access may be needed. A client like a long-lived JCR Session would then need to either periodically refresh to the latest revision or extend its lease on an earlier revision. i think that we should specify that revisions have a certain guaranteed life span, e.g. N minutes, and that a client cannot expect to be able to read a revision exceeding that life span. however, what should the guaranteed life span be? 1 minute, 1 hour, 1 day? i don't think that we should allow clients to explicitly extend the life span of a specific revision. this would IMO unnecessarily complicate the GC logic and it would allow misbehaved clients to compromise the stability of the mk. cheers stefan Or we could combine these approaches by defining a Revision interface for local Java clients and an accompanying Revision-String mapping with defined lease handling for remote access. WDYT? BR, Jukka Zitting
Re: oak-api and move operations
On Fri, Mar 30, 2012 at 3:45 PM, Jukka Zitting jukka.zitt...@gmail.com wrote:
Hi,
On Fri, Mar 30, 2012 at 2:25 PM, Stefan Guggisberg
stefan.guggisb...@gmail.com wrote:
On Fri, Mar 30, 2012 at 2:18 PM, Jukka Zitting jukka.zitt...@gmail.com
wrote:
Exactly. IMHO we should adjust the MK interface to support this. The
solution should also address handling of large imports.
sorry, i can't follow you here. could you please elaborate?
The main trouble here is that oak-core currently needs to keep the
full transient space in memory (or in some other custom non-MK
storage) and then serialize it all to a single JSOP string during
save().
It would be much more convenient if the changes could be incrementally
sent to the MicroKernel and tracked as a standard content tree just
like any other content. This way we wouldn't need to come up with a
separate storage mechanism in oak-core or use temporary subtrees in
the main repository to work around this limitation.
One way of dealing with this on the MK level would be to introduce the
concept of private branches that are only visible to a single client
until explicitly merged back to the main repository. A quick draft of
what this could look like:
String addLotsOfData(MicroKernel mk) {
String baseRevision = mk.getHeadRevision();
String branchRevision = mk.branch(baseRevision);
for (int i = 0; i 100; i++) {
branchRevision = mk.commit(
/, +\node + i + \:{}, branchRevision, null);
}
return mk.merge(branchRevision, baseRevision);
}
good point. the versioning model of the mk has been strictly linear so far
but i guess your proposal makes sense. we'll have to take into account
potential implications (gc, getJournal etc) but currently i don't see any
real problems with supporting 'private' branches.
cheers
stefan
BR,
Jukka Zitting
Re: [Jackrabbit Wiki] Update of Jsop by JulianReschke
On Fri, Mar 23, 2012 at 6:04 PM, Julian Reschke julian.resc...@gmx.de wrote:
I have updated the Wiki page with an IETF ABNF variant of the JSOP Diff
syntax.
Open questions are:
- JSON Patch (the IETF spec) allows set operations with objects; JSOP Diff
didn't need it yet (as it's for overwriting properties); we probably should
allow it in order to remove the mismatch (same for test)
agreed
- are pointers escaped the same way as in JSON Pointers or not? As we do not
need / in names, we probably can get away without escaping, but then
JSOP-Diff wouldn't be able to express all JSON-Patch documents
would that be a problem? if possible i'd rather keep it simple and not
support the '^' escaping.
OTOH if it proofs to be a real iterop issue it might perhaps be worth
the effort.
- extensibility / metadata are not addressed yet
As a next step we should document the mapping between these two formats
(while noting the remaining differences). That way we can define the
semantics of a JSOP-Diff instance in terms of JSON Patch, we should be on
the IETF Standards Track in the not-to distant future.
+1, excellent!
cheers
stefan
Best regards, Julian
Original Message
Subject: [Jackrabbit Wiki] Update of Jsop by JulianReschke
Date: Fri, 23 Mar 2012 16:51:21 -
From: Apache Wiki wikidi...@apache.org
Reply-To: d...@jackrabbit.apache.org
To: Apache Wiki wikidi...@apache.org
Dear Wiki user,
You have subscribed to a wiki page or wiki category on Jackrabbit Wiki for
change notification.
The Jsop page has been changed by JulianReschke:
http://wiki.apache.org/jackrabbit/Jsop?action=diffrev1=43rev2=44
STRING ::= /* See RFC 4627, Section 2.5 */
NUMBER ::= /* See RFC 4627, Section 2.4 */
}}}
+
+ IETF ABNF variant:
+ {{{
+ diffs = *( ws ( add / set / remove / move / copy / test ) ws )
+
+ add = + ws pointer ws : ws ( object / atom / array )
+ copy = * ws pointer ws : ws pointer
+ move = ws pointer ws : ws pointer
+ remove = - ws pointer
+ set = ^ ws pointer ws : ws ( atom / array ) ; object?
+ test = ? ws pointer ws : ws ( atom / array ) ; object?
+
+ atom = string / number / false / null / true
+
+ pointer = string
+
+ ; JSON
+ array = see RFC4627, Section 2.3
+ false = see RFC4627, Section 2.1
+ null = see RFC4627, Section 2.1
+ number = see RFC4627, Section 2.4
+ object = see RFC4627, Section 2.2
+ string = see RFC4627, Section 2.5
+ true = see RFC4627, Section 2.1
+ ws = see RFC4627, Section 2.1
+ }}}
+
+
A
[[http://svn.apache.org/viewvc/jackrabbit/sandbox/jackrabbit-mk/jackrabbit-spi2microkernel/src/main/java/org/apache/jackrabbit/spi2microkernel/json/JsopParser.java?view=markup|parser]],
and a simple
[[http://svn.apache.org/viewvc/jackrabbit/sandbox/jackrabbit-mk/jackrabbit-spi2microkernel/src/test/java/org/apache/jackrabbit/spi2microkernel/json/JsopParserTest.java?view=markup|test
case]] for this syntax is available from the sandbox.
Also available are a
[[http://svn.apache.org/viewvc/jackrabbit/sandbox/microkernel/src/main/java/org/apache/jackrabbit/mk/json/JsopTokenizer.java?view=markup|tokenizer]],
a
[[http://svn.apache.org/viewvc/jackrabbit/sandbox/microkernel/src/main/java/org/apache/jackrabbit/mk/json/JsopBuilder.java?view=markup|builder]],
and a
[[http://svn.apache.org/viewvc/jackrabbit/sandbox/microkernel/src/main/java/org/apache/jackrabbit/mk/json/JsopStream.java?view=markup|Jsop
writer / reader]]. Those implementations are token based, and therefore not
limited to the JSOP format described here.
Re: Value types supported by the MicroKernel
On Wed, Mar 21, 2012 at 8:26 PM, Jukka Zitting jukka.zitt...@gmail.com wrote: Hi, The MicroKernel interface currently says: supported property types: string, number that's outdated, it now says: string, number, boolean In addition there's explicit support for binary blobs. For oak-core (see OAK-33 [1]) we'd need a bit more detailed definition of the supported value types. A few questions: the microkernel as currently implemented does not interpret property value types, it just stores the value token. 1) Are boolean values (the true and false keywords in JSON) also supported? It would be nice if they were. yes 2) Are null values (the null keyword) supported? It would be nice if not. setting a property to null (e.g. ^foo:null ) removes it (as in JCR). however, adding a node with null valued properties is currently possible. that should probably be fixed. 3) Are arrays (i.e. [ ... ]) supported? If yes, are there constraints on element types? It would be nice if arrays indeed were supported. With the restriction that all elements are scalar values of the same type. it does support arrays but doesn't enforce restrictions. the microkernel should IMO just support the basic json model. restrictions should be handled on a higher level. 4) Are there limits on number values? Most notably, can a number that's larger than Long.MAX_VALUE be stored reliably (as a number)? Larger than Double.MAX_VALUE? What about things like Double.NaN? Or weirder, the other 2^51 NaN values allowed by IEEE754? whatever you pass in the commit is stored and returned as-is. as already mentioned, the value is (currently) not interpreted. 5) Are there limits on string values? For example, can I expect to store a string value that's larger than 1MB? Larger than 1GB? the heap is your limit ;) 6) Presumably strings are in Unicode. Are characters beyond the BMP supported? Is it possible for a string value to contain a Unicode non-character, for example U+? erm, haven't thought about that. do you think that there could be a problem? cheers stefan [1] https://issues.apache.org/jira/browse/OAK-33 BR, Jukka Zitting
Re: Value types supported by the MicroKernel
On Thu, Mar 22, 2012 at 4:22 PM, Jukka Zitting jukka.zitt...@gmail.com wrote:
Hi,
On Thu, Mar 22, 2012 at 3:04 PM, Stefan Guggisberg
stefan.guggisb...@gmail.com wrote:
On Wed, Mar 21, 2012 at 8:26 PM, Jukka Zitting jukka.zitt...@gmail.com
wrote:
For oak-core (see OAK-33 [1]) we'd need a bit more detailed definition
of the supported value types. A few questions:
the microkernel as currently implemented does not interpret property
value types, it just stores the value token.
Can we turn that into a harder API contract, i.e. one that should hold
true for any MK implementation? If not, we need to define the
boundaries of what oak-core can expect the MK to store reliably. We
don't need the answers right away, but ultimately this needs to be
defined for us to properly resolve OAK-33.
ack and agreed. i am fine with specifying a harder api contract something
along the lines of 'values are treated as opaque character sequences'.
A related question, will a value like \u0058 be considered equal to
X for example when merging concurrent changes? What about [ ] vs. []
(i.e. extra whitespace within an empty array)?
hmm, now it gets tricky... if the mk treats values as opaque character sequences
it would consider the above examples non-equal. we'll have to see whether
that's an issue. if it turns out to be an issue we'll have to provide
for smarter
merge-logic and perhaps some basic normalization (e.g. removing ws
within arrays).
2) Are null values (the null keyword) supported? It would be nice if not.
setting a property to null (e.g. ^foo:null ) removes it (as in JCR).
however, adding a node with null valued properties is currently possible.
that should probably be fixed.
OK, sounds good. So oak-core can treat a case like {foo:null} being
returned from MicroKernel.getNodes() as an unexpected error, probably
caused by an administrator directly accessing the MicroKernel.
3) Are arrays (i.e. [ ... ]) supported? If yes, are there
constraints on element types? It would be nice if arrays indeed were
supported. With the restriction that all elements are scalar values of
the same type.
it does support arrays but doesn't enforce restrictions.
the microkernel should IMO just support the basic json model.
restrictions should be handled on a higher level.
OK, cool.
4) Are there limits on number values? Most notably, can a number
that's larger than Long.MAX_VALUE be stored reliably (as a number)?
Larger than Double.MAX_VALUE? What about things like Double.NaN? Or
weirder, the other 2^51 NaN values allowed by IEEE754?
whatever you pass in the commit is stored and returned as-is.
as already mentioned, the value is (currently) not interpreted.
OK. Can we get rid of the (currently) qualifier? Also, the not
interpreted part needs to be qualified in terms of things like
conflict merging. I'd be fine with store and used as an opaque,
unparsed character sequence.
i agree with that wording.
cheers
stefan
Basically this means that in oak-core or -jcr we need to either a) not
support all possible Java double values or b) use some custom string
encoding for such values. I suppose a) is the better alternative, but
we need to check whether that will cause problems down the line.
6) Presumably strings are in Unicode. Are characters beyond the BMP
supported? Is it possible for a string value to contain a Unicode
non-character, for example U+?
erm, haven't thought about that. do you think that there could be a problem?
Like with the Double.NaN case, unless the underlying MicroKernel
supports the full range of characters, we need to consider adding some
extra level of encoding (possibly simply the \u escapes) on the
oak-core or -jcr level. But it sounds like this won't be needed.
BR,
Jukka Zitting
[jira] [Commented] (OAK-32) Drop MicroKernel.dispose()
[ https://issues.apache.org/jira/browse/OAK-32?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13234206#comment-13234206 ] Stefan Guggisberg commented on OAK-32: -- +1 cheers stefan Drop MicroKernel.dispose() -- Key: OAK-32 URL: https://issues.apache.org/jira/browse/OAK-32 Project: Jackrabbit Oak Issue Type: Improvement Reporter: Jukka Zitting Just like a client of the MicroKernel interface doesn't know how a MK instance is created, there should not be a need for a client to be able to dispose an instance. For example the lifecycle of a MK instance running as an OSGi service (or any other component framework) is managed by the framework, not by clients. Thus I suggest that the MicroKernel.dispose() method is removed. The only piece of code that's notably affected by this change is the MicroKernelFactory class still in oak-core and any client code that uses it to construct new MicroKernel instances. I think we should replace the MKF class with a more generic solution as outlined in OAK-17. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
Re: JSOP vs JSON-Patch, was: [Jackrabbit Wiki] Update of Jsop by stefan
sorry, i forgot to cc oak-dev. resending...
On Wed, Mar 21, 2012 at 2:07 PM, Julian Reschke julian.resc...@gmx.de wrote:
On 2012-03-21 13:47, Stefan Guggisberg wrote:
On Tue, Mar 20, 2012 at 6:53 PM, Julian Reschkejulian.resc...@gmx.de
wrote:
Hi,
so Stefan is cleaning up the JSOP Wiki page (thanks!).
I think it's time to look again at the difference between the JSOP diff
format (as used in MK.commit()), and the IETF JSON Patch format, now
http://trac.tools.ietf.org/html/draft-ietf-appsawg-json-patch-01.
JSOP has lost it's dependency on member ordering, and JSON Patch now has
test and copy, so it seems the only missing gap is the metadata
feature.
WRT the set of operations (add, replace, copy et al) i agree that there's
a
nice 1:1 match.
Thanks for confirming.
So should I start work on a spec that defines JSOP diff as syntactical
variant of draft-ietf-appsawg-json-patch-01?
yes, i guess it would make sense.
WRT the metadata feature. it might be useful in certain situations but
i am not sure whether we really need it.
I assume it's needed for the commit message in MK, but that of course could
be separated into a separate string argument.
the commit msg is already passed as a separate argument in the commit method.
thomas' proposal referred to an alternative format of getJournal
return value [1].
this has been discussed on the dev list a while ago [2].
IMO it does seem useful but i don't know whether or how we should
specify that as a proposed extension to JSON patch.
[1]
http://svn.apache.org/repos/asf/jackrabbit/oak/trunk/oak-mk/src/main/java/org/apache/jackrabbit/mk/api/MicroKernel.java
[2] http://www.mail-archive.com/dev@jackrabbit.apache.org/msg26796.html
If this is true, we can (mostly) define the JSOP diff format as a
transformation from JSON Patch. That might be useful in order to avoid
bikeshed discussions about syntax.
i agree that it would be useful if we could define JSOP diff as a
transformation
from JSON Patch.
however, after skim-reading the JSON Patch draft i noticed the following
issues:
- adding nested object trees doesn't seem to be explicitly covered,
e.g. +/a/b/c : { foo : bar}
the draft only talks about manipulating non-object members of a
document.
In
http://tools.ietf.org/html/draft-ietf-appsawg-json-patch-01#section-4.1,
the example is setting an array as value. But I agree that it would be good
to clarify the term value.
- the draft does explicitly cover manipulating arrays of values.
that's not applicable
to the mk data model (array values are not supported). furthermore,
there's
no equivalent jcr operation.
Ack. So this would be something that can be syntactically expressed in JSOP
Diff, but wouldn't be supported by the MK. Is this a problem?
i don't know yet, but probably not. we would have to clearly document
that restriction on the microkernel commit method.
One open issue seems to be escaping inside pathString; if a JSON member
contains a forward slash (/), how is it represented in a pathString?
1) We could define that it's not allowed to be in a member name, thus we
don't need escaping, or
2) We could adopt the escaping syntax from JSON Pointer (now ^).
1) has the advantage of being simple.
2) has the advantage that it would work with generic JSON data, something
I
think we need to deal with if we want to use JSOP diff outside the MK
work.
Thoughts?
since forward slashes are not allowed in jcr names i guess it's not an
issue
for JSOP diff since / would need be encoded already according to the
jcr rules. i would prefer 1) but i am fine with either 1) or 2).
My proposal is to start with 1) but to document the restriction; we can
still introduce escaping later on if we find out that this is a problem.
agreed.
cheers
stefan
Best regards, Julian
Re: JSOP vs JSON-Patch, was: [Jackrabbit Wiki] Update of Jsop by stefan
On Wed, Mar 21, 2012 at 5:05 PM, Julian Reschke julian.resc...@gmx.de wrote: On 2012-03-21 16:28, Julian Reschke wrote: ... OK, here's an attempt to rewrite in IETF ABNF: great, thanks! -- snip -- diffs = *( ws ( add / set / remove / move / copy / test ) ws ) add = + ws pointer ws : ws ( object / atom / array ) copy = * ws pointer ws : ws pointer move = ws pointer ws : ws pointer remove = - ws pointer set = ^ ws pointer ws : ws ( atom / array ) ; object? test = ? ws pointer ws : ws ( atom / array ) ; object? atom = string / number / false / null / true pointer = string ; JSON array = see RFC4627, Section 2.3 false = see RFC4627, Section 2.1 null = see RFC4627, Section 2.1 number = see RFC4627, Section 2.4 object = see RFC4627, Section 2.2 string = see RFC4627, Section 2.5 true = see RFC4627, Section 2.1 ws = see RFC4627, Section 2.1 -- snip -- Open issues: commenting below from the microkernel POV... 1) JSON's ws allows CR LF, and it can occur in arrays and objects. Is this true for JSOPDIFF as well? yes 2) I left out METADATA and EXTENSION for now 3) Does JSOPDIFF require anything special between diffs? Are they supposed to start on new lines? there are no special requirements for separating the diffs. 4) The way the grammar is written makes it hard to extend, we may want to generalize it into simply diffs = *( ws op ws ) op = VCHAR ws pointer ws : ws ( object / atom / array ) and then describe the individual operations separately. agreed, makes sense. cheers stefan Feedback appreciated, Julian
Re: Javadoc: {@code...} instead of code/?
On Thu, Mar 15, 2012 at 3:59 PM, Felix Meschberger fmesc...@adobe.com wrote:
Hi,
Am 15.03.2012 um 15:46 schrieb Michael Dürig:
Hi,
What do people feel about using {@code null} instead of
codenull/code in javadoc? Personally I prefer the second one since
it is lighter on the eye but don't have a strong opinion. I think
however we should use all the same format.
I didn't know the {@code} tag exists so I always use the code.../code
style.
same here. i realize that i am getting old;)
i kind of like the {@code} variant but i don't feel strong either way.
cheers
stefan
From a formatting POV (allowing to format segments with tags different from
code) it might really make sense to use the tag instead of the html element.
Regards
Felix
Re: On setting component boundaries in Oak
On Fri, Mar 9, 2012 at 2:15 PM, Angela Schreiber anch...@adobe.com wrote: hi Following up on OAK-5, where the question came up on whether we should put the JCR binding for Oak to a separate oak-jcr component or just under an .oak.jcr package in oak-core. There are good arguments for both approaches, and the balance of the different solutions also depends on the state of the overall architecture. as far as i understand the overall strategy as presented by david at the last f2f in basel the objective was to have a separate API boundary (SPI-like). therefore i would strongly suggest to separate jcr-transient space from an SPI layer from the very beginning. based on my experience both with jackrabbit-core and being the author of the v1 SPI i am convinced that we should start with that separation from the beginning and force ourselves to clearly distinguish between the two. +1, i absolutely agree with angela. cheers stefan we should start with an first proposal of what part of the JCR implementation goes where and keep discussing those decisions as we go ahead and face problems and inconsistencies. Before making the decision on this, here's a few questions to consider: 1) What's the main API through which functionality in oak-core (and the potential low-level extensions it interacts with) shall be accessed? Shall things like node type handling, versioning, locking and query support be included in that API? my answer was that everything that needs validation for overall consistency and JCR compliance goes to the core (spi impl). AFAIUI the MicroKernel interface is intended more as an internal extension point and remoting API instead of something that a client (including one like a specific protocol binding) would directly interact with, so it probably shouldn't be the API through which other components access functionality in oak-core. i agree 2) What kind of clients will interact directly with Oak through JCR vs. some lover level API (call it Oak SPI for now)? For example, do we want to build a WebDAV or JSON/JSOP protocol binding based directly on the Oak SPI or should they work on top of the JCR binding like they currently do in Jackrabbit 2.x? Using an appropriate lower level API might make it easier or even possible to implement things like ETag support for WebDAV. as far as i understood from the discussions we had so far, we have both use cases: a) applications that interact with the JCR API b) applications and non-java JCR implementations like the Jackalope project that directly interact with the Oak SPI as you call it. 3) Given that such an internal Oak SPI is needed above the MK level, are we confident enough that we know where that boundary between oak-core and the proposed oak-jcr components should be set and what the boundary should look like? Making a premature decision on that could easily become a self-fulfilling prophesy that binds us to a specific architectural layout. For example the current Jackrabbit SPI has some inherent design limitations that are very hard to fix anymore given that plenty of client code is already using it. imo we should not be afraid of creating the boundary right from the beginning and feeling free to adjust it as we go ahead. but the prerequisite was to have a first draft of what goes where that allows us to validate our assumptions and change or refine them in the dev process. regards angela BR, Jukka Zitting
[jira] [Commented] (OAK-7) In-memory persistence
[ https://issues.apache.org/jira/browse/OAK-7?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13226125#comment-13226125 ] Stefan Guggisberg commented on OAK-7: - Currently that's not possible since the o.a.j.mk.Repository class always creates the homeDir, even if you switch the DefaultRevisionStore to use InMemPersistence. +1 for changing the current behavior and just creating the homeDir on demand, i.e. lazily when it's actually accessed. In-memory persistence - Key: OAK-7 URL: https://issues.apache.org/jira/browse/OAK-7 Project: Jackrabbit Oak Issue Type: New Feature Reporter: Jukka Zitting Fix For: 0.1 There should be a way to run an Oak repository that requires zero disk access. Currently that's not possible since the o.a.j.mk.Repository class always creates the homeDir, even if you switch the DefaultRevisionStore to use InMemPersistence. Having such a fully in-memory repository makes it easier (and faster) to run test cases for which the actual persistence mechanism is irrelevant. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
