Re: [VOTE] Release Apache Jackrabbit Oak 1.6.4

[Andrei Dulceanu]

2017-08-16 7:45 GMT+03:00 Amit Jain :

> Please vote on releasing this package as Apache Jackrabbit Oak 1.6.4.
> The vote is open for the next 72 hours and passes if a majority of at
> least three +1 Jackrabbit PMC votes are cast.
>
> [ ] +1 Release this package as Apache Jackrabbit Oak 1.6.4
> [ ] -1 Do not release this package because...
>

[X] +1 Release this package as Apache Jackrabbit Oak 1.6.4

Regards,
Andrei

Re: [VOTE] Release Apache Jackrabbit Oak 1.0.39

[Andrei Dulceanu]

2017-08-15 21:25 GMT+03:00 Manfred Baedke :

> Please vote on releasing this package as Apache Jackrabbit Oak 1.0.39.
> The vote is open for the next 72 hours and passes if a majority of at
> least three +1 Jackrabbit PMC votes are cast.
>
> [ ] +1 Release this package as Apache Jackrabbit Oak 1.0.39
> [ ] -1 Do not release this package because...
>
>
[X ] +1 Release this package as Apache Jackrabbit Oak 1.0.39

Regards,
Andrei

Re: [VOTE] Release Apache Jackrabbit Oak 1.0.39

[Robert Munteanu]

On Tue, 2017-08-15 at 20:25 +0200, Manfred Baedke wrote:
> Please vote on releasing this package as Apache Jackrabbit Oak
> 1.0.39.


+1

Robert

signature.asc
Description: This is a digitally signed message part

Re: [VOTE] Release Apache Jackrabbit Oak 1.6.4

[Julian Reschke]

On 2017-08-16 06:45, Amit Jain wrote:

...


[X] +1 Release this package as Apache Jackrabbit Oak 1.6.4

Best regards, Julian

Re: [CompositeBlobStore] Delegate traversal algorithm

[Thomas Mueller]

Hi,

The Bloom filter is something to consider, to speed up reading. It's not 
strictly needed of course.

Yes, I would consider using Bloom filters to more quickly find out where an 
entry is stored, if there are multiple possibilities. So, one filter per 
"delegate". In our case, the most logical place to do that is for the read-only 
stores. They could also be used for read-write stores (created during garbage 
collection for example). Sure, they would not always be up-to-date, but most 
(let's say 90%) binaries are older than the last GC, so it would speedup that 
case (and have basically no cost for new entries, as the filter is in memory).

Regards,
Thomas





On 16.08.17, 02:25, "Matt Ryan"  wrote:

Hi Thomas (and everyone else):

I wanted to ask about a comment you made in the wiki where you said "Bloom
filters should be mentioned (are they used, if yes how, if not why not).”
 I assume since you included that you are thinking they probably should be
used.

I believe the intended use of a Bloom filter in this case would be for read
operations, to quickly determine if a blob id is not stored anywhere in the
system.  Let me know if you had another use in mind.

If that’s the use case, I wonder how we would reasonably come up with a
useful guess as to the appropriate size of the filter.  Someone with more
experience using them could maybe offer some insight here as to appropriate
values for an expected number of insertions and the appropriate expected
false positive probability.

It seems like we could also use more than one Bloom filter, one for each
delegate to say whether the blob id is located in that particular
delegate.  Not sure if you were thinking more along those lines or just a
single Bloom filter for the entire composite as a whole, or both.

-MR

On August 15, 2017 at 4:06:56 PM, Matt Ryan (o...@mvryan.org) wrote:

Hi Thomas,

After emailing I saw you also provided comments in-line on the wiki.  I’ll
work through those and reply back on-list when I think I have addressed
them.  Thanks for doing that also!

-MR


On August 15, 2017 at 2:01:04 PM, Matt Ryan (o...@mvryan.org) wrote:

Hi Thomas,

Thank you for taking the time to offer a review.  I’ve been going through
the suggested readings and will continue to do so.

Some comments inline below.


On August 15, 2017 at 12:25:54 AM, Thomas Mueller 
(muel...@adobe.com.invalid)
wrote:

Hi,

It is important to understand which operations are available in the JCR
API, the DataStore API, and the concept of revisions we use for Oak. For
example,

* The DataStore API doesn’t support updating a binary.


This is of course true.  The interface supports only an “addRecord()”
capability to put a blob into the data store.  The javadoc there clearly
expects the possibility that the record may already exist:  "If the same
stream already exists in another record, then that record is returned
instead of creating a new one.”

Implementations handle the details of what happens when the blob already
exists.  For example, the “write()” method in the S3Backend class clearly
distinguishes between the two as the way to handle this via the AWS SDK is
different for an update versus a create:

https://svn.apache.org/repos/asf/jackrabbit/oak/trunk/oak-blob-cloud/src/main/java/org/apache/jackrabbit/oak/blob/cloud/s3/S3Backend.java

It is still the case that from the data store’s point of view there is no
difference between the two so it doesn’t support a distinction.

The original data store concept can take this approach because it only has
one place for the data to go.  The composite blob store has more than one
place the data could go, so I believe there is a possibility that the data
could exist in a delegate blob store that is not the first blob store that
the data could be written to.

What should happen in that case?  I assumed we should try to find a match
first, and prefer updating to creating new.  I’m not sure exactly how that
would happen though, since the name only matches if the content hash is the
same (unless there’s a collision of course), and otherwise it’s a new blob
anyway.



* A node might have multiple revisions.
* In the Oak revision model, you can't update a reference of an old
revision.


Does the data store even know about this?  I assumed this was all handled
at a higher level, and that once the data store is told to add a record
it’s already been determined that the write is okay, even if it ends up
that the stream being written already exists somewhere.



* The JCR API allows to create binaries without nodes via ValueFactory (so

Re: [VOTE] Release Apache Jackrabbit Oak 1.6.4

[Robert Munteanu]

On Wed, 2017-08-16 at 10:15 +0530, Amit Jain wrote:
> Please vote on releasing this package as Apache Jackrabbit Oak 1.6.4.

+1

Robert

signature.asc
Description: This is a digitally signed message part

Re: [VOTE] Release Apache Jackrabbit Oak 1.6.4

[Thomas Mueller]

> Please vote on releasing this package as Apache Jackrabbit Oak 1.6.4.
+1  

Thomas

Re: [VOTE] Release Apache Jackrabbit Oak 1.0.39

[Amit Jain]

+1 Release this package as Apache Jackrabbit Oak 1.0.39

Thanks
Amit

On Tue, Aug 15, 2017 at 11:55 PM, Manfred Baedke 
wrote:

> A candidate for the Jackrabbit Oak 1.0.39 release is available at:
>
> https://dist.apache.org/repos/dist/dev/jackrabbit/oak/1.0.39/
>
> The release candidate is a zip archive of the sources in:
>
> https://svn.apache.org/repos/asf/jackrabbit/oak/tags/jackrab
> bit-oak-1.0.39/
>
> The SHA1 checksum of the archive is 5cfb85374d99c96f5528697255a0f5
> c626657032.
>
> A staged Maven repository is available for review at:
>
> https://repository.apache.org/
>
> The command for running automated checks against this release candidate is:
>
> $ sh check-release.sh oak 1.0.39 5cfb85374d99c96f5528697255a0f5
> c626657032
>
> Please vote on releasing this package as Apache Jackrabbit Oak 1.0.39.
> The vote is open for the next 72 hours and passes if a majority of at
> least three +1 Jackrabbit PMC votes are cast.
>
> [ ] +1 Release this package as Apache Jackrabbit Oak 1.0.39
> [ ] -1 Do not release this package because...
>
>

Re: svn commit: r1805209 - in /jackrabbit/oak/trunk/oak-segment-tar: ./ src/main/java/org/apache/jackrabbit/oak/segment/file/ src/main/java/org/apache/jackrabbit/oak/segment/file/tar/ src/test/java/or

[Francesco Mari]

Instead of creating IOMonitor.NULL, you could instantiate a new
IOMonitorAdapter when needed. This has already been done many times in
TarFileTest and TarWriterTest.

On Wed, Aug 16, 2017 at 5:26 PM,   wrote:
> Author: mduerig
> Date: Wed Aug 16 15:26:29 2017
> New Revision: 1805209
>
> URL: http://svn.apache.org/viewvc?rev=1805209=rev
> Log:
> OAK-6555: Implement ITs for rolling upgrade
>
> Added:
> 
> jackrabbit/oak/trunk/oak-segment-tar/src/test/java/org/apache/jackrabbit/oak/segment/upgrade/
> 
> jackrabbit/oak/trunk/oak-segment-tar/src/test/java/org/apache/jackrabbit/oak/segment/upgrade/UpgradeIT.java
> jackrabbit/oak/trunk/oak-segment-tar/src/test/upgrade-it-scripts/
> 
> jackrabbit/oak/trunk/oak-segment-tar/src/test/upgrade-it-scripts/create16store.groovy
> Modified:
> jackrabbit/oak/trunk/oak-segment-tar/pom.xml
> 
> jackrabbit/oak/trunk/oak-segment-tar/src/main/java/org/apache/jackrabbit/oak/segment/file/ManifestChecker.java
> 
> jackrabbit/oak/trunk/oak-segment-tar/src/main/java/org/apache/jackrabbit/oak/segment/file/tar/IOMonitor.java
>
> Modified: jackrabbit/oak/trunk/oak-segment-tar/pom.xml
> URL: 
> http://svn.apache.org/viewvc/jackrabbit/oak/trunk/oak-segment-tar/pom.xml?rev=1805209=1805208=1805209=diff
> ==
> --- jackrabbit/oak/trunk/oak-segment-tar/pom.xml (original)
> +++ jackrabbit/oak/trunk/oak-segment-tar/pom.xml Wed Aug 16 15:26:29 2017
> @@ -87,6 +87,40 @@
>  true
>  
>  
> +
> +org.apache.maven.plugins
> +maven-dependency-plugin
> +
> +
> +pre-integration-test
> +copy
> +
> +
> org.apache.jackrabbit:oak-run:1.6.1
> +
> ${project.build.directory}/upgrade-it
> +true
> +
> +
> +
> +
> +
> +org.apache.maven.plugins
> +maven-resources-plugin
> +
> +
> +pre-integration-test
> +copy-resources
> +
> +
> ${project.build.directory}/upgrade-it
> +
> +
> +
> src/test/upgrade-it-scripts
> +false
> +
> +
> +
> +
> +
> +
>  
>  
>
>
> Modified: 
> jackrabbit/oak/trunk/oak-segment-tar/src/main/java/org/apache/jackrabbit/oak/segment/file/ManifestChecker.java
> URL: 
> http://svn.apache.org/viewvc/jackrabbit/oak/trunk/oak-segment-tar/src/main/java/org/apache/jackrabbit/oak/segment/file/ManifestChecker.java?rev=1805209=1805208=1805209=diff
> ==
> --- 
> jackrabbit/oak/trunk/oak-segment-tar/src/main/java/org/apache/jackrabbit/oak/segment/file/ManifestChecker.java
>  (original)
> +++ 
> jackrabbit/oak/trunk/oak-segment-tar/src/main/java/org/apache/jackrabbit/oak/segment/file/ManifestChecker.java
>  Wed Aug 16 15:26:29 2017
> @@ -22,9 +22,9 @@ import static com.google.common.base.Pre
>  import java.io.File;
>  import java.io.IOException;
>
> -class ManifestChecker {
> +public class ManifestChecker {
>
> -static ManifestChecker newManifestChecker(File path, boolean 
> shouldExist, int minStoreVersion, int maxStoreVersion) {
> +public static ManifestChecker newManifestChecker(File path, boolean 
> shouldExist, int minStoreVersion, int maxStoreVersion) {
>  checkArgument(path != null, "path");
>  checkArgument(minStoreVersion > 0, "minStoreVersion");
>  checkArgument(maxStoreVersion > 0, "maxStoreVersion");
> @@ -52,7 +52,7 @@ class ManifestChecker {
>  updateManifest(manifest);
>  }
>
> -void checkManifest() throws IOException, 
> InvalidFileStoreVersionException {
> +public void checkManifest() throws IOException, 
> InvalidFileStoreVersionException {
>  checkManifest(openManifest());
>  }
>
>
> Modified: 
> jackrabbit/oak/trunk/oak-segment-tar/src/main/java/org/apache/jackrabbit/oak/segment/file/tar/IOMonitor.java
> URL: 
> http://svn.apache.org/viewvc/jackrabbit/oak/trunk/oak-segment-tar/src/main/java/org/apache/jackrabbit/oak/segment/file/tar/IOMonitor.java?rev=1805209=1805208=1805209=diff
> ==
> --- 
> jackrabbit/oak/trunk/oak-segment-tar/src/main/java/org/apache/jackrabbit/oak/segment/file/tar/IOMonitor.java
>  (original)
> +++ 
>