Author: amitj
Date: Wed May 3 05:20:45 2017
New Revision: 1793609
URL: http://svn.apache.org/viewvc?rev=1793609&view=rev
Log:
OAK-4933: Create a data store implementation that integrates with Microsoft
Azure Blob Storage
Merged r1788387 from trunk
Added:
jackrabbit/oak/branches/1.6/oak-blob-cloud-azure/
- copied from r1788387, jackrabbit/oak/trunk/oak-blob-cloud-azure/
Modified:
jackrabbit/oak/branches/1.6/ (props changed)
jackrabbit/oak/branches/1.6/oak-blob-cloud-azure/pom.xml
jackrabbit/oak/branches/1.6/oak-doc/src/site/markdown/osgi_config.md
jackrabbit/oak/branches/1.6/oak-doc/src/site/markdown/plugins/blobstore.md
jackrabbit/oak/branches/1.6/oak-parent/pom.xml
jackrabbit/oak/branches/1.6/pom.xml
Propchange: jackrabbit/oak/branches/1.6/
------------------------------------------------------------------------------
--- svn:mergeinfo (original)
+++ svn:mergeinfo Wed May 3 05:20:45 2017
@@ -1,3 +1,3 @@
/jackrabbit/oak/branches/1.0:1665962
-/jackrabbit/oak/trunk:1781068,1781075,1781248,1781386,1781846,1781907,1782000,1782029,1782196,1782447,1782476,1782770,1782945,1782966,1782973,1782990,1783061,1783066,1783089,1783104-1783105,1783619,1783720,1783731,1783733,1783738,1783742,1783773,1783855,1783891,1784023,1784130,1784162,1784251,1784401,1784551,1784574,1785095,1785108,1785283,1785838,1785919,1785946,1787074,1787217,1788378,1790382,1792463,1792742,1793088
+/jackrabbit/oak/trunk:1781068,1781075,1781248,1781386,1781846,1781907,1782000,1782029,1782196,1782447,1782476,1782770,1782945,1782966,1782973,1782990,1783061,1783066,1783089,1783104-1783105,1783619,1783720,1783731,1783733,1783738,1783742,1783773,1783855,1783891,1784023,1784130,1784162,1784251,1784401,1784551,1784574,1785095,1785108,1785283,1785838,1785919,1785946,1787074,1787217,1788378,1788387,1790382,1792463,1792742,1793088
/jackrabbit/trunk:1345480
Modified: jackrabbit/oak/branches/1.6/oak-blob-cloud-azure/pom.xml
URL:
http://svn.apache.org/viewvc/jackrabbit/oak/branches/1.6/oak-blob-cloud-azure/pom.xml?rev=1793609&r1=1788387&r2=1793609&view=diff
==============================================================================
--- jackrabbit/oak/branches/1.6/oak-blob-cloud-azure/pom.xml (original)
+++ jackrabbit/oak/branches/1.6/oak-blob-cloud-azure/pom.xml Wed May 3
05:20:45 2017
@@ -21,7 +21,7 @@
<parent>
<artifactId>oak-parent</artifactId>
<groupId>org.apache.jackrabbit</groupId>
- <version>1.8-SNAPSHOT</version>
+ <version>1.6.2-SNAPSHOT</version>
<relativePath>../oak-parent/pom.xml</relativePath>
</parent>
<modelVersion>4.0.0</modelVersion>
Modified: jackrabbit/oak/branches/1.6/oak-doc/src/site/markdown/osgi_config.md
URL:
http://svn.apache.org/viewvc/jackrabbit/oak/branches/1.6/oak-doc/src/site/markdown/osgi_config.md?rev=1793609&r1=1793608&r2=1793609&view=diff
==============================================================================
--- jackrabbit/oak/branches/1.6/oak-doc/src/site/markdown/osgi_config.md
(original)
+++ jackrabbit/oak/branches/1.6/oak-doc/src/site/markdown/osgi_config.md Wed
May 3 05:20:45 2017
@@ -433,6 +433,7 @@ All the above data stores enable local f
* _PID `org.apache.jackrabbit.oak.plugins.blob.datastore.SharedS3DataStore`_
* _PID `org.apache.jackrabbit.oak.plugins.blob.datastore.S3DataStore`_
* _PID `org.apache.jackrabbit.oak.plugins.blob.datastore.FileDataStore`_
+* _PID `org.apache.jackrabbit.oak.plugins.blob.datastore.AzureDataStore`_
cacheSize
: Default - 68719476736
Modified:
jackrabbit/oak/branches/1.6/oak-doc/src/site/markdown/plugins/blobstore.md
URL:
http://svn.apache.org/viewvc/jackrabbit/oak/branches/1.6/oak-doc/src/site/markdown/plugins/blobstore.md?rev=1793609&r1=1793608&r2=1793609&view=diff
==============================================================================
--- jackrabbit/oak/branches/1.6/oak-doc/src/site/markdown/plugins/blobstore.md
(original)
+++ jackrabbit/oak/branches/1.6/oak-doc/src/site/markdown/plugins/blobstore.md
Wed May 3 05:20:45 2017
@@ -78,6 +78,7 @@ Further Oak ships with multiple BlobStor
4. `S3DataStore` (with wrapper) - Stores the file in Amazon S3
5. `RDBBlobStore` - Store the file contents in chunks in a relational
databases. Typically used with
`DocumentNodeStore`when using a relational DB persistence
+6. `AzureDataStore` (with wrapper) - Stores the file in Microsoft Azure Blob
storage
In addition there are some more implementations which are considered
**experimental**
@@ -98,6 +99,7 @@ can be used
* FileDataStore - This should be used if the blobs/binaries have to be shared
between multiple
repositories. This would also be used when a JR2 repository is migrated to
Oak
* S3DataStore - This should be used when binaries are stored in Amazon S3
+* AzureDataStore - This should be used when binaries are stored in Microsoft
Azure Blob storage
#### DocumentNodeStore
@@ -112,7 +114,7 @@ one of the following can be used
#### Caching DataStore
-The DataStore implementations `S3DataStore` and `CachingFileDataStore` support
local file system caching for the
+The DataStore implementations `S3DataStore`,`CachingFileDataStore` and
`AzureDataStore` support local file system caching for the
files/blobs and extend the `AbstractSharedCachingDataStore` class which
implements the caching functionality. The
`CachingFileDataStore` is useful when the DataStore is on nfs.
The cache has a size limit and is configured by the `cacheSize` parameter.
@@ -198,6 +200,7 @@ Blob Garbage Collection(GC) is applicabl
* FileDataStore
* S3DataStore
* SharedS3DataStore (since Oak 1.2.0)
+ * AzureDataStore
Oak implements a Mark and Sweep based Garbage Collection logic.
@@ -224,7 +227,7 @@ The garbage collection can be triggered
<a name="blobid-tracker"></a>
#### Caching of Blob ids locally (Oak 1.6.x)
-For the `FileDataStore` and `S3DataStore` the blob ids are cached locally on
the disk when they are created which
+For the `FileDataStore`, `S3DataStore` and `AzureDataStore` the blob ids are
cached locally on the disk when they are created which
speeds up the 'Mark BlobStore' phase. The locally tracked ids are synchronized
with the data store periodically to enable
other cluster nodes or different repositories sharing the datastore to get a
consolidated list of all blob ids. The
interval of synchronization is defined by the OSGi configuration parameter
`blobTrackSnapshotIntervalInSecs` for the
@@ -249,13 +252,14 @@ following should be executed.
##### Registration
-On start of a repository configured to use a shared DataStore (same path or S3
bucket), a unique repository id is
+On start of a repository configured to use a shared DataStore (same path, S3
bucket or Azure container), a unique repository id is
generated and registered in the NodeStore as well as the DataStore.
In the DataStore this repository id is registered as an empty file with the
format `repository-[repository-id]`
(e.g. repository-988373a0-3efb-451e-ab4c-f7e794189273). This empty file is
created under:
* FileDataStore - Under the root directory configured for the datastore.
* S3DataStore - Under `META` folder in the S3 bucket configured.
+* AzureDataStore - Under `META` folder in the Azure container configured.
On start/configuration of all the repositories sharing the data store it
should be confirmed that the unique
repositoryId per repository is registered in the DataStore. Refer the section
below on [Checking Shared GC status](#check-shared-datastore-gc).
@@ -387,6 +391,7 @@ the steps:
* Remove the corresponding registered repository file
(`repository-[repositoryId]`) from the DataStore
* FileDataStore - Remove the file from the data store root directory.
* S3DataStore - Remove the file from the `META` folder of the S3 bucket.
+ * AzureDataStore - Remove the file from the `META` folder of the Azure
container.
* Remove other files corresponding to the particular repositoryId e.g.
`markedTimestamp-[repositoryId]` or
`references-[repositoryId]`.
Modified: jackrabbit/oak/branches/1.6/oak-parent/pom.xml
URL:
http://svn.apache.org/viewvc/jackrabbit/oak/branches/1.6/oak-parent/pom.xml?rev=1793609&r1=1793608&r2=1793609&view=diff
==============================================================================
--- jackrabbit/oak/branches/1.6/oak-parent/pom.xml (original)
+++ jackrabbit/oak/branches/1.6/oak-parent/pom.xml Wed May 3 05:20:45 2017
@@ -177,6 +177,7 @@
<filter>!org.apache.jackrabbit.oak.spi.blob.split</filter>
<filter>!org.apache.jackrabbit.oak.spi.blob.stats</filter>
<filter>!org.apache.jackrabbit.oak.blob.cloud.aws.s3</filter>
+
<filter>!org.apache.jackrabbit.oak.blob.cloud.azure.blobstorage</filter>
<filter>!org.apache.jackrabbit.oak</filter>
<filter>!org.apache.jackrabbit.oak.json</filter>
<filter>!org.apache.jackrabbit.oak.management</filter>
Modified: jackrabbit/oak/branches/1.6/pom.xml
URL:
http://svn.apache.org/viewvc/jackrabbit/oak/branches/1.6/pom.xml?rev=1793609&r1=1793608&r2=1793609&view=diff
==============================================================================
--- jackrabbit/oak/branches/1.6/pom.xml (original)
+++ jackrabbit/oak/branches/1.6/pom.xml Wed May 3 05:20:45 2017
@@ -40,6 +40,7 @@
<module>oak-commons</module>
<module>oak-blob</module>
<module>oak-blob-cloud</module>
+ <module>oak-blob-cloud-azure</module>
<module>oak-core</module>
<module>oak-jcr</module>
<module>oak-upgrade</module>
