[jira] [Comment Edited] (OAK-3148) Online migration process for the binaries

2015-08-31 Thread JIRA

[ 
https://issues.apache.org/jira/browse/OAK-3148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14716524#comment-14716524
 ] 

Tomek Rękawek edited comment on OAK-3148 at 8/31/15 11:08 AM:
--

I prepared a [pull request|https://github.com/apache/jackrabbit-oak/pull/36] 
with this feature. I tested it with the recent AEM and it took 10 minutes to 
migrate blobs from {{FileDataStore}} to {{FileBlobStore}}. More information can 
be found in the following migration scenario for an OSGi-based installation:

*Requirements*
* An OSGi-based Oak installation (eg. Sling or AEM).
* Node store with a external or internal blob store

*1a. Enabling SplitBlobStore - external blob store case*
Steps:
# Add {{split.blobstore=old}} OSGi property to the source blob store.
# Configure the destination blob store and add {{split.blobstore=new}} property 
to its OSGi configuration.
# Create a configuration for the 
{{org.apache.jackrabbit.oak.spi.blob.osgi.SplitBlobStoreService}}.
{code}
split.old.blobstore.type=INTERNAL
# optional:
repository.home=crx-quickstart/repository
{code}
#* The directory is used to save the {{migrated_blobs.txt}} file.
# (optional) Restart the instance

*1b. Enabling SplitBlobStore - internal blob store case*
Steps:
# Configure the destination blob store and add {{split.blobstore=new}} property 
to its OSGi configuration.
# Create a configuration for the 
{{org.apache.jackrabbit.oak.spi.blob.osgi.SplitBlobStoreService}}.
{code}
# use DOCUMENT or SEGMENT, depending on the NodeStore type:
split.old.blobstore.type=SEGMENT
# optional:
repository.home=crx-quickstart/repository
{code}
#* The directory is used to save the {{migrated_blobs.txt}} file.
# (optional) Restart the instance

After starting the instance, the {{SplitBlobStoreService}} will wait until blob 
stores with {{split.blobstore}} properties (the {{old}} and the {{new}}) are 
available. They will be bound and the {{SplitBlobStore}} will be registered in 
the OSGi. On the other hand, the {{NodeStoreService}} will ignore blob stores 
configured with the {{split.blobstore}} property and will wait until the 
{{SplitBlobStore}} is available.

>From this point, all the new blobs will be saved in the new blob store. 
>Binaries from the old blob store will be available to read.

The {{split.blobstore}} property support was added to {{FileBlobStore}}, 
{{AbstractDataStoreService}} (handling all Jackrabbit data stores), 
{{DocumentNodeStoreService}} and {{SegmentNodeStoreService}}.

*2. Migration*
Steps:
# Find BlobMigration JMX bean in the Felix console.
# Run {{startBlobMigration(false)}} operation

The migration can be stopped using {{stopBlobMigration()}} and then resumed 
with {{startBlobMigration(true)}}. The current stats are available via the JMX 
as well:

* last processed path,
* number of migrated nodes.

*3. Switching to the new blob store*
When the migration is finished, it's possible to completely switch to the new 
blob store:
# Remove the configuration for the old blob store.
# Remove the configuration for the {{SplitBlobStoreService}}
# Remove the {{split.blobstore=new}} OSGi property from the new blob store, so 
it can be find by the {{NodeStoreService}}.
# (optional) Restart the instance, so there are no JCR sessions bound to the 
old {{NodeState}}.

Migration is complete!


was (Author: tomek.rekawek):
I prepared a [pull request|https://github.com/apache/jackrabbit-oak/pull/36] 
with this feature. I tested it with the recent AEM and it took 10 minutes to 
migrate blobs from {{FileDataStore}} to {{FileBlobStore}}. More information can 
be found in the following migration scenario for an OSGi-based installation:

*Requirements*
* An OSGi-based Oak installation (eg. Sling or AEM).
* Node store should be configured to use an external blob store 
(customBlobStore=true).

*1. Enabling SplitBlobStore*
Steps:
# Add {{split.blobstore=old}} OSGi property to the source blob store.
# Configure the destination blob store and add {{split.blobstore=new}} property 
to its OSGi configuration.
# Create a configuration for the 
{{org.apache.jackrabbit.oak.spi.blob.osgi.SplitBlobStoreService}}.
#* It may be empty or contain just one parameter:
{code}
repository.home=crx-quickstart/repository
{code}
#* The directory is used to save the {{migrated_blobs.txt}} file.
# (optional) Restart the instance

After starting the instance, the {{SplitBlobStoreService}} will wait until blob 
stores with {{split.blobstore}} properties (the {{old}} and the {{new}}) are 
available. They will be bound and the {{SplitBlobStore}} will be registered in 
the OSGi. On the other hand, the {{NodeStoreService}} will ignore blob stores 
configured with the {{split.blobstore}} property and will wait until the 
{{SplitBlobStore}} is available.

>From this point, all the new blobs will be saved in the new blob store. 
>Binaries from the old blob store will be available to read.

Th

[jira] [Comment Edited] (OAK-3148) Online migration process for the binaries

2015-08-27 Thread JIRA

[ 
https://issues.apache.org/jira/browse/OAK-3148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14716524#comment-14716524
 ] 

Tomek Rękawek edited comment on OAK-3148 at 8/27/15 11:58 AM:
--

I prepared a [pull request|https://github.com/apache/jackrabbit-oak/pull/36] 
with this feature. I tested it with the recent AEM and it took 10 minutes to 
migrate blobs from {{FileDataStore}} to {{FileBlobStore}}. More information can 
be found in the following migration scenario for an OSGi-based installation:

*Requirements*
* An OSGi-based Oak installation (eg. Sling or AEM).
* Node store should be configured to use an external blob store 
(customBlobStore=true).

*1. Enabling SplitBlobStore*
Steps:
# Add {{split.blobstore=old}} OSGi property to the source blob store.
# Configure the destination blob store and add {{split.blobstore=new}} property 
to its OSGi configuration.
# Create a configuration for the 
{{org.apache.jackrabbit.oak.spi.blob.osgi.SplitBlobStoreService}}.
#* It may be empty or contain just one parameter:
{code}
repository.home=crx-quickstart/repository
{code}
#* The directory is used to save the {{migrated_blobs.txt}} file.
# (optional) Restart the instance

After starting the instance, the {{SplitBlobStoreService}} will wait until blob 
stores with {{split.blobstore}} properties (the {{old}} and the {{new}}) are 
available. They will be bound and the {{SplitBlobStore}} will be registered in 
the OSGi. On the other hand, the {{NodeStoreService}} will ignore blob stores 
configured with the {{split.blobstore}} property and will wait until the 
{{SplitBlobStore}} is available.

>From this point, all the new blobs will be saved in the new blob store. 
>Binaries from the old blob store will be available to read.

The {{split.blobstore}} property support was added to {{FileBlobStore}}, 
{{AbstractDataStoreService}} (handling all Jackrabbit data stores), 
{{DocumentNodeStoreService}} and {{SegmentNodeStoreService}}.

*2. Migration*
Steps:
# Find BlobMigration JMX bean in the Felix console.
# Run {{startBlobMigration(false)}} operation

The migration can be stopped using {{stopBlobMigration()}} and then resumed 
with {{startBlobMigration(true)}}. The current stats are available via the JMX 
as well:

* last processed path,
* number of migrated nodes.

*3. Switching to the new blob store*
When the migration is finished, it's possible to completely switch to the new 
blob store:
# Remove the configuration for the old blob store.
# Remove the configuration for the {{SplitBlobStoreService}}
# Remove the {{split.blobstore=new}} OSGi property from the new blob store, so 
it can be find by the {{NodeStoreService}}.
# (optional) Restart the instance, so there are no JCR sessions bound to the 
old {{NodeState}}.

Migration is complete!


was (Author: tomek.rekawek):
I prepared a [PR|https://github.com/apache/jackrabbit-oak/pull/36]. I tested it 
with the recent AEM and it took 10 minutes to migrate blobs from 
{{FileDataStore}} to {{FileBlobStore}}. More information can be found in the 
following migration scenario for an OSGi-based installation:

*Requirements*
* An OSGi-based Oak installation (eg. Sling or AEM).
* Node store should be configured to use an external blob store 
(customBlobStore=true).

*1. Enabling SplitBlobStore*
Steps:
# Add {{split.blobstore=old}} OSGi property to the source blob store.
# Configure the destination blob store and add {{split.blobstore=new}} property 
to its OSGi configuration.
# Create a configuration for the 
{{org.apache.jackrabbit.oak.spi.blob.osgi.SplitBlobStoreService}}.
#* It may be empty or contain just one parameter:
{code}
repository.home=crx-quickstart/repository
{code}
#* The directory is used to save the {{migrated_blobs.txt}} file.
# (optional) Restart the instance

After starting the instance, the {{SplitBlobStoreService}} will wait until blob 
stores with {{split.blobstore}} properties (the {{old}} and the {{new}}) are 
available. They will be bound and the {{SplitBlobStore}} will be registered in 
the OSGi. On the other hand, the {{NodeStoreService}} will ignore blob stores 
configured with the {{split.blobstore}} property and will wait until the 
{{SplitBlobStore}} is available.

>From this point, all the new blobs will be saved in the new blob store. 
>Binaries from the old blob store will be available to read.

The {{split.blobstore}} property support was added to {{FileBlobStore}}, 
{{AbstractDataStoreService}} (handling all Jackrabbit data stores), 
{{DocumentNodeStoreService}} and {{SegmentNodeStoreService}}.

*2. Migration*
Steps:
# Find BlobMigration JMX bean in the Felix console.
# Run {{startBlobMigration(false)}} operation

The migration can be stopped using {{stopBlobMigration()}} and then resumed 
with {{startBlobMigration(true)}}. The current stats are available via the JMX 
as well:

* last processed path,
* number of migrated nodes.

*3. Sw

[jira] [Comment Edited] (OAK-3148) Online migration process for the binaries

2015-08-27 Thread JIRA

[ 
https://issues.apache.org/jira/browse/OAK-3148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14716524#comment-14716524
 ] 

Tomek Rękawek edited comment on OAK-3148 at 8/27/15 11:56 AM:
--

I prepared a [PR|https://github.com/apache/jackrabbit-oak/pull/36]. I tested it 
with the recent AEM and it took 10 minutes to migrate blobs from 
{{FileDataStore}} to {{FileBlobStore}}. More information can be found in the 
following migration scenario for an OSGi-based installation:

*Requirements*
* An OSGi-based Oak installation (eg. Sling or AEM).
* Node store should be configured to use an external blob store 
(customBlobStore=true).

*1. Enabling SplitBlobStore*
Steps:
# Add {{split.blobstore=old}} OSGi property to the source blob store.
# Configure the destination blob store and add {{split.blobstore=new}} property 
to its OSGi configuration.
# Create a configuration for the 
{{org.apache.jackrabbit.oak.spi.blob.osgi.SplitBlobStoreService}}.
#* It may be empty or contain just one parameter:
{code}
repository.home=crx-quickstart/repository
{code}
#* The directory is used to save the {{migrated_blobs.txt}} file.
# (optional) Restart the instance

After starting the instance, the {{SplitBlobStoreService}} will wait until blob 
stores with {{split.blobstore}} properties (the {{old}} and the {{new}}) are 
available. They will be bound and the {{SplitBlobStore}} will be registered in 
the OSGi. On the other hand, the {{NodeStoreService}} will ignore blob stores 
configured with the {{split.blobstore}} property and will wait until the 
{{SplitBlobStore}} is available.

>From this point, all the new blobs will be saved in the new blob store. 
>Binaries from the old blob store will be available to read.

The {{split.blobstore}} property support was added to {{FileBlobStore}}, 
{{AbstractDataStoreService}} (handling all Jackrabbit data stores), 
{{DocumentNodeStoreService}} and {{SegmentNodeStoreService}}.

*2. Migration*
Steps:
# Find BlobMigration JMX bean in the Felix console.
# Run {{startBlobMigration(false)}} operation

The migration can be stopped using {{stopBlobMigration()}} and then resumed 
with {{startBlobMigration(true)}}. The current stats are available via the JMX 
as well:

* last processed path,
* number of migrated nodes.

*3. Switching to the new blob store*
When the migration is finished, it's possible to completely switch to the new 
blob store:
# Remove the configuration for the old blob store.
# Remove the configuration for the {{SplitBlobStoreService}}
# Remove the {{split.blobstore=new}} OSGi property from the new blob store, so 
it can be find by the {{NodeStoreService}}.
# (optional) Restart the instance, so there are no JCR sessions bound to the 
old {{NodeState}}.

Migration is complete!


was (Author: tomek.rekawek):
I prepared a patch with this feature. I tested it with the recent AEM and it 
took 10 minutes to migrate blobs from {{FileDataStore}} to {{FileBlobStore}}. 
More information can be found in the following migration scenario for an 
OSGi-based installation:

*Requirements*
* An OSGi-based Oak installation (eg. Sling or AEM).
* Node store should be configured to use an external blob store 
(customBlobStore=true).

*1. Enabling SplitBlobStore*
Steps:
# Add {{split.blobstore=old}} OSGi property to the source blob store.
# Configure the destination blob store and add {{split.blobstore=new}} property 
to its OSGi configuration.
# Create a configuration for the 
{{org.apache.jackrabbit.oak.spi.blob.osgi.SplitBlobStoreService}}.
#* It may be empty or contain just one parameter:
{code}
repository.home=crx-quickstart/repository
{code}
#* The directory is used to save the {{migrated_blobs.txt}} file.
# (optional) Restart the instance

After starting the instance, the {{SplitBlobStoreService}} will wait until blob 
stores with {{split.blobstore}} properties (the {{old}} and the {{new}}) are 
available. They will be bound and the {{SplitBlobStore}} will be registered in 
the OSGi. On the other hand, the {{NodeStoreService}} will ignore blob stores 
configured with the {{split.blobstore}} property and will wait until the 
{{SplitBlobStore}} is available.

>From this point, all the new blobs will be saved in the new blob store. 
>Binaries from the old blob store will be available to read.

The {{split.blobstore}} property support was added to {{FileBlobStore}}, 
{{AbstractDataStoreService}} (handling all Jackrabbit data stores), 
{{DocumentNodeStoreService}} and {{SegmentNodeStoreService}}.

*2. Migration*
Steps:
# Find BlobMigration JMX bean in the Felix console.
# Run {{startBlobMigration(false)}} operation

The migration can be stopped using {{stopBlobMigration()}} and then resumed 
with {{startBlobMigration(true)}}. The current stats are available via the JMX 
as well:

* last processed path,
* number of migrated nodes.

*3. Switching to the new blob store*
When the migration is finis