[
https://issues.apache.org/jira/browse/FLUME-1577?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hari Shreedharan updated FLUME-1577:
------------------------------------
Description:
Since we do not sync to disk immediately after setting CHECKPOINT_INCOMPLETE at
the checkpoint marker, system failure(not JVM exit), can cause bad checkpoints.
was:
Currently our config looks like:
a1.channels.c1.type = FILE
a1.channels.c1.checkpointDir =
/var/run/flume-ng/.flume/file-channel-c1/checkpoint
a1.channels.c1.dataDirs = /var/run/flume-ng/.flume/file-channel-c1/data
a1.channels.c1.encryption.keyAlias = key-1
a1.channels.c1.encryption.cipherProvider = AESCTRNOPADDING
a1.channels.c1.encryption.keyProvider = p0
a1.channels.c1.encryption.keyProvider.p0.type = JCEKSFILE
a1.channels.c1.encryption.keyProvider.p0.keyStoreFile =
/etc/flume-ng/conf/flume.keystore
a1.channels.c1.encryption.keyProvider.p0.keyStorePasswordFile =
/etc/flume-ng/conf/key-1.password
a1.channels.c1.encryption.keyProvider.p0.keys = key-1
a1.channels.c1.encryption.keyProvider.p0.keys.key-1.passwordFile =
/etc/flume-ng/conf/key-1.password
a1.channels.c1.encryption.keyProvider.p0.keys.key-2.passwordFile =
/etc/flume-ng/conf/key-2.password
I propose changing 'keyAlias' to 'activeKey'. Also, AFAIK multiple
keyProvider's can't be specified for a single file channel config so I propose
removing the need for this line:
a1.channels.c1.encryption.keyProvider = p0
...and thus also removing the additional 'p0' namespace. And I don't think
there's any need for specifying a 'type' after keyProvider. So that the entire
file channel config will look like this:
a1.channels.c1.type = FILE
a1.channels.c1.checkpointDir =
/var/run/flume-ng/.flume/file-channel-c1/checkpoint
a1.channels.c1.dataDirs = /var/run/flume-ng/.flume/file-channel-c1/data
a1.channels.c1.encryption.activeKey = key-1
a1.channels.c1.encryption.cipherProvider = AESCTRNOPADDING
a1.channels.c1.encryption.keyProvider = JCEKSFILE
a1.channels.c1.encryption.keyProvider.keyStoreFile =
/etc/flume-ng/conf/flume.keystore
a1.channels.c1.encryption.keyProvider.keyStorePasswordFile =
/etc/flume-ng/conf/key-1.password
a1.channels.c1.encryption.keyProvider.keys = key-1 key-2
a1.channels.c1.encryption.keyProvider.keys.key-1.passwordFile =
/etc/flume-ng/conf/key-1.password
a1.channels.c1.encryption.keyProvider.keys.key-2.passwordFile =
/etc/flume-ng/conf/key-2.password
> CHECKPOINT_INCOMPLETE should be synced to disk before starting the checkpoint.
> ------------------------------------------------------------------------------
>
> Key: FLUME-1577
> URL: https://issues.apache.org/jira/browse/FLUME-1577
> Project: Flume
> Issue Type: Bug
> Components: Channel
> Affects Versions: v1.3.0
> Reporter: Will McQueen
> Assignee: Brock Noland
> Fix For: v1.3.0
>
> Attachments: FLUME-1577-2.patch
>
>
> Since we do not sync to disk immediately after setting CHECKPOINT_INCOMPLETE
> at the checkpoint marker, system failure(not JVM exit), can cause bad
> checkpoints.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
