[
https://issues.apache.org/jira/browse/NIFI-10235?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17572106#comment-17572106
]
David Handermann commented on NIFI-10235:
-----------------------------------------
On further evaluation, the problem is the result of the Content Claim length
not being calculated until the Encrypted Content OutputStream is closed. This
does not happen until the framework commits the ProcessSession, due to the way
the StandardContentClaimWriteCache blocks calls to OutputStream.close().
Updating the StandardContentClaimWriteCache implementation, in conjunction with
introducing a new subclass of OutputStream, should provide a way forward.
> Provenance replay fails when repository encryption is enabled
> -------------------------------------------------------------
>
> Key: NIFI-10235
> URL: https://issues.apache.org/jira/browse/NIFI-10235
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core Framework, Security
> Affects Versions: 1.16.3
> Environment: RHEL 8.5
> Reporter: Peter Kimberley
> Assignee: David Handermann
> Priority: Major
> Labels: encryption, provenance, replay
> Attachments: NiFi_Flow.json, error-base-install.log, error.log
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> h3. Problem summary
> When repository encryption is enabled, replaying a DROP provenance record
> fails, with the following error appearing in the logs:
> {quote}org.apache.nifi.processor.exception.FlowFileAccessException: Failed to
> export
> StandardFlowFileRecord[uuid=df985fc5-23da-4094-8783-2e0186bcb92d,claim=StandardContentClaim
> [resourceClaim=StandardResourceClaim[id=1657864218374-23, container=default,
> section=23], offset=379,
> length=1048576],offset=0,name=b29633c4-324e-42fe-b3e8-1ea455fc3650,size=1048576]
> to /opt/nifi/nifi-current/data/store/.b29633c4-324e-42fe-b3e8-1ea455fc3650
> due to java.io.EOFException: *Attempted to copy {color:#ff8b00}1048576{color}
> bytes but only {color:#ff8b00}1048197{color} bytes were available*
> {quote}
>
> I've observed that the difference between the sizes mentioned in the log is
> {+}*always 379 bytes*{+}, regardless of the length of the input file.
>
> With repository encryption disabled, provenance replay works as expected.
> h3. Configuration
> # NiFi v1.16.3 running as a three-node cluster in Kubernetes.
> # Each node has up to 8GB memory and 4 CPUs available to it.
> # Testing has included both NFS and ephemeral (emptyDir) storage.
> # The encryption key was generated by the following command, using the same
> JDK version:
> ## keytool -genseckey -alias key-1 -keyalg AES -keysize 256 -keystore
> repository.p12 -storetype PKCS12
> h4. nifi.properties
> {quote}nifi.repository.encryption.protocol.version=1
> nifi.repository.encryption.key.id=key-1
> nifi.repository.encryption.key.provider=KEYSTORE
> nifi.repository.encryption.key.provider.keystore.location=conf/repository.p12
> nifi.repository.encryption.key.provider.keystore.password=<password>
> {quote}
> h3. Processor group
> GenerateFlowFile processor generating 1MB random files every second to a
> PutFile processor. Have also tested with InvokeHTTP.
> h3. Other comments
> With repository encryption enabled, I am able to download files via the
> provenance UI (suggesting that encryption/decryption works). The processor
> group also performs all other actions as expected.
> Not having the ability to replay provenance records is a blocker for our
> deployment, which requires data to be encrypted at rest and in transit.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
