Thanks James. So when you say the JRE security provider, are you referring to bouncycastle? If so, I am currently using bcprov-jdk16-1.46.jar.
Thanks, Dan M From: James Wing <[email protected]> Reply-To: <[email protected]> Date: Wednesday, August 17, 2016 at 11:45 PM To: <[email protected]> Subject: Re: PutS3Object Error The IllegalStateException appears to be a response to a CloneNotSupportException thrown by java.security.MessageDigest. That suggests that your JRE's java.security provider does not support the clone method used by the AWS SDK. Is your java.security setup the JRE default? Would it be possible to try a different, maybe newer, JRE? And it's always a bonus when the code throwing your exception is foreshadowed with the ominous "should never occur": private MessageDigest cloneFrom(MessageDigest from) { try { return (MessageDigest)from.clone(); } catch (CloneNotSupportedException e) { // should never occur throw new IllegalStateException("unexpected", e); } } (see https://github.com/aws/aws-sdk-java/blob/master/aws-java-sdk-s3/src/main/java/com/amazonaws/services/s3/internal/MD5DigestCalculatingInputStream.java#L54) Thanks, James On Wed, Aug 17, 2016 at 5:40 PM, dgm <[email protected]> wrote: Sure thing! Version: Nifi 0.7.0, RHEL 6.8, with Java JRE 1.7.0_101 Logs/Stacktrace: 2016-08-17 19:01:09,339 INFO [Timer-Driven Process Thread-3] o.a.n.p.attributes.UpdateAttribute UpdateAttribute[id=7b485633-a6e3-4aee- a671-d5b6f2599040] Updated attributes for StandardFlowFileRecord[uuid=bda2e6c2-7a20-4354-a65f-3ed5b9ef4d06,claim=StandardContentClaim [resourceClaim=StandardResourceClaim[id=1471471716616-15, container=default, section=15], offset=44598, length=3467],offset=0, name=20160817190051056-low-text_json_good.json,size=3467]; transferring to 'success' 29 2016-08-17 19:01:09,346 ERROR [Timer-Driven Process Thread-3] o.a.nifi.processors.aws.s3.PutS3Object PutS3Object[id=a0836995-c787-40b1- a8bb-c6f39bb9f37a] PutS3Object[id=a0836995-c787-40b1-a8bb-c6f39bb9f37a] failed to process due to java.lang.IllegalStateException: unexpected; rolling back session: java.lang.IllegalStateException: unexpected 30 2016-08-17 19:01:09,349 ERROR [Timer-Driven Process Thread-3] o.a.nifi.processors.aws.s3.PutS3Object 31 java.lang.IllegalStateException: unexpected 32 at com.amazonaws.services.s3.internal.MD5DigestCalculatingInputStream.cloneFrom(MD5DigestCalculatingInputStream.java:54) ~[na:na] 33 at com.amazonaws.services.s3.internal.MD5DigestCalculatingInputStream.mark(MD5DigestCalculatingInputStream.java:66) ~[na:na] 34 at com.amazonaws.internal.SdkFilterInputStream.mark(SdkFilterInputStream.java:96) ~[na:na] 35 at com.amazonaws.internal.SdkFilterInputStream.mark(SdkFilterInputStream.java:96) ~[na:na] 36 at com.amazonaws.http.AmazonHttpClient.executeHelper(AmazonHttpClient.java:472) ~[na:na] 37 at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:317) ~[na:na] 38 at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:3595) ~[na:na] 39 at com.amazonaws.services.s3.AmazonS3Client.putObject(AmazonS3Client.java:1382) ~[na:na] 40 at org.apache.nifi.processors.aws.s3.PutS3Object$1.process(PutS3Object.java:446) ~[na:na] 41 at org.apache.nifi.controller.repository.StandardProcessSession.read(StandardProcessSession.java:1851) ~[nifi-framework-core-0.7.0. jar:0.7.0] 42 at org.apache.nifi.controller.repository.StandardProcessSession.read(StandardProcessSession.java:1822) ~[nifi-framework-core-0.7.0. jar:0.7.0] 43 at org.apache.nifi.processors.aws.s3.PutS3Object.onTrigger(PutS3Object.java:400) ~[na:na] 44 at org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27) ~[nifi-api-0.7.0.jar:0.7.0] 45 at org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1054) [nifi-framework-core-0.7.0.jar:0.7.0] 46 at org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:136) [nifi-framework-core-0.7.0. jar:0.7.0] 47 at org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:47) [nifi-framework-core-0.7.0. jar:0.7.0] 48 at org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:127) [nifi-framework-core-0. 7.0.jar:0.7.0] 49 at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [na:1.7.0_101] 50 at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304) [na:1.7.0_101] 51 at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178) [na:1.7. 0_101] 52 at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) [na:1.7.0_101] 53 at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [na:1.7.0_101] 54 at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [na:1.7.0_101] 55 at java.lang.Thread.run(Thread.java:745) [na:1.7.0_101] 56 Caused by: java.lang.CloneNotSupportedException: null 57 at java.security.MessageDigest$Delegate.clone(MessageDigest.java:571) ~[na:1.7.0_101] 58 at com.amazonaws.services.s3.internal.MD5DigestCalculatingInputStream.cloneFrom(MD5DigestCalculatingInputStream.java:52) ~[na:na] 59 ... 23 common frames omitted 60 2016-08-17 19:01:09,350 ERROR [Timer-Driven Process Thread-3] o.a.nifi.processors.aws.s3.PutS3Object PutS3Object[id=a0836995-c787-40b1- a8bb-c6f39bb9f37a] PutS3Object[id=a0836995-c787-40b1-a8bb-c6f39bb9f37a] failed to process session due to java.lang.IllegalStateException: unexpected: java.lang.IllegalStateException: unexpected 61 2016-08-17 19:01:09,354 ERROR [Timer-Driven Process Thread-3] o.a.nifi.processors.aws.s3.PutS3Object 62 java.lang.IllegalStateException: unexpected 63 at com.amazonaws.services.s3.internal.MD5DigestCalculatingInputStream.cloneFrom(MD5DigestCalculatingInputStream.java:54) ~[na:na] 64 at com.amazonaws.services.s3.internal.MD5DigestCalculatingInputStream.mark(MD5DigestCalculatingInputStream.java:66) ~[na:na] 65 at com.amazonaws.internal.SdkFilterInputStream.mark(SdkFilterInputStream.java:96) ~[na:na] 66 at com.amazonaws.internal.SdkFilterInputStream.mark(SdkFilterInputStream.java:96) ~[na:na] 67 at com.amazonaws.http.AmazonHttpClient.executeHelper(AmazonHttpClient.java:472) ~[na:na] 68 at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:317) ~[na:na] 69 at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:3595) ~[na:na] 70 at com.amazonaws.services.s3.AmazonS3Client.putObject(AmazonS3Client.java:1382) ~[na:na] 71 at org.apache.nifi.processors.aws.s3.PutS3Object$1.process(PutS3Object.java:446) ~[na:na] 72 at org.apache.nifi.controller.repository.StandardProcessSession.read(StandardProcessSession.java:1851) ~[nifi-framework-core-0.7.0. jar:0.7.0] 73 at org.apache.nifi.controller.repository.StandardProcessSession.read(StandardProcessSession.java:1822) ~[nifi-framework-core-0.7.0. jar:0.7.0] 74 at org.apache.nifi.processors.aws.s3.PutS3Object.onTrigger(PutS3Object.java:400) ~[na:na] 75 at org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27) ~[nifi-api-0.7.0.jar:0.7.0] 76 at org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1054) ~[nifi-framework-core-0.7.0.jar:0.7.0] 77 at org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:136) [nifi-framework-core-0.7.0. jar:0.7.0] 78 at org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:47) [nifi-framework-core-0.7.0. jar:0.7.0] 79 at org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:127) [nifi-framework-core-0. 7.0.jar:0.7.0] 80 at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [na:1.7.0_101] 81 at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304) [na:1.7.0_101] 82 at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178) [na:1.7. 0_101] 83 at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) [na:1.7.0_101] 84 at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [na:1.7.0_101] 85 at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [na:1.7.0_101] 86 at java.lang.Thread.run(Thread.java:745) [na:1.7.0_101] 87 Caused by: java.lang.CloneNotSupportedException: null 88 at java.security.MessageDigest$Delegate.clone(MessageDigest.java:571) ~[na:1.7.0_101] 89 at com.amazonaws.services.s3.internal.MD5DigestCalculatingInputStream.cloneFrom(MD5DigestCalculatingInputStream.java:52) ~[na:na] 90 ... 23 common frames omitted 91 2016-08-17 19:01:09,354 WARN [Timer-Driven Process Thread-3] o.a.nifi.processors.aws.s3.PutS3Object PutS3Object[id=a0836995-c787-40b1- a8bb-c6f39bb9f37a] Processor Administratively Yielded for 1 sec due to processing failure Thanks, Dan M From: James Wing <[email protected]> Reply-To: <[email protected]> Date: Wednesday, August 17, 2016 at 8:32 PM To: <[email protected]> Subject: Re: PutS3Object Error Dan, Would you please share the version of NiFi you are using? Also, would you please look in logs/nifi-app.log for the stack trace of the exception and any nested exceptions? Thanks, James On Wed, Aug 17, 2016 at 5:05 PM, dgm <[email protected]> wrote: I’m just staring to use nifi and having an issue with the PutS3Object processor. I believe that it's configured consistently with my other S3 processors, using the same accounts/permissions... receiving the following error in nifi: PutS3Object[id=a3d6e547-9f71-4bcd-8873-108575604ab4] PutS3Object[id=a3d6e547-9f71-4bcd-8873-108575604ab4] failed to process session due to java.lang.IllegalStateException: unexpected: java.lang.IllegalStateException: unexpected When I configured the processor, I’m completing all the mandatory fields (i.e. object key, bucket, access key, secret key, storage class, and region). I’ve made sure these are consistent with my other S3 processors (List and Fetch – of which I can successful retrieve files). Any help or guidance would be greatly appreciated! Thanks, Dan M
