Very good! You are correct, I was clicking too fast when creating controllers. I thought it must have been something else as it showed that it's trying to read inexisting file.
Thanks for pointing me to correct source of problem! Martin. 2018-01-22 17:08 GMT+01:00 Joe Witt <[email protected]>: > hello > > just appears to be failing because it is trying to read the data as an avro > object with embedded schema and it appears to not actually be that. > > can you share the data it is failing on? and a flow template? > > thanks > > On Jan 22, 2018 11:06 AM, "Martin Mucha" <[email protected]> wrote: > > > Hi, > > > > I'd like to ask what this could mean. ValidateRecord processor from > master > > branch generates this error for me. I admit, I did some alterations to it > > in past, but I checkouted to origin/master, run mvn clean and started > > completely from scratch only with data in master branch. > > > > Question: is there something outside of nifi folder which has to be > > cleaned? Some /var/whatever or /usr/lib/whatever? > > > > If not -- this message simply means, that in content repository is > > referenced file, which does not exist. > > > > Exception below is taken from log and debugger reveals more; actual error > > happens here: > > sun.nio.fs.UnixException#rethrowAsIOException(sun.nio.fs.UnixPath, > > sun.nio.fs.UnixPath) > > > > I googled out, that it can be caused by wrong limits, so I set them > > appropriately, but issue did not go away: > > > > ulimit -a > > core file size (blocks, -c) unlimited > > data seg size (kbytes, -d) unlimited > > scheduling priority (-e) 0 > > file size (blocks, -f) unlimited > > pending signals (-i) 29595 > > max locked memory (kbytes, -l) 64 > > max memory size (kbytes, -m) unlimited > > open files (-n) 50000 > > pipe size (512 bytes, -p) 8 > > POSIX message queues (bytes, -q) 819200 > > real-time priority (-r) 0 > > stack size (kbytes, -s) 8192 > > cpu time (seconds, -t) unlimited > > max user processes (-u) 10000 > > virtual memory (kbytes, -v) unlimited > > file locks (-x) unlimited > > > > > > from log: > > > > > > 2018-01-22 15:44:35,416 ERROR [Timer-Driven Process Thread-10] > > o.a.n.processors.standard.ValidateRecord > > ValidateRecord[id=1e25a850-0161-1000-788b-4f427cfb5631] > > Failed to process StandardFlow > > FileRecord[uuid=43100221-8ea6-48c4-abe1-d51965ecd8ae,claim= > > StandardContentClaim > > [resourceClaim=StandardResourceClaim[id=1516632234823-1, > > container=default, > > section=1], offset=34, length=17], > > offset=0,name=808180285294,size=17]; will route to failure: > > java.io.IOException: Not a data file. > > java.io.IOException: Not a data file. > > at org.apache.avro.file.DataFileStream.initialize( > > DataFileStream.java:105) > > at org.apache.avro.file.DataFileStream.<init>( > > DataFileStream.java:84) > > at org.apache.nifi.avro.AvroReaderWithEmbeddedSchema.<init>( > > AvroReaderWithEmbeddedSchema.java:38) > > at org.apache.nifi.avro.AvroReader.createRecordReader( > > AvroReader.java:89) > > at org.apache.nifi.serialization.RecordReaderFactory. > > createRecordReader(RecordReaderFactory.java:46) > > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > > at sun.reflect.NativeMethodAccessorImpl.invoke( > > NativeMethodAccessorImpl.java:62) > > at sun.reflect.DelegatingMethodAccessorImpl.invoke( > > DelegatingMethodAccessorImpl.java:43) > > at java.lang.reflect.Method.invoke(Method.java:498) > > at org.apache.nifi.controller.service. > > StandardControllerServiceInvoc > > ationHandler.invoke(StandardControllerServiceInvocationHandler.java:89) > > at com.sun.proxy.$Proxy81.createRecordReader(Unknown Source) > > at org.apache.nifi.processors.standard.ValidateRecord. > > onTrigger(ValidateRecord.java:295) > > at org.apache.nifi.processor.AbstractProcessor.onTrigger( > > AbstractProcessor.java:27) > > at org.apache.nifi.controller.StandardProcessorNode.onTrigger( > > StandardProcessorNode.java:1122) > > at org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask. > > call(ContinuallyRunProcessorTask.java:147) > > at org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask. > > call(ContinuallyRunProcessorTask.java:47) > > at org.apache.nifi.controller.scheduling. > > TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:128) > > at java.util.concurrent.Executors$RunnableAdapter. > > call(Executors.java:511) > > at java.util.concurrent.FutureTask.runAndReset( > > FutureTask.java:308) > > at java.util.concurrent.ScheduledThreadPoolExecutor$ > > ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) > > at java.util.concurrent.ScheduledThreadPoolExecutor$ > > ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) > > at java.util.concurrent.ThreadPoolExecutor.runWorker( > > ThreadPoolExecutor.java:1149) > > at java.util.concurrent.ThreadPoolExecutor$Worker.run( > > ThreadPoolExecutor.java:624) > > at java.lang.Thread.run(Thread.java:748) > > >
