hello just appears to be failing because it is trying to read the data as an avro object with embedded schema and it appears to not actually be that.
can you share the data it is failing on? and a flow template? thanks On Jan 22, 2018 11:06 AM, "Martin Mucha" <[email protected]> wrote: > Hi, > > I'd like to ask what this could mean. ValidateRecord processor from master > branch generates this error for me. I admit, I did some alterations to it > in past, but I checkouted to origin/master, run mvn clean and started > completely from scratch only with data in master branch. > > Question: is there something outside of nifi folder which has to be > cleaned? Some /var/whatever or /usr/lib/whatever? > > If not -- this message simply means, that in content repository is > referenced file, which does not exist. > > Exception below is taken from log and debugger reveals more; actual error > happens here: > sun.nio.fs.UnixException#rethrowAsIOException(sun.nio.fs.UnixPath, > sun.nio.fs.UnixPath) > > I googled out, that it can be caused by wrong limits, so I set them > appropriately, but issue did not go away: > > ulimit -a > core file size (blocks, -c) unlimited > data seg size (kbytes, -d) unlimited > scheduling priority (-e) 0 > file size (blocks, -f) unlimited > pending signals (-i) 29595 > max locked memory (kbytes, -l) 64 > max memory size (kbytes, -m) unlimited > open files (-n) 50000 > pipe size (512 bytes, -p) 8 > POSIX message queues (bytes, -q) 819200 > real-time priority (-r) 0 > stack size (kbytes, -s) 8192 > cpu time (seconds, -t) unlimited > max user processes (-u) 10000 > virtual memory (kbytes, -v) unlimited > file locks (-x) unlimited > > > from log: > > > 2018-01-22 15:44:35,416 ERROR [Timer-Driven Process Thread-10] > o.a.n.processors.standard.ValidateRecord > ValidateRecord[id=1e25a850-0161-1000-788b-4f427cfb5631] > Failed to process StandardFlow > FileRecord[uuid=43100221-8ea6-48c4-abe1-d51965ecd8ae,claim= > StandardContentClaim > [resourceClaim=StandardResourceClaim[id=1516632234823-1, > container=default, > section=1], offset=34, length=17], > offset=0,name=808180285294,size=17]; will route to failure: > java.io.IOException: Not a data file. > java.io.IOException: Not a data file. > at org.apache.avro.file.DataFileStream.initialize( > DataFileStream.java:105) > at org.apache.avro.file.DataFileStream.<init>( > DataFileStream.java:84) > at org.apache.nifi.avro.AvroReaderWithEmbeddedSchema.<init>( > AvroReaderWithEmbeddedSchema.java:38) > at org.apache.nifi.avro.AvroReader.createRecordReader( > AvroReader.java:89) > at org.apache.nifi.serialization.RecordReaderFactory. > createRecordReader(RecordReaderFactory.java:46) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at sun.reflect.NativeMethodAccessorImpl.invoke( > NativeMethodAccessorImpl.java:62) > at sun.reflect.DelegatingMethodAccessorImpl.invoke( > DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:498) > at org.apache.nifi.controller.service. > StandardControllerServiceInvoc > ationHandler.invoke(StandardControllerServiceInvocationHandler.java:89) > at com.sun.proxy.$Proxy81.createRecordReader(Unknown Source) > at org.apache.nifi.processors.standard.ValidateRecord. > onTrigger(ValidateRecord.java:295) > at org.apache.nifi.processor.AbstractProcessor.onTrigger( > AbstractProcessor.java:27) > at org.apache.nifi.controller.StandardProcessorNode.onTrigger( > StandardProcessorNode.java:1122) > at org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask. > call(ContinuallyRunProcessorTask.java:147) > at org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask. > call(ContinuallyRunProcessorTask.java:47) > at org.apache.nifi.controller.scheduling. > TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:128) > at java.util.concurrent.Executors$RunnableAdapter. > call(Executors.java:511) > at java.util.concurrent.FutureTask.runAndReset( > FutureTask.java:308) > at java.util.concurrent.ScheduledThreadPoolExecutor$ > ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) > at java.util.concurrent.ScheduledThreadPoolExecutor$ > ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) > at java.util.concurrent.ThreadPoolExecutor.runWorker( > ThreadPoolExecutor.java:1149) > at java.util.concurrent.ThreadPoolExecutor$Worker.run( > ThreadPoolExecutor.java:624) > at java.lang.Thread.run(Thread.java:748) >
