Tracing the path this is similar to the previous case where this path is triggered when the number of empty windows exceeds a certain number. Try setting the property I mentioned in the last email to a high value so this case is not triggered.
On Wed, May 17, 2017 at 6:22 PM, Vivek Bhide <bhide.vi...@gmail.com> wrote: > Hi Sanjay > > I waited for the application to rollover to nest file but as soon as the > file reaches the size I defined, the operator started failing with below > error > > Any suggestion on this error? FYI. I override the file size into my > application properties to 50 MB from defalut 128MB > > > 2017-05-17 20:18:46,608 INFO stram.StreamingContainerParent > (StreamingContainerParent.java:log(170)) - child msg: Stopped running due > to > an exception. java.lang.NullPointerException > at > com.datatorrent.lib.io.fs.AbstractFileOutputOperator.requestFinalize( > AbstractFileOutputOperator.java:742) > at > com.datatorrent.lib.io.fs.AbstractFileOutputOperator.rotate( > AbstractFileOutputOperator.java:883) > at > com.datatorrent.contrib.hive.AbstractFSRollingOutputOperator.rotateCall( > AbstractFSRollingOutputOperator.java:186) > at > com.datatorrent.contrib.hive.AbstractFSRollingOutputOperator.endWindow( > AbstractFSRollingOutputOperator.java:227) > at > com.datatorrent.stram.engine.GenericNode.processEndWindow( > GenericNode.java:153) > at com.datatorrent.stram.engine.GenericNode.run(GenericNode. > java:397) > at > com.datatorrent.stram.engine.StreamingContainer$2.run( > StreamingContainer.java:1428) > context: > PTContainer[id=9(container_e3092_1491920474239_123256_01_ > 000022),state=ACTIVE,operators=[PTOperator[id=10, > name=hiveOutput$fsRolling,state=PENDING_DEPLOY]]] > 2017-05-17 20:18:47,819 WARN stram.StreamingContainerManager > (StreamingContainerManager.java:processOperatorFailure(1439)) - Operator > failure: PTOperator[id=10,name=hiveOutput$fsRolling,state=INACTIVE] count: > 10 > 2017-05-17 20:18:47,819 ERROR stram.StreamingContainerManager > (StreamingContainerManager.java:processOperatorFailure(1446)) - Initiating > container restart after operator failure > PTOperator[id=10,name=hiveOutput$fsRolling,state=INACTIVE] > 2017-05-17 20:18:47,837 INFO stram.StreamingAppMasterService > (StreamingAppMasterService.java:sendContainerAskToRM(1174)) - Requested > stop > container container_e3092_1491920474239_123256_01_000022 > 2017-05-17 20:18:47,837 INFO impl.NMClientAsyncImpl > (NMClientAsyncImpl.java:run(536)) - Processing Event EventType: > STOP_CONTAINER for Container container_e3092_1491920474239_ > 123256_01_000022 > 2017-05-17 20:18:47,838 INFO impl.NMClientImpl > (NMClientImpl.java:stopContainer(242)) - ok, stopContainerInternal.. > container_e3092_1491920474239_123256_01_000022 > 2017-05-17 20:18:47,838 INFO impl.ContainerManagementProtocolProxy > (ContainerManagementProtocolProxy.java:newProxy(260)) - Opening proxy : > brdn1089.target.com:45454 > 2017-05-17 20:18:48,840 INFO stram.StreamingAppMasterService > (StreamingAppMasterService.java:execute(954)) - Completed > containerId=container_e3092_1491920474239_123256_01_000022, > state=COMPLETE, > exitStatus=-105, diagnostics=Container killed by the ApplicationMaster. > Container killed on request. Exit code is 143 > Container exited with a non-zero exit code 143 > at > com.datatorrent.contrib.hive.AbstractFSRollingOutputOperator.endWindow( > AbstractFSRollingOutputOperator.java:227) > at > com.datatorrent.stram.engine.GenericNode.processEndWindow( > GenericNode.java:153) > at com.datatorrent.stram.engine.GenericNode.run(GenericNode. > java:397) > at > com.datatorrent.stram.engine.StreamingContainer$2.run( > StreamingContainer.java:1428) > context: > PTContainer[id=9(container_e3092_1491920474239_123256_01_ > 000022),state=ACTIVE,operators=[PTOperator[id=10, > name=hiveOutput$fsRolling,state=PENDING_DEPLOY]]] > 2017-05-17 20:18:47,819 WARN stram.StreamingContainerManager > (StreamingContainerManager.java:processOperatorFailure(1439)) - Operator > failure: PTOperator[id=10,name=hiveOutput$fsRolling,state=INACTIVE] count: > 10 > > > > -- > View this message in context: http://apache-apex-users-list. > 78494.x6.nabble.com/HiveOutputModule-creating-extra-directories-than- > specified-while-saving-data-into-HDFS-tp1620p1626.html > Sent from the Apache Apex Users list mailing list archive at Nabble.com. >
