Hi Sanjay
I waited for the application to rollover to nest file but as soon as the
file reaches the size I defined, the operator started failing with below
error
Any suggestion on this error? FYI. I override the file size into my
application properties to 50 MB from defalut 128MB
2017-05-17 20:18:46,608 INFO stram.StreamingContainerParent
(StreamingContainerParent.java:log(170)) - child msg: Stopped running due to
an exception. java.lang.NullPointerException
at
com.datatorrent.lib.io.fs.AbstractFileOutputOperator.requestFinalize(AbstractFileOutputOperator.java:742)
at
com.datatorrent.lib.io.fs.AbstractFileOutputOperator.rotate(AbstractFileOutputOperator.java:883)
at
com.datatorrent.contrib.hive.AbstractFSRollingOutputOperator.rotateCall(AbstractFSRollingOutputOperator.java:186)
at
com.datatorrent.contrib.hive.AbstractFSRollingOutputOperator.endWindow(AbstractFSRollingOutputOperator.java:227)
at
com.datatorrent.stram.engine.GenericNode.processEndWindow(GenericNode.java:153)
at com.datatorrent.stram.engine.GenericNode.run(GenericNode.java:397)
at
com.datatorrent.stram.engine.StreamingContainer$2.run(StreamingContainer.java:1428)
context:
PTContainer[id=9(container_e3092_1491920474239_123256_01_000022),state=ACTIVE,operators=[PTOperator[id=10,name=hiveOutput$fsRolling,state=PENDING_DEPLOY]]]
2017-05-17 20:18:47,819 WARN stram.StreamingContainerManager
(StreamingContainerManager.java:processOperatorFailure(1439)) - Operator
failure: PTOperator[id=10,name=hiveOutput$fsRolling,state=INACTIVE] count:
10
2017-05-17 20:18:47,819 ERROR stram.StreamingContainerManager
(StreamingContainerManager.java:processOperatorFailure(1446)) - Initiating
container restart after operator failure
PTOperator[id=10,name=hiveOutput$fsRolling,state=INACTIVE]
2017-05-17 20:18:47,837 INFO stram.StreamingAppMasterService
(StreamingAppMasterService.java:sendContainerAskToRM(1174)) - Requested stop
container container_e3092_1491920474239_123256_01_000022
2017-05-17 20:18:47,837 INFO impl.NMClientAsyncImpl
(NMClientAsyncImpl.java:run(536)) - Processing Event EventType:
STOP_CONTAINER for Container container_e3092_1491920474239_123256_01_000022
2017-05-17 20:18:47,838 INFO impl.NMClientImpl
(NMClientImpl.java:stopContainer(242)) - ok, stopContainerInternal..
container_e3092_1491920474239_123256_01_000022
2017-05-17 20:18:47,838 INFO impl.ContainerManagementProtocolProxy
(ContainerManagementProtocolProxy.java:newProxy(260)) - Opening proxy :
brdn1089.target.com:45454
2017-05-17 20:18:48,840 INFO stram.StreamingAppMasterService
(StreamingAppMasterService.java:execute(954)) - Completed
containerId=container_e3092_1491920474239_123256_01_000022, state=COMPLETE,
exitStatus=-105, diagnostics=Container killed by the ApplicationMaster.
Container killed on request. Exit code is 143
Container exited with a non-zero exit code 143
at
com.datatorrent.contrib.hive.AbstractFSRollingOutputOperator.endWindow(AbstractFSRollingOutputOperator.java:227)
at
com.datatorrent.stram.engine.GenericNode.processEndWindow(GenericNode.java:153)
at com.datatorrent.stram.engine.GenericNode.run(GenericNode.java:397)
at
com.datatorrent.stram.engine.StreamingContainer$2.run(StreamingContainer.java:1428)
context:
PTContainer[id=9(container_e3092_1491920474239_123256_01_000022),state=ACTIVE,operators=[PTOperator[id=10,name=hiveOutput$fsRolling,state=PENDING_DEPLOY]]]
2017-05-17 20:18:47,819 WARN stram.StreamingContainerManager
(StreamingContainerManager.java:processOperatorFailure(1439)) - Operator
failure: PTOperator[id=10,name=hiveOutput$fsRolling,state=INACTIVE] count:
10
--
View this message in context:
http://apache-apex-users-list.78494.x6.nabble.com/HiveOutputModule-creating-extra-directories-than-specified-while-saving-data-into-HDFS-tp1620p1626.html
Sent from the Apache Apex Users list mailing list archive at Nabble.com.
