Vivek Looking at the code you could have run into a corner case bug where no data is being sent to the HiveOutputModule for a number of windows (could that be happening)?
The corner case is that AbstractFSRollingOutputOperator rotates files if number of empty windows exceeds a certain number. That certain number can be set as the property maxWindowsWithNoData of that operator. Within the HiveOutputModule the name of that operator is fsRolling so you can use ....operator.fsRolling.prop.maxWindowsWithNoData as the name to set a very high value so the operator doesn't try to create a rolling partition for empty windows. On Wed, May 17, 2017 at 2:51 PM, bhidevivek <bhide.vi...@gmail.com> wrote: > While using the HiveOutputModule to save the data into Hive partitioned > table, the application submission fails many times with below error > > 2017-05-17 16:20:13,503 INFO stram.StreamingContainerManager > (StreamingContainerManager.java:processHeartbeat(1486)) - Container > container_e3092_1491920474239_122895_01_000014 buffer server: > brdn1362.target.com:34963 > 2017-05-17 16:20:13,841 INFO stram.StreamingContainerParent > (StreamingContainerParent.java:log(170)) - child msg: Stopped running due > to > an exception. java.lang.NullPointerException > at > com.datatorrent.contrib.hive.AbstractFSRollingOutputOperato > r.getHDFSRollingLastFile(AbstractFSRollingOutputOperator.java:204) > at > com.datatorrent.contrib.hive.AbstractFSRollingOutputOperator.endWindow( > AbstractFSRollingOutputOperator.java:226) > at > com.datatorrent.stram.engine.GenericNode.processEndWindow( > GenericNode.java:153) > at com.datatorrent.stram.engine.GenericNode.run(GenericNode. > java:397) > at > com.datatorrent.stram.engine.StreamingContainer$2.run( > StreamingContainer.java:1428) > context: > PTContainer[id=9(container_e3092_1491920474239_122895_01_ > 000014),state=ACTIVE,operators=[PTOperator[id=10, > name=hiveOutput$fsRolling,state=PENDING_DEPLOY]]] > > I don't see any pattern when this error is reported. I made sure the table > exists in Hive and the location is correct. Is there any particular > configuration or settings I should look for to avoid this? > > > > > -- > View this message in context: http://apache-apex-users-list. > 78494.x6.nabble.com/NullPointerException-at-AbstractFSRollingOutputOperato > r-while-using-HiveOutputModule-tp1625.html > Sent from the Apache Apex Users list mailing list archive at Nabble.com. >
