Most likely the NPE is caused by a bug (regression introduced by
APEXMALHAR-2492 Correct usage of empty Slice in Malhar Library). What is
operator library version (3.8.0)?
Thank you,
Vlad
On 3/23/18 05:48, Bhupesh Chawda wrote:
Hi Vivek,
Can you send across few more details, like the configuration of the
operator and some sample data?
Also do confirm if the key could have been null?
~ Bhupesh
_______________________________________________________
Bhupesh Chawda
E: bhup...@datatorrent.com <mailto:bhup...@datatorrent.com>| Twitter:
@bhupeshsc
www.datatorrent.com <http://www.datatorrent.com> | apex.apache.org
<http://apex.apache.org>
On Fri, Mar 23, 2018 at 1:44 AM, Vivek Bhide <vivek.bh...@target.com
<mailto:vivek.bh...@target.com>> wrote:
Hi All,
While using the TimeBasedDedupOperator for deduping, I see that
operator
keeps failing with below NullPointer exception. I don't know if it has
anything to do with configuration.
I also see that operator is always high on CPU usages. Almost
reaching 100%.
I tried increasing vcores and also memory for operator but its of
no use
2018-03-22 15:10:10,037 INFO stram.FSRecoveryHandler
(FSRecoveryHandler.java:rotateLog(103)) - Creating
hdfs://littleredns/user/SVDATHDP/datatorrent/apps/application_1519410901484_187748/recovery/log
2018-03-22 15:10:10,056 INFO stram.StreamingContainerParent
(StreamingContainerParent.java:log(170)) - child msg: Stopped
running due to
an exception. java.lang.NullPointerException
at
org.apache.hadoop.io.file.tfile.TFile$Writer.append(TFile.java:387)
at
com.datatorrent.lib.fileaccess.TFileWriter.append(TFileWriter.java:66)
at
org.apache.apex.malhar.lib.state.managed.BucketsFileSystem.writeBucketData(BucketsFileSystem.java:179)
at
org.apache.apex.malhar.lib.state.managed.IncrementalCheckpointManager.transferWindowFiles(IncrementalCheckpointManager.java:139)
at
org.apache.apex.malhar.lib.state.managed.IncrementalCheckpointManager$1.run(IncrementalCheckpointManager.java:110)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
context:
PTContainer[id=3(container_e3125_1519410901484_187748_01_000012),state=ACTIVE,operators=[PTOperator[id=3,name=dedupeOperator,state=ACTIVE]]]
2018-03-22 15:10:10,915 WARN stram.StreamingContainerManager
(StreamingContainerManager.java:processOperatorFailure(1439)) -
Operator
failure: PTOperator[id=3,name=dedupeOperator,state=INACTIVE] count: 1
Regards
Vivek
--
Sent from: http://apache-apex-users-list.78494.x6.nabble.com/
<http://apache-apex-users-list.78494.x6.nabble.com/>
