The mapping: tuple -> dedup-key needs to be 1-1; if multiple tuples are mapped
to the same dedup key you'll see problems like this. In your case multiple
tuples can be mapped to the same value of "id + id1". For example, all tuples
with (id, id1) being any of these pairs will all map to a value of 3: (0, 3),
(1, 2), (2, 1), (3, 0).
A simple way to get a unique dedep key is to convert all of your dedup fields
to strings and catenate them. So in your event/tuple class, define a
getDedupKey() method and within it, compute this string and return it. Then,
you can usethe expression {$}.getDedupKey(). Something along those lines should
work.
Ram
On Wednesday, October 25, 2017, 12:21:59 PM PDT, Vivek Bhide
<[email protected]> wrote:
Thank you everyone for reply. Solution that chinmay suggested is working but
then I see one more discrepancy.
After adding more that 1 fields as a dedup key, my expectation was to have
the dedup decision made on combination of these 2 keys. I did run the test
case multiple times with BoundedDedupOperator and what i found is, events
are marked as Duplicate but if search for the corresponding Unique entry in
sysout then that entry is no where to be found. Its not happening for all
but for most of entires marked as Duplicate
Is my expectation of dedup behavior is correct and is this a right way to
validate if its working as expected?
Dedup_test_case_output.txt
<http://apache-apex-users-list.78494.x6.nabble.com/file/t127/Dedup_test_case_output.txt>
Sample entries :
Present as Unique and Duplicate :
Duplicate: TestEvent [id=75, id1=64, eventTime=Wed Oct 25 12:09:44 PDT 2017]
Unique: TestEvent [id=75, id1=64, eventTime=Wed Oct 25 12:09:18 PDT 2017]
Only present at Duplicate :
Duplicate: TestEvent [id=23, id1=77, eventTime=Wed Oct 25 12:09:04 PDT 2017]
Duplicate: TestEvent [id=44, id1=63, eventTime=Wed Oct 25 12:09:40 PDT 2017]
--
Sent from: http://apache-apex-users-list.78494.x6.nabble.com/
