[
https://issues.apache.org/jira/browse/NIFI-3730?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15982392#comment-15982392
]
ASF subversion and git services commented on NIFI-3730:
-------------------------------------------------------
Commit ba2bdf858672e935da60861cfc412e2b55f35652 in nifi's branch
refs/heads/master from [~mattyb149]
[ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=ba2bdf8 ]
NIFI-3730: Allow exclusion of BEGIN/COMMIT event output in CaptureChangeMySQL
This closes #1689.
Signed-off-by: Koji Kawamura <[email protected]>
> Allow exclusion of begin/commit events in CaptureChangeMySQL
> ------------------------------------------------------------
>
> Key: NIFI-3730
> URL: https://issues.apache.org/jira/browse/NIFI-3730
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Extensions
> Affects Versions: 1.2.0
> Reporter: Matt Burgess
> Assignee: Matt Burgess
> Fix For: 1.2.0
>
>
> Notes from [~ijokarumawak]:
> In an end-to-end CDC flow, we would use CaptureChangeMySQL to get database
> events to stream into PutSQL or PutDatabaseRecord. Those database events
> include SQL statements such as begin and commit, and such statements are
> executed literally here.
> I think it would be great if users can configure CaptureChangeMySQL whether
> to emit begin and commit event or not, as these statements do not have
> significant meaning for synchronizing change since CaptureChangeMySQL emits
> FlowFile per updated record, but just for EnforceOrder to order events
> correctly.
> If we can eliminate these events, we can minimize the number of FlowFiles
> which would lead us to a better performance.
> Also, (for MySQL at least) begin and commit is not a database or table
> specific event. When I replicate changes from table A to table B using
> CaptureChangeMySQL and PutDatabaseRecord, I saw following behavior and felt
> begin and commit are a little bit disturbing:
> Insert a row into table A
> 3 events are emitted, begin, insert A and commit. Then CaptureChangeMySQL
> emitted 3 FlowFiles.
> PutDatabaseRecord execute 3 SQLs. begin, insert B and commit.
> Another 3 events are emitted derived from insert B via MySQL bin-log, begin,
> insert B and commit.
> Since I have configured CaptureChangeMySQL only listens to table A, it only
> emitted 2 FlowFiles, begin and commit.
> PutDatabaseRecord executes the begin and commit SQL.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
