github-actions[bot] commented on code in PR #68616:
URL: https://github.com/apache/doris/pull/68616#discussion_r4247181704
##########
fe/fe-core/src/main/java/org/apache/doris/job/extensions/insert/streaming/StreamingInsertJob.java:
##########
@@ -933,6 +950,7 @@ public void onStreamTaskFail(AbstractStreamingTask task)
throws JobException {
public void onStreamTaskSuccess(AbstractStreamingTask task) throws
JobException {
try {
this.needRebuildReader = false;
+ this.tolerateSchemaChange = false;
Review Comment:
[P2] Preserve acceptance across recovery rounds that make no progress. After
repairing an unsupported change and running RESUME, asynchronous reader startup
can exceed `searchTimeoutMs` (`taskTimeoutMs / 2`) without delivering a record.
`PipelineCoordinator` then commits the unchanged starting offset successfully,
and this line clears acceptance before the blocked schema has been checked. The
successor reuses the live reader at that offset, receives the same schema with
tolerance false, and pauses again. Failure rebuilds the reader, so consistently
slow startup makes every RESUME repeat this cycle. Keep acceptance through
setup/no-progress successes and clear it after the rejected change has actually
been accepted or replay has passed it.
##########
fe/fe-core/src/main/java/org/apache/doris/job/extensions/insert/streaming/DataSourceConfigValidator.java:
##########
@@ -287,6 +288,9 @@ private static boolean isValidValue(String key, String
value, String dataSourceT
if (key.equals(DataSourceConfigKeys.SKIP_SNAPSHOT_BACKFILL)) {
return isValidBoolean(value);
}
+ if (key.equals(DataSourceConfigKeys.SCHEMA_CHANGE_BEHAVIOR)) {
+ return "evolve".equalsIgnoreCase(value) ||
"ignore".equalsIgnoreCase(value);
Review Comment:
[P1] Negotiate schema-change support before dispatching the policy. During
an FE-first rolling upgrade, a job with `schema_change_behavior=ignore` can
remain bound to a BE whose bundled CDC client predates this change. Backend
selection checks availability, and that client ignores the new config key:
source `ALTER TABLE t DROP COLUMN legacy` still produces target DROP COLUMN,
which the FE schema-change endpoint executes, deleting historical values
despite ignore. The reverse pairing also cannot recover unsupported changes
because an old FE never sends `tolerateSchemaChange=true`. Negotiate the
feature capability and gate dispatch/rebinding, or enforce a compatible policy
at the receiving boundary; optional JSON fields alone do not preserve these
semantics.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
