0AyanamiRei commented on PR #60051: URL: https://github.com/apache/doris/pull/60051#issuecomment-3999101604
<html> <body> <!--StartFragment--><h1 style="direction: ltr; unicode-bidi: bidi-override; color: rgb(204, 204, 204); font-family: -apple-system, BlinkMacSystemFont, "Segoe UI", Roboto, sans-serif; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; white-space: normal; background-color: rgb(31, 31, 31); text-decoration-thickness: initial; text-decoration-style: initial; text-decoration-color: initial;">Code Review: Kinesis Routine Load (commit<span> </span><code style="direction: ltr; unicode-bidi: bidi-override; font-family: monospace; color: rgb(208, 208, 208); background-color: rgb(60, 60, 60); padding: 2px 4px; border-radius: 3px; font-size: 0.9em; word-break: break-word;">7c79c34a73e</code>)</h1><h2 style="direction: ltr; unicode-bidi: bidi-override; color: rgb(204, 204, 204); font-family: -apple-s ystem, BlinkMacSystemFont, "Segoe UI", Roboto, sans-serif; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; white-space: normal; background-color: rgb(31, 31, 31); text-decoration-thickness: initial; text-decoration-style: initial; text-decoration-color: initial;">BLOCKER Issues</h2><h3 style="direction: ltr; unicode-bidi: bidi-override; color: rgb(204, 204, 204); font-family: -apple-system, BlinkMacSystemFont, "Segoe UI", Roboto, sans-serif; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; white-space: normal; background-color: rgb(31, 31, 31); text-decoration-thickness: initial; text-decoration -style: initial; text-decoration-color: initial;">1.<span> </span><code style="direction: ltr; unicode-bidi: bidi-override; font-family: monospace; color: rgb(208, 208, 208); background-color: rgb(60, 60, 60); padding: 2px 4px; border-radius: 3px; font-size: 0.9em; word-break: break-word;">data_consumer_pool.cpp:89</code><span> </span>— Null pointer dereference for Kinesis tasks</h3><p style="direction: ltr; unicode-bidi: bidi-override; white-space: pre-wrap; margin-top: 0.1em; margin-bottom: 0.2em; color: rgb(204, 204, 204); font-family: -apple-system, BlinkMacSystemFont, "Segoe UI", Roboto, sans-serif; font-size: 13px; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; background-color: rgb(31, 31, 31); text-decoration-thickness: initial; text-decoration-style: initia l; text-decoration-color: initial;"><a href="vscode-webview://12or9qsfo5kq61724hn7thg1f7a1a8i4j2ml00u56orm57h9kk50/be/src/runtime/routine_load/data_consumer_pool.cpp#L89" target="_blank" rel="noopener noreferrer" style="direction: ltr; unicode-bidi: bidi-override; color: rgb(77, 170, 252); text-decoration: rgb(77, 170, 252);">data_consumer_pool.cpp:89</a>: The old check <code style="direction: ltr; unicode-bidi: bidi-override; font-family: monospace; color: rgb(208, 208, 208); background-color: rgb(60, 60, 60); padding: 2px 4px; border-radius: 3px; font-size: 0.9em; word-break: break-word;">ctx->kafka_info->begin_offset.size() == 0</code> is <strong style="direction: ltr; unicode-bidi: bidi-override;">still unconditionally executed before the switch</strong>. For Kinesis tasks, <code style="direction: ltr; unicode-bidi: bidi-override; font-family: monospace; color: rgb(208, 208, 208); background-color: rgb(60, 60, 60); padding: 2px 4px; border-radius: 3px; font-size: 0.9em; wo rd-break: break-word;">ctx->kafka_info</code> is <code style="direction: ltr; unicode-bidi: bidi-override; font-family: monospace; color: rgb(208, 208, 208); background-color: rgb(60, 60, 60); padding: 2px 4px; border-radius: 3px; font-size: 0.9em; word-break: break-word;">nullptr</code>, so this will <strong style="direction: ltr; unicode-bidi: bidi-override;">crash with a segfault</strong>.</p><div class="codeBlockWrapper_-a7MRw" style="direction: ltr; unicode-bidi: bidi-override; position: relative; margin: 8px 0px; color: rgb(204, 204, 204); font-family: -apple-system, BlinkMacSystemFont, "Segoe UI", Roboto, sans-serif; font-size: 13px; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; white-space: normal; background-color: rgb(31, 31, 31); text-decoration-thickness : initial; text-decoration-style: initial; text-decoration-color: initial;"><button class="copyButton_CEmTFw copyButton_-a7MRw" title="Copy code" aria-label="Copy code to clipboard" style="direction: ltr; unicode-bidi: bidi-override; color: rgb(204, 204, 204); font-family: -apple-system, BlinkMacSystemFont, "Segoe UI", Roboto, sans-serif; font-size: 13px; background: none 0% 0% / auto repeat scroll padding-box border-box rgb(31, 31, 31); border-color: rgb(69, 69, 69); border-style: solid; border-width: 0.666667px; border-image: none 100% / 1 / 0 stretch; cursor: pointer; opacity: 0; display: flex; border-radius: 4px; justify-content: center; align-items: center; padding: 4px; transition: opacity 0.15s, background 0.15s; position: absolute; top: 4px; right: 4px;"><svg xmlns="http://www.w3.org/2000/svg"; viewBox="0 0 20 20" fill="currentColor" aria-hidden="true" data-slot="icon" class="copyIcon_CEmTFw"><path fill-rule="evenodd" d="M15.988 3.012A2.25 2.25 0 0 1 18 5.25v6.5A2.2 5 2.25 0 0 1 15.75 14H13.5v-3.379a3 3 0 0 0-.879-2.121l-3.12-3.121a3 3 0 0 0-1.402-.791 2.252 2.252 0 0 1 1.913-1.576A2.25 2.25 0 0 1 12.25 1h1.5a2.25 2.25 0 0 1 2.238 2.012ZM11.5 3.25a.75.75 0 0 1 .75-.75h1.5a.75.75 0 0 1 .75.75v.25h-3v-.25Z" clip-rule="evenodd"></path><path d="M3.5 6A1.5 1.5 0 0 0 2 7.5v9A1.5 1.5 0 0 0 3.5 18h7a1.5 1.5 0 0 0 1.5-1.5v-5.879a1.5 1.5 0 0 0-.44-1.06L8.44 6.439A1.5 1.5 0 0 0 7.378 6H3.5Z"></path></svg></button><pre style="direction: ltr; unicode-bidi: bidi-override; overflow-x: auto; white-space: pre; box-sizing: border-box; border-radius: 4px; max-width: 100%; margin: 0px; padding: 8px;"><code class="language-cpp" style="direction: ltr; unicode-bidi: bidi-override; font-family: monospace; color: rgb(208, 208, 208); background-color: rgb(60, 60, 60); padding: 0px; border-radius: 3px; font-size: 0.9em; word-break: break-word;">// Line 89 — CRASH: ctx->kafka_info is nullptr for Kinesis tasks if (ctx->kafka_info->begin_offset.size() == 0) { return Status::InternalError("PAUSE: The size of begin_offset of task should not be 0."); } </code></pre></div><p style="direction: ltr; unicode-bidi: bidi-override; white-space: pre-wrap; margin-top: 0.1em; margin-bottom: 0.2em; color: rgb(204, 204, 204); font-family: -apple-system, BlinkMacSystemFont, "Segoe UI", Roboto, sans-serif; font-size: 13px; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; background-color: rgb(31, 31, 31); text-decoration-thickness: initial; text-decoration-style: initial; text-decoration-color: initial;"><strong style="direction: ltr; unicode-bidi: bidi-override;">Fix</strong>: Move validation into the per-source-type switch branches:</p><div class="codeBlockWrapper_-a7MRw" style="direction: ltr; unicode-bidi: bidi-override; position: relative; margin: 8px 0px; color: rgb(204, 204, 204); font-family: -apple-system, BlinkMacSystemFo nt, "Segoe UI", Roboto, sans-serif; font-size: 13px; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; white-space: normal; background-color: rgb(31, 31, 31); text-decoration-thickness: initial; text-decoration-style: initial; text-decoration-color: initial;"><button class="copyButton_CEmTFw copyButton_-a7MRw" title="Copy code" aria-label="Copy code to clipboard" style="direction: ltr; unicode-bidi: bidi-override; color: rgb(204, 204, 204); font-family: -apple-system, BlinkMacSystemFont, "Segoe UI", Roboto, sans-serif; font-size: 13px; background: none 0% 0% / auto repeat scroll padding-box border-box rgb(31, 31, 31); border-color: rgb(69, 69, 69); border-style: solid; border-width: 0.666667px; border-image: none 100% / 1 / 0 stretch; cursor: pointer; opacity: 0; display: flex; border-radius: 4px; justify-content: center; align-items: center; padding: 4px; transition: opacity 0.15s, background 0.15s; position: absolute; top: 4px; right: 4px;"><svg xmlns="http://www.w3.org/2000/svg"; viewBox="0 0 20 20" fill="currentColor" aria-hidden="true" data-slot="icon" class="copyIcon_CEmTFw"><path fill-rule="evenodd" d="M15.988 3.012A2.25 2.25 0 0 1 18 5.25v6.5A2.25 2.25 0 0 1 15.75 14H13.5v-3.379a3 3 0 0 0-.879-2.121l-3.12-3.121a3 3 0 0 0-1.402-.791 2.252 2.252 0 0 1 1.913-1.576A2.25 2.25 0 0 1 12.25 1h1.5a2.25 2.25 0 0 1 2.238 2.012ZM11.5 3.25a.75.75 0 0 1 .75-.75h1.5a.75.75 0 0 1 .75.75v.25h-3v-.25Z" clip-rule="evenodd"></path><path d="M3.5 6A1.5 1.5 0 0 0 2 7.5v9A1.5 1.5 0 0 0 3.5 18h7a1.5 1.5 0 0 0 1.5-1.5v-5.879a1.5 1.5 0 0 0-.44-1.06L8.44 6.439A1.5 1.5 0 0 0 7.378 6H3.5Z"></path></svg></button><pre style="direction: ltr; unicode-bidi: bidi-override; overflow-x: auto; white-space: pre; box-sizing: border-box; border-radius: 4px; max-width: 100%; margin: 0px; padding: 8px;"><code class="language-cpp" style="direction: ltr; unicode-bidi: bidi-override; font-family: monospace; color: rgb(208, 208, 208); background-color: rgb(60, 60, 60); padding: 0px; border-radius: 3px; font-size: 0.9em; word-break: break-word;">switch (ctx->load_src_type) { case TLoadSourceType::KAFKA: { DCHECK(ctx->kafka_info); if (ctx->kafka_info->begin_offset.size() == 0) { return Status::InternalError("PAUSE: ..."); } consumer_num = std::min(consumer_num, ctx->kafka_info->begin_offset.size()); break; } case TLoadSourceType::KINESIS: { DCHECK(ctx->kinesis_info); if (ctx->kinesis_info->begin_sequence_number.size() == 0) { return Status::InternalError("PAUSE: ..."); } consumer_num = 1; break; } ... </code></pre></div><h3 style="direction: ltr; unicode-bidi: bidi-override; color: rgb(204, 204, 204); font-family: -apple-system, BlinkMacSystemFont, "Segoe UI", Roboto, sans-serif; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; white-space: normal; background-color: rgb(31, 31, 31); text-decoration-thickness: initial; text-decoration-style: initial; text-decoration-color: initial;">2.<span> </span><code style="direction: ltr; unicode-bidi: bidi-override; font-family: monospace; color: rgb(208, 208, 208); background-color: rgb(60, 60, 60); padding: 2px 4px; border-radius: 3px; font-size: 0.9em; word-break: break-word;">data_consumer_pool.cpp:109</code><span> </span>— Creates base class<span> </span><code style="direction: ltr; unicode-bidi: bidi-override; font-family: monospace; c olor: rgb(208, 208, 208); background-color: rgb(60, 60, 60); padding: 2px 4px; border-radius: 3px; font-size: 0.9em; word-break: break-word;">DataConsumerGroup</code><span> </span>instead of typed subclass</h3><p style="direction: ltr; unicode-bidi: bidi-override; white-space: pre-wrap; margin-top: 0.1em; margin-bottom: 0.2em; color: rgb(204, 204, 204); font-family: -apple-system, BlinkMacSystemFont, "Segoe UI", Roboto, sans-serif; font-size: 13px; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; background-color: rgb(31, 31, 31); text-decoration-thickness: initial; text-decoration-style: initial; text-decoration-color: initial;"><a href="vscode-webview://12or9qsfo5kq61724hn7thg1f7a1a8i4j2ml00u56orm57h9kk50/be/src/runtime/routine_load/data_consumer_pool.cpp#L109" target= "_blank" rel="noopener noreferrer" style="direction: ltr; unicode-bidi: bidi-override; color: rgb(77, 170, 252); text-decoration: rgb(77, 170, 252);">data_consumer_pool.cpp:109</a>:</p><div class="codeBlockWrapper_-a7MRw" style="direction: ltr; unicode-bidi: bidi-override; position: relative; margin: 8px 0px; color: rgb(204, 204, 204); font-family: -apple-system, BlinkMacSystemFont, "Segoe UI", Roboto, sans-serif; font-size: 13px; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; white-space: normal; background-color: rgb(31, 31, 31); text-decoration-thickness: initial; text-decoration-style: initial; text-decoration-color: initial;"><button class="copyButton_CEmTFw copyButton_-a7MRw" title="Copy code" aria-label="Copy code to clipboard" style="direction: ltr; unicode-bidi : bidi-override; color: rgb(204, 204, 204); font-family: -apple-system, BlinkMacSystemFont, "Segoe UI", Roboto, sans-serif; font-size: 13px; background: none 0% 0% / auto repeat scroll padding-box border-box rgb(31, 31, 31); border-color: rgb(69, 69, 69); border-style: solid; border-width: 0.666667px; border-image: none 100% / 1 / 0 stretch; cursor: pointer; opacity: 0; display: flex; border-radius: 4px; justify-content: center; align-items: center; padding: 4px; transition: opacity 0.15s, background 0.15s; position: absolute; top: 4px; right: 4px;"><svg xmlns="http://www.w3.org/2000/svg"; viewBox="0 0 20 20" fill="currentColor" aria-hidden="true" data-slot="icon" class="copyIcon_CEmTFw"><path fill-rule="evenodd" d="M15.988 3.012A2.25 2.25 0 0 1 18 5.25v6.5A2.25 2.25 0 0 1 15.75 14H13.5v-3.379a3 3 0 0 0-.879-2.121l-3.12-3.121a3 3 0 0 0-1.402-.791 2.252 2.252 0 0 1 1.913-1.576A2.25 2.25 0 0 1 12.25 1h1.5a2.25 2.25 0 0 1 2.238 2.012ZM11.5 3.25a.75.75 0 0 1 .75-.75h1.5a.75.75 0 0 1 .75.75v.25h-3v-.25Z" clip-rule="evenodd"></path><path d="M3.5 6A1.5 1.5 0 0 0 2 7.5v9A1.5 1.5 0 0 0 3.5 18h7a1.5 1.5 0 0 0 1.5-1.5v-5.879a1.5 1.5 0 0 0-.44-1.06L8.44 6.439A1.5 1.5 0 0 0 7.378 6H3.5Z"></path></svg></button><pre style="direction: ltr; unicode-bidi: bidi-override; overflow-x: auto; white-space: pre; box-sizing: border-box; border-radius: 4px; max-width: 100%; margin: 0px; padding: 8px;"><code class="language-cpp" style="direction: ltr; unicode-bidi: bidi-override; font-family: monospace; color: rgb(208, 208, 208); background-color: rgb(60, 60, 60); padding: 0px; border-radius: 3px; font-size: 0.9em; word-break: break-word;">std::shared_ptr<DataConsumerGroup> grp = std::make_shared<DataConsumerGroup>(consumer_num); </code></pre></div><p style="direction: ltr; unicode-bidi: bidi-override; white-space: pre-wrap; margin-top: 0.1em; margin-bottom: 0.2em; color: rgb(204, 204, 204); font-family: -apple-system, BlinkMacSystemFont, "Segoe UI", Roboto, sans-serif; font-size: 13px; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; background-color: rgb(31, 31, 31); text-decoration-thickness: initial; text-decoration-style: initial; text-decoration-color: initial;">This creates the <strong style="direction: ltr; unicode-bidi: bidi-override;">base class</strong> <code style="direction: ltr; unicode-bidi: bidi-override; font-family: monospace; color: rgb(208, 208, 208); background-color: rgb(60, 60, 60); padding: 2px 4px; border-radius: 3px; font-size: 0.9em; word-break: break-word;">DataConsum erGroup</code>, not <code style="direction: ltr; unicode-bidi: bidi-override; font-family: monospace; color: rgb(208, 208, 208); background-color: rgb(60, 60, 60); padding: 2px 4px; border-radius: 3px; font-size: 0.9em; word-break: break-word;">KafkaDataConsumerGroup</code> or <code style="direction: ltr; unicode-bidi: bidi-override; font-family: monospace; color: rgb(208, 208, 208); background-color: rgb(60, 60, 60); padding: 2px 4px; border-radius: 3px; font-size: 0.9em; word-break: break-word;">KinesisDataConsumerGroup</code>. The base class <code style="direction: ltr; unicode-bidi: bidi-override; font-family: monospace; color: rgb(208, 208, 208); background-color: rgb(60, 60, 60); padding: 2px 4px; border-radius: 3px; font-size: 0.9em; word-break: break-word;">start_all()</code> simply returns <code style="direction: ltr; unicode-bidi: bidi-override; font-family: monospace; color: rgb(208, 208, 208); background-color: rgb(60, 60, 60); padding: 2px 4px; border-radius: 3px; font- size: 0.9em; word-break: break-word;">Status::OK()</code> without consuming anything. Later in <code style="direction: ltr; unicode-bidi: bidi-override; font-family: monospace; color: rgb(208, 208, 208); background-color: rgb(60, 60, 60); padding: 2px 4px; border-radius: 3px; font-size: 0.9em; word-break: break-word;">routine_load_task_executor.cpp:455,472</code>, <code style="direction: ltr; unicode-bidi: bidi-override; font-family: monospace; color: rgb(208, 208, 208); background-color: rgb(60, 60, 60); padding: 2px 4px; border-radius: 3px; font-size: 0.9em; word-break: break-word;">static_pointer_cast<KafkaDataConsumerGroup></code> / <code style="direction: ltr; unicode-bidi: bidi-override; font-family: monospace; color: rgb(208, 208, 208); background-color: rgb(60, 60, 60); padding: 2px 4px; border-radius: 3px; font-size: 0.9em; word-break: break-word;">static_pointer_cast<KinesisDataConsumerGroup></code> will produce <strong style="direction: ltr; unicode-bidi: bidi -override;">undefined behavior</strong> (casting base to derived when the object is not actually derived).</p><p style="direction: ltr; unicode-bidi: bidi-override; white-space: pre-wrap; margin-top: 0.1em; margin-bottom: 0.2em; color: rgb(204, 204, 204); font-family: -apple-system, BlinkMacSystemFont, "Segoe UI", Roboto, sans-serif; font-size: 13px; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; background-color: rgb(31, 31, 31); text-decoration-thickness: initial; text-decoration-style: initial; text-decoration-color: initial;"><strong style="direction: ltr; unicode-bidi: bidi-override;">Fix</strong>: Create typed consumer groups inside the switch:</p><div class="codeBlockWrapper_-a7MRw" style="direction: ltr; unicode-bidi: bidi-override; position: relative; margin: 8 px 0px; color: rgb(204, 204, 204); font-family: -apple-system, BlinkMacSystemFont, "Segoe UI", Roboto, sans-serif; font-size: 13px; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; white-space: normal; background-color: rgb(31, 31, 31); text-decoration-thickness: initial; text-decoration-style: initial; text-decoration-color: initial;"><button class="copyButton_CEmTFw copyButton_-a7MRw" title="Copy code" aria-label="Copy code to clipboard" style="direction: ltr; unicode-bidi: bidi-override; color: rgb(204, 204, 204); font-family: -apple-system, BlinkMacSystemFont, "Segoe UI", Roboto, sans-serif; font-size: 13px; background: none 0% 0% / auto repeat scroll padding-box border-box rgb(31, 31, 31); border-color: rgb(69, 69, 69); border-style: solid; border-width: 0. 666667px; border-image: none 100% / 1 / 0 stretch; cursor: pointer; opacity: 0; display: flex; border-radius: 4px; justify-content: center; align-items: center; padding: 4px; transition: opacity 0.15s, background 0.15s; position: absolute; top: 4px; right: 4px;"><svg xmlns="http://www.w3.org/2000/svg"; viewBox="0 0 20 20" fill="currentColor" aria-hidden="true" data-slot="icon" class="copyIcon_CEmTFw"><path fill-rule="evenodd" d="M15.988 3.012A2.25 2.25 0 0 1 18 5.25v6.5A2.25 2.25 0 0 1 15.75 14H13.5v-3.379a3 3 0 0 0-.879-2.121l-3.12-3.121a3 3 0 0 0-1.402-.791 2.252 2.252 0 0 1 1.913-1.576A2.25 2.25 0 0 1 12.25 1h1.5a2.25 2.25 0 0 1 2.238 2.012ZM11.5 3.25a.75.75 0 0 1 .75-.75h1.5a.75.75 0 0 1 .75.75v.25h-3v-.25Z" clip-rule="evenodd"></path><path d="M3.5 6A1.5 1.5 0 0 0 2 7.5v9A1.5 1.5 0 0 0 3.5 18h7a1.5 1.5 0 0 0 1.5-1.5v-5.879a1.5 1.5 0 0 0-.44-1.06L8.44 6.439A1.5 1.5 0 0 0 7.378 6H3.5Z"></path></svg></button><pre style="direction: ltr; unicode-bidi: bidi-override; overflow-x: auto; white-space: pre; box-sizing: border-box; border-radius: 4px; max-width: 100%; margin: 0px; padding: 8px;"><code class="language-cpp" style="direction: ltr; unicode-bidi: bidi-override; font-family: monospace; color: rgb(208, 208, 208); background-color: rgb(60, 60, 60); padding: 0px; border-radius: 3px; font-size: 0.9em; word-break: break-word;">std::shared_ptr<DataConsumerGroup> grp; switch (ctx->load_src_type) { case TLoadSourceType::KAFKA: ... grp = std::make_shared<KafkaDataConsumerGroup>(consumer_num); break; case TLoadSourceType::KINESIS: ... grp = std::make_shared<KinesisDataConsumerGroup>(consumer_num); break; ... } </code></pre></div><h3 style="direction: ltr; unicode-bidi: bidi-override; color: rgb(204, 204, 204); font-family: -apple-system, BlinkMacSystemFont, "Segoe UI", Roboto, sans-serif; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; white-space: normal; background-color: rgb(31, 31, 31); text-decoration-thickness: initial; text-decoration-style: initial; text-decoration-color: initial;">3.<span> </span><code style="direction: ltr; unicode-bidi: bidi-override; font-family: monospace; color: rgb(208, 208, 208); background-color: rgb(60, 60, 60); padding: 2px 4px; border-radius: 3px; font-size: 0.9em; word-break: break-word;">data_consumer.cpp:871</code><span> </span>— Use-after-move: accessing<span> </span><code style="direction: ltr; unicode-bidi: bidi-override; font-family: monospace; color: rgb(208, 208, 208); background-color: rgb(60, 60, 60); padding: 2px 4px; border-radius: 3px; font-size: 0.9em; word-break: break-word;">result</code><span> </span>after<span> </span><code style="direction: ltr; unicode-bidi: bidi-override; font-family: monospace; color: rgb(208, 208, 208); background-color: rgb(60, 60, 60); padding: 2px 4px; border-radius: 3px; font-size: 0.9em; word-break: break-word;">std::move</code></h3><p style="direction: ltr; unicode-bidi: bidi-override; white-space: pre-wrap; margin-top: 0.1em; margin-bottom: 0.2em; color: rgb(204, 204, 204); font-family: -apple-system, BlinkMacSystemFont, "Segoe UI", Roboto, sans-serif; font-size: 13px; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; background-color: rgb(31, 31, 31); text-decoration-t hickness: initial; text-decoration-style: initial; text-decoration-color: initial;"><a href="vscode-webview://12or9qsfo5kq61724hn7thg1f7a1a8i4j2ml00u56orm57h9kk50/be/src/runtime/routine_load/data_consumer.cpp#L864-L871" target="_blank" rel="noopener noreferrer" style="direction: ltr; unicode-bidi: bidi-override; color: rgb(77, 170, 252); text-decoration: rgb(77, 170, 252);">data_consumer.cpp:864-871</a>:</p><div class="codeBlockWrapper_-a7MRw" style="direction: ltr; unicode-bidi: bidi-override; position: relative; margin: 8px 0px; color: rgb(204, 204, 204); font-family: -apple-system, BlinkMacSystemFont, "Segoe UI", Roboto, sans-serif; font-size: 13px; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; white-space: normal; background-color: rgb(31, 31, 31); text-decoration- thickness: initial; text-decoration-style: initial; text-decoration-color: initial;"><button class="copyButton_CEmTFw copyButton_-a7MRw" title="Copy code" aria-label="Copy code to clipboard" style="direction: ltr; unicode-bidi: bidi-override; color: rgb(204, 204, 204); font-family: -apple-system, BlinkMacSystemFont, "Segoe UI", Roboto, sans-serif; font-size: 13px; background: none 0% 0% / auto repeat scroll padding-box border-box rgb(31, 31, 31); border-color: rgb(69, 69, 69); border-style: solid; border-width: 0.666667px; border-image: none 100% / 1 / 0 stretch; cursor: pointer; opacity: 0; display: flex; border-radius: 4px; justify-content: center; align-items: center; padding: 4px; transition: opacity 0.15s, background 0.15s; position: absolute; top: 4px; right: 4px;"><svg xmlns="http://www.w3.org/2000/svg"; viewBox="0 0 20 20" fill="currentColor" aria-hidden="true" data-slot="icon" class="copyIcon_CEmTFw"><path fill-rule="evenodd" d="M15.988 3.012A2.25 2.25 0 0 1 18 5.2 5v6.5A2.25 2.25 0 0 1 15.75 14H13.5v-3.379a3 3 0 0 0-.879-2.121l-3.12-3.121a3 3 0 0 0-1.402-.791 2.252 2.252 0 0 1 1.913-1.576A2.25 2.25 0 0 1 12.25 1h1.5a2.25 2.25 0 0 1 2.238 2.012ZM11.5 3.25a.75.75 0 0 1 .75-.75h1.5a.75.75 0 0 1 .75.75v.25h-3v-.25Z" clip-rule="evenodd"></path><path d="M3.5 6A1.5 1.5 0 0 0 2 7.5v9A1.5 1.5 0 0 0 3.5 18h7a1.5 1.5 0 0 0 1.5-1.5v-5.879a1.5 1.5 0 0 0-.44-1.06L8.44 6.439A1.5 1.5 0 0 0 7.378 6H3.5Z"></path></svg></button><pre style="direction: ltr; unicode-bidi: bidi-override; overflow-x: auto; white-space: pre; box-sizing: border-box; border-radius: 4px; max-width: 100%; margin: 0px; padding: 8px;"><code class="language-cpp" style="direction: ltr; unicode-bidi: bidi-override; font-family: monospace; color: rgb(208, 208, 208); background-color: rgb(60, 60, 60); padding: 0px; border-radius: 3px; font-size: 0.9em; word-break: break-word;">auto result = outcome.GetResultWithOwnership(); auto millis_behind = result.GetMillisBehindLatest(); RETURN_IF_ERROR(_process_records(shard_id, std::move(result), queue, ...)); // ... std::string next_iterator = result.GetNextShardIterator(); // UB: result was moved! </code></pre></div><p style="direction: ltr; unicode-bidi: bidi-override; white-space: pre-wrap; margin-top: 0.1em; margin-bottom: 0.2em; color: rgb(204, 204, 204); font-family: -apple-system, BlinkMacSystemFont, "Segoe UI", Roboto, sans-serif; font-size: 13px; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; background-color: rgb(31, 31, 31); text-decoration-thickness: initial; text-decoration-style: initial; text-decoration-color: initial;">After <code style="direction: ltr; unicode-bidi: bidi-override; font-family: monospace; color: rgb(208, 208, 208); background-color: rgb(60, 60, 60); padding: 2px 4px; border-radius: 3px; font-size: 0.9em; word-break: break-word;">std::move(result)</code> on line 864, accessing <code style="direction: ltr; unicode-bidi: bidi-overri de; font-family: monospace; color: rgb(208, 208, 208); background-color: rgb(60, 60, 60); padding: 2px 4px; border-radius: 3px; font-size: 0.9em; word-break: break-word;">result.GetNextShardIterator()</code> on line 871 is <strong style="direction: ltr; unicode-bidi: bidi-override;">undefined behavior</strong>. The moved-from object is in an indeterminate state.</p><p style="direction: ltr; unicode-bidi: bidi-override; white-space: pre-wrap; margin-top: 0.1em; margin-bottom: 0.2em; color: rgb(204, 204, 204); font-family: -apple-system, BlinkMacSystemFont, "Segoe UI", Roboto, sans-serif; font-size: 13px; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; background-color: rgb(31, 31, 31); text-decoration-thickness: initial; text-decoration-style: initial; text-decoration-col or: initial;"><strong style="direction: ltr; unicode-bidi: bidi-override;">Fix</strong>: Save <code style="direction: ltr; unicode-bidi: bidi-override; font-family: monospace; color: rgb(208, 208, 208); background-color: rgb(60, 60, 60); padding: 2px 4px; border-radius: 3px; font-size: 0.9em; word-break: break-word;">next_iterator</code> before the move:</p><div class="codeBlockWrapper_-a7MRw" style="direction: ltr; unicode-bidi: bidi-override; position: relative; margin: 8px 0px; color: rgb(204, 204, 204); font-family: -apple-system, BlinkMacSystemFont, "Segoe UI", Roboto, sans-serif; font-size: 13px; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; white-space: normal; background-color: rgb(31, 31, 31); text-decoration-thickness: initial; text-decoration-style: initial; text-decoration-color: initial;"><button class="copyButton_CEmTFw copyButton_-a7MRw" title="Copy code" aria-label="Copy code to clipboard" style="direction: ltr; unicode-bidi: bidi-override; color: rgb(204, 204, 204); font-family: -apple-system, BlinkMacSystemFont, "Segoe UI", Roboto, sans-serif; font-size: 13px; background: none 0% 0% / auto repeat scroll padding-box border-box rgb(31, 31, 31); border-color: rgb(69, 69, 69); border-style: solid; border-width: 0.666667px; border-image: none 100% / 1 / 0 stretch; cursor: pointer; opacity: 0; display: flex; border-radius: 4px; justify-content: center; align-items: center; padding: 4px; transition: opacity 0.15s, background 0.15s; position: absolute; top: 4px; right: 4px;"><svg xmlns="http://www.w3.org/2000/svg"; viewBox="0 0 20 20" fill="currentColor" aria-hidden="true" data-slot="icon" class="copyIcon_CEmTFw"><path fill-rule="evenodd" d="M15.988 3.012A2.25 2.25 0 0 1 18 5.25v6.5A2.25 2.25 0 0 1 15.75 14H13.5v-3.379a3 3 0 0 0-.879-2.121l-3.12-3.121a3 3 0 0 0-1.402-.791 2.252 2.252 0 0 1 1.913-1.576A2.25 2.25 0 0 1 12.25 1h1.5a2.25 2.25 0 0 1 2.238 2.012ZM11.5 3.25a.75.75 0 0 1 .75-.75h1.5a.75.75 0 0 1 .75.75v.25h-3v-.25Z" clip-rule="evenodd"></path><path d="M3.5 6A1.5 1.5 0 0 0 2 7.5v9A1.5 1.5 0 0 0 3.5 18h7a1.5 1.5 0 0 0 1.5-1.5v-5.879a1.5 1.5 0 0 0-.44-1.06L8.44 6.439A1.5 1.5 0 0 0 7.378 6H3.5Z"></path></svg></button><pre style="direction: ltr; unicode-bidi: bidi-override; overflow-x: auto; white-space: pre; box-sizing: border-box; border-radius: 4px; max-width: 100%; margin: 0px; padding: 8px;"><code class="language-cpp" style="direction: ltr; unicode-bidi: bidi-override; font-family: monospace; color: rgb(208, 208, 208); background-color: rgb(60, 60, 60); padding: 0px; border-radius: 3px; font-size: 0.9em; word-break: break-word;">auto result = outcome.GetResultWithOwnership(); auto millis_behind = result.GetMillisBehindLatest(); std::string next_iterator = result.GetNextShardIterator(); RETURN_IF_ERROR(_process_records(shard_id, std::move(result), queue, ...)); _millis_behind_latest[shard_id] = millis_behind; // use next_iterator below... </code></pre></div><hr style="direction: ltr; unicode-bidi: bidi-override; font-family: -apple-system, BlinkMacSystemFont, "Segoe UI", Roboto, sans-serif; font-size: 13px; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; white-space: normal; background-color: rgb(31, 31, 31); text-decoration-thickness: initial; text-decoration-style: initial; text-decoration-color: initial;"><h2 style="direction: ltr; unicode-bidi: bidi-override; color: rgb(204, 204, 204); font-family: -apple-system, BlinkMacSystemFont, "Segoe UI", Roboto, sans-serif; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-stroke-width : 0px; white-space: normal; background-color: rgb(31, 31, 31); text-decoration-thickness: initial; text-decoration-style: initial; text-decoration-color: initial;">HIGH Issues</h2><h3 style="direction: ltr; unicode-bidi: bidi-override; color: rgb(204, 204, 204); font-family: -apple-system, BlinkMacSystemFont, "Segoe UI", Roboto, sans-serif; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; white-space: normal; background-color: rgb(31, 31, 31); text-decoration-thickness: initial; text-decoration-style: initial; text-decoration-color: initial;">4.<span> </span><code style="direction: ltr; unicode-bidi: bidi-override; font-family: monospace; color: rgb(208, 208, 208); background-color: rgb(60, 60, 60); padding: 2px 4px; border-radius: 3px; font-size: 0.9em; word-break: break-word;">data_cons umer.cpp:928-929</code><span> </span>— Dangerous<span> </span><code style="direction: ltr; unicode-bidi: bidi-override; font-family: monospace; color: rgb(208, 208, 208); background-color: rgb(60, 60, 60); padding: 2px 4px; border-radius: 3px; font-size: 0.9em; word-break: break-word;">const_cast</code><span> </span>+<span> </span><code style="direction: ltr; unicode-bidi: bidi-override; font-family: monospace; color: rgb(208, 208, 208); background-color: rgb(60, 60, 60); padding: 2px 4px; border-radius: 3px; font-size: 0.9em; word-break: break-word;">std::move</code><span> </span>on loop iterator reference</h3><p style="direction: ltr; unicode-bidi: bidi-override; white-space: pre-wrap; margin-top: 0.1em; margin-bottom: 0.2em; color: rgb(204, 204, 204); font-family: -apple-system, BlinkMacSystemFont, "Segoe UI", Roboto, sans-serif; font-size: 13px; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: norma l; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; background-color: rgb(31, 31, 31); text-decoration-thickness: initial; text-decoration-style: initial; text-decoration-color: initial;"><a href="vscode-webview://12or9qsfo5kq61724hn7thg1f7a1a8i4j2ml00u56orm57h9kk50/be/src/runtime/routine_load/data_consumer.cpp#L928-L929" target="_blank" rel="noopener noreferrer" style="direction: ltr; unicode-bidi: bidi-override; color: rgb(77, 170, 252); text-decoration: rgb(77, 170, 252);">data_consumer.cpp:928-929</a>:</p><div class="codeBlockWrapper_-a7MRw" style="direction: ltr; unicode-bidi: bidi-override; position: relative; margin: 8px 0px; color: rgb(204, 204, 204); font-family: -apple-system, BlinkMacSystemFont, "Segoe UI", Roboto, sans-serif; font-size: 13px; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text- align: start; text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; white-space: normal; background-color: rgb(31, 31, 31); text-decoration-thickness: initial; text-decoration-style: initial; text-decoration-color: initial;"><button class="copyButton_CEmTFw copyButton_-a7MRw" title="Copy code" aria-label="Copy code to clipboard" style="direction: ltr; unicode-bidi: bidi-override; color: rgb(204, 204, 204); font-family: -apple-system, BlinkMacSystemFont, "Segoe UI", Roboto, sans-serif; font-size: 13px; background: none 0% 0% / auto repeat scroll padding-box border-box rgb(31, 31, 31); border-color: rgb(69, 69, 69); border-style: solid; border-width: 0.666667px; border-image: none 100% / 1 / 0 stretch; cursor: pointer; opacity: 0; display: flex; border-radius: 4px; justify-content: center; align-items: center; padding: 4px; transition: opacity 0.15s, background 0.15s; position: absolute; top: 4px; right: 4px;"><svg xmlns="http: //www.w3.org/2000/svg" viewBox="0 0 20 20" fill="currentColor" aria-hidden="true" data-slot="icon" class="copyIcon_CEmTFw"><path fill-rule="evenodd" d="M15.988 3.012A2.25 2.25 0 0 1 18 5.25v6.5A2.25 2.25 0 0 1 15.75 14H13.5v-3.379a3 3 0 0 0-.879-2.121l-3.12-3.121a3 3 0 0 0-1.402-.791 2.252 2.252 0 0 1 1.913-1.576A2.25 2.25 0 0 1 12.25 1h1.5a2.25 2.25 0 0 1 2.238 2.012ZM11.5 3.25a.75.75 0 0 1 .75-.75h1.5a.75.75 0 0 1 .75.75v.25h-3v-.25Z" clip-rule="evenodd"></path><path d="M3.5 6A1.5 1.5 0 0 0 2 7.5v9A1.5 1.5 0 0 0 3.5 18h7a1.5 1.5 0 0 0 1.5-1.5v-5.879a1.5 1.5 0 0 0-.44-1.06L8.44 6.439A1.5 1.5 0 0 0 7.378 6H3.5Z"></path></svg></button><pre style="direction: ltr; unicode-bidi: bidi-override; overflow-x: auto; white-space: pre; box-sizing: border-box; border-radius: 4px; max-width: 100%; margin: 0px; padding: 8px;"><code class="language-cpp" style="direction: ltr; unicode-bidi: bidi-override; font-family: monospace; color: rgb(208, 208, 208); background-color: rgb(60, 60, 60); padding: 0px; border-radius: 3px; font-size: 0.9em; word-break: break-word;">auto& records = result.GetRecords(); for (auto& record : records) { ... auto record_ptr = std::make_shared<Aws::Kinesis::Model::Record>(std::move( const_cast<Aws::Kinesis::Model::Record&>(record))); </code></pre></div><p style="direction: ltr; unicode-bidi: bidi-override; white-space: pre-wrap; margin-top: 0.1em; margin-bottom: 0.2em; color: rgb(204, 204, 204); font-family: -apple-system, BlinkMacSystemFont, "Segoe UI", Roboto, sans-serif; font-size: 13px; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; background-color: rgb(31, 31, 31); text-decoration-thickness: initial; text-decoration-style: initial; text-decoration-color: initial;">Even though <code style="direction: ltr; unicode-bidi: bidi-override; font-family: monospace; color: rgb(208, 208, 208); background-color: rgb(60, 60, 60); padding: 2px 4px; border-radius: 3px; font-size: 0.9em; word-break: break-word;">result</code> is taken by value, <code style="direction: ltr; unicode-bidi: bidi-override; font- family: monospace; color: rgb(208, 208, 208); background-color: rgb(60, 60, 60); padding: 2px 4px; border-radius: 3px; font-size: 0.9em; word-break: break-word;">GetRecords()</code> returns <code style="direction: ltr; unicode-bidi: bidi-override; font-family: monospace; color: rgb(208, 208, 208); background-color: rgb(60, 60, 60); padding: 2px 4px; border-radius: 3px; font-size: 0.9em; word-break: break-word;">const Aws::Vector<Record>&</code>. The <code style="direction: ltr; unicode-bidi: bidi-override; font-family: monospace; color: rgb(208, 208, 208); background-color: rgb(60, 60, 60); padding: 2px 4px; border-radius: 3px; font-size: 0.9em; word-break: break-word;">const_cast</code> removes const from a reference to a const member, then <code style="direction: ltr; unicode-bidi: bidi-override; font-family: monospace; color: rgb(208, 208, 208); background-color: rgb(60, 60, 60); padding: 2px 4px; border-radius: 3px; font-size: 0.9em; word-break: break-word;">std::move< /code> moves from it. This is technically <strong style="direction: ltr; unicode-bidi: bidi-override;">undefined behavior</strong> (modifying a const object). Even if it "works" in practice due to the AWS SDK implementation, it's fragile.</p><p style="direction: ltr; unicode-bidi: bidi-override; white-space: pre-wrap; margin-top: 0.1em; margin-bottom: 0.2em; color: rgb(204, 204, 204); font-family: -apple-system, BlinkMacSystemFont, "Segoe UI", Roboto, sans-serif; font-size: 13px; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; background-color: rgb(31, 31, 31); text-decoration-thickness: initial; text-decoration-style: initial; text-decoration-color: initial;"><strong style="direction: ltr; unicode-bidi: bidi-override;">Fix</strong>: Use <code style="direction: ltr; unic ode-bidi: bidi-override; font-family: monospace; color: rgb(208, 208, 208); background-color: rgb(60, 60, 60); padding: 2px 4px; border-radius: 3px; font-size: 0.9em; word-break: break-word;">result.GetRecords()</code> non-const overload or store the result's records via a mutable reference. Since <code style="direction: ltr; unicode-bidi: bidi-override; font-family: monospace; color: rgb(208, 208, 208); background-color: rgb(60, 60, 60); padding: 2px 4px; border-radius: 3px; font-size: 0.9em; word-break: break-word;">result</code> is by value, you can add a helper:</p><div class="codeBlockWrapper_-a7MRw" style="direction: ltr; unicode-bidi: bidi-override; position: relative; margin: 8px 0px; color: rgb(204, 204, 204); font-family: -apple-system, BlinkMacSystemFont, "Segoe UI", Roboto, sans-serif; font-size: 13px; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-inde nt: 0px; text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; white-space: normal; background-color: rgb(31, 31, 31); text-decoration-thickness: initial; text-decoration-style: initial; text-decoration-color: initial;"><button class="copyButton_CEmTFw copyButton_-a7MRw" title="Copy code" aria-label="Copy code to clipboard" style="direction: ltr; unicode-bidi: bidi-override; color: rgb(204, 204, 204); font-family: -apple-system, BlinkMacSystemFont, "Segoe UI", Roboto, sans-serif; font-size: 13px; background: none 0% 0% / auto repeat scroll padding-box border-box rgb(31, 31, 31); border-color: rgb(69, 69, 69); border-style: solid; border-width: 0.666667px; border-image: none 100% / 1 / 0 stretch; cursor: pointer; opacity: 0; display: flex; border-radius: 4px; justify-content: center; align-items: center; padding: 4px; transition: opacity 0.15s, background 0.15s; position: absolute; top: 4px; right: 4px;"><svg xmlns="http://www.w3.org/2000/svg"; viewBox="0 0 20 20" fill="currentColor" aria-hidden="true" data-slot="icon" class="copyIcon_CEmTFw"><path fill-rule="evenodd" d="M15.988 3.012A2.25 2.25 0 0 1 18 5.25v6.5A2.25 2.25 0 0 1 15.75 14H13.5v-3.379a3 3 0 0 0-.879-2.121l-3.12-3.121a3 3 0 0 0-1.402-.791 2.252 2.252 0 0 1 1.913-1.576A2.25 2.25 0 0 1 12.25 1h1.5a2.25 2.25 0 0 1 2.238 2.012ZM11.5 3.25a.75.75 0 0 1 .75-.75h1.5a.75.75 0 0 1 .75.75v.25h-3v-.25Z" clip-rule="evenodd"></path><path d="M3.5 6A1.5 1.5 0 0 0 2 7.5v9A1.5 1.5 0 0 0 3.5 18h7a1.5 1.5 0 0 0 1.5-1.5v-5.879a1.5 1.5 0 0 0-.44-1.06L8.44 6.439A1.5 1.5 0 0 0 7.378 6H3.5Z"></path></svg></button><pre style="direction: ltr; unicode-bidi: bidi-override; overflow-x: auto; white-space: pre; box-sizing: border-box; border-radius: 4px; max-width: 100%; margin: 0px; padding: 8px;"><code class="language-cpp" style="direction: ltr; unicode-bidi: bidi-override; font-family: monospace; color: rgb(208, 208, 208); background-color: rgb(60, 60, 60); padding: 0px; border-radius: 3p x; font-size: 0.9em; word-break: break-word;">// GetRecords returns const& but result is our owned copy, so we can safely cast auto& records = const_cast<Aws::Vector<Aws::Kinesis::Model::Record>&>(result.GetRecords()); for (auto& record : records) { auto record_ptr = std::make_shared<Aws::Kinesis::Model::Record>(std::move(record)); ... } </code></pre></div><p style="direction: ltr; unicode-bidi: bidi-override; white-space: pre-wrap; margin-top: 0.1em; margin-bottom: 0.2em; color: rgb(204, 204, 204); font-family: -apple-system, BlinkMacSystemFont, "Segoe UI", Roboto, sans-serif; font-size: 13px; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; background-color: rgb(31, 31, 31); text-decoration-thickness: initial; text-decoration-style: initial; text-decoration-color: initial;">Or better yet, check if AWS SDK provides <code style="direction: ltr; unicode-bidi: bidi-override; font-family: monospace; color: rgb(208, 208, 208); background-color: rgb(60, 60, 60); padding: 2px 4px; border-radius: 3px; font-size: 0.9em; word-break: break-word;">SetRecords()</code> / mutable access. The cleanest approach is to m ove the entire vector out once.</p><h3 style="direction: ltr; unicode-bidi: bidi-override; color: rgb(204, 204, 204); font-family: -apple-system, BlinkMacSystemFont, "Segoe UI", Roboto, sans-serif; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; white-space: normal; background-color: rgb(31, 31, 31); text-decoration-thickness: initial; text-decoration-style: initial; text-decoration-color: initial;">5.<span> </span><code style="direction: ltr; unicode-bidi: bidi-override; font-family: monospace; color: rgb(208, 208, 208); background-color: rgb(60, 60, 60); padding: 2px 4px; border-radius: 3px; font-size: 0.9em; word-break: break-word;">data_consumer_pool.cpp:83-87,100</code><span> </span>— Leftover debug artifacts</h3><p style="direction: ltr; unicode-bidi: bidi-override; white-space: pre-wrap; margin-top: 0.1em; margin-bottom: 0.2em; color: rgb(204, 204, 204); font-family: -apple-system, BlinkMacSystemFont, "Segoe UI", Roboto, sans-serif; font-size: 13px; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; background-color: rgb(31, 31, 31); text-decoration-thickness: initial; text-decoration-style: initial; text-decoration-color: initial;"><a href="vscode-webview://12or9qsfo5kq61724hn7thg1f7a1a8i4j2ml00u56orm57h9kk50/be/src/runtime/routine_load/data_consumer_pool.cpp#L83-L87" target="_blank" rel="noopener noreferrer" style="direction: ltr; unicode-bidi: bidi-override; color: rgb(77, 170, 252); text-decoration: rgb(77, 170, 252);">data_consumer_pool.cpp:83-87</a>: Commented-out old code remains:</p><div class="codeBlockWrapper_-a7MRw" style="direction: l tr; unicode-bidi: bidi-override; position: relative; margin: 8px 0px; color: rgb(204, 204, 204); font-family: -apple-system, BlinkMacSystemFont, "Segoe UI", Roboto, sans-serif; font-size: 13px; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; white-space: normal; background-color: rgb(31, 31, 31); text-decoration-thickness: initial; text-decoration-style: initial; text-decoration-color: initial;"><button class="copyButton_CEmTFw copyButton_-a7MRw" title="Copy code" aria-label="Copy code to clipboard" style="direction: ltr; unicode-bidi: bidi-override; color: rgb(204, 204, 204); font-family: -apple-system, BlinkMacSystemFont, "Segoe UI", Roboto, sans-serif; font-size: 13px; background: none 0% 0% / auto repeat scroll padding-box border-box rgb(31, 31, 31); border -color: rgb(69, 69, 69); border-style: solid; border-width: 0.666667px; border-image: none 100% / 1 / 0 stretch; cursor: pointer; opacity: 0; display: flex; border-radius: 4px; justify-content: center; align-items: center; padding: 4px; transition: opacity 0.15s, background 0.15s; position: absolute; top: 4px; right: 4px;"><svg xmlns="http://www.w3.org/2000/svg"; viewBox="0 0 20 20" fill="currentColor" aria-hidden="true" data-slot="icon" class="copyIcon_CEmTFw"><path fill-rule="evenodd" d="M15.988 3.012A2.25 2.25 0 0 1 18 5.25v6.5A2.25 2.25 0 0 1 15.75 14H13.5v-3.379a3 3 0 0 0-.879-2.121l-3.12-3.121a3 3 0 0 0-1.402-.791 2.252 2.252 0 0 1 1.913-1.576A2.25 2.25 0 0 1 12.25 1h1.5a2.25 2.25 0 0 1 2.238 2.012ZM11.5 3.25a.75.75 0 0 1 .75-.75h1.5a.75.75 0 0 1 .75.75v.25h-3v-.25Z" clip-rule="evenodd"></path><path d="M3.5 6A1.5 1.5 0 0 0 2 7.5v9A1.5 1.5 0 0 0 3.5 18h7a1.5 1.5 0 0 0 1.5-1.5v-5.879a1.5 1.5 0 0 0-.44-1.06L8.44 6.439A1.5 1.5 0 0 0 7.378 6H3.5Z"></path></svg></button><pre style="d irection: ltr; unicode-bidi: bidi-override; overflow-x: auto; white-space: pre; box-sizing: border-box; border-radius: 4px; max-width: 100%; margin: 0px; padding: 8px;"><code class="language-cpp" style="direction: ltr; unicode-bidi: bidi-override; font-family: monospace; color: rgb(208, 208, 208); background-color: rgb(60, 60, 60); padding: 0px; border-radius: 3px; font-size: 0.9em; word-break: break-word;">// if (ctx->load_src_type != TLoadSourceType::KAFKA) { // return Status::InternalError(...); // } // DCHECK(ctx->kafka_info); </code></pre></div><p style="direction: ltr; unicode-bidi: bidi-override; white-space: pre-wrap; margin-top: 0.1em; margin-bottom: 0.2em; color: rgb(204, 204, 204); font-family: -apple-system, BlinkMacSystemFont, "Segoe UI", Roboto, sans-serif; font-size: 13px; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; background-color: rgb(31, 31, 31); text-decoration-thickness: initial; text-decoration-style: initial; text-decoration-color: initial;"><a href="vscode-webview://12or9qsfo5kq61724hn7thg1f7a1a8i4j2ml00u56orm57h9kk50/be/src/runtime/routine_load/data_consumer_pool.cpp#L100" target="_blank" rel="noopener noreferrer" style="direction: ltr; unicode-bidi: bidi-override; color: rgb(77, 170, 252); text-decoration: rgb(77, 170, 252);">data_consumer_pool.cpp:100</a>: <code st yle="direction: ltr; unicode-bidi: bidi-override; font-family: monospace; color: rgb(208, 208, 208); background-color: rgb(60, 60, 60); padding: 2px 4px; border-radius: 3px; font-size: 0.9em; word-break: break-word;">(Refrain) TODO</code> comment still present:</p><div class="codeBlockWrapper_-a7MRw" style="direction: ltr; unicode-bidi: bidi-override; position: relative; margin: 8px 0px; color: rgb(204, 204, 204); font-family: -apple-system, BlinkMacSystemFont, "Segoe UI", Roboto, sans-serif; font-size: 13px; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; white-space: normal; background-color: rgb(31, 31, 31); text-decoration-thickness: initial; text-decoration-style: initial; text-decoration-color: initial;"><button class="copyButton_CEmTFw copyButton_-a7MRw" title="Co py code" aria-label="Copy code to clipboard" style="direction: ltr; unicode-bidi: bidi-override; color: rgb(204, 204, 204); font-family: -apple-system, BlinkMacSystemFont, "Segoe UI", Roboto, sans-serif; font-size: 13px; background: none 0% 0% / auto repeat scroll padding-box border-box rgb(31, 31, 31); border-color: rgb(69, 69, 69); border-style: solid; border-width: 0.666667px; border-image: none 100% / 1 / 0 stretch; cursor: pointer; opacity: 0; display: flex; border-radius: 4px; justify-content: center; align-items: center; padding: 4px; transition: opacity 0.15s, background 0.15s; position: absolute; top: 4px; right: 4px;"><svg xmlns="http://www.w3.org/2000/svg"; viewBox="0 0 20 20" fill="currentColor" aria-hidden="true" data-slot="icon" class="copyIcon_CEmTFw"><path fill-rule="evenodd" d="M15.988 3.012A2.25 2.25 0 0 1 18 5.25v6.5A2.25 2.25 0 0 1 15.75 14H13.5v-3.379a3 3 0 0 0-.879-2.121l-3.12-3.121a3 3 0 0 0-1.402-.791 2.252 2.252 0 0 1 1.913-1.576A2.25 2.25 0 0 1 12. 25 1h1.5a2.25 2.25 0 0 1 2.238 2.012ZM11.5 3.25a.75.75 0 0 1 .75-.75h1.5a.75.75 0 0 1 .75.75v.25h-3v-.25Z" clip-rule="evenodd"></path><path d="M3.5 6A1.5 1.5 0 0 0 2 7.5v9A1.5 1.5 0 0 0 3.5 18h7a1.5 1.5 0 0 0 1.5-1.5v-5.879a1.5 1.5 0 0 0-.44-1.06L8.44 6.439A1.5 1.5 0 0 0 7.378 6H3.5Z"></path></svg></button><pre style="direction: ltr; unicode-bidi: bidi-override; overflow-x: auto; white-space: pre; box-sizing: border-box; border-radius: 4px; max-width: 100%; margin: 0px; padding: 8px;"><code class="language-cpp" style="direction: ltr; unicode-bidi: bidi-override; font-family: monospace; color: rgb(208, 208, 208); background-color: rgb(60, 60, 60); padding: 0px; border-radius: 3px; font-size: 0.9em; word-break: break-word;">// (Refrain) TODO: For Kinesis, now just set consumer_num = 1 </code></pre></div><p style="direction: ltr; unicode-bidi: bidi-override; white-space: pre-wrap; margin-top: 0.1em; margin-bottom: 0.2em; color: rgb(204, 204, 204); font-family: -apple-system, BlinkMacSystemFont, "Segoe UI", Roboto, sans-serif; font-size: 13px; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; background-color: rgb(31, 31, 31); text-decoration-thickness: initial; text-decoration-style: initial; text-decoration-color: initial;"><strong style="direction: ltr; unicode-bidi: bidi-override;">Fix</strong>: Remove commented-out code and the TODO marker. The code should be clean for PR submission.</p><h3 style="direction: ltr; unicode-bidi: bidi-override; color: rgb(204, 204, 204); font-family: -apple-system, BlinkMacSystemFont, "Segoe UI", Roboto, san s-serif; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; white-space: normal; background-color: rgb(31, 31, 31); text-decoration-thickness: initial; text-decoration-style: initial; text-decoration-color: initial;">6.<span> </span><code style="direction: ltr; unicode-bidi: bidi-override; font-family: monospace; color: rgb(208, 208, 208); background-color: rgb(60, 60, 60); padding: 2px 4px; border-radius: 3px; font-size: 0.9em; word-break: break-word;">data_consumer_group.cpp:279</code><span> </span>—<span> </span><code style="direction: ltr; unicode-bidi: bidi-override; font-family: monospace; color: rgb(208, 208, 208); background-color: rgb(60, 60, 60); padding: 2px 4px; border-radius: 3px; font-size: 0.9em; word-break: break-word;">static_cast<void>(consumer->cancel(ctx))</code><span > </span>silently swallows error</h3><p style="direction: ltr; unicode-bidi: >bidi-override; white-space: pre-wrap; margin-top: 0.1em; margin-bottom: >0.2em; color: rgb(204, 204, 204); font-family: -apple-system, >BlinkMacSystemFont, "Segoe UI", Roboto, sans-serif; font-size: >13px; font-style: normal; font-variant-ligatures: normal; font-variant-caps: >normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: >start; text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px; >-webkit-text-stroke-width: 0px; background-color: rgb(31, 31, 31); >text-decoration-thickness: initial; text-decoration-style: initial; >text-decoration-color: initial;"><a >href="vscode-webview://12or9qsfo5kq61724hn7thg1f7a1a8i4j2ml00u56orm57h9kk50/be/src/runtime/routine_load/data_consumer_group.cpp#L279" > target="_blank" rel="noopener noreferrer" style="direction: ltr; >unicode-bidi: bidi-override; color: rgb(77, 170, 252); text-decoration: >rgb(77, 170, 252);">data_consumer_group.cp p:279</a> (in <code style="direction: ltr; unicode-bidi: bidi-override; font-family: monospace; color: rgb(208, 208, 208); background-color: rgb(60, 60, 60); padding: 2px 4px; border-radius: 3px; font-size: 0.9em; word-break: break-word;">KinesisDataConsumerGroup::start_all</code>):</p><div class="codeBlockWrapper_-a7MRw" style="direction: ltr; unicode-bidi: bidi-override; position: relative; margin: 8px 0px; color: rgb(204, 204, 204); font-family: -apple-system, BlinkMacSystemFont, "Segoe UI", Roboto, sans-serif; font-size: 13px; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; white-space: normal; background-color: rgb(31, 31, 31); text-decoration-thickness: initial; text-decoration-style: initial; text-decoration-color: initial;"><button class="copyButton_CEmTFw copyBu tton_-a7MRw" title="Copy code" aria-label="Copy code to clipboard" style="direction: ltr; unicode-bidi: bidi-override; color: rgb(204, 204, 204); font-family: -apple-system, BlinkMacSystemFont, "Segoe UI", Roboto, sans-serif; font-size: 13px; background: none 0% 0% / auto repeat scroll padding-box border-box rgb(31, 31, 31); border-color: rgb(69, 69, 69); border-style: solid; border-width: 0.666667px; border-image: none 100% / 1 / 0 stretch; cursor: pointer; opacity: 0; display: flex; border-radius: 4px; justify-content: center; align-items: center; padding: 4px; transition: opacity 0.15s, background 0.15s; position: absolute; top: 4px; right: 4px;"><svg xmlns="http://www.w3.org/2000/svg"; viewBox="0 0 20 20" fill="currentColor" aria-hidden="true" data-slot="icon" class="copyIcon_CEmTFw"><path fill-rule="evenodd" d="M15.988 3.012A2.25 2.25 0 0 1 18 5.25v6.5A2.25 2.25 0 0 1 15.75 14H13.5v-3.379a3 3 0 0 0-.879-2.121l-3.12-3.121a3 3 0 0 0-1.402-.791 2.252 2.252 0 0 1 1.913-1.5 76A2.25 2.25 0 0 1 12.25 1h1.5a2.25 2.25 0 0 1 2.238 2.012ZM11.5 3.25a.75.75 0 0 1 .75-.75h1.5a.75.75 0 0 1 .75.75v.25h-3v-.25Z" clip-rule="evenodd"></path><path d="M3.5 6A1.5 1.5 0 0 0 2 7.5v9A1.5 1.5 0 0 0 3.5 18h7a1.5 1.5 0 0 0 1.5-1.5v-5.879a1.5 1.5 0 0 0-.44-1.06L8.44 6.439A1.5 1.5 0 0 0 7.378 6H3.5Z"></path></svg></button><pre style="direction: ltr; unicode-bidi: bidi-override; overflow-x: auto; white-space: pre; box-sizing: border-box; border-radius: 4px; max-width: 100%; margin: 0px; padding: 8px;"><code class="language-cpp" style="direction: ltr; unicode-bidi: bidi-override; font-family: monospace; color: rgb(208, 208, 208); background-color: rgb(60, 60, 60); padding: 0px; border-radius: 3px; font-size: 0.9em; word-break: break-word;">static_cast<void>(consumer->cancel(ctx)); </code></pre></div><p style="direction: ltr; unicode-bidi: bidi-override; white-space: pre-wrap; margin-top: 0.1em; margin-bottom: 0.2em; color: rgb(204, 204, 204); font-family: -apple-system, BlinkMacSystemFont, "Segoe UI", Roboto, sans-serif; font-size: 13px; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; background-color: rgb(31, 31, 31); text-decoration-thickness: initial; text-decoration-style: initial; text-decoration-color: initial;">Per the review skill (Section 2.1.1): <em style="direction: ltr; unicode-bidi: bidi-override;">"<code style="direction: ltr; unicode-bidi: bidi-override; font-family: monospace; color: rgb(208, 208, 208); background-color: rgb(60, 60, 60); padding: 2px 4px; border-radius: 3px; font-size: 0.9em; word-break: break-word;">static_cast& lt;void>(...)</code> silent discard is prohibited. Never allow this pattern in new code."</em></p><p style="direction: ltr; unicode-bidi: bidi-override; white-space: pre-wrap; margin-top: 0.1em; margin-bottom: 0.2em; color: rgb(204, 204, 204); font-family: -apple-system, BlinkMacSystemFont, "Segoe UI", Roboto, sans-serif; font-size: 13px; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; background-color: rgb(31, 31, 31); text-decoration-thickness: initial; text-decoration-style: initial; text-decoration-color: initial;">The Kafka path has the same pattern (pre-existing), but new code should not replicate anti-patterns.</p><p style="direction: ltr; unicode-bidi: bidi-override; white-space: pre-wrap; margin-top: 0.1em; margin-bottom: 0.2em; color: rgb(204, 204, 204); fon t-family: -apple-system, BlinkMacSystemFont, "Segoe UI", Roboto, sans-serif; font-size: 13px; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; background-color: rgb(31, 31, 31); text-decoration-thickness: initial; text-decoration-style: initial; text-decoration-color: initial;"><strong style="direction: ltr; unicode-bidi: bidi-override;">Fix</strong>: At minimum, log the warning:</p><div class="codeBlockWrapper_-a7MRw" style="direction: ltr; unicode-bidi: bidi-override; position: relative; margin: 8px 0px; color: rgb(204, 204, 204); font-family: -apple-system, BlinkMacSystemFont, "Segoe UI", Roboto, sans-serif; font-size: 13px; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; tex t-align: start; text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; white-space: normal; background-color: rgb(31, 31, 31); text-decoration-thickness: initial; text-decoration-style: initial; text-decoration-color: initial;"><button class="copyButton_CEmTFw copyButton_-a7MRw" title="Copy code" aria-label="Copy code to clipboard" style="direction: ltr; unicode-bidi: bidi-override; color: rgb(204, 204, 204); font-family: -apple-system, BlinkMacSystemFont, "Segoe UI", Roboto, sans-serif; font-size: 13px; background: none 0% 0% / auto repeat scroll padding-box border-box rgb(31, 31, 31); border-color: rgb(69, 69, 69); border-style: solid; border-width: 0.666667px; border-image: none 100% / 1 / 0 stretch; cursor: pointer; opacity: 0; display: flex; border-radius: 4px; justify-content: center; align-items: center; padding: 4px; transition: opacity 0.15s, background 0.15s; position: absolute; top: 4px; right: 4px;"><svg xmlns="htt p://www.w3.org/2000/svg" viewBox="0 0 20 20" fill="currentColor" aria-hidden="true" data-slot="icon" class="copyIcon_CEmTFw"><path fill-rule="evenodd" d="M15.988 3.012A2.25 2.25 0 0 1 18 5.25v6.5A2.25 2.25 0 0 1 15.75 14H13.5v-3.379a3 3 0 0 0-.879-2.121l-3.12-3.121a3 3 0 0 0-1.402-.791 2.252 2.252 0 0 1 1.913-1.576A2.25 2.25 0 0 1 12.25 1h1.5a2.25 2.25 0 0 1 2.238 2.012ZM11.5 3.25a.75.75 0 0 1 .75-.75h1.5a.75.75 0 0 1 .75.75v.25h-3v-.25Z" clip-rule="evenodd"></path><path d="M3.5 6A1.5 1.5 0 0 0 2 7.5v9A1.5 1.5 0 0 0 3.5 18h7a1.5 1.5 0 0 0 1.5-1.5v-5.879a1.5 1.5 0 0 0-.44-1.06L8.44 6.439A1.5 1.5 0 0 0 7.378 6H3.5Z"></path></svg></button><pre style="direction: ltr; unicode-bidi: bidi-override; overflow-x: auto; white-space: pre; box-sizing: border-box; border-radius: 4px; max-width: 100%; margin: 0px; padding: 8px;"><code class="language-cpp" style="direction: ltr; unicode-bidi: bidi-override; font-family: monospace; color: rgb(208, 208, 208); background-color: rgb(60, 60, 60); paddin g: 0px; border-radius: 3px; font-size: 0.9em; word-break: break-word;">Status cancel_st = consumer->cancel(ctx); if (!cancel_st.ok()) { LOG(WARNING) << "failed to cancel kinesis consumer: " << consumer->id() << ", error: " << cancel_st; } </code></pre></div><hr style="direction: ltr; unicode-bidi: bidi-override; font-family: -apple-system, BlinkMacSystemFont, "Segoe UI", Roboto, sans-serif; font-size: 13px; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; white-space: normal; background-color: rgb(31, 31, 31); text-decoration-thickness: initial; text-decoration-style: initial; text-decoration-color: initial;"><h2 style="direction: ltr; unicode-bidi: bidi-override; color: rgb(204, 204, 204); font-family: -apple-system, BlinkMacSystemFont, "Segoe UI", Roboto, sans-serif; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-stroke-width : 0px; white-space: normal; background-color: rgb(31, 31, 31); text-decoration-thickness: initial; text-decoration-style: initial; text-decoration-color: initial;">MEDIUM Issues</h2><h3 style="direction: ltr; unicode-bidi: bidi-override; color: rgb(204, 204, 204); font-family: -apple-system, BlinkMacSystemFont, "Segoe UI", Roboto, sans-serif; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; white-space: normal; background-color: rgb(31, 31, 31); text-decoration-thickness: initial; text-decoration-style: initial; text-decoration-color: initial;">7.<span> </span><code style="direction: ltr; unicode-bidi: bidi-override; font-family: monospace; color: rgb(208, 208, 208); background-color: rgb(60, 60, 60); padding: 2px 4px; border-radius: 3px; font-size: 0.9em; word-break: break-word;">data_co nsumer_group.cpp:289</code><span> </span>—<span> </span><code style="direction: ltr; unicode-bidi: bidi-override; font-family: monospace; color: rgb(208, 208, 208); background-color: rgb(60, 60, 60); padding: 2px 4px; border-radius: 3px; font-size: 0.9em; word-break: break-word;">static_cast<void>(kinesis_pipe->finish())</code><span> </span>silently swallows error</h3><p style="direction: ltr; unicode-bidi: bidi-override; white-space: pre-wrap; margin-top: 0.1em; margin-bottom: 0.2em; color: rgb(204, 204, 204); font-family: -apple-system, BlinkMacSystemFont, "Segoe UI", Roboto, sans-serif; font-size: 13px; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; background-color: rgb(31, 31, 31); text-decoration-thickness: initial; text-decoration-style: initial; te xt-decoration-color: initial;"><a href="vscode-webview://12or9qsfo5kq61724hn7thg1f7a1a8i4j2ml00u56orm57h9kk50/be/src/runtime/routine_load/data_consumer_group.cpp#L289" target="_blank" rel="noopener noreferrer" style="direction: ltr; unicode-bidi: bidi-override; color: rgb(77, 170, 252); text-decoration: rgb(77, 170, 252);">data_consumer_group.cpp:289</a>:</p><div class="codeBlockWrapper_-a7MRw" style="direction: ltr; unicode-bidi: bidi-override; position: relative; margin: 8px 0px; color: rgb(204, 204, 204); font-family: -apple-system, BlinkMacSystemFont, "Segoe UI", Roboto, sans-serif; font-size: 13px; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; white-space: normal; background-color: rgb(31, 31, 31); text-decoration-thickness: initial; text-decoration-style: initial ; text-decoration-color: initial;"><button class="copyButton_CEmTFw copyButton_-a7MRw" title="Copy code" aria-label="Copy code to clipboard" style="direction: ltr; unicode-bidi: bidi-override; color: rgb(204, 204, 204); font-family: -apple-system, BlinkMacSystemFont, "Segoe UI", Roboto, sans-serif; font-size: 13px; background: none 0% 0% / auto repeat scroll padding-box border-box rgb(31, 31, 31); border-color: rgb(69, 69, 69); border-style: solid; border-width: 0.666667px; border-image: none 100% / 1 / 0 stretch; cursor: pointer; opacity: 0; display: flex; border-radius: 4px; justify-content: center; align-items: center; padding: 4px; transition: opacity 0.15s, background 0.15s; position: absolute; top: 4px; right: 4px;"><svg xmlns="http://www.w3.org/2000/svg"; viewBox="0 0 20 20" fill="currentColor" aria-hidden="true" data-slot="icon" class="copyIcon_CEmTFw"><path fill-rule="evenodd" d="M15.988 3.012A2.25 2.25 0 0 1 18 5.25v6.5A2.25 2.25 0 0 1 15.75 14H13.5v-3.379a3 3 0 0 0-.879-2.121l-3.12-3.121a3 3 0 0 0-1.402-.791 2.252 2.252 0 0 1 1.913-1.576A2.25 2.25 0 0 1 12.25 1h1.5a2.25 2.25 0 0 1 2.238 2.012ZM11.5 3.25a.75.75 0 0 1 .75-.75h1.5a.75.75 0 0 1 .75.75v.25h-3v-.25Z" clip-rule="evenodd"></path><path d="M3.5 6A1.5 1.5 0 0 0 2 7.5v9A1.5 1.5 0 0 0 3.5 18h7a1.5 1.5 0 0 0 1.5-1.5v-5.879a1.5 1.5 0 0 0-.44-1.06L8.44 6.439A1.5 1.5 0 0 0 7.378 6H3.5Z"></path></svg></button><pre style="direction: ltr; unicode-bidi: bidi-override; overflow-x: auto; white-space: pre; box-sizing: border-box; border-radius: 4px; max-width: 100%; margin: 0px; padding: 8px;"><code class="language-cpp" style="direction: ltr; unicode-bidi: bidi-override; font-family: monospace; color: rgb(208, 208, 208); background-color: rgb(60, 60, 60); padding: 0px; border-radius: 3px; font-size: 0.9em; word-break: break-word;">static_cast<void>(kinesis_pipe->finish()); </code></pre></div><p style="direction: ltr; unicode-bidi: bidi-override; white-space: pre-wrap; margin-top: 0.1em; margin-bottom: 0.2em; color: rgb(204, 204, 204); font-family: -apple-system, BlinkMacSystemFont, "Segoe UI", Roboto, sans-serif; font-size: 13px; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; background-color: rgb(31, 31, 31); text-decoration-thickness: initial; text-decoration-style: initial; text-decoration-color: initial;">If <code style="direction: ltr; unicode-bidi: bidi-override; font-family: monospace; color: rgb(208, 208, 208); background-color: rgb(60, 60, 60); padding: 2px 4px; border-radius: 3px; font-size: 0.9em; word-break: break-word;">finish()</code> fails, the error is silently ignored. This could mask data loss (partial pipe flush).</p> <p style="direction: ltr; unicode-bidi: bidi-override; white-space: pre-wrap; margin-top: 0.1em; margin-bottom: 0.2em; color: rgb(204, 204, 204); font-family: -apple-system, BlinkMacSystemFont, "Segoe UI", Roboto, sans-serif; font-size: 13px; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; background-color: rgb(31, 31, 31); text-decoration-thickness: initial; text-decoration-style: initial; text-decoration-color: initial;"><strong style="direction: ltr; unicode-bidi: bidi-override;">Fix</strong>: Check the return status and propagate or log appropriately.</p><h3 style="direction: ltr; unicode-bidi: bidi-override; color: rgb(204, 204, 204); font-family: -apple-system, BlinkMacSystemFont, "Segoe UI", Roboto, sans-serif; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; white-space: normal; background-color: rgb(31, 31, 31); text-decoration-thickness: initial; text-decoration-style: initial; text-decoration-color: initial;">8.<span> </span><code style="direction: ltr; unicode-bidi: bidi-override; font-family: monospace; color: rgb(208, 208, 208); background-color: rgb(60, 60, 60); padding: 2px 4px; border-radius: 3px; font-size: 0.9em; word-break: break-word;">data_consumer_group.h</code><span> </span>— Missing<span> </span><code style="direction: ltr; unicode-bidi: bidi-override; font-family: monospace; color: rgb(208, 208, 208); background-color: rgb(60, 60, 60); padding: 2px 4px; border-radius: 3px; font-size: 0.9em; word-break: break-word;">compile_check_begin.h</code><span> </span>/<span> </span><code style="direction: ltr; unicode-bidi: bidi-over ride; font-family: monospace; color: rgb(208, 208, 208); background-color: rgb(60, 60, 60); padding: 2px 4px; border-radius: 3px; font-size: 0.9em; word-break: break-word;">compile_check_end.h</code><span> </span>pairing validation</h3><p style="direction: ltr; unicode-bidi: bidi-override; white-space: pre-wrap; margin-top: 0.1em; margin-bottom: 0.2em; color: rgb(204, 204, 204); font-family: -apple-system, BlinkMacSystemFont, "Segoe UI", Roboto, sans-serif; font-size: 13px; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; background-color: rgb(31, 31, 31); text-decoration-thickness: initial; text-decoration-style: initial; text-decoration-color: initial;">The header includes <code style="direction: ltr; unicode-bidi: bidi-override; font-family: monospace; color: rgb(208, 208, 208); background-color: rgb(60, 60, 60); padding: 2px 4px; border-radius: 3px; font-size: 0.9em; word-break: break-word;">compile_check_begin.h</code> inside namespace <code style="direction: ltr; unicode-bidi: bidi-override; font-family: monospace; color: rgb(208, 208, 208); background-color: rgb(60, 60, 60); padding: 2px 4px; border-radius: 3px; font-size: 0.9em; word-break: break-word;">doris</code> (line 39) and <code style="direction: ltr; unicode-bidi: bidi-override; font-family: monospace; color: rgb(208, 208, 208); background-color: rgb(60, 60, 60); padding: 2px 4px; border-radius: 3px; font-size: 0.9em; word-break: break-word;">compile_check_end.h</code> at line 128. The new <code style="direction: ltr; unicode-bidi: bidi-override; font-family: monospace; color: rgb(208, 208, 208); background-color: rgb(60, 60, 60); padding: 2px 4px; border-radius: 3px; font-size: 0.9em; word-break: break-word;">KinesisDataConsumerGroup</code> class is correctly within this scope, whi ch is good. However, the <code style="direction: ltr; unicode-bidi: bidi-override; font-family: monospace; color: rgb(208, 208, 208); background-color: rgb(60, 60, 60); padding: 2px 4px; border-radius: 3px; font-size: 0.9em; word-break: break-word;">Aws::Kinesis::Model::Record</code> type is brought in via <code style="direction: ltr; unicode-bidi: bidi-override; font-family: monospace; color: rgb(208, 208, 208); background-color: rgb(60, 60, 60); padding: 2px 4px; border-radius: 3px; font-size: 0.9em; word-break: break-word;">data_consumer.h</code> which includes AWS SDK headers. Verify that AWS SDK headers don't trigger <code style="direction: ltr; unicode-bidi: bidi-override; font-family: monospace; color: rgb(208, 208, 208); background-color: rgb(60, 60, 60); padding: 2px 4px; border-radius: 3px; font-size: 0.9em; word-break: break-word;">-Wconversion</code> errors under the compile_check scope.</p><h3 style="direction: ltr; unicode-bidi: bidi-override; color: rgb(204, 204, 204) ; font-family: -apple-system, BlinkMacSystemFont, "Segoe UI", Roboto, sans-serif; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; white-space: normal; background-color: rgb(31, 31, 31); text-decoration-thickness: initial; text-decoration-style: initial; text-decoration-color: initial;">9.<span> </span><code style="direction: ltr; unicode-bidi: bidi-override; font-family: monospace; color: rgb(208, 208, 208); background-color: rgb(60, 60, 60); padding: 2px 4px; border-radius: 3px; font-size: 0.9em; word-break: break-word;">routine_load_task_executor.cpp:587-588</code><span> </span>— Accessing<span> </span><code style="direction: ltr; unicode-bidi: bidi-override; font-family: monospace; color: rgb(208, 208, 208); background-color: rgb(60, 60, 60); padding: 2px 4px; border-radius: 3px; font-size: 0.9em; word-break: break-word;">consumer_grp->consumers()[0]</code><span> </span>after consumers returned to pool</h3><p style="direction: ltr; unicode-bidi: bidi-override; white-space: pre-wrap; margin-top: 0.1em; margin-bottom: 0.2em; color: rgb(204, 204, 204); font-family: -apple-system, BlinkMacSystemFont, "Segoe UI", Roboto, sans-serif; font-size: 13px; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; background-color: rgb(31, 31, 31); text-decoration-thickness: initial; text-decoration-style: initial; text-decoration-color: initial;"><a href="vscode-webview://12or9qsfo5kq61724hn7thg1f7a1a8i4j2ml00u56orm57h9kk50/be/src/runtime/routine_load/routine_load_task_executor.cpp#L587-L588" target="_blank" rel="noopener noreferrer" style="direction: ltr; unicode -bidi: bidi-override; color: rgb(77, 170, 252); text-decoration: rgb(77, 170, 252);">routine_load_task_executor.cpp:587-588</a>:</p><div class="codeBlockWrapper_-a7MRw" style="direction: ltr; unicode-bidi: bidi-override; position: relative; margin: 8px 0px; color: rgb(204, 204, 204); font-family: -apple-system, BlinkMacSystemFont, "Segoe UI", Roboto, sans-serif; font-size: 13px; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; white-space: normal; background-color: rgb(31, 31, 31); text-decoration-thickness: initial; text-decoration-style: initial; text-decoration-color: initial;"><button class="copyButton_CEmTFw copyButton_-a7MRw" title="Copy code" aria-label="Copy code to clipboard" style="direction: ltr; unicode-bidi: bidi-override; color: rgb(204, 204, 204); font-fami ly: -apple-system, BlinkMacSystemFont, "Segoe UI", Roboto, sans-serif; font-size: 13px; background: none 0% 0% / auto repeat scroll padding-box border-box rgb(31, 31, 31); border-color: rgb(69, 69, 69); border-style: solid; border-width: 0.666667px; border-image: none 100% / 1 / 0 stretch; cursor: pointer; opacity: 0; display: flex; border-radius: 4px; justify-content: center; align-items: center; padding: 4px; transition: opacity 0.15s, background 0.15s; position: absolute; top: 4px; right: 4px;"><svg xmlns="http://www.w3.org/2000/svg"; viewBox="0 0 20 20" fill="currentColor" aria-hidden="true" data-slot="icon" class="copyIcon_CEmTFw"><path fill-rule="evenodd" d="M15.988 3.012A2.25 2.25 0 0 1 18 5.25v6.5A2.25 2.25 0 0 1 15.75 14H13.5v-3.379a3 3 0 0 0-.879-2.121l-3.12-3.121a3 3 0 0 0-1.402-.791 2.252 2.252 0 0 1 1.913-1.576A2.25 2.25 0 0 1 12.25 1h1.5a2.25 2.25 0 0 1 2.238 2.012ZM11.5 3.25a.75.75 0 0 1 .75-.75h1.5a.75.75 0 0 1 .75.75v.25h-3v-.25Z" clip-rule="evenodd"></path ><path d="M3.5 6A1.5 1.5 0 0 0 2 7.5v9A1.5 1.5 0 0 0 3.5 18h7a1.5 1.5 0 0 0 >1.5-1.5v-5.879a1.5 1.5 0 0 0-.44-1.06L8.44 6.439A1.5 1.5 0 0 0 7.378 >6H3.5Z"></path></svg></button><pre style="direction: ltr; unicode-bidi: >bidi-override; overflow-x: auto; white-space: pre; box-sizing: border-box; >border-radius: 4px; max-width: 100%; margin: 0px; padding: 8px;"><code >class="language-cpp" style="direction: ltr; unicode-bidi: bidi-override; >font-family: monospace; color: rgb(208, 208, 208); background-color: rgb(60, >60, 60); padding: 0px; border-radius: 3px; font-size: 0.9em; word-break: >break-word;">// Line 542: return consumers to pool BEFORE this point consumer_pool->return_consumers(consumer_grp.get()); ... // Line 587-588: accessing consumer from grp AFTER return auto kinesis_consumer = std::static_pointer_cast<KinesisDataConsumer>( consumer_grp->consumers()[0]); ctx->kinesis_info->millis_behind_latest = kinesis_consumer->get_millis_behind_latest(); </code></pre></div><p style="direction: ltr; unicode-bidi: bidi-override; white-space: pre-wrap; margin-top: 0.1em; margin-bottom: 0.2em; color: rgb(204, 204, 204); font-family: -apple-system, BlinkMacSystemFont, "Segoe UI", Roboto, sans-serif; font-size: 13px; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; background-color: rgb(31, 31, 31); text-decoration-thickness: initial; text-decoration-style: initial; text-decoration-color: initial;">At line 542, <code style="direction: ltr; unicode-bidi: bidi-override; font-family: monospace; color: rgb(208, 208, 208); background-color: rgb(60, 60, 60); padding: 2px 4px; border-radius: 3px; font-size: 0.9em; word-break: break-word;">return_consumers()</code> returns all consumers back to the pool. At line 587, code accesses <c ode style="direction: ltr; unicode-bidi: bidi-override; font-family: monospace; color: rgb(208, 208, 208); background-color: rgb(60, 60, 60); padding: 2px 4px; border-radius: 3px; font-size: 0.9em; word-break: break-word;">consumer_grp->consumers()[0]</code>. While <code style="direction: ltr; unicode-bidi: bidi-override; font-family: monospace; color: rgb(208, 208, 208); background-color: rgb(60, 60, 60); padding: 2px 4px; border-radius: 3px; font-size: 0.9em; word-break: break-word;">shared_ptr</code> keeps the consumer alive, another thread could have already obtained this consumer from the pool, called <code style="direction: ltr; unicode-bidi: bidi-override; font-family: monospace; color: rgb(208, 208, 208); background-color: rgb(60, 60, 60); padding: 2px 4px; border-radius: 3px; font-size: 0.9em; word-break: break-word;">reset()</code> on it (which clears <code style="direction: ltr; unicode-bidi: bidi-override; font-family: monospace; color: rgb(208, 208, 208); background- color: rgb(60, 60, 60); padding: 2px 4px; border-radius: 3px; font-size: 0.9em; word-break: break-word;">_millis_behind_latest</code>), and started using it. This is a <strong style="direction: ltr; unicode-bidi: bidi-override;">race condition</strong>.</p><p style="direction: ltr; unicode-bidi: bidi-override; white-space: pre-wrap; margin-top: 0.1em; margin-bottom: 0.2em; color: rgb(204, 204, 204); font-family: -apple-system, BlinkMacSystemFont, "Segoe UI", Roboto, sans-serif; font-size: 13px; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; background-color: rgb(31, 31, 31); text-decoration-thickness: initial; text-decoration-style: initial; text-decoration-color: initial;"><strong style="direction: ltr; unicode-bidi: bidi-override;">Fix</strong>: Move the millis_behind _latest extraction <strong style="direction: ltr; unicode-bidi: bidi-override;">before</strong> <code style="direction: ltr; unicode-bidi: bidi-override; font-family: monospace; color: rgb(208, 208, 208); background-color: rgb(60, 60, 60); padding: 2px 4px; border-radius: 3px; font-size: 0.9em; word-break: break-word;">return_consumers()</code> on line 542, or copy the data before returning consumers.</p><h3 style="direction: ltr; unicode-bidi: bidi-override; color: rgb(204, 204, 204); font-family: -apple-system, BlinkMacSystemFont, "Segoe UI", Roboto, sans-serif; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; white-space: normal; background-color: rgb(31, 31, 31); text-decoration-thickness: initial; text-decoration-style: initial; text-decoration-color: initial;">10.<span> </span><code style="direction: ltr; unicode-bidi: bidi-override; font-family: monospace; color: rgb(208, 208, 208); background-color: rgb(60, 60, 60); padding: 2px 4px; border-radius: 3px; font-size: 0.9em; word-break: break-word;">kinesis_consumer_pipe.h</code><span> </span>— Missing<span> </span><code style="direction: ltr; unicode-bidi: bidi-override; font-family: monospace; color: rgb(208, 208, 208); background-color: rgb(60, 60, 60); padding: 2px 4px; border-radius: 3px; font-size: 0.9em; word-break: break-word;">compile_check_begin.h</code><span> </span>/<span> </span><code style="direction: ltr; unicode-bidi: bidi-override; font-family: monospace; color: rgb(208, 208, 208); background-color: rgb(60, 60, 60); padding: 2px 4px; border-radius: 3px; font-size: 0.9em; word-break: break-word;">compile_check_end.h</code></h3><p style="direction: ltr; unicode-bidi: bidi-override; white-space: pre-wrap; margin-top: 0.1em; margin-bottom: 0.2em; color: rgb(204, 204, 204); font-family: -apple- system, BlinkMacSystemFont, "Segoe UI", Roboto, sans-serif; font-size: 13px; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; background-color: rgb(31, 31, 31); text-decoration-thickness: initial; text-decoration-style: initial; text-decoration-color: initial;"><a href="vscode-webview://12or9qsfo5kq61724hn7thg1f7a1a8i4j2ml00u56orm57h9kk50/be/src/io/fs/kinesis_consumer_pipe.h" target="_blank" rel="noopener noreferrer" style="direction: ltr; unicode-bidi: bidi-override; color: rgb(77, 170, 252); text-decoration: rgb(77, 170, 252);">kinesis_consumer_pipe.h</a> doesn't include <code style="direction: ltr; unicode-bidi: bidi-override; font-family: monospace; color: rgb(208, 208, 208); background-color: rgb(60, 60, 60); padding: 2px 4px; border-radius: 3px; font-size: 0.9em; wo rd-break: break-word;">compile_check_begin.h</code> / <code style="direction: ltr; unicode-bidi: bidi-override; font-family: monospace; color: rgb(208, 208, 208); background-color: rgb(60, 60, 60); padding: 2px 4px; border-radius: 3px; font-size: 0.9em; word-break: break-word;">compile_check_end.h</code>. Per Section 2.1.3 of the review guide: <em style="direction: ltr; unicode-bidi: bidi-override;">"Do new .h files include paired compile_check_begin.h / compile_check_end.h?"</em></p><p style="direction: ltr; unicode-bidi: bidi-override; white-space: pre-wrap; margin-top: 0.1em; margin-bottom: 0.2em; color: rgb(204, 204, 204); font-family: -apple-system, BlinkMacSystemFont, "Segoe UI", Roboto, sans-serif; font-size: 13px; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; b ackground-color: rgb(31, 31, 31); text-decoration-thickness: initial; text-decoration-style: initial; text-decoration-color: initial;">Note that <code style="direction: ltr; unicode-bidi: bidi-override; font-family: monospace; color: rgb(208, 208, 208); background-color: rgb(60, 60, 60); padding: 2px 4px; border-radius: 3px; font-size: 0.9em; word-break: break-word;">kafka_consumer_pipe.h</code> also doesn't have them, so this may be acceptable for consistency with the existing pattern, but ideally new headers should include them.</p><h3 style="direction: ltr; unicode-bidi: bidi-override; color: rgb(204, 204, 204); font-family: -apple-system, BlinkMacSystemFont, "Segoe UI", Roboto, sans-serif; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; white-space: normal; background-color: rgb(31, 3 1, 31); text-decoration-thickness: initial; text-decoration-style: initial; text-decoration-color: initial;">11.<span> </span><code style="direction: ltr; unicode-bidi: bidi-override; font-family: monospace; color: rgb(208, 208, 208); background-color: rgb(60, 60, 60); padding: 2px 4px; border-radius: 3px; font-size: 0.9em; word-break: break-word;">data_consumer_group.cpp:315-316</code><span> </span>—<span> </span><code style="direction: ltr; unicode-bidi: bidi-override; font-family: monospace; color: rgb(208, 208, 208); background-color: rgb(60, 60, 60); padding: 2px 4px; border-radius: 3px; font-size: 0.9em; word-break: break-word;">left_bytes -= len</code><span> </span>potential narrowing</h3><p style="direction: ltr; unicode-bidi: bidi-override; white-space: pre-wrap; margin-top: 0.1em; margin-bottom: 0.2em; color: rgb(204, 204, 204); font-family: -apple-system, BlinkMacSystemFont, "Segoe UI", Roboto, sans-serif; font-size: 13px; font-style: normal; font-variant- ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; background-color: rgb(31, 31, 31); text-decoration-thickness: initial; text-decoration-style: initial; text-decoration-color: initial;"><a href="vscode-webview://12or9qsfo5kq61724hn7thg1f7a1a8i4j2ml00u56orm57h9kk50/be/src/runtime/routine_load/data_consumer_group.cpp#L315-L316" target="_blank" rel="noopener noreferrer" style="direction: ltr; unicode-bidi: bidi-override; color: rgb(77, 170, 252); text-decoration: rgb(77, 170, 252);">data_consumer_group.cpp:315-316</a>:</p><div class="codeBlockWrapper_-a7MRw" style="direction: ltr; unicode-bidi: bidi-override; position: relative; margin: 8px 0px; color: rgb(204, 204, 204); font-family: -apple-system, BlinkMacSystemFont, "Segoe UI", Roboto, sans-serif; font-size: 13px; font-style: normal; font-variant-ligature s: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; white-space: normal; background-color: rgb(31, 31, 31); text-decoration-thickness: initial; text-decoration-style: initial; text-decoration-color: initial;"><button class="copyButton_CEmTFw copyButton_-a7MRw" title="Copy code" aria-label="Copy code to clipboard" style="direction: ltr; unicode-bidi: bidi-override; color: rgb(204, 204, 204); font-family: -apple-system, BlinkMacSystemFont, "Segoe UI", Roboto, sans-serif; font-size: 13px; background: none 0% 0% / auto repeat scroll padding-box border-box rgb(31, 31, 31); border-color: rgb(69, 69, 69); border-style: solid; border-width: 0.666667px; border-image: none 100% / 1 / 0 stretch; cursor: pointer; opacity: 0; display: flex; border-radius: 4px; justify-content: center; align-items: center; padding: 4px; transiti on: opacity 0.15s, background 0.15s; position: absolute; top: 4px; right: 4px;"><svg xmlns="http://www.w3.org/2000/svg"; viewBox="0 0 20 20" fill="currentColor" aria-hidden="true" data-slot="icon" class="copyIcon_CEmTFw"><path fill-rule="evenodd" d="M15.988 3.012A2.25 2.25 0 0 1 18 5.25v6.5A2.25 2.25 0 0 1 15.75 14H13.5v-3.379a3 3 0 0 0-.879-2.121l-3.12-3.121a3 3 0 0 0-1.402-.791 2.252 2.252 0 0 1 1.913-1.576A2.25 2.25 0 0 1 12.25 1h1.5a2.25 2.25 0 0 1 2.238 2.012ZM11.5 3.25a.75.75 0 0 1 .75-.75h1.5a.75.75 0 0 1 .75.75v.25h-3v-.25Z" clip-rule="evenodd"></path><path d="M3.5 6A1.5 1.5 0 0 0 2 7.5v9A1.5 1.5 0 0 0 3.5 18h7a1.5 1.5 0 0 0 1.5-1.5v-5.879a1.5 1.5 0 0 0-.44-1.06L8.44 6.439A1.5 1.5 0 0 0 7.378 6H3.5Z"></path></svg></button><pre style="direction: ltr; unicode-bidi: bidi-override; overflow-x: auto; white-space: pre; box-sizing: border-box; border-radius: 4px; max-width: 100%; margin: 0px; padding: 8px;"><code class="language-cpp" style="direction: ltr; unicode-bidi: bidi-overrid e; font-family: monospace; color: rgb(208, 208, 208); background-color: rgb(60, 60, 60); padding: 0px; border-radius: 3px; font-size: 0.9em; word-break: break-word;">left_bytes -= len; </code></pre></div><p style="direction: ltr; unicode-bidi: bidi-override; white-space: pre-wrap; margin-top: 0.1em; margin-bottom: 0.2em; color: rgb(204, 204, 204); font-family: -apple-system, BlinkMacSystemFont, "Segoe UI", Roboto, sans-serif; font-size: 13px; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; background-color: rgb(31, 31, 31); text-decoration-thickness: initial; text-decoration-style: initial; text-decoration-color: initial;"><code style="direction: ltr; unicode-bidi: bidi-override; font-family: monospace; color: rgb(208, 208, 208); background-color: rgb(60, 60, 60); padding: 2px 4px; border-radius: 3px; font-size: 0.9em; word-break: break-word;">left_bytes</code> is <code style="direction: ltr; unicode-bidi: bidi-override; font-family: monospace; color : rgb(208, 208, 208); background-color: rgb(60, 60, 60); padding: 2px 4px; border-radius: 3px; font-size: 0.9em; word-break: break-word;">int64_t</code>, <code style="direction: ltr; unicode-bidi: bidi-override; font-family: monospace; color: rgb(208, 208, 208); background-color: rgb(60, 60, 60); padding: 2px 4px; border-radius: 3px; font-size: 0.9em; word-break: break-word;">len</code> is <code style="direction: ltr; unicode-bidi: bidi-override; font-family: monospace; color: rgb(208, 208, 208); background-color: rgb(60, 60, 60); padding: 2px 4px; border-radius: 3px; font-size: 0.9em; word-break: break-word;">size_t</code>. Under <code style="direction: ltr; unicode-bidi: bidi-override; font-family: monospace; color: rgb(208, 208, 208); background-color: rgb(60, 60, 60); padding: 2px 4px; border-radius: 3px; font-size: 0.9em; word-break: break-word;">compile_check</code>, this may trigger a <code style="direction: ltr; unicode-bidi: bidi-override; font-family: monospace; color: rgb (208, 208, 208); background-color: rgb(60, 60, 60); padding: 2px 4px; border-radius: 3px; font-size: 0.9em; word-break: break-word;">-Wconversion</code> warning (implicit <code style="direction: ltr; unicode-bidi: bidi-override; font-family: monospace; color: rgb(208, 208, 208); background-color: rgb(60, 60, 60); padding: 2px 4px; border-radius: 3px; font-size: 0.9em; word-break: break-word;">size_t</code> → <code style="direction: ltr; unicode-bidi: bidi-override; font-family: monospace; color: rgb(208, 208, 208); background-color: rgb(60, 60, 60); padding: 2px 4px; border-radius: 3px; font-size: 0.9em; word-break: break-word;">int64_t</code>). The Kafka path has the same pattern but was written before compile_check was enforced.</p><hr style="direction: ltr; unicode-bidi: bidi-override; font-family: -apple-system, BlinkMacSystemFont, "Segoe UI", Roboto, sans-serif; font-size: 13px; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weig ht: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; white-space: normal; background-color: rgb(31, 31, 31); text-decoration-thickness: initial; text-decoration-style: initial; text-decoration-color: initial;"><h2 style="direction: ltr; unicode-bidi: bidi-override; color: rgb(204, 204, 204); font-family: -apple-system, BlinkMacSystemFont, "Segoe UI", Roboto, sans-serif; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; white-space: normal; background-color: rgb(31, 31, 31); text-decoration-thickness: initial; text-decoration-style: initial; text-decoration-color: initial;">LOW Issues</h2><h3 style="direction: ltr; unicode-bidi: bidi-override; color: rgb(204, 204, 204); font-famil y: -apple-system, BlinkMacSystemFont, "Segoe UI", Roboto, sans-serif; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; white-space: normal; background-color: rgb(31, 31, 31); text-decoration-thickness: initial; text-decoration-style: initial; text-decoration-color: initial;">12.<span> </span><code style="direction: ltr; unicode-bidi: bidi-override; font-family: monospace; color: rgb(208, 208, 208); background-color: rgb(60, 60, 60); padding: 2px 4px; border-radius: 3px; font-size: 0.9em; word-break: break-word;">data_consumer_pool.cpp:104-106</code><span> </span>— Using<span> </span><code style="direction: ltr; unicode-bidi: bidi-override; font-family: monospace; color: rgb(208, 208, 208); background-color: rgb(60, 60, 60); padding: 2px 4px; border-radius: 3px; font-size: 0.9em; word- break: break-word;">stringstream</code><span> </span>instead of<span> </span><code style="direction: ltr; unicode-bidi: bidi-override; font-family: monospace; color: rgb(208, 208, 208); background-color: rgb(60, 60, 60); padding: 2px 4px; border-radius: 3px; font-size: 0.9em; word-break: break-word;">fmt</code>/<code style="direction: ltr; unicode-bidi: bidi-override; font-family: monospace; color: rgb(208, 208, 208); background-color: rgb(60, 60, 60); padding: 2px 4px; border-radius: 3px; font-size: 0.9em; word-break: break-word;">Status::Cancelled(fmt, ...)</code></h3><div class="codeBlockWrapper_-a7MRw" style="direction: ltr; unicode-bidi: bidi-override; position: relative; margin: 8px 0px; color: rgb(204, 204, 204); font-family: -apple-system, BlinkMacSystemFont, "Segoe UI", Roboto, sans-serif; font-size: 13px; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-i ndent: 0px; text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; white-space: normal; background-color: rgb(31, 31, 31); text-decoration-thickness: initial; text-decoration-style: initial; text-decoration-color: initial;"><button class="copyButton_CEmTFw copyButton_-a7MRw" title="Copy code" aria-label="Copy code to clipboard" style="direction: ltr; unicode-bidi: bidi-override; color: rgb(204, 204, 204); font-family: -apple-system, BlinkMacSystemFont, "Segoe UI", Roboto, sans-serif; font-size: 13px; background: none 0% 0% / auto repeat scroll padding-box border-box rgb(31, 31, 31); border-color: rgb(69, 69, 69); border-style: solid; border-width: 0.666667px; border-image: none 100% / 1 / 0 stretch; cursor: pointer; opacity: 0; display: flex; border-radius: 4px; justify-content: center; align-items: center; padding: 4px; transition: opacity 0.15s, background 0.15s; position: absolute; top: 4px; right: 4px;"><svg xmlns="http://www.w3.org/2000/sv g" viewBox="0 0 20 20" fill="currentColor" aria-hidden="true" data-slot="icon" class="copyIcon_CEmTFw"><path fill-rule="evenodd" d="M15.988 3.012A2.25 2.25 0 0 1 18 5.25v6.5A2.25 2.25 0 0 1 15.75 14H13.5v-3.379a3 3 0 0 0-.879-2.121l-3.12-3.121a3 3 0 0 0-1.402-.791 2.252 2.252 0 0 1 1.913-1.576A2.25 2.25 0 0 1 12.25 1h1.5a2.25 2.25 0 0 1 2.238 2.012ZM11.5 3.25a.75.75 0 0 1 .75-.75h1.5a.75.75 0 0 1 .75.75v.25h-3v-.25Z" clip-rule="evenodd"></path><path d="M3.5 6A1.5 1.5 0 0 0 2 7.5v9A1.5 1.5 0 0 0 3.5 18h7a1.5 1.5 0 0 0 1.5-1.5v-5.879a1.5 1.5 0 0 0-.44-1.06L8.44 6.439A1.5 1.5 0 0 0 7.378 6H3.5Z"></path></svg></button><pre style="direction: ltr; unicode-bidi: bidi-override; overflow-x: auto; white-space: pre; box-sizing: border-box; border-radius: 4px; max-width: 100%; margin: 0px; padding: 8px;"><code class="language-cpp" style="direction: ltr; unicode-bidi: bidi-override; font-family: monospace; color: rgb(208, 208, 208); background-color: rgb(60, 60, 60); padding: 0px; border-radius: 3px; font-size: 0.9em; word-break: break-word;">std::stringstream ss; ss << "unknown routine load task type: " << ctx->load_type; return Status::Cancelled(ss.str()); </code></pre></div><p style="direction: ltr; unicode-bidi: bidi-override; white-space: pre-wrap; margin-top: 0.1em; margin-bottom: 0.2em; color: rgb(204, 204, 204); font-family: -apple-system, BlinkMacSystemFont, "Segoe UI", Roboto, sans-serif; font-size: 13px; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; background-color: rgb(31, 31, 31); text-decoration-thickness: initial; text-decoration-style: initial; text-decoration-color: initial;">Could use the simpler format: <code style="direction: ltr; unicode-bidi: bidi-override; font-family: monospace; color: rgb(208, 208, 208); background-color: rgb(60, 60, 60); padding: 2px 4px; border-radius: 3px; font-size: 0.9em; word-break: break-word;">return Status::Cancelled("unknown routine load task type: {}", ctx->load_ty pe);</code></p><h3 style="direction: ltr; unicode-bidi: bidi-override; color: rgb(204, 204, 204); font-family: -apple-system, BlinkMacSystemFont, "Segoe UI", Roboto, sans-serif; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; white-space: normal; background-color: rgb(31, 31, 31); text-decoration-thickness: initial; text-decoration-style: initial; text-decoration-color: initial;">13. Test coverage — Zero tests for new Kinesis path</h3><p style="direction: ltr; unicode-bidi: bidi-override; white-space: pre-wrap; margin-top: 0.1em; margin-bottom: 0.2em; color: rgb(204, 204, 204); font-family: -apple-system, BlinkMacSystemFont, "Segoe UI", Roboto, sans-serif; font-size: 13px; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-s pacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; background-color: rgb(31, 31, 31); text-decoration-thickness: initial; text-decoration-style: initial; text-decoration-color: initial;">No unit tests or regression tests were added for the Kinesis routine load feature. At minimum, the <code style="direction: ltr; unicode-bidi: bidi-override; font-family: monospace; color: rgb(208, 208, 208); background-color: rgb(60, 60, 60); padding: 2px 4px; border-radius: 3px; font-size: 0.9em; word-break: break-word;">get_consumer_grp()</code> function should have a test validating it creates the correct consumer group type for each source type.</p><hr style="direction: ltr; unicode-bidi: bidi-override; font-family: -apple-system, BlinkMacSystemFont, "Segoe UI", Roboto, sans-serif; font-size: 13px; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; white-space: normal; background-color: rgb(31, 31, 31); text-decoration-thickness: initial; text-decoration-style: initial; text-decoration-color: initial;"><h2 style="direction: ltr; unicode-bidi: bidi-override; color: rgb(204, 204, 204); font-family: -apple-system, BlinkMacSystemFont, "Segoe UI", Roboto, sans-serif; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; white-space: normal; background-color: rgb(31, 31, 31); text-decoration-thickness: initial; text-decoration-style: initial; text-decoration-color: initial;">Summary Table</h2> # | Severity | File | Description -- | -- | -- | -- 1 | BLOCKER | data_consumer_pool.cpp:89 | Null ptr dereference: ctx->kafka_info accessed for Kinesis tasks 2 | BLOCKER | data_consumer_pool.cpp:109 | Creates base DataConsumerGroup instead of typed subclass → UB on static_pointer_cast 3 | BLOCKER | data_consumer.cpp:871 | Use-after-move: result.GetNextShardIterator() after std::move(result) 4 | HIGH | data_consumer.cpp:928 | const_cast + std::move on const ref is technically UB 5 | HIGH | data_consumer_pool.cpp:83-100 | Leftover commented code and (Refrain) TODO 6 | HIGH | data_consumer_group.cpp:279 | static_cast<void> on cancel return — silently swallows error in new code 7 | MEDIUM | data_consumer_group.cpp:289 | static_cast<void> on finish() return — could mask data loss 8 | MEDIUM | data_consumer_group.h | Verify AWS SDK headers don't conflict with compile_check 9 | MEDIUM | routine_load_task_executor.cpp:587 | Race: accessing consumer after returning to pool 10 | MEDIUM | kinesis_consumer_pipe.h | Missing compile_check begin/end 11 | MEDIUM | data_consumer_group.cpp:315 | Potential size_t → int64_t narrowing under compile_check 12 | LOW | data_consumer_pool.cpp:104 | Use fmt instead of stringstream 13 | LOW | — | Zero test coverage for Kinesis path <p style="direction: ltr; unicode-bidi: bidi-override; white-space: pre-wrap; margin-top: 0.1em; margin-bottom: 0.2em; color: rgb(204, 204, 204); font-family: -apple-system, BlinkMacSystemFont, "Segoe UI", Roboto, sans-serif; font-size: 13px; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; background-color: rgb(31, 31, 31); text-decoration-thickness: initial; text-decoration-style: initial; text-decoration-color: initial;">Issues <strong style="direction: ltr; unicode-bidi: bidi-override;">#1, #2, #3</strong> are <strong style="direction: ltr; unicode-bidi: bidi-override;">critical</strong> and will cause crashes or undefined behavior at runtime. They must be fixed before merging.</p><!--EndFragment--> </body> </html>Code Review: Kinesis Routine Load (commit 7c79c34a73e) BLOCKER Issues 1. data_consumer_pool.cpp:89 — Null pointer dereference for Kinesis tasks [data_consumer_pool.cpp:89](vscode-webview://12or9qsfo5kq61724hn7thg1f7a1a8i4j2ml00u56orm57h9kk50/be/src/runtime/routine_load/data_consumer_pool.cpp#L89): The old check ctx->kafka_info->begin_offset.size() == 0 is still unconditionally executed before the switch. For Kinesis tasks, ctx->kafka_info is nullptr, so this will crash with a segfault. // Line 89 — CRASH: ctx->kafka_info is nullptr for Kinesis tasks if (ctx->kafka_info->begin_offset.size() == 0) { return Status::InternalError("PAUSE: The size of begin_offset of task should not be 0."); } Fix: Move validation into the per-source-type switch branches: switch (ctx->load_src_type) { case TLoadSourceType::KAFKA: { DCHECK(ctx->kafka_info); if (ctx->kafka_info->begin_offset.size() == 0) { return Status::InternalError("PAUSE: ..."); } consumer_num = std::min(consumer_num, ctx->kafka_info->begin_offset.size()); break; } case TLoadSourceType::KINESIS: { DCHECK(ctx->kinesis_info); if (ctx->kinesis_info->begin_sequence_number.size() == 0) { return Status::InternalError("PAUSE: ..."); } consumer_num = 1; break; } ... 2. [data_consumer_pool.cpp:109](vscode-webview://12or9qsfo5kq61724hn7thg1f7a1a8i4j2ml00u56orm57h9kk50/be/src/runtime/routine_load/data_consumer_pool.cpp#L109) — Creates base class DataConsumerGroup instead of typed subclass data_consumer_pool.cpp:109: std::shared_ptr<DataConsumerGroup> grp = std::make_shared<DataConsumerGroup>(consumer_num); This creates the base class DataConsumerGroup, not KafkaDataConsumerGroup or KinesisDataConsumerGroup. The base class start_all() simply returns Status::OK() without consuming anything. Later in routine_load_task_executor.cpp:455,472, static_pointer_cast<KafkaDataConsumerGroup> / static_pointer_cast<KinesisDataConsumerGroup> will produce undefined behavior (casting base to derived when the object is not actually derived). Fix: Create typed consumer groups inside the switch: std::shared_ptr<DataConsumerGroup> grp; switch (ctx->load_src_type) { case TLoadSourceType::KAFKA: ... grp = std::make_shared<KafkaDataConsumerGroup>(consumer_num); break; case TLoadSourceType::KINESIS: ... grp = std::make_shared<KinesisDataConsumerGroup>(consumer_num); break; ... } 3. data_consumer.cpp:871 — Use-after-move: accessing result after std::move [data_consumer.cpp:864-871](vscode-webview://12or9qsfo5kq61724hn7thg1f7a1a8i4j2ml00u56orm57h9kk50/be/src/runtime/routine_load/data_consumer.cpp#L864-L871): auto result = outcome.GetResultWithOwnership(); auto millis_behind = result.GetMillisBehindLatest(); RETURN_IF_ERROR(_process_records(shard_id, std::move(result), queue, ...)); // ... std::string next_iterator = result.GetNextShardIterator(); // UB: result was moved! After std::move(result) on line 864, accessing result.GetNextShardIterator() on line 871 is undefined behavior. The moved-from object is in an indeterminate state. Fix: Save next_iterator before the move: auto result = outcome.GetResultWithOwnership(); auto millis_behind = result.GetMillisBehindLatest(); std::string next_iterator = result.GetNextShardIterator(); RETURN_IF_ERROR(_process_records(shard_id, std::move(result), queue, ...)); _millis_behind_latest[shard_id] = millis_behind; // use next_iterator below... HIGH Issues 4. data_consumer.cpp:928-929 — Dangerous const_cast + std::move on loop iterator reference [data_consumer.cpp:928-929](vscode-webview://12or9qsfo5kq61724hn7thg1f7a1a8i4j2ml00u56orm57h9kk50/be/src/runtime/routine_load/data_consumer.cpp#L928-L929): auto& records = result.GetRecords(); for (auto& record : records) { ... auto record_ptr = std::make_shared<Aws::Kinesis::Model::Record>(std::move( const_cast<Aws::Kinesis::Model::Record&>(record))); Even though result is taken by value, GetRecords() returns const Aws::Vector<Record>&. The const_cast removes const from a reference to a const member, then std::move moves from it. This is technically undefined behavior (modifying a const object). Even if it "works" in practice due to the AWS SDK implementation, it's fragile. Fix: Use result.GetRecords() non-const overload or store the result's records via a mutable reference. Since result is by value, you can add a helper: // GetRecords returns const& but result is our owned copy, so we can safely cast auto& records = const_cast<Aws::Vector<Aws::Kinesis::Model::Record>&>(result.GetRecords()); for (auto& record : records) { auto record_ptr = std::make_shared<Aws::Kinesis::Model::Record>(std::move(record)); ... } Or better yet, check if AWS SDK provides SetRecords() / mutable access. The cleanest approach is to move the entire vector out once. 5. [data_consumer_pool.cpp:83-87](vscode-webview://12or9qsfo5kq61724hn7thg1f7a1a8i4j2ml00u56orm57h9kk50/be/src/runtime/routine_load/data_consumer_pool.cpp#L83-L87),100 — Leftover debug artifacts data_consumer_pool.cpp:83-87: Commented-out old code remains: // if (ctx->load_src_type != TLoadSourceType::KAFKA) { // return Status::InternalError(...); // } // DCHECK(ctx->kafka_info); [data_consumer_pool.cpp:100](vscode-webview://12or9qsfo5kq61724hn7thg1f7a1a8i4j2ml00u56orm57h9kk50/be/src/runtime/routine_load/data_consumer_pool.cpp#L100): (Refrain) TODO comment still present: // (Refrain) TODO: For Kinesis, now just set consumer_num = 1 Fix: Remove commented-out code and the TODO marker. The code should be clean for PR submission. 6. data_consumer_group.cpp:279 — static_cast<void>(consumer->cancel(ctx)) silently swallows error [data_consumer_group.cpp:279](vscode-webview://12or9qsfo5kq61724hn7thg1f7a1a8i4j2ml00u56orm57h9kk50/be/src/runtime/routine_load/data_consumer_group.cpp#L279) (in KinesisDataConsumerGroup::start_all): static_cast<void>(consumer->cancel(ctx)); Per the review skill (Section 2.1.1): "static_cast<void>(...) silent discard is prohibited. Never allow this pattern in new code." The Kafka path has the same pattern (pre-existing), but new code should not replicate anti-patterns. Fix: At minimum, log the warning: Status cancel_st = consumer->cancel(ctx); if (!cancel_st.ok()) { LOG(WARNING) << "failed to cancel kinesis consumer: " << consumer->id() << ", error: " << cancel_st; } MEDIUM Issues 7. data_consumer_group.cpp:289 — static_cast<void>(kinesis_pipe->finish()) silently swallows error [data_consumer_group.cpp:289](vscode-webview://12or9qsfo5kq61724hn7thg1f7a1a8i4j2ml00u56orm57h9kk50/be/src/runtime/routine_load/data_consumer_group.cpp#L289): static_cast<void>(kinesis_pipe->finish()); If finish() fails, the error is silently ignored. This could mask data loss (partial pipe flush). Fix: Check the return status and propagate or log appropriately. 8. data_consumer_group.h — Missing compile_check_begin.h / compile_check_end.h pairing validation The header includes compile_check_begin.h inside namespace doris (line 39) and compile_check_end.h at line 128. The new KinesisDataConsumerGroup class is correctly within this scope, which is good. However, the Aws::Kinesis::Model::Record type is brought in via data_consumer.h which includes AWS SDK headers. Verify that AWS SDK headers don't trigger -Wconversion errors under the compile_check scope. 9. routine_load_task_executor.cpp:587-588 — Accessing consumer_grp->consumers()[0] after consumers returned to pool [routine_load_task_executor.cpp:587-588](vscode-webview://12or9qsfo5kq61724hn7thg1f7a1a8i4j2ml00u56orm57h9kk50/be/src/runtime/routine_load/routine_load_task_executor.cpp#L587-L588): // Line 542: return consumers to pool BEFORE this point consumer_pool->return_consumers(consumer_grp.get()); ... // Line 587-588: accessing consumer from grp AFTER return auto kinesis_consumer = std::static_pointer_cast<KinesisDataConsumer>( consumer_grp->consumers()[0]); ctx->kinesis_info->millis_behind_latest = kinesis_consumer->get_millis_behind_latest(); At line 542, return_consumers() returns all consumers back to the pool. At line 587, code accesses consumer_grp->consumers()[0]. While shared_ptr keeps the consumer alive, another thread could have already obtained this consumer from the pool, called reset() on it (which clears _millis_behind_latest), and started using it. This is a race condition. Fix: Move the millis_behind_latest extraction before return_consumers() on line 542, or copy the data before returning consumers. 10. kinesis_consumer_pipe.h — Missing compile_check_begin.h / compile_check_end.h [kinesis_consumer_pipe.h](vscode-webview://12or9qsfo5kq61724hn7thg1f7a1a8i4j2ml00u56orm57h9kk50/be/src/io/fs/kinesis_consumer_pipe.h) doesn't include compile_check_begin.h / compile_check_end.h. Per Section 2.1.3 of the review guide: "Do new .h files include paired compile_check_begin.h / compile_check_end.h?" Note that kafka_consumer_pipe.h also doesn't have them, so this may be acceptable for consistency with the existing pattern, but ideally new headers should include them. 11. data_consumer_group.cpp:315-316 — left_bytes -= len potential narrowing [data_consumer_group.cpp:315-316](vscode-webview://12or9qsfo5kq61724hn7thg1f7a1a8i4j2ml00u56orm57h9kk50/be/src/runtime/routine_load/data_consumer_group.cpp#L315-L316): left_bytes -= len; left_bytes is int64_t, len is size_t. Under compile_check, this may trigger a -Wconversion warning (implicit size_t → int64_t). The Kafka path has the same pattern but was written before compile_check was enforced. LOW Issues 12. data_consumer_pool.cpp:104-106 — Using stringstream instead of fmt/Status::Cancelled(fmt, ...) std::stringstream ss; ss << "unknown routine load task type: " << ctx->load_type; return Status::Cancelled(ss.str()); Could use the simpler format: return Status::Cancelled("unknown routine load task type: {}", ctx->load_type); 13. Test coverage — Zero tests for new Kinesis path No unit tests or regression tests were added for the Kinesis routine load feature. At minimum, the get_consumer_grp() function should have a test validating it creates the correct consumer group type for each source type. Summary Table # Severity File Description 1 BLOCKER data_consumer_pool.cpp:89 Null ptr dereference: ctx->kafka_info accessed for Kinesis tasks 2 BLOCKER data_consumer_pool.cpp:109 Creates base DataConsumerGroup instead of typed subclass → UB on static_pointer_cast 3 BLOCKER data_consumer.cpp:871 Use-after-move: result.GetNextShardIterator() after std::move(result) 4 HIGH data_consumer.cpp:928 const_cast + std::move on const ref is technically UB 5 HIGH data_consumer_pool.cpp:83-100 Leftover commented code and (Refrain) TODO 6 HIGH data_consumer_group.cpp:279 static_cast<void> on cancel return — silently swallows error in new code 7 MEDIUM data_consumer_group.cpp:289 static_cast<void> on finish() return — could mask data loss 8 MEDIUM data_consumer_group.h Verify AWS SDK headers don't conflict with compile_check 9 MEDIUM routine_load_task_executor.cpp:587 Race: accessing consumer after returning to pool 10 MEDIUM kinesis_consumer_pipe.h Missing compile_check begin/end 11 MEDIUM data_consumer_group.cpp:315 Potential size_t → int64_t narrowing under compile_check 12 LOW data_consumer_pool.cpp:104 Use fmt instead of stringstream 13 LOW — Zero test coverage for Kinesis path Issues #1, #2, #3 are critical and will cause crashes or undefined behavior at runtime. They must be fixed before merging. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
