GaneshPatil7517 commented on issue #17890: URL: https://github.com/apache/dolphinscheduler/issues/17890#issuecomment-3857743417
<html> <body> <!--StartFragment--><p style="margin: 0px 0px 16px; color: rgb(204, 204, 204); font-family: "Segoe WPC", "Segoe UI", sans-serif; font-size: 13px; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; white-space: normal; background-color: rgb(24, 24, 24); text-decoration-thickness: initial; text-decoration-style: initial; text-decoration-color: initial;"><strong>Hi @ruanwenjun,</strong></p><p style="margin: 0px 0px 16px; color: rgb(204, 204, 204); font-family: "Segoe WPC", "Segoe UI", sans-serif; font-size: 13px; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-s troke-width: 0px; white-space: normal; background-color: rgb(24, 24, 24); text-decoration-thickness: initial; text-decoration-style: initial; text-decoration-color: initial;">Thanks for your response. Here's my design proposal for this issue:</p><h3 style="line-height: normal; font-size: 13px; font-weight: unset; margin: 0px 0px 8px; font-family: "Segoe WPC", "Segoe UI", sans-serif; color: rgb(204, 204, 204); font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; white-space: normal; background-color: rgb(24, 24, 24); text-decoration-thickness: initial; text-decoration-style: initial; text-decoration-color: initial;">Problem</h3><p style="margin: 0px 0px 16px; color: rgb(204, 204, 204); font-family: "Segoe WPC", "Segoe UI", sans-serif; font-size: 13px; font-style: n ormal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; white-space: normal; background-color: rgb(24, 24, 24); text-decoration-thickness: initial; text-decoration-style: initial; text-decoration-color: initial;">When a worker group's name is changed,<span> </span><a href="vscode-file://vscode-app/c:/Users/HP/AppData/Local/Programs/Microsoft%20VS%20Code/bdd88df003/resources/app/out/vs/code/electron-browser/workbench/workbench.html" title="" draggable="true" data-href="file:///c%3A/Users/HP/Music/Dol/dolphinscheduler/dolphinscheduler-dao/src/main/java/org/apache/dolphinscheduler/dao/entity/EnvironmentWorkerGroupRelation.java#30%2C14" custom-hover="true" data-keybinding-context="2485" class="chat-inline-anchor-widget show-file-icons" style="border-color: rgba(255, 255, 255, 0.1); border-style: solid; b order-width: 0.8px; border-image: none 100% / 1 / 0 stretch; border-radius: 4px; margin: 0px 1px; padding: 1px 3px; text-wrap: nowrap; width: fit-content; font-weight: 400; text-decoration: rgb(204, 204, 204); color: inherit; user-select: text;"><span class="icon-label" style="padding: 0px 3px; text-wrap: wrap;">EnvironmentWorkerGroupRelation</span></a><span> </span>and<span> </span><a href="vscode-file://vscode-app/c:/Users/HP/AppData/Local/Programs/Microsoft%20VS%20Code/bdd88df003/resources/app/out/vs/code/electron-browser/workbench/workbench.html" title="" draggable="true" data-href="file:///c%3A/Users/HP/Music/Dol/dolphinscheduler/dolphinscheduler-dao/src/main/java/org/apache/dolphinscheduler/dao/entity/TaskDefinition.java#56%2C14" custom-hover="true" data-keybinding-context="2486" class="chat-inline-anchor-widget show-file-icons" style="border-color: rgba(255, 255, 255, 0.1); border-style: solid; border-width: 0.8px; border-image: none 100% / 1 / 0 stretch; border-radius: 4px ; margin: 0px 1px; padding: 1px 3px; text-wrap: nowrap; width: fit-content; font-weight: 400; text-decoration: rgb(204, 204, 204); color: inherit; user-select: text;"><span class="icon-label" style="padding: 0px 3px; text-wrap: wrap;">TaskDefinition</span></a><span> </span>become orphaned because they reference the worker group by name instead of ID.</p><h3 style="line-height: normal; font-size: 13px; font-weight: unset; margin: 0px 0px 8px; font-family: "Segoe WPC", "Segoe UI", sans-serif; color: rgb(204, 204, 204); font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; white-space: normal; background-color: rgb(24, 24, 24); text-decoration-thickness: initial; text-decoration-style: initial; text-decoration-color: initial;">Proposed Solution</h3><p style="margin: 0px 0px 16px; color: rgb(204, 204, 204); font-family: "Segoe WPC", "Segoe UI", sans-serif; font-size: 13px; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; white-space: normal; background-color: rgb(24, 24, 24); text-decoration-thickness: initial; text-decoration-style: initial; text-decoration-color: initial;"><strong>1. Database Schema Changes</strong></p> Table | Change -- | -- t_ds_environment_worker_group_relation | Add worker_group_id INT column (nullable initially for backward compatibility) t_ds_task_definition | Add worker_group_id INT column (nullable initially for backward compatibility) t_ds_task_definition_log | Add worker_group_id INT column <p style="margin: 0px 0px 16px; color: rgb(204, 204, 204); font-family: "Segoe WPC", "Segoe UI", sans-serif; font-size: 13px; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; white-space: normal; background-color: rgb(24, 24, 24); text-decoration-thickness: initial; text-decoration-style: initial; text-decoration-color: initial;"><strong>2. Entity Changes</strong></p><ul style="padding-inline-start: 24px; color: rgb(204, 204, 204); font-family: "Segoe WPC", "Segoe UI", sans-serif; font-size: 13px; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-stroke-width : 0px; white-space: normal; background-color: rgb(24, 24, 24); text-decoration-thickness: initial; text-decoration-style: initial; text-decoration-color: initial;"><li>Add<span> </span><code style="font-family: Consolas, "Courier New", monospace; font-size: 11.999px; color: rgb(208, 208, 208); background-color: rgb(60, 60, 60); padding: 1px 3px; border-radius: 4px; border-color: rgb(208, 208, 208); border-style: none; border-width: 0px; border-image: none 100% / 1 / 0 stretch; white-space: pre-wrap;">workerGroupId</code><span> </span>field to<span> </span><a href="vscode-file://vscode-app/c:/Users/HP/AppData/Local/Programs/Microsoft%20VS%20Code/bdd88df003/resources/app/out/vs/code/electron-browser/workbench/workbench.html" title="" draggable="true" data-href="file:///c%3A/Users/HP/Music/Dol/dolphinscheduler/dolphinscheduler-dao/src/main/java/org/apache/dolphinscheduler/dao/entity/EnvironmentWorkerGroupRelation.java#30%2C14" custom-hover="true" data-keybinding-context="2 487" class="chat-inline-anchor-widget show-file-icons" style="border-color: rgba(255, 255, 255, 0.1); border-style: solid; border-width: 0.8px; border-image: none 100% / 1 / 0 stretch; border-radius: 4px; margin: 0px 1px; padding: 1px 3px; text-wrap: nowrap; width: fit-content; font-weight: 400; text-decoration: none; color: inherit; user-select: text;"><span class="icon-label" style="padding: 0px 3px; text-wrap: wrap;">EnvironmentWorkerGroupRelation</span></a>,<span> </span><a href="vscode-file://vscode-app/c:/Users/HP/AppData/Local/Programs/Microsoft%20VS%20Code/bdd88df003/resources/app/out/vs/code/electron-browser/workbench/workbench.html" title="" draggable="true" data-href="file:///c%3A/Users/HP/Music/Dol/dolphinscheduler/dolphinscheduler-dao/src/main/java/org/apache/dolphinscheduler/dao/entity/TaskDefinition.java#56%2C14" custom-hover="true" data-keybinding-context="2488" class="chat-inline-anchor-widget show-file-icons" style="border-color: rgba(255, 255, 255, 0.1); border-s tyle: solid; border-width: 0.8px; border-image: none 100% / 1 / 0 stretch; border-radius: 4px; margin: 0px 1px; padding: 1px 3px; text-wrap: nowrap; width: fit-content; font-weight: 400; text-decoration: none; color: inherit; user-select: text;"><span class="icon-label" style="padding: 0px 3px; text-wrap: wrap;">TaskDefinition</span></a>, and<span> </span><code style="font-family: Consolas, "Courier New", monospace; font-size: 11.999px; color: rgb(208, 208, 208); background-color: rgb(60, 60, 60); padding: 1px 3px; border-radius: 4px; border-color: rgb(208, 208, 208); border-style: none; border-width: 0px; border-image: none 100% / 1 / 0 stretch; white-space: pre-wrap;">TaskDefinitionLog</code></li><li>Keep<span> </span><a href="vscode-file://vscode-app/c:/Users/HP/AppData/Local/Programs/Microsoft%20VS%20Code/bdd88df003/resources/app/out/vs/code/electron-browser/workbench/workbench.html" title="" draggable="true" data-href="file:///c%3A/Users/HP/Music/Dol/dolphinschedule r/dolphinscheduler-dao/src/main/java/org/apache/dolphinscheduler/dao/entity/EnvironmentWorkerGroupRelation.java#43%2C20" custom-hover="true" data-keybinding-context="2489" class="chat-inline-anchor-widget show-file-icons" style="border-color: rgba(255, 255, 255, 0.1); border-style: solid; border-width: 0.8px; border-image: none 100% / 1 / 0 stretch; border-radius: 4px; margin: 0px 1px; padding: 1px 3px; text-wrap: nowrap; width: fit-content; font-weight: 400; text-decoration: none; color: inherit; user-select: text;"><span class="icon-label" style="padding: 0px 3px; text-wrap: wrap;">workerGroup</span></a><span> </span>(name) field for backward compatibility during transition</li></ul><p style="margin: 0px 0px 16px; color: rgb(204, 204, 204); font-family: "Segoe WPC", "Segoe UI", sans-serif; font-size: 13px; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text -indent: 0px; text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; white-space: normal; background-color: rgb(24, 24, 24); text-decoration-thickness: initial; text-decoration-style: initial; text-decoration-color: initial;"><strong>3. API Changes</strong></p><ul style="padding-inline-start: 24px; color: rgb(204, 204, 204); font-family: "Segoe WPC", "Segoe UI", sans-serif; font-size: 13px; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; white-space: normal; background-color: rgb(24, 24, 24); text-decoration-thickness: initial; text-decoration-style: initial; text-decoration-color: initial;"><li>Modify create/update APIs to accept<span> </span><code style="font-family: Consolas, "Courier New", monospace; font-size: 11.999 px; color: rgb(208, 208, 208); background-color: rgb(60, 60, 60); padding: 1px 3px; border-radius: 4px; border-color: rgb(208, 208, 208); border-style: none; border-width: 0px; border-image: none 100% / 1 / 0 stretch; white-space: pre-wrap;">workerGroupId</code><span> </span>instead of (or in addition to)<span> </span><a href="vscode-file://vscode-app/c:/Users/HP/AppData/Local/Programs/Microsoft%20VS%20Code/bdd88df003/resources/app/out/vs/code/electron-browser/workbench/workbench.html" title="" draggable="true" data-href="file:///c%3A/Users/HP/Music/Dol/dolphinscheduler/dolphinscheduler-dao/src/main/java/org/apache/dolphinscheduler/dao/entity/EnvironmentWorkerGroupRelation.java#43%2C20" custom-hover="true" data-keybinding-context="2490" class="chat-inline-anchor-widget show-file-icons" style="border-color: rgba(255, 255, 255, 0.1); border-style: solid; border-width: 0.8px; border-image: none 100% / 1 / 0 stretch; border-radius: 4px; margin: 0px 1px; padding: 1px 3px; text-wrap: no wrap; width: fit-content; font-weight: 400; text-decoration: none; color: inherit; user-select: text;"><span class="icon-label" style="padding: 0px 3px; text-wrap: wrap;">workerGroup</span></a><span> </span>name</li><li>Add validation to ensure referenced worker group exists</li></ul><p style="margin: 0px 0px 16px; color: rgb(204, 204, 204); font-family: "Segoe WPC", "Segoe UI", sans-serif; font-size: 13px; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; white-space: normal; background-color: rgb(24, 24, 24); text-decoration-thickness: initial; text-decoration-style: initial; text-decoration-color: initial;"><strong>4. Migration Strategy</strong></p><ul style="padding-inline-start: 24px; color: rgb(204, 204, 204); font-family: "Segoe WPC", "Sego e UI", sans-serif; font-size: 13px; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; white-space: normal; background-color: rgb(24, 24, 24); text-decoration-thickness: initial; text-decoration-style: initial; text-decoration-color: initial;"><li>Provide database migration script to populate<span> </span><code style="font-family: Consolas, "Courier New", monospace; font-size: 11.999px; color: rgb(208, 208, 208); background-color: rgb(60, 60, 60); padding: 1px 3px; border-radius: 4px; border-color: rgb(208, 208, 208); border-style: none; border-width: 0px; border-image: none 100% / 1 / 0 stretch; white-space: pre-wrap;">worker_group_id</code><span> </span>based on existing<span> </span><code style="font-family: Consolas, "Courier New", monospace; font-size : 11.999px; color: rgb(208, 208, 208); background-color: rgb(60, 60, 60); padding: 1px 3px; border-radius: 4px; border-color: rgb(208, 208, 208); border-style: none; border-width: 0px; border-image: none 100% / 1 / 0 stretch; white-space: pre-wrap;">worker_group</code><span> </span>name</li><li>Deprecate name-based references in future versions</li></ul><p style="margin: 0px 0px 16px; color: rgb(204, 204, 204); font-family: "Segoe WPC", "Segoe UI", sans-serif; font-size: 13px; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; white-space: normal; background-color: rgb(24, 24, 24); text-decoration-thickness: initial; text-decoration-style: initial; text-decoration-color: initial;"><strong>5. Worker Registration</strong><br>For dynamically registered workers (#179 29):</p><ul style="padding-inline-start: 24px; color: rgb(204, 204, 204); font-family: "Segoe WPC", "Segoe UI", sans-serif; font-size: 13px; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; white-space: normal; background-color: rgb(24, 24, 24); text-decoration-thickness: initial; text-decoration-style: initial; text-decoration-color: initial;"><li>Require worker groups to be pre-created via UI before workers can join</li><li>Workers attempting to join non-existent groups should fail with clear error message</li></ul><h3 style="line-height: normal; font-size: 13px; font-weight: unset; margin: 0px 0px 8px; font-family: "Segoe WPC", "Segoe UI", sans-serif; color: rgb(204, 204, 204); font-style: normal; font-variant-ligatures: normal; font-varia nt-caps: normal; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; white-space: normal; background-color: rgb(24, 24, 24); text-decoration-thickness: initial; text-decoration-style: initial; text-decoration-color: initial;">Implementation Order</h3><p style="margin: 0px 0px 16px; color: rgb(204, 204, 204); font-family: "Segoe WPC", "Segoe UI", sans-serif; font-size: 13px; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; white-space: normal; background-color: rgb(24, 24, 24); text-decoration-thickness: initial; text-decoration-style: initial; text-decoration-color: initial;">Following the sub-issues:</p><ol style="padding-inline-start: 28px; color: rgb(204, 204, 204); font-family: "Segoe WPC", "Segoe UI", sans-serif; font-size: 13px; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; white-space: normal; background-color: rgb(24, 24, 24); text-decoration-thickness: initial; text-decoration-style: initial; text-decoration-color: initial;"><li>#17926 - Create worker groups without workers (prerequisite)</li><li>#17928 - Add workerGroupId to EnvironmentWorkerGroupRelation</li><li>#17930 - Add workerGroupId to TaskDefinition</li><li>#17929 - Prevent workers joining non-existent groups</li></ol><p style="margin: 0px 0px 16px; color: rgb(204, 204, 204); font-family: "Segoe WPC", "Segoe UI", sans-serif; font-size: 13px; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-w eight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; white-space: normal; background-color: rgb(24, 24, 24); text-decoration-thickness: initial; text-decoration-style: initial; text-decoration-color: initial;">I'd be happy to implement any of these sub-issues. Let me know which one you'd like me to start with!</p><hr style="border-color: rgba(255, 255, 255, 0.18); font-family: "Segoe WPC", "Segoe UI", sans-serif; font-size: 13px; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; white-space: normal; background-color: rgb(24, 24, 24); text-decoration-thickness: initial; text-decoration-style: initial; text-decoration-color: initial;"><br class="Apple -interchange-newline"><!--EndFragment--> </body> </html>Hi @ruanwenjun, Thanks for your response. Here's my design proposal for this issue: Problem When a worker group's name is changed, [EnvironmentWorkerGroupRelation](vscode-file://vscode-app/c:/Users/HP/AppData/Local/Programs/Microsoft%20VS%20Code/bdd88df003/resources/app/out/vs/code/electron-browser/workbench/workbench.html) and [TaskDefinition](vscode-file://vscode-app/c:/Users/HP/AppData/Local/Programs/Microsoft%20VS%20Code/bdd88df003/resources/app/out/vs/code/electron-browser/workbench/workbench.html) become orphaned because they reference the worker group by name instead of ID. Proposed Solution 1. Database Schema Changes Table Change t_ds_environment_worker_group_relation Add worker_group_id INT column (nullable initially for backward compatibility) t_ds_task_definition Add worker_group_id INT column (nullable initially for backward compatibility) t_ds_task_definition_log Add worker_group_id INT column 2. Entity Changes Add workerGroupId field to [EnvironmentWorkerGroupRelation](vscode-file://vscode-app/c:/Users/HP/AppData/Local/Programs/Microsoft%20VS%20Code/bdd88df003/resources/app/out/vs/code/electron-browser/workbench/workbench.html), [TaskDefinition](vscode-file://vscode-app/c:/Users/HP/AppData/Local/Programs/Microsoft%20VS%20Code/bdd88df003/resources/app/out/vs/code/electron-browser/workbench/workbench.html), and TaskDefinitionLog Keep [workerGroup](vscode-file://vscode-app/c:/Users/HP/AppData/Local/Programs/Microsoft%20VS%20Code/bdd88df003/resources/app/out/vs/code/electron-browser/workbench/workbench.html) (name) field for backward compatibility during transition 3. API Changes Modify create/update APIs to accept workerGroupId instead of (or in addition to) [workerGroup](vscode-file://vscode-app/c:/Users/HP/AppData/Local/Programs/Microsoft%20VS%20Code/bdd88df003/resources/app/out/vs/code/electron-browser/workbench/workbench.html) name Add validation to ensure referenced worker group exists 4. Migration Strategy Provide database migration script to populate worker_group_id based on existing worker_group name Deprecate name-based references in future versions 5. Worker Registration For dynamically registered workers (#17929): Require worker groups to be pre-created via UI before workers can join Workers attempting to join non-existent groups should fail with clear error message Implementation Order Following the sub-issues: #17926 - Create worker groups without workers (prerequisite) #17928 - Add workerGroupId to EnvironmentWorkerGroupRelation #17930 - Add workerGroupId to TaskDefinition #17929 - Prevent workers joining non-existent groups I'd be happy to implement any of these sub-issues. Let me know which one you'd like me to start with! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
