chris-fast commented on issue #16754: URL: https://github.com/apache/dolphinscheduler/issues/16754#issuecomment-3823118959
Hey! I'd like to work on this issue. I've got the backend part done, but
wanted to get some feedback on the frontend approach before going further.
## Quick summary of what I did
**Backend:** ✅ Done
- Added `batchSize` parameter to `DataxParameters`
- Added JSON generation in `DataxTask` (only includes it when value > 0)
- Added unit tests and i18n support
**Frontend:** 🤔 Need your input here
## The question: How should the UI look?
I'm leaning towards using a dropdown with preset values (like how
`jobSpeedByte` and `jobSpeedRecord` work currently), but with a twist - the
options change based on the target database:
**For ClickHouse / Databend:**
0 (unlimited)
16384
32768
65536 ← default
131072
**For everything else:**
0 (unlimited)
1024
2048 ← default
4096
8192
16384
32768
**Why?**
- ClickHouse needs larger batches (65536, as mentioned in the issue)
- Other databases are fine with the DataX default (2048)
- When you switch the target database, it auto-updates the default
## What do you all think?
1. Dropdown vs free-form number input - which one do you prefer?
2. Are there other databases that should get special treatment? (Like
Hive, etc.)
3. Or should we keep it simple and just use the same defaults for
everyone?
Also just to clarify - `batchSize` is different from
`jobSpeedByte`/`jobSpeedRecord`. Those control how fast data moves, while
`batchSize` controls how many records get written in one batch. They work
together but don't overlap.
Let me know your thoughts! I'm happy to adjust the approach based on
feedback.
@bromine0x23 @SbloodyS
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
