DanielLeens commented on issue #11309: URL: https://github.com/apache/seatunnel/issues/11309#issuecomment-4880124990
Thanks for writing this up so concretely. This reads as a reasonable feature request rather than a bug. The use case is clear: SeaTunnel already has pieces like `Embedding` and the Lance sink, but there is still a gap when the target system is a DeepLake dataset. The most important next step is to keep the first version narrow enough that the community can judge the maintenance cost clearly. To evaluate a DeepLake sink well, it would help to make four points explicit in a follow-up design or PR: 1. the exact DeepLake client / API path the connector would use 2. dependency and licensing implications of that client in the SeaTunnel distribution 3. the minimum type surface for phase 1, such as scalar fields, vector fields, and binary / text metadata 4. expected behavior for id mapping, batch vs streaming, and retry / failure handling A sink-only MVP is probably the right first step here. If someone wants to implement it, keeping the scope limited to write support plus runnable documentation will make review much easier. We have kept this visible as `feature` + `help wanted`, so contributions are very welcome. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
