Re: [CONNECT] New Clients for Go and Rust

[Maciej]

Hi,








--
Best regards,
Maciej Szymkiewicz

Web:https://zero323.net
PGP: A30CEF0C31A501EC



On 5/19/23 14:57, Jia Fan wrote:

Personally, I'm strongly against the second option and have some preference towards the third one (or maybe a mix of the first one and the third one). The project is already pretty large as-is and, with an extremely conservative approach towards removal of APIs, it only tends to grow over time. Making it even larger is not going to make things more maintainable and is likely to create an entry barrier for new contributors (that's similar to Jia's arguments). Moreover, we've seen quite a few different language clients over the years and all but one or two survived while none is particularly active, as far as I'm aware.  Taking responsibility for more clients, without being sure that we have resources to maintain them and there is enough community around them to make such effort worthwhile, doesn't seem like a good idea.

Hi,

Thanks for contribution!
I prefer (1). There are some reason:








https://github.com/ClickHouse/clickhouse-java
https://github.com/ClickHouse/clickhouse-odbc
https://github.com/ClickHouse/clickhouse-cpp

PS: I'm looking forward to the javascript connect client!

Thanks Regards
Jia Fan

Martin Grund <mgr...@apache.org> 于2023年5月19日周五 20:03写道：

    Hi folks,

    When Bo (thanks for the time and contribution) started the work on
    https://github.com/apache/spark/pull/41036 he started the Go
    client directly in the Spark repository. In the meantime, I was
    approached by other engineers who are willing to contribute to
    working on a Rust client for Spark Connect.

    Now one of the key questions is where should these connectors live
    and how we manage expectations most effectively.

    At the high level, there are two approaches:

    (1) "3rd party" (non-JVM / Python) clients should live in separate
    repositories owned and governed by the Apache Spark community.

    (2) All clients should live in the main Apache Spark repository in
    the `connector/connect/client` directory.

    (3) Non-native (Python, JVM) Spark Connect clients should not be
    part of the Apache Spark repository and governance rules.

    Before we iron out how exactly, we mark these clients as
    experimental and how we align their release process etc with
    Spark, my suggestion would be to get a consensus on this first
    question.

    Personally, I'm fine with (1) and (2) with a preference for (2).

    Would love to get feedback from other members of the community!

    Thanks
    Martin

1. Different repository can maintain independent versions, different release times, and faster bug fix releases. 2. Different languages have different build tools. Putting them in one repository will make the main repository more and more complicated, and it will become extremely difficult to perform a complete build in the main repository. 3. Different repository will make CI configuration and execute easier, and the PR and commit lists will be clearer. 4. Other repository also have different client to governed, like clickhouse. It use different repository for jdbc, odbc, c++. Please refer:

OpenPGP_signature
Description: OpenPGP digital signature