DannyVarod commented on issue #13185: URL: https://github.com/apache/arrow/issues/13185#issuecomment-1130220127
I am not use pyarrow directly, but only via libraries. - I had to write a large compatibility test to make sure they all work together on EMR in order to be able update one (as we run a series of applications on EMR and the 3rd party and internal dependencies are install once on EMR cluster creation). * Luckily it passed. In my case I'm using pyspark and snowflake, where pyspark has only a minimal version, but is not tested with any newer version and snowflake has an almost exact version i.e. a.b.0<=ver<a.b+1.0 which means it doesn't play nice with any other 3rd party. Why wouldn't each language-library have its own version? Also, it seems strange that there is at least one breaking change so frequently. If this has to be the case (breaking versions), I would split the project into 2 different libraries (each an articat and namespace of its own) to prevent library nightmare like profobuf 1 vs profobuf 2 which in Java requires shading. However, still, as a library developer myself, I rarely break the API, except for at the beginning when I'm still stabilizing it. Another solution, is to break up artifacts into smaller pieces, so only small parts that aren't usually used by 3rd party libraries break. This of course, would be a breaking change though, so if you do this, please do it, please use a different artifact and namespace/package name for the change. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arrow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
