Some questions for clarification: * What is the maven artifact that Spark can automatically pull (via --packages)? * Does that artifact use shaded dependencies? * Does that artifact depend on the Iceberg Spark bundle? * Is the _code_ running in Spark the same when the Polaris Spark Client is pulled via --packages and via --jars?
I know I could have figured that out from code, but I'm asking here because I think we may need to review our approach to publishing these artifacts. I believe that regardless of the method of including the Client into Spark runtime, the code has to be exactly the same.... and I doubt it is the same now. WDYT? Thanks, Dmitri. On Fri, Jun 20, 2025 at 10:15 AM Dmitri Bourlatchkov <di...@apache.org> wrote: > Hi All, > > Re: PR [1908] let's use this thread to clarify the problems we're trying > to solve and options for solutions. > > As for me, it looks like some refactoring in the way the Spark Client is > built and published may be needed. > > I think it makes sense to clarify this before 1.0 to avoid changes to > Maven coordinates right after 1.0 > > [1908] https://github.com/apache/polaris/pull/1908 > > Thanks, > Dmitri. > >
