I sympathize, but might be less change to just rename the dir. There is more in there like the avro reader; it's kind of miscellaneous. I think we might want fewer rather than more top level dirs.
On Thu, Mar 17, 2022 at 7:33 PM Jungtaek Lim <kabhwan.opensou...@gmail.com> wrote: > We seem to just focus on how to avoid the conflict with the name > "external" used in bazel. Since we consider the possibility of renaming, > why not revisit the modules "external" contains? > > Looks like kinds of the modules external directory contains are 1) Docker > 2) Connectors 3) Sink on Dropwizard metrics (only ganglia here, and it > seems to be just that Ganglia is LGPL) > > Would it make sense if each kind deserves a top directory? We can probably > give better generalized names, and as a side-effect we will no longer have > "external". > > On Fri, Mar 18, 2022 at 5:45 AM Dongjoon Hyun <dongjoon.h...@gmail.com> > wrote: > >> Thank you for posting this, Alkis. >> >> Before the question (1) and (2), I'm curious if the Apache Spark >> community has other downstreams using Bazel. >> >> To All. If there are some Bazel users with Apache Spark code, could you >> share your practice? If you are using renaming, what is your renamed >> directory name? >> >> Dongjoon. >> >> >> On Thu, Mar 17, 2022 at 11:56 AM Alkis Evlogimenos >> <alkis.evlogime...@databricks.com.invalid> wrote: >> >>> AFAIK there is not. `external` has been baked in bazel since the >>> beginning and there is no plan from bazel devs to attempt to fix this >>> <https://github.com/bazelbuild/bazel/issues/4508#issuecomment-724055371> >>> . >>> >>> On Thu, Mar 17, 2022 at 7:52 PM Sean Owen <sro...@gmail.com> wrote: >>> >>>> Just checking - there is no way to tell bazel to look somewhere else >>>> for whatever 'external' means to it? >>>> It's a kinda big ugly change but it's not a functional change. If >>>> anything it might break some downstream builds that rely on the current >>>> structure too. But such is life for developers? I don't have a strong >>>> reason we can't. >>>> >>>> On Thu, Mar 17, 2022 at 1:47 PM Alkis Evlogimenos >>>> <alkis.evlogime...@databricks.com.invalid> wrote: >>>> >>>>> Hi Spark devs. >>>>> >>>>> The Apache Spark repo has a top level external/ directory. This is a >>>>> reserved name for the bazel build system and it causes all sorts of >>>>> problems: some can be worked around and some cannot (for some details on >>>>> one that cannot see >>>>> https://github.com/hedronvision/bazel-compile-commands-extractor/issues/30 >>>>> ). >>>>> >>>>> Some forks of Apache Spark use bazel as a build system. It would be >>>>> nice if we can make this change in Apache Spark without resorting to >>>>> complex renames/merges whenever changes are pulled from upstream. >>>>> >>>>> As such I proposed to rename external/ directory to want to rename the >>>>> external/ directory to something else [SPARK-38569 >>>>> <https://issues.apache.org/jira/browse/SPARK-38569>]. I also sent a >>>>> tentative [PR-35874 <https://github.com/apache/spark/pull/35874>] >>>>> that renames external/ to vendor/. >>>>> >>>>> My questions to you are: >>>>> 1. Are there any objections to renaming external to X? >>>>> 2. Is vendor a good new name for external? >>>>> >>>>> Cheers, >>>>> >>>>
