I've been thinking about this issue. With a bit of work, I think the approach
should probably be to:
a) make a custom build tool that knows which dirs use which other tools
b) add a new file in each dir that can tell precommit where to break
for modules
For example, let's say we have a dir structure like this:
build.sh
c/
Makefile
java/
pom.xml
perl/
Makefile.PL
Let's add a file called multibuild.sh that precommit can use:
build.sh
multibuild.sh
c/
Makefile
multibuild.sh
java/
pom.xml
multibuild.sh
perl/
Makefile.PL
multibuild.sh
In our "multibuild" build tool plugin, we'd define
function multibuild_buildfile
{
echo "multibuild.sh"
}
Now when precommit gets a patch, it will know that a change in c/ is a
different module than a change in java/ or perl/ because of the presence of the
multibuild.sh file. This prevents the "build the world" problem for a largely
diverse project like avro.
The place where this gets hard is our theoretical multibuild_executor
function because precommit doesn't pass as a param the module that's currently
being processed. (we should probably fix that) But we can still make it work
and reality, it might have led us down a false path. If we make this function
look like this:
function multibuild_executor
{
echo "multibuild.sh"
}
and make sure that BUILDTOOLCWD=module (the default), then precommit will run
our file that we placed to actually do the work. This file can then take
whatever parameters we're getting passed and convert it as appropriate to the
build tool for that directory. Since this is a custom build tool, you can
dictate what parameters to pass to which modules via _modules_workers. For
example, an easy route might be:
function multibuild_modules_workers
{
modules_workers $2
}
then multibuild.sh for each language becomes a series of case statements. For
C, it might look like:
testname=$1
shift
case ${testname} in
compile)
make -Dcustomflag=1
;;
doc)
make man
unit)
make -Danotherflag=1 test
;;
*)
;;
esac
It's pretty hacky, but theoretically it would work until we get a
chance to make precommit smarter about multiple build tools in a single
project. (which would be a pretty hard undertaking, but something we should
probably do)
> On Sep 10, 2016, at 6:32 PM, suraj acharya <[email protected]> wrote:
>
> Yes.
> The ./build.sh test
> <https://github.com/apache/avro/blob/master/build.sh#L40> command helps run
> all the tests from the top level.
> However, I am not sure if we need to run all the tests when a change is
> made?
> I was thinking of running language specific tests for changes in the
> specific directory.
> For example for changes in python will trigger ant test
> <https://github.com/apache/avro/blob/master/build.sh#L43>.
> If this is not a good idea let me know. Ill start another tread on dev@avro
> to see what the community thinks about it
>
> Thanks
>
> -Suraj Acharya
>
> On Sat, Sep 10, 2016 at 5:45 PM, Sean Busbey <[email protected]> wrote:
>
>> my understanding of Avro's build system is that there are per-language
>> build tools and everything gets unified via shell scripts. is there
>> just the top level build.sh file, or do the individual components have
>> their own?
>>
>> On Sat, Sep 10, 2016 at 11:48 AM, suraj acharya <[email protected]>
>> wrote:
>>> Hi,
>>> I have been working on AVRO-1887
>>> <https://issues.apache.org/jira/browse/AVRO-1887> to add pre-commit
>> test to
>>> avro.
>>> I have been successful in adding the personality for java file using
>> maven.
>>> I having some questions about the other plugins.
>>> Example :ant
>>>
>>> The structure of avro is that there is a top level pom.
>>> And then there is different langs in subdirectory below it.
>>> So there is avro/lang/py
>>> <https://github.com/apache/avro/tree/master/lang/py>which has python
>> code
>>> in it. The build tool for that is ant.
>>> My understanding is running add_test ant will run ant test.
>>> However, that doesn't actually run any tests. My assumption is that since
>>> there is no build.xml present there it is unable to run any ant tests.
>>>
>>> I tried cd to lang/py and run add_test ant from there. However, it throws
>>> an error saying unable to find pom.xml.
>>>
>>> Can someone give me some pointers on how to continue?
>>>
>>>
>>> Thanks
>>> -Suraj Acharya
>>
