Antoine,
Thank you for responding. I really appreciate you taking the time to discuss
this.
I agree that this is the key question:
> ... I'm not sure why you can't just copy and paste the tests in
> your own source tree.
I'm looking at this from within a large bureaucratic organisation in which
people move from job to job over time, internal systems evolve as do the
libraries we bring in and rely upon, such as pyarrow.
In this case my intent is to ensure that tests are present which exercise the
relationship between our implementations and your base libraries, now and over
time as the code evolves. Taking a snapshot of your tests now would mean that
future tests would be static (as of 'now') even if your implementation and
testing evolved to address new needs, to tackle corner case issues etc.
By testing our code using your tests we'll get immediate notice if our
assumptions / implementation no longer match your expectation no matter if the
change was at our end or your end.
In an organisation which evolves over time having tests which evolve with the
libraries we rely on is a Good Thing.
Thanks to you Antoine, and to your team mates, for your work on Arrow.
Very best wishes,
Bruce
-----Original Message-----
From: Antoine Pitrou [mailto:[email protected]]
Sent: Thursday, July 8, 2021 6:42 PM
To: [email protected]
Subject: Re: [Python] testing custom pyarrow.fs filesystems
Hi Bruce,
On Thu, 08 Jul 2021 08:58:10 +0000
"Badger, Bruce" <[email protected]> wrote:
>
> I understand that you did not intend that your tests be used in this way, and
> the caveats you gave about the scope of the tests (i.e. most tests are in
> C++), but it really has been very useful for us so thank you anyway!
>
> Needless to say, having found out how to do this we would like to be able to
> do the same with future Arrow releases.
>
> Could we ask that you maintain the shape of the tests around
> self.pyarrow.tests.test_fs.filesystem_config or, if you have to change that,
> that you provide an alternative way to plug in external filesystem classes?
Well, I don't think we have any plans to significantly refactor those
tests, so you *can* probably rely on them in the near future.
That said, I'm not sure why you can't just copy and paste the tests in
your own source tree. They're under the Apache license after all, so
there should not be much impediment in doing so.
Best regards
Antoine.
----------------------------------------------------------------------
This message, and any attachments, is for the intended recipient(s) only, may
contain information that is privileged, confidential and/or proprietary and
subject to important terms and conditions available at
http://www.bankofamerica.com/emaildisclaimer . If you are not the intended
recipient, please delete this message. Please note you may be contacted by a
different BofA entity acting for and on behalf of your service provider where
permitted by applicable law. This does not change your service provider.
Where applicable please note:
Merrill Lynch International is Registered in England (No.2312079). Registered
Office: 2 King Edward Street, London EC1A, 1HQ. VAT No. GB 245 1224 93.
Authorised by the Prudential Regulation Authority and regulated by the
Financial Conduct Authority and the Prudential Regulation Authority. Member of
the London Stock Exchange.
