I believe tests against real storage systems are highly valuable, especially since Polaris relies entirely on three distinct cloud storage providers at this moment, each with its own quirks and unique features. These tests help ensure robustness and compatibility across these platforms.
To balance usability, we can make these tests optional, allowing them to be skipped if users don’t provide the necessary credentials. Yufei On Fri, Jan 10, 2025 at 4:24 PM Dmitri Bourlatchkov <di...@apache.org> wrote: > Real storage implies real credentials, which will necessarily restrict OSS > users. > > For example, outside contributors will have no ability to run (and debug) > the full set of tests. > > My previous experience in OSS projects that had a private CI component is > that this model does not scale well in terms of handling contributions. > > I'd advocate for a model when CI in Apache Polaris is 100% transparent for > all users and is the only "normative" CI for OSS PRs, but vendors building > custom Polaris-based systems run extra tests on their infrastructure for > extra coverage. > > If issues are discovered on "real" storage the corresponding OSS code fix > would still be tested on emulators. > > Of course there will be cases when complete test coverage of edge cases is > not possible, but I think this should not be a reason for reducing the > openness of CI processes. > > WDYT? > > Cheers, > Dmitri. > > On Fri, Jan 10, 2025 at 7:06 PM Eric Maynard <eric.w.mayn...@gmail.com> > wrote: > > > While I think more tests that mocked or emulated storage would be great, > I > > think we probably want tests hitting "real" storage in addition to those. > > > > I would be in favor of running the "real" storage tests less often. For > > example, only prior to merging or to cutting a release. > > > > --EM > > > > On Fri, Jan 10, 2025 at 3:36 PM Dmitri Bourlatchkov <di...@apache.org> > > wrote: > > > > > Hi All, > > > > > > I see that some tests (e.g. for GCP and Azure) apparently require > special > > > settings to be able to run. > > > > > > Would there be any objections to gradually converting those tests to > use > > > emulators or containerized storage (like MinIO) and hence have no > special > > > settings or secrets? > > > > > > Thanks, > > > Dmitri. > > > > > >
