I've already found the assumptions, they were in elastic_test_data.py. The relevant PR is at https://github.com/apache/beam/pull/13085
On 2020/10/22 18:15:11, Tyson Hamilton <[email protected]> wrote: > IMO it really comes down to stability & runtime differences. If there are > no significant changes to either of these then keeping it as a > precommit and using test containers is fine. Where are the assumptions in > the IT test, in HadoopFormatIOElasticTest? > > On Mon, Oct 12, 2020 at 10:10 AM Piotr Szuberski < > [email protected]> wrote: > > > I'm trying to update elasticsearch version to 7.9.2 but I've encountered a > > problem with HadoopFormatIOElasticTest that uses ES in-memory emulator that > > is no longer supported: > > https://stackoverflow.com/questions/51316813/elastic-node-on-local-in-6-2 > > > > It's recommended to use testcontainers as proposed here > > https://github.com/allegro/embedded-elasticsearch but it would transform > > the in-memory test to integration test (which has to be done anyway) > > > > There is also Elasticsearch test framework with ESSingleNodeTestCase but > > it causes Jar Hell problem and I don't think it's easily solvable - the > > dependencies in "java core" and "java core test". > > I tried to > > > > Is running the precommit test with testcontainers acceptable? It's the > > easiest fix. > > > > About the integration test: > > I'd like to enable the IT test in Java PostCommit but there are some > > assumptions about the data that is already written to Elasticsearch but I > > can't find anywhere what that data should be (Probably something like > > Item_Price0, Item_Price1 etc but I'm not sure) > > >
