iilyak commented on pull request #3189: URL: https://github.com/apache/couchdb/pull/3189#issuecomment-748073005
> @iilyak are these efforts still valid? I saw you merged a PR (#3286) to overcome an issue I was stuck in front of 👍 I didn't know that inability to skip tests is what motivated you for this work. I implemented #3286 as an additional target to call instead of `make elixir` for people who run customized versions of CouchDB. The idea was that CouchDB CI would use your work. We possibly use different tags for things which are not implemented yet for FDB based CouchDB. Maybe `3.x` tag for all elixir tests and `4.x` for tests which already working in `main` branch. This should cover all possible combinations: | 3x | 4x | description | |-|-|-| | | + | New APIs (new pagination API from #2870 for example) | | + | | APIs which are not yet implemented in `main` or removed | | + | + | APIs which already work for `main` and `master` | However these new tags might interfere with `kind`. Which probably means we probably need to comment out 3x tests which are testing not implemented yet features. Again the #3286 switches to `make elixir-suite` as a temporary solution. Nick noticed that the coverage via #3286 is greater than the one via `make elixir tests=test/elixir/test/basics_test.exs,test/elixir/test/replication_test.exs,test/elixir/test/map_test.exs,test/elixir/test/all_docs_test.exs,test/elixir/test/bulk_docs_test.exs` so he proposed to use new way for now. The deficiency of #3286 is the need to maintain `test/elixir/test/config/suite.elixir`. It is ok for people who run customized versions, but IMO it is not good enough as a primary mechanism. Ideally the ability to skip tests via blacklist should be supported by testing framework. I considered opening PR for mix. However I realized that CouchDB wouldn't benefit from the feature because it is going to be a long time before we migrate to version of Elixir which would have my patch. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected]
