Using rebar 3.15.2 would work for now. Good find. For _build directory, a good number of dependencies already have their own unit tests for PRs, so they should be ok. Some we just skip altogether anyway https://github.com/apache/couchdb/blob/main/Makefile#L76 (folsom,meck,mochiweb,triq,proper,snappy,bcrypt,hyper,ibrowse).
But the issue is that those dependencies just won't run their own tests or that they won't even be available as dependencies for other eunit tests. If it's later, we may have to do some additional magic, like you suggested. Thanks, -Nick On Fri, Nov 26, 2021 at 8:57 AM Adam Kocoloski <kocol...@apache.org> wrote: > > The pc plugin has handled everything I’ve thrown at it so far. I recall Paul > mentioning some issue with transitive dependencies; I’ll keep an eye out for > any trouble. I didn’t try anything with Fauxton and docs yet. > > Rebar3 supported Erlang 20 as recently as 3.15.2, so I can imagine an option > to just use that version for Erlang 20 / 21 builds and the latest version for > 22+. That’s what I ended up doing for the GH Actions CI for the b64url lib. > For what it’s worth, the rebar3 team only intends to support 3 major releases > at a time going forward, see https://github.com/erlang/rebar3/pull/2555 > <https://github.com/erlang/rebar3/pull/2555>. They argue that this is > consistent with the support statement from Erlang/OTP itself. > > One notable difference I’ve found so far is that our current setup considers > dependencies to be peers to the internal apps in the umbrella couchdb repo: > we put all of them in src/, we run their tests as part of our test suite, > etc. Rebar3 installs dependencies to the _build directory and does not test > them. If it’s essential that we consider code in these other repositories to > be part of an Apache CouchDB source release we might need to get a little > creative. > > As an aside, I didn’t intend to weigh in on the rebar3 vs. mix debate here. I > can see good arguments for both. I just found myself gaining some familiarity > with rebar3 recently, and figured there would be a fair amount of common work > involved in aligning our build system to either modern option. > > Cheers, Adam > > > On Nov 24, 2021, at 2:07 PM, Nick Vatamaniuc <vatam...@gmail.com> wrote: > > > > Good idea to try rebar3! I tried it a few years back and got stumped > > at building some NIFs and a few other issues. Most of those can be > > overcome with a bit of work. > > > > I saw you used the pc compiler for NIFs, and I think rebar3 can set > > overrides now too > > https://github.com/blt/port_compiler#use-with-existing-dependency for > > individual dependencies. Perhaps the issue I encountered might be > > solved that way. > > > > The other issue was our "raw" dependencies - fauxton and docs. The > > approach I had tried was to make them look like Erlang apps. I don't > > recall how successful that was. But I think there are some newer > > things like https://rebar3.readme.io/docs/custom-dep-resources which > > we could leverage there. > > > > One more issue I recently discovered was that rebar3 is not Erlang 20 > > compatible. We ship our releases with the Erlang 20 as the minimum > > version and if we update to use rebar3 our release branch might be > > left behind. If the issue is just using Tag:Error:Stack catch > > patterns, we could potentially have our own fork of rebar3, as we > > already do for rebar2. > > > > Cheers, > > -Nick > > > > > > On Tue, Nov 23, 2021 at 5:39 PM Adam Kocoloski <kocol...@apache.org> wrote: > >> > >> Bah, ignore the message below. The issue isn’t that EUnit has a problem > >> with nested directories, it’s that it expects each test module to have a > >> `_tests` suffix, not a `_test` one. We’re inconsistent about adhering to > >> that rule in our codebase I’ll submit a PR to fix the module names so no > >> one else bumps into that issue. > >> > >> Adam > >> > >>> On Nov 23, 2021, at 3:48 PM, Adam Kocoloski <kocol...@apache.org> wrote: > >>> > >>> Hi, > >>> > >>> TL;DR - I think it would be a good idea to move EUnit tests back into > >>> `test` instead of `test/eunit`. > >>> > >>> Recently I’ve been spending some time porting specific libraries to use > >>> rebar3 and I thought it might be a nice project to hack on supporting > >>> rebar3 in CouchDB proper during the holiday weekend. I noticed a few > >>> small issues, but one in particular is that it was difficult to get > >>> rebar3 to pick up unit tests that were in subdirectories of `test`. > >>> > >>> Eventually I figured out that you could tell the compiler to do recursive > >>> compilation on these test directories and that worked for running the > >>> end-to-end test suite, but then I found that rebar3’s `—app` flag was > >>> still broken, e.g. if I ran `rebar3 eunit —app couch_log` it would only > >>> find tests in the actual source modules but not any of the modules in the > >>> test/eunit directory. I thought this might be an inconsistency bug in > >>> rebar3, but I realized that when you invoked rebar3 this way it was > >>> really just calling `eunit:test({application, couch_log})`, and EUnit > >>> doesn’t allow any configurability of its search path when invoked this > >>> way. > >>> > >>> Bottom line, it feels like we’re going against the grain of eunit itself > >>> when we try to move the eunit tests to anywhere else other than `test`. > >>> Currently we have a small handful of apps following this pattern: > >>> > >>> chttpd > >>> couch > >>> couch_epi > >>> couch_log > >>> couch_prometheus > >>> couch_replicator > >>> > >>> Of those, only the first two have any ExUnit unit tests that (I think) > >>> motivated the creation of separate eunit and exunit tests in the first > >>> place. Would it be OK if we moved those test modules back into `test` so > >>> we can stay on the “paved path” for EUnit usage going forward? > >>> > >>> Cheers, Adam > >> >
