Hi! I might be saying some nonsense, but...
On Sun, 31 Mar 2019 at 17:25, Nadav Har'El <[email protected]> wrote: > n Sun, Mar 31, 2019 at 5:58 PM Waldek Kozaczuk <[email protected]> > wrote: > >> Here is a list of things we should try to do: >> > > I think there are two very different issues involved here, that don't > really need the same solution: > The OSv kernel's makefile, and the "apps" build system. > > >> - Propose and define model app layout (Makefile, GET, ...) under >> osv-apps >> - Typically each app has GET shell script, Makefile and module.py: >> - GET typically downloads ad puts source files into upstream >> directory (sometimes using latest script) >> - Makefile typically does what needs to be done (sometime >> compiles) and puts source files into install subdirectory (sometimes >> usr.manifest is generated, sometime usr.manifest is pre-defined). >> - module.py is called to execute extra logic - sometimes to >> generate usr.manifest, define cmdline and possibly multiple flavors >> - This is somewhat documented in the README.md of osv-apps project >> but is not clear enough and the apps are all over the place following >> this >> loose convention and I am myself guilty of not following it (see >> python2x/3x apps that do too much in GET). I doubt it is worth fixing >> existing apps but I think we could improve the documentation in this >> area >> and and possibly point to a good model app. >> >> I agree. Also, note that both the "GET" and "makefile" there are rather > arbitrary conventions - > all we need from the Makefile is to have the "make" and "make clean" > target. That's it. Because people > were too lazy to write a proper makefile around this requirement, "make" > just calls "GET" which is a > shell script that just does everything. > We could easily have any other convention to build and clean a package. > In the past, I did start writing a python script to handle OSv apps in a uniform way the code is lost somewhere here But the idea was to have a JSON file with 2 sections, one for configuration, another for stages and based on some reserved names, we do operations like downloading files, checksum, etc pretty much like and dependency/build manager like maven/gradle/pip/conan/... Of course, any suggestion is welcomed :) Also, because we are limited in terms of developers, I would suggest leaving documentation until the new app builder is ready, otherwise, we will take a lot of time to write documentation for current build that will be lost and people will certainly ask for help if they are really interested anyway > I think the more difficult part is understand what the heck is module.py > and what is usr.manifest, etc. > if I understood, the script that handles OSv modules uses module.py to expose OSv apps as OSv modules, otherwise, we would need to move OSv apps inside <osv-dir>/modules (or change the python script that handles modules in other directories accordingly) Well, in fact, it might be a future refactoring, moving non-essential modules to OSv apps, for example, I think java, go, openssl, httpserver-*, monitoring-agent could be moved to OSv apps As for usr.manifest, personally, I would prefer to not have user.manifest and instead have a default directory inside app directory and everything that you puts there will be added to the final osv file system image I think most of the apps already generates a ROOTFS directory that creates such a structure not sure also, we could create tests to build each app automatically and see if they're still working I guess we have a number of apps that are not working due to various reasons I think you're right that just documenting the conventions we used will go > a long way in helping people to use this system, even if we don't change > anything. > > >> >> - >> - I know our build system (scripts/build and related python scripts) >> is complicated enough. But it would be super nice to allow passing >> parameters from scripts/build to app Makefile. The best use case for that >> is building java9 (10,11,12 as well) JREs using jlink where it would be >> nice to pass a list of JRE modules to incorporate - see >> >> https://github.com/cloudius-systems/osv-apps/blob/master/openjdk9-zulu-java-base/Makefile#L26; >> I think there are many other apps that would benefit >> >> Possibly. Indeed if properly documented, it shouldn't matter that it's > even more complicated. > >> >> - >> - Eliminate external - >> https://github.com/cloudius-systems/osv/issues/743 >> - It would be nice to detail what to do to each subdirectory (I am >> not positive we want to eliminate all - see x64/acpica) >> >> I agree. We should do that. "external" is an idea that really outlived > its usefulness, I think. > Indeed we don't need to remove everything at once, we can do it piece by > piece (and stop using pieces before actually removing them). > It might also be interesting updating acpica to the latest version Of course, I understand that talk is cheap (speaking for myself...) > > >> >> - Convert to CMakefile >> - I am not sure how difficult it would be, but one benefit of it >> would be the ability to load a project in IDEs like CLion that are >> able to >> work with CMakefile, >> >> I would hate for this to happen. I really think OSv's makefile is well > tuned and works well. > Switching to cmake just to make an editor happy would be very sad. > I don't know anything about clion, but is it possible to tell it OSv's > include paths, etc., *without* switching the Makefile language? > > E.g., could we have "make clion" which automatically generates a file > CMakeLists.txt > which somehow includes the right incantations to tell CLIon what are OSv's > include path and predefined macros needed to have the source files properly > formatted? > The file doesn't need to contain real instructions on how to compile > anything, I hope... > > >> >> - Some apps (like python2x/python3x) pull most of the binaries from >> host instead of re-compiling from scratch. It would be nice to have a >> shared app/tool to build apps from host like that. >> >> I agree. It would really be nice to have scripts or python code for > module.py or something, of how to copy a bunch of files from the host and > possibly the shared libraries they need, etc., and then just use this same > script from various apps instead of repeating the same code (with > variations nobody understands) in a dozen apps. > > While I'm always in favor of "newer"/"experimental" work, I would also stick to the things that we can help and support by now because once again we have limited resources to develop OSv > > >> Feel free to add any other things you would like to be improved in the >> build system. >> >> Waldek >> > > Let me know if I can help with anything :) -- You received this message because you are subscribed to the Google Groups "OSv Development" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/d/optout.
