On Sun, Feb 18, 2018 at 5:27 AM, Geraldo Netto <geraldone...@gmail.com> wrote:
> Hello Waldek/All, > > Indeed, I must say, not only this, but many other apps have broken urls or > broken builds > This was the main thing that bothered me about your "change wget to wget -c" series of 50 patches - in many of these changes, the fix didn't really fix anything - either the module did not compile at all before your patch (and also after it), or the Makefile was written in a way that it doesn't matter at all whether you did wget or wget -c because, for example, the files were deleted and re-downloaded from scratch every time. I think as a general comment, we need to at least try to *test* every patch. If a patch can't be tested because the code doesn't actually work both before and after the patch, it's a hint there's a more important patch to be written instead of the patch that doesn't fix anything :-) > For example, this is my 'quick list' after running random builds in random > apps: > - busybox that was updated recently (somewhere in the end of 2017) looks > broken for me now > - Apache Derby that is compiled with a newer jdk and because of that, does > not run > Although it would be great to fix all of these issues, it will take work, and in many cases the original module was written by someone who had a personal interest in this specific application, and today there's nobody who cares. So if we don't fix some of these modules, the least we could do is to open issues in the bug tracker and present what you found didn't run, and why you think it doesn't - perhaps someone else would care enough to fix them. I created https://github.com/cloudius-systems/osv/issues/947 now about what needs to be done to fix apache-derby (with fairly easy instructions how, in case you want to contribute a patch :-) ). > - Erlang, Free Pascal and some other apps require their own > compilers/tools/sdk to be installed before building their packages and we > don't have a check for them > While it would be nice to have a programatic check, an almost-good-enough alternative is to have a README in the module directory explaining what you need to get it running. In any case, I think it should be considered a must to have a README in each module directory - in some of those, I am not even sure what this module does or how I'm supposed to use it after building an image with it. > - we don't have a unified structure to handle many common cases > This I agree but see more on that below (it's a great idea, but will require work). > - netperf url seems broken > - rpm url is broken for me and on mint (ubuntu based) it requires libmagic > and some other libs to be installed > - leveldb i think i have submitted some patches to fix and upgrade the > build > - rhino url seems broken > - embulk url seems broken > - groonga build seems broken > - plus all the problems you're facing > > > Actually, I would like to suggest a different approach for the building, > maybe use a single unified Makefile or shell script > that uses a configuration file per app like capstran, in case some custom > shell script is required, we can also define some standard 'hooks' > We could do this, but it would be a lot of work to provide something useful enough for all application needs. The nice thing about the shell or Makefile (or both) is that it lets you do anything :-) I wonder if we can learn from existing open source distributions projects. For example, for RPM building too, they take upstream tars, mix them with local patches, run the Makefile, and collect part of the resulting files into an RPM. Could we use some of their tooling? I guess we could do better with our own special tools, and it will definitely be very useful, it just will be an effort to write them. > > By now, I see we need the follow tasks: > > check > - could be used to check for dependencies, like check if any host library > is missing > - could be used to check for the downloaded file > if we don't have the md5 of the file, we could pre-compute the hash in > our local host and put it to compare > wget -c just makes sure the file is there but it does not check for the > file size/hash as You/Nadav noticed > > build > - could be a Makefile or a shell script, I would suggest to define a > standard way of creating it - currently, we have many different approaches > to do the same thing > we could pass the cpu number automatically, I don't know why, i'm still > getting the make message "jobserver unavailable: using -j1. Add `+' to > parent make rule" in some builds > > unpack (to untar/unzip/un<anything>) > - just needs to unpack the source/binary > > rootfs > - put all the content on rootfs, etc > > manifest > - prepares the usr.manifest file > > clean > - a standard way to clean the apps without removing the source/binary > > Do you have any suggestion/comment? > > > Kind Regards, > > Geraldo Netto > Sapere Aude => Non dvcor, dvco > http://exdev.sf.net/ > > On 17 February 2018 at 21:20, Waldek Kozaczuk <jwkozac...@gmail.com> > wrote: > >> After I applied this patch and previous and this is an error I am getting: >> >> 2018-02-17 18:12:45 7786 [Note] InnoDB: Waiting for purge to start >> 2018-02-17 18:12:45 7786 [Note] InnoDB: 5.6.21 started; log sequence >> number 1625987 >> ERROR: 1193 Unknown system variable 'table_type' >> 2018-02-17 18:12:45 7786 [ERROR] Aborting >> >> 2018-02-17 18:12:45 7786 [Note] Binlog end >> 2018-02-17 18:12:45 7786 [Note] InnoDB: FTS optimize thread exiting. >> 2018-02-17 18:12:45 7786 [Note] InnoDB: Starting shutdown... >> 2018-02-17 18:12:47 7786 [Note] InnoDB: Shutdown completed; log sequence >> number 1625997 >> 2018-02-17 18:12:47 7786 [Note] ./bin/mysqld: Shutdown complete >> >> Importing /home/wkozaczuk/projects/osv/modules/java-base/module.py >> Importing /home/wkozaczuk/projects/osv/apps/fonts/module.py >> No module.py in /home/wkozaczuk/projects/osv/modules/ca-certificates >> No module.py in /home/wkozaczuk/projects/osv/modules/libz >> Preparing usr.manifest >> Appending /home/wkozaczuk/projects/osv/apps/mysql/usr.manifest to >> usr.manifest >> Preparing bootfs.manifest >> Saving command line to /home/wkozaczuk/projects/osv/b >> uild/release.x64/cmdline >> Building into build/release.x64 >> GEN gen/include/osv/version.h >> MKBOOTFS build/release.x64/bootfs.bin >> AS bootfs.s >> LINK loader.elf >> LIBOSV.SO >> STRIP loader.elf -> loader-stripped.elf >> LZ loader-stripped.elf >> OBJCOPY loader-stripped.elf.lz -> loader-stripped.elf.lz.o >> LINK lzloader.elf >> ALIGN lzloader.elf >> DD loader.img boot.bin >> DD loader.img lzloader.elf >> IMGEDIT build/release.x64/loader.img >> IMGEDIT build/release.x64/loader.img >> Traceback (most recent call last): >> File "/home/wkozaczuk/projects/osv/scripts/upload_manifest.py", line >> 161, in <module> >> main() >> File "/home/wkozaczuk/projects/osv/scripts/upload_manifest.py", line >> 144, in main >> manifest = read_manifest(options.manifest) >> File "/home/wkozaczuk/projects/osv/scripts/manifest_common.py", line >> 59, in read_manifest >> hostpath = components[1].strip() >> IndexError: list index out of range >> ./scripts/build failed: $SRC/scripts/upload_manifest.py -o usr.img -m >> usr.manifest -D jdkbase=$jdkbase -D gccbase=$gccbase -D >> glibcbase=$glibcbase -D miscbase=$miscbase >> >> Also as Nadav pointed out when one stops (Ctrl-C) after download (wget) >> next run happily does NOT do anything. >> >> Waldek >> >> On Tuesday, January 16, 2018 at 2:44:29 PM UTC-5, Geraldo Netto wrote: >>> >>> Signed-off-by: geraldo netto <gerald...@gmail.com> >>> --- >>> mysql/GET | 5 +---- >>> 1 file changed, 1 insertion(+), 4 deletions(-) >>> >>> diff --git a/mysql/GET b/mysql/GET >>> index 57ed3f8..8871124 100755 >>> --- a/mysql/GET >>> +++ b/mysql/GET >>> @@ -6,12 +6,9 @@ BUILDDIR=$BASEDIR/build >>> ROOTDIR=$BASEDIR/install >>> SRCDIR=$BASEDIR/mysql-$VERSION >>> >>> -if [ ! -f mysql-$VERSION.tar.gz ]; then >>> - wget -c http://dev.mysql.com/get/Downl >>> oads/MySQL-5.6/mysql-$VERSION.tar.gz >>> -fi >>> <http://dev.mysql.com/get/Downloads/MySQL-5.6/mysql-$VERSION.tar.gz-fi> >>> +wget -c http://dev.mysql.com/get/Downloads/MySQL-5.6/mysql-$VERSION. >>> tar.gz >>> tar zxf mysql-$VERSION.tar.gz >>> >>> - >>> mkdir -p $BUILDDIR >>> mkdir -p $ROOTDIR/etc >>> >>> -- >>> 2.7.4 >>> >>> -- >> You received this message because you are subscribed to the Google Groups >> "OSv Development" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to osv-dev+unsubscr...@googlegroups.com. >> For more options, visit https://groups.google.com/d/optout. >> > > -- > You received this message because you are subscribed to the Google Groups > "OSv Development" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to osv-dev+unsubscr...@googlegroups.com. > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "OSv Development" group. To unsubscribe from this group and stop receiving emails from it, send an email to osv-dev+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
