Hi All,
Well, I decided to spend more time off the computer during the holiday. I
wanted to spend more time with family and relaxing. But, I did get a lot of
things knocked off “The Schedule” for the upcoming 1.3-RC5 release since my
last update. I won’t go into all the fine details for everything.
The "over the holiday" work mostly boils down to this kind of stuff…
A number of package updates.
Changing some stuff in the RBE and installer to handle the COMMAND.COM
differently (again).
Reworked the RBE to use the GitLab repositories for the package origins
instead of downloading the ibiblio repo CD. (I go into reasons for doing it
this way later on)
Changed some package lists.
Did some other modifications to the RBE and installers.
So, the all the changes I wanted or needed to make for the RBE and upcoming
release are done now. At present, I just need to do testing and look for any
bugs. With all the major changes to the RBE and installer since RC4, there may
be some bugs that creeped inside. I may also find some issues that pre-existed
and have not yet been reported or fixed. But, I don’t foresee any major issues.
RC5 will be coming in the very near future. :-)
In a previous discussion in the FreeDOS-User list in response to a user, I
confirmed that there is at least one broken/corrupt zip archive in the current
repository. I count not say if there was more than one from my memory. Only,
that one was buried inside another zip in the OpenGEM package. During a release
build, the RBE checks many things and stores warnings and errors in a log. Here
is the error log from todays test build.
2021-11-28 15:52:33-05:00 Warning: FDI using development branch
2021-11-28 16:01:31-05:00 warning: no sources detected for 'cpidos'
2021-11-28 16:07:40-05:00 warning: no sources detected for 'romos'
2021-11-28 16:08:28-05:00 warning: no sources detected for 'djgpp_bs'
2021-11-28 16:08:41-05:00 warning: no sources detected for 'djgpp_bn'
2021-11-28 16:08:50-05:00 warning: no sources detected for 'djgpp_fq'
2021-11-28 16:08:58-05:00 warning: no sources detected for 'djgpp_fx'
2021-11-28 16:09:17-05:00 warning: no sources detected for 'djgpp_db'
2021-11-28 16:09:25-05:00 warning: no sources detected for 'djgpp_mk'
2021-11-28 16:10:07-05:00 warning: no sources detected for 'djgpp_ob'
2021-11-28 16:10:14-05:00 warning: no sources detected for 'djgpp_rh'
2021-11-28 16:10:21-05:00 warning: no sources detected for 'djgpp_gc'
2021-11-28 16:10:44-05:00 warning: no sources detected for 'djgpp_tx'
2021-11-28 16:12:09-05:00 warning: no sources detected for 'djgpp'
2021-11-28 16:15:01-05:00 warning: no sources detected for 'i16budoc'
2021-11-28 16:15:36-05:00 warning: no sources detected for 'djgpp_gp'
2021-11-28 16:16:16-05:00 warning: no sources detected for 'i16gcdoc'
2021-11-28 16:18:22-05:00 warning: no sources detected for 'i16nlelk'
2021-11-28 16:20:49-05:00 warning: no sources detected for 'msa'
2021-11-28 16:26:55-05:00 warning: no sources detected for 'tinyasm'
2021-11-28 16:33:29-05:00 warning: no sources detected for 'fmines'
2021-11-28 16:33:32-05:00 warning: no sources detected for 'watcomf'
2021-11-28 16:37:06-05:00 warning: no sources detected for 'watcomc'
2021-11-28 16:46:19-05:00 warning: no sources detected for 'bladeenc'
2021-11-28 16:46:22-05:00 warning: no sources detected for 'cdp'
2021-11-28 16:47:31-05:00 package 'opengem', error extracting
'/SOURCE/OPENGEM/SOURCES.ZIP/PROGRAMMING TOOLKITS AND UTILITIES AND BUILD
TOOLS/atari gem programming library hexaid manual for games.zip'
2021-11-28 16:47:31-05:00 be74c0f6 crc, package error.
2021-11-28 16:48:49-05:00 warning: no sources detected for 'clamdb'
2021-11-28 16:50:21-05:00 warning: no sources detected for 'gnufonts'
2021-11-28 17:00:01-05:00 warning: iso image 'temp/fd-bonus0.iso' 622mb. It is
over 618mb and getting close to the 650mb limit
As you can see. On the GitLab repository, only one nested zip throws an error.
As a rule, nearly all zips and nested zips are fully extracted prior to storing
them in GitLab. But, I left that one as-is because it is broken. There a couple
other nested zips I leave intact as well. But, those are because I think they
are meant to be left that way.
Ok, now for the reasons I put support into the RBE to use GitLab for the origin
of packages. First, this allows updating or minor changes to the software
packages without the need to push an update to the Official Software Repository
on ibiblio. During a previous RC, there were a number of language, description
or simple metadata updates made that required updating version numbers. This
confused and/or upset some people. That will no longer be a problem going
forward. Such changes can be made (and a lot were for upcoming RC5) without
needing to update the ibiblio package. However, since the GitLab repo is
upstream of ibiblio, any changes there will propagate to ibiblio when we
eventually update those packages.
Further more, an additional option was added to the RBE specifically for
testing things. When the RBE is fetching things like the installer or packages
from GitHub and GitLab, it can be told to use development branches when they
exist. This allows for making large changes to the RBE, installers and packages
without breaking things for anyone who may wish to build a release.
Normally when you checkout/clone a Git project, the files get stamped with the
current date/time. This was an issue that annoyed many people (including me) in
RC4. A while back, I deployed a solution to remedy that issue. And now when the
RBE fetches those Git Packages, it adjusts the date/time stamps.
Unfortunately... Instead of just downloading a RepoCD, checkout/cloning of
projects is much slower. I knew that would be the case. However, it is a LOT
slower. Like 15x or more. So… I made some adjustments to the RBE and added
multi-threaded support in two very time consuming areas. The
cloning/timestamp/compression of packages and the metadata/analysis processes.
Those two processes make up roughly 60-70% (maybe more) of the compute time to
build a release. Personally, I am only running the RBE in a VirtualBox instance
with 1 CPU core and see little benefit from the threading. But if I was in a
hurry, I could throw more cores at it an cut down build time drastically.
Eventually, I may drop some of the other stuff the RBE does into threads as
well. Maybe someday, I’ll also do some optimizing to the metadata & analysis
process to speed it up some more. But, it works and is “good-enough” for now.
:-)
Jerome
_______________________________________________
Freedos-devel mailing list
Freedos-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/freedos-devel
