[Freedos-devel] a proposed compress_dir algo

2022-09-06 Thread Paul Dufresne via Freedos-devel
Thinking about the problem... I think about writing a small program to compress 
a directory.
Here is the pseudocode I have come up to now:

Function compress_topdir(path)
  if path reference a zip file
decompress zip file to some_dir
compress_dir(some_dir)
  else
compress_dir(path)

Function compress_dir(dir)
  if dir contains only one directory file /* Gitlab zip file */
compress_dir(dir,"")
  else
compress_dir(dir,dir)

Function compress_dir (dir, tuncated_path)
  For each file in dir hereby named f
f_truncated=truncate_name(f) /* to 8.3 format */
if f is a dir
  truncated_path = truncated_path + '/' + f_truncated
  add truncated_path/f_truncated empty directory to zip
  compress_dir(dir, truncated_path)
else /* a file */
  copy dir/f to a temporary file named f_truncated
  update zip by adding truncated_path + "/" + f_truncated
  delete temporary file named f_truncated

Hope it make some sense... and that you don't feel flooded with useless 
information.
Might consider to make it as a Perl script.
Mostly because I consider learning Rakudo (Perl v. 6) but really would do 
it in plain Perl v.5. for it is installed on most Linux.


___
Freedos-devel mailing list
Freedos-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/freedos-devel


Re: [Freedos-devel] Ré : When a system with limited filenames length combine directories into one

2022-09-06 Thread Paul Dufresne via Freedos-devel
>If that sounds like it would solve the problem, I could add it to the fdpkg.sh 
>script. Then if it works, implement the equivalent in the RBE.

Thanks for this internal description of fdnpkg.sh. I'll take a look at it...___
Freedos-devel mailing list
Freedos-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/freedos-devel


Re: [Freedos-devel] Ré : When a system with limited filenames length combine directories into one

2022-09-06 Thread Jerome Shidel
Hi Paul, 

Well….

> On Sep 6, 2022, at 12:18 AM, Paul Dufresne via Freedos-devel 
>  wrote:
> 
> I wrongly said:
> >It's probably ok... because fdnpkg probably won't too.
> Nonsense! fdnpkg have no way to know that many directories have been combined 
> into one.
> 
> So I just need fdpkg to accept to combine directories with same 8 chars 
> truncated length to be combined together.


If the DJGPP program (DJ_GPP) works fine when the long directory names are 
truncated and combined, a possible solution would be to create a utility (or 
add an option/modify fdpkg.sh) to perform the task. Basically, scan directory 
names and if the would be truncated to 8.3 chars (and collide with another 
directory), to just do that in advance. 

I don’t know how familiar you are with bash scripting. However, the fdpkg.sh 
shell script was something thrown together quickly and is not very complex.

A general overview of what happens when it is executed…

First, it sets a couple internal global variables. Everything else is broken 
into functions to make it more organized. Then it calls the PKGU_main function 
with the command-line passed to the script.

PKGU_main - parses the command line options and either expands zips or tries to 
compress paths into archives. When compressing and it is LFN mode is enabled 
(the default, set by "LFN_ZIP=true”), it will call the a function 
(PKGU_SmashLFN) to see if any LFNS are present and any LFNFILES.ZIP archives 
are required. Otherwise, it will just proceed without checking. Next, it will 
attempt to compress the path as DOS 8.3 compatible. If this cannot be done, the 
script will abort with an error.  

PKGU_SmashLFN - looks at the the primary directory names and determines if they 
are SOURCE, NORMAL or SPECIAL directories (like LINKS). Those are either 
handled like sources (PKGU_CheckSources), the binaries (PKGU_CheckPath) or 
specifically exempt from LFN scanning and compression. 

PKGU_CheckSources and PKGU_CheckPath - Basically do the same things in a 
slightly different way with with slightly different settings. The search for 
the LFNs and Compress those when found. Their are slightly different conditions 
used by each to determine what qualifies as a Long File Name. PKGU_CheckPath 
uses the PKGU_SearchLFN to scan for any files that are NOT compatible with 8.3. 
However, PKGU_CheckSources uses PKGU_SearchEFN that will do the same but 
include any filename that is not uppercase. 

When the RBE performs the compression of packages, it does some similar 
scan/compressing. However for several reasons, it always compresses the SOURCE 
directory. Primarily, this tends to reduce the overall size of the archive. It 
also requires less disk space when sources are installed. A side benefit is 
that when always compressing that path, it never needs to actually scanned. I 
should update fdpkg.sh to just always compress those files without scanning to 
make it more like process used by the RBE.

So….

There are several ways to add a TRUNCATE and COMBINE long path names to the 
script. First, probably add a global variable to en/disable allowing the 
action. Something like LFN_CRUSH=false. Then a command line option like ‘-c’ to 
enable and maybe ‘-n’ to disable it. (default probably -c)

Then under PKGU_SmashLFN for specific paths call a new function to perform the 
action. Or, do that under PKGU_CheckSources and/or PKGU_CheckPath. The new 
could be called PKGU_CrushDirs. 

PKGU_CrushDirs could be implemented several ways. The easiest would be to just 
figure out what the truncated directory name would be and see if that collides 
with anything else. When that happens, just move that stuff into the same 
truncated directory. 

If that sounds like it would solve the problem, I could add it to the fdpkg.sh 
script. Then if it works, implement the equivalent in the RBE.

:-)

Jerome





> 
> ___
> Freedos-devel mailing list
> Freedos-devel@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/freedos-devel

___
Freedos-devel mailing list
Freedos-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/freedos-devel