Re: [fpc-devel] Interface to compressed files and archives
On Friday 31 December 2004 14:24, [EMAIL PROTECTED] wrote: > Normally one never uses a fully qualified identifier. So I'm not normal. ;) I *always* use them. Makes life easier. > Only when a possible name conflict exists, which > - Should be very rare, and avoided in the first place. IBTD. I even often use the same names for same things, but in different units (like Open/Close, ...). I don't see any reason to not use the unit-name for identifiers. This also avoids C-like prefixes to each name. One usually does the same with OOP techniques where each constructor is called Init or Create. ;-) And BTW, it is not very long ago that this actually saved me from intermixing Objects.NewStr with SysUtils.NewStr. Vinzent. ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] Interface to compressed files and archives
On Sat, 1 Jan 2005, DrDiettrich wrote: > [EMAIL PROTECTED] wrote: > > > > E.g.: gzip.xyz, is this based on a gzip unit or a gzip variable or... > > > > Does this matter to you ? > > > > Normally one never uses a fully qualified identifier. > > And that can become a problem, when a variable and a unit has the same > name. That's why I do not only prefer to prefix type names with "T", but > also unit names with "u", form (unit) names with "f" etc. As prefixes > for specific kinds of units seem to be in use by other people as well, > why not prefix all units? Because it doesn't seem necessary. > > > Only when a possible name conflict exists, which > > - Should be very rare, and avoided in the first place. > > - In such cases it will be obvious from the context. > > Okay, name clashes between unit and variable names should be detectable > easily. But then a decision has to be made, which of both names should > stay unchanged, and which one to decorate. My preference then is to > decorate the unit names, because these occur less frequently in source > code, and almost only in obvious Uses clauses. You are free to do so :) > > I know that my private prefix style is a bit uncommon, as is my coding > style (indentation...). In shareable contributions I'm willing to follow > the more widely accepted standards, of course :-) It depends a bit on the context. Large Delphi component suites all have their unit names prefixed with a prefix that identifies the suite. There is even a registration system somewhere on the net, to 'register' your prefix. If you're going to make an all-purpose archiving suite, I suggest you do the same: use a common prefix (as proposed in an earlier mail) like ar, vfs, dca or whatever. Michael. ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] Interface to compressed files and archives
[EMAIL PROTECTED] wrote: > > E.g.: gzip.xyz, is this based on a gzip unit or a gzip variable or... > > Does this matter to you ? > > Normally one never uses a fully qualified identifier. And that can become a problem, when a variable and a unit has the same name. That's why I do not only prefer to prefix type names with "T", but also unit names with "u", form (unit) names with "f" etc. As prefixes for specific kinds of units seem to be in use by other people as well, why not prefix all units? > Only when a possible name conflict exists, which > - Should be very rare, and avoided in the first place. > - In such cases it will be obvious from the context. Okay, name clashes between unit and variable names should be detectable easily. But then a decision has to be made, which of both names should stay unchanged, and which one to decorate. My preference then is to decorate the unit names, because these occur less frequently in source code, and almost only in obvious Uses clauses. I know that my private prefix style is a bit uncommon, as is my coding style (indentation...). In shareable contributions I'm willing to follow the more widely accepted standards, of course :-) DoDi ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] Interface to compressed files and archives
On Fri, 31 Dec 2004, DrDiettrich wrote: > [EMAIL PROTECTED] wrote: > > > Naming a unit with 'u' standard does not seem useful to me, but this is > > a matter of taste. > ... > > All other files are assumed to be units. > > (projects/packages have distinct extensions anyway) > > No problem at the directory level, but how to distinguish names of > units, types, variables etc. in qualified references? - Types should always be prepended with T. - One should avoid the use of globally visible variables. They are inherently evil. Specially when doing multithread programming. > E.g.: gzip.xyz, is this based on a gzip unit or a gzip variable or... Does this matter to you ? Normally one never uses a fully qualified identifier. Only when a possible name conflict exists, which - Should be very rare, and avoided in the first place. - In such cases it will be obvious from the context. Michael. ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] Interface to compressed files and archives
[EMAIL PROTECTED] wrote: > Naming a unit with 'u' standard does not seem useful to me, but this is > a matter of taste. ... > All other files are assumed to be units. > (projects/packages have distinct extensions anyway) No problem at the directory level, but how to distinguish names of units, types, variables etc. in qualified references? E.g.: gzip.xyz, is this based on a gzip unit or a gzip variable or... DoDi ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] Interface to compressed files and archives
Marco van de Voort wrote: > Better have a separate way. Otherwise you can't set e.g. a compressionlevel > for that stream, _or_ you have to have lots of different constructors. Compressors can require any kind and number of arguments, that must be reflected somewhere, e.g. in the specific constructor. It's not guaranteed that a compressor will have usable defaults for all parameters. For now only decompression will be supported by a general function, where the decompressor (class) can be selected from the file extension or header, and the decompressor then should be able to determine the appropriate parameters from the concrete data source. > One other thing to keep in mind (iirc) is that some algo's require the > uncompressed size to unpack, and some the compressed size. So probably > your interface has to support both. Good to know. But the uncompressed size cannot be known in the general case, so that such information must be supplied together with the compressed data. The wrapper processor then must know how to retrieve the unpacked size and the data from its own input stream. > And use a 64-bit size and an endianness indicator if possible. The 64 bit issue should be handled by the basic TStream class, where also the according data types for size_t, off_t (or equivalent) should be defined appropriately. The endianness of the target system also can be selected at compile time - how? > Search for "zfs" (zip filesystem). It was FPC compat for a while. Can you be a bit more specific? I have traditional problems in searching :-( My first search resulted in >6000 hits, almost related to zip drives. My second search retrieved two messages about errors in a StarKit... package. > > I already > > decided to replace my own "stdc" unit by the FPC "libc" unit, with > > hopefully no changes to that unit. > > Then change them again to use BaseUnix/Unix :-) I could locate 2 BaseUnix.pp units, for Linux and BSD. But I'm developing under Windows :-( What I really need is a unit with commonly used type names, so that all modules ported from C to Pascal share the same types. The module specific types are less important, because these are not shared with other modules; nonetheless even these types should be defined based on common types, like cUInt8. > The "libc" unit is not a base unit of FPC, but exists merely for Kylix > porting purposes, since it is pretty much only a direct x86 glibc > translation, and not a general POSIX abstraction. It is only supported for > Linux/x86. Good to know. I already wondered how the implementation could work on Windows. > As said, a portable subset is available in units baseunix/unix Where? I couldn't find any such directory and unit in the FPC sources :-( A related question: which (source...) directories must I add to the search path, for using FPC and (primarily) Lazarus on Windows? > See http://www.freepascal.org/docs-html/rtl/ for some docs > http://www.stack.nl/~marcov/unixrtl.pdf Just downloaded, but I couldn't find neither on the server nor in the expanded docs-html.zip a /rtl directory. DoDi ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] Interface to compressed files and archives
On Thu, 30 Dec 2004, Marco van de Voort wrote: > > Just don't use anything conflicting with other prefixes: lib, z, db, im, > > sys. > > > > If we can agree on an implementation for archive handling that can be > > considered a standard implementation for FPC (i.e. distributed with the > > FCL), 'fp' can (and should) be used. > > > > Naming a unit with 'u' standard does not seem useful to me, but this is > > a matter of taste. In general, in Delphi, I use only 4 fixed prefixes: > > > > fra for Frames > > frm for Forms > > dm for DataModules > > mgr for Manager classes. > > Only additional general one i use is > > dlg for dialog In my view, dialog=form, the only 'dialogs' are the windows stock dialogs. But we have some forms that qualify. Thanks for the idea!! :-) Michael. ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] Interface to compressed files and archives
> Just don't use anything conflicting with other prefixes: lib, z, db, im, sys. > > If we can agree on an implementation for archive handling that can be > considered a standard implementation for FPC (i.e. distributed with the > FCL), 'fp' can (and should) be used. > > Naming a unit with 'u' standard does not seem useful to me, but this is > a matter of taste. In general, in Delphi, I use only 4 fixed prefixes: > > fra for Frames > frm for Forms > dm for DataModules > mgr for Manager classes. Only additional general one i use is dlg for dialog ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] Interface to compressed files and archives
On Thu, 30 Dec 2004, DrDiettrich wrote: I split up my answers to your mail, because the topics are a bit different. > Now you should have gotten the big picture of my intended activities. > Many more questions will arise when I proceed with my work. I already > decided to replace my own "stdc" unit by the FPC "libc" unit, with > hopefully no changes to that unit. None other than bugfixes will be accepted: libc is the interface to the real C library (glibc) on Linux. The interface is therefore fixed and not subject to change, unless libc changes. You must explain what you want: + An interface to libc - then the libc is OK for you. + An imitation of libc (on unix) - then you should do as marco implied: use Unix/BaseUnix + An imitation of libc (general) - then you must implement a new unit, the above ones are for Unices only. + A cross-platform uniform pascal API - Then you should stick to sysutils/strutils/dateutils etc. > For further compatibility it will be > necessary to find compromises between my coding style, and the style > used by the FPC community. E.g. I prefer to prefix all my units with an > "u", so that the base names remain available for procedures or > variables. I also use upper case characters in the unit names, what may > not be appreciated by users from the Unix world. As a compromise it may > be possible to use a "lib" prefix, but this may conflict with existing > library names (libz...). Any ideas? You can name the unit identifier with uppercase characters, but not the filename. Filenames lowercase only, please. As for prefixes: for a compression/decompression set of units, I would use dc or cd as prefix. If it is 'archiving' in general, 'ar' seems best suited. And if you want a 'virtual file system', use vfs. Just don't use anything conflicting with other prefixes: lib, z, db, im, sys. If we can agree on an implementation for archive handling that can be considered a standard implementation for FPC (i.e. distributed with the FCL), 'fp' can (and should) be used. Naming a unit with 'u' standard does not seem useful to me, but this is a matter of taste. In general, in Delphi, I use only 4 fixed prefixes: fra for Frames frm for Forms dm for DataModules mgr for Manager classes. All other files are assumed to be units. (projects/packages have distinct extensions anyway) Michael. ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] Interface to compressed files and archives
On Thu, 30 Dec 2004, DrDiettrich wrote: > Hi there, > > I'm new to this list and want to introduce myself and my intended > contributions to FreePascal. > > My name is Dr. Hans-Peter Diettrich, and I live in Flensburg (Germany). > For brevity I use to sign my messages as DoDi. My main interests are > decompilers and (tools for) porting code. Usually I work with Delphi, > but this behaviour may change ;-) > > Recently I came across some interesting library modules of FPC, that I > want to use in my own projects. Some of these modules deserve updates, > in general and for use with Delphi, and I want to contribute my > according work to the FPC community. > > Currently I'm implementing an RPM clone for Windows, which in detail > should support source rpm's, better than the original RPM. Hereby I have > to deal with compressed files in various formats (gzip, bzip2), and > archive files (cpio, tar...). I've already update or implemented some > of these modules, now I want to define a common interface and API for > compressed and archive streams, based on TStreams. The zstream unit is > dedicated to a single compressor, but it has an handy name. How should I > name a more general unit, would "zstreams" be acceptable? > > My idea of a general (de-)compression interface is as follows: > Your idea is already implemented in Abbrevia from TurboPower. Abbrevia is open source, and is maintained on SourceForge. Free Pascal has a copy of it that compiles in CVS, check out contrib/abbrevia Michael. ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] Interface to compressed files and archives
> I'm new to this list and want to introduce myself and my intended > contributions to FreePascal. You are not new to everybody:-) > archive files (cpio, tar...). I've already update or implemented some > of these modules, now I want to define a common interface and API for > compressed and archive streams, based on TStreams. The zstream unit is > dedicated to a single compressor, but it has an handy name. How should I > name a more general unit, would "zstreams" be acceptable? Maybe just streams. I also have a encryption stream (Rijndael) based on a PD unit that I could try to adapt. Of course that depends on how specific the interface is for compression (compres//decompress crypt/decrypt are similar) > Then a general Open or Decompress procedure can determine which > decompressor to use for an given stream, and can create the appropriate > decompressor object. For compressors it may be better to create the > according object directly, according to the desired compression format, > in which case the according arguments also can be passed to the > constructor of that class in the appropriate form. Better have a separate way. Otherwise you can't set e.g. a compressionlevel for that stream, _or_ you have to have lots of different constructors. One other thing to keep in mind (iirc) is that some algo's require the uncompressed size to unpack, and some the compressed size. So probably your interface has to support both. And use a 64-bit size and an endianness indicator if possible. > Archive files deserve a more elaborate API, so that the files in an > archive can be extracted to individual files or streams. There was > already a suggestion, to define something like a virtual file system > interface for archive files. I suspect that something like this already > exists for use in the GUI browsers of both Linux and Windows. This may > deserve some research, before an accordingly compatible interface can be > defined. Now I'm waiting for according contributions from the OS gurus > before proceeding with this approach. Search for "zfs" (zip filesystem). It was FPC compat for a while. > Now you should have gotten the big picture of my intended activities. > Many more questions will arise when I proceed with my work. I already > decided to replace my own "stdc" unit by the FPC "libc" unit, with > hopefully no changes to that unit. Then change them again to use BaseUnix/Unix :-) The "libc" unit is not a base unit of FPC, but exists merely for Kylix porting purposes, since it is pretty much only a direct x86 glibc translation, and not a general POSIX abstraction. It is only supported for Linux/x86. As said, a portable subset is available in units baseunix/unix, with threadsafe errno and most calls have "fp" prefixed to avoid nameclashes with older FPC unix units (oldlinux) and Kylix' libc. See http://www.freepascal.org/docs-html/rtl/ for some docs, and http://www.stack.nl/~marcov/unixrtl.pdf > For further compatibility it will be > necessary to find compromises between my coding style, and the style > used by the FPC community. E.g. I prefer to prefix all my units with an > "u", so that the base names remain available for procedures or > variables. I also use upper case characters in the unit names, what may > not be appreciated by users from the Unix world. As a compromise it may > be possible to use a "lib" prefix, but this may conflict with existing > library names (libz...). Any ideas? unit prefixes for units belonging together are not uncommon (see e.g. the image and db routines), but I would avoid the uppercase unitnames and don't use already taken prefixes like "lib", "db", "unix" and "fp" ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
[fpc-devel] Interface to compressed files and archives
Hi there, I'm new to this list and want to introduce myself and my intended contributions to FreePascal. My name is Dr. Hans-Peter Diettrich, and I live in Flensburg (Germany). For brevity I use to sign my messages as DoDi. My main interests are decompilers and (tools for) porting code. Usually I work with Delphi, but this behaviour may change ;-) Recently I came across some interesting library modules of FPC, that I want to use in my own projects. Some of these modules deserve updates, in general and for use with Delphi, and I want to contribute my according work to the FPC community. Currently I'm implementing an RPM clone for Windows, which in detail should support source rpm's, better than the original RPM. Hereby I have to deal with compressed files in various formats (gzip, bzip2), and archive files (cpio, tar...). I've already update or implemented some of these modules, now I want to define a common interface and API for compressed and archive streams, based on TStreams. The zstream unit is dedicated to a single compressor, but it has an handy name. How should I name a more general unit, would "zstreams" be acceptable? My idea of a general (de-)compression interface is as follows: In the general decompression unit a list of all available compressors is maintained, every implemented and used compressor adds itself to this list, in the initialization section of it's main unit. Then a general Open or Decompress procedure can determine which decompressor to use for an given stream, and can create the appropriate decompressor object. For compressors it may be better to create the according object directly, according to the desired compression format, in which case the according arguments also can be passed to the constructor of that class in the appropriate form. The use of the de/compression stream objects should be obvious, Read or Write is called until the EOF. The legacy C code of the compressors is based on error codes and conditions that must be checked after almost every call to an internal function, and which are available as the final result after the information is fully processed. I want to modify that model, so that errors will raise the predefined stream exceptions. This approach will simplify, and make more transparent, the existing code as well as the application code. It also will allow to hide the compressor specific error codes from the application. Such a change will be incompatible with the inherited decompressor API's, but does anybody see a need to further support alternative and specialized access to de/compressors, beyond the stream support? If we can agree about the above details, I plan to convert the gzip, bzip2 and zip modules to that common interface. I'm also willing to update further modules for use of that interface, provided that the modules already exist as Pascal source code. --- Archive files deserve a more elaborate API, so that the files in an archive can be extracted to individual files or streams. There was already a suggestion, to define something like a virtual file system interface for archive files. I suspect that something like this already exists for use in the GUI browsers of both Linux and Windows. This may deserve some research, before an accordingly compatible interface can be defined. Now I'm waiting for according contributions from the OS gurus before proceeding with this approach. A much simpler interface could be based on enumeration and callback procedures, that will allow to process existing archive files sequentially. It also may be possible to create an directory tree for an archive, but for now I will leave such an implementation to somebody else ;-) For the creation of new archive files, methods are required to add files to the archive directory. The simplest approach will be based on physical (existing) files, whose attributes can be retrieved by the archiver from the existing file system. Then the application code must not care about all related details. --- Now you should have gotten the big picture of my intended activities. Many more questions will arise when I proceed with my work. I already decided to replace my own "stdc" unit by the FPC "libc" unit, with hopefully no changes to that unit. For further compatibility it will be necessary to find compromises between my coding style, and the style used by the FPC community. E.g. I prefer to prefix all my units with an "u", so that the base names remain available for procedures or variables. I also use upper case characters in the unit names, what may not be appreciated by users from the Unix world. As a compromise it may be possible to use a "lib" prefix, but this may conflict with existing library names (libz...). Any ideas? I'll stop now and thank you for your patient reading. Feel free to modify the subject or to open new threads for discussing details. Happy New Year DoDi ___ fpc-devel maillist - fpc-devel@lists.freepas
