Re: [fpc-pascal] Adding file to string to the RTL
> On Oct 6, 2020, at 1:42 AM, Jer Haan via fpc-pascal > wrote: > > I use this function to read a file into a string: Not sure how Michael implemented it but we have this in the RTL now, so we can throw away our ReadFile functions. I have many of those littered around in various projects so I'm looking forward to killing it. ;) Regards, Ryan Joseph ___ fpc-pascal maillist - fpc-pascal@lists.freepascal.org https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Re: [fpc-pascal] Adding file to string to the RTL
Since we're on the topic how about another one-liner for reading all the files
in directory into a dynamic array? This has the added benefit of getting
enumeration for free. This is standard stuff for working with files in
scripting languages so I think the FPC RTL should include something like this
also.
type
TStringArray = array of string;
function FindAllFiles(path: string): TStringArray;
var
info: TSearchRec;
begin
if not DirectoryExists(path) then
raise Exception.Create('Directory "'+path+'"" doesn''t exist');
path := path+DirectorySeparator+'*';
result := [];
if FindFirst(path, faAnyFile, info) = 0 then
begin
repeat
result += [info.name];
until FindNext(info) <> 0;
FindClose(info);
end;
end;
begin
// fastest way to read all text files in a directory.
// only 2 lines doesn't leak memory
for name in FindAllFiles('/usr/local/lib/fpc') do
writeln(GetFileAsString('/usr/local/lib/fpc/'+name));
end.
Regards,
Ryan Joseph
___
fpc-pascal maillist - fpc-pascal@lists.freepascal.org
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Re: [fpc-pascal] Merging units
> On Oct 6, 2020, at 2:52 PM, Zamrony P. Juhara via fpc-pascal > wrote: > > I use a secondary unit as alias of all separate units as Marco suggests in > > https://github.com/fanoframework/fano/blob/master/src/fano.pas > > However I am curious if merging units like this has drawback. AFAIK smart > linking works. Got around to trying this and the namespace aliases are nice. Thanks. What about functions though? This will require lots of inline wrappers. Having a function alias would be a really nice feature but I think Sven actually rejected this idea some while ago when I asked. Regards, Ryan Joseph ___ fpc-pascal maillist - fpc-pascal@lists.freepascal.org https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Re: [fpc-pascal] Merging units
I use a secondary unit as alias of all separate units as Marco suggests in https://github.com/fanoframework/fano/blob/master/src/fano.pas However I am curious if merging units like this has drawback. AFAIK smart linking works. Sent from Yahoo Mail on Android On Sat, Oct 3, 2020 at 3:31, Marco van de Voort via fpc-pascal wrote: Op 2020-10-02 om 19:55 schreef Ryan Joseph via fpc-pascal: > Something that's bothered me for a while I wanted to ask about. > > I have a package I want to distribute which has 2 or more units, all of which > are required to use the package. Technically all the code should be in one > file but because units are a nice feature of Pascal we like to keep our code > separate when we develop. > > Usually what happens is we compromise and do either: > > 1) Have the user of the package add all the units to the uses clause of every > file which needs it. This is annoying for the user because it bloats the uses > section but for the developer our code is nicely separated into units. > > 2) Break the units into include files and wrap the interface/implementation > blocks in ifdefs. This is nice for the user because they only have a single > unit to use but now the developers of the package have to work in what are > essentially header files (like in C). The RTL is of course filled with > examples like this. In some cases you can also move types and consts to a secondary unit, but alias them in the primary one. Like e.g. Unixtype (what unit you say? Never heard of it? That is because the types are aliased into core units baseunix and unix) ___ fpc-pascal maillist - fpc-pascal@lists.freepascal.org https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal ___ fpc-pascal maillist - fpc-pascal@lists.freepascal.org https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Re: [fpc-pascal] Adding file to string to the RTL
On Tue, 6 Oct 2020, Bart via fpc-pascal wrote: On Tue, Oct 6, 2020 at 10:12 AM Michael Van Canneyt via fpc-pascal wrote: // Assume TEncoding.SystemEncoding Function GetFileAsString(Const aFileName : RawByteString) : RawByteString; // Specify encoding Function GetFileAsString(Const aFileName : RawByteString; aEncoding : TEncoding) : RawByteString; // Assume TEncoding.Unicode contents Function GetFileAsString(Const aFileName : UnicodeString) : UnicodeString; // Specify encoding, return Unicode string. Function GetFileAsString(Const aFileName : UnicodeString; aEncoding : TEncoding) : UnicodeString; Now I want to get the contents of a file, which contains either UTF8 or some single-byte encoding, and the filename is NOT in my locale (e.g. it may be chinese, which I cannot represent in RawByteString), Of course you can. That is why we use RawByteString for file opening functions. You specify it as UTF8 in that case, the RTL will do the rest. The whole of lazarus works on this principle. Michael. ___ fpc-pascal maillist - fpc-pascal@lists.freepascal.org https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Re: [fpc-pascal] Adding file to string to the RTL
On Tue, 6 Oct 2020, Ryan Joseph via fpc-pascal wrote: On Oct 6, 2020, at 2:12 AM, Michael Van Canneyt via fpc-pascal wrote: I added the following functions to the sysutils unit (rev 47056): Great, thanks Michael. I've always used AnsiString so why is UnicodeString preferable here? If the original file contains UTF16, it will save you a conversion. I usually do not use UTF16, you clearly also not, but other people do, and we try to cater for them as well :-) So is the idea we need to specify an UTF-8 encoding for unicode otherwise it assumes ASCII format? It will assume the system unit default encoding. What that is depends on your OS. Mac, Linux: UTF8, windows: ANSI, unless you use Lazarus in which case it will also be UTF8. (in fact you can set it to any codepage you want, but the above are the defaults) I didn't specify the encoding with TStringList and I always seemed to get back what I wanted (maybe it sniffed the encoding from the file?). Depending on your environment TStringList will use ANSI or UTF8, just as above. Michael. ___ fpc-pascal maillist - fpc-pascal@lists.freepascal.org https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Re: [fpc-pascal] Adding file to string to the RTL
On Tue, Oct 6, 2020 at 10:12 AM Michael Van Canneyt via fpc-pascal wrote: > // Assume TEncoding.SystemEncoding > Function GetFileAsString(Const aFileName : RawByteString) : RawByteString; > // Specify encoding > Function GetFileAsString(Const aFileName : RawByteString; aEncoding : > TEncoding) : RawByteString; > // Assume TEncoding.Unicode contents > Function GetFileAsString(Const aFileName : UnicodeString) : UnicodeString; > // Specify encoding, return Unicode string. > Function GetFileAsString(Const aFileName : UnicodeString; aEncoding : > TEncoding) : UnicodeString; Now I want to get the contents of a file, which contains either UTF8 or some single-byte encoding, and the filename is NOT in my locale (e.g. it may be chinese, which I cannot represent in RawByteString), but when I specify the filename as UnicodeString I am forced to receive the contents Unicode as well. From my perspective that sucks. -- Bart ___ fpc-pascal maillist - fpc-pascal@lists.freepascal.org https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Re: [fpc-pascal] Adding file to string to the RTL
Well, the answer is twofold: 1. IOUtils does not exist yet. Now the OP has a solution. 2. I don't use IOUtils and don't plan to. Its .NET-like approach has no added value for me. Just like I don't use rtii unit, but always use the more low-level typinfo unit. But once TFile will be implemented, it can obviously reuse the functions from Sysutils. (like it presumably will for 99% of all other functions in it). If need be the API can be extended in SysUtils to match what IOUtils needs.. Michael. On Tue, 6 Oct 2020, Nico Neumann wrote: Wouldn't it be better to implement it directly as *IOUtils.TFile* (like Deplhi)? *class function ReadAllBytes(const Path: string): TBytes; static;class function ReadAllLines(const Path: string): TStringDynArray;overload; static;class function ReadAllLines(const Path: string; const Encoding: TEncoding): TStringDynArray; overload; static;class function ReadAllText(const Path: string): string; overload; inline; static;class function ReadAllText(const Path: string; const Encoding: TEncoding): string; overload; inline; static;* Souce: http://docwiki.embarcadero.com/Libraries/Sydney/en/System.IOUtils.TFile_Methods Then there would be only one way in the RTL for this task and not two once *IOUtils* (foundation webpage says it's planned anyway) is implemented. Am Di., 6. Okt. 2020 um 10:12 Uhr schrieb Michael Van Canneyt via fpc-pascal : On Mon, 5 Oct 2020, Ryan Joseph via fpc-pascal wrote: On Oct 5, 2020, at 5:08 PM, Jean SUZINEAU via fpc-pascal < fpc-pascal@lists.freepascal.org> wrote: In my own code I use BlockRead/BlockWrite, but I'm wondering if I've not seen this somewhere in RTL. This looks good to me what about the concerns raised by Michael? I don't know enough about text formats but using AnsiString always just seems to work for me (I assume the compiler did something magic behind the scenes). No, we don't deal in magic, only bits and bytes :-) I added the following functions to the sysutils unit (rev 47056): // Read raw content as bytes Function GetFileContents(Const aFileName : RawByteString) : TBytes; Function GetFileContents(Const aFileName : UnicodeString) : TBytes; Function GetFileContents(Const aHandle : THandle) : TBytes; // Read content as string // Assume TEncoding.SystemEncoding Function GetFileAsString(Const aFileName : RawByteString) : RawByteString; // Specify encoding Function GetFileAsString(Const aFileName : RawByteString; aEncoding : TEncoding) : RawByteString; // Assume TEncoding.Unicode contents Function GetFileAsString(Const aFileName : UnicodeString) : UnicodeString; // Specify encoding, return Unicode string. Function GetFileAsString(Const aFileName : UnicodeString; aEncoding : TEncoding) : UnicodeString; These functions will raise an exception if the file cannot be opened or read. They cannot be used on handles that do not support FileSeek() (sockets, pipes, stdin/stdout etc.). I did some tests on encoding conversion but not extensively. If you find any errors, please report them through the bugtracker. Michael. ___ fpc-pascal maillist - fpc-pascal@lists.freepascal.org https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal ___ fpc-pascal maillist - fpc-pascal@lists.freepascal.org https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Re: [fpc-pascal] Adding file to string to the RTL
Wouldn't it be better to implement it directly as *IOUtils.TFile* (like Deplhi)? *class function ReadAllBytes(const Path: string): TBytes; static;class function ReadAllLines(const Path: string): TStringDynArray;overload; static;class function ReadAllLines(const Path: string; const Encoding: TEncoding): TStringDynArray; overload; static;class function ReadAllText(const Path: string): string; overload; inline; static;class function ReadAllText(const Path: string; const Encoding: TEncoding): string; overload; inline; static;* Souce: http://docwiki.embarcadero.com/Libraries/Sydney/en/System.IOUtils.TFile_Methods Then there would be only one way in the RTL for this task and not two once *IOUtils* (foundation webpage says it's planned anyway) is implemented. Am Di., 6. Okt. 2020 um 10:12 Uhr schrieb Michael Van Canneyt via fpc-pascal : > > > On Mon, 5 Oct 2020, Ryan Joseph via fpc-pascal wrote: > > > > > > >> On Oct 5, 2020, at 5:08 PM, Jean SUZINEAU via fpc-pascal < > fpc-pascal@lists.freepascal.org> wrote: > >> > >> In my own code I use BlockRead/BlockWrite, but I'm wondering if I've > not seen this somewhere in RTL. > >> > >> > > > > This looks good to me what about the concerns raised by Michael? I don't > > know enough about text formats but using AnsiString always just seems to > > work for me (I assume the compiler did something magic behind the > scenes). > > No, we don't deal in magic, only bits and bytes :-) > > I added the following functions to the sysutils unit (rev 47056): > > // Read raw content as bytes > > Function GetFileContents(Const aFileName : RawByteString) : TBytes; > Function GetFileContents(Const aFileName : UnicodeString) : TBytes; > Function GetFileContents(Const aHandle : THandle) : TBytes; > > // Read content as string > > // Assume TEncoding.SystemEncoding > Function GetFileAsString(Const aFileName : RawByteString) : RawByteString; > // Specify encoding > Function GetFileAsString(Const aFileName : RawByteString; aEncoding : > TEncoding) : RawByteString; > // Assume TEncoding.Unicode contents > Function GetFileAsString(Const aFileName : UnicodeString) : UnicodeString; > // Specify encoding, return Unicode string. > Function GetFileAsString(Const aFileName : UnicodeString; aEncoding : > TEncoding) : UnicodeString; > > These functions will raise an exception if the file cannot be opened or > read. > They cannot be used on handles that do not support FileSeek() (sockets, > pipes, stdin/stdout etc.). > > I did some tests on encoding conversion but not extensively. > If you find any errors, please report them through the bugtracker. > > Michael. > ___ > fpc-pascal maillist - fpc-pascal@lists.freepascal.org > https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal > ___ fpc-pascal maillist - fpc-pascal@lists.freepascal.org https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Re: [fpc-pascal] Adding file to string to the RTL
Not 100% on this but I think the gist is that UnicodeString is compatible with AnsiString and a conversion is done on assignment. -- Alexander Grotewohl https://dcclost.com From: fpc-pascal on behalf of Ryan Joseph via fpc-pascal Sent: Tuesday, October 6, 2020 12:39:43 PM To: FPC-Pascal users discussions Cc: Ryan Joseph Subject: Re: [fpc-pascal] Adding file to string to the RTL > On Oct 6, 2020, at 2:12 AM, Michael Van Canneyt via fpc-pascal > wrote: > > I added the following functions to the sysutils unit (rev 47056): Great, thanks Michael. I've always used AnsiString so why is UnicodeString preferable here? So is the idea we need to specify an UTF-8 encoding for unicode otherwise it assumes ASCII format? I didn't specify the encoding with TStringList and I always seemed to get back what I wanted (maybe it sniffed the encoding from the file?). Regards, Ryan Joseph ___ fpc-pascal maillist - fpc-pascal@lists.freepascal.org https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal ___ fpc-pascal maillist - fpc-pascal@lists.freepascal.org https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Re: [fpc-pascal] Adding file to string to the RTL
> On Oct 6, 2020, at 2:12 AM, Michael Van Canneyt via fpc-pascal > wrote: > > I added the following functions to the sysutils unit (rev 47056): Great, thanks Michael. I've always used AnsiString so why is UnicodeString preferable here? So is the idea we need to specify an UTF-8 encoding for unicode otherwise it assumes ASCII format? I didn't specify the encoding with TStringList and I always seemed to get back what I wanted (maybe it sniffed the encoding from the file?). Regards, Ryan Joseph ___ fpc-pascal maillist - fpc-pascal@lists.freepascal.org https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Re: [fpc-pascal] Adding file to string to the RTL
On Mon, 5 Oct 2020, Ryan Joseph via fpc-pascal wrote: On Oct 5, 2020, at 5:08 PM, Jean SUZINEAU via fpc-pascal wrote: In my own code I use BlockRead/BlockWrite, but I'm wondering if I've not seen this somewhere in RTL. This looks good to me what about the concerns raised by Michael? I don't know enough about text formats but using AnsiString always just seems to work for me (I assume the compiler did something magic behind the scenes). No, we don't deal in magic, only bits and bytes :-) I added the following functions to the sysutils unit (rev 47056): // Read raw content as bytes Function GetFileContents(Const aFileName : RawByteString) : TBytes; Function GetFileContents(Const aFileName : UnicodeString) : TBytes; Function GetFileContents(Const aHandle : THandle) : TBytes; // Read content as string // Assume TEncoding.SystemEncoding Function GetFileAsString(Const aFileName : RawByteString) : RawByteString; // Specify encoding Function GetFileAsString(Const aFileName : RawByteString; aEncoding : TEncoding) : RawByteString; // Assume TEncoding.Unicode contents Function GetFileAsString(Const aFileName : UnicodeString) : UnicodeString; // Specify encoding, return Unicode string. Function GetFileAsString(Const aFileName : UnicodeString; aEncoding : TEncoding) : UnicodeString; These functions will raise an exception if the file cannot be opened or read. They cannot be used on handles that do not support FileSeek() (sockets, pipes, stdin/stdout etc.). I did some tests on encoding conversion but not extensively. If you find any errors, please report them through the bugtracker. Michael. ___ fpc-pascal maillist - fpc-pascal@lists.freepascal.org https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Re: [fpc-pascal] Adding file to string to the RTL
Op 2020-10-05 om 20:45 schreef Ryan Joseph via fpc-pascal: I often need to use a function which reads a file into a string, as is so common in so many scripting languages. Can it be considered to add something like this to the RTL? Since we have a refcounted Ansistring type it's natural for this to be a one-liner. Not saying we should use TStringList but that's the closest thing I found in the RTL. function ReadFile(path: Ansistring): Ansistring; var list: TStringList; begin list := TStringList.Create; list.LoadFromFile(ExpandFileName(path)); result := list.Text; list.Free; end; Stringlist static method loadstringfromfile ? ___ fpc-pascal maillist - fpc-pascal@lists.freepascal.org https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Re: [fpc-pascal] Adding file to string to the RTL
I use this function to read a file into a string:
function ReadFile(const FileName: TFileName): String;
var
InputFile: THandle;
FileSize, BytesRead: Integer;
Buffer: String='';
begin
try
InputFile := FileOpen(FileName, fmOpenRead);
if InputFile = -1 then
begin
WriteLn(Format('Error opening file "%s".', [FileName]));
Halt; // Exit?
end;
FileSize := FileSeek(InputFile, 0, fsFromEnd);
SetLength(Buffer, FileSize);
FileSeek(InputFile, 0, fsFromBeginning);
BytesRead := FileRead(InputFile, Buffer[1], FileSize);
if BytesRead < FileSize then
begin
WriteLn(Format('Error reading file "%s".', [FileName]));
Halt; // Exit?
end;
Result := Buffer;
finally
FileClose(InputFile);
end;
end;
On 6 Oct 2020, at 09:25, Luca Olivetti via fpc-pascal
wrote:
El 6/10/20 a les 9:01, Michael Van Canneyt via fpc-pascal ha escrit:
> A simple filecreate, allocate buffer, fileread, fileclose will probably be
> easiest.
Lazarus has a ReadFileToString in fileutil.
Bye
--
Luca
___
fpc-pascal maillist - fpc-pascal@lists.freepascal.org
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
___
fpc-pascal maillist - fpc-pascal@lists.freepascal.org
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Re: [fpc-pascal] Adding file to string to the RTL
El 6/10/20 a les 9:01, Michael Van Canneyt via fpc-pascal ha escrit: A simple filecreate, allocate buffer, fileread, fileclose will probably be easiest. Lazarus has a ReadFileToString in fileutil. Bye -- Luca ___ fpc-pascal maillist - fpc-pascal@lists.freepascal.org https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Re: [fpc-pascal] Adding file to string to the RTL
On Mon, 5 Oct 2020, Ryan Joseph via fpc-pascal wrote: On Oct 5, 2020, at 3:23 PM, Michael Van Canneyt via fpc-pascal wrote: So I think you're looking at 6 or even 8 versions of this hypothetical function... Ouch. :) I'm sure this code already exists in the RTL though, right? I assume it's part of some classes like TStringList and we can just pull it up into a collection of functions which are easily searchable in the RTL docs. Using TStringList would be a mistake, since it will parse your file while there is no need. No, I think that if you want this, it must be built from the basic calls. If not what is lowest level cross-platform layer which already exists and can we build up from? Maybe the "TextFile" type helps us? (I know it's ancient and I've never used it before honestly). I don't think that would be a good strategy. A simple filecreate, allocate buffer, fileread, fileclose will probably be easiest. Michael. ___ fpc-pascal maillist - fpc-pascal@lists.freepascal.org https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
