[Freeciv-Dev] [patch #2843] Make secfile reading cope with \r\n and \n\r line endings on all platforms

[Jacob Nevins]

Update of patch #2843 (project freeciv):

  Status:  Ready For Test => Done   
 Open/Closed:Open => Closed 
 Summary: Make secfile reading cope with \r\n and \n\r line
endings on all platforms => Make secfile reading cope with rn and nr line
endings on all platforms


___

Reply to this item at:

  

___
  Message sent via/by Gna!
  http://gna.org/


___
Freeciv-dev mailing list
Freeciv-dev@gna.org
https://mail.gna.org/listinfo/freeciv-dev

Re: [Freeciv-Dev] [patch #2843] Make secfile reading cope with \r\n and \n\r line endings on all platforms

[Marko Lindqvist]

On 30 July 2011 17:25, Jacob Nevins
<0jacobnk.fc...@chiark.greenend.org.uk> wrote:
>>  btw. I plan to make only builds like these (freeciv in one zip) in
>> the future. As you are the only one who has ever successfully used
>> these builds, would you mind writing short instructions how you setup
>> your system, including l10n stuff? I could only give untested theories
>> about the setup myself. Forum thread about my builds is one place
>> where such instructions should exist, so feel free to comment there
>> directly.
>
> Sure. Do you yet have a public example of such a build that I can cite?

 2.3.0-RC2 build will probably be the first one, unless official build
by cproc turns up before I've made mine.


 - ML

___
Freeciv-dev mailing list
Freeciv-dev@gna.org
https://mail.gna.org/listinfo/freeciv-dev

Re: [Freeciv-Dev] [patch #2843] Make secfile reading cope with \r\n and \n\r line endings on all platforms

[Jacob Nevins]

>>> Did this build also contain the uncommitted patch for bug #18396 (the
>>> bzip2 one, file #13687)?
>>
>>  No, I considered having them both patches in that build, but thought
>> that you probably want to test effects of single patch without
>> worrying what the other patch does to results. I can make another
>> build with both patches in, of course, if you want to test that.
> 
>  Uploading...

OK, will try this next time I'm in Windows.
(Feel free to delete the previous build from cazfi.net, BTW; I have a
copy.)

>  btw. I plan to make only builds like these (freeciv in one zip) in
> the future. As you are the only one who has ever successfully used
> these builds, would you mind writing short instructions how you setup
> your system, including l10n stuff? I could only give untested theories
> about the setup myself. Forum thread about my builds is one place
> where such instructions should exist, so feel free to comment there
> directly.

Sure. Do you yet have a public example of such a build that I can cite?

___
Freeciv-dev mailing list
Freeciv-dev@gna.org
https://mail.gna.org/listinfo/freeciv-dev

Re: [Freeciv-Dev] [patch #2843] Make secfile reading cope with \r\n and \n\r line endings on all platforms

[Marko Lindqvist]

On 30 July 2011 17:06, Marko Lindqvist  wrote:
> On 30 July 2011 16:53, Jacob Nevins
> <0jacobnk.fc...@chiark.greenend.org.uk> wrote:
>> Marko Lindqvist writes:
>>> On 29 July 2011 02:30, Jacob Nevins  
>>> wrote:
 Attached patch makes secfile reading treat \r\n and \n\r line endings the
 same as \n, if it sees them (e.g., if opening a file with DOS/Windows line
 endings on Unix).
>>>
>>>  I don't know if there's anything in this patch making Windows testing
>>> necessary, but I made a Windows build with it regardless for your
>>> testing pleasure.
>>>
>>>  http://www.cazfi.net/~cazfi/fcwintest/freeciv-2.3.0-RC1+-r20033.zip
>>
>> Thanks, that was useful. Detailed results in bug digest.
>>
>> Did this build also contain the uncommitted patch for bug #18396 (the
>> bzip2 one, file #13687)? This presumably affects whether the files are
>> opened in text or binary mode (although I haven't checked thoroughly).
>> I don't think it should materially affect the conclusions, but it would
>> be good to know.
>
>  No, I considered having them both patches in that build, but thought
> that you probably want to test effects of single patch without
> worrying what the other patch does to results. I can make another
> build with both patches in, of course, if you want to test that.

 Uploading...

 btw. I plan to make only builds like these (freeciv in one zip) in
the future. As you are the only one who has ever successfully used
these builds, would you mind writing short instructions how you setup
your system, including l10n stuff? I could only give untested theories
about the setup myself. Forum thread about my builds is one place
where such instructions should exist, so feel free to comment there
directly.


 - ML

___
Freeciv-dev mailing list
Freeciv-dev@gna.org
https://mail.gna.org/listinfo/freeciv-dev

Re: [Freeciv-Dev] [patch #2843] Make secfile reading cope with \r\n and \n\r line endings on all platforms

[Marko Lindqvist]

On 30 July 2011 16:53, Jacob Nevins
<0jacobnk.fc...@chiark.greenend.org.uk> wrote:
> Marko Lindqvist writes:
>> On 29 July 2011 02:30, Jacob Nevins  wrote:
>>> Attached patch makes secfile reading treat \r\n and \n\r line endings the
>>> same as \n, if it sees them (e.g., if opening a file with DOS/Windows line
>>> endings on Unix).
>>
>>  I don't know if there's anything in this patch making Windows testing
>> necessary, but I made a Windows build with it regardless for your
>> testing pleasure.
>>
>>  http://www.cazfi.net/~cazfi/fcwintest/freeciv-2.3.0-RC1+-r20033.zip
>
> Thanks, that was useful. Detailed results in bug digest.
>
> Did this build also contain the uncommitted patch for bug #18396 (the
> bzip2 one, file #13687)? This presumably affects whether the files are
> opened in text or binary mode (although I haven't checked thoroughly).
> I don't think it should materially affect the conclusions, but it would
> be good to know.

 No, I considered having them both patches in that build, but thought
that you probably want to test effects of single patch without
worrying what the other patch does to results. I can make another
build with both patches in, of course, if you want to test that.


 - ML

___
Freeciv-dev mailing list
Freeciv-dev@gna.org
https://mail.gna.org/listinfo/freeciv-dev

Re: [Freeciv-Dev] [patch #2843] Make secfile reading cope with \r\n and \n\r line endings on all platforms

[Jacob Nevins]

Marko Lindqvist writes:
> On 29 July 2011 02:30, Jacob Nevins  wrote:
>> Attached patch makes secfile reading treat \r\n and \n\r line endings the
>> same as \n, if it sees them (e.g., if opening a file with DOS/Windows line
>> endings on Unix).
> 
>  I don't know if there's anything in this patch making Windows testing
> necessary, but I made a Windows build with it regardless for your
> testing pleasure.
> 
>  http://www.cazfi.net/~cazfi/fcwintest/freeciv-2.3.0-RC1+-r20033.zip

Thanks, that was useful. Detailed results in bug digest.

Did this build also contain the uncommitted patch for bug #18396 (the
bzip2 one, file #13687)? This presumably affects whether the files are
opened in text or binary mode (although I haven't checked thoroughly).
I don't think it should materially affect the conclusions, but it would
be good to know.

___
Freeciv-dev mailing list
Freeciv-dev@gna.org
https://mail.gna.org/listinfo/freeciv-dev

[Freeciv-Dev] [patch #2843] Make secfile reading cope with \r\n and \n\r line endings on all platforms

[Jacob Nevins]

Update of patch #2843 (project freeciv):

 Planned Release: => 2.3.1,2.4.0
 Summary: Make secfile reading cope with \r\n and \n\r line
endings on all platforms => Make secfile reading cope with rn and nr line
endings on all platforms

___

Follow-up Comment #1:

After some experimentation, I believe the principle behind this patch is safe
for Windows.

Compiling and running the attached test program for Windows (with MinGW,
which I believe will use MSVCRT for the standard library) suggests the
following facts about fgets() on Windows:
* When reading a DOS file (0D 0A) in text mode, it treats 0D 0A as the line
ending character, and 0A is what ends up at the end of the fetched string (all
as expected/standard)
* When reading a DOS file in _binary_ mode, it treats _0A_ as the line ending
character, so the strings returned by fgets() are XX XX... 0D 0A
* When reading a Unix file (0A), it breaks lines at 0A; what you see doesn't
depend on whether it was opened in binary mode, of course. (So Windows fgets()
has no trouble reading Unix files; I don't know why Notepad can't cope,
grumble grumble.)
* fgets() doesn't treat 0A 0D as a valid line ending; the 0Ds leak into the
string even in text mode. (I've no reason to believe such files are common,
I'm just handling them for completeness.)
* fgets() entirely fails to spot old-Mac line endings (0D), treating the file
as one long line. (But we established in bug #18396 that that's irrelevant
these days.)

So, I believe that Windows and Unix builds with this patch will be able to
cope with each of Windows and Unix files, regardless of whether the file is
opened in text or binary mode. (This may enable more files to be
gzipped/bzipped/etc, as I suspect that any compressed file read is effectively
in binary mode.)

Separately, testing an S2_3 Windows build from cazfi with this patch, the
resulting freeciv-gtk2 coped with helpdata.txt regardless of whether it had
Windows or Unix line endings, with localisation working correctly in both
cases. Other secfile reading such as rulesets seemed to work too (these files
would have had Unix line endings). I don't actually know whether the files
were being opened in binary or text mode, but from the above, it doesn't
matter.

(file #13697)
___

Additional Item Attachment:

File name: fgets.cSize:0 KB


___

Reply to this item at:

  

___
  Message sent via/by Gna!
  http://gna.org/


___
Freeciv-dev mailing list
Freeciv-dev@gna.org
https://mail.gna.org/listinfo/freeciv-dev

Re: [Freeciv-Dev] [patch #2843] Make secfile reading cope with \r\n and \n\r line endings on all platforms

[Marko Lindqvist]

On 29 July 2011 02:30, Jacob Nevins  wrote:
>
> Attached patch makes secfile reading treat \r\n and \n\r line endings the
> same as \n, if it sees them (e.g., if opening a file with DOS/Windows line
> endings on Unix).
>
> Tested on Unix. I haven't checked what happens with a Windows build of
> Freeciv opening a Unix file in text mode; it depends if fgets() treats the
> lone \n as a line ending.
>
> This probably wants some testing...

 I don't know if there's anything in this patch making Windows testing
necessary, but I made a Windows build with it regardless for your
testing pleasure.

 http://www.cazfi.net/~cazfi/fcwintest/freeciv-2.3.0-RC1+-r20033.zip


 - ML

___
Freeciv-dev mailing list
Freeciv-dev@gna.org
https://mail.gna.org/listinfo/freeciv-dev

[Freeciv-Dev] [patch #2843] Make secfile reading cope with \r\n and \n\r line endings on all platforms

[Jacob Nevins]

URL:
  

 Summary: Make secfile reading cope with \r\n and \n\r line
endings on all platforms
 Project: Freeciv
Submitted by: jtn
Submitted on: Fri Jul 29 00:30:49 2011
Category: general
Priority: 5 - Normal
  Status: Ready For Test
 Privacy: Public
 Assigned to: jtn
Originator Email: 
 Open/Closed: Open
 Discussion Lock: Any
 Planned Release: 

___

Details:

Following on from bug #18396:

Attached patch makes secfile reading treat \r\n and \n\r line endings the
same as \n, if it sees them (e.g., if opening a file with DOS/Windows line
endings on Unix).

Tested on Unix. I haven't checked what happens with a Windows build of
Freeciv opening a Unix file in text mode; it depends if fgets() treats the
lone \n as a line ending.

(I briefly wondered if there might be crazy-but-theoretically-supported
multibyte character encodings where 0x0D might be a valid thing to see at the
start or end of a line. But any such encoding was pretty much doomed anyway if
opened in text mode. Certainly UTF-8 doesn't fall into this category.)

This probably wants some testing...



___

File Attachments:


---
Date: Fri Jul 29 00:30:49 2011  Name: trunk-S2_3-secfile-crlf.diff  Size: 1kB
  By: jtn



___

Reply to this item at:

  

___
  Message sent via/by Gna!
  http://gna.org/


___
Freeciv-dev mailing list
Freeciv-dev@gna.org
https://mail.gna.org/listinfo/freeciv-dev