On Thu, Jul 19, 2007 at 12:35:52AM +0300, Dov Feldstern wrote:
> Dov Feldstern wrote:
> > José Matos wrote:
> >> On Wednesday 18 July 2007 01:21:52 Dov Feldstern wrote:
> >>> Also, this has only convinced me even more that it's wrong to apply a
> >>> lyx2lyx fix for this patch: this is a bug fix, there were previously
> >>> .lyx files which were ok, but which lyx was generating incorrect latex
> >>> for, and patch 1820 fixes that. If there are lyx documents out there
> >>> which patch 1820 affects, it can only do them good. All this lyx2lyx
> >>> patch does is to explicitly set the language to the wrong values in all
> >>> kinds of situations.
> >>
> >> OK you convinced me. I will accept the fix to bug 1820 if you add a
> >> comment in release notes. Do we have a deal?
> >>
> > Deal. What exactly should the comment say --- just explain where the
> > problems were, and that they are now hopefully fixed?
> > I actually like Martin's idea of just having a placeholder format change to
> > go along with this --- just in case we do ever decide that some kind of
> > format change is needed, so we can identify what is before and what is
> > after the fix was applied. Does this sound right?
> >> BTW in terms of final output what could it be the difference with and
> >> without this patch applied? Was the previous output always wrong to RTL
> >> users and so people learned to avoid it?
> >>
> >> This type of answer would justify our response to this bug.
> > As far as I can determine, there are two main changes:
> > 1. In certain specific cases, the text inside a footnote is displayed in
> > the wrong language (i.e., in the GUI it shows that it's in Hebrew, but is
> > printed in latex in English characters and in LTR, or vice-versa).
> > 2. A footnote breaks the RTL sequence, so instead of logical
> > {A...M[1]N...Z} being displayed as {Z...N[1]M...A}, it's displayed as
> > {M...A}[1]{Z...N}.
> > Regarding case 1, I doubt that many users were actually creating documents
> > which included these situations. If there are such documents, then I would
> > guess that it's for one of the following reasons:
> > (a) the user did not notice that the latex was generated incorrectly (in
> > the GUI it displays correctly, and maybe the user didn't go over the latex
> > output carefully --- after all, it's only a footnote...);
> > (b) the user knew about this, but assumed that someday the bug would be
> > corrected;
> > (c) the user was generating the document for a non-latex backend, which
> > generates correct output in these situations (I don't really know how the
> > other backends deal with these situations; plaintext output is correct, as
> > far as I can tell).
> > In all these cases, I think that including the lyx2lyx patch would only be
> > detrimental.
> > Regarding case 2, a user could conceivably realize what was happening, and
> > therefore decide to write the text in the wrong logical order:
> > {N...Z[1]A...M}, so that it would be displayed correctly as
> > {Z...N}[1]{M...A}. In this case, they *would* want a lyx2lyx fix. However,
> > creating documents that are logically wrong is bad practice, and could lead
> > to other problems. For example, if such text is broken by a line-break, it
> > gets totally mangled... So it's not a good situation to have gotten into in
> > the first place, and I would argue that we should not perpetuate it by
> > providing a "fix" for it.
> > Regarding non-RTL users, (2) doesn't affect them so much, but (1) does.
> > However, if the languages being mixed are same-alphabet languages (e.g.,
> > European languages), then they may not even notice something is wrong,
> > because it would display more-or-less correctly. So in this case, I don't
> > think that the fix would make much of a difference, nor would the lyx2lyx
> > patch. But I'm not sure about this.
> > So, what's the upshot?
> > 1. Can I commit the patch for 1820 (the patch itself, the cleaned-up
> > version at http://permalink.gmane.org/gmane.editors.lyx.devel/90016)?
> > 2. I provide a comment explaining this (I'll try to make it as shrt as
> > possible) for the release notes.
> > 3. Preferably, also add a placeholder format change to go along with it,
> > but no lyx2lyx for the moment?
> > Thanks!
> > Dov
>
> Attached find a patch which includes:
>
> - a placeholder format change for bug 1820
> - a comment for the release notes explaining the situation
>
> The release notes is done against Jose's patch for them, and also includes
> some minor fixes to that. The only significant change is that I changed the
> "please report to the users list" to "developers list" (I think Abdel also
> suggested that change).
>
> Jose --- can I commit the patch itself, along with these additions?
>
> Dov
I like this solution. Indeed, a behaviour of generating
malformed documents should not be carried forward by lyx2lyx,
as no user can reasonably have relied on it.
- Martin
> Index: lyx-devel/src/Buffer.cpp
> ===================================================================
> --- lyx-devel.orig/src/Buffer.cpp 2007-07-18 23:55:43.000000000 +0300
> +++ lyx-devel/src/Buffer.cpp 2007-07-19 00:01:26.000000000 +0300
> @@ -141,7 +141,7 @@
>
> namespace {
>
> -int const LYX_FORMAT = 276;
> +int const LYX_FORMAT = 277;
>
> } // namespace anon
>
> Index: lyx-devel/development/FORMAT
> ===================================================================
> --- lyx-devel.orig/development/FORMAT 2007-07-18 23:55:43.000000000 +0300
> +++ lyx-devel/development/FORMAT 2007-07-19 00:01:26.000000000 +0300
> @@ -1,6 +1,21 @@
> LyX file-format changes
> -----------------------
>
> +2007-07-19 Dov Feldstern <[EMAIL PROTECTED]>
> +
> + * format incremented to 277: this is just a placeholder format change,
> + to accompany the fix for bug 1820. After much deliberation, we
> have
> + decided that a lyx2lyx patch will not be provided: since LyX's
> old
> + interpretation of the situations fixed by this patch is wrong,
> creating
> + a lyx2lyx patch would mean intentionally creating .lyx files
> which
> + are wrong (e.g., display text in one language, but marked as
> belonging
> + to a different language).
> + However, just in case someone may want to be able to preserve
> the old
> + formatting, we are upgrading the version. An initial lyx2lyx
> patch
> + for this change is actually already available on the newsgroup
> + (http://permalink.gmane.org/gmane.editors.lyx.devel/90061), but
> it
> + should be applied with care, as it may corrupt some files!
> +
> 2007-06-26 Uwe Stöhr <[EMAIL PROTECTED]> and Dov Feldstern <[EMAIL
> PROTECTED]>
>
> * format incremented to 276: switching exsting language 'arabic' to
> Index: lyx-devel/lib/lyx2lyx/LyX.py
> ===================================================================
> --- lyx-devel.orig/lib/lyx2lyx/LyX.py 2007-07-18 23:55:43.000000000 +0300
> +++ lyx-devel/lib/lyx2lyx/LyX.py 2007-07-19 00:01:26.000000000 +0300
> @@ -77,7 +77,7 @@
> ("1_2", [220], generate_minor_versions("1.2" , 4)),
> ("1_3", [221], generate_minor_versions("1.3" , 7)),
> ("1_4", range(222,246), generate_minor_versions("1.4" ,
> 4)),
> - ("1_5", range(246,277), generate_minor_versions("1.5" ,
> 0))]
> + ("1_5", range(246,278), generate_minor_versions("1.5" ,
> 0))]
>
>
> def formats_list():
> Index: lyx-devel/RELEASE-NOTES
> ===================================================================
> --- lyx-devel.orig/RELEASE-NOTES 2007-07-19 00:01:39.000000000 +0300
> +++ lyx-devel/RELEASE-NOTES 2007-07-19 00:28:09.000000000 +0300
> @@ -37,14 +37,30 @@
>
> - Reverting 1.5 LyX documents to previous versions
>
> -Since unicode is supported in version 1.5 the dowgrade option to 1.4
> +Since unicode is supported in version 1.5 the downgrade option to 1.4
> and 1.3 lyx documents may be lossy. In those cases where no meaningful
> -convertion was possible the characters will be replaced by a
> +conversion was possible the characters will be replaced by a
> placeholder ???.
>
> -We have done our best effort to make the transition the smoothest
> -possible, if problems remain please report them to the users mailing
> -list.
> +We have done our best effort to make the transition as smooth possible,
> +if problems remain please report them to the developers mailing list.
> +
> +- Languages/encodings and insets
> +
> +One of the bugs fixed in LyX 1.5.0 is that previously, there were certain
> +specific cases in which the LaTeX generated did not correctly reflect
> +language/encoding transitions in and around insets (footnotes, LyX notes).
> +After much deliberation, it was decided not to change older files such that
> +they will still reflect the old LaTeX output; rather, they will now correctly
> +reflect the situation as it appears in the GUI. This means, however, that if
> +you mangled the text in the GUI in the older versions, in order that it
> +generate the correct LaTeX output, the LaTeX will now generate the mangled
> +text. If this is problematic for you, please get in touch with us on the
> +developers mailing list, we do have some possible solutions for this.
> +
> +The effects of this will be more pronounced for RTL (Hebrew, Arabic, Farsi)
> +users --- though they affect users of other languages as well.
> +
>
> Note: There may later be an updated list of known issues online at
> http://wiki.lyx.org/LyX/ReleaseNotes
> Index: lyx-devel/lib/lyx2lyx/lyx_1_5.py
> ===================================================================
> --- lyx-devel.orig/lib/lyx2lyx/lyx_1_5.py 2007-07-19 00:28:42.000000000
> +0300
> +++ lyx-devel/lib/lyx2lyx/lyx_1_5.py 2007-07-19 00:29:29.000000000 +0300
> @@ -2026,10 +2026,12 @@
> [273, []],
> [274, [normalize_font_whitespace_274]],
> [275, [convert_graphics_rotation]],
> - [276, [convert_arabic]]
> + [276, [convert_arabic]],
> + [277, []],
> ]
>
> revert = [
> + [276, []],
> [275, [revert_arabic]],
> [274, [revert_graphics_rotation]],
> [273, []],
