Re: dash documentation patch
On Sat, Sep 02, 2017 at 09:19:50PM +, Guenter Milde wrote: > On 2017-09-02, Scott Kostyshak wrote: > > In the RELEASE-NOTES file, we could reference > > https://wiki.lyx.org/LyX/ReleaseNotes for release notes relating to > > previous versions of LyX. Actually, I need to update that Wiki page. > > e.g. we could put "If upgrading from a pre-2.2 LyX version, please see > > https://wiki.lyx.org/LyX/ReleaseNotes;, > A link to the ReleaseNotes wiki page is still a good idea, as we do not > ship a "Changelog" file. Done in master at 0bf31368, and 2.3.x at fa32ad6a. Scott signature.asc Description: PGP signature
Re: dash documentation patch
On Mon, Sep 11, 2017 at 10:53:27PM -0400, Scott Kostyshak wrote: > On Wed, Sep 06, 2017 at 07:14:26PM +, Guenter Milde wrote: > > On 2017-08-31, Guenter Milde wrote: > > > > Dear LyX developers, > > > > is there a chance to fix the following problems instead of just > > documenting them? > > Fine with me as long as you get a +1. Has anyone taken a look at the patch? Scott signature.asc Description: PGP signature
Re: dash documentation patch
On Mon, Sep 04, 2017 at 06:55:57AM +, Guenter Milde wrote: > Dear Scot, dear Docutils developers, > > On 2017-09-04, Scott Kostyshak wrote: > > > >> The problem is, however, that users upgrading from 2.2 have read the > >> 2.2.0 RELEASE NOTES already and won't see the need to re-read them, > > > Agreed. > > >> thus missing the information. Nevertheless, they are still affected by > >> the changes when opening pre-2.2 documents (and will forever, as we > >> voted not to revert the decision to merge literal and ligature dashes > >> in the LyX source). > > > You are thinking maybe of a user who created their document with 2.1.x, > > edited it briefly in 2.2.x but not enough to realize a change/problem, > > and now are upgrading to 2.3.0 and will be reading the release notes? > > I am looking for the right place of a warning to all users of LyX 2.2 and > newer, as they may see problems with dashes in documents created with > version up to 2.1. > > > I think we can address it with e.g. a concise "if you created the first > > version of your document in LyX pre-2.2, your dashes may have been > > changed. For more information, please see..." I think that makes more > > sense that putting a lot of details about this issue in the 2.3.0 > > release notes. I think that the important thing is to let the user know > > the conditions under which they might experience a caveat. > > > I don't really have a strong opinion on this, but I think a concise > > description of the conditions for a caveat, combined with a reference > > for further information, could be a reasonable compromise. > > Agreed. I will prepare a patch for review. Günter sent me a patch privately since he cannot post to the list. It is attached. I reviewed the changes to RELEASE-NOTES and they look good to me. The changes relating to dashes are concise and clear. There was an objection about the ZWSP entry as being too technical, but I personally don't think that's a problem: We should try to make the notes user-friendly, but some notes are hard to make more user-friendly. And since this involves potential data loss, I think it does belong in the RELEASE-NOTES. I would remove the exclamation mark, but I recognize that is just personal taste. Any final thoughts before Günter commits? Note that Günter has a pending proposal to fix the ZWSP issue, in which case the note regarding it would not be needed. But that proposal would need a +1 from another developer. To review it, please see: https://www.mail-archive.com/search?l=mid=oophei%246e1%241%40blaine.gmane.org Scott From 3684015c5cf180cb3fd4229e45171ecc9658754d Mon Sep 17 00:00:00 2001 From: Günter Milde Date: Tue, 29 Aug 2017 21:52:31 +0200 Subject: [PATCH] Update documentation regarding em- and en-dashes. --- lib/RELEASE-NOTES | 22 +- lib/doc/UserGuide.lyx | 1083 --- 2 files changed, 1077 insertions(+), 28 deletions(-) diff --git a/lib/RELEASE-NOTES b/lib/RELEASE-NOTES index 591fcae..9b91f00 100644 --- a/lib/RELEASE-NOTES +++ b/lib/RELEASE-NOTES @@ -13,11 +13,13 @@ be safely dissolved, as it will be automatically inserted at export time if needed, as usual. -* LyX now outputs en- and em-dashes as -- and --- ligatures when exporting to - latex using TeX fonts, as done in version 2.1 and earlier. In version 2.2 - they were instead output as the macros \textendash and \textemdash, causing - changed output with old documents and bugs. The 2.2 behavior can be restored - by don't allowing using dash ligatures in Document->Settings->Fonts. +* The new setting + "Document->Settings->Fonts->Output em- and en-dash as ligatures" forces + output of en- and em-dashes as -- and --- when exporting to LaTeX. + It is is "true" by default but "false" when opening documents edited + with LyX 2.2. + See chapter 3.9.1.1 "Dashes and line breaks" of the User Guide and + "Caveats when upgrading
Re: dash documentation patch
On Wed, Sep 06, 2017 at 07:14:26PM +, Guenter Milde wrote: > On 2017-08-31, Guenter Milde wrote: > > Dear LyX developers, > > is there a chance to fix the following problems instead of just > documenting them? Fine with me as long as you get a +1. Scott signature.asc Description: PGP signature
Re: dash documentation patch
On 2017-08-31, Guenter Milde wrote:
Dear LyX developers,
is there a chance to fix the following problems instead of just
documenting them?
> lyx2lyx deletes ZWSP characters following literal em- and en-dashes when
> converting to 2.3 format. If you used literal ZWSP characters (u200b) as
> optional line breaks after dashes, convert them to 0dd wide space insets
> before opening your document with LyX 2.3 or the optional line breaks will
> be lost!
...
> If using TeX fonts and en- and em-dashes are output as font ligatures,
> when exporting documents containing en- and em-dashes to the format of
> LyX 2.0 or earlier, the following line has to be manually added to the
> unicodesymbols file of that LyX version:
> 0x200b "\\hspace{0pt}" "" "" "" "" # ZERO WIDTH SPACE
> This avoids "uncodable character" issues if the document is actually
> loaded by that LyX version. LyX 2.1 and later versions already have the
> necessary definition in their unicodesymbols file
Export to 2.1 and older changes dashes from ligature to literal.
Proposed changes (see patch below):
1. detect presence of literal as well as ligature dashes in old documents and
use it for setting \use_dash_ligatures:
\use_dash_ligatures = default if none are found
\use_dash_ligatures = Trueif ligature dash(s) (\t*hyphens) found
\use_dash_ligatures = False if literal dash(s) found
\use_dash_ligatures = default if both are found (+ issue a warning)
2. back-convert dashes to \twohyphens / \threehyphens if
\use_dash_ligatures = True
This means
* unchanged behaviour with LyX <= 2.1
* unchanged behaviour after round-trip (unless edited with 2.2)
3. do not use invisible optional line break characters (ZWSP) in backwards
conversion.
Günter
diff --git a/lib/lyx2lyx/lyx_2_3.py b/lib/lyx2lyx/lyx_2_3.py
index 73ac45cf00..6305d417b0 100644
--- a/lib/lyx2lyx/lyx_2_3.py
+++ b/lib/lyx2lyx/lyx_2_3.py
@@ -1841,103 +1841,57 @@ def revert_chapterbib(document):
def convert_dashligatures(document):
-" Remove a zero-length space (U+200B) after en- and em-dashes. "
-
+" Set use_dash_ligatures according to content (literal vs. 'ligature'
dashes) "
+# Default:
+use_dash_ligatures = False # TODO: Get the default from stdtemplate.lyx
+# Look for dashes (followed by a word or no-break space):
+# (Documents by LyX 2.1 or older have "\twohyphens\n" or "\threehyphens\n"
+# as interim representation for dash ligatures in 2.2.)
+has_literal_dashes = has_ligature_dashes = False
+for i, line in enumerate(document.body):
+if re.search(u"[\u2013\u2014]([\w\u00A0]|$)", line, flags=re.UNICODE):
+has_literal_dashes = True
+if re.search(ur"(\\twohyphens|\\threehyphens)", line,
flags=re.UNICODE):
+# print "dash in line ", i, document.body[i+1].encode('utf8')
+if re.match(u"[\w\u00A0]", document.body[i+1], flags=re.UNICODE):
+has_ligature_dashes = True
+if has_literal_dashes and has_ligature_dashes:
+# TODO: insert a warning note in the document?
+document.warning("""This document contained both literal and
"ligature" dashes.
+Line break may have changed. See UserGuide chapter 3.9.1 for
details.""")
+elif has_literal_dashes:
+# print "has literal dashes"
+use_dash_ligatures = False
+elif has_ligature_dashes:
+# print "has ligature dashes"
+use_dash_ligatures = True
+# insert the setting
i = find_token(document.header, "\\use_microtype", 0)
if i != -1:
-if document.initial_format > 474 and document.initial_format < 509:
-# This was created by LyX 2.2
-document.header[i+1:i+1] = ["\\use_dash_ligatures false"]
-else:
-# This was created by LyX 2.1 or earlier
-document.header[i+1:i+1] = ["\\use_dash_ligatures true"]
-
-i = 0
-while i < len(document.body):
-words = document.body[i].split()
-# Skip some document parts where dashes are not converted
-if len(words) > 1 and words[0] == "\\begin_inset" and \
- words[1] in ["CommandInset", "ERT", "External", "Formula", \
-"FormulaMacro", "Graphics", "IPA", "listings"]:
-j = find_end_of_inset(document.body, i)
-if j == -1:
-document.warning("Malformed LyX document: Can't find end of " \
- + words[1] + " inset at line " + str(i))
-i += 1
-else:
-i = j
-continue
-if len(words) > 0 and words[0] in ["\\leftindent", \
-"\\paragraph_spacing", "\\align", "\\labelwidthstring"]:
-i += 1
-continue
-
-start = 0
-while True:
-j = document.body[i].find(u"\u2013", start) # en-dash
-k = document.body[i].find(u"\u2014",
Re: dash documentation patch
Dear Scot, dear Docutils developers, On 2017-09-04, Scott Kostyshak wrote: >> The problem is, however, that users upgrading from 2.2 have read the >> 2.2.0 RELEASE NOTES already and won't see the need to re-read them, > Agreed. >> thus missing the information. Nevertheless, they are still affected by >> the changes when opening pre-2.2 documents (and will forever, as we >> voted not to revert the decision to merge literal and ligature dashes >> in the LyX source). > You are thinking maybe of a user who created their document with 2.1.x, > edited it briefly in 2.2.x but not enough to realize a change/problem, > and now are upgrading to 2.3.0 and will be reading the release notes? I am looking for the right place of a warning to all users of LyX 2.2 and newer, as they may see problems with dashes in documents created with version up to 2.1. > I think we can address it with e.g. a concise "if you created the first > version of your document in LyX pre-2.2, your dashes may have been > changed. For more information, please see..." I think that makes more > sense that putting a lot of details about this issue in the 2.3.0 > release notes. I think that the important thing is to let the user know > the conditions under which they might experience a caveat. > I don't really have a strong opinion on this, but I think a concise > description of the conditions for a caveat, combined with a reference > for further information, could be a reasonable compromise. Agreed. I will prepare a patch for review. Günter
Re: dash documentation patch
On Sat, Sep 02, 2017 at 09:19:50PM +, Guenter Milde wrote: > Dear Scott, dear Enrico, dear lyx developers, > > On 2017-09-02, Scott Kostyshak wrote: > > On Sat, Sep 02, 2017 at 12:34:05AM +0200, Enrico Forestieri wrote: > >> On Thu, Aug 31, 2017 at 08:53:41PM +, Guenter Milde wrote: > > >> See above. Release notes should be used to illustrate current changes, not > >> changes occurred in past releases. > > > From what I understand, if we could go back in time (but could not fix > > anything in the code), we would add notes to release notes for the LyX > > 2.2.0 release. That is not possible. So the question is do we put them > > in the release notes for 2.3.0 or leave them absent from any release > > notes? > > We cannot go back in time, but git branches allow at least a fix: we > can amend the RELEASE NOTES in the 2.2.x branch. I'm not sure about this. I do not think users expect the release notes to change. > The question is, who is > going to read it? > > > I wonder if a similar situation has come up before and what we did. > > > It is true that we have a subsection "Caveats when upgrading from > > earlier versions to 2.3.x", and indeed 2.1.x is an "earlier version". > > However, the title of the file is "Important Changes in LyX 2.3.0", > > which in my opinion dominates any implication of a subtitle. > > Some consequences of the changes in 2.2.0 came only to our attention during > the 2.3 development cycle. IMO, it is a pragmatic solution to add this > insight in the 2.3 RELEASE NOTES. I see your point and agree that the information needs to go somewhere that could be accessed by users. > > One line of logic is: suppose the release notes for 2.2.0 were accurate. > > Do we really think that 2.1.x users would read both the release notes of > > 2.2.0 and of 2.3.0 before upgrading? Probably not. So the fact that the > > release notes for 2.2.0 are inaccurate might not be so relevant. This is > > all just guessing about user-behavior on my part though. And I must > > admit that when I personally upgrade software that's important to me, I > > would read all of the release notes in-between. > > > I think the best solution might be the following: > > > In the RELEASE-NOTES file, we could reference > > https://wiki.lyx.org/LyX/ReleaseNotes for release notes relating to > > previous versions of LyX. Actually, I need to update that Wiki page. > > e.g. we could put "If upgrading from a pre-2.2 LyX version, please see > > https://wiki.lyx.org/LyX/ReleaseNotes;, and on that Wiki page we can > > list both the original release notes as well as any "post 2.2-release > > addendums", where issues such as what Günter details could be listed. > > If it is decided that this proposal is too complicated, we could > > perhaps include a very concise note in the 2.3 release notes along the > > lines of "if upgrading from pre-2.2, please read for > > considerations related to dashes". > > The problem is, however, that users upgrading from 2.2 have read the > 2.2.0 RELEASE NOTES already and won't see the need to re-read them, Agreed. > thus > missing the information. Nevertheless, they are still affected by the > changes when opening pre-2.2 documents (and will forever, as we voted not > to revert the decision to merge literal and ligature dashes in the LyX > source). You are thinking maybe of a user who created their document with 2.1.x, edited it briefly in 2.2.x but not enough to realize a change/problem, and now are upgrading to 2.3.0 and will be reading the release notes? I think we can address it with e.g. a concise "if you created the first version of your document in LyX pre-2.2, your dashes may have been changed. For more information, please see..." I think that makes more sense that putting a lot of details about this issue in the 2.3.0 release notes. I think that the important thing is to let the user know the conditions under which they might experience a caveat. I don't really have a strong opinion on this, but I think a concise description of the conditions for a caveat, combined with a reference for further information, could be a reasonable compromise. Scott > A link to the ReleaseNotes wiki page is still a good idea, as we do not > ship a "Changelog" file. > > Günter > > signature.asc Description: PGP signature
Re: dash documentation patch
Dear Scott, dear Enrico, dear lyx developers, On 2017-09-02, Scott Kostyshak wrote: > On Sat, Sep 02, 2017 at 12:34:05AM +0200, Enrico Forestieri wrote: >> On Thu, Aug 31, 2017 at 08:53:41PM +, Guenter Milde wrote: >> See above. Release notes should be used to illustrate current changes, not >> changes occurred in past releases. > From what I understand, if we could go back in time (but could not fix > anything in the code), we would add notes to release notes for the LyX > 2.2.0 release. That is not possible. So the question is do we put them > in the release notes for 2.3.0 or leave them absent from any release > notes? We cannot go back in time, but git branches allow at least a fix: we can amend the RELEASE NOTES in the 2.2.x branch. The question is, who is going to read it? > I wonder if a similar situation has come up before and what we did. > It is true that we have a subsection "Caveats when upgrading from > earlier versions to 2.3.x", and indeed 2.1.x is an "earlier version". > However, the title of the file is "Important Changes in LyX 2.3.0", > which in my opinion dominates any implication of a subtitle. Some consequences of the changes in 2.2.0 came only to our attention during the 2.3 development cycle. IMO, it is a pragmatic solution to add this insight in the 2.3 RELEASE NOTES. > One line of logic is: suppose the release notes for 2.2.0 were accurate. > Do we really think that 2.1.x users would read both the release notes of > 2.2.0 and of 2.3.0 before upgrading? Probably not. So the fact that the > release notes for 2.2.0 are inaccurate might not be so relevant. This is > all just guessing about user-behavior on my part though. And I must > admit that when I personally upgrade software that's important to me, I > would read all of the release notes in-between. > I think the best solution might be the following: > In the RELEASE-NOTES file, we could reference > https://wiki.lyx.org/LyX/ReleaseNotes for release notes relating to > previous versions of LyX. Actually, I need to update that Wiki page. > e.g. we could put "If upgrading from a pre-2.2 LyX version, please see > https://wiki.lyx.org/LyX/ReleaseNotes;, and on that Wiki page we can > list both the original release notes as well as any "post 2.2-release > addendums", where issues such as what Günter details could be listed. > If it is decided that this proposal is too complicated, we could > perhaps include a very concise note in the 2.3 release notes along the > lines of "if upgrading from pre-2.2, please read for > considerations related to dashes". The problem is, however, that users upgrading from 2.2 have read the 2.2.0 RELEASE NOTES already and won't see the need to re-read them, thus missing the information. Nevertheless, they are still affected by the changes when opening pre-2.2 documents (and will forever, as we voted not to revert the decision to merge literal and ligature dashes in the LyX source). A link to the ReleaseNotes wiki page is still a good idea, as we do not ship a "Changelog" file. Günter
Re: dash documentation patch
On Sat, Sep 02, 2017 at 12:34:05AM +0200, Enrico Forestieri wrote: > On Thu, Aug 31, 2017 at 08:53:41PM +, Guenter Milde wrote: > > This also one of the caveats then upgrading from earlier versions (which are > > not limited to upgrading from 2.2). > > See above. Release notes should be used to illustrate current changes, not > changes occurred in past releases. From what I understand, if we could go back in time (but could not fix anything in the code), we would add notes to release notes for the LyX 2.2.0 release. That is not possible. So the question is do we put them in the release notes for 2.3.0 or leave them absent from any release notes? I wonder if a similar situation has come up before and what we did. It is true that we have a subsection "Caveats when upgrading from earlier versions to 2.3.x", and indeed 2.1.x is an "earlier version". However, the title of the file is "Important Changes in LyX 2.3.0", which in my opinion dominates any implication of a subtitle. One line of logic is: suppose the release notes for 2.2.0 were accurate. Do we really think that 2.1.x users would read both the release notes of 2.2.0 and of 2.3.0 before upgrading? Probably not. So the fact that the release notes for 2.2.0 are inaccurate might not be so relevant. This is all just guessing about user-behavior on my part though. And I must admit that when I personally upgrade software that's important to me, I would read all of the release notes in-between. I think the best solution might be the following: In the RELEASE-NOTES file, we could reference https://wiki.lyx.org/LyX/ReleaseNotes for release notes relating to previous versions of LyX. Actually, I need to update that Wiki page. e.g. we could put "If upgrading from a pre-2.2 LyX version, please see https://wiki.lyx.org/LyX/ReleaseNotes;, and on that Wiki page we can list both the original release notes as well as any "post 2.2-release addendums", where issues such as what Günter details could be listed. If it is decided that this proposal is too complicated, we could perhaps include a very concise note in the 2.3 release notes along the lines of "if upgrading from pre-2.2, please read for considerations related to dashes". Scott signature.asc Description: PGP signature
Re: dash documentation patch
On Thu, Aug 31, 2017 at 08:53:41PM +, Guenter Milde wrote:
> On 2017-08-31, Enrico Forestieri wrote:
> > On Wed, Aug 30, 2017 at 11:06:25PM +0200, Guenter Milde wrote:
>
> >> +* Since version 2.2, -- and --- in the LyX source are output as -{}- and
> >> + -{}-{}- to prevent conversion to en- and em-dashes by TeX.
> >> + Occurences in pre-2.2 documents are converted to literal Unicode dashes.
> >> + In some cases this leads to different line breaks, as:
> >> + + there is an optional line break after hyphens (also -- and ---) but
> >> not
> >> +after literal dashes, and
> >> + + hyphenation is suppressed in words following hyphens but allowed after
> >> +literal dashes.
>
> > I think the above does not belong to the release notes for 2.3. It
> > would have been appropriate for the 2.2 release notes. IMO, this has to
> > go in the docs.
>
> Description of past behaviour is IMO not out of place in a section called
> "!!Caveats when upgrading from earlier versions to 2.3.x".
It is out of place because this is not illustrating something introduced
in the current release but in past releases.
> >> + It is no longer possible to differentiate dashes with/without optional
> >> + line break using --- and -- vs. literal dashes. Either convert one sort
> >> to
> >> + ERT or insert optional line break characters.
>
> > This is true since 2.2, so this does not belong to the release notes for
> > 2.3 and is better explained in the docs.
>
> This also one of the caveats then upgrading from earlier versions (which are
> not limited to upgrading from 2.2).
See above. Release notes should be used to illustrate current changes, not
changes occurred in past releases.
--
Enrico
Re: dash documentation patch
On 2017-08-31, Enrico Forestieri wrote:
> On Wed, Aug 30, 2017 at 11:06:25PM +0200, Guenter Milde wrote:
>> +* Since version 2.2, -- and --- in the LyX source are output as -{}- and
>> + -{}-{}- to prevent conversion to en- and em-dashes by TeX.
>> + Occurences in pre-2.2 documents are converted to literal Unicode dashes.
>> + In some cases this leads to different line breaks, as:
>> + + there is an optional line break after hyphens (also -- and ---) but not
>> +after literal dashes, and
>> + + hyphenation is suppressed in words following hyphens but allowed after
>> +literal dashes.
> I think the above does not belong to the release notes for 2.3. It
> would have been appropriate for the 2.2 release notes. IMO, this has to
> go in the docs.
Description of past behaviour is IMO not out of place in a section called
"!!Caveats when upgrading from earlier versions to 2.3.x".
Is there a CHANGELOG where this could be find a better place?
>> + LyX 2.3 exports literal dashes as -- and --- by default. If you used
>> + literal em- and en-dashes in pre-2.2 documents, you must manually unselect
>> + "Document->Settings->Fonts->Output em- and en-dash as ligatures" to ensure
>> + unchanged behaviour.
> The above might be useful information and thus is Ok.
>> + It is no longer possible to differentiate dashes with/without optional
>> + line break using --- and -- vs. literal dashes. Either convert one sort to
>> + ERT or insert optional line break characters.
> This is true since 2.2, so this does not belong to the release notes for
> 2.3 and is better explained in the docs.
This also one of the caveats then upgrading from earlier versions (which are
not limited to upgrading from 2.2).
>> + lyx2lyx deletes ZWSP characters following literal em- and en-dashes when
>> + converting to 2.3 format. If you used literal ZWSP characters (u200b) as
>> + optional line breaks after dashes, convert them to 0dd wide space insets
>> + before opening your document with LyX 2.3 or the optional line breaks will
>> + be lost!
> I find the above so technical to be not understood by the vast majority
> of users. Moreover, the chance that someone used ZWSP to achieve that
> effect is negligible. Better removed from release notes and maybe explained
> in a footnote in the docs.
There is evidence of this usage and the consequences are easily overseen
and (because of the data loss) hard to revert. Even if not understood by
a majority of users, it should be clear to affected users. In any case,
it is not more technical then the next paragraph:
If using TeX fonts and en- and em-dashes are output as font ligatures,
when exporting documents containing en- and em-dashes to the format of
LyX 2.0 or earlier, the following line has to be manually added to the
unicodesymbols file of that LyX version:
0x200b "\\hspace{0pt}" "" "" "" "" # ZERO WIDTH SPACE
This avoids "uncodable character" issues if the document is actually
loaded by that LyX version. LyX 2.1 and later versions already have the
necessary definition in their unicodesymbols file
IMO, it is best to render both of them superfluous by not adding ZWSP in the
back conversion and not removing it in the forward conversion.
This also opens the way for true compatibility when exporting to 2.1 and
older formats.
See patch below.
Günter
diff --git a/lib/lyx2lyx/lyx_2_3.py b/lib/lyx2lyx/lyx_2_3.py
index 73ac45cf00..6305d417b0 100644
--- a/lib/lyx2lyx/lyx_2_3.py
+++ b/lib/lyx2lyx/lyx_2_3.py
@@ -1841,103 +1841,57 @@ def revert_chapterbib(document):
def convert_dashligatures(document):
-" Remove a zero-length space (U+200B) after en- and em-dashes. "
-
+" Set use_dash_ligatures according to content (literal vs. 'ligature'
dashes) "
+# Default:
+use_dash_ligatures = False # TODO: Get the default from stdtemplate.lyx
+# Look for dashes (followed by a word or no-break space):
+# (Documents by LyX 2.1 or older have "\twohyphens\n" or "\threehyphens\n"
+# as interim representation for dash ligatures in 2.2.)
+has_literal_dashes = has_ligature_dashes = False
+for i, line in enumerate(document.body):
+if re.search(u"[\u2013\u2014]([\w\u00A0]|$)", line, flags=re.UNICODE):
+has_literal_dashes = True
+if re.search(ur"(\\twohyphens|\\threehyphens)", line,
flags=re.UNICODE):
+# print "dash in line ", i, document.body[i+1].encode('utf8')
+if re.match(u"[\w\u00A0]", document.body[i+1], flags=re.UNICODE):
+has_ligature_dashes = True
+if has_literal_dashes and has_ligature_dashes:
+# TODO: insert a warning note in the document?
+document.warning("""This document contained both literal and
"ligature" dashes.
+Line break may have changed. See UserGuide chapter 3.9.1 for
details.""")
+elif has_literal_dashes:
+# print "has literal dashes"
+use_dash_ligatures = False
+elif has_ligature_dashes:
+
Re: dash documentation patch
On Wed, Aug 30, 2017 at 11:06:25PM +0200, Guenter Milde wrote:
>
> +* Since version 2.2, -- and --- in the LyX source are output as -{}- and
> + -{}-{}- to prevent conversion to en- and em-dashes by TeX.
> + Occurences in pre-2.2 documents are converted to literal Unicode dashes.
> + In some cases this leads to different line breaks, as:
> + + there is an optional line break after hyphens (also -- and ---) but not
> +after literal dashes, and
> + + hyphenation is suppressed in words following hyphens but allowed after
> +literal dashes.
I think the above does not belong to the release notes for 2.3. It would have
been appropriate for the 2.2 release notes. IMO, this has to go in the docs.
> + LyX 2.3 exports literal dashes as -- and --- by default. If you used
> + literal em- and en-dashes in pre-2.2 documents, you must manually unselect
> + "Document->Settings->Fonts->Output em- and en-dash as ligatures" to ensure
> + unchanged behaviour.
The above might be useful information and thus is Ok.
> + It is no longer possible to differentiate dashes with/without optional
> + line break using --- and -- vs. literal dashes. Either convert one sort to
> + ERT or insert optional line break characters.
This is true since 2.2, so this does not belong to the release notes for
2.3 and is better explained in the docs.
> + lyx2lyx deletes ZWSP characters following literal em- and en-dashes when
> + converting to 2.3 format. If you used literal ZWSP characters (u200b) as
> + optional line breaks after dashes, convert them to 0dd wide space insets
> + before opening your document with LyX 2.3 or the optional line breaks will
> + be lost!
I find the above so technical to be not understood by the vast majority
of users. Moreover, the chance that someone used ZWSP to achieve that
effect is negligible. Better removed from release notes and maybe explained
in a footnote in the docs.
--
Enrico
Re: dash documentation patch
On 2017-08-31, Scott Kostyshak wrote: > On Wed, Aug 30, 2017 at 11:06:25PM +0200, Guenter Milde wrote: >> Dear LyX developers, >> The following patch brings RELEASE_NOTES and UserGuide up to the current >> state of dash handling in lyx. (The UserGuide was still at 2.1 behaviour.) > I get the following output: > $ git am > /tmp/tmp.ua7NmO0Ljj/0001-Update-documentation-regarding-em--and-en-dashes.patch > Applying: Update documentation regarding em- and en-dashes. > .git/rebase-apply/patch:92: trailing whitespace. ... > warning: 39 lines add whitespace errors. > Warning: commit message did not conform to UTF-8. > You may want to amend it after fixing the message, or set the config > variable i18n.commitencoding to the encoding your project uses. > $ > I think the whitespace errors are normal (from the .lyx files), Yes, currently LyX leaves a lot of trailing spaces in the source files. May be an idea to check if they can be safely removed when saving or if there are side-effects or places where they change the semantics of the file. (But this is a different topic.) > but is the "did not confform to UTF-8" message expected? I think as a > result, the Author name of the commit is mangled. Thank you for pointing out this problem. Maybe the author name was mangled before (I don't know the reason) and I just did not realize it. I'll have to check before the next commit. Does it work if you replace the "From:" with mi...@lyx.org? How about the content? OK to commit? Thanks, Günter
Re: dash documentation patch
On Wed, Aug 30, 2017 at 11:06:25PM +0200, Guenter Milde wrote: > Dear LyX developers, > > The following patch brings RELEASE_NOTES and UserGuide up to the current > state of dash handling in lyx. (The UserGuide was still at 2.1 behaviour.) I get the following output: $ git am /tmp/tmp.ua7NmO0Ljj/0001-Update-documentation-regarding-em--and-en-dashes.patch Applying: Update documentation regarding em- and en-dashes. .git/rebase-apply/patch:92: trailing whitespace. .git/rebase-apply/patch:98: trailing whitespace. , this .git/rebase-apply/patch:115: trailing whitespace. : either .git/rebase-apply/patch:125: trailing whitespace. or .git/rebase-apply/patch:127: trailing whitespace. the bitmap font warning: squelched 34 whitespace errors warning: 39 lines add whitespace errors. Warning: commit message did not conform to UTF-8. You may want to amend it after fixing the message, or set the config variable i18n.commitencoding to the encoding your project uses. $ I think the whitespace errors are normal (from the .lyx files), but is the "did not confform to UTF-8" message expected? I think as a result, the Author name of the commit is mangled. Scott signature.asc Description: PGP signature
