In other words, it tells the library something significant about the module
that the engine needs to know about in order to perform a certain function
correctly.
Aside: I didn't mean to take this offline. I simply forgot to edit the
addresses.
David
Sent from ProtonMail Mobile
On Sun, Jan 7, 2018 at 13:46, DM Smith <dmsm...@crosswire.org> wrote:
> Library uses it to convert bytes to string.
>
> — DM Smith
> From my phone. Brief. Weird autocorrections.
>
> On Jan 7, 2018, at 8:44 AM, David Haslam <dfh...@protonmail.com> wrote:
>
>> Then answer this question, please.
>>
>> What value has the Encoding key ?
>>
>> David
>>
>> Sent from ProtonMail Mobile
>>
>> On Sun, Jan 7, 2018 at 12:52, DM Smith <dmsm...@crosswire.org> wrote:
>>
>>> SWORD too.
>>>
>>> I don’t yet see a value in the suggested conf entry.
>>>
>>> — DM Smith
>>> From my phone. Brief. Weird autocorrections.
>>>
>>> On Jan 7, 2018, at 4:03 AM, David Haslam <dfh...@protonmail.com> wrote:
>>>
>>>> You mean in the JSword API ?
>>>>
>>>> If so, that a start. Thanks, DM. :)
>>>>
>>>> Does that mean you now support the proposed new config key being accepted
>>>> and documented?
>>>>
>>>> Best regards,
>>>>
>>>> David
>>>>
>>>> Sent from ProtonMail Mobile
>>>>
>>>> On Sat, Jan 6, 2018 at 23:43, DM Smith <dmsm...@crosswire.org> wrote:
>>>>
>>>>> I added -N. To make search work.
>>>>>
>>>>> — DM Smith
>>>>> From my phone. Brief. Weird autocorrections.
>>>>>
>>>>> On Jan 6, 2018, at 4:41 PM, David Haslam <dfh...@protonmail.com> wrote:
>>>>>
>>>>>> Thanks DM.
>>>>>>
>>>>>> Interesting observations.
>>>>>>
>>>>>> It prompts the question whether either engine includes the capability to
>>>>>> normalize the search index (assuming that it does normalize the search
>>>>>> key).
>>>>>> And that it does this by default ????
>>>>>> Or does indexing assume that all modules were made without using the -N
>>>>>> option and are therefore already in NFC.
>>>>>> Yet it also remains the case that some front-ends also provide for
>>>>>> non-indexed search options.
>>>>>>
>>>>>> Moreover, it raise questions as to how the front-end actually displays
>>>>>> the set of search results when all or part of the underlying module is
>>>>>> not NFC.
>>>>>>
>>>>>> It must be the case that the developers of osis2mod had a valid reason
>>>>>> to provide the -N option.
>>>>>> Are those involved back then still with CrossWire?
>>>>>>
>>>>>> Best regards,
>>>>>>
>>>>>> David
>>>>>>
>>>>>> Sent from ProtonMail Mobile
>>>>>>
>>>>>> On Sat, Jan 6, 2018 at 21:20, DM Smith <dmsm...@crosswire.org> wrote:
>>>>>>
>>>>>>> The purpose of normalization was for the sake of search. Only when the
>>>>>>> search index and the search request are normalized to the same form can
>>>>>>> a result be found.
>>>>>>>
>>>>>>> It doesn’t matter if the normalized form is not readable. If SWORD (or
>>>>>>> JSword) normalizes both the same, then it doesn’t matter what Unicode
>>>>>>> Normalization or lack of it is used for displaying the text.
>>>>>>>
>>>>>>> Assuming that SWORD (or JSword) handles search properly, the only
>>>>>>> advantage of canonical over decomposed in the module itself is space.
>>>>>>>
>>>>>>> In Him,
>>>>>>> DM
>>>>>>>
>>>>>>>> On Jan 6, 2018, at 2:26 PM, David Haslam <dfh...@protonmail.com> wrote:
>>>>>>>>
>>>>>>>> Good question, Tom.
>>>>>>>>
>>>>>>>> Assuming that the Latin script part of the source text actually
>>>>>>>> required normalization to NFC,
>>>>>>>> and that at least some of the Biblical Hebrew should not be converted
>>>>>>>> to NFC,
>>>>>>>> you'd build the module using the -N switch of osis2mod, after first
>>>>>>>> applying a script
>>>>>>>> to the source text to ensure that both the requirements were
>>>>>>>> implemented.
>>>>>>>>
>>>>>>>> It would be a very simple task for a bespoke TextPipe filter with a
>>>>>>>> restrict filter
>>>>>>>> designed to limit the Convert to NFC subfilter to the text that was
>>>>>>>> not Hebrew.
>>>>>>>>
>>>>>>>> Ignoring alphabetical presentation forms, all the Hebrew characters
>>>>>>>> are in one Unicode block.
>>>>>>>> A PCRE to exclude the Hebrew would be very simple.
>>>>>>>> I could almost do it in my sleep after 17 years using TextPipe.
>>>>>>>> No doubt other programmers could do likewise with Perl or Python, etc.
>>>>>>>>
>>>>>>>> Best regards,
>>>>>>>>
>>>>>>>> David
>>>>>>>>
>>>>>>>> Sent from ProtonMail Mobile
>>>>>>>>
>>>>>>>> On Sat, Jan 6, 2018 at 19:14, Tom Sullivan <i...@beforgiven.info>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> Y'all: For text, such as in a commentary, which includes both Hebrew
>>>>>>>>> and English (or another modern Latin script using language), what do
>>>>>>>>> you put for the normalization? Tom Tom Sullivan
>>>>>>>>> info@BeForgiven.INFOFAX: 815-301-2835 --------------------- Great
>>>>>>>>> News! God created you, owns you and gave you commands to obey. You
>>>>>>>>> have disobeyed God - as your conscience very well attests to you.
>>>>>>>>> God's holiness and justice compel Him to punish you in Hell. Jesus
>>>>>>>>> Christ became Man, was crucified, buried and rose from the dead as a
>>>>>>>>> substitute for all who trust in Him, redeeming them from Hell. If you
>>>>>>>>> repent (turn from your sin) and believe (trust) in Jesus Christ, you
>>>>>>>>> will go to Heaven. Otherwise you will go to Hell. Warning! Good works
>>>>>>>>> are a result, not cause, of saving trust. More info is at
>>>>>>>>> www.esig.beforgiven.infoDo you believe this? Copy this signature into
>>>>>>>>> your email program and use the Internet to spread the Great News
>>>>>>>>> every time you email. On 01/06/2018 12:32 PM, David Haslam wrote: >
>>>>>>>>> Hi Greg, > > One area where it might turn out to be useful is for the
>>>>>>>>> search features > of front-end apps. > It could be important to know
>>>>>>>>> that the underlying module text is _not_ > *NFC*. > > That's not to
>>>>>>>>> lay down a requirement as to how search features should be >
>>>>>>>>> designed, > but at least to provide the information in case it does
>>>>>>>>> matter for some > types of search option. > > Like other things in
>>>>>>>>> .conf files, a key can also be _educational_. > It may prompt
>>>>>>>>> developers and users to ask, /*Why did they do this?*/ > > cf. It was
>>>>>>>>> _almost by accident_ that in 2014, I first came across this > aspect
>>>>>>>>> of using Unicode for Biblical Hebrew. > /It applies only to texts
>>>>>>>>> with _both_ vowel accents and cantillation./ > > Even though it's
>>>>>>>>> mentioned in our developers' wiki, it's all too easily > missed by
>>>>>>>>> other CrossWire volunteers. > > Best regards, > > David > > Sent with
>>>>>>>>> ProtonMail Secure Email. > >> -------- Original Message -------- >>
>>>>>>>>> Subject: Re: [sword-devel] Module .conf files, Unicode Normalization
>>>>>>>>> >> Local Time: 6 January 2018 5:19 PM >> UTC Time: 6 January 2018
>>>>>>>>> 17:19 >> From: greg.helli...@gmail.com >> To: David Haslam , SWORD
>>>>>>>>> Developers' >> Collaboration Forum >> >> Why would the front end or
>>>>>>>>> engine need to know this information? Would >> it help the front end
>>>>>>>>> developers or users to know it? What do we gain >> by adding this?
>>>>>>>>> (I'm not implying it wouldn't be beneficial. But the >> only thing I
>>>>>>>>> know about Unicode is how the different UTF encodings >> work, so I
>>>>>>>>> have no idea what use this information could be. I also >> think
>>>>>>>>> changes to formats and information standards should be >>
>>>>>>>>> conservative instead of liberal) >> >> --Greg >> >> On Jan 6, 2018
>>>>>>>>> 11:01, "David Haslam" > > wrote: >> >> Dear all, >> >> We've known
>>>>>>>>> for quite a few years that there are aspects of >> *Biblical Hebrew*
>>>>>>>>> that mean we should _avoid_ converting the >> Unicode source text to
>>>>>>>>> *NFC* when we build a module. >> >> This prompts me to suggest that
>>>>>>>>> we ought to define a new *key* for >> .conf files. >> >>
>>>>>>>>> *Normalization=NFC* (this would be the default, and may be >>
>>>>>>>>> _omitted_ for the vast majority of modules) >> *Normalization=Custom*
>>>>>>>>> (we should include this in certain Biblical >> Hebrew modules) >> >>
>>>>>>>>> This would make it clear to front-end developers and users alike >>
>>>>>>>>> that the source text was _not_ converted to NFC during module build.
>>>>>>>>> >> i.e. *osis2mod* was used intentionally with the *-N* switch, in >>
>>>>>>>>> _accordance with the requirements of the source text provider_. >> >>
>>>>>>>>> The Unicode source text may already be encoded in *UTF-8* ; this >>
>>>>>>>>> memo is /only /about normalization. >> >> In the rare eventuality
>>>>>>>>> that there could arise a requrement for >> any of the other three
>>>>>>>>> normalization forms (*NFD*, *NFKC*, *NFKD*) >> defined by the Unicode
>>>>>>>>> Consortium, >> these would also be permitted values for the conf file
>>>>>>>>> key. >> >> A further benefit arises when a module needs to be
>>>>>>>>> updated. >> If the modules team sees that the .conf file includes the
>>>>>>>>> line >> *Normalization=Custom* >> they would be forewarned against
>>>>>>>>> converting to NFC through >> /inadvertently/ omitting the *-N* switch
>>>>>>>>> during module build. >> >> _Aside_: Another language with a need for
>>>>>>>>> non-standard >> normalization is *Tibetan*. We don't yet have a
>>>>>>>>> module in that script. >> >> Best regards, >> >> David >> >> Sent
>>>>>>>>> with ProtonMail Secure Email. >> >> >>
>>>>>>>>> _______________________________________________ >> sword-devel
>>>>>>>>> mailing list: sword-devel@crosswire.org >> >>
>>>>>>>>> http://www.crosswire.org/mailman/listinfo/sword-devel >> >>
>>>>>>>>> Instructions to unsubscribe/change your settings at above page > > >
>>>>>>>>> ______________________________________________________________________
>>>>>>>>> > This email has been scanned by the Symantec Email Security.cloud
>>>>>>>>> service. > For more information please visit
>>>>>>>>> http://www.symanteccloud.com>
>>>>>>>>> ______________________________________________________________________
>>>>>>>>> > > > _______________________________________________ > sword-devel
>>>>>>>>> mailing list: sword-devel@crosswire.org>
>>>>>>>>> http://www.crosswire.org/mailman/listinfo/sword-devel > Instructions
>>>>>>>>> to unsubscribe/change your settings at above page >
>>>>>>>>> _______________________________________________ sword-devel mailing
>>>>>>>>> list: sword-devel@crosswire.org
>>>>>>>>> http://www.crosswire.org/mailman/listinfo/sword-develInstructions to
>>>>>>>>> unsubscribe/change your settings at above page @crosswire.org>
>>>>>>>>> @protonmail.com> @protonmail.com> @crosswire.org> @protonmail.com>
>>>>>>>>
>>>>>>>> _______________________________________________
>>>>>>>> sword-devel mailing list: sword-devel@crosswire.org
>>>>>>>> http://www.crosswire.org/mailman/listinfo/sword-devel
>>>>>>>> Instructions to unsubscribe/change your settings at above page
>>>>>
>>>>>> _______________________________________________
>>>>>> sword-devel mailing list: sword-devel@crosswire.org
>>>>>> http://www.crosswire.org/mailman/listinfo/sword-devel
>>>>>> Instructions to unsubscribe/change your settings at above page
>>>
>>>> _______________________________________________
>>>> sword-devel mailing list: sword-devel@crosswire.org
>>>> http://www.crosswire.org/mailman/listinfo/sword-devel
>>>> Instructions to unsubscribe/change your settings at above page
_______________________________________________
sword-devel mailing list: sword-devel@crosswire.org
http://www.crosswire.org/mailman/listinfo/sword-devel
Instructions to unsubscribe/change your settings at above page
