Re: new features with no documentation

2020-07-02 Thread Anatoli
Ellie, Ken, All,

I have prepared a PR [1] with some changes to the compiling page. Your
feedback is highly appreciated.

The changes are as follows:
 1. Completed the descriptions of a number of libraries with the
explanations of when they are needed and what happens if they are
not included (based on the information from this thread). For this I
also added to the tables a new column "Required".
 2. Completed the libs package names on Debian & RH for all libs
(except for the perl packages).
 3. Reordered some libs, in particular brotli, nghttp2, shapelib, wslay,
etc. (they were in Other, moved them to DAV/JMAP/httpd and renamed a
bit that section).
 4. Added links to the libs official websites where they were missing.
 5. Other minor fixes & cleanup.
 6. Removed 2 libs that IMO are not used anymore:
1. net-snmp: in my understanding, it was used for the SNMP code in
   master which is pending removal (BTW, is it possible to close the
   issues/1765 [2]? It looks like everything is ready); so no need
   to include obsolete features in the description;
2. tcp_wrappers: not 100% sure, but it looks like it's not used
   anymore.


Ellie, with respect to the point 6, please take this into account if you
would like to backport the changes to the previous versions. I guess 3.x
is ok, but not sure about 2.x.

For those who have RedHat instances, please confirm the libs package
names are correct. I have a Debian one so those should all be ok, but
for RH I had to look for the package names on the web.


On the other hand, Ellie, I see that the cld2 detection code is still in
configure.ac. Shouldn't it be removed based on what we've discussed here
(i.e. cld2 is not used anywhere and was included by mistake)?

And related, recently there was a thread ("cyrus-imap buildin: sse
extention") in info-cyrus@ about SSE4.2 compilation for CRC32c algo.
I've commented (but got no reply) that maybe the detection and
informational code should also be removed from configure.ac as it is not
used in any way at this moment and there are no plans to make it work,
so to remove the confusion and simplify the code.

Regards,
Anatoli

[1]: https://github.com/cyrusimap/cyrus-imapd/pull/3095
[2]: https://github.com/cyrusimap/cyrus-imapd/issues/1765


On 19/6/20 02:09, Anatoli wrote:
> Ellie,
> 
> Thanks for the explanation!
> 
> I'll try to prepare a PR for the compiling page these days.
> 
> Regards,
> Anatoli
> 
> On 19/6/20 00:29, ellie timoney wrote:
>> I think transfig is the package that provides fig2dev?  I think that's an 
>> artifact from the old days, and is only used for generation of a couple of 
>> png files in the legacy documentation (doc/legacy/murder.png and 
>> doc/legacy/netnews.png).  I think it's only needed when configure is run 
>> with --enable-maintainer-mode (i.e. if you're building a release or package, 
>> rather than just building the software).  Those files are pre-built in the 
>> release tarballs, so additionally you would only need this when building 
>> from git.  It would be nice to one day finish merging this legacy 
>> documentation into the current documentation, cause then we can get rid of 
>> it: https://github.com/cyrusimap/cyrus-imapd/issues/1769
>>
>> libsrs2 looks like it's used for implementing Sender Rewriting Scheme 
>> functionality for messages forwarded by sieve script.  Without it, messages 
>> forwarded by sieve script will not have this functionality (and, I suppose, 
>> might have difficulty delivering to servers that insist on it).
>>
>> The section of configure.ac that looks for libsrs2 is missing an 
>> AC_MSG_RESULT call to match the AC_MSG_CHECKING one.  It probably also wants 
>> an AC_NOTICE when it's not found, explaining the consequences of it being 
>> left out, like we have for the various http dependencies.  I'll fix this up 
>> in a moment.
>>
>> shapelib is already explained by configure.  For example, I don't have it 
>> installed myself, so I see:
>>
>>> checking for SHAPELIB... no
>>> configure: tzdist will not have geolocation support.  Consider installing 
>>> shapelib
>>
>> If you don't run a tzdist service, you don't need it.  If you do, you 
>> probably also know whether geolocation support is useful; I don't.
>>
>> I've already updated configure.ac to provide a similar notice when chardet 
>> is not found (https://github.com/cyrusimap/cyrus-imapd/commit/30fef51485); 
>> it'll be in 3.2.2. :)
>>
>> If you want to have a go at making the compiling page a bit clearer, I'd be 
>> happy to review/merge a pull request.  Please submit it against the master, 
>> and once we're happy with it, I'll get it cherry-picked appropriately to the 
>> other branches.
>>
>> Cheers,
>>
>> ellie
>>
>> On Fri, Jun 19, 2020, at 12:50 PM, Anatoli wrote:
>>> Ellie (or anyone else, Ken?),
>>>
>>> Could you please let us know what's the purpose of transfig, libsrs2 and
>>> shapelib libs in cyrus-imapd, and if they are required for any
>>> 

Re: new features with no documentation

2020-06-18 Thread Anatoli
Ellie,

Thanks for the explanation!

I'll try to prepare a PR for the compiling page these days.

Regards,
Anatoli

On 19/6/20 00:29, ellie timoney wrote:
> I think transfig is the package that provides fig2dev?  I think that's an 
> artifact from the old days, and is only used for generation of a couple of 
> png files in the legacy documentation (doc/legacy/murder.png and 
> doc/legacy/netnews.png).  I think it's only needed when configure is run with 
> --enable-maintainer-mode (i.e. if you're building a release or package, 
> rather than just building the software).  Those files are pre-built in the 
> release tarballs, so additionally you would only need this when building from 
> git.  It would be nice to one day finish merging this legacy documentation 
> into the current documentation, cause then we can get rid of it: 
> https://github.com/cyrusimap/cyrus-imapd/issues/1769
> 
> libsrs2 looks like it's used for implementing Sender Rewriting Scheme 
> functionality for messages forwarded by sieve script.  Without it, messages 
> forwarded by sieve script will not have this functionality (and, I suppose, 
> might have difficulty delivering to servers that insist on it).
> 
> The section of configure.ac that looks for libsrs2 is missing an 
> AC_MSG_RESULT call to match the AC_MSG_CHECKING one.  It probably also wants 
> an AC_NOTICE when it's not found, explaining the consequences of it being 
> left out, like we have for the various http dependencies.  I'll fix this up 
> in a moment.
> 
> shapelib is already explained by configure.  For example, I don't have it 
> installed myself, so I see:
> 
>> checking for SHAPELIB... no
>> configure: tzdist will not have geolocation support.  Consider installing 
>> shapelib
> 
> If you don't run a tzdist service, you don't need it.  If you do, you 
> probably also know whether geolocation support is useful; I don't.
> 
> I've already updated configure.ac to provide a similar notice when chardet is 
> not found (https://github.com/cyrusimap/cyrus-imapd/commit/30fef51485); it'll 
> be in 3.2.2. :)
> 
> If you want to have a go at making the compiling page a bit clearer, I'd be 
> happy to review/merge a pull request.  Please submit it against the master, 
> and once we're happy with it, I'll get it cherry-picked appropriately to the 
> other branches.
> 
> Cheers,
> 
> ellie
> 
> On Fri, Jun 19, 2020, at 12:50 PM, Anatoli wrote:
>> Ellie (or anyone else, Ken?),
>>
>> Could you please let us know what's the purpose of transfig, libsrs2 and
>> shapelib libs in cyrus-imapd, and if they are required for any
>> functionality (and if not (i.e. they are optional), what's their
>> benefit)?
>>
>> Also, Ellie, would it be useful to add to configure checks with warnings
>> for these libs (as well as for chardet) to inform those users that build
>> cyrus-imapd from sources that these libs are actually needed if certain
>> features are enabled (like Cal/CardDAV & JMAP)? I could prepare a patch.
>>
>> And I'd like to add these clarifications to the compiling page and
>> reformat a bit the tables. If you consider it appropriate, I could
>> prepare a patch too.
>>
>> Regards,
>> Anatoli
>>
>> On 5/6/20 05:45, Anatoli wrote:
 Thanks for the explanation. What happens is that at least Cal/CardDAV
 compile and work well without libchardet (at least under normal
 circumstances) and, as it was not marked as required, I thought it's an
 optional lib and here is the question of why to use it.
>>>
>>> Actually, the same happens with transfig, libsrs2 and shapelib.
>>>
>>> The page mentiones these libs, but "Required for `make check`" (not sure
>>> what that means in relation to the lib being reqiured for cyrus-imapd as
>>> such) for all of them is "no".
>>>
>>> What would help is a column "required" yes/no and another column "usage
>>> in cyrus-imapd" explaining for what exactly it's used. Say if it's for
>>> tzdata processing in the TZDist module, then I probably won't build with
>>> it as I don't see much use for it (or maybe package it as a flavor).
>>>
>>> But if it's something essential for a broader functionality like CalDAV
>>> itself, then definitely it would be included. But at this moment the
>>> Compiling page doesn't provide these details and here we are.
>>>
>>>
>>> On 5/6/20 05:30, Anatoli wrote:
 Ellie,

> libchardet is already listed on the compiling page as being used by
> the CalDAV, CardDAV, and/or JMAP features.  You would decide to
> install libchardet based on whether you need these features: if you
> don't enable these features, you don't need it; if you do, you
> probably do.
>
> If you don't have libchardet, Cyrus will not do character-set
> detection.  If some piece of data has no character set coming in, it
> will have no character set.  I don't know why you would decide to not
> use this, and it may simply become a mandatory requirement for these
> features in 3.4.  It might have been 

Re: new features with no documentation

2020-06-18 Thread ellie timoney
I think transfig is the package that provides fig2dev?  I think that's an 
artifact from the old days, and is only used for generation of a couple of png 
files in the legacy documentation (doc/legacy/murder.png and 
doc/legacy/netnews.png).  I think it's only needed when configure is run with 
--enable-maintainer-mode (i.e. if you're building a release or package, rather 
than just building the software).  Those files are pre-built in the release 
tarballs, so additionally you would only need this when building from git.  It 
would be nice to one day finish merging this legacy documentation into the 
current documentation, cause then we can get rid of it: 
https://github.com/cyrusimap/cyrus-imapd/issues/1769

libsrs2 looks like it's used for implementing Sender Rewriting Scheme 
functionality for messages forwarded by sieve script.  Without it, messages 
forwarded by sieve script will not have this functionality (and, I suppose, 
might have difficulty delivering to servers that insist on it).

The section of configure.ac that looks for libsrs2 is missing an AC_MSG_RESULT 
call to match the AC_MSG_CHECKING one.  It probably also wants an AC_NOTICE 
when it's not found, explaining the consequences of it being left out, like we 
have for the various http dependencies.  I'll fix this up in a moment.

shapelib is already explained by configure.  For example, I don't have it 
installed myself, so I see:

> checking for SHAPELIB... no
> configure: tzdist will not have geolocation support.  Consider installing 
> shapelib

If you don't run a tzdist service, you don't need it.  If you do, you probably 
also know whether geolocation support is useful; I don't.

I've already updated configure.ac to provide a similar notice when chardet is 
not found (https://github.com/cyrusimap/cyrus-imapd/commit/30fef51485); it'll 
be in 3.2.2. :)

If you want to have a go at making the compiling page a bit clearer, I'd be 
happy to review/merge a pull request.  Please submit it against the master, and 
once we're happy with it, I'll get it cherry-picked appropriately to the other 
branches.

Cheers,

ellie

On Fri, Jun 19, 2020, at 12:50 PM, Anatoli wrote:
> Ellie (or anyone else, Ken?),
> 
> Could you please let us know what's the purpose of transfig, libsrs2 and
> shapelib libs in cyrus-imapd, and if they are required for any
> functionality (and if not (i.e. they are optional), what's their
> benefit)?
> 
> Also, Ellie, would it be useful to add to configure checks with warnings
> for these libs (as well as for chardet) to inform those users that build
> cyrus-imapd from sources that these libs are actually needed if certain
> features are enabled (like Cal/CardDAV & JMAP)? I could prepare a patch.
> 
> And I'd like to add these clarifications to the compiling page and
> reformat a bit the tables. If you consider it appropriate, I could
> prepare a patch too.
> 
> Regards,
> Anatoli
> 
> On 5/6/20 05:45, Anatoli wrote:
> >> Thanks for the explanation. What happens is that at least Cal/CardDAV
> >> compile and work well without libchardet (at least under normal
> >> circumstances) and, as it was not marked as required, I thought it's an
> >> optional lib and here is the question of why to use it.
> > 
> > Actually, the same happens with transfig, libsrs2 and shapelib.
> > 
> > The page mentiones these libs, but "Required for `make check`" (not sure
> > what that means in relation to the lib being reqiured for cyrus-imapd as
> > such) for all of them is "no".
> > 
> > What would help is a column "required" yes/no and another column "usage
> > in cyrus-imapd" explaining for what exactly it's used. Say if it's for
> > tzdata processing in the TZDist module, then I probably won't build with
> > it as I don't see much use for it (or maybe package it as a flavor).
> > 
> > But if it's something essential for a broader functionality like CalDAV
> > itself, then definitely it would be included. But at this moment the
> > Compiling page doesn't provide these details and here we are.
> > 
> > 
> > On 5/6/20 05:30, Anatoli wrote:
> >> Ellie,
> >>
> >>> libchardet is already listed on the compiling page as being used by
> >>> the CalDAV, CardDAV, and/or JMAP features.  You would decide to
> >>> install libchardet based on whether you need these features: if you
> >>> don't enable these features, you don't need it; if you do, you
> >>> probably do.
> >>>
> >>> If you don't have libchardet, Cyrus will not do character-set
> >>> detection.  If some piece of data has no character set coming in, it
> >>> will have no character set.  I don't know why you would decide to not
> >>> use this, and it may simply become a mandatory requirement for these
> >>> features in 3.4.  It might have been mandatory in 3.2, if this had
> >>> been brought to our attention during the nearly 3 month beta window;
> >>> but now that it's in a stable release, it will remain as it is.
> >>
> >> Thanks for the explanation. What happens is that at least Cal/CardDAV
> >> 

Re: new features with no documentation

2020-06-18 Thread Anatoli
Ellie (or anyone else, Ken?),

Could you please let us know what's the purpose of transfig, libsrs2 and
shapelib libs in cyrus-imapd, and if they are required for any
functionality (and if not (i.e. they are optional), what's their
benefit)?

Also, Ellie, would it be useful to add to configure checks with warnings
for these libs (as well as for chardet) to inform those users that build
cyrus-imapd from sources that these libs are actually needed if certain
features are enabled (like Cal/CardDAV & JMAP)? I could prepare a patch.

And I'd like to add these clarifications to the compiling page and
reformat a bit the tables. If you consider it appropriate, I could
prepare a patch too.

Regards,
Anatoli

On 5/6/20 05:45, Anatoli wrote:
>> Thanks for the explanation. What happens is that at least Cal/CardDAV
>> compile and work well without libchardet (at least under normal
>> circumstances) and, as it was not marked as required, I thought it's an
>> optional lib and here is the question of why to use it.
> 
> Actually, the same happens with transfig, libsrs2 and shapelib.
> 
> The page mentiones these libs, but "Required for `make check`" (not sure
> what that means in relation to the lib being reqiured for cyrus-imapd as
> such) for all of them is "no".
> 
> What would help is a column "required" yes/no and another column "usage
> in cyrus-imapd" explaining for what exactly it's used. Say if it's for
> tzdata processing in the TZDist module, then I probably won't build with
> it as I don't see much use for it (or maybe package it as a flavor).
> 
> But if it's something essential for a broader functionality like CalDAV
> itself, then definitely it would be included. But at this moment the
> Compiling page doesn't provide these details and here we are.
> 
> 
> On 5/6/20 05:30, Anatoli wrote:
>> Ellie,
>>
>>> libchardet is already listed on the compiling page as being used by
>>> the CalDAV, CardDAV, and/or JMAP features.  You would decide to
>>> install libchardet based on whether you need these features: if you
>>> don't enable these features, you don't need it; if you do, you
>>> probably do.
>>>
>>> If you don't have libchardet, Cyrus will not do character-set
>>> detection.  If some piece of data has no character set coming in, it
>>> will have no character set.  I don't know why you would decide to not
>>> use this, and it may simply become a mandatory requirement for these
>>> features in 3.4.  It might have been mandatory in 3.2, if this had
>>> been brought to our attention during the nearly 3 month beta window;
>>> but now that it's in a stable release, it will remain as it is.
>>
>> Thanks for the explanation. What happens is that at least Cal/CardDAV
>> compile and work well without libchardet (at least under normal
>> circumstances) and, as it was not marked as required, I thought it's an
>> optional lib and here is the question of why to use it.
>>
>> I guess what you could do for 3.2 is to include a warning in configure
>> explaining the issue, so the maintainers see it and take appropriate
>> action. I suppose there are not that many installations with 3.2 yet so
>> if the warning is included with 3.2.2 it would be an appropriate time.
>>
>> Regards,
>> Anatoli
>>
>> On 4/6/20 21:58, ellie timoney wrote:
>>> On Thu, Jun 4, 2020, at 12:52 PM, Anatoli wrote:
> chardet is used by the JMAP module of httpd to detect the character
 set of untagged 8-bit headers.

 Does that mean that these libs are required? If not, what would happen
 if not included? How Cyrus would detect the charset? Or put in other
 words, if they are not required, what's the benefit to use them?

 On the other hand, may I suggest to please add these details to the
 Compiling [1] page? There's already a lot of useful information, but
 some of the libs lack any details on what's their purpose in Cyrus and
 why they are beneficial/required (and if they are not used yet, maybe
 they shouldn't be mentioned there at all?). This could help porters and
 maintainers to make more educated decisions on how to package Cyrus.
>>>
>>> libchardet is already listed on the compiling page as being used by the 
>>> CalDAV, CardDAV, and/or JMAP features.  You would decide to install 
>>> libchardet based on whether you need these features: if you don't enable 
>>> these features, you don't need it; if you do, you probably do.
>>>
>>> If you don't have libchardet, Cyrus will not do character-set detection.  
>>> If some piece of data has no character set coming in, it will have no 
>>> character set.  I don't know why you would decide to not use this, and it 
>>> may simply become a mandatory requirement for these features in 3.4.  It 
>>> might have been mandatory in 3.2, if this had been brought to our attention 
>>> during the nearly 3 month beta window; but now that it's in a stable 
>>> release, it will remain as it is.
>>>
>>> cld2 was used by one (or maybe both) of the experimental search 

Re: new features with no documentation

2020-06-05 Thread Anatoli
> Thanks for the explanation. What happens is that at least Cal/CardDAV
> compile and work well without libchardet (at least under normal
> circumstances) and, as it was not marked as required, I thought it's an
> optional lib and here is the question of why to use it.

Actually, the same happens with transfig, libsrs2 and shapelib.

The page mentiones these libs, but "Required for `make check`" (not sure
what that means in relation to the lib being reqiured for cyrus-imapd as
such) for all of them is "no".

What would help is a column "required" yes/no and another column "usage
in cyrus-imapd" explaining for what exactly it's used. Say if it's for
tzdata processing in the TZDist module, then I probably won't build with
it as I don't see much use for it (or maybe package it as a flavor).

But if it's something essential for a broader functionality like CalDAV
itself, then definitely it would be included. But at this moment the
Compiling page doesn't provide these details and here we are.


On 5/6/20 05:30, Anatoli wrote:
> Ellie,
> 
>> libchardet is already listed on the compiling page as being used by
>> the CalDAV, CardDAV, and/or JMAP features.  You would decide to
>> install libchardet based on whether you need these features: if you
>> don't enable these features, you don't need it; if you do, you
>> probably do.
>>
>> If you don't have libchardet, Cyrus will not do character-set
>> detection.  If some piece of data has no character set coming in, it
>> will have no character set.  I don't know why you would decide to not
>> use this, and it may simply become a mandatory requirement for these
>> features in 3.4.  It might have been mandatory in 3.2, if this had
>> been brought to our attention during the nearly 3 month beta window;
>> but now that it's in a stable release, it will remain as it is.
> 
> Thanks for the explanation. What happens is that at least Cal/CardDAV
> compile and work well without libchardet (at least under normal
> circumstances) and, as it was not marked as required, I thought it's an
> optional lib and here is the question of why to use it.
> 
> I guess what you could do for 3.2 is to include a warning in configure
> explaining the issue, so the maintainers see it and take appropriate
> action. I suppose there are not that many installations with 3.2 yet so
> if the warning is included with 3.2.2 it would be an appropriate time.
> 
> Regards,
> Anatoli
> 
> On 4/6/20 21:58, ellie timoney wrote:
>> On Thu, Jun 4, 2020, at 12:52 PM, Anatoli wrote:
 chardet is used by the JMAP module of httpd to detect the character
>>> set of untagged 8-bit headers.
>>>
>>> Does that mean that these libs are required? If not, what would happen
>>> if not included? How Cyrus would detect the charset? Or put in other
>>> words, if they are not required, what's the benefit to use them?
>>>
>>> On the other hand, may I suggest to please add these details to the
>>> Compiling [1] page? There's already a lot of useful information, but
>>> some of the libs lack any details on what's their purpose in Cyrus and
>>> why they are beneficial/required (and if they are not used yet, maybe
>>> they shouldn't be mentioned there at all?). This could help porters and
>>> maintainers to make more educated decisions on how to package Cyrus.
>>
>> libchardet is already listed on the compiling page as being used by the 
>> CalDAV, CardDAV, and/or JMAP features.  You would decide to install 
>> libchardet based on whether you need these features: if you don't enable 
>> these features, you don't need it; if you do, you probably do.
>>
>> If you don't have libchardet, Cyrus will not do character-set detection.  If 
>> some piece of data has no character set coming in, it will have no character 
>> set.  I don't know why you would decide to not use this, and it may simply 
>> become a mandatory requirement for these features in 3.4.  It might have 
>> been mandatory in 3.2, if this had been brought to our attention during the 
>> nearly 3 month beta window; but now that it's in a stable release, it will 
>> remain as it is.
>>
>> cld2 was used by one (or maybe both) of the experimental search features 
>> that were accidentally included in 3.2.0, and removed in 3.2.1 (see the 
>> release notes).  Looks like we forgot to remove the configure checks for the 
>> library when removing the feature(s) that used it.  You don't need it, and 
>> if you have it, it won't be used anyway.
>>
>> Cheers,
>>
>> ellie
>>


Re: new features with no documentation

2020-06-05 Thread Anatoli
Ellie,

> libchardet is already listed on the compiling page as being used by
> the CalDAV, CardDAV, and/or JMAP features.  You would decide to
> install libchardet based on whether you need these features: if you
> don't enable these features, you don't need it; if you do, you
> probably do.
>
> If you don't have libchardet, Cyrus will not do character-set
> detection.  If some piece of data has no character set coming in, it
> will have no character set.  I don't know why you would decide to not
> use this, and it may simply become a mandatory requirement for these
> features in 3.4.  It might have been mandatory in 3.2, if this had
> been brought to our attention during the nearly 3 month beta window;
> but now that it's in a stable release, it will remain as it is.

Thanks for the explanation. What happens is that at least Cal/CardDAV
compile and work well without libchardet (at least under normal
circumstances) and, as it was not marked as required, I thought it's an
optional lib and here is the question of why to use it.

I guess what you could do for 3.2 is to include a warning in configure
explaining the issue, so the maintainers see it and take appropriate
action. I suppose there are not that many installations with 3.2 yet so
if the warning is included with 3.2.2 it would be an appropriate time.

Regards,
Anatoli

On 4/6/20 21:58, ellie timoney wrote:
> On Thu, Jun 4, 2020, at 12:52 PM, Anatoli wrote:
>>> chardet is used by the JMAP module of httpd to detect the character
>> set of untagged 8-bit headers.
>>
>> Does that mean that these libs are required? If not, what would happen
>> if not included? How Cyrus would detect the charset? Or put in other
>> words, if they are not required, what's the benefit to use them?
>>
>> On the other hand, may I suggest to please add these details to the
>> Compiling [1] page? There's already a lot of useful information, but
>> some of the libs lack any details on what's their purpose in Cyrus and
>> why they are beneficial/required (and if they are not used yet, maybe
>> they shouldn't be mentioned there at all?). This could help porters and
>> maintainers to make more educated decisions on how to package Cyrus.
> 
> libchardet is already listed on the compiling page as being used by the 
> CalDAV, CardDAV, and/or JMAP features.  You would decide to install 
> libchardet based on whether you need these features: if you don't enable 
> these features, you don't need it; if you do, you probably do.
> 
> If you don't have libchardet, Cyrus will not do character-set detection.  If 
> some piece of data has no character set coming in, it will have no character 
> set.  I don't know why you would decide to not use this, and it may simply 
> become a mandatory requirement for these features in 3.4.  It might have been 
> mandatory in 3.2, if this had been brought to our attention during the nearly 
> 3 month beta window; but now that it's in a stable release, it will remain as 
> it is.
> 
> cld2 was used by one (or maybe both) of the experimental search features that 
> were accidentally included in 3.2.0, and removed in 3.2.1 (see the release 
> notes).  Looks like we forgot to remove the configure checks for the library 
> when removing the feature(s) that used it.  You don't need it, and if you 
> have it, it won't be used anyway.
> 
> Cheers,
> 
> ellie
> 


Re: new features with no documentation

2020-06-04 Thread ellie timoney
On Thu, Jun 4, 2020, at 12:52 PM, Anatoli wrote:
> > chardet is used by the JMAP module of httpd to detect the character
> set of untagged 8-bit headers.
> 
> Does that mean that these libs are required? If not, what would happen
> if not included? How Cyrus would detect the charset? Or put in other
> words, if they are not required, what's the benefit to use them?
> 
> On the other hand, may I suggest to please add these details to the
> Compiling [1] page? There's already a lot of useful information, but
> some of the libs lack any details on what's their purpose in Cyrus and
> why they are beneficial/required (and if they are not used yet, maybe
> they shouldn't be mentioned there at all?). This could help porters and
> maintainers to make more educated decisions on how to package Cyrus.

libchardet is already listed on the compiling page as being used by the CalDAV, 
CardDAV, and/or JMAP features.  You would decide to install libchardet based on 
whether you need these features: if you don't enable these features, you don't 
need it; if you do, you probably do.

If you don't have libchardet, Cyrus will not do character-set detection.  If 
some piece of data has no character set coming in, it will have no character 
set.  I don't know why you would decide to not use this, and it may simply 
become a mandatory requirement for these features in 3.4.  It might have been 
mandatory in 3.2, if this had been brought to our attention during the nearly 3 
month beta window; but now that it's in a stable release, it will remain as it 
is.

cld2 was used by one (or maybe both) of the experimental search features that 
were accidentally included in 3.2.0, and removed in 3.2.1 (see the release 
notes).  Looks like we forgot to remove the configure checks for the library 
when removing the feature(s) that used it.  You don't need it, and if you have 
it, it won't be used anyway.

Cheers,

ellie


Re: new features with no documentation

2020-06-03 Thread Anatoli
Ken, Partha,

Thanks for the explanation.

> chardet is used by the JMAP module of httpd to detect the character
set of untagged 8-bit headers.

Does that mean that these libs are required? If not, what would happen
if not included? How Cyrus would detect the charset? Or put in other
words, if they are not required, what's the benefit to use them?

On the other hand, may I suggest to please add these details to the
Compiling [1] page? There's already a lot of useful information, but
some of the libs lack any details on what's their purpose in Cyrus and
why they are beneficial/required (and if they are not used yet, maybe
they shouldn't be mentioned there at all?). This could help porters and
maintainers to make more educated decisions on how to package Cyrus.

Thanks,
Anatoli

[1] https://www.cyrusimap.org/imap/developer/compiling.html


On 3/6/20 19:01, Ken Murchison wrote:
> 
> On 6/3/20 5:44 PM, Partha Susarla wrote:
>> Hello Anatoli,
>> On Wed, 3 Jun 2020, at 6:24 PM, Anatoli wrote:
>>> Cyrus developers,
>>>
>>> What is the purpose/benefit of zeroskip?
>>>
>> Zeroskip is an append-only key-value DB, currently in experimental stage.
>>
>>> What is the purpose of chardet and cld2 in cyrus-imapd?
>>>
>> Both the character detection and language detection libraries although are 
>> listed in the dependencies, i don't think are being used. But I will let 
>> ellie/Ken respond to this.
> 
> 
> chardet is used by the JMAP module of httpd to detect the character set of 
> untagged 8-bit headers.
> 
> cld2 is also used by JMAP for language detection but it doesn't look like its 
> being used by any code on the master branch yet.  I believe that Robert S. 
> has a dev branch that uses it.  We can't ask him right now because he's on 
> holiday the rest of June.
> 
> 


Re: new features with no documentation

2020-06-03 Thread Ken Murchison



On 6/3/20 5:44 PM, Partha Susarla wrote:

Hello Anatoli,
On Wed, 3 Jun 2020, at 6:24 PM, Anatoli wrote:

Cyrus developers,

What is the purpose/benefit of zeroskip?


Zeroskip is an append-only key-value DB, currently in experimental stage.


What is the purpose of chardet and cld2 in cyrus-imapd?


Both the character detection and language detection libraries although are 
listed in the dependencies, i don't think are being used. But I will let 
ellie/Ken respond to this.



chardet is used by the JMAP module of httpd to detect the character set 
of untagged 8-bit headers.


cld2 is also used by JMAP for language detection but it doesn't look 
like its being used by any code on the master branch yet.  I believe 
that Robert S. has a dev branch that uses it.  We can't ask him right 
now because he's on holiday the rest of June.



--

Kenneth Murchison
Senior Software Developer
Fastmail US LLC



Re: new features with no documentation

2020-06-03 Thread Partha Susarla
Hello Anatoli,
On Wed, 3 Jun 2020, at 6:24 PM, Anatoli wrote:
> Cyrus developers,
> 
> What is the purpose/benefit of zeroskip?
> 
Zeroskip is an append-only key-value DB, currently in experimental stage. 

> What is the purpose of chardet and cld2 in cyrus-imapd?
> 
Both the character detection and language detection libraries although are 
listed in the dependencies, i don't think are being used. But I will let 
ellie/Ken respond to this.

> Should any be considered in production environments running 3.2 branch?
> 
You can skip installing Zeroskip, as it is not being used in Cyrus-imapd 
currently. 

..Partha


new features with no documentation

2020-06-03 Thread Anatoli
Cyrus developers,

What is the purpose/benefit of zeroskip?

What is the purpose of chardet and cld2 in cyrus-imapd?

Should any be considered in production environments running 3.2 branch?

Thanks,
Anatoli