AW: Re: [SOGo] Outlook EAS with mailboxes > 3GB

2015-10-14 Thread tfu 
Peter,

  IMHO the parameter SOGoMaximumSyncResponseSize = 524288 is set too large.

  [...]
  The value is in kilobytes. Setting 
this to 512 means the response size 
will be of 524288 bytes or less.
  Note that if you set the value 
too low and a mail message (or any other object) surpasses 
it, it will still be synced but only this item will be.
  [...]

  Depending on the email/attachment size it takes some time to fill response 
buffer.
  Setting this parameter to a lower value can help to avoid the "disconnects" 
in OL.
  
  Id suggest to try different value (starting with 10240) and check the 
outcome.

  Regards,
  Thomas
  
 users@sogo.nu schrieb 
   > On 10/12/2015 08:37 AM, Sascha Kasch wrote: >
>>  also tried a lot with activesync and had the same problems with
>> mailboxes
>>  above 3 or 5 GB. Some of them are even 20+ GB.
>>  These are my settings with a prefork (/etc/default/sogo) of 15:
>  Hey Sascha,
>
>  I've just tried your settings (on an already "pre"-synced
> account) and
>  now it finally says "all folders are up to date" and does not
> disconnect
>  every few minutes, seems to be not that bad with your values, thanks a 
> lot!
>
>  But still got a user with an subfolder containg about 5000 messages, it
>  downloads about 100 messages, then ActiveSysnc get's disconnected in
>  Outlook. After reconncting, about another 100 messages get downloaded,
>  then it disconnects againbut that mailbox is bigger than 5GB...
>
>  Also found an Microsoft-KB about disconnecting, they tell to set the
>  registry key "PushDuration" to 30 [1]. Just started testing
> with this key.
>   > With these settings (and IMAP wordlenght of 4MB) i am able to 
> sync
>>  approx 4-5GB in an hour. And for small mailboxes this i enough and
>> mostly stable.
>>
>>  one is do move this mail item to another folder and back and hopefully
>>  it gets resynced and displayed.
>>  Another is to delete the .ost cache file and let Outlook resync
>>  everything. Obviously this is not a way to
>>  go regulary. This problem is a show stopper for me as i can not say
> for
>>  sure if my inbox is always properly synced
>  this shouldn't happen, I already can hear them screaming if they
> finally
>  find out two weeks later that they've missed some messages...
>  By the way: I've just deleted one of these ost-files on one of the
>  clients. Now Outlook creates new folders (Drafts (only this computer),
>  Trash[1], Sent[1]). Oh god how I hate this piece of software!!
>   > As i already saw that behavior with Outlook 2013
>>  being connected via IMAP i cannot blame it on EAS or SOGo.
>  Did not know this also happens with IMAP. Mail would be such a great
>  thing to work on - without Outlook. This is the most market-manipulating
>  thing...life would be much better without that piece of crap
>  designed to work with Exchange only, everything else is (intentionally)
>  poorly supported...
>
>
>  Thanks again
>
>  Best Regards
>  Peter
>
>  [1] https://support.microsoft.com/en-us/kb/2781579
>
>  --
>  users@sogo.nu
>  https://inverse.ca/sogo/lists
>  
-- 
users@sogo.nu
https://inverse.ca/sogo/lists

Re: [SOGo] Outlook EAS with mailboxes > 3GB

2015-10-14 Thread Sascha Kasch 

Am 14.10.2015 um 01:42 schrieb Peter Beck:

On 10/12/2015 08:37 AM, Sascha Kasch wrote:

also tried a lot with activesync and had the same problems with mailboxes
above 3 or 5 GB. Some of them are even 20+ GB.
These are my settings with a prefork (/etc/default/sogo) of 15:

Hey Sascha,

I've just tried your settings (on an already "pre"-synced account) and
now it finally says "all folders are up to date" and does not disconnect
every few minutes, seems to be not that bad with your values, thanks a lot!

But still got a user with an subfolder containg about 5000 messages, it
downloads about 100 messages, then ActiveSysnc get's disconnected in
Outlook. After reconncting, about another 100 messages get downloaded,
then it disconnects againbut that mailbox is bigger than 5GB...

Also found an Microsoft-KB about disconnecting, they tell to set the
registry key "PushDuration" to 30 [1]. Just started testing with this key.


Hi Peter,

when using WorkersCount/Prefork of 15 and virtual Memory of 1024MB
your machine should have at least 16GB RAM for SOGo.
Have a look at sogo.log and take note of "child terminated with signal X".
I am not sure which signal is ok in terms of the parent process kills 
the child
due to reaching memory limit but that should be stated right before 
killing the child.


Had some headaches with working simultaneously with IMAP when Outlook is 
currently

syncing. Creating and Renaming Folders broke the sync.
Some folders wouldn't show as the customer also worked with Mac and 
somehow managed

to name folders with CRLF or prefix them with a space.
Another problem were some contacts that did not sync with customers 
Outlook while i had his
account synced to another Outlook which showed all contacts. Deleting 
the contact within SOGo
and readd them made them visible with customers Outlook (but to be 
honest: comparing 1000 contacts

and mark those missing is not a task i'd like to do regulary)



With these settings (and IMAP wordlenght of 4MB) i am able to sync
approx 4-5GB in an hour. And for small mailboxes this i enough and mostly 
stable.

one is do move this mail item to another folder and back and hopefully
it gets resynced and displayed.
Another is to delete the .ost cache file and let Outlook resync
everything. Obviously this is not a way to
go regulary. This problem is a show stopper for me as i can not say for
sure if my inbox is always properly synced

this shouldn't happen, I already can hear them screaming if they finally
find out two weeks later that they've missed some messages...
By the way: I've just deleted one of these ost-files on one of the
clients. Now Outlook creates new folders (Drafts (only this computer),
Trash[1], Sent[1]). Oh god how I hate this piece of software!!


afaik drafts are always stored locally with activesync. I never got 
Outlook to store them

on the server.
Another thing to note is, that deleting the ost-file leaves you with 
lost follow ups, rules no longer
find their target folder (moving mails rules based) and all categories 
will be deleted.
To have the resync more clean you could also "sogo-tool manage-eas 
resetdevice username deviceID"

That should eliminate your [1] folders.




As i already saw that behavior with Outlook 2013
being connected via IMAP i cannot blame it on EAS or SOGo.

Did not know this also happens with IMAP. Mail would be such a great
thing to work on - without Outlook. This is the most market-manipulating
thing...life would be much better without that piece of crap
designed to work with Exchange only, everything else is (intentionally)
poorly supported...
There's nothing left to add! Recently read that Microsoft stated with 
the release of 2016

"Implementing calDAV and cardDAV might be done within the next years (!)"


Thanks again

Best Regards
Peter

Hoepfully you will be more successful and report back to the list.

Cheers,
Sascha


[1] https://support.microsoft.com/en-us/kb/2781579



--
users@sogo.nu
https://inverse.ca/sogo/lists

Re: [SOGo] Outlook EAS with mailboxes > 3GB

2015-10-13 Thread Peter Beck 
On 10/12/2015 08:37 AM, Sascha Kasch wrote:
> 
> also tried a lot with activesync and had the same problems with mailboxes
> above 3 or 5 GB. Some of them are even 20+ GB.
> These are my settings with a prefork (/etc/default/sogo) of 15:

Hey Sascha,

I've just tried your settings (on an already "pre"-synced account) and
now it finally says "all folders are up to date" and does not disconnect
every few minutes, seems to be not that bad with your values, thanks a lot!

But still got a user with an subfolder containg about 5000 messages, it
downloads about 100 messages, then ActiveSysnc get's disconnected in
Outlook. After reconncting, about another 100 messages get downloaded,
then it disconnects againbut that mailbox is bigger than 5GB...

Also found an Microsoft-KB about disconnecting, they tell to set the
registry key "PushDuration" to 30 [1]. Just started testing with this key.

> With these settings (and IMAP wordlenght of 4MB) i am able to sync
> approx 4-5GB in an hour. And for small mailboxes this i enough and mostly 
> stable.
> 
> one is do move this mail item to another folder and back and hopefully
> it gets resynced and displayed.
> Another is to delete the .ost cache file and let Outlook resync
> everything. Obviously this is not a way to
> go regulary. This problem is a show stopper for me as i can not say for
> sure if my inbox is always properly synced 

this shouldn't happen, I already can hear them screaming if they finally
find out two weeks later that they've missed some messages...
By the way: I've just deleted one of these ost-files on one of the
clients. Now Outlook creates new folders (Drafts (only this computer),
Trash[1], Sent[1]). Oh god how I hate this piece of software!!

> As i already saw that behavior with Outlook 2013
> being connected via IMAP i cannot blame it on EAS or SOGo.

Did not know this also happens with IMAP. Mail would be such a great
thing to work on - without Outlook. This is the most market-manipulating
thing...life would be much better without that piece of crap
designed to work with Exchange only, everything else is (intentionally)
poorly supported...


Thanks again

Best Regards
Peter

[1] https://support.microsoft.com/en-us/kb/2781579

-- 
users@sogo.nu
https://inverse.ca/sogo/lists

Re: [SOGo] Outlook EAS with mailboxes > 3GB

2015-10-12 Thread Mattias Fliesberg 
We have the same problem here, people with large mailboxes using EAS with
Outlook 2013 have issues with initial sync being slow, some mails never even
show up in Outlook and the same goes for the calendar, some events never sync.
The same people are using EAS on their Android phones and it's working fine
there.

Right now I too recommend them to use IMAP and CalDavSynchronizer.

Now that I took a quick peek in our logs I found this one that I haven't seen
before I think...

"Fatal error occured - tried to call -processSyncAddCommand: ... on a mail 
folder. We abort"

... other than that it's mostly just "found no viewer for MIME type". I suspect
this might have nothing to do with it though and it's probably more than one 
issue.

// Mattias

* Peter Beck (pe...@datentraeger.li) wrote:
> On 10/10/2015 08:22 PM, Alain Abbas wrote:
> > for now we had stopped the deployement of activesync and outlook2013 
> 
> Hi Alain,
> 
> good to know I am not the only one ;-)
> I've also switched (at least the big mailboxes) to IMAP and
> CalDavSynchronizer (which will hopefully soon get CardDAV support).
> 
> But still hoping someone can enlighten us here on the list...
> 
> Regards
> Peter
> -- 
> users@sogo.nu
> https://inverse.ca/sogo/lists
-- 
users@sogo.nu
https://inverse.ca/sogo/lists

Re: [SOGo] Outlook EAS with mailboxes > 3GB

2015-10-12 Thread Sascha Kasch 

Am 11.10.2015 um 22:29 schrieb Peter Beck:

On 10/10/2015 08:22 PM, Alain Abbas wrote:

for now we had stopped the deployement of activesync and outlook2013

Hi Alain,

good to know I am not the only one ;-)
I've also switched (at least the big mailboxes) to IMAP and
CalDavSynchronizer (which will hopefully soon get CardDAV support).

But still hoping someone can enlighten us here on the list...

Regards
Peter

Hi Peter,

also tried a lot with activesync and had the same problems with mailboxes
above 3 or 5 GB. Some of them are even 20+ GB.
These are my settings with a prefork (/etc/default/sogo) of 15:

SxVMemLimit = 1024;
SOGoMaximumPingInterval = 3540;
SOGoMaximumSyncInterval = 3540;
SOGoInternalSyncInterval = 30;
SOGoMaximumSyncWindowSize = 128;
SOGoMaximumSyncResponseSize = 524288;
WOWatchDogRequestTimeout = 60;

With these settings (and IMAP wordlenght of 4MB) i am able to sync 
approx 4-5GB

in an hour. And for small mailboxes this i enough and mostly stable.

But the most difficult problem is, that Outlook sometimes does not 
display a message in INBOX
whereas another device connected via IMAP does list all elements. So 
there are two workarounds,
one is do move this mail item to another folder and back and hopefully 
it gets resynced and displayed.
Another is to delete the .ost cache file and let Outlook resync 
everything. Obviously this is not a way to
go regulary. This problem is a show stopper for me as i can not say for 
sure if my inbox is always properly
synced (not everyone has another device to check consistency). As i 
already saw that behavior with Outlook 2013

being connected via IMAP i cannot blame it on EAS or SOGo.
Yet another issue i saw with EAS and a mailbox with 35+ GB is 
performance when switching context to calendar or
contacts or just searching an element... it all happens at unacceptable 
slowlyness.


I read many complaints with mailboxes that big on the list here and also 
did not succeed to make it run. I suspended implementation

of EAS until it will be more stable or I get openchange running flawlessly.

best,
sascha
--
users@sogo.nu
https://inverse.ca/sogo/lists

Re: [SOGo] Outlook EAS with mailboxes > 3GB

2015-10-11 Thread Alain Abbas 

Hello 
We have the same problem with big mailboxes , the first sync take for ever we 
even crashed the server 
for now we had stopped the deployement of activesync and outlook2013 

Regards


Le Samedi 10 Octobre 2015 13:02 CEST, Peter Beck  a 
écrit:
 Hi there,

I've configured some Outlook 2013 users with EAS. This works great with
smaller mailboxes, no issues (or none of the users told me ;-)

But with mailboxes > 2.5/3GB users are complaining about these issues:

* the initial sync takes "forever" (seems never to complete)
* connection very often drops after a few mails (Outlook displaying
"disconnected" and sometimes does not reconnect automatically)
* some messages are downloaded each time and are multiple times listed
in Outlook
* some attachments are "scrumbled" and listed in plaintext instead as an
attachment

Most (if not all) of these users have lots of folders and messages
inside these folders (mails from 2004 until now...)

The server is hosting about 30 mailboxes (small business), about 10 of
them are connecting with Outlook, the rest is using Thunderbird. On
mobiles active sync works as expected.

I found some advice in bug 3236 [1] and some mailinglist posts [2] but
none of these settings helped. Also tried resyncing some folders or
completely removed and reconfigured accounts with manage-eas.

Tried increasing max_line_length in dovecot from 2MB to 4MB, the timeout
in the apache virtual host is set to 3600. The worker count is set to 30.

The same accounts connected with IMAP sync very fast, no "scrumbled"
attachments.

My question now is, how do I find the optimal settings for EAS are
(MaximumSyncWindowSize, MaximumSyncResponseSize,
WOWatchDogRequestTimeout, SOGoMaximumPingInterval, etc...) ?

I've tried lots of values and nothing seemed to fix these issues. Is
that a trial-and-error solution or are there any optimal settings I can
calculate somehow ?

The server is running Debian Jessie 8.2 with SOGo 2.3.2 / Dovecot
2.2.13. It's a virtual machine with currently 4GB RAM and two
processors, average load never gets higher than 0.5...

but sogo-common is still listed as 2.2.9 - is that intended ? (using the
inverse repo)

ii sogo:amd64 2.3.2-1
ii sogo-activesync 2.3.2-1
ii sogo-common 2.2.9+git20141017-1


Hopefully someone can give me advice how to solve these issues...

Best Regards
Peter


[1]http://www.sogo.nu/bugs/view.php?id=3236
[2]https://lists.inverse.ca/sogo/arc/users/2014-10/msg00015.html
--
users@sogo.nu
https://inverse.ca/sogo/lists
--
Alain Abbas
11 rue Robert Schumann
54500 Vandoeuvre
Tel : +333 83 18 02 70
skype: alain.abbas
-- 
users@sogo.nu
https://inverse.ca/sogo/lists

Re: [SOGo] Outlook EAS with mailboxes > 3GB

2015-10-11 Thread Peter Beck 
On 10/10/2015 09:14 PM, t...@aon.at wrote:

> I assume you have tried to set SOGoMaximumSyncResponseSize. Is there any
> exception in sogo.log?

Hi Thomas,

(replying back to the list)

I could not find any exception but lots of messages complaining about
encoding in the logs. Sometimes Outlook shows "disconnected", but in the
sogo.log i still can see ActiveSync entries from this user a while.

I "played" with SOGoMaximumSyncResponseSize - 512, 1024, 2048, 4096 or
even 6144 but none of them solved that issue. My "feeling" was, that 2MB
had the best results from all of them. But then - how does that interact
with all the other parameters ? That is a bit confusing for me. Just
tried to set SOGoMaximumSyncResponseSize to 2048 (and another time to
4096) and commented everything else regarding ActiveSync I've added to
sogo.conf - did not really help. And yes, I also tried the settings
mentioned under "EAS Tuning" in the documentation.

I' fiddling for hours just trying different combinations but none of
them seems to work for me.

> * connection very often drops after a few mails (Outlook displaying
> "disconnected" and sometimes does not reconnect automatically)
> 
> Are there any excpetions / signal .. in sogo.log when the connection drops?
> What do you see in sogo.log when this happens?  ?SOGoEASDebugEnabled?

Tried EASDebugEnabled, but that's lot of information and mostly it's
just xml/plain mail I can see then (lots of attachments listed in
base64). Did not really help me to find the problem - did not even know
what to look for ;-)

Sometimes outlook shows disconnected while Sogo.log still is listing
attachments in base64. When that is finished it looks like that (without
debugging it's showing just the last line):

--8<--8<--
  ...
  
   mail%2Fpublic
   Email
  
 


Oct 10 21:01:22 sogod [1767]:
<0x0x7fa152a406d0[SOGoActiveSyncDispatcher]> Sleeping 10 seconds while
detecting changes...
Oct 10 21:01:32 sogod [1767]: 62.167.130.32, 192.168.0.14 "POST
/SOGo/Microsoft-Server-ActiveSync?jQAHCBDskHrEKrIy0nMyx5aEJkQLBAADV1A4
HTTP/1.1" 200 0/149 32.932 - - -36K
Oct 10 21:01:36 sogod [1793]:
<0x0x7fa1533c7f30[SOGoActiveSyncDispatcher]> Change detected, we push
the content.
Oct 10 21:01:36 sogod [1793]:
<0x0x7fa1533c7f30[SOGoActiveSyncDispatcher]> EAS - response for device
E49E4F080FA643E383B7FB2C2A69E280: 
http://www.microsoft.com/;>

 2
 
  mail%2Fc828a1012d691056ee42619592fc
  mail%2Fe728a1012d691056ee42619592fc
  mail%2Fee28a1012d691056ee42619592fc
 


Oct 10 21:01:36 sogod [1793]: 192.168.0.203 "POST
/SOGo/Microsoft-Server-ActiveSync?Cmd=Ping=u...@domain.com=E49E4F080FA643E383B7FB2C2A69E280=WindowsOutlook15
HTTP/1.1" 200 144/8902 15.703 - - 0

--8<--8<--

Can't really see any exception (did also grep for [E|e]xception in
sogo.log - not one entry found.

> * some messages are downloaded each time and are multiple times listed in
> Outlook
> * some attachments are "scrumbled" and listed in plaintext instead as an
> attachment
> 
> Is the same message ok on other devices?

As far as I recognized: yes. Even on the same devices it's ok when
getting the messages with IMAP (Tried in webmail, Thunderbird and
Outlook with IMAP). Tried that with two different accounts having that
issue and it was working with both of them when not using EAS.

Regards
Peter
-- 
users@sogo.nu
https://inverse.ca/sogo/lists

Re: [SOGo] Outlook EAS with mailboxes > 3GB

2015-10-11 Thread Peter Beck 
On 10/10/2015 08:22 PM, Alain Abbas wrote:
> for now we had stopped the deployement of activesync and outlook2013 

Hi Alain,

good to know I am not the only one ;-)
I've also switched (at least the big mailboxes) to IMAP and
CalDavSynchronizer (which will hopefully soon get CardDAV support).

But still hoping someone can enlighten us here on the list...

Regards
Peter
-- 
users@sogo.nu
https://inverse.ca/sogo/lists

[SOGo] Outlook EAS with mailboxes > 3GB

2015-10-10 Thread Peter Beck 
Hi there,

I've configured some Outlook 2013 users with EAS. This works great with
smaller mailboxes, no issues (or none of the users told me ;-)

But with mailboxes > 2.5/3GB users are complaining about these issues:

* the initial sync takes "forever" (seems never to complete)
* connection very often drops after a few mails (Outlook displaying
"disconnected" and sometimes does not reconnect automatically)
* some messages are downloaded each time and are multiple times listed
in Outlook
* some attachments are "scrumbled" and listed in plaintext instead as an
attachment

Most (if not all) of these users have lots of folders and messages
inside these folders (mails from 2004 until now...)

The server is hosting about 30 mailboxes (small business), about 10 of
them are connecting with Outlook, the rest is using Thunderbird. On
mobiles active sync works as expected.

I found some advice in bug 3236 [1] and some mailinglist posts [2] but
none of these settings helped. Also tried resyncing some folders or
completely removed and reconfigured accounts with manage-eas.

Tried increasing max_line_length in dovecot from 2MB to 4MB, the timeout
in the apache virtual host is set to 3600. The worker count is set to 30.

The same accounts connected with IMAP sync very fast, no "scrumbled"
attachments.

My question now is, how do I find the optimal settings for EAS are
(MaximumSyncWindowSize, MaximumSyncResponseSize,
WOWatchDogRequestTimeout, SOGoMaximumPingInterval, etc...) ?

I've tried lots of values and nothing seemed to fix these issues. Is
that a trial-and-error solution or are there any optimal settings I can
calculate somehow ?

The server is running Debian Jessie 8.2 with SOGo 2.3.2 / Dovecot
2.2.13. It's a virtual machine with currently 4GB RAM and two
processors, average load never gets higher than 0.5...

but sogo-common is still listed as 2.2.9 - is that intended ? (using the
inverse repo)

ii  sogo:amd64   2.3.2-1
ii  sogo-activesync  2.3.2-1
ii  sogo-common  2.2.9+git20141017-1


Hopefully someone can give me advice how to solve these issues...

Best Regards
Peter


[1]http://www.sogo.nu/bugs/view.php?id=3236
[2]https://lists.inverse.ca/sogo/arc/users/2014-10/msg00015.html
-- 
users@sogo.nu
https://inverse.ca/sogo/lists