On 2014-08-28 12:12, Konstantin wrote:
i make search by body and subject, so pilerexport can not help me
here.
I can execute search directly in sphinx and find id of the message.
It would be good to use pilerexport to export by id.
how would you pass 100000 id values to pilerexport?
I have following modifications
sprinx max_matches = 100000
piler iu $config['MAX_SEARCH_HITS'] = 100000;
Will enable more verbose is php-fpm log and see what happen.
ok, be sure to monitor real time resource usage.
Janos
2014-08-28 12:13 GMT+03:00 Janos SUTO <[email protected]>:
On 2014-08-27 23:33, John Kenyon wrote:
I can see why people would want to use the WEBUI to export email,
however Janos is right, for a huge amount of messages it is better
to
use the pilerexport command.
Janos, perhaps to bridge the gap we could look at a dedicated
pilerexport feature in the WEBUI?
Maybe a pilerexport form which auditors/admins have access to, they
input variables for pilerexport, once submitted it then runs an
export
in the background, upon completion piler emails them back with a
link
to download the zip?
This approach would allow people to batch exports, thoughts?
good idea, however pilerexport doesn't use sphinx, so you are limited
to
specifying the sender, recipient, size, and date.
However it might be possible to overcome the sphinx limit using some
sort
of iteration, though.
Janos
Regards,
John Kenyon
-----Original Message-----
From: Janos SUTO [mailto:[email protected]]
Sent: Thursday, 28 August 2014 6:34 AM
To: Piler User
Subject: Re: Download all count
Hello,
recent versions of the gui show you the total number of emails the
search
terms can be found. However sphinx poses a limit on the returned
hits, so
you may get the following response in the middle status bar:
1-50, total: 1000 (27271)
meaning that sphinx found 27271 messages matching the search query,
however sphinx itself returned you 1000 hits only, because of the
sphinx
internal (though configurable) limits, see Q6 in the FAQ.
Also note that downloading 40000 emails in a single zip file
requires huge
amount of resources, not to mention what if the http client times
out, or the
php interpreter decides to end the script for excessing time
limits.
Janos
On 2014-08-27 18:46, Konstantin wrote:
Hi
When i make a search in my piler ui i see ~40 000 results for my
request.
I used 'download all' button to download them in zip file.
The problem is that inside zip file i see only 5 000 - 6 000 eml
file.
Count is different for the same search request. The same problem
for
another search.
piler 0.1.25-master-branch, build 869.
I do not see any errors in syslog, maillog, nginx log.
Is there any limits or known issues?
Thanks.
--
THIS MESSAGE WAS DELIVERED USING 100% RECYCLED ELECTRONS.
!DSPAM:5,53fe40c1155447497421850!
--
THIS MESSAGE WAS DELIVERED USING 100% RECYCLED ELECTRONS.
