Hello Hugo,
I'm so sorry that the fix took quite some time to be made. Please
get this commit
https://bitbucket.org/jsuto/piler/commits/8685574ede416cfce65fb8b1ebc950764778c898
and it solves this issue.
Janos
On 2021-01-27 16:35, Hugo Alvarez wrote:
Hi Janos,
First thanks for all.
Regarding the python version, sorry for the mistake, we are indeed
using python3 (3.6).
Regarding the problem with the import script, the process is actually
finished correctly, the issue is that the relevant fields in the
'import' database table does not seem to get updated.
Upon creation of the task through the gui, the first run of the
import.sh script will actually import all the emails, but only the
following fields get updated in the database:
status: 2 (from imapfetch.py and then from import.c)
total: abc (from imapfetch.py)
imported: xyz (from import.c)
The problem here is that when imapfech.py runs again (when cron reruns
import.sh), the executed query is:
SELECT id, server, username, password FROM import WHERE started=0
which in turn will return all the import jobs again.
Hope this makes the problem clear.
Thanks
Regards
[1]
Libre de virus. www.avast.com [1]
El mar, 26 ene 2021 a las 20:34, <s...@acts.hu> escribió:
Hello Hugo,
thank you for the pacthes. Have you tried to use imapfetch.py with
python3?
Also can you track what happens or at what point pilerimport stucks?
Anything
relevant in the mail logs?
I've just re-run the import test, and pilerimport has properly
updated
the
import table.
Janos
On 2021-01-25 13:27, Hugo Alvarez wrote:
We have installed Piler 1.3.10 and we cannot get imap importing
(Lotus
Domino server) from GUI. Here is what we have been able to
trackdown
so far:
1 - In read_folder_list we had to decode to utf-8 the case for
the
tupple type. A type missmatch error would occur otherwise:
elif isinstance(folder, type(())):
folder = re.sub(r'\{\d+\}$', '',
folder[0].decode('utf-8')) + folder[1].decode('utf-8')
2 - Domino's Imap server returns a backslash the field separator:
[b'(\\Noinferiors \\HasNoChildren) "\\\\" Inbox', b'(\\Noinferiors
\\HasNoChildren) "\\\\" Trash', b'(\\HasNoChildren) "\\\\"
Drafts',
b'(\\HasNoChildren) "\\\\" Sent']
Which does not match against the exiting regexp in
read_folder_list in
imapfetch.py. This is fixed adding the backslash to the regexp: f
=
re.split(r' \"[\/\.\\]+\" ', folder)
3 - We are using python 2.7.5 under Centos 7.9.2009 which seens to
have a bug in imaplib's _checkquote(), which prevents folder names
with special characters from being correctly escaped making
imapfetch.py to fail. We fixed this by using a sane _checkquote()
version from a previous imaplib. Maybe there is a cleaner solution
...
result.append(_checkquote(conn, f[1]))
4 - In process_folder, empty emails will fail when trying to write
to
disk. This is fixed by checking for a tupple:
for num in data[0].split():
rc, data = conn.fetch(num, '(RFC822)')
if opts['verbose']:
print(rc, num)
if isinstance(data[0], tuple):
opts['counter'] += 1
with open("{}.eml".format(opts['counter']), "wb")
as
f: f.write(data[0][1])
We could make it work with these changes, but pilerimport -T does
not
seem to mark the fields 'updated' or 'finished' in the import
table,
so subsequent executions of import.sh starts the whole process
again
(failing the import due to duplicates).
Links:
------
[1]
https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=webmail
