Tue Dec 24 2013 07:03:27 EST from dothebart @ Uncensored

hm, it seems as somewhere in the migrate chain is a 4k limit which can be hit by the seen-records.

my hot guess right now is sendcommand.



Before:

time /usr/sbin/sendcommand migr export > /tmp/blarg.dmp
sendcommand: started (pid=29199) connecting to Citadel server at /var/run/citadel/citadel-admin.socket
200 potzblitz Citadel server ADMIN CONNECTION ready.
migr export
100 Exporting all Citadel databases.
sendcommand: processing ended.

real    2m35.401s
user    0m21.811s
sys     2m5.756s

After:

time ./sendcommand MIGR export > /tmp/out.dmp
sendcommand: started (pid=29014) connecting to Citadel server at /var/run/citadel/citadel-admin.socket
200 potzblitz Citadel server ADMIN CONNECTION ready.
MIGR export
100 Exporting all Citadel databases.
sendcommand: processing ended.

real    0m10.288s
user    0m2.417s
sys     0m1.823s

Plus it now doesn't cut overlong lines anymore.

The second place cutting overlong lines:

/*
 * Import begins here
 */
void migr_do_import(void) {
    StrBuf *Buf;
    XML_Parser xp;
    int linelen;
   
    unbuffer_output();
    Buf = NewStrBufPlain(NULL, SIZ);
    xp = XML_ParserCreate(NULL);
    if (!xp) {
        cprintf("%d Failed to create XML parser instance\n", ERROR+INTERNAL_ERROR);
        return;
    }
    XML_SetElementHandler(xp, migr_xml_start, migr_xml_end);
    XML_SetCharacterDataHandler(xp, migr_xml_chardata);

    CC->dont_term = 1;

    cprintf("%d sock it to me\n", SEND_LISTING);
    unbuffer_output();

    while (CtdlClientGetLine(Buf) >= 0 && strcmp(ChrPtr(Buf), "000")) {
        linelen = StrLength(Buf);
        StrBufAppendBufPlain(Buf, HKEY("\n"), 0);

        if (server_shutting_down)
            break;    // Should we break or return?
       
        if (linelen == 0)
            continue;

        XML_Parse(xp, ChrPtr(Buf), linelen, 0);
    }

    XML_Parse(xp, "", 0, 1);
    XML_ParserFree(xp);
    FreeStrBuf(&Buf);
    rebuild_euid_index();
    rebuild_usersbynumber();
    CC->dont_term = 0;
}

probably also gained speed next to being non-4k limited anymore.

Reply via email to