Re: [Evolution-hackers] Refactoring EWS GAL sqlitedb
On Wed, 2014-09-03 at 12:04 +0100, David Woodhouse wrote: Any other ideas on how to make this happen efficiently? Hi, I also do not follow SQLite's efficiency in the described situation, but I would expect that the remove insert is significantly slower than rewrite of certain fields (supposing that the actual rewritten value is not longer than the current value, which may move the record to a new place). Such massive database rewrites should be also finished with a 'vacuum;' call, to help with further reads, disk space and so on. Reindexing is also another performance hit in current implementation. Are you able to recognize affected contacts from the binary diff, or from anything after the diff is applied? I can imagine that the transform from MAPI properties to vCard also takes its time, especially with your 200K database, not talking that comparing raw vCards can lead to false positives. Being able to recognize the changes directly from the OAB file would help the most, from my point of view, because then you can update only contact offsets. Maybe PidTagLastModified can be used for this? In any case, I agree that the actual merge of changes with minimal impact on the SQLite database is a good idea to have. Bye, Milan ___ evolution-hackers mailing list evolution-hackers@gnome.org To change your list options or unsubscribe, visit ... https://mail.gnome.org/mailman/listinfo/evolution-hackers
Re: [Evolution-hackers] Refactoring EWS GAL sqlitedb
On Wed, 2014-09-03 at 14:18 +0200, Milan Crha wrote:
Are you able to recognize affected contacts from the binary diff, or
from anything after the diff is applied?
Well... theoretically perhaps. With a binary diff of course a lot of
data from the original file is preserved. So any contact contained
entirely within an unchanged part of the file should be unchanged...
although it *might* have changed its offset within the file¹.
It would be fairly non-trivial to hook into the low-level LZX
decompression code and get a list of '{un,}changed ranges' though.
Especially when we're applying more than one delta at a time.
--
dwmw2
¹ which matters if we're storing the file offset so that we can recreate
the vcard on demand from the original.
smime.p7s
Description: S/MIME cryptographic signature
___
evolution-hackers mailing list
evolution-hackers@gnome.org
To change your list options or unsubscribe, visit ...
https://mail.gnome.org/mailman/listinfo/evolution-hackers
Re: [Evolution-hackers] Refactoring contact editor
On Wed, 2014-09-03 at 07:54 +0200, Milan Crha wrote: On Tue, 2014-09-02 at 21:39 +0200, schaarsc wrote: Are there any objections to do so? Hi, definitely no objection, just the opposite, it'll be appreciated to do the cleanup. By the way, can the expander draw the arrow on the right of its label? It seems to me like a better option, to have a label and after that an option to expand or collapse it. But maybe I'm just used to the custom code. And most important ;-), is someone willing to commit such a patch? Sure, please open a bug report and we can review the patch there. While we're here, do you think it would be a good idea to migrate away from the hard coded list of vcard labels/ids ? If you're going to redo the UI anyway, it might be a good idea to move towards something more flexible, with UI functionalities such as: o Add a new email/phonenumber/address - When adding a new phone number, let the user enter their own label, a user may have one or more HOME phone numbers, etc o Delete an email/phonenumber/address o View all emails/phonenumbers/addresses regardless of whether they are in any hard coded list of labels With the possibility of synchronizing your addressbook with another addressbook (on your mobile or wherever), it would be interesting to support the data which can be found in a given vCard that may not have been created by evolution, in which case it becomes important to read the whole vCard for what it is. However, I suppose there might be localization issues with this, I suppose the hard coded fields that evolution lets you edit have translatable labels in the UI, perhaps there is some not-too-difficult way to allow both (to be honest, it's possible that evolution at least lets you view all of the emails/phonenumbers/addresses, I'm not sure, but at least allowing that would be worthwhile). Anyway, just a passing thought I had while seeing this mail, for your consideration :) Cheers, -Tristan Thanks and bye, Milan ___ evolution-hackers mailing list evolution-hackers@gnome.org To change your list options or unsubscribe, visit ... https://mail.gnome.org/mailman/listinfo/evolution-hackers ___ evolution-hackers mailing list evolution-hackers@gnome.org To change your list options or unsubscribe, visit ... https://mail.gnome.org/mailman/listinfo/evolution-hackers
Re: [Evolution-hackers] Refactoring contact editor
On Wed, 2014-09-03 at 12:55 -0300, Tristan Van Berkom wrote: While we're here, do you think it would be a good idea to migrate away from the hard coded list of vcard labels/ids ? If you're going to redo the UI anyway, it might be a good idea to move towards something more flexible, with UI functionalities such as: o Add a new email/phonenumber/address - When adding a new phone number, let the user enter their own label, a user may have one or more HOME phone numbers, etc o Delete an email/phonenumber/address o View all emails/phonenumbers/addresses regardless of whether they are in any hard coded list of labels With the possibility of synchronizing your addressbook with another addressbook (on your mobile or wherever), it would be interesting to support the data which can be found in a given vCard that may not have been created by evolution, in which case it becomes important to read the whole vCard for what it is. However, I suppose there might be localization issues with this, I suppose the hard coded fields that evolution lets you edit have translatable labels in the UI, perhaps there is some not-too-difficult way to allow both (to be honest, it's possible that evolution at least lets you view all of the emails/phonenumbers/addresses, I'm not sure, but at least allowing that would be worthwhile). Hi, it sounds like Patrick Ohly's [Evolution-hackers] custom labels discussion [1] from this year's spring. That's much more work than just replace custom widgets with a stock expander. See thread [1] for more details. But, if Christian will be willing to dive into it, then I'm all for it and I'll be glad to help him. Bye, Milan [1] https://mail.gnome.org/archives/evolution-hackers/2014-April/msg6.html which continues as https://mail.gnome.org/archives/evolution-hackers/2014-May/msg0.html ___ evolution-hackers mailing list evolution-hackers@gnome.org To change your list options or unsubscribe, visit ... https://mail.gnome.org/mailman/listinfo/evolution-hackers
Re: [Evolution-hackers] Refactoring EWS GAL sqlitedb
On Wed, 2014-09-03 at 15:48 +0100, David Woodhouse wrote:
On Wed, 2014-09-03 at 14:18 +0200, Milan Crha wrote:
Are you able to recognize affected contacts from the binary diff,
or
from anything after the diff is applied?
Well... theoretically perhaps. With a binary diff of course a lot of
data from the original file is preserved. So any contact contained
entirely within an unchanged part of the file should be unchanged...
although it *might* have changed its offset within the file¹.
It would be fairly non-trivial to hook into the low-level LZX
decompression code and get a list of '{un,}changed ranges' though.
Especially when we're applying more than one delta at a time.
Hi,
then I'd go by the PidTagLastModified way, and rely on it in case it's
available. That way you'll update only offsets and changed contacts
(plus drop those removed). Having the PidTagLastModified easily
accessible from the table as such, without a need to parse the vCard,
will be a requirement.
Bye,
Milan
___
evolution-hackers mailing list
evolution-hackers@gnome.org
To change your list options or unsubscribe, visit ...
https://mail.gnome.org/mailman/listinfo/evolution-hackers
Re: [Evolution-hackers] Refactoring EWS GAL sqlitedb
On Wed, 2014-09-03 at 18:22 +0200, Milan Crha wrote: then I'd go by the PidTagLastModified way, and rely on it in case it's available. That way you'll update only offsets and changed contacts (plus drop those removed). Having the PidTagLastModified easily accessible from the table as such, without a need to parse the vCard, will be a requirement. I don't have PidTagLastModified. I have the list below. But I don't think I need it. The sha1 of the binary record ought to suffice. Total records is 238198 prop 681f prop 6804001e prop 68010003 prop 6802001e prop 8c98001e prop 3003001e prop 39fe001f prop 3a19001f prop 800f101f prop 8c8e001f prop 3a06001f prop 8c8f001f prop 3a11001f prop 3a1f prop 8c92001f prop 3001001f prop 8ce30003 prop 8ce20003 prop 8cb5000b prop 806a0003 prop 8cac101f prop 8c6d0102 prop 8cdd000b prop 8ca3 prop 39050003 prop 39ff001e prop 8006001e prop 8c6a1102 prop 3a701102 prop 3a220102 prop 3004001f prop 3a21001f prop 3a2e001f prop 3a1c001f prop 3a23001f prop 3a2f101f prop 3a1b101f prop 3a09001f prop 8011001f prop 8c90001f prop 3a18001f prop 3a30001f prop 8c91001f prop 3a16001f prop 3a17001f prop 3a26001f prop 3a2a001f prop 3a28001f prop 3a27001f prop 3a29001f prop 3a0a001f prop 3a08001f prop 3a4b prop ffe0003 prop 3903 prop 8009101e prop 8008101e prop 8cdb000d prop 8cda000d prop 8cd9000d prop 8cd8000d prop 8cc20102 prop 8c9e0102 prop 68051003 -- dwmw2 smime.p7s Description: S/MIME cryptographic signature ___ evolution-hackers mailing list evolution-hackers@gnome.org To change your list options or unsubscribe, visit ... https://mail.gnome.org/mailman/listinfo/evolution-hackers
Re: [Evolution-hackers] Refactoring EWS GAL sqlitedb
On Wed, 2014-09-03 at 20:20 +0100, David Woodhouse wrote: I don't have PidTagLastModified. I have the list below. But I don't think I need it. The sha1 of the binary record ought to suffice. Hi, I see. Do you compute the hash on your own, or it is provided by the OAB? I do not see such property in yours, neither, as you said, the PidTagLastModified. In any case, anything which will speedup change compare should be better than vCard comparison. prop 681f - PidTagOfflineAddressBookName prop 6804001e - PidTagOfflineAddressBookDistinguishedName prop 68010003 - PidTagOfflineAddressBookSequence prop 6802001e - PidTagOfflineAddressBookContainerGuid prop 8c98001e - PidTagAddressBookHierarchicalRootDepartment prop 3003001e - PR_EMAIL_ADDRESS (as string8) prop 39fe001f - PidTagPrimarySmtpAddress prop 3a19001f - PidTagOfficeLocation prop 800f101f - PidTagAddressBookProxyAddresses prop 8c8e001f - PidTagAddressBookPhoneticGivenName prop 3a06001f - PidTagGivenName prop 8c8f001f - PidTagAddressBookPhoneticSurname prop 3a11001f - PidTagSurname prop 3a1f - PidTagAccount prop 8c92001f - PidTagAddressBookPhoneticDisplayName prop 3001001f - PidTagDisplayName prop 8ce30003 - PidTagAddressBookDistributionListExternalMemberCount prop 8ce20003 - PidTagAddressBookDistributionListMemberCount prop 8cb5000b - PidTagAddressBookModerationEnabled prop 806a0003 - PidTagAddressBookDeliveryContentLength prop 8cac101f - PidTagAddressBookSenderHintTranslations prop 8c6d0102 - PidTagAddressBookObjectGuid prop 8cdd000b - PidTagAddressBookHierarchicalIsHierarchicalGroup prop 8ca3 - PidTagAddressBookSeniorityIndex prop 39050003 - PidTagDisplayTypeEx prop 39ff001e - PR_7BIT_DISPLAY_NAME (as string8) prop 8006001e - PidTagAddressBookHomeMessageDatabase prop 8c6a1102 - PidTagAddressBookX509Certificate prop 3a701102 - PidTagUserX509Certificate prop 3a220102 - PidTagUserCertificate prop 3004001f - PidTagComment prop 3a21001f - PidTagPagerTelephoneNumber prop 3a2e001f - PidTagAssistantTelephoneNumber prop 3a1c001f - PidTagMobileTelephoneNumber prop 3a23001f - PidTagPrimaryFaxNumber prop 3a2f101f - PidTagHome2TelephoneNumbers prop 3a1b101f - PidTagBusiness2TelephoneNumbers prop 3a09001f - PidTagHomeTelephoneNumber prop 8011001f - PidTagAddressBookTargetAddress prop 8c90001f - PidTagAddressBookPhoneticDepartmentName prop 3a18001f - PidTagDepartmentName prop 3a30001f - PidTagAssistant prop 8c91001f - PidTagAddressBookPhoneticCompanyName prop 3a16001f - PidTagCompanyName prop 3a17001f - PidTagTitle prop 3a26001f - PidTagCountry prop 3a2a001f - PidTagPostalCode prop 3a28001f - PidTagStateOrProvince prop 3a27001f - PidTagLocality prop 3a29001f - PidTagStreetAddress prop 3a0a001f - PidTagInitials prop 3a08001f - PidTagBusinessTelephoneNumber prop 3a4b - PidTagSendRichInfo prop 0ffe0003 - PidTagObjectType prop 3903 - PidTagDisplayType prop 8009101e - PidTagAddressBookMember prop 8008101e - PidTagAddressBookIsMemberOfDistributionList prop 8cdb000d - PidTagAddressBookDistributionListRejectMessagesFromDLMembers prop 8cda000d - PidTagAddressBookDistributionListMemberSubmitRejected prop 8cd9000d - PidTagAddressBookUnauthorizedSenders prop 8cd8000d - PidTagAddressBookAuthorizedSenders prop 8cc20102 - PidTagSpokenName prop 8c9e0102 - PidTagThumbnailPhoto prop 68051003 - PidTagOfflineAddressBookTruncatedProperties Bye, Milan ___ evolution-hackers mailing list evolution-hackers@gnome.org To change your list options or unsubscribe, visit ... https://mail.gnome.org/mailman/listinfo/evolution-hackers
