Re: [SOGo] Extracting address books and calendars content
On 2019-06-14 08:58, Christian Mack wrote: Am 14.06.19 um 09:47 schrieb Christian Mack (christian.m...@uni-konstanz.de): Am 13.06.19 um 17:59 schrieb André Rodier (an...@rodier.me): On Thu, 2019-06-13 at 17:33 +0200, Christian Mack wrote: < ... > You know sogo-tool backup? Thanks, Christian, I will use this. I have successfully tested it, and it works. I had trouble the last time I tested, with AppArmor. However, how can I restore a backup for some users, now? When you have a sogo-tool backup for user anna in directory ./backup/ you can list her calendars and addressbooks with: /usr/sbin/sogo-tool restore -l ./backup/ anna This will give you something like this: <0x0x191ea60[SOGoCache]> Cache cleanup interval set every 900.00 seconds <0x0x191ea60[SOGoCache]> Using host(s) 'localhost' as server(s) 2019-06-14 09:31:43.685 sogo-tool[26667] Restorable folders: 2019-06-14 09:31:43.688 sogo-tool[26667] Calendar/2013-5614DA00-3-4EA7FE00 (Import Example) 2019-06-14 09:31:43.688 sogo-tool[26667] Calendar/55D4-57CD2C80-47-7C164080 (Superduper) 2019-06-14 09:31:43.689 sogo-tool[26667] Calendar/c81a287ec3c4719de08fae31306f851d (Pickwick) 2019-06-14 09:31:43.689 sogo-tool[26667] Calendar/e6dd60a375bd4c01a6416cacf763b33a (Normal-Tasks) 2019-06-14 09:31:43.689 sogo-tool[26667] Calendar/personal (Persönlicher Kalender) 2019-06-14 09:31:43.689 sogo-tool[26667] Contacts/collected (Gesammelte Adressen) 2019-06-14 09:31:43.689 sogo-tool[26667] Contacts/personal (Persönliches Adressbuch) If you want to restore calendar "Import Example" to the state of your backup and discard all changes made to it since then, use: /usr/sbin/sogo-tool restore -F Calendar/2013-5614DA00-3-4EA7FE00 ./backup/ anna If you just want to restore accidently deleted events and keep all changes made since your backup, use: /usr/sbin/sogo-tool restore -f Calendar/2013-5614DA00-3-4EA7FE00 ./backup/ anna For addressbooks you just have to exchange the Calendar/... part with Contacts/... one. If the user deleted the whole calendar or address book, you also can restore its settings afterwards with: usr/sbin/sogo-tool restore -p ./backup/ anna This will restore all settings, not just the ones for the restored calendar/addressbook. If you use SIEVE in SOGo, then you have to use: usr/sbin/sogo-tool restore -p -c /etc/sogo/sieve.creds ./backup/ anna This will also restore all SIEVE rules in your IMAP/SIEVE server. I forgot 2 things ;-) 1) After restore operations always restart memcached, or you will get strange effects. 2) If you want to restore all calendars and addressbooks of user anna, use: /usr/sbin/sogo-tool restore -F ALL ./backup/ anna With that you can restore all users with a loop over all users for restoring their calendars and addressbooks. Restarting memcached. Then another loop over all users for restoring their privileges and SIEVE rules. Restarting memcached again. Kind regards, Christian Mack Thanks, Christian, This is very very helpful. I will test and integrate this. Kind regards, André -- https://github.com/progmaticltd/homebox -- users@sogo.nu https://inverse.ca/sogo/lists
Re: [SOGo] Extracting address books and calendars content
Am 14.06.19 um 09:47 schrieb Christian Mack (christian.m...@uni-konstanz.de): > Am 13.06.19 um 17:59 schrieb André Rodier (an...@rodier.me): >> On Thu, 2019-06-13 at 17:33 +0200, Christian Mack wrote: > < ... > >>> >>> You know sogo-tool backup? >>> >> >> Thanks, Christian, I will use this. >> >> I have successfully tested it, and it works. I had trouble the last >> time I tested, with AppArmor. >> >> However, how can I restore a backup for some users, now? >> > > When you have a sogo-tool backup for user anna in directory ./backup/ > you can list her calendars and addressbooks with: > > /usr/sbin/sogo-tool restore -l ./backup/ anna > > This will give you something like this: > > <0x0x191ea60[SOGoCache]> Cache cleanup interval set every 900.00 seconds > <0x0x191ea60[SOGoCache]> Using host(s) 'localhost' as server(s) > 2019-06-14 09:31:43.685 sogo-tool[26667] Restorable folders: > 2019-06-14 09:31:43.688 sogo-tool[26667] > Calendar/2013-5614DA00-3-4EA7FE00 (Import Example) > 2019-06-14 09:31:43.688 sogo-tool[26667] > Calendar/55D4-57CD2C80-47-7C164080 (Superduper) > 2019-06-14 09:31:43.689 sogo-tool[26667] > Calendar/c81a287ec3c4719de08fae31306f851d (Pickwick) > 2019-06-14 09:31:43.689 sogo-tool[26667] > Calendar/e6dd60a375bd4c01a6416cacf763b33a (Normal-Tasks) > 2019-06-14 09:31:43.689 sogo-tool[26667] Calendar/personal > (Persönlicher Kalender) > 2019-06-14 09:31:43.689 sogo-tool[26667] Contacts/collected > (Gesammelte Adressen) > 2019-06-14 09:31:43.689 sogo-tool[26667] Contacts/personal > (Persönliches Adressbuch) > > If you want to restore calendar "Import Example" to the state of your > backup and discard all changes made to it since then, use: > > /usr/sbin/sogo-tool restore -F Calendar/2013-5614DA00-3-4EA7FE00 > ./backup/ anna > > If you just want to restore accidently deleted events and keep all > changes made since your backup, use: > > /usr/sbin/sogo-tool restore -f Calendar/2013-5614DA00-3-4EA7FE00 > ./backup/ anna > > For addressbooks you just have to exchange the Calendar/... part with > Contacts/... one. > > If the user deleted the whole calendar or address book, you also can > restore its settings afterwards with: > > usr/sbin/sogo-tool restore -p ./backup/ anna > > This will restore all settings, not just the ones for the restored > calendar/addressbook. > > If you use SIEVE in SOGo, then you have to use: > > usr/sbin/sogo-tool restore -p -c /etc/sogo/sieve.creds ./backup/ anna > > This will also restore all SIEVE rules in your IMAP/SIEVE server. > I forgot 2 things ;-) 1) After restore operations always restart memcached, or you will get strange effects. 2) If you want to restore all calendars and addressbooks of user anna, use: /usr/sbin/sogo-tool restore -F ALL ./backup/ anna With that you can restore all users with a loop over all users for restoring their calendars and addressbooks. Restarting memcached. Then another loop over all users for restoring their privileges and SIEVE rules. Restarting memcached again. Kind regards, Christian Mack -- Christian Mack Universität Konstanz Kommunikations-, Informations-, Medienzentrum (KIM) Abteilung Basisdienste 78457 Konstanz +49 7531 88-4416 smime.p7s Description: S/MIME Cryptographic Signature
Re: [SOGo] Extracting address books and calendars content
Am 13.06.19 um 17:59 schrieb André Rodier (an...@rodier.me): > On Thu, 2019-06-13 at 17:33 +0200, Christian Mack wrote: < ... > >> >> You know sogo-tool backup? >> > > Thanks, Christian, I will use this. > > I have successfully tested it, and it works. I had trouble the last > time I tested, with AppArmor. > > However, how can I restore a backup for some users, now? > When you have a sogo-tool backup for user anna in directory ./backup/ you can list her calendars and addressbooks with: /usr/sbin/sogo-tool restore -l ./backup/ anna This will give you something like this: <0x0x191ea60[SOGoCache]> Cache cleanup interval set every 900.00 seconds <0x0x191ea60[SOGoCache]> Using host(s) 'localhost' as server(s) 2019-06-14 09:31:43.685 sogo-tool[26667] Restorable folders: 2019-06-14 09:31:43.688 sogo-tool[26667] Calendar/2013-5614DA00-3-4EA7FE00 (Import Example) 2019-06-14 09:31:43.688 sogo-tool[26667] Calendar/55D4-57CD2C80-47-7C164080 (Superduper) 2019-06-14 09:31:43.689 sogo-tool[26667] Calendar/c81a287ec3c4719de08fae31306f851d (Pickwick) 2019-06-14 09:31:43.689 sogo-tool[26667] Calendar/e6dd60a375bd4c01a6416cacf763b33a (Normal-Tasks) 2019-06-14 09:31:43.689 sogo-tool[26667] Calendar/personal (Persönlicher Kalender) 2019-06-14 09:31:43.689 sogo-tool[26667] Contacts/collected (Gesammelte Adressen) 2019-06-14 09:31:43.689 sogo-tool[26667] Contacts/personal (Persönliches Adressbuch) If you want to restore calendar "Import Example" to the state of your backup and discard all changes made to it since then, use: /usr/sbin/sogo-tool restore -F Calendar/2013-5614DA00-3-4EA7FE00 ./backup/ anna If you just want to restore accidently deleted events and keep all changes made since your backup, use: /usr/sbin/sogo-tool restore -f Calendar/2013-5614DA00-3-4EA7FE00 ./backup/ anna For addressbooks you just have to exchange the Calendar/... part with Contacts/... one. If the user deleted the whole calendar or address book, you also can restore its settings afterwards with: usr/sbin/sogo-tool restore -p ./backup/ anna This will restore all settings, not just the ones for the restored calendar/addressbook. If you use SIEVE in SOGo, then you have to use: usr/sbin/sogo-tool restore -p -c /etc/sogo/sieve.creds ./backup/ anna This will also restore all SIEVE rules in your IMAP/SIEVE server. Kind regards, Christian Mack -- Christian Mack Universität Konstanz Kommunikations-, Informations-, Medienzentrum (KIM) Abteilung Basisdienste 78457 Konstanz +49 7531 88-4416 smime.p7s Description: S/MIME Cryptographic Signature
Re: [SOGo] Extracting address books and calendars content
On Thu, 2019-06-13 at 17:33 +0200, Christian Mack wrote: > Hello > > Am 13.06.19 um 11:19 schrieb André Rodier (an...@rodier.me): > > > Am 12.06.19 um 19:18 schrieb André Rodier (an...@rodier.me): > > > > Hello, wonderful SOGo team, > > > > > > > > I need a way to export and import an address book, and maybe a > > > > calendar, from the command line. > > > > > > > > I am interested even if your answer is partial, for instance just the > > > > export. The bare minimum would be exporting an address book in a text > > > > file, in any appropriate format. > > > > > > > > I am happy to write SQL as well if this is necessary, although I would > > > > prefer use the http protocol. > > > > > > > > Also, can I restore, still using the command line, the user's data from > > > > a SOGo backup, or the whole database is the way to go. > > > > > > > > Thanks a lot for your hard work, this is great, and thanks to you and > > > > Debian, my project is taking shape. > > > > > > > > Thanks a lot for your hard work and your insights. > > > > > > On Wed, 2019-06-12 at 21:43 +0200, Anselm Martin Hoffmeister wrote: > > > Hallo André, > > > > > > I guess that CALDAV is the way to go (via HTTPS, preferrably). There are > > > php snippets that demonstrate Web DAV accesses, for example, so you > > > could "roll your own", for both data read and write, and have full > > > flexibility to access single items, while keeping sogo happy with > > > synchronizing changed data to whichever user/program/device needs it. > > > For mere backups though, the SQL database dump would be the obvious > > > choice. > > > > > > I admit that for a certain purpose I was a bit lazy and used a database > > > dump: We need a plain csv file of phone number and associated names for > > > our PABX to display caller names. There is some script that I will not > > > disclose (as it is not pretty at all), but it is fed with the output of > > > > > > echo "SELECT c_content FROM sogoanshoffm00112288562" | mysql -u > > > sogodbuser thesogodb | parsevcardscript > > > > > > where the table name has been hand-picked, obviously, and the mysql uses > > > an options file (which contains the password in this case). > > > > > > The data is - you may have guesses - in vcard format, which to me seems > > > like the shabby brother of an LDAP record - it seems to have been kind > > > of a standard for some time, or rather, a set of "we use it like this" > > > that are mostly compatible. Don't trust much on any application to > > > behave, and with a large range of software and devices accessing your > > > sogo, you will for example have several TEL;TYPE= and EMAIL;TYPE= > > > variants that not every program uses in the same way. YMMV. > > > > > > For restoring the complete dataset, I had to do this with several SOGo > > > instances a while back when changing hardware, and on that occasion > > > doing a full software install instead of just moving the root partition. > > > Just shutting down sogo, exporting the database, importing on the new > > > server and firing up sogo there with the mostly identical config file > > > worked as expected. > > > > > > Best regards > > > Anselm > > > > Hello Anselm, > > > > Thanks for your help. > > > > The final purpose is to backup and restore SOGo from scratch, so maybe > > I will end up restoring the whole database. > > > > Nothing in the method above seems appropriate for me. > > > > Since I want to do the backup on the local machine, http or https is > > not very relevant. What is more, however, is that I need some "special" > > account, with read access to all user's address books and calendars, > > and I did not find anything close to this in the documentation. > > > > Thanks, > > André > > > > You know sogo-tool backup? > > > Kind regards, > Christian Mack > Thanks, Christian, I will use this. I have successfully tested it, and it works. I had trouble the last time I tested, with AppArmor. However, how can I restore a backup for some users, now? Thanks for your help. André -- André Rodier HomeBox: https://github.com/progmaticltd/homebox -- users@sogo.nu https://inverse.ca/sogo/lists
Re: [SOGo] Extracting address books and calendars content
Hello Am 13.06.19 um 11:19 schrieb André Rodier (an...@rodier.me): >> Am 12.06.19 um 19:18 schrieb André Rodier (an...@rodier.me): >>> Hello, wonderful SOGo team, >>> >>> I need a way to export and import an address book, and maybe a >>> calendar, from the command line. >>> >>> I am interested even if your answer is partial, for instance just the >>> export. The bare minimum would be exporting an address book in a text >>> file, in any appropriate format. >>> >>> I am happy to write SQL as well if this is necessary, although I would >>> prefer use the http protocol. >>> >>> Also, can I restore, still using the command line, the user's data from >>> a SOGo backup, or the whole database is the way to go. >>> >>> Thanks a lot for your hard work, this is great, and thanks to you and >>> Debian, my project is taking shape. >>> >>> Thanks a lot for your hard work and your insights. >> >> On Wed, 2019-06-12 at 21:43 +0200, Anselm Martin Hoffmeister wrote: >> Hallo André, >> >> I guess that CALDAV is the way to go (via HTTPS, preferrably). There are >> php snippets that demonstrate Web DAV accesses, for example, so you >> could "roll your own", for both data read and write, and have full >> flexibility to access single items, while keeping sogo happy with >> synchronizing changed data to whichever user/program/device needs it. >> For mere backups though, the SQL database dump would be the obvious choice. >> >> I admit that for a certain purpose I was a bit lazy and used a database >> dump: We need a plain csv file of phone number and associated names for >> our PABX to display caller names. There is some script that I will not >> disclose (as it is not pretty at all), but it is fed with the output of >> >> echo "SELECT c_content FROM sogoanshoffm00112288562" | mysql -u >> sogodbuser thesogodb | parsevcardscript >> >> where the table name has been hand-picked, obviously, and the mysql uses >> an options file (which contains the password in this case). >> >> The data is - you may have guesses - in vcard format, which to me seems >> like the shabby brother of an LDAP record - it seems to have been kind >> of a standard for some time, or rather, a set of "we use it like this" >> that are mostly compatible. Don't trust much on any application to >> behave, and with a large range of software and devices accessing your >> sogo, you will for example have several TEL;TYPE= and EMAIL;TYPE= >> variants that not every program uses in the same way. YMMV. >> >> For restoring the complete dataset, I had to do this with several SOGo >> instances a while back when changing hardware, and on that occasion >> doing a full software install instead of just moving the root partition. >> Just shutting down sogo, exporting the database, importing on the new >> server and firing up sogo there with the mostly identical config file >> worked as expected. >> >> Best regards >> Anselm > > Hello Anselm, > > Thanks for your help. > > The final purpose is to backup and restore SOGo from scratch, so maybe > I will end up restoring the whole database. > > Nothing in the method above seems appropriate for me. > > Since I want to do the backup on the local machine, http or https is > not very relevant. What is more, however, is that I need some "special" > account, with read access to all user's address books and calendars, > and I did not find anything close to this in the documentation. > > Thanks, > André > You know sogo-tool backup? Kind regards, Christian Mack -- Christian Mack Universität Konstanz Kommunikations-, Informations-, Medienzentrum (KIM) Abteilung Basisdienste 78457 Konstanz +49 7531 88-4416 smime.p7s Description: S/MIME Cryptographic Signature
Re: [SOGo] Extracting address books and calendars content
> Am 12.06.19 um 19:18 schrieb André Rodier (an...@rodier.me): > > Hello, wonderful SOGo team, > > > > I need a way to export and import an address book, and maybe a > > calendar, from the command line. > > > > I am interested even if your answer is partial, for instance just the > > export. The bare minimum would be exporting an address book in a text > > file, in any appropriate format. > > > > I am happy to write SQL as well if this is necessary, although I would > > prefer use the http protocol. > > > > Also, can I restore, still using the command line, the user's data from > > a SOGo backup, or the whole database is the way to go. > > > > Thanks a lot for your hard work, this is great, and thanks to you and > > Debian, my project is taking shape. > > > > Thanks a lot for your hard work and your insights. > > On Wed, 2019-06-12 at 21:43 +0200, Anselm Martin Hoffmeister wrote: > Hallo André, > > I guess that CALDAV is the way to go (via HTTPS, preferrably). There are > php snippets that demonstrate Web DAV accesses, for example, so you > could "roll your own", for both data read and write, and have full > flexibility to access single items, while keeping sogo happy with > synchronizing changed data to whichever user/program/device needs it. > For mere backups though, the SQL database dump would be the obvious choice. > > I admit that for a certain purpose I was a bit lazy and used a database > dump: We need a plain csv file of phone number and associated names for > our PABX to display caller names. There is some script that I will not > disclose (as it is not pretty at all), but it is fed with the output of > > echo "SELECT c_content FROM sogoanshoffm00112288562" | mysql -u > sogodbuser thesogodb | parsevcardscript > > where the table name has been hand-picked, obviously, and the mysql uses > an options file (which contains the password in this case). > > The data is - you may have guesses - in vcard format, which to me seems > like the shabby brother of an LDAP record - it seems to have been kind > of a standard for some time, or rather, a set of "we use it like this" > that are mostly compatible. Don't trust much on any application to > behave, and with a large range of software and devices accessing your > sogo, you will for example have several TEL;TYPE= and EMAIL;TYPE= > variants that not every program uses in the same way. YMMV. > > For restoring the complete dataset, I had to do this with several SOGo > instances a while back when changing hardware, and on that occasion > doing a full software install instead of just moving the root partition. > Just shutting down sogo, exporting the database, importing on the new > server and firing up sogo there with the mostly identical config file > worked as expected. > > Best regards > Anselm Hello Anselm, Thanks for your help. The final purpose is to backup and restore SOGo from scratch, so maybe I will end up restoring the whole database. Nothing in the method above seems appropriate for me. Since I want to do the backup on the local machine, http or https is not very relevant. What is more, however, is that I need some "special" account, with read access to all user's address books and calendars, and I did not find anything close to this in the documentation. Thanks, André -- André Rodier HomeBox: https://github.com/progmaticltd/homebox -- users@sogo.nu https://inverse.ca/sogo/lists
Re: [SOGo] Extracting address books and calendars content
Hallo André, I guess that CALDAV is the way to go (via HTTPS, preferrably). There are php snippets that demonstrate Web DAV accesses, for example, so you could "roll your own", for both data read and write, and have full flexibility to access single items, while keeping sogo happy with synchronizing changed data to whichever user/program/device needs it. For mere backups though, the SQL database dump would be the obvious choice. I admit that for a certain purpose I was a bit lazy and used a database dump: We need a plain csv file of phone number and associated names for our PABX to display caller names. There is some script that I will not disclose (as it is not pretty at all), but it is fed with the output of echo "SELECT c_content FROM sogoanshoffm00112288562" | mysql -u sogodbuser thesogodb | parsevcardscript where the table name has been hand-picked, obviously, and the mysql uses an options file (which contains the password in this case). The data is - you may have guesses - in vcard format, which to me seems like the shabby brother of an LDAP record - it seems to have been kind of a standard for some time, or rather, a set of "we use it like this" that are mostly compatible. Don't trust much on any application to behave, and with a large range of software and devices accessing your sogo, you will for example have several TEL;TYPE= and EMAIL;TYPE= variants that not every program uses in the same way. YMMV. For restoring the complete dataset, I had to do this with several SOGo instances a while back when changing hardware, and on that occasion doing a full software install instead of just moving the root partition. Just shutting down sogo, exporting the database, importing on the new server and firing up sogo there with the mostly identical config file worked as expected. Best regards Anselm Am 12.06.19 um 19:18 schrieb André Rodier (an...@rodier.me): > Hello, wonderful SOGo team, > > I need a way to export and import an address book, and maybe a > calendar, from the command line. > > I am interested even if your answer is partial, for instance just the > export. The bare minimum would be exporting an address book in a text > file, in any appropriate format. > > I am happy to write SQL as well if this is necessary, although I would > prefer use the http protocol. > > Also, can I restore, still using the command line, the user's data from > a SOGo backup, or the whole database is the way to go. > > Thanks a lot for your hard work, this is great, and thanks to you and > Debian, my project is taking shape. > > Thanks a lot for your hard work and your insights. -- users@sogo.nu https://inverse.ca/sogo/lists
[SOGo] Extracting address books and calendars content
Hello, wonderful SOGo team, I need a way to export and import an address book, and maybe a calendar, from the command line. I am interested even if your answer is partial, for instance just the export. The bare minimum would be exporting an address book in a text file, in any appropriate format. I am happy to write SQL as well if this is necessary, although I would prefer use the http protocol. Also, can I restore, still using the command line, the user's data from a SOGo backup, or the whole database is the way to go. Thanks a lot for your hard work, this is great, and thanks to you and Debian, my project is taking shape. Thanks a lot for your hard work and your insights. -- André Rodier HomeBox: https://github.com/progmaticltd/homebox -- users@sogo.nu https://inverse.ca/sogo/lists
