Re: [CalendarServer-dev] Error with Files backend in Debian Jessie
Thanks for the quick responses. I have finished packaging and testing calendarserver 5.1 for Debian. It should be in the Debian repos in a couple of days. Thanks Rahul, I started working a bit on it, but I will wait for this package and see if encounter any problems. I can then help debug or whatever. That is better than doing twice the packaging work. As for issues I went package to package so I have not yet reached the 5x series. I do not have any issues to report, but have not tested the intermediate packages very well. If I can help with anything let me know. /Fred ___ calendarserver-dev mailing list calendarserver-dev@lists.macosforge.org https://lists.macosforge.org/mailman/listinfo/calendarserver-dev
Re: [CalendarServer-dev] Debian updated package 4.2
Hi Patryk, Fredrik, I wanted to try your repo, but had a certificate problem. Rahul, what is the status of the 5.0 package? Are you working on it? I also wanted to check the status etc the other day, but never got around to finnish the email. I wanted to pick it up again but I was too busy. The certificate is just expired, and I have not got around to renew it. You can still download the packages ? I did not really care to test. (I used a proper startssl certificate as backend for gnupg over smartcard http://article.tree.se/server/identity and I did not want to open that can of worms again) But if there are no news from Rahul or someone else I will give the new calendarserver a go again (but for sid). I've been trying to get the calendarserver to run, but I'm failed. I used the repo from https://svn.calendarserver.org/repository/... (5.1) on debian wheezy. After install of all dependencies and a succfull ./run -s, I was able to start the server. With ./run -i, I perform a system installation. But at the start of caldavd, i got an error ImportError: Twisted requires zope.interface 3.6.0 or later: no module named zope.interface. zope.interface is installed. I think it is a wrong path definition somewhere?! Do you have a tip? Could it be the version ? This is a OLD table I made for calendarserver back in Feb 2012 (2 years ago) http://article.tree.se/server/calendarserver The debian package to be installed is 'python-zope.interface' if that did not change ? I think 4.2 with Postgres backend is not too far behind 5.x ? You could at least just download the debs on https://tree.se/debian/ and install them and play around a bit. setting up the caldavd.plist for your needs etc. IF the devs dissagree, please let me know. I have it at least running but not in use. Or you have to wait and hope I get some time to get to it. I have 2 kids (3 6), work, plus a TON of other projects, that have produced more feedback than the calendarserver :) Was anyone succefull with running a system installation of the calendarserver on debian? For any suggestions I would be very happy I am happy for any colaboration.. :) Depending on your background, if you really know what you are doing you can run from svn, otherwise I would suggest using the debs. BUT disclaimer, before you put real data in the calendarserver, be aware that I am not a debian developer, I just do it for fun, and I just packaged the calendarserver so far. I did not try to do migrations between versions, although I hope that the calendarserver developers take care of that without relying too much on the packager :) /Fred ___ calendarserver-dev mailing list calendarserver-dev@lists.macosforge.org https://lists.macosforge.org/mailman/listinfo/calendarserver-dev
Re: [CalendarServer-dev] Debian updated package 4.2
Thanks Rahul, I did not notice that 5.0 was released. If I can help I will try. Feel free to take any patches and let me know if you need more detailed information. I have them here in a git-buildpackage and *might* have some more details in the commit messages :) When you get 5.0 to unstable I will drop my repo. If anyone need my repo to be online, once 5.0 hits sid, let me know. (do not think there are many people using the 4.2 package.. possibly only me :) ) /Fred On 2013-11-02 19:06, Rahul Amaram wrote: Hi Fredrik, Thanks for the package and all your feedback. I have just bundled calendarserver 5.0 and have taken a couple of patches from your package. There is a bug with files storage backend and I still need to do a more thorough testing of upgrading from previous release. Once that is done, I will push to unstable. Thanks, Rahul. On Friday 15 February 2013 08:42 PM, Fredrik Unger wrote: Hi again, Already found a problem.. Debian centric, not Calendarserver. On 02/15/2013 02:05 PM, Fredrik Unger wrote: Hi, I updated my personal package to 4.2. https://tree.se/debian/ There were three minor changes, fixing a few SQL statements for 0.1.4, adding python-psutil as dependency, debianizing PGSOCKETDIR. ___ calendarserver-dev mailing list calendarserver-dev@lists.macosforge.org https://lists.macosforge.org/mailman/listinfo/calendarserver-dev
Re: [CalendarServer-dev] Exception in recvfd --was: Re: continuing work on FreeBSD ports
Hi, # ls -l /tmp/debug.dat.485* -rw-r--r-- 1 root wheel 0 Feb 13 23:07 /tmp/debug.dat.48563 -rw-r--r-- 1 root wheel 0 Feb 13 23:07 /tmp/debug.dat.48577 -rw-r--r-- 1 root wheel 0 Feb 13 23:07 /tmp/debug.dat.48578 -rw-r--r-- 1 root wheel 0 Feb 13 23:07 /tmp/debug.dat.48579 -rw-r--r-- 1 root wheel 0 Feb 13 23:07 /tmp/debug.dat.48580 # --- I have no explanation for that. The buffers are not flushed [1], I might have added a manual flush after writes, or that it writes as the threads are exited.. eg you stop the server. If I remember correctly in my case a crashed calendarserver did not terminate the calendarserver. Could have been that a /etc/init.d/calendarserver stop stopped the crashed server and flushed the files. Not sure anymore. [1] http://www.gnu.org/software/libc/manual/html_node/Flushing-Buffers.html (Gnu libc is probably not in FreeBSD :) ) /Fred ___ calendarserver-dev mailing list calendarserver-dev@lists.macosforge.org https://lists.macosforge.org/mailman/listinfo/calendarserver-dev
Re: [CalendarServer-dev] Exception in recvfd --was: Re: continuing work on FreeBSD ports
On 02/14/2013 09:56 AM, Axel Rau wrote: No, shutting down the server does not flush them. I did not recognized in the 1st place that you have no close. Closing them on function return and opening in append mode, in case file exist, on next function entry might be the way to go. I will try fflush(3)... Hi, Yes it was a quick hack, as my bug also broke the test at the time I did most debugging there. At the end we could condense the problem to a small testcase and at the end the bug was found.. For the previous bug the problem was in sendmsg.c, I can not really tell if this is the case here. I just figured that FreeBSD might do something different. You should maybe also log the sending side, so you know if what is sent with sendmsg is received.. if that is all ok, then the problem might be elsewhere.. It is a tricky thing and there might be better methods, I just went for the dirty hack to get some more information after looking at the code. /Fred ___ calendarserver-dev mailing list calendarserver-dev@lists.macosforge.org https://lists.macosforge.org/mailman/listinfo/calendarserver-dev
Re: [CalendarServer-dev] Exception in recvfd --was: Re: continuing work on FreeBSD ports
On 02/14/2013 11:37 AM, Fredrik Unger wrote: 2. After the error occured: # cat /tmp/debug.dat.63431 recvmsg: fd [24] [0] result [3] recvmsg: value: d 0 0 0 --- Does this enlighten someone? You could try something like : Index: sendfd.py === --- sendfd.py (revision 10711) +++ sendfd.py (working copy) @@ -15,7 +15,7 @@ # limitations under the License. ## -from struct import pack, unpack +from struct import pack, unpack,calcsize from socket import SOL_SOCKET from twext.python.sendmsg import sendmsg, recvmsg, SCM_RIGHTS @@ -62,5 +62,7 @@ # cmsg_level and cmsg_type really need to be SOL_SOCKET / SCM_RIGHTS, but # since those are the *only* standard values, there's not much point in # checking. +if len(packedFD)!=calcsize(i): +print Error packedFD [%s] (%d) not %d bytes long % (packedFD,len(packedFD),calcsize(ipackedFD,len(packedFD),calcsize(i))) [unpackedFD] = unpack(i, packedFD) return (unpackedFD, data) Not sure if just printing works, might be a better way to log it. Sorry that I can not provide better help. /Fred ___ calendarserver-dev mailing list calendarserver-dev@lists.macosforge.org https://lists.macosforge.org/mailman/listinfo/calendarserver-dev
[CalendarServer-dev] Debian calendarserver 4.1.1 package
Hi, After trying to get 4.1.1 to run on Debian some month ago I decided to bite the bullet and also package it. I am not a debian developer, so it is only available from my own repository : https://tree.se/debian/ DISCLAIMER - USE AT YOUR OWN RISK. It might not be ready for production but I'd like some feedback to the quality of the package. Instructions how to install it is also available on https://tree.se/debian/ The release do not support ipv6, i had to disable the patches.py Runtime error in response to not finding a solution to the problem with --reactor=select as described here : http://lists.macosforge.org/pipermail/calendarserver-dev/2012-November/001568.html Still a new twisted seems to be in the works. It is available in experimental. Debian bug #689983 already requests better IPv6 support. The changelog is : calendarserver (4.1.1-1) unstable; urgency=low * Updated to new released version v4.1.1 * Disabled patches.py until twisted 12.2.0-1 is released * Applied upstream patch for python-sqlparse 0.1.4 ( http://trac.calendarserver.org/ticket/759 ) * Debianized bootstrapdatabase.py * Added caldavd uid/gid to default file -- Fredrik Unger deb...@tree.se Mon, 10 Dec 2012 16:15:11 +0100 It should be just to install, enable in /etc/default/calendarserver, start, and http://localhost:8008/ should be available. It would be iteresting to get some feedback and cooperation from the people running calendarserver on debian, best practices etc that I can put in the documentation. Currently I am running an old instance of the package on my server, and have limited experience with configuring calendarserver for groups etc. I do use LDAP for my installation and will try to document that too. The package is not extensively tested. Sincerely, Fredrik Unger ___ calendarserver-dev mailing list calendarserver-dev@lists.macosforge.org http://lists.macosforge.org/mailman/listinfo/calendarserver-dev
[CalendarServer-dev] [debian] testing trunk
Hi,
This discussion is more debian related, but it might help trunk as well.
This is a list of steps and fixes I made for the packaging of trunk with regards
to the base I have for 4.1.1 which in turn builds on the debian 3.2 package.
First attempt to build trunk as a package failed in dependency on pycalendar
[1].
(upgrading pycalendar to r214 took 3 attempts as it was not clear what branch
to use)
With the dependencies installed the server died with [2]
twext.enterprise.dal.parseschema.ViolatedExpectation: Expected Token.Keyword
got None:
This was fixed with patch [3] (simple whitespace addition).
Now the same error as before appeared.
I also got some information from Peter Mogensen (off list) that detailed that
he saw the same problem but only when the debian package was installed.
(eg, a pure SVN checkout worked, but failed when the debian package was present)
He reported it was totally dependent on the presens
/usr/share/pyshared/twisted/plugins/caldav.py
I did several tests, but at the end I fixed it by removing the RuntimeError
exception. [4] The logging in the patch did not work, it can be left out.
I probably do not use the log as it is meant to be used.
I added a printout in the twext/backport/internet/tcp.py
and it got printed, so I assume that the patching went fine.
Lastly psutil in Debian is version 0.5.1 and version 0.6.0 is needed to
have virtual_memory method. [5]
I have not yet backported the change to 4.1.1 but do not expect problems, I just
wanted to check if the change (removing the exception) is reasonable ?
If ok, I will prepare the 4.1.1 debian package and put it online for testing.
/Fred
[1]
File /usr/lib/python2.7/dist-packages/twistedcaldav/__init__.py, line 70,
in module
PyCalendarProperty.registerDefaultValue(X-CALENDARSERVER-PRIVATE-COMMENT,
PyCalendarValue.VALUETYPE_TEXT)
AttributeError: type object 'PyCalendarProperty' has no attribute
'registerDefaultValue'
[2]
File /usr/bin/twistd, line 14, in module
run()
File /usr/lib/python2.7/dist-packages/twisted/scripts/twistd.py, line 27,
in run
app.run(runApp, ServerOptions)
File /usr/lib/python2.7/dist-packages/twisted/application/app.py, line 647,
in run
config.parseOptions()
File /usr/lib/python2.7/dist-packages/twisted/application/app.py, line 614,
in parseOptions
usage.Options.parseOptions(self, options)
File /usr/lib/python2.7/dist-packages/twisted/python/usage.py, line 261, in
parseOptions
for (cmd, short, parser, doc) in self.subCommands:
File /usr/lib/python2.7/dist-packages/twisted/application/app.py, line 631,
in subCommands
for plug in sorted(plugins, key=attrgetter('tapname')):
File /usr/lib/python2.7/dist-packages/twisted/plugins/caldav.py, line 31,
in getProperty
return getattr(reflect.namedClass(self.serviceMakerClass), propname)
File /usr/lib/python2.7/dist-packages/twisted/python/reflect.py, line 351,
in namedObject
module = namedModule('.'.join(classSplit[:-1]))
File /usr/lib/python2.7/dist-packages/twisted/python/reflect.py, line 339,
in namedModule
topLevel = __import__(name)
File
/usr/lib/python2.7/dist-packages/twistedcaldav/scheduling/imip/mailgateway.py,
line 26, in module
from calendarserver.tap.util import getRootResource, directoryFromConfig
File /usr/lib/python2.7/dist-packages/calendarserver/tap/util.py, line 51,
in module
from twistedcaldav.directory import calendaruserproxy
File
/usr/lib/python2.7/dist-packages/twistedcaldav/directory/calendaruserproxy.py,
line 46, in module
from twistedcaldav.directory.principal import formatLink
File /usr/lib/python2.7/dist-packages/twistedcaldav/directory/principal.py,
line 70, in module
from twistedcaldav.resource import CalendarPrincipalCollectionResource,
CalendarPrincipalResource
File /usr/lib/python2.7/dist-packages/twistedcaldav/resource.py, line 82,
in module
from twistedcaldav.sharing import SharedCollectionMixin, SharedHomeMixin
File /usr/lib/python2.7/dist-packages/twistedcaldav/sharing.py, line 33, in
module
from txdav.common.datastore.sql_tables import _BIND_MODE_OWN, \
File /usr/lib/python2.7/dist-packages/txdav/common/datastore/sql_tables.py,
line 48, in module
schema = _populateSchema()
File /usr/lib/python2.7/dist-packages/txdav/common/datastore/sql_tables.py,
line 44, in _populateSchema
return SchemaSyntax(schemaFromPath(pathObj))
File /usr/lib/python2.7/dist-packages/twext/enterprise/dal/parseschema.py,
line 92, in schemaFromPath
addSQLToSchema(schema, schemaData)
File /usr/lib/python2.7/dist-packages/twext/enterprise/dal/parseschema.py,
line 121, in addSQLToSchema
t = tableFromCreateStatement(schema, stmt)
File /usr/lib/python2.7/dist-packages/twext/enterprise/dal/parseschema.py,
line 76, in tableFromCreateStatement
cp.parse()
File /usr/lib/python2.7/dist-packages/twext/enterprise/dal/parseschema.py,
line 228, in parse
while self.nextColumn():
File
[CalendarServer-dev] Attempt to package 4.1.1 for debian - unofficial
Hi,
As there has not been any upgrades to the debian package, with regards to major
versions I decided to have a go at trying to find out the problems with the
current released version 4.1.1. I am not the debian maintainer, nor a debian
developer.
I am working out some details but one thing I have not been able
to bypass, and would like some advice :
The server fails in :
exceptions.RuntimeError: twisted.internet.addressalready loaded, cannot load
required backport
later other parts fail in :
exceptions.AttributeError: 'module' object has no attribute 'backport'
See backtrace below [1],[2]
I have tried to locate the problem, but I am so far stuck.
The __import__(twext) in caldav.py in twisted/plugins is supposed to
ensure that twisted get update with the new ipv6 modules that
the patches.py in twext is patching ?
Loading the twext.backport module dynamically standalone is possible [3].
I run debian sid with Twisted 12.0.0 but as far as I can tell without
the ipv6 patches that patches.py checks for.
The (wild) ideas I have so far:
- Twisted loads twisted.internet.tcp etc before starting caldav in version
12.0.0 ?
- The many different modules (groupcacher, caldavd, notifications) race to patch
twisted, and once it is done the others fails ?
- Some other package that happens to be in the python path confuses caldavd ?
I assume it is some simple bug, and if you have a hint to the direction
I will try to track it down. It can be that the configuration is out of date,
or a version problem with Debian sid.
I am trying to upgrade the standard configuration as well.
- I also have a vague memory that xmlfile directory is depricated/not up to
date/not recommended ? What is best to use for testing ?
I am using postgres as data backend and have already patches for debian done
(database_bootstrap).
If someone wants to help or try the changes I have made so far let me know.
Once I have a working package I will post the details.
Thank you
Fredrik Unger
[1]
2012-11-13 17:53:59+0100 [-] LimitingInheritingProtocolFactory starting on 8008
2012-11-13 17:54:00+0100 [-] [groupcacher] Unhandled Error
2012-11-13 17:54:00+0100 [-] [groupcacher] Traceback (most recent call last):
2012-11-13 17:54:00+0100 [-] [groupcacher] File
/usr/lib/python2.7/dist-packages/twisted/application/app.py, line 647, in run
2012-11-13 17:54:00+0100 [-] [groupcacher] config.parseOptions()
2012-11-13 17:54:00+0100 [-] [groupcacher] File
/usr/lib/python2.7/dist-packages/twisted/application/app.py, line 614, in
parseOptions
2012-11-13 17:54:00+0100 [-] [groupcacher] usage.Options.parseOptions(self,
options)
2012-11-13 17:54:00+0100 [-] [groupcacher] File
/usr/lib/python2.7/dist-packages/twisted/python/usage.py, line 261, in
parseOptions
2012-11-13 17:54:00+0100 [-] [groupcacher] for (cmd, short, parser, doc) in
self.subCommands:
2012-11-13 17:54:00+0100 [-] [groupcacher] File
/usr/lib/python2.7/dist-packages/twisted/application/app.py, line 631, in
subCommands
2012-11-13 17:54:00+0100 [-] [groupcacher] for plug in sorted(plugins,
key=attrgetter('tapname')):
2012-11-13 17:54:00+0100 [-] [groupcacher] --- exception caught here ---
2012-11-13 17:54:00+0100 [-] [groupcacher] File
/usr/lib/python2.7/dist-packages/twisted/plugin.py, line 213, in getPlugins
2012-11-13 17:54:00+0100 [-] [groupcacher] adapted = interface(plugin, None)
2012-11-13 17:54:00+0100 [-] [groupcacher] File
/usr/lib/python2.7/dist-packages/zope/interface/interface.py, line 631, in
_call_conform
2012-11-13 17:54:00+0100 [-] [groupcacher] return conform(self)
2012-11-13 17:54:00+0100 [-] [groupcacher] File
/usr/lib/python2.7/dist-packages/twisted/plugin.py, line 68, in __conform__
2012-11-13 17:54:00+0100 [-] [groupcacher] return self.load()
2012-11-13 17:54:00+0100 [-] [groupcacher] File
/usr/lib/python2.7/dist-packages/twisted/plugin.py, line 63, in load
2012-11-13 17:54:00+0100 [-] [groupcacher] return
namedAny(self.dropin.moduleName + '.' + self.name)
2012-11-13 17:54:00+0100 [-] [groupcacher] File
/usr/lib/python2.7/dist-packages/twisted/python/reflect.py, line 463, in
namedAny
2012-11-13 17:54:00+0100 [-] [groupcacher] topLevelPackage =
_importAndCheckStack(trialname)
2012-11-13 17:54:00+0100 [-] [groupcacher] File
/usr/lib/python2.7/dist-packages/twisted/python/reflect.py, line 400, in
_importAndCheckStack
2012-11-13 17:54:00+0100 [-] [groupcacher] return __import__(importName)
2012-11-13 17:54:00+0100 [-] [groupcacher] File
/usr/lib/python2.7/dist-packages/twisted/plugins/caldav.py, line 17, in
module
2012-11-13 17:54:00+0100 [-] [groupcacher] __import__(twext) # install
patches before doing anything
2012-11-13 17:54:00+0100 [-] [groupcacher] File
/usr/lib/python2.7/dist-packages/twext/__init__.py, line 21, in module
2012-11-13 17:54:00+0100 [-] [groupcacher] from twext import patches
2012-11-13 17:54:00+0100 [-] [groupcacher] File
/usr/lib/python2.7/dist
Re: [CalendarServer-dev] Attempt to package 4.1.1 for debian - unofficial
Hi, The debian maintainer (Rahul Amaram) did some work not too long ago packaging 3.2. The attempt started from his 3.2 version. Can you attempt to reproduce this problem with trunk? Ok, will do, thanks for the pointer. /Fred ___ calendarserver-dev mailing list calendarserver-dev@lists.macosforge.org http://lists.macosforge.org/mailman/listinfo/calendarserver-dev
