On Dec 28, 2011, at 11:29 AM, Jim J wrote: > Thanks Andre! > > The idea works, not so much the implementation though. I'll post what I've > found, perhaps I missed something otherwise it might be something to look > into or at least help others. > > I was able to set up the database schema but I couldn't do it using the > bootstrap Python script. The script imports > calendarserver.tools.bootstrapdatabase and runs main(). The > bootstrapdatabase.py file makes some improper (and un-overrideable) > assumptions. First, CONNECTNAME is assumed to be "_postgres" but on Ubuntu > it's just "postgres" (no leading underscore). Maybe this isn't a bad > assumption on Mac OS X Server? Second it assumes the existence of database > user "caldav" and database "caldav". Unfortunately I'd already committed to > 'caldavd' (trailing 'd') being the username and group for the daemon so that > didn't work either.
See the 'database connectivity' section in this document: http://trac.calendarserver.org/wiki/MultiServerDeployment Basically you just need to set the postgres DSN as appropriate for your setup. Cheers, -dre > > I went into the Python file and changed those. Then it assumed that the SQL > script that it needed to execute was located at > /usr/share/caldavd/lib/python/.../current.sql. However, when I installed > calendarserver using ./run -i that's not where it put the SQL script! It put > it in /usr/local/lib/python2.7/dist-packages/.../current.sql. I ended up > quitting at this point and logged into PostgresSQL as the caldavd user and > manually executed the SQL script. > > Finally I was able to get calendarserver fully running *without SSL > encryption*. When I went to enable SSL, I seem to have tripped over some > crazy bug in the twisted framework. I didn't dig too deep but in Twisted > 11.0.0 there's a file internet/tcp.py. In the Connection class there's an "if > SSL:" and then a bunch of functions. Inside _startTLS there's a line: > > self.fileno = self.socket.fileno > > This line asserts. I don't really know why. Something about > OpenSSH.SSL.Connection doesn't have attribute 'fileno'. Anyway, I gave up at > this point, locked the unencrypted calendarserver to localhost and wrapped it > in stunnel4. That seems to work. That line doesn't seem to exist in Twisted > 10.x or even in 11.1.0 so I'm guessing it was a problem. > > Thanks for the help. > > Jim > > On Tue, Dec 27, 2011 at 7:10 PM, Andre LaBranche <d...@apple.com> wrote: > Forgot to cc the list... See quoted text for answers :) > > -dre > > Sent from my iPhone > > On Dec 27, 2011, at 6:34 PM, Andre LaBranche <d...@apple.com> wrote: > >> We only provision schema if we spawn Postgres. To use an externally managed >> DB, provision schema with calendarserver_bootstrap_database. >> >> Cheers, >> -dre >> >> Sent from my iPhone >> >> On Dec 27, 2011, at 4:33 PM, Jim J <jahose...@gmail.com> wrote: >> >>> Andre, >>> >>> Thanks! That's a very useful link actually. I've been trying to figure out >>> what all the options are. For some reason it didn't catch my attention >>> during my 'grep's. That fixes all the above problems, I can even use -u and >>> -g now. >>> >>> Next problem: >>> I created the caldavd user in PostgreSQL and created the caldavd database >>> (I'm using -u caldavd -g caldavd again). I noticed this connects to the >>> database using the caldavd user. Now I get this error: >>> >>> 2011-12-27 16:17:47-0500 [-] >>> [txdav.common.datastore.util.UpgradeDatabaseSchemaService#warn] Beginning >>> database schema check. >>> 2011-12-27 16:17:47-0500 [-] >>> [txdav.common.datastore.util.UpgradeDatabaseSchemaService#warn] Required >>> schema version: 3. >>> 2011-12-27 16:17:47-0500 [-] Unhandled error in Deferred: >>> 2011-12-27 16:17:47-0500 [-] Unhandled Error >>> Traceback (most recent call last): >>> File >>> "/usr/lib/python2.7/dist-packages/twisted/internet/defer.py", line 388, in >>> errback >>> self._startRunCallbacks(fail) >>> File >>> "/usr/lib/python2.7/dist-packages/twisted/internet/defer.py", line 455, in >>> _startRunCallbacks >>> self._runCallbacks() >>> File >>> "/usr/lib/python2.7/dist-packages/twisted/internet/defer.py", line 542, in >>> _runCallbacks >>> current.result = callback(current.result, *args, **kw) >>> File >>> "/usr/lib/python2.7/dist-packages/twisted/internet/defer.py", line 1076, in >>> gotResult >>> _inlineCallbacks(r, g, deferred) >>> --- <exception caught here> --- >>> File >>> "/usr/lib/python2.7/dist-packages/twisted/internet/defer.py", line 1018, in >>> _inlineCallbacks >>> result = result.throwExceptionIntoGenerator(g) >>> File >>> "/usr/lib/python2.7/dist-packages/twisted/python/failure.py", line 350, in >>> throwExceptionIntoGenerator >>> return g.throw(self.type, self.value, self.tb) >>> File >>> "/usr/local/lib/python2.7/dist-packages/txdav/common/datastore/util.py", >>> line 271, in doUpgrade >>> actual_version = yield sqlTxn.schemaVersion() >>> File >>> "/usr/lib/python2.7/dist-packages/twisted/internet/defer.py", line 1018, in >>> _inlineCallbacks >>> result = result.throwExceptionIntoGenerator(g) >>> File >>> "/usr/lib/python2.7/dist-packages/twisted/python/failure.py", line 350, in >>> throwExceptionIntoGenerator >>> return g.throw(self.type, self.value, self.tb) >>> File >>> "/usr/local/lib/python2.7/dist-packages/txdav/common/datastore/sql.py", >>> line 233, in schemaVersion >>> result = yield self._schemaVersion.on(self) >>> File >>> "/usr/local/lib/python2.7/dist-packages/twext/internet/threadutils.py", >>> line 64, in _oneWorkUnit >>> result = instruction() >>> File >>> "/usr/local/lib/python2.7/dist-packages/twext/enterprise/adbapi2.py", line >>> 155, in <lambda> >>> lambda : self._reallyExecSQL(*args, **kw) >>> File >>> "/usr/local/lib/python2.7/dist-packages/twext/enterprise/adbapi2.py", line >>> 136, in _reallyExecSQL >>> result = self._reallyExecSQL(sql, args, raiseOnZeroRowCount) >>> File >>> "/usr/local/lib/python2.7/dist-packages/twext/enterprise/adbapi2.py", line >>> 130, in _reallyExecSQL >>> self._cursor.execute(sql, args) >>> File >>> "/usr/local/lib/python2.7/dist-packages/txdav/base/datastore/dbapiclient.py", >>> line 76, in execute >>> self.realCursor.execute(sql, args) >>> File "/usr/local/lib/python2.7/dist-packages/pgdb.py", line 259, >>> in execute >>> self.executemany(operation, (params,)) >>> File "/usr/local/lib/python2.7/dist-packages/pgdb.py", line 289, >>> in executemany >>> raise DatabaseError("error '%s' in '%s'" % (msg, sql)) >>> pg.DatabaseError: error 'ERROR: relation "calendarserver" does not >>> exist >>> LINE 1: select VALUE from CALENDARSERVER where NAME = 'VERSION' >>> ^ >>> ' in 'select VALUE from CALENDARSERVER where NAME = 'VERSION'' >>> >>> >>> I would have expected calendarserver to set up its own tables/schema (as it >>> appears to be attempting to do). Can I force it to rebuild the database? >>> >>> Thanks, >>> >>> Jim >>> >>> On Tue, Dec 27, 2011 at 1:40 PM, Andre LaBranche <d...@apple.com> wrote: >>> Hi, >>> >>> Try setting UseDatabase to Postgres, as described here: >>> http://trac.calendarserver.org/browser/CalendarServer/tags/release/CalendarServer-3.1/twistedcaldav/stdconfig.py >>> >>> You will also need to specify DSN which tells calendar server how to >>> connect to postgres. >>> >>> You can also set ServerEnabled to false for memcached to keep calendar >>> server from spawning memcached. >>> >>> I can provide examples once I get back home to a real machine... >>> >>> Cheers, >>> -dre >>> >>> Sent from my iPhone >>> >>> On Dec 27, 2011, at 1:21 PM, Jim J <jahose...@gmail.com> wrote: >>> >>>> All, >>>> >>>> I'm quite new to this calendarserver thing. I'm trying to set up 3.1 (from >>>> tags/releases in Subversion) on a 64-bit Ubuntu 11.10 server installation. >>>> It's been a disaster. I was able to run calendarserver from the SVN >>>> directory using ./run and then I "installed" it using ./run -i. That's >>>> where things went bad. Some issues I've worked around, but I'd like to get >>>> good solutions for. Currently I'm stuck trying to figure out what's going >>>> on with PostgreSQL. Perhaps you guys can clue me in on how to properly fix >>>> the following issues. >>>> >>>> This is how I'm currently executing caldavd. Once it works I'll put it >>>> into init.d and run it as a proper server. The () are because I *was* >>>> running with those flags (see part 1) and have given up on them. >>>> >>>> sudo caldavd -X -L -f <path to caldavd.plist file> (-u caldavd -g caldavd) >>>> >>>> 1. As I understand it, it is good practice to run Linux services that are >>>> exposed to the internet as a restricted user. Given that and existence of >>>> the -u and -g flags for caldavd, I figured those flags would be the right >>>> way to do things. That doesn't work at all. >>>> >>>> The first problem is that caldavd wants to write to >>>> /var/run/ccs_postgresql_xxxxx which it can't do because /var belongs to >>>> root and it attempts to do this after the setuid() call to become the >>>> caldavd user. So, I went into caldavd.plist and changed the "DataRoot" to >>>> a different directory and gave it to caldavd:caldavd. >>>> >>>> Then I tried again. This time it failed because it appears that >>>> calendarserver attempts to start postgresql whether or not the postgresql >>>> service is already running on the local box. To do this, it tries to >>>> setuid() again, this time to ROOT! If the entire point of setting a user >>>> for a service is to restrict it's privileges, why would one of those privs >>>> be to allow setuid to root? That makes no sense. >>>> >>>> So I gave up, now I'm trying to run it as root to see if the 3.1 release >>>> even functions. How do I make this server run as somebody other than root? >>>> >>>> 2. Running as root (without the -u and -g flags shown above) gets the >>>> application started and then it fails attempting to start memcached. I get >>>> the error "cannot listen on tcp port". This is because memcachd is a >>>> system service and is *already running*. Again, why is calendarserver >>>> attempting to start a service that's already running? >>>> >>>> So I stopped memcachd and tried again. >>>> >>>> 3. Calendarserver now successfully starts and connects to memcached but it >>>> cannot connect to postgresql. This I haven't yet worked around. I get the >>>> following error: >>>> 2011-12-27 12:43:11-0500 [-] [txdav.base.datastore.subpostgres#error] >>>> [Failure instance: Traceback: <class 'pg.InternalError'>: could not >>>> connect to server: No such file or directory >>>> Is the server running locally and accepting >>>> connections on Unix domain socket >>>> "/tmp/ccs_postgres_f5e263275ce7de576bd42ca08db4fc56/.s.PGSQL.5432"? >>>> >>>> >>>> /usr/lib/python2.7/dist-packages/twisted/internet/_baseprocess.py:60:maybeCallProcessEnded >>>> >>>> /usr/local/lib/python2.7/dist-packages/txdav/base/datastore/subpostgres.py:92:processEnded >>>> >>>> /usr/lib/python2.7/dist-packages/twisted/internet/defer.py:361:callback >>>> >>>> /usr/lib/python2.7/dist-packages/twisted/internet/defer.py:455:_startRunCallbacks >>>> --- <exception caught here> --- >>>> >>>> /usr/lib/python2.7/dist-packages/twisted/internet/defer.py:542:_runCallbacks >>>> >>>> /usr/local/lib/python2.7/dist-packages/txdav/base/datastore/subpostgres.py:367:gotReady >>>> >>>> /usr/local/lib/python2.7/dist-packages/txdav/base/datastore/subpostgres.py:269:ready >>>> >>>> /usr/local/lib/python2.7/dist-packages/txdav/base/datastore/subpostgres.py:261:produceConnection >>>> >>>> /usr/local/lib/python2.7/dist-packages/txdav/base/datastore/dbapiclient.py:196:connect >>>> /usr/local/lib/python2.7/dist-packages/pgdb.py:482:connect >>>> ] >>>> >>>> Here I'm stuck. Quite clearly calendarserver is expecting to launch the >>>> postgresql service (which, again, doesn't make sense to me), but then it >>>> can't connect when it does. Well, at least I'm assuming calendarserver >>>> successfully launches postgresql because I don't get the setuid() error I >>>> got in part 1 when it tries to setuid() to root in order to launch >>>> postgresql. I can't figure out how to fix this. Any thoughts? >>>> >>>> Thanks for the help, >>>> >>>> Jim >>>> _______________________________________________ >>>> calendarserver-users mailing list >>>> calendarserver-users@lists.macosforge.org >>>> http://lists.macosforge.org/mailman/listinfo.cgi/calendarserver-users >>> > > _______________________________________________ > calendarserver-users mailing list > calendarserver-users@lists.macosforge.org > http://lists.macosforge.org/mailman/listinfo.cgi/calendarserver-users > >
_______________________________________________ calendarserver-users mailing list calendarserver-users@lists.macosforge.org http://lists.macosforge.org/mailman/listinfo.cgi/calendarserver-users
