Jason Tackaberry wrote:
> Modified: trunk/WIP/beacon2/src/client.py
> ==============================================================================
> --- trunk/WIP/beacon2/src/client.py (original)
> +++ trunk/WIP/beacon2/src/client.py Fri Jun 29 22:18:40 2007
> @@ -277,7 +277,11 @@
> self.rpc('db.lock')
> result = self._db.query(**kwargs)
> if isinstance(result, kaa.notifier.InProgress):
> - result.connect(self.rpc, 'db.unlock')
> + cb = kaa.notifier.Callback(self.rpc, 'db.unlock')
> + # Call args will be the result set of the query. We don't want
> to
> + # send this over RPC.
> + cb.set_ignore_caller_args()
> + result.connect(cb)
> else:
> self.rpc('db.unlock')
> return result
Oops
> Modified: trunk/WIP/beacon2/src/db.py
> ==============================================================================
> --- trunk/WIP/beacon2/src/db.py (original)
> +++ trunk/WIP/beacon2/src/db.py Fri Jun 29 22:18:40 2007
> @@ -355,6 +355,8 @@
> parent structure. For files / directories this function won't check
> if they are still there.
> """
> + # FIXME: this function needs optimizing; adds at least 6 times the
> + # overhead on top of kaa.db.query
> result = []
> cache = {}
> counter = 0
I know, but it is not that easy. We need a whole parent tree for all
items and that takes time. The normal kaa.db query returns just an
item and this functions needs to create the whole parent tree for all
items. I tried to make it faster with the internal cache.
> Modified: trunk/WIP/beacon2/src/query.py
> ==============================================================================
> --- trunk/WIP/beacon2/src/query.py (original)
> +++ trunk/WIP/beacon2/src/query.py Fri Jun 29 22:18:40 2007
> @@ -74,6 +74,7 @@
> self._query = query
> self._client = client
> self.monitoring = False
> + # XXX: maybe 'completed' is better than 'valid'?
> self.valid = False
> self.result = []
> self._beacon_start_query(query, False)
> @@ -165,6 +166,9 @@
> wait = kaa.notifier.YieldCallback()
> self._client.signals['connect'].connect_once(wait)
> yield wait
> + emit_signal = True
> + # Since query was async, we need to emit 'changed' signal, so
> + # override emit_signal argument.
>
> if 'parent' in query and isinstance(query['parent'], Item) and \
> not query['parent']._beacon_id:
> @@ -175,10 +179,16 @@
> log.info('force data for %s', parent)
> parent._beacon_request(self._beacon_start_query, query, True)
> return
> +
> self.result = self._client._beacon_query(**query)
> +
> if isinstance(self.result, kaa.notifier.InProgress):
> yield self.result
> self.result = self.result()
> + # Since query was async, we need to emit 'changed' signal, so
> + # override emit_signal argument.
> + emit_signal = True
> +
> self.valid = True
> if emit_signal:
> self.signals['changed'].emit()
I have to think about that. I did not change that part of the code but
I had a reason for doing it this way. :)
> Modified: trunk/WIP/beacon2/src/server/db.py
> ==============================================================================
> --- trunk/WIP/beacon2/src/server/db.py (original)
> +++ trunk/WIP/beacon2/src/server/db.py Fri Jun 29 22:18:40 2007
> @@ -49,7 +49,7 @@
> # get logging object
> log = logging.getLogger('beacon.db')
>
> -MAX_BUFFER_CHANGES = 30
> +MAX_BUFFER_CHANGES = 200
Some notes about this:
1. The client does not get notified on changes when they are done to
the db, they are notified on commit. That is why there is a commit
in the crawler and some other parts. I'm not sure if 200 may be a
too long time.
2. A commit takes time itself. The client with _block_ when the server
does the commit. The client sends a lock() and does a query. If the
server has 199 entries not commited the client has to wait until
the server reads the lock and commit is done. This time should be
as small as possible.
... doing a small test
ok, since the commit function now only calls commit and not the add
and update queued as before I get
db(171): *** db.commit 201 items: 0.00125
on my 800MHz laptop. Sounds good to me.
Dischi
--
Original Pentium of Borg: Division is futile - your decimals will be
approximated.
pgpxKc9qsyTvl.pgp
Description: PGP signature
------------------------------------------------------------------------- This SF.net email is sponsored by DB2 Express Download DB2 Express C - the FREE version of DB2 express and take control of your XML. No limits. Just data. Click to get it now. http://sourceforge.net/powerbar/db2/
_______________________________________________ Freevo-cvslog mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/freevo-cvslog
