On Fri, Aug 18, 2017 at 10:40 AM, RjOllos <rjol...@gmail.com> wrote:
>
> On Thursday, August 17, 2017 at 3:38:40 PM UTC-4, Jun Omae wrote:
>>
>> Enabling [git] persistent_cache, it checks whether it needs to refresh
>> the cache when Repository.sync() is invoked via explicit
>> synchronization. If explicit synchronization is not set up, new
>> commits in the repository wouldn't be shown.
>>
>> Disabling [git] persistent_cache, it checks whether it needs to
>> refresh the cache each web request. That is `git show-ref` command is
>> invoked each web request. However, new commits in the repository would
>> be shown even if no explicit synchronization.
>
>
> We have a sync_per_request attribute that determines when the db cache is
> synchronized. If you've setup explicit synchronization then I assume you'd
> want persistent_cache = True and sync_per_request = False. If you haven't
> setup explicit synchronization, you'd want persistent_cache = False and
> sync_per_request = True.
>
> Would it make sense to just remove the persistent_cache option and use
> sync_per_request to determine whether the Git revision cache is
> reconstructed on every web request?
I consider that is not good.
GitCachedRepository.sync() checks whether each revision of the
repository is cached in revision table to execute a SELECT query. That
would lead performance down if too many revisions.
However, I think we could improve the sync() for already synchronized
cached git repository, to check each ref is cached before checking
whether all revisions are cached. I agree removing persistent_cache
option if improving sync() by the following patch.
This patch is for 1.0-stable:
diff --git a/tracopt/versioncontrol/git/PyGIT.py
b/tracopt/versioncontrol/git/PyGIT.py
index 47f397389..32aac9184 100644
--- a/tracopt/versioncontrol/git/PyGIT.py
+++ b/tracopt/versioncontrol/git/PyGIT.py
@@ -596,6 +596,11 @@ class Storage(object):
key=lambda (name, rev, head): (not head, name))
return [(name, rev) for name, rev, head in branches]
+ def get_refs(self):
+ for refname, rev in self.rev_cache.refs_dict.iteritems():
+ if refname != 'HEAD':
+ yield refname, rev
+
def get_commits(self):
return self.rev_cache.rev_dict
diff --git a/tracopt/versioncontrol/git/git_fs.py
b/tracopt/versioncontrol/git/git_fs.py
index 031f68c2b..c33eb7a5d 100644
--- a/tracopt/versioncontrol/git/git_fs.py
+++ b/tracopt/versioncontrol/git/git_fs.py
@@ -100,6 +100,21 @@ class GitCachedRepository(CachedRepository):
return count > 0
return False
+ def needs_sync():
+ max_holders = 999
+ revs = sorted(set(rev for refname, rev in repos.git.get_refs()))
+ for idx in xrange(0, len(revs), max_holders):
+ revs_ = revs[idx:idx + max_holders]
+ holders = ','.join(('%s',) * len(revs_))
+ args = [self.id]
+ args.extend(revs_)
+ query = 'SELECT COUNT(*) FROM revision ' \
+ 'WHERE repos=%s AND rev IN (' + holders + ')'
+ for count, in self.env.db_query(query, args):
+ if count < len(revs_):
+ return True
+ return False
+
def traverse(rev, seen):
revs = []
merge_revs = []
@@ -121,9 +136,7 @@ class GitCachedRepository(CachedRepository):
revs[idx:idx] = traverse(rev, seen)
return revs
- while True:
- repos.sync()
- repos_youngest = repos.youngest_rev or ''
+ def sync_revs():
updated = False
seen = set()
@@ -148,9 +161,13 @@ class GitCachedRepository(CachedRepository):
if feedback:
feedback(rev)
- if updated:
- continue # sync again
+ return updated
+ while True:
+ repos.sync()
+ repos_youngest = repos.youngest_rev or ''
+ if needs_sync() and sync_revs():
+ continue # sync again
if meta_youngest != repos_youngest:
with self.env.db_transaction as db:
db("""
--
Jun Omae <jun6...@gmail.com> (大前 潤)
--
You received this message because you are subscribed to the Google Groups "Trac
Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to trac-dev+unsubscr...@googlegroups.com.
To post to this group, send email to trac-dev@googlegroups.com.
Visit this group at https://groups.google.com/group/trac-dev.
For more options, visit https://groups.google.com/d/optout.
