On 7/30/14, 2:32 PM, Mike McLean wrote:
On 07/29/2014 05:43 PM, Mike McLean wrote:
These changes cause the hub to use generators for some potentially large
queries internally. The point is to reduce memory usage.
In the case that a large query is returned via rpc, the marshaller has
been extended to handle generators. The marshaller will still create a
giant pile of xml, but we at least still save the memory for the
original data.
I also have some work on an iterating marshaller that will not keep the
entire xml return in memory, but that's a bit more complicated, so I'm
going to leave that for another day.
There are other cases (e.g. repo_init) where we have very large queries
internally. In these cases, this patch set will result in significant
memory savings.
A few more related changes, additional logging and a bugfix.
Also, will probably drop the chunksize patch, doesn't seem to matter
when unless the db connection has a lot of lag.
These changes are working well for me and are likely to land in git soon.
Here's a small patch to fix chain-maven, which can override a G:A:V in
the tag with a build from the chain. Tested locally and it fixes the
regression in chain-maven builds for me.
From 7f391b08bf04e3872c57ede21d651cfb7c4f957b Mon Sep 17 00:00:00 2001
From: Mike Bonnet <[email protected]>
Date: Thu, 31 Jul 2014 19:39:32 -0400
Subject: [PATCH] maven-chain builds can override a G:A:V from the tag with a
previous build in the chain, update the index accordingly
---
hub/kojihub.py | 5 +++--
1 file changed, 3 insertions(+), 2 deletions(-)
diff --git a/hub/kojihub.py b/hub/kojihub.py
index 62220cf..ed17f23 100644
--- a/hub/kojihub.py
+++ b/hub/kojihub.py
@@ -10428,8 +10428,9 @@ class HostExports(object):
archive['artifact_id'], {}).setdefault(
archive['version'], archive['build_id'])
if idx_build != archive['build_id']:
- logger.error("Found multiple builds for
%(group_id)s:%(artifact_id)s:%(version)s. Current build: %(build_id)i", archive)
- logger.error("Indexed build id was %i", idx_build)
+ logger.error("Overriding build for
%(group_id)s:%(artifact_id)s:%(version)s.", archive)
+ logger.error("Current build is %s, new build is %s.",
idx_build, archive['build_id'])
+
maven_build_index[archive['group_id']][archive['artifact_id']][archive['version']]
= archive['build_id']
ignore.extend(task_deps.values())
--
1.8.2.1
--
buildsys mailing list
[email protected]
https://admin.fedoraproject.org/mailman/listinfo/buildsys
