On Sat, Jul 3, 2010 at 10:51 PM, Scott Parkerson <scott.parker...@gmail.com>wrote:
> On Sat, Jul 3, 2010 at 4:27 PM, Andres Vargas <zod...@foresightlinux.org > >wrote: > > > > > The high memory usage its cause on the update-system. this process not > use > > the XMLRepo. > > > > Not true, you instantiate it (which _opens and parses the XML repos when > you > do this): > > jobPath = Cache().checkCachedUpdateJob(applyList) > > However, that only happens once during update-system. > MMM you write let me check if that not use th xml read > > > > ¿smerp where you seen the high memory usages its caused by xmlcache ? > > > > > Well, I saw it before the changes you mentioned below were made to > gnome-backend to use update-system. Every package that was updated > instantiated a instance of Cache(). I tracked this down with guppy; making > it a singleton basically "fixed" PK's excessive memory utilization. IIRC, > it's much, much worse during the get-updates phase. > man then i understand the singleton is implemented on XMLCache.py ? are you commit it ? > * Changing the object model. The object model of using dicts of dicts and > > > tuple was a mess. I tried to simpify this by using objects with _slots_ > > for > > > the specific things, and using dicts for things that couldn't be > > predicted. > > > > > > I think about it > > http://wiki.foresightlinux.org/display/DEV/Redesign+Conary+Backend and > > write > > objectives > > > > Ok, a quick review of that page is a good start; however, I seriously doubt > we need to do an ORM (Django or otherwise). First-class objects would be > far > better than just dicts, though. > Jejeje a ORM not necesary but some like that with find( metadata = info) or some like data what return a list of objects. > > > * What I considered a *major* showstopper (and why I think that the > Conary > > > PK backend is currently *fundamentally* broken is this: > > > > > > 1. When asking for updates, PK essentially gets the output from conary > > > updateall --items. This is fine, and is what the command line does when > > you > > > run updateall. The output of this updateall is frozen to disk. This is > > what > > > is presented to the user as the list of packages that are in need of > > > updating. > > > > > > > the method for asking updates is get-updates and its executed by > > gpk-icon-viewer > > > > This "frezze" its contains a Jobs this jobs are package:components what > > will > > updated. and showed... > > > > > Ok, we're on the same page here. > > > > > > > > 2. When *applying* the updates, it then queries the repository *again* > > > using the package names in the list! This results in a new updatejob, > > which > > > is then applied. This updatejob is (a) not the same as ther result from > > > conary updateall --items, and (b) is not in optimal order for applying > to > > > the system. Therefore, most of the time, on large updates, the system > is > > > hosed after using PK to apply a set of updates. > > > > > > > This its fixed! on gnome-packagekit. > > the gnome-packagekit execute a get-updates and for each package do and > > update on the past. NOW()! this phase replaced by update-system... > > > > ref: http://bit.ly/amDZpj > > > > > Alright, I have to admit I have not checked your changes to > gnome-packagekit. That means that I guess a user is forced to take all > changes. I suppose that's OK (and probably preferable since a Conary update > is kind of an all or nothing proposition; if you change it by removing > things, you need to re-resolve your entire update job, which was a problem > when you could individually deselect packages when they were presented to > the user after a get-updates action). > > No problem. for review. Thank to you i can seen the gnome-packagekit problem and implement a empty method of simulates. > > > > > I had rectified this by freezing the updatejob *and* a normalized list > of > > > package objects (sorted) which was hashed. This hash -- not the > > updatejob's > > > trovelist -- was checked against a list built from what the GUI > > persisted. > > > If it matched, then the cached updatejob stored with that list was > > applyed. > > > This was faster, and was more accurate > > > Unfortanately, I lost all of this work. :( It was *almost* a complete > > > rewrite of 60% of the backend. That'll teach me. > > > > > > > > > I have to say the update-system as the same methods and operations what > > updateall do. > > > > what as: > > > > client = ConaryClient(cfg) > > updateItems = client.fullUpdateItemList() > > applyList = [ (x[0], (None, None), x[1:], True) for x in updateItems ] > > updJob = client.newUpdateJob() > > sugMap = client.prepareUpdateJob(updJob,applyList) > > dir = client.applyUpdateJob(updJob) > > updJob.close() > > > > > > this basically as conary updateall do... but with verbose callbacks > > http://hg.rpath.com/conary/file/411f4e71df45/conary/callbacks.py#22 > > > > the unique difference is the fronzen dir for getting the updJob > > > > So the idea of conary backend do a update-system on a single Job its > wrong. > > Because internally the backends download jobs, commit,rollback and > install > > to system. This its showed on /tmp/conarybackend.log when packagekit > > running.(this not show now because i remove the logs as embee explain ( > > http://lists.rpath.org/pipermail/foresight-devel/2010-July/001801.html)) > > > > So if the packagekit backend do the same/equal of conary. i ask to my > self > > ¿what cause the memory usage? other think of conary backend haves and > > conary > > updateall not. It is a alot of loggers in code for debug on callbacks. > > > > So the on the last hack i remove all logs on Callbacks and seen a change > on > > my machine .. on other machine (eMbee not work) so i take care the memory > > usage its not on logs. > > > > > Yeah, logs are probably not the issue. > > > next i try to use profiler like python-gup...@fl:2-devel what seen what > > cause the memory usage ... > > > > > Good luck. Let me know what you find. > > --sgp > _______________________________________________ > Foresight-devel mailing list > Foresight-devel@lists.rpath.org > http://lists.rpath.org/mailman/listinfo/foresight-devel > _______________________________________________ Foresight-devel mailing list Foresight-devel@lists.rpath.org http://lists.rpath.org/mailman/listinfo/foresight-devel
