Honestly, Justin, that might solve more than one piece of this for me,
and what you're saying makes sense. I'm going to have to apologize for
the wall of text in advance, but I need it to explain what I'm trying to
accomplish.
My code is a lot more intensive than the sample is, but I'm imagining
it's the same general issue--this is my first dive into mesh processing
outside of simple deformers (I'm normally a rigger, so this is the most
complex piece of code I've written as a single function), so I'm not
always entirely sure what I'm doing as I work through it. The system
works great on small meshes, but once I get up to full character scale,
it slows down exponentially, and on a full-res mesh it's actually
crashing my system (after eating up 32 gigs of memory). I've not been
able to find any unnecessary loops, so I don't think I'm running a
flawed algorithm, but breaking it up to give the system a chance to
catch up is probably the solution. After some research, it looks like
the garbage collector might not be getting time to do its job, so I'm
getting a high-water effect from processing each vert, edge, and face.
I'm still not entirely sure how I'm getting a 6 meg .ma file to take up
32+ gigs in memory, but so it goes (and, as I said, the reason I was
putting the progress bars in in the first place, to give myself feedback
on what the system is doing when the memory starts to get eaten). The
reason I think it's the garbage collector is this
article--http://python.dzone.com/articles/diagnosing-memory-leaks-python
. I haven't installed objgraph yet, or heapy, but I'm imagining I'm
going to find lots of small objects that haven't been deallocated.
As I understand it, if I need to proceed sequentially from one task to
another, I don't think things will benefit from breaking them into
individual functions, but then I'm used to lower-level systems (C and
Assembly at the lowest end--I spent my first year in the field
developing new hardware controllers for the games we were developing),
so I'm used to manual memory management, and Garbage Collection might as
well be magic. So, does the event loop work work off of a function
boundary? I also noticed that you're using QTimer. Quick info, just so
I understand, is that similar to .NET's Timer in that it sets up a
second thread under the hood? And, if so, is that the key to making
this work in a way that can help with refreshing? I ask because that's
the only thing I could think of that would benefit from smaller
functions other than code readability. As I understand it, chaining a
series of functions together (or calling one function after the other)
doesn't really change the resulting bytecode.
Here's my use case: I'm writing a tool to serialize a mesh, and then
optionally mirror it intelligently. I'm starting with just working
across the X axis, but from there I plan to add in other modes of
mirroring--arbitrary plane for sure, possibly even radial.
The steps I'm using are like this:
1) Capture the verts.
2) Capture the polygon index arrays
3) Capture the edge smoothness for each edge.
4) Wrap these into a JSON data structure (a dict) for later use.
5) Iterate through the vertices and mirror those that should be
mirrored. Add the mirrored verts to the data structure.
6) Iterate through the polys. Mirror. Add.
7) Iterate through the edges. Mirror smoothness. Add.
Steps 1-4 are in one function currently, and 5-7 are in another. I have
a third function that rebuilds a mesh from the resulting data
structure. As I said, for small meshes it has no issue. The mesh I'm
using for my testing is actually the beast from Hyper-Real creature
Creation (the 2005 masterclasses), modeled by a WETA guy, and sitting at
about 70,000 polys. I figure that if I can make my tool work on it,
it's about as bulletproof as it can get.
That's the first stage. The second stage, once I have the first part
done, is to use the serialized mesh data to build a rig. So, I can
store a database of defined meshes, hand off a base mesh to a modeler,
bring it back in after sculpting, and have my tool build the rig to it
as long as the modeler didn't mess with the topology. I've seen these
systems in the wild, but what I haven't seen (although I'm sure it
exists) is the part I'm really focused on, which is tying a graphical
workflow setup into it, so as a rigger I can add an arbitrary mesh and
then define how it should build the rig, all graphically. All of that,
I'm confident in my ability on, but it all depends on getting this
initial processing to work--the whole point is that I can define the rig
for the portion being mirrored, and then the tool can build the rig for
the entire system, with all necessary mirroring. So, in a nutshell, as
a one-man show, I'm trying to replicate the rigging systems big houses
with a dedicated development team have :). Yep, I'm just crazy that way.
So, you mentioned QThreads. Threads are still an area of voodoo to me.
I work in games, and Unity (the engine my current company uses) actively
works against you if you even try to bring in threading. I get that
they're useful for parallel processing, or not blocking the UI, but I'm
not sure I need them for a sequential series of mesh analyses. If using
them gives the garbage collector/UI the opportunity to recover though,
I'm glad to give it a shot. I'd read that Maya doesn't like threads
either, especially when you're calling the API. I've done basic
threading before (mostly using BackgroundWorker in .NET to handle
keeping the UI responsive), but most of the information I've found on
threading assumes a much higher level of knowledge about it than I have
(I only took a year of actual CS classes before changing majors to
animation, the rest is 10 years of experience teaching myself)--so, it
says "Here's how you do X with Y Language's threading," when I have no
idea what X means or why I'd want to do it. I've always assumed that
when I needed it and had a specific use case, it would make sense. Any
good resources on QThreads (or how the event loop runs) you could point
me towards? Information at that level seems to be sparse from my searches.
In the meantime, I'll try integrating the example you sent back and see
if it makes a difference in the results.
Thanks again!
Joe
On 11/10/2014 1:59 AM, Justin Israel wrote:
Hey Joe,
The thing is, since your example is really trivial, I can really only
comment on the trivial example itself. Normally one wouldn't want to
process a whole bunch of long running/blocking tasks one after the
next, in a single method, while adding and removing widgets from a
layout. The layout is obviously having a drawing issue related to
timing, from everything happening within that same stack.
I can offer a tweak for the example that shows how it could be
corrected, by splitting up your tasks into pieces that allow the stack
to return to the event loop again.
http://pastebin.com/5s5mx6KE
In a more real world example, you might abstract these tasks into
objects emit their own progress (QThread already does this), and then
be able to chain them up so one starts after the next. Or when one is
done, a queue is checked and the next pending task runs. Are these
tasks that can be run in a thread? If so, you can use a thread pool or
QThreads and coordinate with their signals.
Anyways, basically the issue is that the layout is unhappy having
itself modified so much from a single stack.
- Justin
On Mon Nov 10 2014 at 7:03:33 PM Joe Weidenbach <[email protected]
<mailto:[email protected]>> wrote:
Hey All,
I'm having an issue with a tool I'm working on regarding slot-based
widget deletion.
I'm trying to create a UI where my object which handles data
processing
reports it's progress on various (possibly nested) tasks back to the
UI. It's a mesh processor, so it can be pretty intensive. I'm
actually
using these progress bars to try and identify where a memory leak is
right now, so I can track it down, but it would be dead useful to have
for general purposes as well.
I've made up a simpler script to demo the situation, here:
http://pythonfiddle.com/progress-bar-test
The relevant code is in the final class, in the three slots that
connect
to PBTester (lines 84-113). I don't think I need to call
self.update()
on every loop, but it does slow things down for the visual of the
progress bars.
The issue I'm having is that when the processCompleted Signal
fires, it
doesn't seem to actually delete the given progress bar. It's removed
from the layout, but the visual stays in place and gets overlaid
by the
new progress bar. My guess from the research I've done is that
this has
something to do with deleteLater not being called while in the
process,
but I don't understand the event loop well enough yet to make much
sense
of it. The closest page I've been able to find recommended calling
emit() with Qt.QueuedConnection, and I did try that, but I don't
think I
was doing it right, as it didn't seem to do anything. I'm not using
threads either, so I don't think it's a sync issue, but I'm not
sure if
Qt is using them under the hood or not.
Hopefully that makes sense. Any thoughts from the pros?
Thanks,
Joe
---
This email is free from viruses and malware because avast!
Antivirus protection is active.
http://www.avast.com
--
You received this message because you are subscribed to the Google
Groups "Python Programming for Autodesk Maya" group.
To unsubscribe from this group and stop receiving emails from it,
send an email to [email protected]
<mailto:python_inside_maya%[email protected]>.
To view this discussion on the web visit
https://groups.google.com/d/msgid/python_inside_maya/5460552B.3000500%40gmail.com.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google
Groups "Python Programming for Autodesk Maya" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to [email protected]
<mailto:[email protected]>.
To view this discussion on the web visit
https://groups.google.com/d/msgid/python_inside_maya/CAPGFgA3AZ8gG9zLc%3Dsa7aH1ME1P%3D9yp9nSnOR7A0ENKmjVDoiQ%40mail.gmail.com
<https://groups.google.com/d/msgid/python_inside_maya/CAPGFgA3AZ8gG9zLc%3Dsa7aH1ME1P%3D9yp9nSnOR7A0ENKmjVDoiQ%40mail.gmail.com?utm_medium=email&utm_source=footer>.
For more options, visit https://groups.google.com/d/optout.
---
This email is free from viruses and malware because avast! Antivirus protection
is active.
http://www.avast.com
--
You received this message because you are subscribed to the Google Groups "Python
Programming for Autodesk Maya" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/python_inside_maya/5460F261.2050708%40gmail.com.
For more options, visit https://groups.google.com/d/optout.
