Hi Sebastian, I read your interesting thread on blender import improvements. Did you make any progress on this topic? Would this be something that a summer-of-code student can work on as well?
Regards, Paul On 01/05/14 23:00, Sebastian A. Brachi wrote: > Hi all, > this is the first time writing to the list, I'd like to start learning a > lot and hopefully help to improve blender in some areas. > My main interest right now is Blender's import/export pipeline. > Currently I'm making an addon to import a lot of different formats from > different game engines and also rewriting/improving some existing ones like > PSK/PSA and MD5. > > I'd like to research the best possible ways to import to blender; my first > concern besides code style is speed in large or medium files, and I have a > couple questions. I've been reading > http://wiki.blender.org/index.php/User:Ideasman42/ImportExport_TODO , and > the idea of using C/C++ modules is very interesting. Here are some > observations and questions for importing, > > (expect some misconceptions): > > > 1) Python file reading/parsing. > > *Seems fast enough to me in binary data, even with processing many GB's. I > haven't tested XML/text data though (seems many users complain of the obj > importer, but where are the main bottlenecks is unknown to me) > Also best practices for reading seems to improve the speed (see 2)) > > Q: C + Ctypes doesn't seem very good since we don't want to rewrite the > reading part in C if it's done in python, and compile dlls or OS with the > addons, right? But if someone > like me would do it, does it seem like the best option because we can > still keep the mesh building or other data handling in more readable > python? > > Q: Is it worth to investigate Cython/pypy for speed improvements here and > was it used in past addons pre 2.5? > I haven't done any tests so far and I'd like to know opinions first, > haven't found more than a couple threads in the list that mention it, and a > few in BA: > e.g., > http://blenderartists.org/forum/showthread.php?278213-TEST-Cython-performances-test > > > > 2) Python binary data unpack (struct module). > > * This seems to add a lot of overhead, specially in big files. Also best > practices allow for some speedups (like the use of struct.Struct or > list.extend). > Besides the main document in the API for best practices with a few tips > in string handling when importing, I couldn't fine much info. > > Q: Is it worth to start/modify a document for best practices, and also > add benchmarks? Who could I ask to review it? > > * What if Blender could accept/interpret python bytes objects to build > geometry, avoiding the unpacking in python? E.g., reading a face index > buffer all at once, and just passing the count, > type (short in most cases) and bytes object to Blender. In case of Vertex > buffer seems more complicated, since many parameters need to be defined, > such as stride, type of each element, > ignore vertex normals if included, etc. > > Q: Would this be a reasonable option to investigate, or even possible to > do? > > * Another bottleneck is when binary data uses half-floats (used > extensively in game formats for UV data). > The struct module doesn't have support for them (there is a patch > though: http://bugs.python.org/issue11734), so a python > function must be used instead. I'm using this one: > http://davidejones.com/blog/1413-python-precision-floating-point/ which is > pretty slow. > > Q: Is the python function optimal? I couldn't find better ones. > Q: If not, is it feasible to do one of the following? : > a) Implement the patch mentioned above in our blender python > b) Create the function in C and expose it to the python API > Q: If b) is the best option, do you think is an OK task for me, as a > first approach to Blender's C code? (no much experience in C besides > college first year) > > > 3) Python data to blender data (converting lists to C arrays with > mesh.vertices.add, mesh.polygons.add, etc): > > *I've been doing a lot of tests but not much digging into C code. I don't > seem to understand the process very well. > Using ctypes arrays, or array.array don't seem to have any performance > improvement when used in reading the file (it's actually a little slower) > nor building the mesh. > > Q: When using python data to import geometry, the python objects need to > be converted to C arrays and that's the overhead right? > Q: When using c-like objects in python like ctypes array or array.array, > they still need to be converted to C arrays, so that's why performance is > not improved? > Q: What could be done to avoid the conversion without using C? Something > like passing a pointer to a ctypes array instead? > > > Thanks! > Regards > _______________________________________________ > Bf-committers mailing list > [email protected] > http://lists.blender.org/mailman/listinfo/bf-committers _______________________________________________ Bf-committers mailing list [email protected] http://lists.blender.org/mailman/listinfo/bf-committers
