Andy, could you provide a link to an example to do some benchmarks? I'd prefer to work with real user cases than a super-subdivided suzzane.
On Sun, Mar 9, 2014 at 8:42 PM, metalliandy <[email protected]>wrote: > I have been praying for a faster obj loader for years! > I use massive objs on a regular basis (up to 900mb) so this would be > amazing. IMHO all I/O plugins should be done in C/C++ as python just > doesn't have the speed. > FWIW, Blender takes about 30mins to export a 31.5m poly mesh while > ZBrush takes around 2 mins for the same mesh. > > Cheers, > > -Andy > On 09/03/2014 19:21, Fredrik hansson wrote: > > hi, > > i > > have recently written a obj loader myself just for fun. now i did try > > to > > optimize it quite a bit by making it multithreaded and a few other > > things. > > i have put it up on https://github.com/FredrikHson/fast_obj_loader > > now > > granted this is a much simpler example than doing everything that > > blender does > > with pushing it into blenders internal structure and all that. > > anyway tried it + blenders current importer. > > some stats > > blender total 17.8 sec. > > 8.6sec. parse time > > > > mine: > > sigle threaded 0.6seonds > > multi threaded 0.26sec > > > > this is on a 36mb obj file on a ssd drive with 8threads. > > > > so yes it could probably benefit from being done in c/c++ the question > is still if its worth the trouble > > i personally never import anything much heavier than that 36mb file > myself due to > > slow viewport/manipulation speeds after actually getting the file into > blender and 18seconds is > > a bit annoying but i don't do it that often that its really much of an > issue. > > what is much worse imo is export speeds ( i often forget export selected > only) when dealing with > > dense subsurfed meshes. > > > > // Fredrik > > > > > > > > > > > > On Saturday, March 8, 2014 3:30 AM, Sebastian A. Brachi < > [email protected]> wrote: > > > > Hi Paul, > > I'm pretty satisfied with the performance of my importers right now, and > > specially compared to max-script. > > Turned out the major bottleneck was from UV data, since I wasn't using > > foreach_set which is a much efficient method than adding uv data in a > loop. > > check > > > http://blenderartists.org/forum/showthread.php?321210-Importing-UV-coordinates > > This is for video game data that usually isn't very heavy though > (although > > I'm also importing whole levels with hundreds of assets, all in ~40 > > seconds), > > and also not using an efficient method for unpacking half-floats (used a > > lot in video games for uvs). > > > > As I said before, I heard users complaining .obj importer performance vs > > zbrush's for example in >500 mb data (sculpts mainly). > > There might be room for improvement there, but consider the topics > > discussed (would be good to rewrite it? is C an better option? can small > > tweaks improve performance considerably?) > > > > Regards > > > > > > > > On Thu, Mar 6, 2014 at 12:26 PM, Paul Melis <[email protected]> > wrote: > > > >> Hi Sebastian, > >> > >> I read your interesting thread on blender import improvements. Did you > >> make any progress on this topic? Would this be something that a > >> summer-of-code student can work on as well? > >> > >> Regards, > >> Paul > >> > >> > >> > >> On 01/05/14 23:00, Sebastian A. Brachi wrote: > >> > >>> Hi all, > >>> this is the first time writing to the list, I'd like to start learning > a > >>> lot and hopefully help to improve blender in some areas. > >>> My main interest right now is Blender's import/export pipeline. > >>> Currently I'm making an addon to import a lot of different formats from > >>> different game engines and also rewriting/improving some existing ones > >>> like > >>> PSK/PSA and MD5. > >>> > >>> I'd like to research the best possible ways to import to blender; my > first > >>> concern besides code style is speed in large or medium files, and I > have a > >>> couple questions. I've been reading > >>> http://wiki.blender.org/index.php/User:Ideasman42/ImportExport_TODO , > and > >>> the idea of using C/C++ modules is very interesting. Here are some > >>> observations and questions for importing, > >>> > >>> (expect some misconceptions): > >>> > >>> > >>> 1) Python file reading/parsing. > >>> > >>> *Seems fast enough to me in binary data, even with processing many > >>> GB's. I > >>> haven't tested XML/text data though (seems many users complain of the > obj > >>> importer, but where are the main bottlenecks is unknown to me) > >>> Also best practices for reading seems to improve the speed (see > 2)) > >>> > >>> Q: C + Ctypes doesn't seem very good since we don't want to > rewrite the > >>> reading part in C if it's done in python, and compile dlls or OS with > the > >>> addons, right? But if someone > >>> like me would do it, does it seem like the best option because > we can > >>> still keep the mesh building or other data handling in more readable > >>> python? > >>> > >>> Q: Is it worth to investigate Cython/pypy for speed improvements > here > >>> and > >>> was it used in past addons pre 2.5? > >>> I haven't done any tests so far and I'd like to know opinions > first, > >>> haven't found more than a couple threads in the list that mention it, > and > >>> a > >>> few in BA: > >>> e.g., > >>> http://blenderartists.org/forum/showthread.php?278213- > >>> TEST-Cython-performances-test > >>> > >>> > >>> > >>> 2) Python binary data unpack (struct module). > >>> > >>> * This seems to add a lot of overhead, specially in big files. > Also > >>> best > >>> practices allow for some speedups (like the use of struct.Struct or > >>> list.extend). > >>> Besides the main document in the API for best practices with a > few tips > >>> in string handling when importing, I couldn't fine much info. > >>> > >>> Q: Is it worth to start/modify a document for best practices, and > also > >>> add benchmarks? Who could I ask to review it? > >>> > >>> * What if Blender could accept/interpret python bytes objects to > build > >>> geometry, avoiding the unpacking in python? E.g., reading a face index > >>> buffer all at once, and just passing the count, > >>> type (short in most cases) and bytes object to Blender. In case of > >>> Vertex > >>> buffer seems more complicated, since many parameters need to be > defined, > >>> such as stride, type of each element, > >>> ignore vertex normals if included, etc. > >>> > >>> Q: Would this be a reasonable option to investigate, or even > possible > >>> to > >>> do? > >>> > >>> * Another bottleneck is when binary data uses half-floats (used > >>> extensively in game formats for UV data). > >>> The struct module doesn't have support for them (there is a > patch > >>> though: http://bugs.python.org/issue11734), so a python > >>> function must be used instead. I'm using this one: > >>> http://davidejones.com/blog/1413-python-precision-floating-point/which > >>> is > >>> pretty slow. > >>> > >>> Q: Is the python function optimal? I couldn't find better ones. > >>> Q: If not, is it feasible to do one of the following? : > >>> a) Implement the patch mentioned above in our blender python > >>> b) Create the function in C and expose it to the python API > >>> Q: If b) is the best option, do you think is an OK task for me, > as a > >>> first approach to Blender's C code? (no much experience in C besides > >>> college first year) > >>> > >>> > >>> 3) Python data to blender data (converting lists to C arrays with > >>> mesh.vertices.add, mesh.polygons.add, etc): > >>> > >>> *I've been doing a lot of tests but not much digging into C code. > I > >>> don't > >>> seem to understand the process very well. > >>> Using ctypes arrays, or array.array don't seem to have any > performance > >>> improvement when used in reading the file (it's actually a little > slower) > >>> nor building the mesh. > >>> > >>> Q: When using python data to import geometry, the python objects > need > >>> to > >>> be converted to C arrays and that's the overhead right? > >>> Q: When using c-like objects in python like ctypes array or > >>> array.array, > >>> they still need to be converted to C arrays, so that's why performance > is > >>> not improved? > >>> Q: What could be done to avoid the conversion without using C? > >>> Something > >>> like passing a pointer to a ctypes array instead? > >>> > >>> > >>> Thanks! > >>> Regards > >>> _______________________________________________ > >>> Bf-committers mailing list > >>> [email protected] > >>> http://lists.blender.org/mailman/listinfo/bf-committers > >>> > >> > > _______________________________________________ > > Bf-committers mailing list > > [email protected] > > http://lists.blender.org/mailman/listinfo/bf-committers > > _______________________________________________ > > Bf-committers mailing list > > [email protected] > > http://lists.blender.org/mailman/listinfo/bf-committers > > > > _______________________________________________ > Bf-committers mailing list > [email protected] > http://lists.blender.org/mailman/listinfo/bf-committers > _______________________________________________ Bf-committers mailing list [email protected] http://lists.blender.org/mailman/listinfo/bf-committers
