I'm sure I can get something to you, mate. Be aware that the files are often 900mb+ though
I will upload something tomorrow. Cheers, -Andy On 10/03/2014 03:12, Sebastian A. Brachi wrote: > Andy, could you provide a link to an example to do some benchmarks? > I'd prefer to work with real user cases than a super-subdivided suzzane. > > > On Sun, Mar 9, 2014 at 8:42 PM, metalliandy > <[email protected]>wrote: > >> I have been praying for a faster obj loader for years! >> I use massive objs on a regular basis (up to 900mb) so this would be >> amazing. IMHO all I/O plugins should be done in C/C++ as python just >> doesn't have the speed. >> FWIW, Blender takes about 30mins to export a 31.5m poly mesh while >> ZBrush takes around 2 mins for the same mesh. >> >> Cheers, >> >> -Andy >> On 09/03/2014 19:21, Fredrik hansson wrote: >>> hi, >>> i >>> have recently written a obj loader myself just for fun. now i did try >>> to >>> optimize it quite a bit by making it multithreaded and a few other >>> things. >>> i have put it up on https://github.com/FredrikHson/fast_obj_loader >>> now >>> granted this is a much simpler example than doing everything that >>> blender does >>> with pushing it into blenders internal structure and all that. >>> anyway tried it + blenders current importer. >>> some stats >>> blender total 17.8 sec. >>> 8.6sec. parse time >>> >>> mine: >>> sigle threaded 0.6seonds >>> multi threaded 0.26sec >>> >>> this is on a 36mb obj file on a ssd drive with 8threads. >>> >>> so yes it could probably benefit from being done in c/c++ the question >> is still if its worth the trouble >>> i personally never import anything much heavier than that 36mb file >> myself due to >>> slow viewport/manipulation speeds after actually getting the file into >> blender and 18seconds is >>> a bit annoying but i don't do it that often that its really much of an >> issue. >>> what is much worse imo is export speeds ( i often forget export selected >> only) when dealing with >>> dense subsurfed meshes. >>> >>> // Fredrik >>> >>> >>> >>> >>> >>> On Saturday, March 8, 2014 3:30 AM, Sebastian A. Brachi < >> [email protected]> wrote: >>> Hi Paul, >>> I'm pretty satisfied with the performance of my importers right now, and >>> specially compared to max-script. >>> Turned out the major bottleneck was from UV data, since I wasn't using >>> foreach_set which is a much efficient method than adding uv data in a >> loop. >>> check >>> >> http://blenderartists.org/forum/showthread.php?321210-Importing-UV-coordinates >>> This is for video game data that usually isn't very heavy though >> (although >>> I'm also importing whole levels with hundreds of assets, all in ~40 >>> seconds), >>> and also not using an efficient method for unpacking half-floats (used a >>> lot in video games for uvs). >>> >>> As I said before, I heard users complaining .obj importer performance vs >>> zbrush's for example in >500 mb data (sculpts mainly). >>> There might be room for improvement there, but consider the topics >>> discussed (would be good to rewrite it? is C an better option? can small >>> tweaks improve performance considerably?) >>> >>> Regards >>> >>> >>> >>> On Thu, Mar 6, 2014 at 12:26 PM, Paul Melis <[email protected]> >> wrote: >>>> Hi Sebastian, >>>> >>>> I read your interesting thread on blender import improvements. Did you >>>> make any progress on this topic? Would this be something that a >>>> summer-of-code student can work on as well? >>>> >>>> Regards, >>>> Paul >>>> >>>> >>>> >>>> On 01/05/14 23:00, Sebastian A. Brachi wrote: >>>> >>>>> Hi all, >>>>> this is the first time writing to the list, I'd like to start learning >> a >>>>> lot and hopefully help to improve blender in some areas. >>>>> My main interest right now is Blender's import/export pipeline. >>>>> Currently I'm making an addon to import a lot of different formats from >>>>> different game engines and also rewriting/improving some existing ones >>>>> like >>>>> PSK/PSA and MD5. >>>>> >>>>> I'd like to research the best possible ways to import to blender; my >> first >>>>> concern besides code style is speed in large or medium files, and I >> have a >>>>> couple questions. I've been reading >>>>> http://wiki.blender.org/index.php/User:Ideasman42/ImportExport_TODO , >> and >>>>> the idea of using C/C++ modules is very interesting. Here are some >>>>> observations and questions for importing, >>>>> >>>>> (expect some misconceptions): >>>>> >>>>> >>>>> 1) Python file reading/parsing. >>>>> >>>>> *Seems fast enough to me in binary data, even with processing many >>>>> GB's. I >>>>> haven't tested XML/text data though (seems many users complain of the >> obj >>>>> importer, but where are the main bottlenecks is unknown to me) >>>>> Also best practices for reading seems to improve the speed (see >> 2)) >>>>> Q: C + Ctypes doesn't seem very good since we don't want to >> rewrite the >>>>> reading part in C if it's done in python, and compile dlls or OS with >> the >>>>> addons, right? But if someone >>>>> like me would do it, does it seem like the best option because >> we can >>>>> still keep the mesh building or other data handling in more readable >>>>> python? >>>>> >>>>> Q: Is it worth to investigate Cython/pypy for speed improvements >> here >>>>> and >>>>> was it used in past addons pre 2.5? >>>>> I haven't done any tests so far and I'd like to know opinions >> first, >>>>> haven't found more than a couple threads in the list that mention it, >> and >>>>> a >>>>> few in BA: >>>>> e.g., >>>>> http://blenderartists.org/forum/showthread.php?278213- >>>>> TEST-Cython-performances-test >>>>> >>>>> >>>>> >>>>> 2) Python binary data unpack (struct module). >>>>> >>>>> * This seems to add a lot of overhead, specially in big files. >> Also >>>>> best >>>>> practices allow for some speedups (like the use of struct.Struct or >>>>> list.extend). >>>>> Besides the main document in the API for best practices with a >> few tips >>>>> in string handling when importing, I couldn't fine much info. >>>>> >>>>> Q: Is it worth to start/modify a document for best practices, and >> also >>>>> add benchmarks? Who could I ask to review it? >>>>> >>>>> * What if Blender could accept/interpret python bytes objects to >> build >>>>> geometry, avoiding the unpacking in python? E.g., reading a face index >>>>> buffer all at once, and just passing the count, >>>>> type (short in most cases) and bytes object to Blender. In case of >>>>> Vertex >>>>> buffer seems more complicated, since many parameters need to be >> defined, >>>>> such as stride, type of each element, >>>>> ignore vertex normals if included, etc. >>>>> >>>>> Q: Would this be a reasonable option to investigate, or even >> possible >>>>> to >>>>> do? >>>>> >>>>> * Another bottleneck is when binary data uses half-floats (used >>>>> extensively in game formats for UV data). >>>>> The struct module doesn't have support for them (there is a >> patch >>>>> though: http://bugs.python.org/issue11734), so a python >>>>> function must be used instead. I'm using this one: >>>>> http://davidejones.com/blog/1413-python-precision-floating-point/which >>>>> is >>>>> pretty slow. >>>>> >>>>> Q: Is the python function optimal? I couldn't find better ones. >>>>> Q: If not, is it feasible to do one of the following? : >>>>> a) Implement the patch mentioned above in our blender python >>>>> b) Create the function in C and expose it to the python API >>>>> Q: If b) is the best option, do you think is an OK task for me, >> as a >>>>> first approach to Blender's C code? (no much experience in C besides >>>>> college first year) >>>>> >>>>> >>>>> 3) Python data to blender data (converting lists to C arrays with >>>>> mesh.vertices.add, mesh.polygons.add, etc): >>>>> >>>>> *I've been doing a lot of tests but not much digging into C code. >> I >>>>> don't >>>>> seem to understand the process very well. >>>>> Using ctypes arrays, or array.array don't seem to have any >> performance >>>>> improvement when used in reading the file (it's actually a little >> slower) >>>>> nor building the mesh. >>>>> >>>>> Q: When using python data to import geometry, the python objects >> need >>>>> to >>>>> be converted to C arrays and that's the overhead right? >>>>> Q: When using c-like objects in python like ctypes array or >>>>> array.array, >>>>> they still need to be converted to C arrays, so that's why performance >> is >>>>> not improved? >>>>> Q: What could be done to avoid the conversion without using C? >>>>> Something >>>>> like passing a pointer to a ctypes array instead? >>>>> >>>>> >>>>> Thanks! >>>>> Regards >>>>> _______________________________________________ >>>>> Bf-committers mailing list >>>>> [email protected] >>>>> http://lists.blender.org/mailman/listinfo/bf-committers >>>>> >>> _______________________________________________ >>> Bf-committers mailing list >>> [email protected] >>> http://lists.blender.org/mailman/listinfo/bf-committers >>> _______________________________________________ >>> Bf-committers mailing list >>> [email protected] >>> http://lists.blender.org/mailman/listinfo/bf-committers >>> >> _______________________________________________ >> Bf-committers mailing list >> [email protected] >> http://lists.blender.org/mailman/listinfo/bf-committers >> > _______________________________________________ > Bf-committers mailing list > [email protected] > http://lists.blender.org/mailman/listinfo/bf-committers > _______________________________________________ Bf-committers mailing list [email protected] http://lists.blender.org/mailman/listinfo/bf-committers
