Ghpython Parallel Component Slower than Native Ghopper Module

Replies to This Discussion

Permalink Reply by Mathias s n on November 18, 2014 at 9:06am

I'm pretty interested in this also. I tried your script on a larger amount of points but it took ridiculously much longer time than the normal meshray component!

Best, Mathias

Permalink Reply by Arend on November 18, 2014 at 12:48pm

Ray intersections are typically very lightweight calculations, but there's overhead (getting the results, kickboxing them in the right order, waiting for threads to complete). When working with large amounts of data, I've found sometimes that actually the GH input output logic takes longer than the buisness logic (especially with C#/Script blocks).

Which is logical, because these take care of all kinds of casting which are not really fun to do yourself. It's a convenience/performance trade off.

My conclusion: The python parallel may not be faster for a lot of non-heavy small operations.

Second conclusion: Where do you want to save time? In my tests - the datatree input/output seem to be most costly. Just executing a C#/Python block 6000 times also takes about 50ms, same amount the ray intersections take.

Third conclusion: This applies to this specific case. I do not exclude that cases exist where this specific setup is faster than the normal C# block. (Big meshes perhaps?)

Permalink Reply by Giulio Piacentino on November 18, 2014 at 12:52pm

I would add that, if multithreading is so beneficial, it would be better David's task to add it to the native XRay component. However, I think that the goal is also to provide multithreading in general while solving the whole graph. Let's see if David joins this conversation and exposes his plans for GH2 :)

Multithreading can be very beneficial but, as you noticed, there are cases where the overhead of the operation of making the task parallel consumes the whole advantage of using several processors.

Giulio
--
Giulio Piacentino
for Robert McNeel & Associates
giulio@mcneel.com

Permalink Reply by David Rutten on November 18, 2014 at 3:40pm

The idea is to rewrite the component solve cascade logic using the Task library, but that won't solve a great deal. Mostly bottlenecks in Grasshopper files are either single components or they are components that logically depend on each other, so throwing all components onto different threads won't actually help much.

The real benefit is to be had by converting the iterations of individual components to multi-threaded tasks. This is a bit more complicated both theoretically and practically, but doable in principle.

The big benefit of doing multi-threading on this level is that basically all the hard work has to be done by me and it has to be done only once. The drawback is that it won't speed up single big jobs like massive solid booleans or heavy mesh smooths, but if those turn out to be serious bottlenecks we can always look into multi-threading individual components at a later date.