C# multithreading failure

Replies to This Discussion

Permalink Reply by David Rutten on April 17, 2015 at 2:19pm

List.Add probably isn't a thread-safe method. List in general isn't. You should consider using a concurrent collection of some sort.

Permalink Reply by Dimitar Baldzhiev on April 17, 2015 at 3:47pm

I am sorry for asking you to spell it out for me but is your suggestion to try to split the list into 8 separate lists perform a loop with one thread on each list and at the end join the lists together? 

Permalink Reply by David Rutten on April 18, 2015 at 12:31am

There are several ways to approach it. One is to populate one list per thread, then aggregate all lists afterwards. Another solution is to use a concurrent collection, where adding items is threadsafe. Another solution still is to prepare a list or array ahead of time, and assign values to the appropriate index during your loops.

The last method is fast, retains ordinality, and conceptually easy, though it may be harder to implement given the details of the problem at hand.

Permalink Reply by David Rutten on April 18, 2015 at 12:35am

Using a Parallel.Foreach or Task based approach tends not to yield too much improvement when the iterations are many and short. There's too much overhead to make it worth the multithreading. Instead it makes more sense to split the array into N runs and perform each run in a single task. In my (limited) experience the best value for N is about twice the number of logical processors on your machine.

Permalink Reply by Dimitar Baldzhiev on April 17, 2015 at 5:39pm

Well if someone is interested in my benchmarks here is the file with 5 different ways of multithreading with C#, a python option and two ways of doing the same operation with native components. Oddly for me it seems that python multithreading works quicker than everything tested here!

Attachments:

• multithreading.gh, 188 KB

Permalink Reply by Anders Holden Deleuran on April 18, 2015 at 2:25am

A note on profiling. You should be careful with relying too much on the GH profiler when timing code within a script. Especially if you have an output parameter with many items as this may be a larger bottleneck than the actual code! One way of getting around this is to wrap the data in a Python list and read it downstream as an item (see attached for an example).

Attachments:

• multithreading_AHD.gh, 175 KB