Possible to add background thread to a custom component

Replies to This Discussion

Permalink Reply by Damien Alomar on May 5, 2010 at 12:30pm

All in all, I think it would be possible, but the question is at what cost. I think that maybe a nifty caching of the results of the text file might be more useful than trying to juggle threads. Threading is not for the faint of heart and is quite a significant understanding. However, I don't know what your dealing with, so that may be the way to go about it.

Your component if its "finished" has to supply some sort of results that are then used downstream. AFAIK there isn't a way to "prevent" down stream components from calculating until your finished. They have to get some sort of information or else they'll just be waiting. Considering how the results of those components are likely to be invalid until the information gets calculated, it may be better off supplying them with nulls until you have some actual information to give them.

Anyway, I think that you should think very closely about the structure of your routine, and specifically how it will interact and update itself. The way I'm thinking about it now is that there really isn't anything that's done in the "solve instance" function if you will. Essentially the "solve instance" function would either A) start the reading of the file if no data is found, or B) output some data if it is found. This is an extreme undersimplification, but the simpler you keep this the more likely this will work. Here are a few more "details", i guess, of how I could see this potentially working...

Thread A - Initial call to Solve Instance function
+ Check and see if there are any results that exist from reading your file - at this point there shouldn't be. These results should be stored in some sort of class variable that is accessible to both threads. It might also be a good idea to have some boolean flag that will also be accessible that represents whether your reading/writing those variables.
+ Fire a function in another thread that begins the read process. Note that you'll likely have to do this through a delegate and an invoke call, but I'm not 100% sure
+ Fill in some null values for the variables you must supply
+ Output the nulls, thus finishing the Solve Instance function

Thread B - File Read Function running in separate thread
+ Open up the file. Note that its probably a good idea just to pass the file path (as a string) between the different threads. Leave the creation of the file/text stream to the one thread that's using it.
+ Perform all the necessary reading from the file
+ Copy all your data to the variables that are accessible to both threads.
+ Expire either the solution on either the component in question or (at last resort) the whole canvas. I know expiring the whole canvas is defenitely possible, but it should be possible to just expire the one component that's doing the reading.

Thread A - "Second" call to Solve Instance after being manually expired
+ Check and see if there are any results that exist from reading your file, which there now should be.
+ Output those shared results
+ Clear the last results (or cache them in some way) so that the next time the Solve Instance function is fired, you don't find any results and reread the file.

I think there are a few variations to this that could happen too, including having a separate function for reading and writing through the data that's called using its own delegate/invoke call to make sure that its extra safe.

If you haven't already, you should really look into event driven programming, delegates, and asyncronous messaging. These are going to be the 3 things that you'll need to have a decent hold on to make sure this things works. Just to let you know, debugging these things can be a bitch.

Permalink Reply by Steve Lewis on May 5, 2010 at 12:55pm

Awesome reply. Some initial comments, I wont be able too touch the code until later this evening so I may have some other quesitons/updates later.

what you say makes sense and i have tried using an invoke/delegate statement so I guess im on the right path.

One thing I note, i thought expire solution is only available for the canvas. I dont think i tried using it at the component level. Initially on entering the solve instance of the component, i always clear any existing data in the output parameter which will eventually contain the data read from a file, and then the component phase is set to computed/blank. So may be setting an initial null value instaed of clearing "out right" the output parameter data might have a benificial effect on the recalculaiton action of the downstream component? I also want to keep the last results in the output parameter of the component so that it can be used by other components - i anticipate i will set a boolean as to whether to re-read a file.

It would be really handy if there was documentation as to the whole solution process with respect to the change in phase of upstream and down stream components, and effect of solution expired on component/canvas level on any upstream/down stream components.

I.e. initialize parameters, component phase = computing, do somehting phase = computing. exit solve instance phse = computed/blank

Permalink Reply by Damien Alomar on May 5, 2010 at 10:19pm

I'm not sure exactly what the method is, but I *believe* that there is a way to expire the solution. I could be wrong there, but David mentioned it a number of months ago, and I believe that Guilio uses it in some of his C# tools, so I might check those out.

As I understand it (pieced together from various smaller explanations from David), the canvas actually "solves" itself from right to left as opposed to left to right (or searches upstream as opposed to search downstream). This isn't to say that a downstream component tries to calculate itself before an upstream component has, just that seeking through the document is done by finding if any dependants have expires as opposed to an expired component telling its dependants. After the component which has expired, yet has no expired dependants is found, the Solve Instance function is called. This goes down the line until all the components have been solved, and there you go. I'm pretty sure that I'm completely misinformed, but until David (or someone with some more knowledge that me) lays the whole process on the table I hope that the description helps.

You can keep the results from the last time, but keep in mind that if the SolveInstance function has been called, GH doesn't care what you output from it. What I mean by that is that it doesn't matter if you supply nothing or exactly the same value as before, it will be considered a new solution and all downstream components will recalculate based on that data. So I would advise against resupplying the old data, since that will force a recalculation that will just slow down the process. It will be replaced in n seconds anyway, and the user just saw that solution, so what's the use.

The last thing is that 0.7 is changing a bunch more stuff. I doubt what you're doing will be completely trash or unusable, just be aware that you're on "shakey ground" and things might change.

Permalink Reply by Steve Lewis on May 6, 2010 at 10:26am

I can’t seem to get this to work. Very frustrating.

I’m able to get the background thread B to read data from a file and output the data back into the component. (I do this by passing a value reference for the "Da" property of the component to the thread B method that is taken from the solve instance method). I update the data in the component by launching the solve instance again within Thread B using the statement "Me.Da" - where Me refers to the current component class which is reading/writing data.

The problem is, I cannot for the life of me, force a connected downstream element (Component 2) to not recalculate until the process i) Thread A (first solve instance run) -ii) Thread B (read data store to common data container object) - iii) Thread A (write data to output parameter) has finished in Component 1. It seems as soon as Thread A step i) has finished, the component will start calculating and store a null reference value as the data has not been updated.

I have tried Me.ExpireSolution(arg) on the Component 1 object within both stage i) ii) and iii). However, it doesn’t appear to do anything and often hangs the solution totally. This usually happens when I use arg = True.

Some things I'd like to try but can’t seem to do.

a) call method Da form a direct external reference to component 1 at the end of thread A Thread B Thread A through finding the object in the canvas form its GUID. i.e. dim obj as ghcomponent; obj.Da.setdata(arg); this doesn’t seem to be possible.
b) expire the solution from a direct external reference to component 1; again this doesn’t seem to be possible
c) Disable the component until end of stage iii)

Permalink Reply by David Rutten on May 6, 2010 at 1:22pm

Hi Steve,

I'm a little fuzzy on what exactly you are trying to accomplish with the threading. Either the tread processes data and you wait for it to finish or the thread processes data in the background while you allow the solution to complete with null data.

In the former case the Grasshopper GUI will be frozen, in the latter case the solution will be peppered with nulls since it's running on -as of yet- missing data.

While a solution is running inside Grasshopper a lot of things cannot (or rather, should not) be messed with. Me.ExpireSolution(bool) can only effectively be called when there isn't a solution running already.

If you really must use a thread to process data while you keep the Grasshopper GUI alive, then the approach would be something like the following:

1) Your component is called upon to Solve itself for a Grasshopper Solution.

2) It determines that a lot of data needs to be handled, so it starts a thread to do the heavy lifting and yields execution right back at Grasshopper, which then continues with null data (you'll get a lot of orange components).

36) Your thread is done processing data, it now wants to insert this data into the Grasshopper network, so it must Expire the original component, but it can only do this if no solution is running. So, it has to loop indefinitely until it finds a moment when the GH_Document.SolutionState is not GH_ProcessStep.Process. It should also check to see whether the GH_Document that contains the Component is in fact still in existence and active and whether or not the original component is still available or if it's been deleted in the mean time.

37) The thread must now Invoke a method on the Grasshopper GUI thread that will expire your component and initiate a new solution. If you call ExpireSolution(True) from a background thread, you will crash. Not just you, you will also crash me and probably Rhino. You can never, never ever call code that results in UI changes from a thread other than the main UI thread.

38) Your component has now been expired and a new solution is running. It must have some mechanism in place which tells it whether it should start a new Background thread (a la point #2) or whether the thread it started last time is now ready for harvest. If the latter, extract the data from the thread and fill out the output parameters.



I'm having a hard time figuring out what the real benefit to this would be. Threading is difficult. Really difficult. It's even more difficult when you have to deal with UI threads. It's even more difficult when you're running inside an application that is subject to many events and routinely goes into and out of states that prohibit certain actions.

--
David Rutten
david@mcneel.com
London, UK

Permalink Reply by Steve Lewis on May 6, 2010 at 2:20pm

Thank you so much for the reply David. I dont want to go down the route of threading to be honest but I dont see the alternative if I am working with a huge data file. May be I’m doing things a** backward.

If my component is reading this data and I don’t allow this process to act in the background, it seems that I will be left with an unresponsive GH canvas while the component is chugging through the data. The other option is to stick with the back ground thread method as it is working for me now ,and just assume that before connecting the output parameters with another component, the user has to wait until the data is up to date. I guess I can add a Boolean flag to say if it is completed

The problem is I cannot rerun a later solution for, components 1 and 2 connected, and get the downstream component (component 2) to not read the data until component 1 has finished updating.

It seems to be working fine at the moment if I accept this limitation. However, it seems that it will require the user to connect and un-connect the wires every time a new solution is started and a new file needs to be processed. (Is there a gate component that can be used to open and close the flow of data along a wire based on a Boolean flag? Actually maybe I can use the “Null” component to pas a Boolean value if the data is currently null?)

The current process is pretty stable until I try to force solutions to expire to try and force a down-stream component to re-read data. I understand now why it was crashing because the background thread was in fact calling Me.ExpireSolution

A small thing I still don’t understand is what is the significance of the Boolean argument to the SolutionExpire method on stability and is it possible to setdata on the Da method of a GhCompoennt from the canvas i..e form the GHComponent Instance GUID, select the object and from the active GH Document and change data from there.

Permalink Reply by David Rutten on May 15, 2010 at 2:02am

Me.ExpireSolution(False) erases all data on the current object and all objects that depend on it.

Me.ExpireSolution(True) does the same as above, but it tells the document that owns the current object to recompute a solution after all the objects in question have been wiped.

I think it would be ok for your user to wait until data processing has been handled. You have to wait anyway, the only difference would be that the application is unresponsive. If you can cache the data, then you'll only have to wait once.

--
David Rutten
david@mcneel.com
London, UK

Permalink Reply by Steve Lewis on May 15, 2010 at 11:06am

I agree and I will allow the option to run in background or not. The problem is I'm pretty sure some tasks for getting data will take a long time as it's accessing data from anothercprograms API. I'm talking about five to 10 minutes. I think if the GUI is unresponsive for that period of time it will be a problem. Esoecially if the data is accessed more than once and a solution is recompited more than once.

Permalink Reply by Steve Lewis on May 15, 2010 at 1:41pm

David et al. Maybe I am overcomplicating things if caching can solve the problem. Can u elaborate a bit on caching. It seems this will still require the user to wait until processing is done before caching can start.

Permalink Reply by Steve Lewis on May 14, 2010 at 12:40pm

I give up on this!! Is there a simple way of telling a downstream compoennt not to inherit a parameter form an upstream object until it is a value other than null? Is there an easier way?

Permalink Reply by Steve Lewis on May 14, 2010 at 9:20pm

OK so I managed to get it to work, hope to post something soon. Basically it allows me to interactively hook in to a running instance of excel and get data in back ground and update it in GH/Rhino once the data has been parsed. once the work flow is set up data can be chnaged in excel and it will be updated automatically after a solution has run. Everything you said to do got it to work..............

Permalink Reply by David Rutten on May 15, 2010 at 2:03am

Nope. The only way you can get data to go from component A to component B is to actually run a Solution.

--
David Rutten
david@mcneel.com
London, UK

• ‹ Previous
• 1
• 2
• Next ›
• Page