Synchronously sort data while creating a set

I have two sets of data of the same size, one of is a list of points and the other contains generic data. I can create a set from the list of points to remove all duplicate data, but I also want to be able to see which items in the generic data list correspond to each of the unique items in the set. In other words each of the unique items in the first data set will have a corresponding list of items from the second data set.

My current process is to use the replace branches component to go back and extract the corresponding items, but that component is quite slow due I suspect to its having to search through the data (I have large amounts of data, 40-50k items)

I may be missing something obvious here, but is there no simpler / faster way? It would be a nice option for the Create Set component to accept synchronous data sets which are then filtered into paths corresponding to each of the unique set items.

Any ideas?

Cheers,

Luke

Replies to This Discussion

Permalink Reply by Vicente Soler on April 5, 2012 at 5:41pm

Is this what you are looking for?

Permalink Reply by Vicente Soler on April 5, 2012 at 5:49pm

I missed the part when you say you have 50k items and it's working slow.

Maybe a scripting component will make it faster, try the following:

Create a VB component and paste the following code:

Dim dt As New datatree(Of Object) For i As int32 = 0 To x.Count - 1 dt.add(x(i), New gh_path(y(i))) Next a = dt

Set both x and y inputs as "list" and set the y input as "integer". Connect the generic data to the x input and the M output of the create set component to the y input.

Permalink Reply by Lucas Epp on April 5, 2012 at 5:58pm

Thanks Vicente - I had tried the first method you showed but for some reason hadn't clued into the tree structure of the member index component - exactly what I wanted. Using those native components takes about 40% of the time of my original solution, and using your script is a fraction slower.

I guess my point with this all was to try to do it natively (as the information necessary must be created when the set is first formed) and thus avoid extra computation downstream. I still think adding an optional input to the create set component would be useful (could use the ZUI as well..) I wouldn't imagine it would be much more overhead and it would be very handy (similar to the sort component).

Cheers!

Luke

Permalink Reply by Vicente Soler on April 6, 2012 at 8:52am

Is the script really slower? I just tested it out with 1 million items and the script was much much faster. (11 sec vs 2.4 min)

Permalink Reply by Lucas Epp on April 6, 2012 at 10:09am

No, the script is in actuality faster. For some reason on the first solution run I did it was slower (I've noticed the profiler does that occasionally - unless you change data somewhere, a recompute doesn't necessarily cause the profiler to show the right computation times).

Thanks again for the help.

Permalink Reply by Olivier Suire on January 24, 2017 at 8:39am

Hi Lucas, I completely agree that a synchronous sorting in the CSet component would be great.

I have tried using the "Map" output, but it's a pain in the neck.

Permalink Reply by David Rutten on January 24, 2017 at 9:03am

Sets can be made of all sorts of data types, only a few of which have obvious sorting behaviour (how does one sort colours, or points?). And what should happen if a set is made up of different data types?

Permalink Reply by Olivier Suire on January 25, 2017 at 2:25am

Hi David,

I don't think that we are speaking of the same thing.

The "Sort List" component has a "Values A" input for lists of values to sort synchronously.

My wish (and Lucas's) is just to have the same with Cset.

Here's a very practical example : I have a tree made of lists of named objects.

In some lists, some objects have multiple occurences, and I want my lists to have only one occurence of these named objects.

Cset is the obvious tool for this.

But these objects also have properties contained in corresponding trees :

-Materials the objects are made of

-Weight of the objects

-Color of the objects

-You name it ... !

I need a simple way to sort/purge these trees in the same way Cset did for the "Object names" tree.

I managed to work my way around this, but I think it would be nice to embed this synchronous sorting in the Cset component.

Permalink Reply by David Rutten on January 25, 2017 at 2:37am

My wish (and Lucas's) is just to have the same with Cset.

Ah I see, I indeed misunderstood. I suppose this sort of pattern makes sense for a lot of other components as well. Certainly something to keep in mind when writing the GH2 components.

But these objects also have properties contained in corresponding trees :

-Materials the objects are made of

-Weight of the objects

-Color of the objects

-You name it ... !

This is also something which may be done differently in GH2 to begin with. With custom meta data attached to each value, you wouldn't have to keep separate trees containing associated values, you can just store them directly on the data itself. But of course that'll --at best-- only happen sometimes. Your request still makes sense.

RSS