DataTree selection rules

Dear Users,

I've been working on data tree selection rules this weekend and when 0.9.0063 is released (hopefully tomorrow, 4th November) the [Path Compare], [Tree Split] and [Replace Path] components will work slightly different from before. Sorry about breaking this, but it proved impossible to improve the selection logic with the fairly ambiguous notation that was implemented already.

Not every change is breaking though and I hope that most simple matching rules will work as before. There will be a McNeel webinar on Wednesday the 6th of November where I discuss the new selection rules (as well as path mapping syntax and relative offsets within one or more data trees). This will be a pretty hard-core webinar aimed at expert users. The event will be recorded so you can always go and watch it later. I figured I'd briefly explain the new selection rules on Ning before I release the update though.

-------------------------------------------------------------------------------

Imagine we have the following data tree, containing a bunch of textual characters:

{0;0} = [a,e,i,o,u,y]
{0;1} = [ä,ë,ê,ï,î,ö,ô,õ,ü,û,ÿ,ý]
{1;0} = [b,c,d,f,g,h,j,k,l,m,n,p,q,r,s,t,v,w,x,z]
{1;1} = [ç,ĉ,č,ĝ,ř,š,ş,ž]

There are a total of four branches {0;0}, {0;1}, {1;0} and {1;1}. The first branch contains all the vowels that are part of the standard English alphabet. The second branch contains all non-standard vowels and branches three and four contain the standard and non-standard consonants respectively.

So what if we want to select from this tree only the standard vowels? Basically include everything in the first branch and disregard everything else. We can use the [Tree Split] component with a selection rule to achieve this:

{0;0}

This selection rule hard-codes the number zero in both tree path locations. It doesn't define an item index rule, so all items in {0;0} will be selected.

If we want all the vowels (both standard and non-standard), then we have several options:

{0;?} = select all branches that start with 0

{0;(0,1)} = select all branches that start with 0 and end in either 0 or 1

{0;(0 to 1)} = ......................................... and end in the range 0 to 1.

Conversely, selecting all standard vowels and consonants while disregarding all non-standard character can be achieved with rules as follows:

{?;0}

{(0,1);0}

{(0 to 1);0}

It is also possible to select items from each branch in addition to limiting the selection to specific branches. In this case another rule stated in square brackets needs to be appended:

{0;?}[0 to 2]

The above rule will select the first three vowels from the standard and the non-standard lists.

Basically, rules work in a very consistent way, but there are some syntax conventions you need to know. The first thing to realize is that every individual piece of data in a data-tree can be uniquely and unambiguously identified by a collection of integers. One integer describes its index within the branch and the others are used to identify the branch within the tree. As a result a rule for selection items always looks the same:

{A;B;C;...;Z}[i] where A, B, C, Z and i represent rules.

It's very similar to the Path Mapper syntax except it uses square brackets instead of parenthesis for the index (the Path Mapper will follow suit soon, but that won't be a breaking change). You always have to define the path selector rule in between curly brackets. You can supply any number of rules as long as you separate them with semi-colons.

The index rule is optional, but -when provided- it has to be encased in square brackets after the path selection rule(s).

The following rule notations are allowed:

* Any number of integers in a path

? Any single integer

6 Any specific integer

!6 Anything except a specific integer

(2,6,7) Any one of the specific integers in this group.

!(2,6,7) Anything except one of the integers in this group.

(2 to 20) Any integer in this range (including both 2 and 20).

!(2 to 20) Any integer outside this range.

(0,2,...) Any integer part of this infinite sequence. Sequences have to be at least two integers long, and every subsequent integer has to be bigger than the previous one (sorry, that may be a temporary limitation, don't know yet).

(0,2,...,48) Any integer part of this finite sequence. You can optionally provide a single sequence limit after the three dots.

!(3,5,...) Any integer not part of this infinite sequence. The sequence doesn't extend to the left, only towards the right. So this rule would select the numbers 0, 1, 2, 4, 6, 8, 10, 12 and all remaining even numbers.

!(7,10,21,...,425) Any integer not part of this finite sequence.

Furthermore, it is possible to combine two or more rules using the boolean and/or operators. If you want to select the first five items in every list of a datatree and also the items 7, 12 and 42, then the selection rule would look as follows:

{*}[(0 to 4) or (6,11,41)]

The asterisk allows you to include all branches, no matter what their paths looks like.

It is at present not possible to use the parenthesis to define rule precedence, rules are always evaluated from left to right. It is at present also not possible to use negative integers to identify items from the end of a list.

If you want to know more, join the Webinar on Wednesday!

David Rutten

david@mcneel.com

Seattle, WA

Replies to This Discussion

Permalink Reply by phillip on November 4, 2013 at 12:30pm

Fantastic! Brave New World Order.

Do you plan to apply a similar notation on [RelItem] as well?

Permalink Reply by David Rutten on November 4, 2013 at 2:44pm

Relative items will also have to be re-thought, and path mapper notation too. Though I'm not sure the same kind of notation can be used for all as they do represent very different operations.

David Rutten

david@mcneel.com

Seattle, WA

Permalink Reply by AnewGHy on November 4, 2013 at 11:28pm

Finally. Now just list addresses to sort out (make dynamic), and the GH data management system will at long last be rational, intuitive and robust - critical requisites if attracting [and keeping] new users upon the v1.0 release is being taken seriously!

Permalink Reply by David Rutten on November 5, 2013 at 9:20am

How would you like to see 'list addresses' implemented?

David Rutten

david@mcneel.com

Seattle, WA

Permalink Reply by Danny Boyes on November 5, 2013 at 9:32am

I think AnewGHy is referring to his previous post about the addition of extra branches (0's) that get added after the "possibility" of multiple outcomes.

I.e. only add extra paths if there is additional paths to create

Permalink Reply by AnewGHy on November 5, 2013 at 4:32pm

It would be good if the existing data matching logic is overhauled: currently, list addresses are created by default as {0;0}. I understand why GH operates in this way...after some more 'enlightened' individuals told me so.

And therein lies the problem. Its just plain confusing if you're new to GH - and even after stumbling upon the discovery, (which I suspect most inquisitive minds will do at some point), it forever remains kind of an oddity to anyone of a rational/logical disposition when considering how these rules 'adapt' depending on different scenarios. A quasi-inconsistency seems to manifest. I have illustrated this in a previous post of mine:

http://www.grasshopper3d.com/forum/topics/gh-addresses-just-me-or-u...

To reiterate, the key points I made were as follows:

[...] why doesn't GH simply create lists dynamically rather than 'second guessing' what the potential length of a list might actually be [more specifically, why does it default to an address of {0;0}].

[...], if i have one circle with points on it, logically the list structure should be {0}; if i add more circles naturally {0;0} makes sense.

Here's one example of why the existing GH logic is somewhat flawed:

{0;0} - default

{0;0;0} - I've added additionally components and list length has grown

{0;0;0;0} - similar scenario

The flaw:

If I am splitting a tree, potentially the branches will change as the list length changes. I either have to manually edit my definition or accept that GH is actually restrictive, as it predefines a list length, yet still grows it when its forced to. Why not just grow from the off set and maintain a logic, rather than one rule to start, another to adapt (if the first rule gets broken).

[...] GH appears to do something similar [dynamically adjust lists] once its default data management techniques are exceeded by inserting a new address index.

If addresses are added when forced to, why not just have that as the default behaviour in the first place?

My final point:

circle with points should have a list address of {0}

multiple circles with points should have list address of {0;0}

multiple circles in multiple locations with points should have list address of {0;0;0} etc

I really don't see how that is any less consistent for highly complex data structures. To any rational individual this is predicable and follows a logic. What advantage is there in fixing the address at {0;0} yet still allow for new address sequences to be added further down stream? Logic is the key thing to keep in mind here, not peculiar nuances only the initiated can ever be aware of.

So to summarise, why have the default list addresses start at {0;0}, yet change this structure if data exceeds such constraints? Why not implement a more consistent, logical and predictable data management system which is representative and appropriate for the object(s) being created. Most importantly, the behaviour should be dynamic; if objects change from singletons to collections, just as addresses adapt beyond the default {0;0}, why cant the same happen from {0} to {0;0} and beyond? It would make a lot more sense, especially to new users.

Permalink Reply by David Rutten on November 5, 2013 at 6:29pm

Thanks for reposting this again. I'll put it at the top of the list of things to think deeply about on my flight back to Europe come Friday. I probably won't have time before then to form an opinion worth listening to...

David Rutten

david@mcneel.com

Seattle, WA

Permalink Reply by AnewGHy on November 12, 2013 at 2:15am

Appreciated. The direction you're taking is most certainly a step in the right direction- the data controls are getting a lot more intuitive with a common logic. On your webinar you did hit upon precisely the problem I am referring to; you called them "arbitary zeros". Get the list addresses dynamic (ie adapt according to singleton or collections, collection if collections, and so forth) and finally you'll have a sound, robust logical data control/management system.

Permalink Reply by David Stasiuk on November 12, 2013 at 3:20am

While I appreciate your belief that a singleton output from a component shouldn't activate a new data path index whereas a collection should, I disagree that this should be implemented wholesale.

It is common to have in a definition a component that, depending on the input data, would in such a case sometimes have a singleton output and other times have a collection output. Having the path indices adjust in such a way has the potential to break data structure downstream. In this way, data structure becomes inconsistent.

I totally get that it's counterintuitive and odd to have a new zero added when the output isn't a collection...but automatically making all data structure this fluid makes for breakages, particularly in more complex definitions, whereas the opposite isn't true: you can always predict the data structure that will come out of a component and it always anticipates the highest possible complexity in regards to ordering information, regardless of the singleton/collection nature of the output. But I think you're right that for people first learning (and most likely in 90% of instances of usage) it'd be easier to grasp data structure to not automatically add incremental indices.

I wonder if a potential solution could be to have a component level toggle - maybe the default value for new components could be set in preferences -which could also be switched on and off within each component individually? Something like "dynamic data structure"?

Permalink Reply by Frans Magnusson on November 5, 2013 at 9:49am

Is there a case for Lists as a datatype similar to what groups are doing to geometry?

Pack the branches of a tree into items (of list type) and do what you can do to lists of items in terms of culling and weaving and when you are finished: pop up the branches again.

In many situations I find myself flipping matrixes to get to use various list components and then flipping back again, but it is often hard to really get a full understanding of what actually happens in these situations.

Sorry for off topicing a bit!
//fr

Permalink Reply by Nick Tyrer on November 5, 2013 at 11:09am

If i'm understanding you correctly you can, if you use the 'param viewer' to output the branch paths, you can use any of the list components on the branch paths, then input those paths into the 'tree branch' component.