Text to integer very slow

I have a big list of data coming from a .csv file. I seperate everything into what used to be the columns and feed it to an integer parameter. The list only contains integers, <empty> and <null>.

For a list with around 22000 entries it takes almost 14 seconds. The parameter also says that it failed in the conversion. It turns all <empty> entries to <null>.

So the question is, why is it so slow? I want all <empty> entries to remain empty.

Strangely the Profiler only sometimes shows the timing under the integer parameter. Most of the times it only shows the long time at the bottom.

Attachments:

text to integer slow.gh, 56 KB

Replies to This Discussion

Permalink Reply by ng5 Alex on June 21, 2016 at 10:45am

Hello Armin,

this happens because integer component tries to convert text to integers internally making it slow to calculate.

In case you want to retain the empty from csv (empty is text with no legth "") you can do the following

best

alex

Permalink Reply by ng5 Alex on June 21, 2016 at 10:46am

and the file

Attachments:

texttointeger.gh, 64 KB

Permalink Reply by David Rutten on June 21, 2016 at 11:22am

I want all <empty> entries to remain empty.

This is not possible. Empty is a text only property, in that it represents a piece of text with no characters. There is no equivalent for numeric types such as Integer for this. Empty text will either become a null integer, or maybe you can choose to have it set to a specific integer, like -1 so you can detect it later.

So the question is, why is it so slow?

That's because Grasshopper treats the text as an expression. It doesn't just assume the text represents an integer directly. Admittedly, this was a poor decision on my part, the conversion should have used the direct method first, and only switch to slow expression evaluation when that first attempt fails.

I attached a C# script component which converts a list of text into a list of integers, using quick initial testing and it also provides ways to provide special placeholders for null and empty values. It still won't be as quick as it could be because the script components have some overhead, but hopefully it'll be faster than what you have now.

Attachments:

text to int.gh, 12 KB

Permalink Reply by ng5 Alex on June 21, 2016 at 12:24pm

i tried davids strategy with native components. turns out pretty quick, though if data set is too large, at some point member index gets slow.

best

alex

Attachments:

texttointeger2.gh, 64 KB

Permalink Reply by Armin Seltz on June 22, 2016 at 2:00am

Thanks to both of you for the explanation and solutions! I have restructered things a bit now and actually doing the conversion from text to integer later on and in some parts using your replacement methods. Its now down to <1 sec for everything (the dataset from the file is just one of 10 branches).

RSS