Grasshopper

algorithmic modeling for Rhino

Format function returning unexpected number value

I've simplified down a problem that I have in a different model.  Simplified, I created a series and then output via a format function.  For some reason the format is changing the value of the input and I don't see any reason why it should.  Is there an explanation for this or is it a bug?

Views: 524

Attachments:

Replies to This Discussion

Hi Norm, you could use {0:0.###} as formatting rule to fix that.
(Or use {0:0.00#} to have at least 2 digits, and 3 digits if the third digit is relevant (not 0).
To have all values use 3 digits you could also use {0:N3}

Sorry, I meant decimals/decimal places instead of digits ...

Hey Pieter...  Thanks for the suggestions.  It kinda works, but the specific usage that I have for this requires that the result not be rounded.  It's important to know if the result is 3.46 or if it is 3.4599999999994.

It really seems to be an anomaly with the Format function... also interesting is that the last digit starts a repeating pattern as the number of results increases.

But your ideas got me to experimenting and I think that the format of:

{0:0.############}    (12 #'s)

will work as it seems to give the correct results in the list.

If I use  {0:0.#############}  (13 #'s) the output fails again.

Well, the "correct results" in the original series are already rounded: Try {0:R} on the series output to see 'all' decimals (depending on your GH Display settings). I'm not sure how come the noise is introduced. I think it's a floating point precision issue. (Try a search for that).

But since your series increments in steps of 0.0010 I figured three (or four) decimals would suffice in you case (plus it kills the noise).

RSS

About

Translate

Search

Photos

  • Add Photos
  • View All

Videos

  • Add Videos
  • View All

© 2024   Created by Scott Davidson.   Powered by

Badges  |  Report an Issue  |  Terms of Service