Terrain Generator : Invalid Brep

Replies to This Discussion

Permalink Reply by djordje on June 9, 2017 at 8:31am

Hi Mathieu,

Can you please attach your .gh file?
Thank you.

Permalink Reply by Mathieu Venot on June 9, 2017 at 8:46am

Hi Djordje,

I just send it via PM, don't know why it doesn't want to upload from the post.

I get another problem since previous post, the OSM Shapes components don't want to works too now. They tell me "The component generated invalid shapes data. To fix this issue, simply rerun the component (set the "_runIt" input to "False", and then again to "True")". I try to switch between True and False from the toggle on each but no luck, it fails. 2h ago all was working fine...

Thanks,

Edit : I just try to restart computer and delete gismo on C:/ but same error for the OSM Shapes

Permalink Reply by Mathieu Venot on June 9, 2017 at 9:17am

Edit 2 : I tried your file from the Retrieving elevation discussion (gismogetelevation2.gh), and without touching anything, just toggle the runIt input to True, if I check the terrain outputand it's "Invalid Brep" too.

Permalink Reply by djordje on June 9, 2017 at 10:10am

Hi Mathieu,

I will answer your questions, but before that:
The .gh file that you sent me has a Paris as a location. This one.
While in your opening reply, you state that your location is: Lac du Bourdon.
So which one of these two locations do you need?

Permalink Reply by Mathieu Venot on June 9, 2017 at 10:33am

Oh yes, sorry for the mess in the file.

The Paris location has Terrain Generator working correctly, but OSM Shapes returning the "invalid shapes data" error, even pressing multiple time the toggle.

The Lac du Bourdon location return invalid brep by the Terrain Generator, and also the "invalid shapes data" by OSM Shapes.

In fact, I don't really need specific location. I try some tests on several location and since that, every location I try return the OSM Shapes error (I try in Manhattan, San Francisco, Paris, little french towns...). The Terrain Generator return sometimes the invalid brep but not on all locations tested (error onto Manhattan, and Lac du Bourdon for example, others are good geometry).

I try to have a generic .gh file that allow quick context generation.

Permalink Reply by djordje on June 9, 2017 at 12:27pm

Thank you for the detailed reply Mathieu,
Can you please attach the .gh file with the exact location of the "Lac du Bourdon" and the "radius_" you used to generate a terrain for that location?

Permalink Reply by Mathieu Venot on June 9, 2017 at 1:48pm

Ok, I resend you another file via MP (still can't upload via the post, very strange).

The file contains red round Groups with the settings used and components performing errors.

Basically, the settings are 47.607407 (lon), 3.120446 (lat), 250 (rad), and with this setup the terrain generator output invalid brep (if rad is 125 or 500 the brep is clean, no error). But unfortunately, the OSM Shapes always gives the error no matter what is the rad size.

Permalink Reply by djordje on June 9, 2017 at 5:14pm

Thank you for sending the LocationGrabber03_Gismo6 - Bourdon.gh file and for reporting the "Invalid Brep" bug! The "Invalid Brep" issue you had was a tolerance bug, which appeared when splitting the terrain. It has been fixed now (the newest "Terrain Generator" component is v0.0.2 JUN 09 2017). This is the result of the AW3D30 source_ and 250 radius_ inputs with performed elevation analysis with "Terrain Analysis" component:

The LocationGrabber03_Gismo6.gh file you sent me, ran without problems. Here is the result that I get:

Instead of Gismo "Terrain Generator" component, I used Ladybug "Terrain Generator" component. Not because the first one did not work, it did, both for SRTMGL1 and AW3D30 source_ inputs. But because in case you are looking for a terrain in large cities, the Ladybug "Terrain Generator" will in cases of large USA cities, and as we can see, Paris (so maybe large European cities as well too?) produce much more precise results. You can always try yourself, 

and check which of the mentioned sources and two components produces the best terrain.

I made a few modifications to your .gh file, mainly adding the data to the requiredKeys_ input of "OSM shapes" component. We mentioned this previously. The "OSM shapes" component would still work even without these but if the radius_ is large or if there are a lot of things tagged in that location, then the number of keys may exceed the shapefiles' maximum, which could result in some keys not even being included in the "keys" output of the "OSM shapes" component. This is why it is recommended to always add specific keys to the requiredKeys_ input.

I attached the modified .gh file you sent me below, from which the upper screenshot comes from.

The "invalid shapes" warning happens due to a bug in the MapWinGIS. The authors couldn't fix it. What happens is that the openstreetmap file format is converted to a shapefile format. Sometimes this conversion fails. Luckily the problem is easily solvable by running the conversion again: by turning the "OSM shapes" component off, and then on again (runIt_ to "False" and then to "True").

If you are constantly getting this warning, then it can be that something is wrong with your system.
In April, you solved your own issue with MapWinGIS folder and osmconf.ini file permissions.
Maybe you can check once again if the permissions are still the ones you reported back in April?
And maybe you can try to set the same permissions to the "c:\gismo" folder, as well as each of its subfolders ("c:\gismo\osm_files",
"c:\gismo\terrain_files"...).

Also try setting the same permissions to this folder:

"c:/gismo/osm_files/3_Rue_de_Bretonvilliers_Paris_48.850763_2.35891_radius=0.13KM"

It's the folder where your openstreetmap and shapefiles are located for the location_: "3_Rue_de_Bretonvilliers_Paris, lat:48.850763, lon:2.35891, radius_:125".

Attachments:

• LocationGrabber03_Gismo6b.gh, 941 KB

Permalink Reply by Mathieu Venot on June 10, 2017 at 6:09am

Thanks a lot Djordje !

As you said, the Terrain Generator seems to be good now. Clean closed brep on every location I tried.

For the invalid shapes data from the OSM Shapes, I check and re-applied the full control priority on files and folders but no luck. So I re-install MapWinGIS, and it seems that was this. Now all is working fine !

I have several questions also :

- About the terrain generation, is it possible to have the texture from google or other provider mapped onto the terrain surface from gismo component ? (Same as using the ladybug terrain generator in fact). I try to used the image extracted by ladybug component and then applied it to the gismo terrain but the texture is rotated by 90°.

- About the requiredKeys_ input of OSM shapes, I understand what you mean and your advice, but in most cases when I use it, the component was working fine even without input. If possible, I prefer to use it without input to check all data contains on the selected area (values, keys and tags). As I don't always know what data I want.

Permalink Reply by djordje on June 11, 2017 at 8:57am

Hey, thank you, for finding the bug!

For the invalid shapes data from the OSM Shapes, I check and re-applied the full control priority on files and folders but no luck. So I re-install MapWinGIS, and it seems that was this. Now all is working fine !

Glad that it worked! But I am still a bit worried. Gismo components only modify the gdal-data/osmconf.ini file and no other MapWinGIS file. So your MapWinGIS installation files should not be compromised. The fact that you did not get the "COM CLSID" error message when running the "Gismo Gismo" component suggests that MapWinGIS has been properly installed. So I wonder if the cause for the permanent "invalid shapes" warning has again something with the fact that your system is again not allowing the MapWinGIS to properly edit the osmconf.ini. Maybe this problem will appear again, and again, and reinstallation of MapWinGIS every time can be somewhat bothersome.

- About the terrain generation, is it possible to have the texture from google or other provider mapped onto the terrain surface from gismo component ? (Same as using the ladybug terrain generator in fact). I try to used the image extracted by ladybug component and then applied it to the gismo terrain but the texture is rotated by 90°.

The issue with the rotation can be solved by swapping/reversing the U,V directions of the terrain surface. A slightly more important issue is that terrain surface generated with Gismo "Terrain Generator" component might have a bit smaller radius than what the radius_ input required. This stems from the fact that the terrain data first needs to be downloaded in geographic coordinate system, and then projected. Some projecting issues may occur at the very edges of the projected terrain, so I had to slightly cut out the very edges of the terrain which results in the actual terrain diameters being slightly shorted in both directions. This means that if you apply the same satellite image from Ladybug "Terrain Generator" component to Gismo "Terrain Generator" component the results may not be the same.
I attached below a python component which tries to solve this issue by extending the edges of Gismo "Terrain Generator" terrain, and then cutting them with the cuboid of the exact dimensions as the radius_ input. Have in mind that this extension of the original terrain at its edges is not a correct representation of the actual terrain in that location. But rather just an extension of the isoparameteric curve of the terrain surface. So basically: some 0 to 10% (0 to 10 percent of the width and length) of the terrain around all four edges is not the actual terrain for that location, but rather just its extension.
The python component is located at the very right of the definition attached below.

Also, if you would like to use the satellite images from Ladybug "Terrain Generator" component along with "OSM shapes", sometimes you may find slight differences in position of the shapes. This is due to openstreetmap data not being based on Google Maps (that's what Ladybug "Terrain Generator" component is using), but rather on Bing, MapQuest and a few others.

- About the requiredKeys_ input of OSM shapes, I understand what you mean and your advice, but in most cases I use it, the component was working fine even without input. I think it's better to extract all tags, values and keys of the selected area, instead of searching for specific ones as I try to find all data related to what I want after, isn't it ? To check what keys are present on the area also.

Ineed, you are correct.
I though you were trying to only create a terrain, 3d buildings and maybe find some school or similar 3d building, for these two locations.
The recommendation I mentioned previously is due to shapefiles having a limit (2044) to how many keys it can contain. This requires further testing of some big cities locations with maybe larger radii, which I haven't performed due to my poor PC configuration. But in theory, I imagine that it may happen that a downloaded .osm file may have more than 2044 keys. In that case shapefile will only record 2044 of them, and disregard the others. That was my point.
But again 2044 is a lot of keys, and I haven't been checking much this in practice. For example, when I set the radius_ to 1000 meters, and use your "3 Rue de Bretonvilliers Paris" location I get around 350 something keys, which is way below the 2044.
Another reason why one should use the requiredKeys_ input is to make the Gismo OSM components run quicker: for example, the upper mentioned 350 something keys will result in 350 values for each branch of the "OSM shapes" component's "values" output.
Which means if you have 10 000 shapes, the "OSM shapes" component will have 10 000 branches with 350 items on each branch (values). This can make all Gismo OSM components very heavy, and significantly elongate the calculation process.
With requiredKeys_ input you may end up with only a couple of tens of items per each branch.

Sorry for the long reply.

Attachments:

• LocationGrabber03_Gismo6c.gh, 944 KB

Permalink Reply by Mathieu Venot on June 12, 2017 at 6:30am

Wah, thanks a lot for all your help Djordje, and sorry to have bothered you on this week end.

About the MapWinGIS problem, I'll keep you updated if the error come back. For now, it seems to be good, and maybe it was of my fault (I tweak my system quite often and it could be a mistake I did).

I just try your Python component to swap and reverse UV, it works great ! As you said, there is a little offset between the Google image and the OSM footprints. It will be awesome if we could choose between multiple providers for aerial image (especially Bing or MapBox as they seems to be used by OSM). I know that the Mosquito's Get RASTER component is doing it but for ArcGIS image.

I'll keep in mind what you explained so clearly about the required_keys. I known that is really important for performance issues, but I didn't know for the 2044 limit.

Anyway, thanks a lot for your time. I'll continue to test and follow your updates with great interest !

Permalink Reply by Mathieu Venot on June 12, 2017 at 6:48am

Edit : My mistake, it's the Heron "Get REST Raster" component

• ‹ Previous
• 1
• 2
• Next ›
• Page