How to using delaunay triangulation to wrap around a geometry?

I am trying to use delaunay meshes to wrap around a prism. The only way I could think of is to randomly creat some points on each surface of the prism and then use delaunay mesh respectively. However, that turns out to be the following image which I don't how to connet them in the end.

I tried to delaunay them all at once as in the second image, but the mesh is not following the prism as you can define only one plane for it.