Version control

Hi, I often make things that work, and then gradually morph them into things that don't work. When I'm coding I can use git or some other version control tool to track my changes, but GH files are binary, and GHX files seem to change so dramatically when I make changes that they don't seem to be worth tracking.

Has anyone had any luck using any sort of version control with grasshopper?

As a side thought, if it turns out that version control is silly, is there a good workflow to capture what you've done at each save?

Replies to This Discussion

Permalink Reply by Danny Boyes on April 16, 2013 at 2:44am

I use the international date convention as a suffix for file names to keep track of ever changing projects. that way when you order them on name they are sequential. e.g.

Version_20130101.gh

Version_20130228.gh

Version_20130416.gh

There is also an inbuilt revision notes section to Grasshopper which helps keep a track of major changes especially if you coincide this with the saved suffix date.

File>Document Properties>Add Revision

Permalink Reply by Ben Doherty on April 16, 2013 at 2:57am

The "File>Document Properties>Add Revision" model is interesting!

In 3dsMax there is an option to increment on save, which just adds a number to your file name that is one bigger, this means that every time you [ctrl]+[s] it saves a version, it'd be nice if saving could be made to trigger the version dialogue.

One of the nice things about version control is that it only stores the delta between the two files. For things like GH files which will always be (relatively) small this isn't really an issue, but for a big max file it can mean that a day of work is several gigs of data. It'd take some really substantial remodelling of the file format to make normal version control work, but I wonder if there is anything that could be done to fake it, maybe with a massive undo buffer etc.

This is all very well for linear progression, but when it comes to forking something like git gives you great visualisations of who is where on a project. When there are multiple people working on something, crafting referenced clusters etc. then maybe this sort of thing will become important.

Permalink Reply by Danny Boyes on April 16, 2013 at 3:15am

Rhino has a similar, if not the same, feature with IncrementalSave on the File Menu

Permalink Reply by Chintan Pathak on August 4, 2015 at 4:32am

I believe there should be a better workflow than this.

As the capability of 'diff', merge etc. is not available this way, only saving multiple versions does little. You still have to open/run each and every saved iteration to find which one works.

I was wondering, if all the components we see in a grasshopper sketch are essentially code-blocks, a plug-in/option to convert-to-code and then convert-to-component could help enable version control.

Permalink Reply by Ben Doherty on August 4, 2015 at 5:17am

I've thought this over for a while. There are three components to a grasshopper file:

The topology of the DAG
The position of the nodes on the canvas
The data in the file.

You could be picky and say that the data in the DAG should be separate from the data at the root, but lets not for the moment.

Dot language could be used to capture the topology (could, not should)
Position could be captured as a list of tupples, as json or as a csv or as some other arbitrary, text based file format.
The data could be binary.

Those three files could then be zipped up like a JAR or a DOCX file and stored in your windows file system, but you could diff of the topology file (1) or the topology and then positions (1&2) or all three.

This is probably overly simplified, but it's a start!

Permalink Reply by David Rutten on August 4, 2015 at 5:53am

We have made some forays into supporting something quite similar to dot language for GH. That goal was to have a human readable language which could be parsed by software as well. This language would only support a subset of the GH file format (namely the connections and options on components, but not internalised data). It is unlikely that GH1 will get support for this but I'm still hopeful that the next version will.

Permalink Reply by Chintan Pathak on August 6, 2015 at 3:55am

For cases, when the ultimate geometry is totally generated by the GH sketch, then it means it is totally dependent on code, and may be easily version-able.

Permalink Reply by Yasushi Sakai on April 25, 2017 at 8:55am

Hi community,

Were there any follow-ups or attempts regarding version controlling (Git) GH files?

I see other discussions in other threads, but this post's title seems to fit well.

situation

- We have multiple people working on one project with different roles (connecting components, scripting, etc).

- GH file communicates with different programs, so repo includes scripts.

- Rollbacks, branching, tags, blame, diff,, etc to manage different versions

- forking to using (partial) GH files for future projects

considerations top of my head

* Methods that integrate with git(hub) fits with the current trend

* Putting binary .gh files in a git repository just doesn't seem right.

* Despite Ben's remark that .ghx files change so rapidly, .ghx seems the closest way to go...

* Is there a way to make GH save as .ghx by default?(prone to human error with inconsistent .gh and .ghx files)

(prone to human error with inconsistent .gh and .ghx files)

* Does sorting the xml elements to make it trackable help?

might as well need to consider package management, but slightly different topic.

Permalink Reply by Gonzalo Casas on April 26, 2017 at 3:49am

Hi!

I totally share your pain.

We started using .ghx in hopes it git merges/diffs would be manageable (or remotely manageable at least), but it's not really working out very well. Off the top of my head, the main problems we see are:

The position of components is constantly creating merge conflicts for the most trivial of changes, the lack of snap-to-grid or lock-component-position feature makes it a recurring issue.
Clusters: I was thrilled when I notice that clusters can be externally referenced, and then immediately disappointed when I understood that: a) even if external, the main GHX file stores a binary stream of the cluster's content! b) the .ghcluster format is only binary, not XML.
XML size: once you start doing slightly bigger things, the .ghx file grows to several MB of size, which make it a pain to merge. Especially when long binary stream lines are interspersed throughout the file, those tend to crash my diff/merge tool very often.

IMO the biggest issue is the lack of proper composability: it's impossible to break down a grasshopper file into smaller, discrete parts that can be imported into a larger one. That alone (assuming the part-file format is not binary) would make everyone's life so much easier.

Cheers

Gonzalo

Permalink Reply by Yasushi Sakai on April 26, 2017 at 6:59am

Your reaction already helps :)

Maybe asking for a comprehensive solution both having code and gh files version in a single repo is too much. I'm starting to look one step deeper on ways to diff XML files (https://msdn.microsoft.com/en-us/library/aa302295.aspx) where we have interest on the change of the logic, not strings + it's unlikely we will be writing ghx on text editors.

If we want to stick with git, (and very much like to) another possibility is to use git hook, (or webhook in Github) where we can intercept and run scripts before/after a commit.

Like in your comment, I think still there are issues regarding size and "clusters", but I see potential on versioning gh files not just for efficiency but for learning/sharing grasshopper. Even in this forum, the current way is to simply upload the definition file, back and forth...

One non-technical concern is whether it has that much of a demand, I see a lot of mental gap between saving incremental files (project-final-final-latest-presentation.gh) from [stage]-[commit]-[push] workflow....

Anyway thanks! I might start with small experiments, and update when appropriate.

Permalink Reply by Gonzalo Casas on May 2, 2017 at 8:03am

The idea of XML pre-processing tools is also something I considered. I've used it very successfully in the past (not for Grasshopper, but for translation files that had the same problem: the change was semantically not that big, but from a text-only perspective, it was pretty large)

I haven't had time to play with that but if you would be interested in writing such a tool as open source, I can definitely contribute to it. (And I totally agree with nobody will write ghx on text editors, that's not what we're aiming for at all ;) )

Regarding the mental gap you mention, absolutely. It is a big issue, we're trying to tackle it with internal training but it takes time.

Cheers

Permalink Reply by Ben Doherty on May 2, 2017 at 11:30pm

On the issue of [stage]-[commit]-[push] vs incremental files, it's one of those things that nobody sees the value in until they've been saved by it.

All my students learn a basic version of the git workflow, and in week 1 they write quite good commit messages, but by week 4 or so their messages are "sdlkgjsdf" or "avroabsvd". Then they get into the meat of the course and they have to use their git log to keep track of what they did, or to revert, and the commit messages become really high quality.