02 Jan 2015

Mad Mash Versioning

Through 2014, I had every once in a while heard people casually mention that they were versioning their projects in a combination of git and SVN. I didn’t really take the time to ask for details on it, but it was one of those things, stuck in the back of my mind.

Filed under “delightfully mad hacks”, I finally found some time to look into it over Christmas – as I was dodging various boring tasks in finalising Behave 2.5.

The Excuse

Git is lovely for its decentralisation, easy branching and a bunch of other stuff. However where it is not so delightful is in its handling of binary files.

One alternative VCS which is pretty good at handling binary files is SVN. So by putting binary project files in SVN and text files in git, you should be able to get the best of both worlds, right?

That is the idea anyway.

The Hack

Regardless of whether it is a good idea or not, it is an interesting hack. Immediately, it seems like a terrible idea with way too high a potential of user error from its silly workflow overhead. So that is where we’ll attack.

The goal is then to have the user place binary files in one folder and text files in other folders – all under the same core project folder. And then simply commit, update and branch as normally, in the knowledge that everyones local drives will not explode with tons of versioned binary data.

Our primary concern is keeping SVN and git in sync – with the proper commits connected. No-one likes SVN, so we’ll hide that away and put git in the driver seat.

Luckily, git operations are also fairly easily customised on a per-repository basis, which is perfect for this case.

The idea will then be to have git operations keep track of the SVN working copy, making sure that everything there is committed before a dependent git revision is committed, at which point a versioned text file with the corresponding SVN version number, should be added to the commit.

This way, SVN becomes a dictionary of collected binary files, into which git adds new versions when committing and fetches copies when updating – using SVN version numbers as key.

Git Hooks

Our entry point in automating this stuff with git is in the so called “git hook”. These are scripts located at “.git/hooks/” for any git repository. Per default, this folder is populated with a few demo scripts, each containing an example of what to do in response to that particular git event.

For this hack, the two points of interest are:

  1. The commit process – cancelling the commit if the SVN working copy is not clean, otherwise getting its current version and adding it to a text file in that same commit.
  2. The checkout process – updating the SVN working copy to the version specified in the text file, every time a git checkout has completed.

This means that the files we are interested in are:

  1. (Git clone path)/.git/hooks/pre-commit
  2. (Git clone path)/.git/hooks/post-checkout

Note that these files are not by default versioned with the rest of your git repository, though I do recommend doing so anyway. For this hack, I chose to address this via a setup script which needs to be run once after first cloning the repository – as well as after performing any updates to the versioned hook scripts (which I’d estimate happening just about never).

Solution

My weapon of choice for this sort of hackery is Perl. If you would rather use something else, that is fine too – as long as your shell can run it, you should be fine.

Since we’re in glorious hack land, I chose to take it further over the top by expanding the text file with the SVN version to be a full JSON-formatted configuration file, with the option of overriding parts with per-user local configuration.

The project structure is as follows:

  • .gitignore – containing ignores for the local user config as well as the SVN folder.
  • VCS/ – this is where the core perl scripts are kept, as well as configuration.
  • Setup.pl – the run-once script for copying the hooks in etc.

In the linked Git repository, the example configuration has also been set up for a Unity project – with the SVN working copy placed in Assets/SVN. Though aside from this, nothing here is specific to any technology aside from Git and SVN.

Additional over-the-top-ness:

  • Place a JSON formatted file at VCS/LocalUserConfiguration.txt to override select values from what is set in the versioned configuration file.
  • Setup.pl will check for a working copy at the specified SVN path and run a checkout from the specified repository if none is found. So this is really a one-stop-shop kinda thing.
  • “autoCommit” and “autoAdd” options for mad driver-less running.
    • “autoCommit” will, if enabled, see the commit script run SVN commit to push any local SVN changes as part of the git commit process.
    • “autoAdd” is for off-your-rocker “Look, ma! No hands!” setups, where any unversioned files found in the SVN folder are automatically added to SVN as part of the git commit process.

Why no “ignore unversioned” flag? In the example configuration, the SVN folder is put inside a Unity Assets folder. This means that contained files are imported as part of the project and project files outside of SVN can depend on them.

If you have no problem risking gun-to-foot related injuries with commits missing dependencies, that’s just fine. However I will not assist you in the loading of said gun.

I hope that you find this stuff interesting as well – if not useful. You can find everything at the other end of the below github link and I would be very interested in any feedback and discussion (preferably over twitter).

Are you employing a similar combined VCS setup? Are you in fact one of those who mentioned using the git+SVN combination in particular? Do you have a suggested alternative to this madness?

CoreObject
Unity Protocol Buffers
Behave 2.7
Learn
Behave 2.6
Trusted Gear
Mad Mash Versioning
Behave 2.5
Behave 2.4
Co-very-routine
Construct
The Engine Wars: Numbers
GDC 14: The Quest For Fun
Moving in Unity
Behave 2.3
Unity and .net assemblies
Behave 2.2
ReView
Behave 2.1
Behave 2.0
Unity Hacks: Dual sticks
Unity Hacks: Cameras
Unity Hacks: Touch gestures
OnRenderTextureGUI
Unite 13 video "Unity Hacks" available
The implicit local network interface
Talks and progress
Five years of Unity expertise looking for contracts
Automagic Unity Android Java gadget OF DOOM!
Invading Planet from your couch
Mountain Lion and laggy bluetooth and duct-tape
Unite 12 video and new videos section available
Asia Bootcamp videos now available
Path is now MIT licensed
Behave 1.4 released
So I've been a bit busy lately
Behave 1.3 released
IGDA Unity SIG slides
Second Unity IGDA SIG this evening: Scene construction and AI
First IGDA Unity SIG this evening
Alternative licensing available
Pathfinding in two lines
Path 2 released
Assembling and assimilating
Path 2 intro screencast
Path 2 beta release for GGJ
AIgameDev master class video now online
Expanding beta
Behave AIgameDev master class public stream
Behave master class on open AIgameDev stream tomorrow
Interview with AIGameDev
New video: From tree to code
Issue tracking on github Behave release project
IT University Copenhagen Unity course completed
IT University Copenhagen Unity course files Thursday
CPH IT University Unity course files
Behave 1.2 released
Video: Behave - starting from scratch
Behave runtime documentation updated
Behave 1.1 released
FAFF cleanup: Sketch
Building a menu of delegates and enums
Pick me! Pick me!
Optimising coroutine yielding in C#
Downloading the hydra
New license of Path: GPL
CopyInspector
Magnetic
GUI drag-drop
Logging an entire GameObject
I bet you can't type an A!
Where did that component go?
UnitySteer
New and improved: Behave 1.0 released
Behave 0.3b and unity 2.5
Behave 0.3b hotfix
Path tutorial video available
Path 1.0 launched!
Continued community tutorials
Community tutorial
New tutorial
First tutorial available
Behave 0.3b
unite '08 open-mic session
Behave 0.2b
Behave 0.1b
Behave pre-release
Path beta 0.3b
Path beta 0.2b
Path beta 0.1b
Path pre-release