As I continue on my tenth year of trying to find a game project to focus on, I decided to bring an old procedural map generation experiment back from the dead, upgrade it to Unity 2019.4 LTS and also replace my Git submodule setup with Unity packages. I’ve not quite succeeded with that last part.
Until earlier this year, I’ve used Git submodules to share reusable code between projects. A typical project has usually had three of these submodules: my dependency injection library, a “gadgets” library which I originally intended to open source and an “internal gadgets” library for stuff that I didn’t want to release. As Unity’s package manager system has evolved over time, I’ve started looking at it as a possible alternative due to some problems I’ve run into with using submodules.
The main problem has been that submodules can be a bit clunky to use and I’ve never managed to quite come to terms with how they work. Adding, moving, renaming and removing them tends to require a series of cryptic Git commands which seem to change from version to version and make the process of managing several submodules in a project feel brittle. As a result of this, I’ve gathered my reusable code to larger libraries (gadgets and internal-gadgets as mentioned above) which has resulted in a second problem: over time these libraries have become bloated with code which I no longer actively use but which is difficult to remove or even update because some older project which I may want to return to might still depend on it.
To emphasize: the problem is the fact that everything is in a monolithic submodule. If I have my inventory system and my touch input library in the same submodule, any project that uses that submodule must use the same version of both libraries, ie. I can’t use an older version of the inventory system with a newer version of the touch input library. The answer is obviously to split the two libraries into separate submodules, but I’ve avoided doing that because submodules are so awkward to work with.
My main problem is that like NPM, UPM can fetch packages from a registry or from a Git URL but only as a direct dependency of the project, not of another package. Wth UPM, Git URLs only work in the project’s manifest.json and not as a package’s dependency. This means that if I want to have my pathfinding package depend on my priority queue package, I’d have to set up a private package registry. I can’t just use the priority queue package’s Git URL like I could with NPM. UPM’s registry protocol is similar enough to NPM that you can use a private NPM registry tool like Verdaccio to host UPM packages. Unfortunately, the protocol isn’t exactly the same. I can’t use Github Packages, for example, because UPM uses reverse domain names (com.mattihiltunen.mypackage) instead of NPM’s text names (@mtti/mypackage).
I could of course set up a personal package registry with Verdaccio, but I’m not at all interested in setting up and paying for another server especially since all of this is still technically a hobby for me.
It’s not the end of the world for me though. The aforementioned pathfinding-priority queue situation still works if both packages are added to the project directly but that means I have to keep track of the dependency graph myself, which negates a lot of the point of using a package manager in the first place.
My other problem has been with package development. When you’re still working on the package, you don’t really want to install it in your project like you would a “finished” package because then you’d have to make changes separately and publish a new version after every change and that’s really stupid. You want to bring your package’s Git repository into your project so you can work on it, test it with your project and then commit your changes.
There’s two ways to do this with UPM: file URLs and embedded packages.
The first means that you put a URL pointing to the local file path of your package’s repository into your project’s manifest.yml. That works fine but makes editing the package’s files awkward if you want the VSCode integration to work at all. You either have the package’s source code open in a separate VSCode so you can access all the source code files neatly from the VSCode sidebar but you lose all the useful OmniSharp stuff because that VSCode instance isn’t connected to the Unity project.
Alternatively, you can open your packages file by clicking them in the Unity project tab, under Packages, but it’s really inconvenient because it means you need to jump over to Unity every time you need to open a file you don’t already have open in a tab because VSCode won’t show your package’s source files in its sidebar unless you add the directories there manually.
The other alternative is embedded packages, which means you put your package directly into your project directory under the Packages directory. I can see my files nicely in the same VSCode window as the main project and the OmniSharp integration works fine.
Well, it never works fine, but it works about as well now as it does with any project.
Which brings us back full circle. Code that used to live under Assets/mtti/Pathfinding now goes to Packages/com.mattihiltunen.pathfinding. I could just clone the package’s repo there and add the path to the project’s .gitignore so I don’t commit it by accident but at that point it’s simpler to just use… a submodule.
God damn it.
So in the end, I decided to just try to live with submodules for now. I’ve started splitting my big gadget libraries into smaller ones which will live as submodules under the Packages directory. They will also have package.json files and will look like proper packages so that they can be transitioned to real packages when it becomes viable to do so.
Starting now and going forward, I’m rebranding these blog posts from devLog to devBlog and reserving the -log name to video devlogs on my mtti tech YouTube channel. I was supposed to start making them at the beginning of this year and fortunately it’s only the end of June so I’m only six months behind schedule. I’ll hopefully get started with them sometime during the coming decade.