20+ GIT Repos and Counting – How We Cope

At work, we’ve built up quite a few code repositories in Gitlab. On the cloud team alone we have 18 at the time I post this. We’ve talked about using git modules, subtrees, symlinks, and more but for one reason or another we eventually voted against adopting each of them. Instead each project gets a stand alone repo separate from, but perhaps dependent on, the others.

Primarily the cloud side consists of Go microservices, and Angular 2 4 5 websites but there’s also an Ethereum solidity contract and a node service for interacting with it (at least until a certain bug is fixed, then it’ll be converted to Go). Usually implementing a feature involves touching two or three of these repos. Want to add something to the database? We have a common package that’s has all the models, a microservice that will accept the new data, a website back end that reads the new data, and a website front end that shows it. Being a startup, we’re still adding features daily so it quickly becomes a guessing game of “Which repos did I touch?” or “Which repos have I commited to?” as well as “Am I on the right branch in all of those projects?” Back when we only had 9 or 10 repos we knew we had to fix this.

I ended up writing a small Go tool to help manage the current state of affairs. Git-status. Not the best name but does cover what happens when you run it. You add a repo usually by running git-status -add . inside the project’s main folder after it’s been added to a remote. This’ll add the path to ~/.git-status one path per line. Running it without any parameters will cause it to iterate over the saved repos running git status and a few other helpful commands in order to build a report on what repos have been modified, a clue to how they were modified, and what branch they’re on.

Here, the persist project on branch feature/session_auth has 1 available pull and 1 modified file. It has made a huge difference at work. It was quickly aliased as gs and I routinely run it a dozen times a day at least.

The project is open source over at bitbucket. A simple go get and you can be up and running with it in no time. The project’s goal is to never modify anything on disk or on your remotes but having the .git-status file in your home directory has been helpful for writing up various scripts to run against all the saved repos.

grep -v -E ^# ~/.git-status | xargs -n 1 -P 1 -I % sh -c "basename %; git -C % fetch"

This allows for lines in the file to be commented out by a # and it’ll run one at a time printing out it’s name followed by any output from git fetch. This script is easily modified for mass pulls, mass tags, or anything else you may want to do to the bulk of repos.

This little project has turned the headache that is managing 20+ repos from a drinking problem into a one line sanity check.

Leave a Reply

Your email address will not be published. Required fields are marked *