In the discussion thread for my DevOps Skills class at Pluralsight, I got a question about how to do TFS Builds and TFS Release Management with code that spans multiple Git repositories. For good reasons, his team has separated their code into multiple repos. But when they go to create a build in TFS, the build templates only know about a single Git repository. So how do you pull that "best practice" Git repo structure into one unified automated build and create something that can be released?
Why Multiple Git Repositories?
If you're a TFS with TFVC developer, you never really worried too much about how big your version control repository got. TFVC (aka. Team Foundation Version Control) is fine with giant repositories. But when you make the move to Git, you'll probably start getting a little more thoughtful about how big your Git repository is getting. Since cloning a git repo puts the entire version control history on everyone's disk -- well, it can get big and you definitely care. TFVC (or any other centralize version control) only give you a single version of the code on your workstation at once. Git gives you everything.
There's the obvious stuff you should avoid doing if you care about about keeping your Git repo size to something manageable. Don't put dependencies in the repo (use NuGet or something like NuGet instead). Don't put your whole photo library or a bunch of zip files or Word docs or Excel spreadsheets into the repo. Basically, keep non-mergable files outside of the repo.
But what about your big sprawling code bases? Surely you can put all that into a single Git repository, right? Maybe. But also maybe not. It's pretty easy for your applications (especially enterprise apps and line-of-business apps) to get HUGE. Chances are very high that when you're working on that app that you don't need the whole thing all the time.
If your codebase is getting big, you might want to consider multiple Git repositories. But how do you bring this all together?
Git Submodules
The answer to this problem is Git Submodules. A submodule is a reference from one Git repository to one or more other repositories. This helps you to keep stuff organized without having to have one huge, giant git repo that rules the universe.
Assuming that you've got two Git repositories: Repo1 and Repo2. Let's say that they're hosted on Visual Studio Team Services (VSTS). They both have URLs:
- https://.visualstudio.com/DefaultCollection//_git/repo1
- https://.visualstudio.com/DefaultCollection//_git/repo2
Let's say that the code in Repo1 needs the code in Repo2 in order to be released. So you clone repo1 by going to the command line and typing:
git clone https://.visualstudio.com/DefaultCollection//_git/repo1
Great. That brings down the code for Repo1.
Next, create a submodule reference from Repo1 to Repo2.
git submodule add https://.visualstudio.com/DefaultCollection//_git/repo2
You've now got the code from Repo2 in a folder off the root of Repo1. You also get a new file named '.gitmodules'. That .gitmodules file contains the mapping from Repo1 to Repo2 and this file should be added, committed, and pushed to Repo1 along with any other changes that you make.
[caption id="attachment_9081" align="alignnone" width="462"]
The code from Repo1, the repo2 submodule code, and a .gitmodules file[/caption]
At this point, you've got the local workstation stuff working with submodules. Next, let's work on the automated build.
TFS Builds with Git Submodules
To make this work with TFS Build vNext, it's really just a matter of a single checkbox. In a web browser, go to the build definition and click 'Edit' to bring up the build editor. Click on the Repository tab to view the git repo settings for this build definition. Towards the bottom of the list, you'll see a checkbox option that says "Checkout submodules". Check that and hit save. That's it. When your build runs, the Git checkout step will follow the submodules that you defined in the .gitmodules file and put all that code on disk. After that, you can do whatever you need to do with that code from the second repository.
[caption id="attachment_9091" align="alignnone" width="665"]
On the repository tab of the build definition, check 'Checkout submodules'[/caption]
I hope this helped.
-Ben
-- Git gettin' ya down? Trying to get your Git repos into your DevOps pipelines? Not even sure what a 'DevOps' is? We can help. Drop us a line at info@benday.com.
