In the last article about Automation (101), I presented you quickly how Microsoft managed across their teams to follow a “well-understood and consistent mechanisms to consume, update, and share engineering [culture]”. Thankfully to the Open Sourcing of the code but not only, the opening of their process. Now that we know the what and partially the how, can we reproduce something alike? Welcome to this guided tour 😊

Before all, this article is about to share a vision and experiment: it is not yet a guideline neither a rule! If you are interested to follow or adapt it, see with your Team and/or EM|PA before :-) It will also imply that we are working in a .NET Core project with Visual Studio 2019 as main IDE. If you are using an alternative IDE like Rider or VSCode, the Making your IDE your friend won’t help you directly (check the compatibility matrix of your IDE).

We will suppose that your environment is setup and running: Visual Studio, git4windows, a terminal (Windows Terminal, cmder or even Git Bash) and PowerShell. Btw, I will try to follow as much as possible the conventional commits specification.

Preparing the battle ground

First to be able to do automation we need that the VSC file structure can support our needs. As said before we will follow a modern global folder pattern: docs, src, tests, eng, scripts, tools. Let us see what go where:

• docs: all the related docs for your project and team.
• src: all the source code of your project, but only the main one.
• tests: all the tests related source code
• eng: all the configuration related to Engineering guideline and other.
• scripts: all the scripts used in your solution (not sql, powershell | bacch one)
• tools: all the source code of tools implemented or used alongside your project

From here we will begin from scratch with a new, empty, repository from TFS: Automation.Playground

Creating the base folders

• mkdir docs,src,tests,eng,scripts,tools

Adding basic git files: .gitignore & .gitattributes

I will import the gitignore file from Github template one, and the gitattributes file from dotnet/runtime and remove the specific lines from the project.

• curl -O https://raw.githubusercontent.com/dotnet/runtime/master/.gitattributes
• curl -o .gitignore https://raw.githubusercontent.com/github/gitignore/master/VisualStudio.gitignore

Adding a global.json

Here we know that our team works with .NET Core and for the moment the latest 2.X LTS version.

• dotnet new globaljson --sdk-version 2.1.800

For safety reason, we will block to only use the latest LTS in our project by following this doc.

1
2
3
4
5
6
7

{
  "sdk": {
    "version": "2.1.800",
    "allowPrerelease": false,
    "rollForward": "latestFeature"
  }
}

Adding a basic project

Before continuing further, we will need to setup a little empty project!

1. Go to src/ and do a new project: dotnet new console -n Playground

   

2. Open the generated project and save the solution file in the root folder
3. Add the other existing files into the solution, (Right click on solution > Add existing item > select them * Add) and Save

   

4. Add these solution folders: src, docs, tests
   a. Move the Playground project inside src
5. Add a XUnit Test Project into the solution folder tests and the corresponding physical folder
6. Add a .NET Standard class library into the solution folder src and the corresponding physical folder
7. Link everything correctly
8. Build

Now we have a basic but solid base ground to continue our work, we have:

• Clean folder structure
• Clean .NET friendly git setup
• Failsafe .NET Core environment

Making the IDE your friend

In the developer worlds, there are two kinds of developer: the one who use IDE and the one that use editor. The first kind prefers to have a one tools for all, the second one use multiple light tools with automated behavior. Happily, for us most of .NET Software Engineer use IDEs for the sake of the learning curve and Microsoft support: Visual Studio. But do they use it fully? Let make the usage percentage from 10% to 15%! Before putting or hand in the oils, what can we do with it? Remember your notes from the last experiment with your friends? Let’s take a look:

• He had to setup Visual Studio correctly for the app.
• He didn’t find the documentation about the needed Visual Studio extensions
• He had problem to follow the code convention
• He used dangerous behavior in his code
• He created a project using a to advanced runtime
• ?

Adding a Visual Studio Configuration files

Go to the Solution > right click > Installation Configuration File

But before, review the configuration to only include what the project needs:

• .NET Core cross-platform development
• Other for the fun?
• Export it!

Adding a coding convention

We will follow the one used by Microsoft .NET Core teams for simple reason: first all the code your will read from their documentation will be the same as the one used at https://source.dot.NET/ as the one in the .NET Core context!

• curl -O https://raw.githubusercontent.com/dotnet/runtime/master/.editorconfig

Don’t forget to add the file in the Solution Items folder too!

Add the Teams VS extensions

Now that we have a coding convention, it could be wise to make it easy to follow. How? By using some useful Visual Studio extensions 😊 You can find them on the dedicated marketplace, for now we will use the one that we introduced into Titan project, as:

• Format document on Save: to help use to make it automatic and headache free
• Editor Guidelines: to setup + see the maximum length of a line of code
• Indent Guides: to see easily scopes
• Trailing Whitespace Visualizer: to spot unwanted space
• Visual Studio Spell Checker (VS2017 and Later): to correct our English
• Visual Studio IntelliCode: to help use with the Intellisence
• File Icons: to help us to distinguish more easily the type of file
• Extension Manager 2019: to make easy to setup everything

After having installed everything you can go to the Solution > right click > Manage Extensions and save the file (Playground.vsext) in the root folder.

The only things know to document in our Team charter is to install at minima the Extension Manager 2019 extension and that it’s. Of course each extension has it own set of settings that you can export/save, be free to do so.

Adding a one for all configuration file

After all this simple advancement, let’s begin to touch Visual Studio and MsBuild configuration file, can we? The goal will be to help VS to know where is the source code folder, what configuration we want to apply to them etc.. If you want to learn more about it, check this out!

Create a Directory.Build.props in the root folder

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12

<Project TreatAsLocalProperty="RepoRoot">
  <PropertyGroup>
    <ImportDirectoryBuildProps>false</ImportDirectoryBuildProps>
  </PropertyGroup>

  <PropertyGroup>
    <RepoRoot>$([MSBuild]::EnsureTrailingSlash('$(MSBuildThisFileDirectory)'))</RepoRoot>
    <RepositoryEngineeringDir>$([MSBuild]::NormalizeDirectory('$(RepoRoot)', 'eng'))</RepositoryEngineeringDir>
  </PropertyGroup>

  <Import Project="$(RepositoryEngineeringDir)Configurations.props" />
</Project>

Here we define that the main root folder musn’t be seen as the root source code one, so not imported and seen as the RepoRoot. We define the engineering folder and a Configuration.props configuration properties file.

Create a Directory.Build.targets in the root folder too

1
2
3
4
5
6
7
8
9

<Project>
  <PropertyGroup>
    <!--
    For non-SDK projects that import this file and then import Microsoft.Common.targets,
    tell Microsoft.Common.targets not to import Directory.Build.targets again
    -->
    <ImportDirectoryBuildTargets>false</ImportDirectoryBuildTargets>
  </PropertyGroup>
</Project>

Adding the configuration file

1
2
3
4
5
6
7
8

  <!-- Package configuration -->
  <PropertyGroup>
    <Company>Mantu</Company>
    <Product>Automation.Playground</Product>
    <Authors>DevOps</Authors>
    <RepositoryUrl>https://tfs.amaris.com/tfs/ITArchitecture/DevOps/_git/Automation.Playground</RepositoryUrl>
    <RepositoryType>git</RepositoryType>
  </PropertyGroup>

Here we setup some global configuration settings as the team name, product name and repository url etc.. Mostly used in the assembly and nuget package output :-) PS: using `latest` in your option could break your build in the CI in some rare cases!

1
2
3
4
5

<!-- Language configuration -->
  <PropertyGroup>
    <LangVersion>latest</LangVersion>
    <Deterministic>true</Deterministic>
  </PropertyGroup>

This one permit us to setup for all projects inside the solution which version of the C# language we want to use and asking the compiler to make a deterministic build (for example only*).

1
2
3
4
5
6
7

<!-- Build configuration -->
  <PropertyGroup Condition="'$(Configuration)|$(Platform)'=='Debug|AnyCPU'">
    <CheckForOverflowUnderflow>true</CheckForOverflowUnderflow>
  </PropertyGroup>
  <PropertyGroup Condition="'$(Configuration)|$(Platform)'=='Release|AnyCPU'">
    <CheckForOverflowUnderflow>true</CheckForOverflowUnderflow>
  </PropertyGroup>

To finish we add some build configuration for the fun 👿

Adding source code analyzers

One of the most famous things about .NET world are the source code analyzers who help developer to detect, explain and in the best case fix know problems. We will setup StyleCop.Analyzers and FxCopAnalyzers.

Create a Analyzers.props in the eng folder

For .NET Framework related

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14

<Project>
  <PropertyGroup>
    <CodeAnalysisRuleset>$(MSBuildThisFileDirectory)CodeAnalysis.ruleset</CodeAnalysisRuleset>
    <RunAnalyzersDuringBuild>false</RunAnalyzersDuringBuild>
  </PropertyGroup>
  <ItemGroup>
    <PackageReference Include="Microsoft.CodeAnalysis.FxCopAnalyzers" Version="3.0.0" PrivateAssets="all">
      <IncludeAssets>runtime; build; native; contentfiles; analyzers; buildtransitive</IncludeAssets>
    </PackageReference>
    <PackageReference Include="StyleCop.Analyzers" Version="1.2.0-beta.261" PrivateAssets="all">
      <IncludeAssets>runtime; build; native; contentfiles; analyzers; buildtransitive</IncludeAssets>
    </PackageReference>
  </ItemGroup>
</Project>

For .NET Core related (need .NET 5 SDK installed locally)

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12

<Project>
  <PropertyGroup>
    <CodeAnalysisRuleset>$(MSBuildThisFileDirectory)CodeAnalysis.ruleset</CodeAnalysisRuleset>
    <AnalysisLevel>latest</AnalysisLevel>
    <AnalysisMode>AllEnabledByDefault</AnalysisMode>
    <EnforceCodeStyleInBuild>true</EnforceCodeStyleInBuild>
  </PropertyGroup>
  <ItemGroup>
    <PackageReference Include="Microsoft.CodeAnalysis.NETAnalyzers" Version="5.0.*" PrivateAssets="all" />
    <PackageReference Include="StyleCop.Analyzers" Version="1.2.*-*" PrivateAssets="all" />
  </ItemGroup>
</Project>

Create a Analyzers.test.props in the eng folder

1
2
3
4
5
6

<Project>
  <PropertyGroup> 
      <CodeAnalysisRuleset>$(MSBuildThisFileDirectory)CodeAnalysis.test.ruleset</CodeAnalysisRuleset>
      <RunAnalyzersDuringBuild>false</RunAnalyzersDuringBuild>
  </PropertyGroup>
</Project>

Of course, we need to import it too we need to add this line into the root props file

1
2

  <Import Project="$(RepositoryEngineeringDir)Analyzers.props" /> 
  <Import Project="$(RepositoryEngineeringDir)Analyzers.test.props" Condition="'$(IsTestProject)' == 'true'"/>

Import CodeAnalysis.ruleset & CodeAnalysis.test.ruleset in the eng folder.

Something hard to do when you arrive in the world of source code analysis are the rules.. Here we will follow the one from Microsoft .NET Core team too (for the ease)

For .NET Framework

• curl -Ohttps://raw.githubusercontent.com/dotnet/runtime/83e56c474af7127d2909e5d4a11400964abb3ac9/eng/CodeAnalysis.ruleset
• curl -Ohttps://raw.githubusercontent.com/dotnet/runtime/master/eng/CodeAnalysis.test.ruleset

For .NET Core

• curl -Ohttps://raw.githubusercontent.com/dotnet/runtime/master/eng/CodeAnalysis.ruleset
• curl -Ohttps://raw.githubusercontent.com/dotnet/runtime/master/eng/CodeAnalysis.test.ruleset

Linking our configuration file

Now that we have setup all our basic, we still need to tell which folder is what. You don’t want to run test.rules on your Web/Api project =P

Create a Directory.Build.props & Directory.Build.targets in the src folder

1
2
3
4
5
6
7
8

<Project>
  <Import Project="..\Directory.Build.props" />

  <!-- Common properties -->
  <PropertyGroup>
    <IsTestProject>false</IsTestProject>
  </PropertyGroup>
</Project>

1
2
3

<Project>
  <Import Project="..\Directory.Build.targets" />
</Project>

Create a Directory.Build.props & Directory.Build.targets in the test folder

1
2
3
4
5
6
7

<Project>
  <Import Project="..\Directory.Build.props" /> 
  <!-- Common properties -->
  <PropertyGroup>
    <IsTestProject>true</IsTestProject> 
  </PropertyGroup> 
</Project>

1
2
3

<Project>
  <Import Project="..\Directory.Build.targets" />
</Project>

Using an existing battleground

Sometime we don’t have the luck to have green-space project but this don’t mean we can’t do it to. Begin the process of preparing the battleground, after creating the new needed folders you will needs to move all the stuff around. For this I advice you to use git mv (doc), why? Git need to know where the file was and where it goes, to permit your team to have a painless experience of moving/renaming files. (and allow the tracking too)

In some lines:

• Do a clean (in VS, dotnet cmd or even git)
• Create the base structure folder
• Move folder/file with git mv
• Edit your sln file to update the csproj link
• Update your nuget path/restore by reinstalling them
• Update your .gitignore file
• Follow the rest of this post

After pushing the modification

After that the SE have pull the change, it’s adviced to run:

• dotnet clean XXX.sln, to remove temporary build & cache files
• git clean -f -d -x -e src/**/.csproj.user -e .vs/ -e tools/**/*.csproj.user -e tests/**/*.csproj.user -n, to check any untracked files who could have been miss
  • if nothing important is displayed, please redo it without the -n, to remove any files not needed

To sum up

If you followed this series of “how to automate”, you could ask yourself why we need to have a structured repository structure? Consequences of these changes aren’t visible directly but here are they:

• Separation of concern from the repository / code
• CI enabler
  • Faster checkout for release build (need only to checkout src folder)
  • Easier CI setup for tests (need only to build and run every project in tests folder)
  • Easier & automated PR review
    • Invite groups / people only if a change from a specific folder happens (QC for any tests/**  changes, SE for any src/** changes etc.)
    • PR Reviewer know the impact of changes
• Guidelines / convention over configuration
• Product cross-team alignment

What we have done:

• Enforce Visual Studio tooling/setup per repository/application
• Enforce code guideline/convention
• Enforce code style guideline/convention
• Enforce quality by setting up analyzer
• Align our SE on the Microsoft code standard
  • Easier to read code from https://referencesource.microsoft.com/ or https://source.dot.NET/ or related github
  • Less behavior surprise by using Microsoft.* packages

What could be done to improve:

• Use the docs folder to store the public wiki. and setup a Wiki as code
• Setup a guideline for the name of tests projects
  • Like, {NameOfLayer}.{TypeOfTest}.csproj
    • Application.UT.csproj, Application.IT.csproj, Application.FZ.csproj..
• ?

Open questions

• It would be interesting to have a TFS template repository (as Github do) to have a clean updated out-of-the-box experience. How can we do it?
• Analyzer rules and other configuration could be centralized as the dotnet/arcade project, with git subtree or manually sync, and maintained by the Engineering Manager and Product Architect

After settings all the necessary software, we still miss some point as:

• Documentation
  • for our Teams
  • for our Users
• Scripting
  • for our Team
• Enforcing the rules, locally or remotely (CI)

In the next article Automation 103 I will present you an example of  scripting for making the developer environment setup, how to test it and how to enforce some of our today work.

Photo by Rob Lambert on Unsplash