Known Issues Support YAML Policies Task Parameters Task Variables Policy Results Common Usage Scenarios FAQ

Build Quality Checks

The Build Quality Checks task allows you to add quality gates to your build process.

Change Notes

You can find the changes notes for this task here.

Known Issues

Support

If you need help with the extension or run into issues, please contact us at psgerextsupport@microsoft.com or create an issue here.

Adding the Task to a Build Definition

The Build Quality Checks (in task category Build) task needs to be placed after the tasks it should inspect within the same job. If you only need the code coverage policy, you may place the task in a separate job that runs after all other jobs that generate code coverage; code coverage data is aggregated across multiple jobs. In a Visual Studio build definition, e.g., an appropriate place would be after build, test, and symbol indexing/publishing. This ensures that, even if the policy breaks the build, you still get test results as well as the compile output and symbols.

Task Placement

Note: When you want to use the Code Coverage Policy you need to make sure that you publish code coverage values calculated by your test tool first. The Policy itself does not calculate the code coverage. Before you add the task and activate the policy, please make sure that you can already see code coverage values in your build summary. If you are using a test tool other then MSTest (i.e., anything other than the Visual Studio Test task), please use the Publish Code Coverage Results task to publish your coverage data before you add the policy.

Adding the Task to a YAML Build Definition

To add the Build Quality Checks task to a YAML build definition, use the task name and major version like this - task: BuildQualityChecks@9 and set a display name using the displayName property. Then add all task inputs as described under Task Parameters or in the Policies section.

YAML snippet:

- task: BuildQualityChecks@9
  displayName: 'Check build quality'
  inputs:
    # ===== Warnings Policy Inputs =====
    #checkWarnings: false # Optional
    #warningFailOption: build # Optional; Valid values: build, fixed
    #warningThreshold: '0' # Optional
    #forceFewerWarnings: false # Optional
    #allowWarningVariance: false # Optional
    #warningVariance: # Required if allowWarningVariance = true
    #showStatistics: false # Optional
    #evaluateTaskWarnings: true # Optional
    #warningTaskSelectors: '/^((vs|ms)build|ant(\s+.+)?|gradle(w)?(\s+.+)?|grunt|gulp|maven(\s+.+)?|xamarin(android|ios)|xcode(\s+.+)?|cmake|build\s+.+)$/i' # Optional (alias: warningTaskFilters)
    #warningSelectors: # Optional (alias: warningFilters)
    #inclusiveSelection: false # Optional (alias: inclusiveFiltering)
    #warningExclusions: # Optional
    #evaluateFileWarnings: false # Optional
    #warningFilesFolder: # Optional
    #warningFiles: # Required if evaluateFileWarnings = true
    #fileWarningSelectors: # Required if evaluateFileWarnings = true (alias: warningFileFilters)
    #fileWarningExclusions: # Optional
    #warningFilesArtifact: # Required if evaluateFileWarnings = true and (warningFailOption = build or showStatistics = true)
    # ===== Code Coverage Policy Inputs =====
    #checkCoverage: false # Optional
    #coverageFailOption: build # Optional; Valid values: build, fixed
    #coverageType: blocks # Optional; Valid values: blocks, lines, branches, custom
    #customCoverageType: # Required if coverageType = custom
    #treat0of0as100: false # Optional
    #coverageThreshold: '60' # Optional
    #forceCoverageImprovement: false # Optional
    #coverageUpperThreshold: '80' # Optional
    #ignoreDecreaseAboveUpperThreshold: true # Optional
    #useUncoveredElements: false # Optional
    #allowCoverageVariance: false # Optional
    #coverageVariance: # Required if allowCoverageVariance = true
    #coverageDeltaType: percentage # Optional; Valid values: absolute, percentage
    #coveragePrecision: '4' # Optional
    #buildConfiguration: # Optional
    #buildPlatform: # Optional
    #explicitSelector: false # Optional (alias: explicitFilter)
    # ===== Baseline Inputs =====
    #includePartiallySucceeded: true # Optional
    #fallbackOnPRTargetBranch: true # Optional
    #baseDefinitionSelector: # Ignored - only used by UI editor
    #baseDefinitionId: # Optional
    #baseRepoId: # Ignored - only used by UI editor
    #baseBranchRef: # Optional
    # ===== Reporting Inputs =====
    #runTitle: # Optional
    #fileAnalysisTitle: # Optional
    # ===== Advanced Inputs =====
    #disableCertCheck: false # Optional
    #createBuildIssues: true # Optional
    #addAttachment: true # Optional

Policies

The Build Quality Checks task currently supports two policies (click the link for details):

Task Parameters

Baseline

If you choose Previous Value for the Fail Build On option for one of the policies, the policy value (e.g., number of warnings) is, by default, compared to the corresponding value from the last build that ran for the current build definition. If the build definition targets a Git repository hosted in Azure DevOps Services or Team Foundation Server, the policy will look for the last build that ran against the same branch as the current build. This behavior can be customized with the following parameters:

Baseline Parameters

Reporting Options

Reporting Options

Advanced

Advanced Parameters

Task Variables

In addition to parameters visible in the task UI there are a few variables you can set to affect the tasks behavior:

Policy Results

Policy results are returned in various ways to make it easy for you to see and use them in different scenarios:

Build Issue List

If the advanced option Log task results as build issues is enabled (default), the task reports policy violations as regular build issues. These are listed with all other issues on the build’s summary page:

Policy Violations

Extension Summary Section

The Build Quality Checks task creates its own summary section in the build summary view. This section displays all success, warning, and error messages for all activated policies. If you run a multi-configuration build, a subsection for each build job is created so you can see exactly which configuration might have quality issues.

Policy Result

Note: Please see the Limitations and Special Cases section of the Code Coverage Policy for possible issues with multi-configuration builds and code coverage.

Policy Result Variables

In addition, each policy you run creates an output variable that you can use to conditionally run other tasks in your pipeline (e.g., create a technical debt work item if quality metrics get worse). See policy documentation for more information.

Pull Request Statuses

If you run the task as part of a pull request validation build, each policy also publishes its result as a pull request status:

Branch Policy Results

See Pull Requests and TFVC Feature Branches for more information.

Important: If you want to use the pull request features of the task, please ensure that your build account has the Contribute to pull requests permission! See Git repository permissions for more information. Depending on your build job authorization scope you need to either give that permission to your project build account (e.g., YourProject Build Service (YourOrganization)) or your project collection build account (e.g., Project Collection Build Service (YourOrganization)).

Common Usage Scenarios

FAQ

We have put together a list of frequently asked questions and answers in our FAQ document. If you feel we need to add a specific question to the list, feel free to send it to our support address.

Checklist board icon Icon designed by Vexels.com