When it comes to your build analysis, you are likely running a nightly integration build - for more information, see qacli validate build. Helix QAC also offers the ability to run analysis for CI/CD pipelines in conjunction with your regular integration analysis. Helix QAC tools are designed to be DevOps ready, which makes it easy to include static code analysis as part of your continuous delivery pipelines.
Helix QAC’s analysis for CI/CD pipelines capability enables your organization to identify and communicate errors faster, without waiting for nightly builds. By identifying and communicating issues faster, developers can fix problematic code earlier.
Optionally, it is possible to analyze only the files that have changed since the last full build. This provides you with the shortest possible analysis times.
As developers update and commit code, the CI build system picks up the changes and performs builds throughout the day. Instead of waiting for nightly builds, Helix QAC CI analyzes the new code and notifies developers of problems right away. Here’s an overview of the process:
- Developers make changes to the code and commit them.
- The CI build system detects the changes and builds the software.
- Helix QAC CI analyzes the changes and generates a list of new issues and uploads them to Validate.
To implement, follow these steps:
- Determine when and where issues will be reported.
- Determine how to triage issues.
- Integrate qacli commands (such as
qacli validate cibuild) into your jobs. - Integrate reporting scripts into your jobs.
- Test to ensure everything is working as expected.
Limitations and Suggested Configuration
To accurately reflect defects introduced in a CI build, the analysis must be consistently repeatable and deterministic for a given set of source code. Otherwise, a CI build may report new defects with a build, which are in fact due to the non-determinism of an analysis run.
Behavior you may observe can include:
- You get an “unstable” CI build for code that analyzed successfully on a desktop, or the messages raised were from source that was not changed
- The messages raised are from Dataflow, or other known non-deterministic messages
- Subsequent CI builds show several “fixed” issues – which upon inspection turn out to be timeout messages.
Where possible you should remove or minimize any non-deterministic behavior from the CI build or accept that some reported defects will be introduced consequently.
Dataflow
A key aspect of the Dataflow component is query timeout, which strikes a balance between performance and query completion. Essentially, queries are sent to the solver and given a certain amount of time to be resolved or the solver times out. The duration of the timeout impacts the ability of Dataflow to determine if an issue is possible, definite or not an issue.
Depending upon factors such as machine load the same query may complete on one run, but not on another and this can lead to Dataflow determining that an issue is definite for one run where the query ran to completion, where it was only marked possible previously.
Should you find your CI builds produce inconsistent results as described above you can try the following:
- Fix issues raised if possible / feasible
- Suppress the messages that are likely to appear because dataflow timed out
- Suppressions can be done either with a code suppression, or in Validate itself
Incremental Analysis
If you choose to use the ‑‑incremental option, i.e. disable whole program analysis and only analyze files that have changed, then this option will automatically disable any whole program components, such as Dataflow, RCMA and MTA.
Server Build License Restrictions
To prevent issues where the server builds exhaust the license pool for analysis qacli validate build and qacli validate cibuild should be run with the same user across all servers.