To get started building your own mainframe DevOps pipeline, we have provided pipeline examples and setup instructions using Jenkins. To implement a pipeline using other tools, refer to Mainframe CI using alternatives Jenkins.
Example code and documentation are provided for:
- A suggested starting point for Jenkins CI pipelines
- Compuware and 3rd party tools used in the pipelines
- Instructions setting up and configuring Jenkins and SonarQube
- Code snippets or sample code for:
The code published serves as example code, using Compuware example applications and environments. It needs to be adjusted to site specific needs and requirements.
The descriptions and tutorials assume a certain level of familiarity with using Jenkins, Topaz for Total Test and other Compuware tools. The required level of knowledge expected, is not too high. E.g. it helps to know, how to define a new job in Jenkins.
Code examples are stored in this GitHub repository https://github.com/cpwr-devops/DevOps-Examples/.
We have published several examples of "complete" pipelines which show different process steps and techniques in Jenkins.
Mainframe-CI-Example-pipeline - (jenkinsfile) - a scripted pipeline using parameters. This is a simple approach to a DevOps pipeline that allows you to get up and going quickly, but may not be the best when scaling pipelines across your enterprise. The job is intended to be triggered after promoting code within ISPW
This pipeline serves as the model for most of the other pipelines examples. Meaning, other pipelines follow the same structure, implementing bascially the process and just differing in nuances, e.g. the types of test being executed at different stages of the development process, and demonstrating certain techniques that prove to be helpful.
Mainframe_CI_Pipeline_from_Shared_Lib - (groovy) - a pipeline loaded from a Jenkins shared library. Shared Libraries are a useful approach to scale pipelines across an enterprise since it moves the bulk of the pipeline logic to shared components that individual pipelines can reference as steps. This allows organizations to develop pipelines in a more standard way. The job is also intended to be triggered after promoting code within ISPW.
Older versions of this pipeline made use helper classes to encapsulate some of the complexity and make the flow of the job easier to folow. Thanks to the changes in the Compuware CLI and plugins the complexity of the code required to implement such a pipeline has been reduced so much that the use of these classes does not seem to be necessary anymore.
Combined Pipeline: The following two pipeline definitions are part of a more elaborate process. They are called by a Jenkins job "distrubuting" the work to either of the scripts based on the ISPW operation (generat or promote) triggering the webhook and Jenkins job.
- Mainframe_Generate_Pipeline.groovy - a pipeline that is triggered by an ISPW Generate that executes virtualized tests against those components that have been generated.
- Mainframe_Integration_Pipeline.groovy - a pipeline that is triggered by an ISPW Promote that executes non virtualized tests against the project/assignment.
Also using the Shared Library technique, these two example show how Shared Library scripts can be used for modularization and re-use of existing code to implement such more elaborate processes.
Other Code examples
Next to the pipeline examples, we also publishe code snippets and short Jenkins jobs that demonstrate individual features and use cases of the Compuware plugins or CLIs.
The GitHub repository is organized as follows:
- vars folder: example pipelines using Shared Library technology.
- src folder: class definitions for code related to these pipelines.
src/Jenkinsfilefolder: example code not directly related, but that define Jenkins jobs. Currently these are:
- JCL_Pipeline_Example contains a simple example of mainframe jobs being submitted from Jenkins — both with the JCL residing on the mainframe and the JCL stored/generated in the pipeline code itself.
- Three examples of downloading sources (COBOL programs and copybooks) from the mainframe using different download stores for the code and methods to download, pushing the sources to SonarQube using the Sonar scanner and querying the resulting Sonar Quality Gate.
- Scan_Sources_from_ISPW_Container_with_Sonar - using the container downloader for sources stored in ISPW.
- Scan_Sources_from_ISPW_Repository_with_Sonar - using the repository downloader for sources stored in ISPW.
- Scan_Sources_from_PDS_with_Sonar - uses the PDS downloader for sources stored in PDS's (inside or outside a mainframe SCM tool).
- Push_TTT_results_to_Git - (jenkinsfile) - shows how to push results of unit test execution back to GitHub for a developer to consume locally.
- misc-examples/Powershell sub-folder: that resides within the src directory of the repository, Powershell scripts and examples for using alternatives to Jenkins or the Jenkins plugins**. These scripts make use of the ISPW REST API and the Topaz CLI, and we describe them in detail in Mainframe CI using alternatives Jenkins.
What has changed in April 2021
Over the recent few releases of Topaz for Total Test, the Topaz CLI, ISPW, the ISPW CLI and the corresponding Jenkins plugins, integration between different tools has been greatly improved, moving much of the complexity of implementing certain requirements from the code of the scripts into functionality provided by the plugins/CLI. Therefore, many of the example pipelines will have become far simpler, compared to the older versions, while providing the same flexibility and functionality as before. For customers this means that the coding effort and complexity of the resulting pipeline scripts has been reduced considerably.
The example pipelines use a development scenario based on:
- ISPW as the SCM to store and govern mainframe sources
- Git (GitHub) as the SCM to store unit test assets.
- Topaz for Total Test as the mainframe unit, functional, and integration testing tool to create and maintain test assets.
- Xpediter Code Coverage as the tool to gather code coverage data during execution of the unit tests.
- SonarQube as the server for code analysis and setting up quality gates.
- XLRelease as the CD server for release steps following the initial CI process Jenkins.
Instructions for configuring the various tools can be found in the Configuration section of this site.
The code repository folder structure
Based on the description above and due to the requirements for the use of Pipeline Shared Libraries in Jenkins, the folder structure of the DevOps-Examples repository is as follows:
(root) +- resources # Configuration files used by the pipelines +- src # (Groovy) source files | +- Jenkinsfile # "simple" example job scripts | +- misc-examples # non Jenkins related (non Groovy) code examples | +- AzureDevOps | +- PipelineYAML # YAML file(s) describing Azure DevOps pipelines (builds or releases) | +- Powershell # Powershell scripts being used by an Azure DevOps example pipeline +- vars # Shared Library Pipeline Examples