Jenkins X Pipelines
In continuous delivery (CD) environments, a pipeline is a process (expressed as a collection of commands or plugins and a configuration file to express the development process) that automates the life cycle from repository source files to production deployment.
Jenkins X Pipelines is a serverless pipeline execution engine based on the Tekton Pipelines open source project. Tekton has been designed to be a modern cloud native solution for running pipelines.
Jenkins X pipelines are configured in YAML configuration files. The files can be found in two locations serving distinct purposes:
- In the Jenkins X project repository, called
- In the build packs for creating applications, if it is specified in the project repository
Each pipeline YAML file has a number of separate logical pipelines:
releasefor processing merges to the master branch which typically creates a new version and release then triggers promotion
pullRequestfor processing Pull Requests
featurefor processing merges to a feature branch. Consider using trunk based development which is a practice of high performing teams.
Jenkins X has various steps in building, validating, and releasing your application through the development lifecycle. The lifecycle phases in the Jenkins X pipeline YAML configuration are:
setup- Steps to create the build environment, such as checking out code with git checkout or generating credentials files for Git provider authentication
preBuild- Steps to perform before a build occurs, such as ensuring a Docker image registry is available for building
build- Steps performed to build your application
postBuild- Steps performed after the build occurs, such as validating for Common Vulnerability Exposure (CVE) in any code changes.
promote- Shifting the state of an application (after build and validation) to another environment, such as Staging or Production.
Understanding Jenkins X pipelines
The Jenkins X cluster configuration process creates a YAML-based pipeline configuration file called jenkins-x.yml. This file configures the default development pipeline for building applications on kubernetes clusters with Jenkins X.
buildPack: none pipelineConfig: pipelines: release: pipeline: agent: image: gcr.io/jenkinsxio/builder-go
buildPack specifies a build pack which contains a
pipeline.yml file that
jenkins-x.yml file in the project directory. If none is
specified, there is no build pack and Jenkins X uses the default pipeline
The configuration defines the pipeline agent, in this case a Google Container Registry image for the Go language build tools.
environment: - name: DEPLOY_NAMESPACE value: jx
environment specifies environment variables used in the pipeline
configuration. In this instance, the
DEPLOY_NAMESPACE variable is used with a
jx for the Jenkins X namespace.
stages: - name: release steps: - name: verify-preintall dir: /workspace/source/env command: jx args: ['step','verify','preinstall']
stages are unique groups of steps (or nested stages sequentially run within a
stage) that specify commands, directories, and arguments for a particular
pipeline stage. In this instance, there is a step within the
verify-preinstall that runs a
jx command that verifies whether cloud
infrastructure (such as the presence of the
kubectl binary and the correct
version of git is installed) was setup in the preinstallation process.
- name: install-vault dir: /workspace/source/systems/vault command: jx args: ['step', 'boot','vault']
name calls out a unique step in the pipeline configuration that defines
development steps to verify and apply arguments to various commands necessary
for the stage under which it is nested. In this instance,
installs the Hashicorp Vault tool for secrets management.
- name: apply-repositories dir: /workspace/source/repositories command: jx args: ['step','helm','apply', '--name', 'repos']
This step creates and applies the Helm Package Manager for installation and management of helm kubernetes applications.
- name: apply-pipeline-schedulers dir: /workspace/source/prowConfig command: jx args: ['step','scheduler','config', 'apply', '--direct=true']
This step allows the pipeline to work with a scheduler, which executes program jobs unattended in the background.
- name: update-webhooks dir: /workspace/source/repositories command: jx args: ['update','webhooks','--verbose', '--warn-on-fail']
This step updates webhooks, which is a service that listens for GitHub activity and trigger jobs, send automated messages to chat clients such as Slack, and other configurable actions.
- name: verify-install dir: /workspace/source/env command: jx args: ['step','verify','install', '--pod-wait-time', '30m']
This step verifies the project installation, downloading and installing or updating components when necessary.
pullRequest: pipeline: agent: image: gcr.io/jenkinsxio/builder-go
pullRequest is a logical pipeline within the project pipeline that specifies how pull requests are managed when changes are made to the project repository in GitHub.
stages: - name: release steps: - name: helm-build dir: /workspace/source/env command: make args: ['build']
pullRequest pipeline contains a stage wherein steps can also be executed. In this instance, make is run to create a helm chart and validate that a build has been completed.
A pipeline YAML can extend another YAML file. You can reference a base pipeline YAML using the following methods:
Using file to reference a relative file path in the same build pack
extends: file: ../jenkins-x.yaml
Using import to reference a YAML file:
extends: import: classic file: maven/pipeline.yaml
which then refers to a named imported module via git:
modules: - name: classic gitUrl: https://github.com/jenkins-x-buildpacks/jenkins-x-kubernetes.git gitRef: master
Users can override steps in a pipeline YAML from a base pipeline YAML, similar to overriding classes in languages like Java. This allows users reuse the steps in a base pipeline’s lifecycle, then add additional steps.
By default any steps you define are added after the base pipeline YAML steps. For example:
extends: file: base-pipeline.yaml pipelines: pullRequest: build: steps: - sh: export VERSION=$PREVIEW_VERSION && skaffold build -f skaffold.yaml
You can add steps before the base pipeline steps using the preSteps: property:
extends: file: base-pipeline.yaml pipelines: release: setup: preSteps: - sh: echo BEFORE BASE SETUP steps: - sh: echo AFTER BASE SETUP build: replace: true steps: - sh: mvn clean deploy -Pmyprofile comment: this command is overridden from the base pipeline
If you want to completely replace all the steps from a base pipeline for a particular lifecycle you can use replace: true:
replace: true steps: - sh: mvn clean deploy -Pmyprofile comment: this command is overridden from the base pipeline
Was this page helpful?
Glad to hear it! Please tell us how we can improve.
Sorry to hear that. Please tell us how we can improve.