Jenkins X Pipelines
In continuous delivery (CD) environments, a pipeline is a process (expressed as a collection of commands or plugins and a configuration file to express the development process) that automates the life cycle from repository source files to production deployment.
Jenkins X Pipelines is a serverless pipeline execution engine based on the Tekton Pipelines open source project. Tekton has been designed to be a modern cloud native solution for running pipelines.
Jenkins X pipelines are configured in YAML configuration files. The files can be found in two locations serving distinct purposes:
- In the Jenkins X project repository, called
jenkins-x.yml. - In the build packs for creating applications, if it is specified in the project repository
pipeline.yamlfile underbuildPack.
Pipeline types
Each pipeline YAML file has a number of separate logical pipelines:
releasefor processing merges to the master branch which typically creates a new version and release then triggers promotionpullRequestfor processing Pull Requestsfeaturefor processing merges to a feature branch. Consider using trunk based development which is a practice of high performing teams.
Lifecycles
Jenkins X has various steps in building, validating, and releasing your application through the development lifecycle. The lifecycle phases in the Jenkins X pipeline YAML configuration are:
-
setup- Steps to create the build environment, such as checking out code with git checkout or generating credentials files for Git provider authentication -
preBuild- Steps to perform before a build occurs, such as ensuring a Docker image registry is available for building -
build- Steps performed to build your application -
postBuild- Steps performed after the build occurs, such as validating for Common Vulnerability Exposure (CVE) in any code changes. -
promote- Shifting the state of an application (after build and validation) to another environment, such as Staging or Production.
Understanding Jenkins X pipelines
The Jenkins X cluster configuration process creates a YAML-based pipeline configuration file called jenkins-x.yml. This file configures the default development pipeline for building applications on kubernetes clusters with Jenkins X.
buildPack: none
pipelineConfig:
pipelines:
release:
pipeline:
agent:
image: gcr.io/jenkinsxio/builder-go
buildPack specifies a build pack which contains a pipeline.yml file that
supersedes the jenkins-x.yml file in the project directory. If none is
specified, there is no build pack and Jenkins X uses the default pipeline
configuration.
The configuration defines the pipeline agent, in this case a Google Container Registry image for the Go language build tools.
environment:
- name: DEPLOY_NAMESPACE
value: jx
environment specifies environment variables used in the pipeline
configuration. In this instance, the DEPLOY_NAMESPACE variable is used with a
value of jx for the Jenkins X namespace.
stages:
- name: release
steps:
- name: verify-preintall
dir: /workspace/source/env
command: jx
args: ['step','verify','preinstall']
stages are unique groups of steps (or nested stages sequentially run within a
stage) that specify commands, directories, and arguments for a particular
pipeline stage. In this instance, there is a step within the release stage
called verify-preinstall that runs a jx command that verifies whether cloud
infrastructure (such as the presence of the kubectl binary and the correct
version of git is installed) was setup in the preinstallation process.
- name: install-vault
dir: /workspace/source/systems/vault
command: jx
args: ['step', 'boot','vault']
name calls out a unique step in the pipeline configuration that defines
development steps to verify and apply arguments to various commands necessary
for the stage under which it is nested. In this instance, install-vault
installs the Hashicorp Vault tool for secrets management.
- name: apply-repositories
dir: /workspace/source/repositories
command: jx
args: ['step','helm','apply', '--name', 'repos']
This step creates and applies the Helm Package Manager for installation and management of helm kubernetes applications.
- name: apply-pipeline-schedulers
dir: /workspace/source/prowConfig
command: jx
args: ['step','scheduler','config', 'apply', '--direct=true']
This step allows the pipeline to work with a scheduler, which executes program jobs unattended in the background.
- name: update-webhooks
dir: /workspace/source/repositories
command: jx
args: ['update','webhooks','--verbose', '--warn-on-fail']
This step updates webhooks, which is a service that listens for GitHub activity and trigger jobs, send automated messages to chat clients such as Slack, and other configurable actions.
- name: verify-install
dir: /workspace/source/env
command: jx
args: ['step','verify','install', '--pod-wait-time', '30m']
This step verifies the project installation, downloading and installing or updating components when necessary.
pullRequest:
pipeline:
agent:
image: gcr.io/jenkinsxio/builder-go
pullRequest is a logical pipeline within the project pipeline that specifies how pull requests are managed when changes are made to the project repository in GitHub.
stages:
- name: release
steps:
- name: helm-build
dir: /workspace/source/env
command: make
args: ['build']
The pullRequest pipeline contains a stage wherein steps can also be executed. In this instance, make is run to create a helm chart and validate that a build has been completed.
Extending pipelines
A pipeline YAML can extend another YAML file. You can reference a base pipeline YAML using the following methods:
-
Using file to reference a relative file path in the same build pack
extends: file: ../jenkins-x.yaml -
Using import to reference a YAML file:
extends: import: classic file: maven/pipeline.yaml
which then refers to a named imported module via git:
modules:
- name: classic
gitUrl: https://github.com/jenkins-x-buildpacks/jenkins-x-kubernetes.git
gitRef: master
Overriding steps
Users can override steps in a pipeline YAML from a base pipeline YAML, similar to overriding classes in languages like Java. This allows users reuse the steps in a base pipeline’s lifecycle, then add additional steps.
By default any steps you define are added after the base pipeline YAML steps. For example:
extends:
file: base-pipeline.yaml
pipelines:
pullRequest:
build:
steps:
- sh: export VERSION=$PREVIEW_VERSION && skaffold build -f skaffold.yaml
You can add steps before the base pipeline steps using the preSteps: property:
extends:
file: base-pipeline.yaml
pipelines:
release:
setup:
preSteps:
- sh: echo BEFORE BASE SETUP
steps:
- sh: echo AFTER BASE SETUP
build:
replace: true
steps:
- sh: mvn clean deploy -Pmyprofile
comment: this command is overridden from the base pipeline
If you want to completely replace all the steps from a base pipeline for a particular lifecycle you can use replace: true:
replace: true
steps:
- sh: mvn clean deploy -Pmyprofile
comment: this command is overridden from the base pipeline
Feedback
Was this page helpful?
Glad to hear it! Please tell us how we can improve.
Sorry to hear that. Please tell us how we can improve.