Build Packs

We use draft style build packs for different languages, runtimes and build tools to add the necessary configuration files to projects as we import them or create them so that we can build and deploy them in kubernetes.

The build packs are used to default the following files if they do not already exist in the project being created/imported:

• Dockerfile to turn the code into an immutable docker image for running on kubernetes
• Jenkinsfile to define the declarative Jenkins pipeline to define the CI/CD steps for the application
• helm chart in the charts folder to generate the kubernetes resources to run the application on kubernetes
• a preview chart in the charts/preview folder to define any dependencies for deploying a preview environment on a Pull Request

The default build packs are at https://github.com/jenkins-x-buildpacks/jenkins-x-kubernetes with a folder for each language or build tool.

The jx command line clones the build packs to your .~/.jx/draft/packs/ folder and updates them via a git pull each time you try create or import a project.

Pipeline extension model

We have refactored our build packs so that they are more modular and easier to compose and reuse across workloads.

For example the jenkins-x-kubernetes build pack inherits from the jenkins-x-classic build pack, reusing the CI and release pipelines but then adding the kubernetes specific workloads (e.g. building docker images, creating helm charts, Preview Environments and Promotion via GitOps)

To do this we’ve introduced a simple new YAML file format for defining pipelines.

Pipelines

Each Pipeline YAML file has a number of separate logical pipelines:

• release for processing merges to the master branch which typically creates a new version and release then triggers promotion
• pullRequest for processing Pull Requests
• feature for processing merges to a feature branch. Though note that the accelerate book recommends against long term feature branches. Instead consider using trunk based development which is a practice of high performing teams.

Life Cycles

Then each pipeline has a number of distinct life cycle phases - rather like maven has clean, compile, compile-test, package etc.

The life cycle phases in Jenkins X Pipeline YAML are:

• setup
• preBuild
• build
• postBuild
• promote

Extending

A Pipeline YAML can extend another YAML file. You can reference a base pipeline YAML via:

• using file to reference a relative file path in the same build pack like this example using file
• using import to reference a YAML file which is imported like this example using import which then refers to a named imported module via git

Overriding steps

Rather like classes in languages like Java you can override steps in a Pipeline YAML from a base Pipeline YAML. This lets you reuse the steps in a base pipeline’s life cycle then add your own additional steps.

By default any steps you define are added after the base pipeline YAML steps like in this example.

You can add steps before the base pipeline steps using the preSteps: property like this example

If you want to completely replace all the steps from a base pipeline for a particular life cycle you can use replace: true like in this example

Example Pipeline

For example for maven libraries we use this pipeline.yaml file which:

• extends the common pipeline that sets up git and defines common post build steps
• configures the agent in terms of pod template and container name
• defines the steps for the pull request pipeline build steps
• defines the release pipeline set version steps and build steps

Then the maven kubernetes pipeline.yaml then extends from the classic pipeline to add the kubernetes steps

Creating new build packs

We love contributions so please consider adding new build packs and pod templates.

Here are instructions on how to create a new build pack - please if anything is not clear come join the community and just ask we are happy to help!

The best place to start with is a quickstart application. A sample project that you can use as a test. So create/find a suitable example project and then import it.

Then manually add a Dockerfile and Jenkinsfile if one is not already added for you. You could start with files from the current build pack folders - using the most similar language/framework to yours.

If your build pack is using build tools which are not yet available in one of the existing pod templates then you will need to submit a new pod template probably using a new build container image too.

Once you have a pod template to use, say, jenkins-foo then refer to it in your Jenkinsfile:

// my declarative Jenkinsfile

pipeline {
    agent {
      label "jenkins-foo"
    }
    environment {
      ...
    }
    stages {
      stage('CI Build and push snapshot') {
        steps {
          container('foo') {
            sh "foo deploy"
          }

Once your Jenkinsfile is capable of doing CI/CD for your language/runtime on your sample project then we should be able to take the Dockerfile, Jenkinsfile and charts folder and copy them into a folder in your fork of the jenkins-x/draft-packs repository.

You can try that out locally by adding these files to your local clone of the build packs repository at ~/.jx/draft/packs/github.com/jenkins-x/draft-packs/packs

e.g.

export PACK="foo"
mkdir ~/.jx/draft/packs/github.com/jenkins-x/draft-packs/packs/$PACK
cp Dockerfile Jenkinsfile  ~/.jx/draft/packs/github.com/jenkins-x/draft-packs/packs/$PACK

# the charts will be in some folder charts/somefoo
cp -r charts/somefoo ~/.jx/draft/packs/github.com/jenkins-x/draft-packs/packs/$PACK/charts

Once your build pack is in a folder at ~/.jx/draft/packs/github.com/jenkins-x/draft-packs/packs/ then it should be usable by the jx import code which uses programming language detection to find the most suitable build pack to use when importing a project. If your build pack requires custom logic to detect it then let us know and we can help patch jx import to work better for your build pack. For example, we have some custom logic for handling Maven and Gradle better.

Using custom build packs

Sometimes you may want to specify a custom buildpack to use when setting up your applications. There are two distinct ways to create/setup clusters with Jenkins-x and the approach you take will determine how you specify a custom buildpack.

If a cluster was setup using the jx install method then we have this command that allows users to edit which buildpack to use, to build applications.

However if a cluster is being setup with jx boot, then users have to edit their jx-requirements.yml file. You have to add the following fields to the jx-requirements.yml file before running boot pipeline again. All the fields have to be added together otherwise jx won’t be able to figure out where to get the buildpack from.

buildPacks:
  buildPackLibrary:
    name: "Test name"
    gitURL: "github.com/jx-user/test-repo"
    gitRef: "master"

If you need any more help join the community

Feedback

Was this page helpful?

Glad to hear it! Please tell us how we can improve.

Sorry to hear that. Please tell us how we can improve.