Progressive delivery allows you to gradually rollout new versions of your application to an environment using canaries and gradually giving traffic to the new version until you are happy to fully rollover to the new version.
Our recommendation for using progressive delivery with Jenkins X is to use:
- flagger as the progressive delivery controller
- istio as the service mesh to provider advanced load balancing capabilities across internal or external networking
Configuring Progressive Delivery
Please follow the usual getting started guide for boot and helm 3
Add the istio and flagger apps
releases: - chart: jx-labs/istio - chart: flagger/flagger - chart: flagger/grafana - chart: jx-labs/flagger-metrics - chart: stable/kuberhealthy ...
So that you remove
stable/nginx-ingress from your
helmfile.yaml file and ensure the above apps are added at the top of the file.
Also for now I’m afraid you will have to remove the
jenkins-x/jxui chart as its currently not working yet with istio ingress - but we can hopefully get that working soon.
Enable istio based ingress
To avoid having 2
LoadBalancer services for both
nginx (which costs more money) its easier to switch to pure istio for both internal and external load balancing. This also results in a smaller footprint.
To do that ensure that
kind: istio is added to the
jx-requirements.yml file in the top level
ingress: section like this:
ingress: domain: "" kind: istio ...
Now your development git repository should be setup and be ready. Now:
- make sure you have setup any secrets you need to boot
When it is all complete you should see istio, flagger, grafana pods running in the
istio-system namespace something like this:
$ kubectl get pod -n istio-system NAME READY STATUS RESTARTS AGE flagger-66dc49cd-g6ptp 1/1 Running 0 32h grafana-7d7d7476f6-ff6bm 1/1 Running 0 32h istio-ingressgateway-598796f4d9-sq8b7 1/1 Running 0 32h istiod-7d9c7bdd6-vjp9j 1/1 Running 0 32h kuberhealthy-f54f7f7df-b5gbf 1/1 Running 2 32h kuberhealthy-f54f7f7df-j6qwt 1/1 Running 0 32h prometheus-b47d8c58c-n974m 2/2 Running 0 32h
From 1.5 onwards istio is pretty small; just 2 pods. Note that those
kuberhealthy pods are optional and just help with reporting.
Enable istio in staging/production
To enable istio in staging:
kubectl label namespace jx-staging istio-injection=enabled
To enable istio in production:
kubectl label namespace jx-production istio-injection=enabled
Defaulting to use Canary
jx edit deploy --team --canary --hpa
This will enable all new quickstarts and imported projects to use canary rollouts and use horizontal pod autoscaling in all environments.
You can switch the defaults back again at any time or configure any app to change its defaults by running
jx edit deploy inside a git clone of an application.
Enabling/Disabling Canary/HPA in an Environment
If you want to enable/disable canary or horizontal pod autoscaling for a specific app in an environment then you can follow the app customisation approach.
Assuming your app is called
myapp then in the git repository for the environment (e.g.
Staging) you can add/edit a file called
apps/mychart/values.yaml to look like this:
canary: enabled: true hpa: enabled: true
you can enable/disable those 2 flags for canary releases and horizontal pod autoscaler at any point in any environment.
Using Progressive Delivery
Once you have followed the above steps create a quickstart application in the usual way.
As you merge changes to the master branch of your application Jenkins X will create a new release and promote it to the staging environment.
However if Canary deployment is enabled your new version will gradually be rolled out progressively:
- the defaults are that 20% of the traffic will go to the new version
- flagger will keep monitoring the metrics used in the Canary resource to determine if the canary is good
- after the configured time period is over the traffic will be increased to 40% then 60%
- eventually if things look good the new version will fully rollout to 100% traffic
- if anything goes bad during the rollout time period the old version is restored
There is an excellent video showing this in action.
Was this page helpful?
Glad to hear it! Please tell us how we can improve.
Sorry to hear that. Please tell us how we can improve.