How to debug your Tekton pipelines
TaskRunbreakpoint functionality is no longer supported since Tekton 0.29.0 upgrades in
3.2.298. For more info see Kubernetes 1.22 - Breaking change!.
Tekton recently introduced a debug feature when you create
TaskRun resources so that steps can be paused at a breakpoint until told to move forwards so that you can diagnose why pipeline steps fail.
The latest Tekton release only supports breakpoints on
TaskRun resources but there is a Pull Request #4145 to add support also to debugging
PipelineRun resources as well. If you are reading this please add your thumbs up emoji feedback to the PR #4145
We’ve switched Jenkins X to use a preview image of Tekton with PR #4145 included so that Jenkins X developers can easily debug their pipelines (which typically are
How to debug Tekton Pipelines
Here is a demo which shows how to debug pipelines:
If you intend to use the
jx in the below examples make sure you upgrade the CLI too
Enable a breakpoint
To enable a breakpoint you can:
- use the Lens UI as shown in the above video by:
- right click on a
Breakpoint -> Add
- right click on a
- you can use the jx pipeline debug command then select the pipeline to add/remove a breakpoint.
You can view breakpoints in the Lens UI in the
Breakpoints tab or via:
kubectl get lighthousebreakpoints # you can use the short name: kubectl get lhbp
Using a breakpoint
Once you have set a breakpoint defined for a particular Pipeline you need to trigger the pipeline. e.g. perform a git commit on the git branch to trigger a new pipeline to execute.
The pipeline will execute as normal; you’ll be able to view it execute via:
- Lens UI
- run jx pipeline grid to watch pipelines run and select the one you wish to view the log
- run jx pipeline log to watch the log of a specific pipeline
Opening a shell
Once your breakpoint is reached the pipeline pod will pause, waiting to continue.
At this point you can then open a shell inside the container.
The easiest way to do this is via the Lens UI, click on the Pipeline action menu then
latest step and a shell will open.
Otherwise you can use:
kubectl exec -it -c $name-of-container $name-of-pod (sh | bash | ash)
Continuing after the breakpoint
If you wish to continue the execution of a pipeline there are multiple scripts you can run inside the shell you can run inside the shell in the pipeline to tell the pipeline to continue:
||Mark the step as completed with success by writing to
||Mark the step as completed with failure by writing to
There are a few ways to delete breakpoints.
You can run jx pipeline debug and toggle off any existing breakpoints.
You can use the
Breakpoints tab in Lens UI then click the breakpoints action menu then
Or find the one you want via:
kubectl get lhbp kubectl delete lhbp whatever-the-name-is
So there you have it; nice and easy debugging of pipelines so you can diagnose why pipelines fail and try incrementally fix things up from inside the pipeline pods! Pretty cool eh!
Let us know via slack or the issue tracker if you can think of any ways we can make this even easier to use! Also check out the Tekton enhancement proposal 42 that covers this capability in the underlying tekton controller and pods.
Finally please add your thumbs up emoji to the tekton PR #4145 :)