How to debug your Tekton pipelines

Tekton recently introduced a debug feature when you create TaskRun resources so that steps can be paused at a breakpoint until told to move forwards so that you can diagnose why pipeline steps fail.

The latest Tekton release only supports breakpoints on TaskRun resources but there is a Pull Request #4145 to add support also to debugging PipelineRun resources as well. If you are reading this please add your thumbs up emoji feedback to the PR #4145

We’ve switched Jenkins X to use a preview image of Tekton with PR #4145 included so that Jenkins X developers can easily debug their pipelines (which typically are PipelineRun resources).

How to debug Tekton Pipelines

Here is a demo which shows how to debug pipelines:

Prerequisites

Make sure your cluster is upgraded to the latest version stream.

If you intend to use the jx in the below examples make sure you upgrade the CLI too

Enable a breakpoint

To enable a breakpoint you can:

• use the Lens UI as shown in the above video by:
  • right click on a Pipeline action menu
  • select Breakpoint -> Add
• you can use the jx pipeline debug command then select the pipeline to add/remove a breakpoint.

Viewing breakpoints

You can view breakpoints in the Lens UI in the Breakpoints tab or via:

kubectl get lighthousebreakpoints

# you can use the short name:
kubectl get lhbp

Using a breakpoint

Once you have set a breakpoint defined for a particular Pipeline you need to trigger the pipeline. e.g. perform a git commit on the git branch to trigger a new pipeline to execute.

The pipeline will execute as normal; you’ll be able to view it execute via:

• Lens UI
• run jx pipeline grid to watch pipelines run and select the one you wish to view the log
• run jx pipeline log to watch the log of a specific pipeline

Opening a shell

Once your breakpoint is reached the pipeline pod will pause, waiting to continue.

At this point you can then open a shell inside the container.

The easiest way to do this is via the Lens UI, click on the Pipeline action menu then Shell -> latest step and a shell will open.

Otherwise you can use:

kubectl exec -it -c $name-of-container $name-of-pod (sh | bash | ash)

Continuing after the breakpoint

If you wish to continue the execution of a pipeline there are multiple scripts you can run inside the shell you can run inside the shell in the pipeline to tell the pipeline to continue:

Removing breakpoints

There are a few ways to delete breakpoints.

You can run jx pipeline debug and toggle off any existing breakpoints.

You can use the Breakpoints tab in Lens UI then click the breakpoints action menu then Remove

Or find the one you want via:

kubectl get lhbp
kubectl delete lhbp whatever-the-name-is

Conclusion

So there you have it; nice and easy debugging of pipelines so you can diagnose why pipelines fail and try incrementally fix things up from inside the pipeline pods! Pretty cool eh!

Let us know via slack or the issue tracker if you can think of any ways we can make this even easier to use! Also check out the Tekton enhancement proposal 42 that covers this capability in the underlying tekton controller and pods.

Finally please add your thumbs up emoji to the tekton PR #4145 :)

• ←Previous
Next→