K3s
NOTE
Ensure you are logged into GitHub else you will get a 404 error when clicking the links below
This guide will walk you though how to setup Jenkins X on your laptop using k3s
If you are on Mac OS, you can follow this guide to set up k3s. You do not need to install kalm for the rest of the tutorial.
Prerequisites
K3s
Make sure you have created a cluster using k3s.
If you dont have an existing k3s cluster, you can install one by running:
To verify that k3s has been installed successfully, and configured run:
- You will need to open multiple terminals later, so setting these env variables in the bashrc or zshrc might help you
Optional
- If the above method didn’t work, copy the configurations to the ~/.kube/config instead (if you don’t have any other clusters, that should be fine)
- If still got
permission denied
This value of the node will be used later during installation and configuring of Jenkins X.
Check k3s install guide for more installation options.
Vault
Install vault cli. Refer to the vault docs on how to install vault for your platform.
Internal vault (Preferred)
Helm
Follow the helm documentation to install the helm binary. Once helm is installed, proceed to the next steps.
To install vault inside the newly created k3s cluster, you need to install the vault operator and vault instance chart.
Wait for the vault pods to be in running status, before proceeding to the next steps.
Output should be similar to
External vault
Docker
You need to install docker and manage it as a non root user
Make sure you have vault running in a docker container with kubernetes auth enabled.
To verify if vault started properly use docker logs jx-k3s-vault
.
Next enable kubernetes auth in vault.
Note: If you get the error Error enabling kubernetes auth: Post "https://127.0.0.1:8200/v1/sys/auth/kubernetes": http: server gave HTTP response to HTTPS client
, try the command vault login myroot
.
Github
- Create a git bot user (different from your own personal user) e.g. https://github.com/join and generate a a personal access token, this will be used by Jenkins X to interact with git repositories. e.g. https://github.com/settings/tokens/new?scopes=repo,read:user,read:org,user:email,write:repo_hook,delete_repo,admin:repo_hook,write:packages
- This bot user needs to have write permission to write to any git repository used by Jenkins X. This can be done by adding the bot user to the git organisation level or individual repositories as a collaborator Add the new bot user to your Git Organisation, for now give it Owner permissions, we will reduce this to member permissions soon.
Jenkins-X
- Make sure you have installed jx 3.x binary and put it on your
$PATH
as thejx admin operator
will be used
Jenkins X v3 installation
-
Generate a cluster git repository from the jx3-k3s-vault template, by clicking here
-
Clone the generated repository and cd into the repository folder
-
Set up ingress and webhook
- Get the external IP of the traefik service (loadbalancer)
- Edit the jx-requirements.yaml file by editing the ingress domain:
-
set up Ngrok
-
Refer to these docs to set up ngrok
-
Once this tunnel is open, paste the ngrok url (without http) which is forwarding the traffic to port 8080 in the hook field in the helmfiles/jx/jxboot-helmfile-resources-values.yaml file in the cluster git repository.
-
commit and push the changes.
-
-
Make these changes only when using external vault
- Add the value of the vault url in the
jx-requirements.yaml
file.
The jx-requirements file should look like this for external vault:
- Commit and push your changes:
- Add the value of the vault url in the
-
Set the GIT_USERNAME (bot username) and GIT_TOKEN (bot personal access token) env variable and run:
- Internal vault
- External vault
Note (Only for external vault): The first job will fail as it cannot authenticate against vault. The errors will be of the form
error: failed to populate secrets: failed to create a secret manager for ExternalSecret
. Once the secret-infra namespace has been created, we can configure vault. If you get an error connecting to the cluster, try runningkubectl config view --raw >~/.kube/config
as well as checking the permissions/owner of~/.kube/config
Vault configuration (Only for external vault)
Install jq before running these commands.
Remember to run the following commands in a terminal where you have set the value of VAULT_ADDR
- Create a vault config
- Create a vault role:
- Create a policy attached to vault role:
Once vault is configured, pull the changes commited to the cluster git repository by the bootjob, and push a dummy job
tail the logs of jx-git-operator
pod in the jx-git-operator
namespace.
Kill that job.
Once it’s killed a new boot job will be triggered. This job will create the secrets in vault which will be used by external secrets to create kubernetes secrets.
- To verify the job succeeded, run
jx admin log
- To verfiy the secrets were created, run
kubectl get es -A
andjx secret verify
- If this didn’t work try and repeat the steps but commit your dummy changes through github repository directly other than the
git push origin main
command
Port forwarding for webhooks
In another terminal run the following command to enable webhooks via ngrok
- Once the bootjob has succeeded, you should see:
Next steps
Feedback
Was this page helpful?
Glad to hear it! Please tell us how we can improve.
Sorry to hear that. Please tell us how we can improve.