Peter Lee's Blog

0%

hpa+auto deploy

Posted on In

Horizontal Pod Autoscaler

deployment placement

the container -> resource property must set, otherwise the scaler cant get your container runtime metrics

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
kind: Deployment
apiVersion: apps/v1
metadata:
  name: <deployment name>
  namespace: <namespace>
  labels:
    app: <deployment name>
    k8s-app: <deployment name>
  annotations:
    deployment.kubernetes.io/revision: '21'
spec:
  selector:
    matchLabels:
      k8s-app: <deployment name>
  template:
    metadata:
      name: <deployment name>
      creationTimestamp: null
      labels:
        app: <deployment name>
        k8s-app: <deployment name>
    spec:
      containers:
        - name: <deployment name>
          image: <container-registery/image>
          ports:
            - name: http
              containerPort: 9000
              protocol: TCP
            - name: grpc
              containerPort: 9001
              protocol: TCP
          resources:
            limits:
              cpu: 200m
              # memory: 1024Mi
            requests:
              cpu: 50m
              # memory: 128Mi
          imagePullPolicy: Always
          securityContext:
            privileged: false
      restartPolicy: Always
      terminationGracePeriodSeconds: 30
      dnsPolicy: ClusterFirst
      securityContext: {}
      imagePullSecrets:
        - name: ghcr
      schedulerName: default-scheduler
  strategy:
    type: RollingUpdate
    rollingUpdate:
      maxUnavailable: 25%
      maxSurge: 25%
  revisionHistoryLimit: 10
  progressDeadlineSeconds: 600

hpa placement

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: <hpa-name>
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: <your desired deployment name>
  minReplicas: 1
  maxReplicas: 10
  metrics:
  - type: Resource
    resource:
      name: cpu
      target:
        type: Utilization
        averageUtilization: 50 # unit: percent
2023-12-20T022516

gitlab runner auto rollout to kubernetes cluster

gitlab -> runner -> auto trigger kubectl roll update (/home/ubuntu/.kube)

runner on kubernetes

if we want to let multiple repo run on same gitlab runner, we need to use register group runner (need to create a group)

create agent config file in project (optional)

To create an agent configuration file:

Choose a name for your agent. The agent name follows the DNS label standard from RFC 1123. The name must: - Be unique in the project. - Contain at most 63 characters. - Contain only lowercase alphanumeric characters or -. - Start with an alphanumeric character. - End with an alphanumeric character. - In the repository, in the default branch, create an agent configuration file at the root:

1
.gitlab/agents/<agent-name>/config.yaml

You can leave the file blank for now, and configure it later.

Register the agent with GitLab

1. Select Operate > Kubernetes clusters

2023-12-20T022538

2. Select Connect a cluster (agent)

* If you want to create a configuration with CI/CD defaults, type a name.
* If you already have an agent configuration file, select it from the list.
2023-12-20T022604

3. if you already have config file in your project, selec one. otherwise create a new one

2023-12-20T022614

4. get the access token

2023-12-20T022625

Update your .gitlab-ci.yml file to run kubectl commands

1. install kubernetes agent in your cluster through helm with the provided token

1
2
3
4
5
6
7
8
helm repo add gitlab https://charts.gitlab.io
helm repo update
helm upgrade --install <xxxxx> gitlab/gitlab-agent \
    --namespace gitlab-agent-<xxxxx> \
    --create-namespace \
    --set image.tag=v16.5.0 \
    --set config.token=glagent-YoxxFv-5HxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxUeA \
    --set config.kasAddress=wss://kas.gitlab.com

2. check if gitlab has connected to the cluster agent

2023-12-20T021043

3. run kubectl in .gitlab-ci.yml

1
2
3
4
5
6
7
8
deploy:
  image:
    name: bitnami/kubectl:latest
    entrypoint: ['']
  script:
    - kubectl config get-contexts
    - kubectl config use-context path/to/agent/repository:agent-name
    - kubectl get pods

notice: If you are not sure what your agent’s context is, open a terminal and connect to your cluster. Run kubectl config get-contexts. (in my case, I need to execute the command directly in the gitlab ci triggered job)

2023-12-20T021027
  1. push the change to trigger the gitlab workflow and run your own custom kubectl command in the gitlab ci

reference