How to automate your tasks with Kuber­netes CronJob

Kuber­netes actively monitors the im­ple­ment­a­tion of CronJobs and executes user-defined actions if errors occur. Con­fig­ur­ing Kuber­netes CronJobs is straight­for­ward with YAML files.

What is a CronJob?

A CronJob is a method for auto­mat­ing tasks, a bit like an alarm clock that rings at a certain time. You can also configure a CronJob so that it auto­mat­ic­ally executes defined tasks at set times. A CronJob can automate various types of tasks, such as updating databases, backing up files, executing scripts or sending emails at regular intervals. By in­teg­rat­ing CronJobs into Kuber­netes, tasks can be monitored and managed in isolation in con­tain­ers.

How to set up CronJobs in Kuber­netes

Creating a CronJob is very similar to creating a regular job in Kuber­netes. You have to create a YAML manifest file to do this. This is a struc­tured de­scrip­tion that contains all the relevant details for the CronJob. You define important para­met­ers in this file, such as the schedule for executing the job, the con­tain­ers and the exact commands to be executed in these con­tain­ers.

Creating a YAML file

Open a text editor of your choice to create the YAML con­fig­ur­a­tion file for the CronJob. Make sure to define the schedule of the CronJob in Cron format. Add the job defin­i­tion, including the task to be executed. Then, save the file with the extension .yaml.

apiVersion: batch/v1 
kind: CronJob 
metadata: 
    name: new_cronjob 
spec: 
    schedule: "0    **     ** " 
    jobTemplate: 
        spec: 
            template: 
                spec: 
                    containers: 
                    - name: container 
                        image: image:latest 
                        command:    
                     - /bin/sh

Add resource re­quire­ments and other settings to the Pod spe­cific­a­tion if necessary:

spec: 
    containers: 
    - name: container 
        resources: 
            requests: 
                memory: "64Mi" 
                cpu: "250m" 
            limits: 
                memory: "128Mi" 
                cpu: "500m"

Ac­tiv­at­ing CronJob

Enter the following command in a terminal to create the CronJob in the Kuber­netes cluster:

kubectl apply -f [filename].yaml

Mon­it­or­ing CronJob

If you execute the following command, you will receive a list of the existing Kuber­netes CronJobs and Watch mode will ensure that the output is auto­mat­ic­ally updated when the status of CronJobs changes.

kubectl get cronjob --watch

CronJob schedule syntax

The Schedule syntax for Kuber­netes CronJobs is based on the classic cron format with five fields in the following order: minute, hour, day of the month, month and day of the week. Here is a brief overview:

• Minute (0–59)
• Hour (0–23)
• Day of the month (1–31)
• Month (1–12 or Jan–Dec)
• Day of the week (0–7 or Sun–Sat)

Special char­ac­ters:

• *: Each valid value for the field
• ,: Spe­cific­a­tion of multiple values
• /: Spe­cific­a­tion of steps

Examples:

0 ** **: Every hour 15 2 ** *: Daily at 2:15 0 0 1 **: On the first day of each month at midnight 0 0 *3*: Daily at midnight in March 0 2 ** 1: Every Monday at 2 a.m.

CronJob con­cur­rency policy

The con­cur­rency policy can be specified in the con­fig­ur­a­tion of a CronJob and affects how parallel jobs within the same Kuber­netes CronJob are handled.

• Allow (default): If the con­cur­rency policy is set to __Allow__, a new job is started even if the previous job has not yet been completed. This means that several instances of the same job can run sim­ul­tan­eously.
• Forbid: With this setting, a new job is not started if a previous job has not yet been completed. This ensures that only one instance of the job is executed at any given time.
• Replace: Un­fin­ished jobs are cancelled so that new jobs can continue. No instances of the same job may run at the same time.

Set a deadline for the execution run

The startingDeadlineSeconds field in a Kuber­netes CronJob specifies the maximum number of seconds after the scheduled execution time that a job may be started. If the job does not run within this time limit, it is con­sidered failed.

apiVersion: batch/v1 
kind: CronJob 
metadata: 
    name: new_cronjob 
spec: 
    schedule: "0    **     ** " 
    startingDeadlineSeconds: 300    
    jobTemplate: 
        spec: 
            template: 
                spec: 
                    containers: 
                    - name: container 
                        image: image:latest 
                        command: 
                     - /bin/sh

In this example, the job defined by the CronJob must be started within 300 seconds (5 minutes) of the scheduled time, otherwise it will be deemed as failed.

Limit job history

In Kuber­netes CronJobs, the settings spec.successfulJobsHistoryLimit and spec.failedJobsHistoryLimit allow you to limit the number of jobs kept in the history of the CronJob. spec.successfulJobsHistoryLimit is an optional field that specifies how many suc­cess­fully completed jobs should be saved in the history. This is useful to control resource util­isa­tion and ensure that the history is not cluttered with an excessive number of suc­cess­fully completed jobs. Similarly, spec.failedJobsHistoryLimit allows control over the number of failed jobs.

apiVersion: batch/v1beta1 
kind: CronJob 
metadata: 
    name: new_cronjob 
spec: 
    schedule: "0    **     ** " 
    successfulJobsHistoryLimit: 3    # Keep only the last 3 successfully completed jobs in history. 
    failedJobsHistoryLimit: 1                    # Keep only the last failed job in history. 
    jobTemplate: 
        spec: 
            template: 
                spec: 
                    containers: 
                    - name: container 
                        image: image:latest 
                        command: 
                     - /bin/sh

Here, we specify keeping only the last three suc­cess­fully completed jobs and the last failed job in the course of the Kuber­netes CronJob new_cronjob.

Kuber­netes Cronjob errors and troubleshoot­ing

Various errors can occur in Kuber­netes CronJobs, so you should know some effective troubleshoot­ing tech­niques. Here are a few common error sources and troubleshoot­ing tips:

Job will not start

If a CronJob does not start in Kuber­netes, this can have various causes including an un­avail­able or incorrect container image and missing au­thor­isa­tions for the pod’s service account. To diagnose the problem, examine the container logs with kubectl logs <pod-name>, the events of the CronJob with kubectl describe cronjob <cronjob-name>, and the events of the pod with kubectl describe pod <pod-name>. Make sure that the service account has the necessary au­thor­isa­tions by checking the role and role binding.

Incorrect CronJob con­fig­ur­a­tion

An incorrect CronJob con­fig­ur­a­tion may be due to syntax errors in the YAML file or invalid settings for Schedule and Re­start­Policy. To identify these problems, check the YAML file for correct syntax and settings. You can use kubectl describe cronjob <cronjob-name> to obtain detailed in­form­a­tion about the con­fig­ur­a­tion and identify any errors or in­con­sist­en­cies.

Resource limits exceeded

Exceeding resource limits can cause CronJobs to not run properly. To resolve the problem, check the resource limits in the CronJob and the as­so­ci­ated pods with kubectl describe cronjob <cronjob-name> and kubectl describe pod <pod-name>. In addition, mon­it­or­ing cluster resource util­isa­tion with kubectl top nodes and kubectl top pods can provide valuable insights. If necessary, adjust the resource limits in the YAML file.

To get started with cluster man­age­ment in Kuber­netes, we recommend the Kuber­netes tutorial from our Digital Guide.