I had this K3s v1.26.4+k3s1 running on Nvidia box somewhere out in the sticks. Then the device started running low on disk space, largely due to crappy containers running on it. /var/log/pods was taking up space that could be used elsewhere.

Here is how to limit size of /var/log/pods on K3s. Create /etc/rancher/k3s/config.yaml or if existent add the following arguments kubelet arguments:

  - "container-log-max-files=2"
  - "container-log-max-size=2Mi"

This will cause k3s to create 2 log files, each with maximum size of 2M.

Alternatively the arguments can be passed to k3s binary itself.

k3s server --kubelet-arg container-log-max-files=4 --kubelet-arg container-log-max-size=50Mi