I need some assistance with managing storage for KumoMTA. Specifically, I’m facing issues with logs and spool files, and I’d like to optimize their usage. Here’s a detailed overview of my situation:
Logs
Current Setup: My KumoMTA setup generates logs that accumulate over more than 2 days.
Problem: The increasing volume of logs is consuming significant disk space.
Goal: I want to retain only the logs from the last 2 days to manage storage effectively.
Spool Files
Current Setup: The spool directory where emails and related data are stored is growing in size and using up a lot of inodes.
Problem: This excessive growth is leading to high storage usage and potential inode exhaustion.
Goal: I need a strategy to manage and clean up spool files to optimize storage and inode usage.
Could anyone provide guidance or best practices for managing these aspects in KumoMTA? Specifically:
For Logs: How can I set up KumoMTA or system configurations to ensure that only the logs from the last 2 days are kept? What’s the best way to automate the cleanup of older logs?
For Spool Files: What are effective methods to manage the spool directory to prevent excessive growth? How can I automate the removal of old or unnecessary spool files?
Any advice or tips would be greatly appreciated. Thank you in advance for your help!
About the logs:
The logfiles are already compressed so they should be already as small as they possibly can be. The logs are not used by KumoMTA in any way, so feel free to create a way to remove them after two days:, something like this:
find /var/log/kumomta -type f -mtime +2 -delete
About the Spool size:
Not sure on this, unless you do massive amounts of mail and they are not delivered then you should run into inode or disk space issues. Other then that it is also possible that webhook events are not delivered and filling up your spool dir. For example if you feed the logs into logstash or Kafka or any other queue for central logging and the receiving side is not accepting it then you should see undelivered webhook entries using:
kcli queue-summary
Then especially look at webook related entries and if there are any numbers on the T column.
To help you more on that I guess we would need a bit more background on the volumes and when you see the inode use being high.
Thank you for your input. I’ve reviewed the kcli queue-summary output and noticed the following:
webhook.log_hook 1167096
SITE SOURCE PROTO D T C Q
http://127.0.0.1:8008.tsa.kumomta unspecified lua:make.http://127.0.0.1:8008.tsa.kumomta 3912 0 0 0
webhook.log_hook unspecified lua:make.webhook.log_hook 0 0 0 1024
Observations:
http://127.0.0.1:8008.tsa.kumomta: Shows 3912 delivery attempts with no issues and no items in the queue.
webhook.log_hook: Shows 1024 items in the queue, indicating a backlog.
Clarification Needed:
Webhook Configuration: I’m not using any webhooks in my setup, so I’m unclear why webhook.log_hook is appearing with a queue size of 1024 items. Could this be a default configuration or an artifact from KumoMTA’s internal processes?
Handling Backlog: Since I don’t use webhooks, how should I address the backlog shown in the webhook.log_hook queue? Is there a specific configuration or cleanup process I should follow?
Any insights or guidance on why these entries are appearing and how to resolve the backlog would be very helpful.
I guess here we come at the configuration file part check if there is any webhook defined in the init.lua or maybe if you used the examaple using the webhook policy-helper:
local log_hooks = require ‘policy-extras.log_hooks’
It seems that webhooks are indeed configured in my setup, even though I don’t actively use them. The log_hooks:new_json section appears to be defining a webhook for logs at http://10.0.0.1:4242/log.
check if there is any webhook defined in the init.lua or maybe if you used the examaple using the webhook policy-helper: