I’m having a bit of trouble with deleting logs on my KumoMTA server and was hoping for some advice. I used this command to clear out old logs:
find /var/log/kumomta/ -type f -name '2024*' ! -newermt '2024-08-23' -exec rm {} \;
After running it, I started getting a bunch of errors in the service status. Here are a few examples:
logger kumod::logging: failed to log: expected user@domain
logger kumod::logging: failed to log: expected user@domain
readyq-1 kumod::ready_queue: error requeuing message: failed to load 0c28061a61d511efa9311225eb9f0d8d from "/var/spool/kumomta/meta/data/0c/28/06/1a/61d511efa9311225eb9f0d8d": No such file or directory
readyq-0 kumod::ready_queue: error requeuing message: failed to load 1028923261d511efa9d11225eb9f0d8d from "/var/spool/kumomta/meta/data/10/28/92/32/61d511efa9d11225eb9f0d8d": No such file or directory
It looks like the log deletion might have caused issues with logging and message processing. I’m guessing I might have missed something important in the process.
Can anyone share the best way to safely delete logs in KumoMTA without causing these kinds of errors? Is there a specific method or command that’s recommended to avoid disrupting the system?
logger kumod::logging: failed to log: open log file “/var/log/kumomta/20240824-080257”: No space left on device (os error 28) .
This error appears along with the above ones. How can I deal with on a regular basis because I cannot keep adding more space to it.
I believe the problem is in the last line of the error.
“No space left on device.”
You may need to stop and start the kumod process to release the space.
I’m seeing that my spool and logs are taking up a lot of space, and I want to manage this better moving forward. A few things I could use some advice on:
Safe Deletion: What’s the safest way to delete spool files and logs without causing issues like I just had? Is there a specific procedure I should be following?
Limiting Growth: How can I prevent the spool and log directories from growing too large over time? Should I be setting up automated log rotation or are there other best practices I should follow?
Monitoring Disk Usage: What’s the best way to keep an eye on disk usage to avoid running into this problem again? Are there any tools or configurations in KumoMTA that could help me stay on top of this?
For log file deletion, consider configuring max_segment_duration - KumoMTA Docs and max_file_size - KumoMTA Docs so that you have an upper bound on the time the system will hold open a given log file segment. You can then process/delete log files according to whatever schedule and retention policy makes sense for you.
For log file growth: kumomta doesn’t use the log files at all; they are for you to consume and discard as you see fit, so it is up to you to decide on a retention and processing policy for them. The directories for this are chosen by you in your policy when you configure_local_logs.
If you don’t have anything that processes the log files, then you may wish to disable using the logs and instead configuring webhooks to push the log information to external infrastructure: Publishing Log Events Via Webhooks - KumoMTA Docs