SMTP rejection with email in spool on lua error

get-help
1

I’m running Kumo version 2024.09.02.062344.c5476b89 and I observed an odd behavior.
Due to an error in queue management lua code - queue config could not be created (threw a lua error).

What is odd is that the feeder got temp rejection (421 4.3.0 technical difficulties while the message actually hit the spool. Problem is - the application will obviously retry on a temp error, leading to multiple copies of the same email being spooled and (when lua error fixed) - sent. Thankfully this happened in dev/test environment.

Any advice on how to handle this since from what I understand technical difficulties is generic response and there is not much to differentiate from application side

3

Please start by reading and following the links provided by the bot.

4

My appologies. Here’s a gist with policy file and an SMTP trace and kumo log
https://gist.github.com/GeekOfAges/2a08d2610055b1814c5376b99d8b7807

5

I added an error statement where the issue occurred. Ofc the client retried a few times. Removed error and got 6 emails since client tried 6 times

image.png
image.png536×553 33.6 KB

6

Added logs from tailer as well to the gist. What I had was syslog

7

It actually looks like the problem may be in the queue.toml helper. The line 36 thing may be a red herring

8

I think the issue here is that the 421 came back during data but we still spooled the message

9

we should do one or the other

10

@free-spirited-yorksh That’s exactly it. SMTP transaction was rejected, but mail landed in spool

11

You can replicate by simply adding error('Example error location') as first thing in your get_queue_config handler

12

Ah

13

Any advice here? I was just about to push kumo into service. If it’s only the get_queue_config that’s affected - can test the code up-the-wazoo and say all good since it’s basically “select egress pool with default”, other queue configs are static

14

FWIW, no one has hit this in production so far

15

the issue is that the QueueManager::insert doesn’t try to roll back and remove messages from the spool if the queue config and/or egress path config hooks propagate an error

16

You can use kumod --validate to verify your configs (especially the helpers) before you push them into production, but if you are doing custom lua for your hooks, you will need to find a way to satisfy yourself that the configs are good

17

I’ll take a look at unwinding the spooled message in this situation, but I don’t consider this to be a blocking issue

18

Since on the other end of the feeder interface is a queue-backed email forming app, would running with deferred_spool mitigate this since forming app will retry if it does not get a 2xx back?

19

hmm, maybe

20

We’ll test our lua code as much as possible, but is the get_queue_config hook only one affected or is there anything more that needs the deepest depths of scrutiny while this issue is addressed?

21

this should be resolved in Improve handling of errors in get_queue_config · KumoCorp/kumomta@e997cb4 · GitHub