OOB log messages don't contain the meta and headers

get-help
1

The JSON log posted to my webhook endpoint for OOB messages include the meta and header values in the log entries for “Reception” and “Delivery” (for custom-lua delivery queue) of the message, but the header and meta values in the “OOB” log entry of the same message all have empty values. This is preventing my application from marking the original message (indicated by the “References” header) as bounced.

I’ve attached sample logs to this issue - not sure how to debug this further.
oob-logs.json (4.43 KB)

2

Hey there @original-baboon, thanks for posting. Please read the “Troubleshooting” and “How to Ask for Help” buttons below. If you would like a 1:1 support session from the KumoMTA team, details are at the “Book a Support Session” button below.

3

Do you have a copy of the OOB bounce message?

4

Sent a new one to capture the message data
msg.eml (5.88 KB)
oob-logs.json (4.4 KB)

5

So the meta and header fields are missing because we are not sending the message, we are receiving a bounce. That bounce may or may not have the full message included, and often does not. We can’t control what the remote host sends to us.

Many ESPs handle this using VERP, so instead of using a static MAIL FROM on their sends, they use a dynamic one, with important information encoded in it for retrieval. This is because the one thing a remote MTA can redact or change is where they send the OOB to, as defined by your MAIL FROM.

In fact, the format of your references header is formatted as an email address, so using that address as your MAIL FROM would be an easy way to ensure that information is preserved, because the OOB recipient would be that address, and you apparently know what you need to from the format of that address.

6

Some orgs embed the relevant info directly into the VERP address, such as recipient ID and campaign ID, others embed an identifier which they can later use to look up the relevant information.

7

Thanks for the explanation - I’m gonna use msg:set_sender() to set the MAIL FROM, but in any case I’m actually extracting the domain_id and customer_id meta from the msg:recipient().domain. So they should have valid values right now, no?

8

Your domain is the recipient in the case of an OOB message.

9

It’s best to think of messages as being essentially stateless when thinking about OOBs. We queue a message, deliver it. Since the MBP accepted it we offload the message, log it, and forget about it. When the OOB comes in that’s an inbound message addressed to your bounce domain.

10

does this happen for OOB only or all all inbound messages? would I get these meta and header values if someone actually replied to one of the emails sent by KumoMTA?

11

No, because that reply has its own headers. Headers of the original message are pretty much never preserved in a reply, and while some MBPs will preserve them for an FBL or OOB, there’s no guarantee, which again is why we look at the MAIL FROM address for that info using VERP (note that replies go to the From header address, which often is not the same as the MAIL FROM, so catching data in a reply is especially troublesome, which is why clicks and opens are usually the way to go for tracking engagement, but you could also use tracking in the From or Reply-To header address to capture things based on incoming address.

12

I think I’m not explaining myself clearly, I’m not looking to get the original email headers. Here’s the relevant part of the code extracting data from the message in smtp_server_message_received:

  local tenant = cached_tenant_id(msg:from_header().domain)
  local domain = cached_domain_id(msg:from_header().domain)
  if tenant and domain then
    -- this message is from one of our tenants.
    msg:set_meta('source', 'smtp')
    msg:set_meta('direction', 'outbound')
    msg:set_meta('tenant', tenant)
    msg:set_meta('customer_id', tenant)
    msg:set_meta('domain_id', domain)
  else
    -- This message is not from one of our tenants, it's either a reply or OOB.
    tenant = cached_tenant_id(msg:recipient().domain)
    domain = cached_domain_id(msg:recipient().domain)
    if tenant and domain then
      msg:set_meta('source', 'smtp')
      msg:set_meta('direction', 'inbound')
      msg:set_meta('tenant', tenant)
      msg:set_meta('customer_id', tenant)
      msg:set_meta('domain_id', domain)
      msg:set_meta('campaign', 'direction-inbound')

      local response = save_message(msg)
      if response:status_is_success() then
        queue_helper:apply(msg)
        return
      else
        kumo.reject(451, "Internal server error, please try again later.")
      end
    else
      kumo.reject(
        556,
        string.format("Invalid account. Tenant or Domain not found for '%s'.", msg:from_header().email)
      )
    end
  end
13

Here in the else section I’m trying to extract data from the reply message (whether it’s OOB or an actual reply by a user), and I need to at least set the tenant and domain_id in order to be able to accept the message - this is working. So I know that it’s actually finding tenant and domain here. But it’s not including them in the log message.

14

How is it what are your cached_ functions doing?

15 
function get_tenant_id(domain)
  local ok, tenant_id = pcall(kumo.secrets.load, {
    vault_mount = "secret",
    vault_path = "tenants/" .. domain,
  })
  if ok then
    return tenant_id
  else
    return false
  end
end

function get_domain_id(domain)
  local ok, domain_id = pcall(kumo.secrets.load, {
    vault_mount = "secret",
    vault_path = "domains/" .. domain,
  })
  if ok then
    return domain_id
  else
    return false
  end
end

local cached_tenant_id = kumo.memoize(get_tenant_id, {
  name = 'tenants',
  ttl = '5 minutes',
  capacity = 1000,
})

local cached_domain_id = kumo.memoize(get_domain_id, {
  name = 'domains',
  ttl = '5 minutes',
  capacity = 1000,
})
16

Aah, ok. There’s a good chance some or all of that is getting bypassed as soon as the message is identified as an OOB. @free-spirited-yorksh can you confirm?

17

confirmed as a bug; should be fixed by fix: OOB didn't respect headers and meta config from logger · KumoCorp/kumomta@0d61040 · GitHub

18

@yearning-hyena Thanks for the explanation on VERP by the way, just implemented it in my system :slightly_smiling_face:

19

Just updated to the latest dev version @free-spirited-yorksh, now I’m not getting the OOB log at all.
oob-logs.json (3.12 KB)

20

Searching in the output of tailer for the message id ccfd6176da6e11eebe66960002cafe7c:

{"type":"Reception","id":"ccfd6176da6e11eebe66960002cafe7c","sender":"","recipient":"b31391d4da6e11eebe5c960002cafe7c@psrp.kumo.taskulu.com","queue":"direction-inbound:4cdd7bdd-294e-4762-892f-83d40abf5a87@psrp.kumo.taskulu.com","site":"","size":6409,"response":{"code":250,"enhanced_code":null,"content":"","command":null},"peer_address":{"name":"traffic02.clientspanel.com","addr":"162.55.128.206"},"timestamp":1709587991,"created":1709587991,"num_attempts":0,"bounce_classification":"Uncategorized","egress_pool":null,"egress_source":null,"feedback_report":null,"meta":{"tenant":"4cdd7bdd-294e-4762-892f-83d40abf5a87"},"headers":{"Subject":"Mail delivery failed: returning message to sender"},"delivery_protocol":null,"reception_protocol":"ESMTP","nodeid":"2f6489ba-9d9d-47d5-83bb-8492ac3fdd93"}
{"type":"Delivery","id":"ccfd6176da6e11eebe66960002cafe7c","sender":"","recipient":"b31391d4da6e11eebe5c960002cafe7c@psrp.kumo.taskulu.com","queue":"direction-inbound:4cdd7bdd-294e-4762-892f-83d40abf5a87@psrp.kumo.taskulu.com","site":"unspecified->psrp.kumo.taskulu.com@lua:make.inbound","size":6409,"response":{"code":200,"enhanced_code":null,"content":"200 OK: OK","command":null},"peer_address":{"name":"Lua via make.inbound","addr":"0.0.0.0"},"timestamp":1709587991,"created":1709587991,"num_attempts":0,"bounce_classification":"Uncategorized","egress_pool":"unspecified","egress_source":"unspecified","feedback_report":null,"meta":{"tenant":"4cdd7bdd-294e-4762-892f-83d40abf5a87"},"headers":{"Subject":"Mail delivery failed: returning message to sender"},"delivery_protocol":"Lua","reception_protocol":"ESMTP","nodeid":"2f6489ba-9d9d-47d5-83bb-8492ac3fdd93"}