Throttle_insert_ready_queue seems to be throttling webhook calls

KumoMTA Traffic Shaping
1

It looks like throttle_insert_ready_queue is throttling webhooks as well as actual messages.

kumo.on('throttle_insert_ready_queue', function(msg)
  local tenant = msg:get_meta('tenant')
  local throttle = aha.per_tenant_throttle(tenant)
  if throttle then
    throttle:delay_message_if_throttled(msg)
  end
end)


-- in `aha` module
mod.per_tenant_throttle = function (tenant_id)
  -- cached_tenant_throttle returns nil if tenant does not exist. 
  local rate = mod.cached_tenant_throttle(tenant_id)
  if not rate then
    return nil
  end

  return kumo.make_throttle(
    string.format('tenant-send-limit-%s', tenant_id),
    rate
  )
end

shaping.toml:

["webhook.log_hook"]
mx_rollup = false
connection_limit = 100000
max_deliveries_per_connection = 10000
max_message_rate = "10000000/s"
max_ready = 100000
max_connection_rate = "1000000/s"

When I add the throttle_insert_ready_queue handler to init.lua, the number of queued messages for webhook.log_hook in the output of kcli provider-summary starts going up rapidly, and as soon as comment that code block, it quickly goes back down to 0.

2

log hooks copy the meta field from the log record into the meta of the log event message (I’m not sure if this is necessarily the best idea, it’s just what I can see in the code right now), so if you are capturing tenant in your logs meta, then the resulting webhook msg will have tenant set the same way, so it will match your logic. I’d suggest that you check msg:recipient().domain == ‘webhook.log_hook’ and skip throttling in your throttle_insert_ready_queue event callback implementation