If I don’t want to send all webhook logs but based on the tenant, when is the most appropriate time?
configure_log_hook, should_enqueue_log_record?
When using the helper, in log_hooks:new, the constructor = function(domain, tenant, campaign)
I tried to print the logs to see the results, and they were all nil. I thought this would be a good place to handle it.
You can do tht any time. If you capture teh tenant in meta data at injection, you can selectively log based on that.
One possibility is to selectively fire in the webhook code. Alternately, you can set a table of webhooks with different destinations.
If you take a look at the sampel here:
You can selectively fire the connector. EG:
if domain == 'webhook' and tenant == "mytenant" then
if domain == 'webhook' and tenant == "mytenant" then
-- Use the `make.webhook` event to handle delivery
-- of webhook log records
return kumo.make_queue_config {
protocol = {
custom_lua = {
constructor = 'make.webhook',
},
},
}
end
return kumo.make_queue_config {}
end)```Yeah, this I have try . but i still want to use webhook helper . the helper is relly so cool
If you want to fire the webhook selectively, you should probably use the more complex version.
when i use webhook helper , the constructor = function(domain, tenant, campaign) , i can not get tenant and campain . i print it all be nil
Then you can conditionally select it when calling the custom constructor
Yeah, the helper is really meant to fire unconditionally.
It might be possible to selectively use the helper but I’d have to think through that.
Thanks for your thinking . Indeed, the kumomta’s helper is a great creation.
Another question, thank you for your time.
How can I pass the tenant and campaign parameters from kumo.on(‘get_queue_config’, function(domain, tenant, campaign, routing_domain) to the webhook so that I can obtain them in kumo.on(‘make.webhook’, function(domain, tenant, campaign) in the webhook? Or will it be automatically passed? I’ll test it later.
The best way to do that is to include the meta data. Tenant and campaign are automatically saved to metadata but you can add any header or random variable to metadata that you choose.
To log the metadata, in the webhook constructor (either helper or long form) add the list of metadata to the log_parameters list.
If you want to use the data for decision making, you can extract it from the message object with msg:get_meta()
Some further reading:
If you are specifically looking for tenant and campaign, they are automatically passed to get_queue_config()
Thank you, Tom, for your patient responses.
Today, I did a quick test and found that I could retrieve the header information “X-campaign: campaignname|1|2|3|4|” in smtp_server_message_received (assuming I indeed passed it in the header when sending the email via SMTP). Using local test = msg:get_first_named_header_value(‘X-campaign’), I successfully obtained the expected value: campaignname|1|2|3|4.
To maintain the integrity of the campaign, I retrieved the campaignname using Lua and set it in smtp_server_message_received as follows:
msg:set_meta(‘campaign’, campaign)
msg:set_meta(‘test1’, 1)
msg:set_meta(‘test2’, 2)
msg:set_meta(‘test3’, 3)
msg:set_meta(‘test4’, 4)
Unfortunately, I couldn’t retrieve the campaign data in get_queue_config.
print(‘get_queue_config campaign’, campaign) The result was nil.
Similarly, in kumo.on(‘make.kafka’), the values for campaign, test1, test2, test3, and test4 were all nil.
I believe that all the data retrieved in make.kafka should be based on message:get_meta(‘log_record’). The values I passed were not accessible.
Regarding the method you mentioned using import_x_headers, I will try it out tomorrow.
I am new to both Lua and KumoMTA, and I am committed to learning diligently. I hope my questions do not cause you any inconvenience. Once again, thank you, Tom.
I am not sure it is possible in the helper, I’ll have to investigate.
If you do the webhook by long form then you can launch a function that pulls in the message variable similar to how it is done in the webhook sample.
local client = kumo.http.build_client {}
-- The send method is called for each log event
function connection:send(message)
local response = client
:post('http://10.0.0.1:4242/log')
:header('Content-Type', 'application/json')
:body(message:get_data())
:send()
I do not use helper. code like this .
kumo.on('smtp_server_message_received', function(msg)
-- Protect against SMTP Smuggling (https://sec-consult.com/blog/detail/smtp-smuggling-spoofing-e-mails-worldwide/)
local failed = msg:check_fix_conformance(
-- check for and reject messages with these issues:
'NON_CANONICAL_LINE_ENDINGS',
-- fix messages with these issues:
''
)
local campaign = msg:get_first_named_header_value('X-campaign')
if campaign then
local campaignsigle, msg_reception, msg_delivery, msg_bounce,
msg_transient = string.match(campaign or "", "([^|]+)|([01])|([01])|([01])|([01])|?")
if campaignsigle == 'whatiwant' then
msg:set_meta('campaign', campaignsigle)
msg:set_meta('test1', msg_reception)
msg:set_meta('test2', msg_delivery)
msg:set_meta('test3', msg_bounce)
msg:set_meta('test4', msg_transient)
end
end
if failed then
kumo.reject(552, string.format('5.6.0 %s', failed))
end
-- Call the queue helper to set up the queue for the message.
queue_helper:apply(msg)
-- SIGNING MUST COME LAST OR YOU COULD BREAK YOUR DKIM SIGNATURES
dkim_signer(msg)
end)
kumo.on('should_enqueue_log_record', function(msg, hook_name)
local log_record = msg:get_meta 'log_record'
if log_record.queue ~= 'kafka' then
msg:set_meta('queue', 'kafka')
return true
end
return false
end)
kumo.on('make.kafka', function(domain, tenant, campaign)
local connection = {}
local kafka_config = {
["bootstrap.servers"] = "xxx.xxx.xxxx.xxx:port",
["queue.buffering.max.ms"] = 1000,
["message.timeout.ms"] = 5000,
["retries"] = 1,
["retry.backoff.ms"] = 100,
["compression.type"] = "snappy",
}
local producer = kumo.kafka.build_producer(kafka_config)
local topics_map = {
['Reception'] = "test1",
['Delivery'] = "test2",
['Bounce'] = "test3",
['TransientFailure'] = "test4",
}
function connection:send(message)
local connection = {}
local campaign = message:get_meta('campaign')
local msg_reception = message:get_meta('msg_reception')
local msg_delivery = message:get_meta('msg_delivery')
local msg_bounce = message:get_meta('msg_bounce')
local msg_transient = message:get_meta('msg_transient')
print('make.kafka --- campaign, msg_reception, msg_delivery, msg_bounce, msg_transient',campaign, msg_reception, msg_delivery, msg_bounce, msg_transient)
-- some other code
return connection
end
function connection:close()
producer:close()
end
return connection
end)
kumo.on('get_queue_config', function(domain, tenant, campaign, routing_domain)
print('get_queue_config --- domain, tenant, campaign, routing_domain',domain, tenant, campaign, routing_domain)
if domain == 'kafka' then
-- Use the `make.webhook` event to handle delivery
-- of webhook log records
return kumo.make_queue_config {
protocol = {
custom_lua = {
constructor = 'make.kafka',
-- how to send other params?
},
},
}
end
return kumo.make_queue_config {}
end)
In that case, the tenant, campaign and full message object are available in several places.
Option 1: decide at should_enqueue_log_record event:
if msg:get_meta('tenant') == "mytenant" then
local log_record = msg:get_meta 'log_record'
if log_record.queue ~= 'kafka' then
msg:set_meta('queue', 'kafka')
return true
end
end
return false
end) ```
Option 2: do it in the `make.kafka` function:
``` kumo.on('make.kafka', function(domain, tenant, campaign)
if tenant ~= ""mytenant" then
return
end
local connection = {}
...
Option 3: in get_queue_config
if domain == 'kafka' and tenant == "mytenant" then
...```Maybe somethings wrong . from I send the code , I never get tenant and campaign . SO I think
lua kumo.on('make.kafka', function(domain, tenant, campaign)
print('make.kafka - tenant ' , tenant) -- it will be nil
if tenant ~= ""mytenant" then
return
end
local connection = {} ```
I'll take a closer look. I think the data for the make.kafka event can only be obtained from LogRecord, and the tenant inside might be something like default-queue@gmail.com.