log_hooks:new_json does not include custom header in the webhook payload

get-help
21

I can’t reproduce this; when I add a header it shows up in the headers map.
My theory is:


 local tenant = cached_tenant_id(msg:from_header().domain)
  if tenant then -- tenant here is nil so none of this block is taken
    msg:set_meta('source', 'http')  -- you have no meta.source
    msg:set_meta('tenant', tenant)  -- meta.tenant is actually set by the queue helper
                                    -- based on the rules in your toml file(s)   
    msg:append_header('X-CUSTOMER-ID', tenant)
22

That’s the thing - I suspected that and therefore added a `print(msg:get_data()):

kumo.on('http_message_generated', function(msg)
  local tenant = cached_tenant_id(msg:from_header().domain)
  if tenant then
    msg:set_meta('source', 'http')
    msg:set_meta('tenant', tenant)
    msg:append_header('X-CUSTOMER-ID', tenant)
  else
    kumo.reject(
      500,
      string.format("from domain '%s' is not allowed.", msg.from_header().domain)
    )
  end
  print(msg:get_data())
  queue_helper:apply(msg)
  dkim_signer(msg)
end)
23

It shows the X-CUSTOMER-ID header in the output:

Content-Type: multipart/related;
        boundary="rE1UwM9AQliTUAG4rlKLAw"
To: <recipient@example.com>
From: Someone <someone@example.com>
Reply-To: Help <help@example.com>
Subject: This is the subject
X-Something: Something!
Mime-Version: 1.0
Date: Mon, 4 Dec 2023 20:50:27 +0000
X-CUSTOMER-ID: test

--rE1UwM9AQliTUAG4rlKLAw
Content-Type: multipart/alternative;
        boundary="xAVvEjkCQjexFplwNB5Qqw"

--xAVvEjkCQjexFplwNB5Qqw
Content-Type: text/plain;
        charset="us-ascii"

This is the plain text part
--xAVvEjkCQjexFplwNB5Qqw
Content-Type: text/html;
        charset="us-ascii"

<p>This is the <b>HTML</b> part</p>
--xAVvEjkCQjexFplwNB5Qqw--
--rE1UwM9AQliTUAG4rlKLAw
Content-Type: image/gif
Content-Transfer-Encoding: base64
Content-Disposition: inline;
        filename="pixel.gif"
Content-ID: <my-image>

R0lGODlhAQABAIAAAAAAAP///yH5BAEAAAAALAAAAAABAAEAAAIBRAA7
--rE1UwM9AQliTUAG4rlKLAw--
24

so tenant is not nil - it’s actually going into the if body and adding the X-CUSTOMER-ID header, right?

25

can you add msg:append_header('X-CUSTOMER-ID', 'unconditional') as the first line of your kumo.on('http_message_generated', function(msg) handler? That way I’d expect to see the header remain set as unconditional and see it show up in the headers list if that tenant mapping function is returning nil

26

sure.

27

what I tested locally was:

kumo.on('http_message_generated', function(msg)
  msg:append_header('X-CUSTOMER-ID', '1234')
end)

and I could see the header show up in the webhook json payload

28

the output is

Content-Type: multipart/related;
        boundary="tgv8y57ZR4avZGy+eBAFNQ"
To: <recipient@example.com>
From: Someone <someone@example.com>
Reply-To: Help <help@example.com>
Subject: This is the subject
X-Something: Something!
Mime-Version: 1.0
Date: Mon, 4 Dec 2023 20:56:50 +0000
X-CUSTOMER-ID: unconditional
X-CUSTOMER-ID: test
29

webhook payload is unchanged -

{"type":"Reception","id":"a572ad4192e711eebc2c080027be1609","sender":"noreply@example.com","recipient":"recipient@example.com","queue":"unconditional@example.com","site":"","size":883,"response":{"code":250,"enhanced_code":null,"content":"","command":null},"peer_address":{"name":"","addr":"192.168.0.100"},"timestamp":1701723410,"created":1701723410,"num_attempts":0,"bounce_classification":"Uncategorized","egress_pool":null,"egress_source":null,"feedback_report":null,"meta":{"tenant":"unconditional"},"headers":{"Subject":"This is the subject"},"delivery_protocol":null,"reception_protocol":"HTTP","nodeid":"6c48c1dc-5f6a-487c-ab67-c3e6f426d6a6"}
30

"meta":{"tenant":"unconditional"

31

you mean msg:set_meta('tenant', 'unconditional') at the top of the function?

32

I mean, the payload is changed; it nows logs meta.tenant = 'unconditional'

33

Then how is it that the X-CUSTOMER-ID: test is added here?

34

I can’t explain it!

35

not sure if this is related


# Set the tenant from this header
tenant_header = "X-CUSTOMER-ID"
remove_tenant_header = true

# Set the campaign from this header
campaign_header = "X-Campaign"
remove_campaign_header = true

in queue.toml

36

ah, yes, when you apply your queue helper, the header will be removed

37

unless you set remove_tenant_header = false

38

Sorry for wasting your time! :man_facepalming: