Headers not in logs

get-help
1

I just noticed that in my setup I kinda had the default config for the kumomta logs like this:

  kumo.configure_local_logs {
    log_dir = '/var/log/kumomta',
    max_segment_duration = '10 minute',
    headers = { 'Subject' },
  }

I also use the policy-extras.log_hooks to log stuff into logstash… but no Subject logging there too.

log_hooks:new_json {
  name = 'webhook',
  url = 'http://x.y.z.a:5080/',
  log_parameters = {
    headers = { 'Subject' },
  },
  queue_config = {
    retry_interval = '1m',
    max_retry_interval = '20m',
  },
}

But when looking in the logs I don’t see the Subject header in any of the logs. I did want to add extra logging of headers due to the NDR messages not being send. But the Subject not being in any of the logs make s me think that either my config is broken or that there is a bug in logging the headers.
In the logs I see this:

{
  "type": "Delivery",
  "id": "3c6b9825134a11efa6e002666fd40793",
  "sender": "firstname.lastname@xxxx.com",
  "recipient": "firstname.lastname@yyyy.com",
  "queue": "http://127.0.0.1:8008.tsa.kumomta",
  "site": "unspecified->http://127.0.0.1:8008.tsa.kumomta@lua:make.http://127.0.0.1:8008.tsa.kumomta",
  "size": 895,
  "response": {
    "code": 200,
    "enhanced_code": null,
    "content": "200 OK: ",
    "command": null
  },
  "peer_address": {
    "name": "Lua via make.http://127.0.0.1:8008.tsa.kumomta",
    "addr": "0.0.0.0"
  },
  "timestamp": 1715839503,
  "created": 1715839503,
  "num_attempts": 0,
  "bounce_classification": "Uncategorized",
  "egress_pool": "unspecified",
  "egress_source": "unspecified",
  "feedback_report": null,
  "meta": {},
  "headers": {},
  "delivery_protocol": "Lua",
  "reception_protocol": "LogRecord",
  "nodeid": "234f64a1-0345-419b-a259-8aad0bbf8ea3"
}

OS: Debian 10
KumoMTA version(s): 2024.04.12.081810.af61dee6 and 2024.05.29.111636.d8cc6d98

2

Hey there @magnanimous-umbrella, thanks for posting. Please read the “Troubleshooting” and “How to Ask for Help” buttons below. If you would like a 1:1 support session from the KumoMTA team, details are at the “Book a Support Session” button below.

3

Darnit.. Just realised I posted a wrong log :slightly_smiling_face:

{
  "type": "Delivery",
  "id": "2f3f072723a911efa16902666fd40793",
  "sender": "firstname.lastname@xxxxx.com",
  "recipient": "firstname.lastname@yahoo.com",
  "queue": "all-ips@yahoo.com",
  "site": "smtp-out2->(mta5|mta6|mta7).am0.yahoodns.net@smtp_client",
  "size": 6148,
  "response": {
    "code": 250,
    "enhanced_code": null,
    "content": "ok dirdel",
    "command": ".\r\n"
  },
  "peer_address": {
    "name": "mta5.am0.yahoodns.net.",
    "addr": "67.195.228.106"
  },
  "timestamp": 1717639504,
  "created": 1717639502,
  "num_attempts": 0,
  "bounce_classification": "Uncategorized",
  "egress_pool": "pool-all-ips",
  "egress_source": "smtp-out2",
  "feedback_report": null,
  "meta": {},
  "headers": {},
  "delivery_protocol": "ESMTP",
  "reception_protocol": "ESMTP",
  "nodeid": "234f64a1-0345-419b-a259-8aad0bbf8ea3",
  "tls_cipher": "TLS13_AES_128_GCM_SHA256",
  "tls_protocol_version": "TLSv1_3",
  "tls_peer_subject_name": [
    "C=US",
    "ST=California",
    "L=Sunnyvale",
    "O=Oath Holdings Inc.",
    "CN=*.mail.am0.yahoodns.net"
  ]
}

But no headers here either. I do know that my test mail which I am sending to yahoo does have a subject.

4

FWIW, we do have integration tests for logging headers, and they are passing.

5

thanks @free-spirited-yorksh I’ll dive in it with a test setup.. not sure yet… it will probably be my fault somewhere

6

well on a test server I copied over my production config and after a few confg changes it worked and I also get the headers in the logs when testing… I am still a bit confused how this is possible.. but it must be either the config or my Debian 12 package… I tested on Debian 11 with the Ubuntu 20.04 package kumomta-dev using the openrepo sources.list. So I will need to investigate more :frowning_face_with_open_mouth:

7

Well I stopped kumomta and kumo-tsa-daemon… upgradede to the latest Debian 12 build 2024.06.10.140609.74a0f2be and it started logging. I don’t get it.. but I will take it :slightly_smiling_face: In addition I did setup a Debian 11 system with the Ubuntu 22.04 stable release and that also worked (with the same config as the other server). I guess Murphy was in the house… Case closed for me.