Way to get message metadata in get_queue_config event

get-help
1

Hello,

Is there a way to retrieve metadata from a message in the get_queue_config event?

I want to do this because I would like to choose the queue config (e.g., ingress_pool) based on metadata from the message.

Examples:

  • First example: I want to send a marketing email using pool A and a transactional email using pool B. The “transactional/marketing” info could be in the message metadata.
  • Second example: I have a score for a tenant outside of Kumo and want to choose the pool according to the score.

I have a few potential solutions:

  • The tenant could be an aggregation of information, such as tenant_transactional or tenant_score5. I’m not really convinced by this option.

  • Before pushing an email to Kumo, I could update the Kumo queue config (in Redis or a config file), invalidate the cache, and then push the email to Kumo. I like this option, but I’m concerned about handling high email volumes and frequent Kumo config updates.

Additional question: Is it a good practice to load configs from Redis instead of a TOML/JSON file?

Hope this make sense for you.

Thank you for your help.

3

Overloading the tenant meta has worked for me in the past, but it depends on how complicated your needs are.

4

The previous example was just for illustration, but what we have in mind is much more complicated.

5

Another slightly hackish possibility is to write the vars you want to script locals that you can retrieve anywhere. This has limitations though.

6

Is there a way to retrieve metadata from a message in the get_queue_config event?

get_queue_config is for configuring a queue. A queue is instantiated based on its queue name. Queue naming is discussed in Queues - KumoMTA Docs

If you want different messages to be placed in queues with different configurations, you need to ensure that the message metadata names a different queue. The easiest way to achieve that is to use tenant, campaign or routing_domain metadata.

7

get_queue_config must always return the same configuration when it is called with the same parameters. It’s okay if it varies over time to reflect changes in the configuration, but if you are trying to vary it based on any other state that you’re trying to carry through from a message, the system behavior will be undefined and inconsistent.

8

Additional question: Is it a good practice to load configs from Redis instead of a TOML/JSON file?

It’s fine, but you need to have a strategy for what you want to happen if Redis is unavailable, and you need to ensure that you handle that in your policy logic.

Handling that can be more of a headache than going a bit lower tech and having your configuration deployment pipeline just pre-build the config as a set of json files and pushing those to the nodes.

My personal preference is to push config files to the node as it feels easier to reason about overall, and you can then use eg: kumod --validate as part of your pipeline to validate the overall setup before it goes live.

9

Thank you for you feedback. :slightly_smiling_face: