How to set rate limits for a specific source IP under the Yahoo provider

KumoMTA Helpers
, , , ,
1

For example, currently we can only do:

["yahoo.com".sources."sourceip"]
max_message_rate = "50000/h"

Is it possible to support something like:

["provider"."yahoo".sources."sourceip"]
max_message_rate = "500000/h"

Yahoo has too many domains, such as yahoo.com, yahoo.com.br, yahoo.com.au, yahoo.co.jp.

UseCase: We need to apply rate limiting to specific IPs to warm-up.

we really like using the helper approach because it allows non-technical support staff to maintain these configuration files. However, once Lua code is involved, technical personnel have to step in. Thank you.

2

You can apply using provider instead of domain, just like your example. Configuring Traffic Shaping - KumoMTA Docs

3

Yeah, it is the `sources part that does not work.
This works
[provider.“mairs.org”]
This does not
[provider.“mairs.org”.sources.“demo-IP1”]

4

Yes, you can do per-provider-per-source rules:

[provider."yahoo"]
match=[{MXSuffix=".yahoodns.net"}]
max_deliveries_per_connection = 20

[provider."yahoo".sources."sourcename"]
max_message_rate = "500000/h"

Pro Tip: you can use validate-shaping to parse and confirm that the syntax is correct:

$ /opt/kumomta/sbin/validate-shaping /tmp/prov.toml
INFO: approx memory used = 2.7 KiB
OK
5

also worth calling out: make sure that you understand the difference between max_message_rate and provider_max_message_rate. The former will apply a limit per-source-per-site-name, while the latter will be per-source-per-provider, so make sure that you pick the one that makes sense for your use case

6

Can confirm. My test had combined provider config with shaping config and you need to separate them.

This is bad:

[provider.“mairs.org”.sources.“demo-IP1”]
match=[{MXSuffix=“mairs.org”}]
idle_timeout = “2m”
max_ready = 1000
connection_limit = 200

This is good:

[provider.“mairs.org”]
match=[{MXSuffix=“mairs.org”}]

[provider.“mairs.org”.sources.“demo-IP1”]
idle_timeout = “2m”
max_ready = 1000
connection_limit = 200

7

I believe it’s:

“This sparks joy”

“This does not spark joy”

8

Thank you! I really love it here — it’s much more fun than Discord.

9

Based on my tests, this configuration:

[provider.“yahoo”.sources.“haproxy-ip1”]
max_message_rate = “10/h”

only supports max_message_rate; it does not support provider_max_message_rate.

Error message:

/opt/kumomta/share/policy-extras/shaping.lua:571: in function </opt/kumomta/share/policy-extras/shaping.lua:566>
Error: Initialization raised an error: call validate_config callback: provider: yahoo: interpreting provider 'yahoo' source 'haproxy-ip1' provider_max_message_rate = "10/h"
 as EgressPathConfig: unknown field provider_max_message_rate  
stack traceback:
    [C]: in local 'poll'
    [string "?"]:27: in field 'load'
    /opt/kumomta/share/policy-extras/shaping.lua:571: in function </opt/kumomta/share/policy-extras/shaping.lua:566>

However, based on my understanding, what I need is per-source-per-provider.

10

As mentioned above, you need to keep the provider config and the shaping config separated.

11

Max_message_rate is only valid in shaping config

Provider_max_message_rate is only valid in the provider config

12

Yes, exactly. I’ve already separated them and confirmed through testing.
This is good:

[provider."yahoo".sources."haproxy-ip1"]
max_message_rate = "10/h"

This is bad:

[provider."yahoo".sources."haproxy-ip1"]
provider_max_message_rate = "10/h"

:grinning_face: