Hi there,
We have many IP addresses we want to use as egress sources, and we want to apply a warmup process.
We have a warm-up process that defines the daily rate limit per IP (and per domain). As I found in the documentation, I can add source_selection_rate configuration entry in my shapping.toml configuration that sets the rate limit per domain per source.
The question is whether there is an existing solution or component in Kumo that can help manage the warmup plan?
By an existing solution, I mean a mechanism that can read the plan and manage the current state of each source and domain, and according to this, apply the current rate limit.
As we have tens of IPs, the shaping configuration for the warmup is massive. Therefore, we are also looking for a centralized configuration that allows us to configure a list of sources and a list of stages, rather than a massive shaping file with hundreds of lines.
I looked for any solution on the internet and AI, and got two general options:
Implement such a mechanism inside the MTA that will run on get_egress_path_config event, which will check the stage of a selected source, calculate the current stage, and maintain warmup state using the defined warmup plan configuration
Create a script that takes as input the warmup plan and current warmup state, and populate an updated shapping configuration. Stage duration is a minimum of 1 day. The script will be configured with CRON and a Lambda function, and will maintain the state as an S3 file or any DB table
To my knowledge, anyone who has done this on KumoMTA so far has written their own script to modify the shaping. This can be an external script that walks your plan and changes the shaping.toml file daily. It is crude, but it works.
Our goal is to make that much more elegant and intelligent, but it will require enough interest and funding to do it right. Let us know if you are interested in a sponsorship agreement. Most of the really cool features were driven by paid sponsors.
Well, we handle this directly at our own level. Each IP address has a emails_sent, emails_max in the database and that is used to control the number of emails that go through it every day.
Ideally, you should be doing MX Rollups (is that the right term??), querying the domain’s MX records and mapping them to the ESP, and having a default max limit and a per-provider limit.
It is similar to one of my earlier experiments. First, I preferred to use source_selection_rate with the shaping policy, as I give to the MTA, the responsibility for counting and applying limits.
Second, as I have a “main switch” for emails_max, it will take effect for all sources and domains. However, for my use case, I need the option to control each source and domain separately.
Currently, we have over 200 egress sources which can be used by any MTA instance in the cluster (using Kumo SOCKS5 Proxy), and not all the sources are at warmup or at the same warmup stage
Our migration to KumoMTA was a bit rushed, so we didn’t get time to carefully plan and use every feature it offers. I remember that we migrated to KumoMTA in like 2 or 3 hours because our MTA at that time simply failed.