[BALUG-Admin] anti-spam, exim4, postfix, mailman/mailman2/mailman3, integration, ... web abuse & bots, ...

[Michael Paoli]

> From: "Rick Moen" <rick@linuxmafia.com>
> Subject: Re: [BALUG-Admin] much better now: Re: /var space issues on  
> balug VM, almost	certainly from Internet miscreants & their bots
> Date: Mon, 26 Apr 2021 10:46:26 -0700

> Quoting Michael Paoli (Michael.Paoli@cal.berkeley.edu):
>
> Well, what can I say?  EximConfig (no longer maintained and too crufty
> to even be a reasonable point of departure, any more) was in its heyday

Yep, once-upon-a-time.  But alas, that time is gone.
Might still possible "mine" it for anti-spam ideas in general,
though ... and carefully pick and choose from that - along with
considerations of modern, best practices, etc.

> It was always the case, however, that occasionally you would find that
> some of the ruleset entries were overaggressive and unwise, so you would
> comment those few out as you found them.

Oh yes, (anti-)spam will probably always continue to be an escalation
war.  And probably never quite be 100%.  Yeah, sure, there's always
content ... the definitive deciding difference, eh?  But even that gets
rather dicey, at best.  Say you're running a list ... for mail admins,
say it's a list that mostly discusses (anti-)spam, and often cites
example excerpts - perhaps even sometimes rather to large chunks of
such cited examples of spam content.  So, I ask those that say, "sure,
can always tell by content" - okay, how 'ya gonna do it by content
for that case of legitimate vs. illegitimate (spam) mail?

But a lot of the simpler cases can be covered by egregious and
non-conformant behavior, and lack of adherence to reasonable
best practices which are pretty much effectively "required" if one
is expecting email sent across The Internet to land in an "inbox".

> Pretty soon, I'm going to pick your brain on some parts of modern Postfix
> antispam setup -- assuming you're current on that.  I'm fond of exim4,
> but am thinking it might finally be the right time to bail.  Not decided
> on that.  I might instead pick your brain on some parts of modern exim4
> antispam setup.

My brain may likely be more ripe for the pickin' in another week (or two
or three or so).  Haven't done a huge bit 'o anti-spam yet ...
actually current is rather highly minimal.  It's presently approximately:
o sure as hell don't be a relay
o only accept to legitimate domains
o reject some cruft that exim doesn't even like by default
o most everything else is for mailman, as for lists, it's mostly
   o not authorized to post, held for moderator (I should reconfirm no
     backscatter there)
   o other mailman aliases - mostly handled reasonably (not suitable command
     or format, or not from subscriber, or lacks subscriber's password ...)
o the moderate bit of other target addresses, not so great, but mostly
   just "deal with it" - for now, e.g. list -owner and postmaster@ and
   the like probably have lots 'o spam coming in, but for the moment that's
   tolerable, as it doesn't impact subscribers, and the volume is low enough
   it's not a systems resource issue.
o outbound: SPF & appropriate source IP(s) & "reverse" DNS (and don't relay,
   etc.)

Oh, and one semi-simle(ish) that I want to add, that would make things
much less annoying for listadmins (e.g. me), is reject at SMTP attempts
to sent to list from those not authorized to post to list - period.
And with suitably customized text on the reject error.  Should be rather
easy to tie a hard whitelist of address(es) in file(s) to exim, and should
be easy enough to write program(s) to fill/update those files and have
exim pickup any relevant changes thereof.  One of the slightly tricker
bits I'm presently looking at, is for subscribers, do they or do they not
have the moderate(d) flag set (if it's set, don't whitelist 'em).
Maybe that gets easier with mailman3?  Who knows.

But, hopefully soonish, I'll have the anti-spam much better.  I'm thinking
such that, e.g. postmaster@ email gets down to a (semi-?)reasonable
(near) trickle of email ... likewise for other email addresses.  Not there
yet, but ... hopefully soon?

As for exim[4] "vs." Postfix (vs. ... sendmail 8-O ... or what have you).
Some year(s) back - and I occasionally check again - where's the
critical mass - what seems best (or at least reasonably) supported to
e.g. integrate with probable likely or needed stuff, e.g. mailman,
anti-spam, dokuwiki, wordpress, ... whatever.
The most logical choices seem exim or postfix, after that the numbers
dwindle very fast - at least on Debian.  Yeah, popcon ... still looks
fairly similar:
https://qa.debian.org/popcon.php?package=exim4
https://qa.debian.org/popcon.php?package=postfix
Also, looking over the more complex (not horribly so, but not exactly
trivial) integrations, ... e.g. mailman2.  Looks like (at least on
Debian) that's mostly oriented towards exim or postfix ... with actually
better documentation/examples, etc. for exim than postfix.  Not sure
about mailman3 yet, though I suspect I'll be finding that out in the
not-too-horribly-distant-future.  I'd also guestimate, again for Debian,
mailman2 very well having documentation for exim/postfix integration, and
especially exim, I'd guestimate the documentation/examples for mailman3
will be similar.  But ... shall see.
So, ... thus far I've been leaning towards exim over postfix ... though
postfix seems a strong reasonable contender worthy of consideration,
and periodic reevaluation/reconsideration.
Can't say exim thrills me, but, given the options/possibilities, for
a modern MTA, and one that needs well integrate with other stuff,
presently seems to me exim is the "best" (least horrible?) option.
No matter what, it'll end up fairly complex.  Question is how complex
and nasty(?), and how well/reasonably supported/documented ... and/or not.

Also seems Debian's wiki could use some more TLC ... notably more current
examples, e.g. mailman/mailman3, and exim integration and anti-spam, etc.
Maybe I'll end up doing some 'o that.  I was already thinking of
doing the mailman/exim integration stuff on wiki there, as it seems kind'a
not well covered (though relevant documentation is found elsewhere within
Debian - notably files within the packages) - but would be good to have wiki
page that points that out and gives examples and/or makes relevant
recommendations.

Oh, and "of course" ... bad bots ... not just SMTP abuse, ... but web
too ... if the find any form that can cause an email to be generated,
they'll often do so. 332 queued email messages stuck trying to go
out (to only 2 target email addresses) were from such web abuse.
Sure, subscribe such-and-such email address to the list ...
that causes a confirmation email to be sent (attempted to be sent).
Well, ... that can be be repeated many times ... at least 332 apparently.
Even worse if spammers can abuse the form to at all control the content of
what's emailed ... if they can, they'll put their spam message in
that.  Not sure an easy counter-measure for such.  Maybe mailman3 is
better on that?  There's captcha ... slows or even stops bots, but
dang annoying for the humans - many of 'em are to the point where
AI can solve 'em better and faster than the humans, so ... AI CPU
resource consumption might slow the bots, but won't stop 'em ... and
as AI gets better and better and CPU(/GPU) resources faster and cheaper,
I think in fairly short order AI is gonna win the captcha wars, so I don't
see captcha being all that useful for all that long.  So at best
captcha is more about throttling, and not so much about stopping.

Bot abuse basically killed self-registration for dokuwiki on balug VM,
and I think that may be the case for wordpress on that VM too.