Quoting Michael Paoli (Michael.Paoli(a)cal.berkeley.edu):
>
> Did notice you changed the default on monthly password reminder to
> no ... was hoping I'd get to resetting users on that to no
> before this month's reminder ... alas, didn't get to that in time,
> but hoping/expecting I'll probably get that wiggled in soon.
> I've got earlier notes on the bit in the Python Pickle files,
> and also have programs I used for dump/restore of user options (all
> but password) when earlier doing list migrations ... and may just
> tweak some of that code to turn the password reminder off on all the users -
> at least for the shorter term, if not indefinitely (and probably note
> which ones then got their preference flipped, in case there's ever
> need/reason to reverse that).
Yes, indeed I didn't change the "Get password reminder email for this
list?" toggle for any existing user, just (as you say) for new
prospective users -- the latter by setting "Send monthly password
reminders?" (send_reminders, General options) to "no" on all four lists,
instead of the default yes.
linuxmafia:/tmp# su - list
No directory, logging in with HOME=/
list@linuxmafia:/$ cd /var/lib/mailman/bin
list@linuxmafia:/var/lib/mailman/bin$ ls
add_members convert.pyc invite_members postfix-to-mailman.py show_qfiles
arch discard list_admins postfix-to-mailman.pyc sync_members
b4b5-archfix dumpdb list_lists qmail-to-mailman.py transcheck
change_pw export.py list_members qmail-to-mailman.pyc unshunt
check_db export.pyc list_owners qrunner update
check_perms find_member mailmanctl rb-archfix version
cleanarch fix_url.py mmsitepass remove_members withlist
clone_member fix_url.pyc newlist reset_pw.py
config_list genaliases paths.py reset_pw.pyc
convert.py inject paths.pyc rmlist
list@linuxmafia:/var/lib/mailman/bin$
If memory serves, the way you massage individual subscribers' options
from the command line is using /var/lib/mailman/bin/withlist , where
you need to tell withlist to follow a very Python-ey instruction script.
_However_, the wording on https://wiki.list.org/DOC/How%20do%20I%20turn%20off%20passwords%20completel…
suggests that frobbing off "Get password reminder email for this list?"
in the per-list admin WebUI causes _nobody_ to receive password
reminders.
I actually didn't notice whether I got password reminders from
lists.balug.org, yesterday, or not.
BTW, that some page says that password reminders can be disabled
_sitewide_ by the siteadmin, by simply removing
/usr/lib/mailman/cron/mailpasswds .
linuxmafia:/usr/lib/mailman# mkdir cron.disabled
linuxmafia:/usr/lib/mailman# mv cron/mailpasswds cron.disabled/
linuxmafia:/usr/lib/mailman#
> See also: https://www.balug.org/wiki/doku.php?id=balug:mail_and_lists
> https://www.balug.org/wiki/doku.php?id=balug:balug_todo_list (alas,
> even the "todo" list needs some significant updating)
Thank you for the gentle reminder. I will attend to that, as to my
bits.
So ... did this a while ago,
for "vicki" and the balug VM, changed the system default timezone
2019-05-26T23:23:22+0000 Michael Paoli
reconfigured default system timezone to Etc/UTC
... and rather close in time to that, likewise on the vicki host.
And, more recently, updated some cron[tab] jobs,
most notably I pushed most all the daily/weekly/monthly ones forward 8
hours. Why? To better correlate to / coordinate with local times,
e.g. monthly email reminders/stats - to continue 1st of month, rather
than local last of month. Also, sf-lug list backups, better correlate
to grabbing those relatively freshly after their available,
as opposed to - 7 or - 8 + 24 = about 16 or 17 hours after they
become available, likewise keep those data transfers generally towards
wee/early hours of the AM local time (at/towards minimum, or at least lower
traffic periods).
> From: "Michael Paoli" <Michael.Paoli(a)cal.berkeley.edu>
> Subject: BALUG VM (& "vicki" - sometimes physical host of BALUG):
> default timezone --> UTC (?
> Date: Sun, 26 May 2019 11:09:34 -0700
> Unless someone convinces me otherwise (unlikely),
> I'll probably proceed fairly soon, to reconfigure the
> BALUG VM (Virtual Machine) - (host balug-sf-lug-v2.balug.org)
> and likewise "vicki" (hostname vicki - and the sometimes /
> semi-regular physical host of the BALUG VM)
> to default to using UTC.
> To minimize surprises/disruptions - in addition to this notice,
> I'll probably make that change upon a (re)boot of those hosts
> (e.g. on reboot, go to single user mode, reconfigure,
> then reboot per normal to multi-user).
>
> Rationale:
> o security, etc. - many won't even consider looking at logs if they're
> not in UTC
> o no matter who uses it from where on the planet, one zone all can (well
> approximately) reasonably agree upon
> o no need/reason to change it in the (unlikely) even it moves to another
> physical location, or timezone at existing physical location changes
> o "principle of least surprise" - if it was a whole bunch 'o local folks
> doing admin, etc. on the box, and especially more "jr." folks, local
> would be of least surprise. But alas, yours truly does >>~=99.7 %
> of the systems administration, etc. on those hosts, so that being
> the case, and having been the case quite a while, and seeming improbable
> to change ... for me, that "principle of least surprise", and other
> reasons/advantages ... UTC
> o Users can always use TZ setting to whatever they wish that's available,
> we're only talking about the system default timezone, e.g.:
> $ TZ=America/Los_Angeles; export TZ
So,
The the exim4 + eximconfig configuration is pretty darn good about
blocking spam ... but it's not quite 100%.
And ... bad bots, etc., have already started to pick up the
@temp.balug.org list posting addresses - no surprise there,
and I'm sure they'll likewise also pick up @lists.balug.org -
and/or already have those addresses from earlier.
So ... fortunately has been quite low volume :-) ... but there has
been some spam/UCE making it through exim4 + eximconfig to the
list posting addresses (and then held for moderation as "post" from
non-member). Have reviewed those, and ... heck no, haven't added 'em
to Mailman's automatic rejection (that would be relatively uselessly
"too late" (post-SMTP)) ... but did make some selective additions
in the exim4 + eximconfig configuration.
Most notably, the wee bit 'o spam/UCE that had made it through to list
posting addresses, seemed to fit roughly in 2 general categories (at
least from that seen thus far):
Unsolicited Commercial Email (UCE) spam - from senders/businesses that
might otherwise be/seem quite legitimate, but are sending email that's
very much unsolicited, not confirmed opt-in, typically in violation of
relevant regulations, etc. regarding anti-spam ... but they're sending
anyway (they ought be hit by a clue-by-four). And also not a
"Joe job" - someone/something impersonating an otherwise legitmate
sender (spoofed). So ... for the few of those (I think about two thus
far), I added their domains to be blocked by exim4 + eximconfig as spam.
And the others ... sufficiently well constructed to make it past the
spam filters, but most likely fly-by-night spam operations - e.g. buy
up some cheap or ultra-cheap domain, set up some DNS/email infrastructure,
and ... start spamming. Not much to be done about those ... except,
where they're yet another crud TLD I'd not already so added, added the
TLD for explicit greylisting (e.g. TLD bid). That doesn't necessarily stop
spam from those, but ends up thwarting a quite large percentage of them -
while also letting through any (theoretical) legitimate email - without
too much additional burden/delay on sender (e.g. one would think .xzy is
utter cr*p, ... but I know of one person that legitimately has and
uses such a domain ... perhaps not at all for email, but ... at least
hypothetically). So ... all those "crud" domains ... (and spammy
country code TLDs, etc.), generally inclined to treat 'em with additional
scrutiny (notably also adding greylisting), but not absolutely outright
block/reject 'em. Also, might not / probably wouldn't be a good idea,
but could also add additional layer of requiring callouts on those too ...
but that could be a bad thing - even for such highly suspect domains -
so not even seriously considering it at this time.
I might also do some blanket adding of "crud" TLDs for additional
checks (notably greylisting). So far I've based it upon what
eximconfig gave as default starting point, plus some adjustments
notably for those I've never seen send a legitimate email but I see
(notably elsewhere) sending plenty of spam (enough for me to recall
and groan about thinking of the TLD, without even looking over
spam collection details/stats).
Oh, random bit - I did see a very small percentage of list subscriber
addresses that do callouts (as evidenced in the logs).
And also, mailman has pretty good logs ... I don't have to worry too
much about "oh, I just deleted all those spam emails ... what if I wanted
to add some more anti-spam measure at SMTP time for them?" ... not an
issue - can find most relevant bits about those held for moderation
of post attempt from non-member in the mailman logs - at least for some
reasonable period.
Some web stats by domains, etc.
I was interested, thought others might be curious/interested too, so ...
I've got the log rotation set up so it retains bit over a year's worth of
web server logs
$ awk '{if($1=="rotate" || $1 !~ /^#/ && $1 ~ /ly$/)print}'
/etc/logrotate.d/apache2
weekly
rotate 60
so, fair bit of data available.
This web server runs on the balug Virtual Machine (VM) host,
which covers for not only the BALUG [Linux] User Group ([L]UG),
but others too. Anyway, grabbed the data from the logs, and did a wee
bit of analysis. This is from the {ssl_,}access.log* files.
So, basically analyzed domain and port, here's the information/analysis,
and counts, from the highest levels on down to that:
13945791 Total
That's the total traffic seen, all sites/domains that got logged.
That's approximately:
232430 per week (13945791/60)
33204 per day (13945791/60/7)
1384 per hour (13945791/60/7/24)
23 per minute (13945791/60/7/24/60)
0.38 per second (13945791/60/7/24/60/60)
So about one per 2.6 seconds (1/(13945791/60/7/24/60/60))
by [L]UG or project or whatever and the like. Note also that test
traffic isn't excluded, so, e.g. some domains that aren't in DNS or
so delegated (or not yet or no longer in DNS) also show wee bit 'o
traffic.
9504843 BALUG
3463857 BerkeleyLUG
968724 SF-LUG
8283 digitalwitness.org
51 BAD
33 BUUG
In a bit more detail, by relevant TLD:
9504843 balug.org
3457611 berkeleylug.com
896560 sf-lug.org
40847 sf-lug.com
13698 sflug.org
8283 digitalwitness.org
7458 sflug.net
6246 berkeleylug.org
6084 sflug.com
4077 sf-lug.net
51 bad.debian.net
33 buug.org
Note that for all of SF-LUG's various domains,
traffic drops by >20x once we leave the canonical TLD ([www.]sf-lug.org)
and drop to 1st runner up in SF-LUG traffic, >65x to 2nd runner up, and
> 120x for 3rd runner up.
For BerkeleyLUG, the obscure and almost completely unknown (was never
canonical nor particularly promoted) BerkeleyLUG.org,
The canonical BerkeleyLUG.com has >550x the traffic ...
however BerkeleyLUG.org no longer exists, so it was something >1/550 of
the traffic when it still existed (but even back then I recall it being
relatively negligible portion of traffic).
So, by domain ... secure.balug.org is somewhat surprising. It's probably
mostly from "bad" or not so well behaved bots, poking at the BALUG wiki,
trying to login there, and getting redirected ... or maybe such bots
thinking there's something more "interesting" to go after because it's
got "secure" in the name? Maybe ought phase that one out, not nearly
so relevant anymore. I think the original idea is that one would
redirect to force https, but I believe now all the domains offer https,
and as/where relevant (e.g. wiki login and after doing so with cookie
that has authenticated state) redirection can be done from http to
https with same domain - no need for a separate domain for that.
Also a bit surprising too, is BerkeleyLUG.com being as high as it is.
It's WordPress, and bots - legitimate, or not so, and search engines,
well, WordPress effectively "expands" to a quite huge set of unique
URLs, even if the content of each isn't all that incredibly unique.
E.g. I remember earlier trying to crawl the site when it was still
hosted by WordPress.com - it expanded into a huge amount of content
that wasn't particularly feasible to replicate in-place as-is without
WordPress - especially relatively to the actual full data in/behind
the site - a much more manageable and smaller set of data. So that
may, at least partially, explain the somewhat surprising large number
there.
Likewise, BALUG's wiki - lots of content - especially if one crawls the
archive of all older versions of all pages.
And BALUG's list - every posting can be individually crawled, so, lots of
URLs, and probably keeps search engines / bots relatively busy.
And one other that's slightly surprising ... the mx entries.
Really noting promoting - or even configured for that as web,
other than it existing in DNS. So that's probably mostly "bad" bots
and/or a bit of test traffic.
Also, pi.berkeleylug.com is slightly, but not entirely redundant with
https://berkeleylug.com/Pi.BerkeleyLUG/
Notably pi.berkeleylug.com 302 redirects to the above ...
however also, there's account and sudo and dynamic DNS highly available
to pi.berkelelylug - so if/whenever they / that SIG wants it to go to
somewhere else in DNS, it's readily available for that. But if the
DNS & web traffic hits the BALUG host's web server, it 302 ("temporary")
redirects as noted above.
4467098 secure.balug.org
3408433 berkeleylug.com
2083824 www.balug.org
1692238 www.wiki.balug.org
998636 lists.balug.org
875806 www.sf-lug.org
185888 balug.org
47829 www.berkeleylug.com
42223 www.archive.balug.org
23561 www.sf-lug.com
17286 sf-lug.com
15402 sf-lug.org
14875 old-debian.balug.org
8993 sflug.org
5372 www.digitalwitness.org
5175 wiki.balug.org
5047 berkeleylug.org
4705 www.sflug.org
4295 sflug.com
4052 sflug.net
3841 www.ipv4.sf-lug.org
3406 www.sflug.net
3291 www.new.balug.org
3291 sf-lug.net
2911 digitalwitness.org
2819 www.test.balug.org
2519 www.ipv4.balug.org
2016 www.beta.balug.org
1789 www.sflug.com
1225 www.php.test.balug.org
1199 www.berkeleylug.org
1182 ipv4.sf-lug.org
1096 mx.balug.org
966 ipv4.balug.org
786 www.sf-lug.net
753 www.pi.berkeleylug.com
611 mx.lists.balug.org
596 pi.berkeleylug.com
266 www.ipv6.balug.org
262 www.ipv6.sf-lug.org
77 ipv6.balug.org
67 ipv6.sf-lug.org
51 bad.debian.net
25 www.buug.org
8 buug.org
And, nothing horribly surprising here, given the above. This is
essentially same again, but broken down by by port.
Also, if we total up all of http (:80) and https (:443) we get:
7263155 :80 (http)
6682636 :443 (https)
And by domain and port:
3570578 secure.balug.org:80
2942325 berkeleylug.com:443
1762326 www.balug.org:443
1355057 www.wiki.balug.org:80
896520 secure.balug.org:443
834027 www.sf-lug.org:80
616148 lists.balug.org:443
466108 berkeleylug.com:80
382488 lists.balug.org:80
337181 www.wiki.balug.org:443
321498 www.balug.org:80
140573 balug.org:80
45315 balug.org:443
43816 www.berkeleylug.com:80
41779 www.sf-lug.org:443
29510 www.archive.balug.org:80
22723 www.sf-lug.com:80
15730 sf-lug.com:80
14827 old-debian.balug.org:80
12713 www.archive.balug.org:443
11427 sf-lug.org:80
6618 sflug.org:80
4604 berkeleylug.org:80
4420 www.digitalwitness.org:80
4134 wiki.balug.org:80
4013 www.berkeleylug.com:443
3975 sf-lug.org:443
3548 sflug.com:80
3263 sflug.net:80
3251 www.sflug.org:80
2738 sf-lug.net:80
2485 digitalwitness.org:80
2462 www.test.balug.org:80
2394 www.ipv4.sf-lug.org:80
2375 sflug.org:443
2244 www.new.balug.org:80
1964 www.sflug.net:80
1767 www.beta.balug.org:80
1556 sf-lug.com:443
1454 www.sflug.org:443
1447 www.ipv4.sf-lug.org:443
1442 www.sflug.net:443
1409 www.sflug.com:80
1392 www.ipv4.balug.org:443
1127 www.ipv4.balug.org:80
1047 www.new.balug.org:443
1041 wiki.balug.org:443
975 www.php.test.balug.org:80
952 www.digitalwitness.org:443
879 ipv4.sf-lug.org:80
838 www.sf-lug.com:443
789 sflug.net:443
787 www.berkeleylug.org:80
747 sflug.com:443
736 ipv4.balug.org:80
718 www.pi.berkeleylug.com:80
599 mx.balug.org:80
553 sf-lug.net:443
497 mx.balug.org:443
463 www.sf-lug.net:80
443 berkeleylug.org:443
426 digitalwitness.org:443
412 www.berkeleylug.org:443
381 mx.lists.balug.org:80
380 www.sflug.com:443
376 pi.berkeleylug.com:80
357 www.test.balug.org:443
323 www.sf-lug.net:443
303 ipv4.sf-lug.org:443
250 www.php.test.balug.org:443
249 www.beta.balug.org:443
230 mx.lists.balug.org:443
230 ipv4.balug.org:443
220 pi.berkeleylug.com:443
194 www.ipv6.sf-lug.org:80
173 www.ipv6.balug.org:80
93 www.ipv6.balug.org:443
68 www.ipv6.sf-lug.org:443
67 ipv6.balug.org:443
60 ipv6.sf-lug.org:443
48 old-debian.balug.org:443
45 bad.debian.net:80
35 www.pi.berkeleylug.com:443
18 www.buug.org:80
10 ipv6.balug.org:80
7 www.buug.org:443
7 ipv6.sf-lug.org:80
6 bad.debian.net:443
4 buug.org:80
4 buug.org:443
Anyway, maybe some day I'll get some "real" web reporting in place.
It is on the todo list, but ...
$ wc todo
6156 22307 188878 todo
$
It's also not a FIFO list, nor FILO.
It's mostly a priority interrupt driven stack that gets a lot
of reordering applied to it, and tends to grow much more than it
shrinks. Well, at least I don't have to worry about running
out of stuff to do - have well over lifetime's worth 'o stuff on the
list.
More BALUG and [L]UG specific lists may be found on BALUG's wiki,
though they're not necessarily complete nor current.
Quoting Michael Paoli (Michael.Paoli(a)cal.berkeley.edu):
> I also restricted access of the list roster to the list admins - to prevent
> potential abuse (e.g. by spammers).
Uh-oh.
I think we need to talk about that, because I think it's an extremely bad
idea, and am delaying un-doing your change pending discussion primarily
in the name of the spirit of consultation among admins. I.e., I _do_
expect to revert that change soon, but wish to discuss the matter first.
(In the future, I'd actually respectfully suggest you check with the
other admins before making fundamental changes, rather than having us
merely react to them retrospectively. I hope you don't take that as any
form of personal criticism, as I highly respect you for showing
leadership. Very likely, you assumed this was a no-brainer improvement,
and I gladly acknowledge your benign intent.)
Making the roster accessible to listadmins only, as opposed to
subscribed members, really doesn't help the "I'm hiding my address from
spammers" people much, in the first place: Spammers can easily
programmatically "harvest" the addresses of anyone who's ever posted,
from the back-postings archives. Moreover, the "I'm hiding from
spammers" people already have a less-drastic remedy that doesn't injure
the transparency of the roster generally: Anyone who's _that_ concerned
about his/her address never being seen in public need only set the
"hidden" flag on his/her individual subscription.[1]
The harm done by setting the roster listadmin-accessible-only is
twofold: (1) It prevents people from seeing whom they're sharing the
list with, i.e., whom they're speaking to at the moment -- for no
compelling reason whatsoever. I really think this is an important
concern, and part of what it means to be a community -- to know _who_
(or at least what e-mail address) is participating with you, other than
those who've explicitly hidden their addresses.
(2) Using the listadmin-accessible-only setting creates an implication
of hypercontrol and creates the suspicion in the minds of many of us
mailing list old-timers that this is yet another forum run by control
freaks who like to "disappear" people they dislike while making sure
that, lacking access to the roster, they write to their fellow members
to protest the action. (You can rightly protest that we're not that
sort of admins. The point, however, is that's the impression such
settings naturally convey.)
I hope the above analysis doesn't come across as cranky or kneejerk:
I've been through this discussion many, many times over a period of
decades, and so please accept my apologies in advance if it seems harsh,
peremptory, or insufficiently well explained.
[1] People here who are subscribed to the main SVLUG mailing list, whose
roster is viewable by any subscribed party, can see the effect of that
flag's use at http://lists.svlug.org/lists/roster/svlug . For the benefit
of those who aren't, the roster starts out with this header data:
436 Non-digested Members of svlug: 166 Digested Members of svlug:
(11 private members not shown) (4 private members not shown)
[list follows below that] [list follows below that]
The "11 private members" and "4 private members" are those who've set
the "hidden" flag on their individual subscriptions.
The SVLUG list's listinfo page includes this advisory in strong-tagged
text, to further warn in advance any I'm-hiding-from-spammers people:
Our public message archives display unobscured posting addresses. If
you're trying to hide your e-mail address from spammers, do not post
from that address.