1. On https://temp.balug.org/cgi-bin/mailman/admin/balug-test/general ,
added myself to lisadmin roster.
Changed max_days_to_hold from 0 to 5. Rationale: Makes held spam
expire out with reasonable speed, while still allowing for listadmin
reaction time. (One of my guiding principles of Mailman administration
is that if you find yourself trying to outwork mailbots, you're doing it
wrong. Along those lines, if you're repeatedly visiting the admin queue
to clear it even though you know from the held-mail summaries that it's
100% spam, then you're trying to outwork mailbots.) I personally lead
towards 3 rather than 5, sufficient to notice something on a holiday
weekend. (On 'conspire', I actually have this as '2'.)
Corrected subject_prefix from '[Balug-test]' to '[balug-test]'.
Noted without particular comment (except 'OK with me'): You've
seemingly toggled send_reminders (monthly password reminder) to 'no'.
Last I saw, Mailman installation default is 'yes'.
Changed 'respond_to_post_requests' aka 'Send mail to poster when their
posting is held for approval?' to 'no'.
2. On https://temp.balug.org/cgi-bin/mailman/admin/balug-test/privacy/sender :
Changed 'dmarc_moderation_action' from 'Accept' to 'Munge From'. This
is a huge topic where mailing list are collateral damage in the war
against SMTP forgery, and where listadmins have only sucky options to
choose among. We can discuss this further.
(The committee of jackbooted fascists who designed DMARC either hated
mailing lists or simply didn't give a damn about them.)
[posting to more fitting BALUG-Admin]
> From: "Rick Moen" <rick(a)linuxmafia.com>
> To: balug-test(a)temp.balug.org
> Subject: Re: [BALUG-Test] Oooh, ... archive tests ... :-)
> Date: Wed, 20 Sep 2017 15:18:59 -0700
> Quoting aaronco36 (aaronco36(a)SDF.ORG):
>
>> Am wondering what the TL;DR of all this is, i.e., the summarized
>> outcome and current status of all Michael P's efforts here?
>
> I think he means he successfully used balug-test as a test case (pun
> semi-intended) for import of mail to the mailing list's mbox and
> generation of the archive. This was to gain confidence that doing
> likewise to the main mailing lists has a good plan.
>
> For balug-talk in particular (maybe some others), Michael has in mind to
> attempt to assemble a more-complete history, adding posting salvaged
> from other sources such as subscribers' saved mail, and merging that
> into the mailing list's mbox as a basis for running
> $MAILMAN_HOME/bin/arch to make a new pipermail archive. Seems
> worthwhile, but doubtless it'll take some time, and he might have to
> bodge headers on some of the saved mail to make it work. (You ask
> subscribers whether they have old postings in mbox format, and in my
> experience they send you the damnedest random formats, disregarding the
> qualifier about 'mbox' because they didn't understand it and weren't
> smart enough to ask. I experienced this when sf-lug mailing list
> subscribers supplied backfill material for that archive. A lot of what
> I got was not at all what I asked.)
>
> Sufficent scripting-fu can, in my experience, reassemble an mbox that
> $MAILMAN_HOME/bin/arch will parse correctly. Just make sure that you
> keep a safety copy of the most recent mbox _verified_ to parse right,
> so you don't burn your bridges, only your spare time.
>
> I believe backfilling the archives is the only task remaining.
> (Dreamhost "oops"-discarded quite a lot, over the years.)
So, yes, DreamHost.com lost a lot of BALUG's list archives over the years.
:-(
Fortunately I *do* have much of that ... but alas, most all of that that
I have ... mostly notably started saving regularly after DreamHost started
messing up - and quite repeatedly ... is in "cooked" format - as
mailman serves it up on the web, and alas, too, with @ --> " at " munging.
So, ... let's see ...
$ hostname && pwd -P && id
balug-sf-lug-v2.balug.org
/home/balug/e-mail_lists
uid=10582(balug) gid=10582(balug) groups=10582(balug)
$ grep . */archive_date_ranges
balug-admin/archive_date_ranges:2005-03-18--2013-05-24,2014-01-11--2015-01-31,2015-05-01--2015-11-30,2016-02-18--
balug-announce/archive_date_ranges:2001-06-15--2013-07-12,2013-11-18--2014-09-30,2014-11-14--2015-01-31,2015-04-20--2015-11-30--2015-12-15,2016-02-15--
balug-talk/archive_date_ranges:2001-06-15--2013-07-13,2013-11-09--2014-10-19,2014-10-22--2015-01-31,2015-04-06--2015-12-05,2016-01-23--
$
So, ... see the (--) ranges? Those are the "cooked" ranges I have.
See the ,'s between the ranges? Those are DreamHost.com's repeated
f*ck ups[1].
$ du -sh balug-*/archive/
832K balug-admin/archive/
664K balug-announce/archive/
3.3M balug-talk/archive/
$
And that's almost entirely gzip compressed stuff ... so quite a bit 'o
material ... but also with some significant gaps.
Anyway, not top of the priorities, but now also much more feasible
to merge that back into archives under mailman ... without the
encumberances of DreamHost being in the way.
I do also have some partial archives stuff from pre-DreamHost too.
But again, that's in "cooked" format (sucked from archive.org), and has
some gaps too (notably from when archive.org last grabbed that, and
when those lists went bye-bye and got replaced with new lists started
from scratch under DreamHost.com hosting).
Anyway, (more) stuff to do (archivist) among many BALUG tasks on the
quite long todo list ... for the volunteer(s) to do in their
copious <cough, cough> free time.
Oh, ... been a while since he mentioned it, but I have heard from
Jim Stockford, that he has "all his old mail" - notably from earlier
list traffic, etc. ... at least somewhere on some drive(s). If I
ever get access to that - or can have Jim run relevant programs against
such ... might be able to well get all or most all our list messages,
and may also be able to much better unmunge messages where we may only
have munged versions at least thus far.
Can also put out more calls/mentions/requests for email
archivists/hoarders ... if we can get the data (to pull from, or
hand over programs to extract the BALUG stuff, and then get that),
then we could better fill in and complete our archives. In the meantime,
... gaps. :-/
references/footnotes:
1. More references on DreamHost's repeated f*ck ups, not a complete list,
but ... the support reference identifiers ... at least from when I started
explicitly tracking them ... until I totally gave up on that as being
quite futile:
$ cat balug/dreamhost_f\*ckups
#7395573 list archives broken again, please fix ASAP
DreamHost Support Ticket #6693834 (backup request)
DreamHost Support Ticket #6693826
DreamHost Support Ticket #6614786
DreamHost Support Ticket #6532542
DreamHost Support Ticket #5972304
DreamHost Support Ticket #5933596
DreamHost Support Ticket #5865872
DreamHost Support Ticket #5855851
[micpao 98044886]
[micpao 97906756]
[micpao 95718194]
[micpao 95607623]
[micpao 95543944]
[micpao 95510775]
[micpao 82295833]
[micpao 80000975]
[micpao 79685471]
[micpao 78726933]
[micpao 77934213]
[micpao 77127808]
[micpao 77039262]
[micpao 75952335]
[micpao 75625344]
[micpao 75481168]
[micpao 75473081]
BALUG.org DNS,
For anyone that might notice, and might be wondering ...
$ dig +noall +answer +multiline balug.org. SOA | fgrep serial
3909849755 ; serial
$
... WFT scheme is *that*? ;->
Ah, ... the master zone file comments make that much more clear:
# fgrep -i serial balug.org
3909849755; SERIAL ; DO NOT EXCEED
4164575247 BEFORE (3W EXPIRY + 1 day) 1507640615
2017-10-10T06:03:35-0700 2017-10-10T13:03:35+0000 ; rolling SERIAL
from 2017091600 date +'%Y%m%d00' to date +%s (absolute max is
4294967295 2^32-1)
#
So yes, I know, YYYYMMDDNN is "preferred" per RFC(s) (more human readable),
but it's by no means the only permissible scheme. For various reasons,
I mostly much prefer seconds since ephoch - it's likely runner-up in
most commonly supported/used schemes, possibly excepting simple
monotonically increasing, and (by sheer volume) whatever the heck
AWS is doing (Route 53 always giving serial of 0).
Anyway, bit beyond 3 weeks from now, and should then be converged to
seconds since epoch ... in the meantime, something in the
range between 3909849755 and 4164575247.
Also, delegated subdomains - those:
{beta,ipv6,new,temp,test}.balug.org.
have now been rolled up into the balug.org. zone - possibly
notwihstanding any relevant still pending cached TTL expirations
(for the roll-up, excepting NS and SOA, and possibly some TTLs, the data
is thus far same - just has been rolled up to balug.org zone).
Nearly ready to start the major DNS changes to kick off last stages of
migration of BALUG.org away from DreamHost.com hosting.
This should mostly be highly seamless except ...
for the domains themselves:
www.balug.orgbalug.orglists.balug.org
There may be some minor issues/discrepancies apparent, as for a while,
(up to about 24 hours, due to DNS TTLs) those DNS names may go to either
the old legacy DreamHost.com hosting, or the new "hosting" (self-hosted
VM).
There are also a fair number of DreamHost.com hosting specific DNS names
that will be going away or changing - but those changes should
effectively pass without notice or detectable impact (e.g.:
www.mailboxes.balug.org.
ftp.balug.org.
mysql.balug.org.
etc.)
Note also that at the present time,
lists.balug.org. is (temporarily) deprecated - so don't expect
it to work yet (but it will be revived to full functionality again
in near future, as it will replace temp.balug.org which will be
phased out over suitable adjustment period).
If you notice any more significant issues, certainly feel free to
contact me.
Quoting Michael Paoli (Michael.Paoli(a)cal.berkeley.edu):
> Thanks!
>
> Looks like DreamHost assembled and made the files available by
> approximately 2017-09-16T16:10:15-0700.
> I've successfully retrieved 'em and run some basic sanity checks -
> looks like they are in fact what we're expecting from DreamHost
> (at least of what they still have and hadn't otherwise lost earlier).
>
> I'll update when clear to "pull the plug" on DreamHost (notably after
> I complete a mostly queued set of final migration config changes,
> and then the last of the longest relevant DNS TTLs have expired).
> Anyway - fairly soon (guestimating "all clear" by about
> mid-week +- a bit - will update when it's all done 'n clear).
I've just now tried to do some remedial looking around for third-party
utilities to deal with problems like mbox files needing cleanup.
I've been ffeeling some sense of loss, in that I'm _sure_ Marc Merlin
pointed me to a bunch of such tools years ago, but I failed to capture
that knowledge.
But not quite so fast. Lookie here:
rick@linuxmafia]
/usr/local/src/rickstuff $ cd svlug
[rick@linuxmafia]
/usr/local/src/rickstuff/svlug $ ls -al
total 27516
drwxrwsr-x 10 rick rick 4096 Sep 3 15:55 .
drwsr-sr-x 27 rick rick 4096 Mar 25 01:38 ..
drwxr-sr-x 3 root rick 4096 May 28 2015 etc
-rw-r--r-- 1 rick rick 5880 Jun 22 2009 marc-merlin-mailman-scripts.tar.gz
drwxrwsr-x 2 469 469 4096 Mar 16 2012 mboxes
-rw-r--r-- 1 rick rick 27614753 Sep 3 15:55 svlug-rosters
-rw------- 1 rick rick 27960 Jan 5 2016 svlug-rosters~.
-r-xr-x--- 1 rick rick 3590 Jun 22 2009 svlug.svlug.org-backup-scripts.tar.gz
-r-xr-x--- 1 rick rick 365822 Jun 22 2009 svlug.svlug.org-usr.local.sbin.tar.gz
-r-xr-x--- 1 rick rick 58947 Jul 19 2009 svlug.svlug.org-var.lib.scr.tar.gz
-r-xr-x--- 1 rick rick 5154 Jun 22 2009 svlug.svlug.org-var.www.bin.tar.gz
drwxr-sr-x 2 root rick 4096 May 28 2015 usr-local-sbin
drwxr-sr-x 3 root rick 4096 May 28 2015 var-local-mailman-backup
drwxr-sr-x 3 root rick 4096 May 28 2015 var-local-mailman-bin
drwxr-sr-x 2 root rick 4096 May 28 2015 var-local-mailman-cron
drwxr-sr-x 3 root rick 4096 May 28 2015 var-local-scr
drwxrwsr-x 2 rick rick 4096 Apr 30 2016 www
[rick@linuxmafia]
/usr/local/src/rickstuff/svlug $
Oh, joy. Looks like I did something useful in the way of looking ahead
to situations just like this one.
/usr/local/src/rickstuff/svlug $ cp marc-merlin-mailman-scripts.tar.gz /tmp
[rick@linuxmafia]
/usr/local/src/rickstuff/svlug $ cd /tmp
[rick@linuxmafia]
/tmp $ tar xvzf marc-merlin-mailman-scripts.tar.gz
marc-merlin-mailman-scripts/
marc-merlin-mailman-scripts/addusertolists
marc-merlin-mailman-scripts/checkalllists
marc-merlin-mailman-scripts/dumplistconfigs
marc-merlin-mailman-scripts/excludelists
marc-merlin-mailman-scripts/findemail
marc-merlin-mailman-scripts/findemailpattern
marc-merlin-mailman-scripts/findvauser
marc-merlin-mailman-scripts/listlists
marc-merlin-mailman-scripts/listsoutside
marc-merlin-mailman-scripts/mailman_force_settings
marc-merlin-mailman-scripts/mailman_setowner_settings
marc-merlin-mailman-scripts/README
marc-merlin-mailman-scripts/reconfigalllistsfromdump
marc-merlin-mailman-scripts/removevauser
marc-merlin-mailman-scripts/renameuser
marc-merlin-mailman-scripts/renamevauser
marc-merlin-mailman-scripts/resetlistconfigs
marc-merlin-mailman-scripts/savelistinfo
marc-merlin-mailman-scripts/listsconfig/
marc-merlin-mailman-scripts/crontab
[rick@linuxmafia]
/tmp $
Lots of good tricks and script snippets in there. Here's one I loved
when I discovered it. (This is all stuff captured from lists.svlug.org,
by the way.)
The crontab file you see in there is the main system crontab file from
lists.svlug.org, and it includes:
14 5 * * 1-5 root bash -c 'export IFS=" "; A=`/var/local/mailman/bin/scripts/resetlistconfigs 2>&1`; if [ z"$A" != z ]; then echo $A | Mail -s "~mailman/bin/scripts/resetlistconfigs output" owner-mailman; fi'
And what does resetlistconfigs do?
/tmp/marc-merlin-mailman-scripts $ cat resetlistconfigs
#!/bin/bash
cd ~mailman/lists/
for i in *
do
~mailman/bin/config_list -o - $i | tail +3 > /var/tmp/tmp_mm_settings.before
~mailman/bin/config_list -i ~mailman/bin/scripts/mailman_force_settings $i
~mailman/bin/config_list -o - $i | tail +3 > /var/tmp/tmp_mm_settings.after
if ! diff -q /var/tmp/tmp_mm_settings.before /var/tmp/tmp_mm_settings.after &>/dev/null; then
echo "Fixing config of list $i"
diff -u0 /var/tmp/tmp_mm_settings.before /var/tmp/tmp_mm_settings.after
fi
done
[rick@linuxmafia]
/tmp/marc-merlin-mailman-scripts $
You get the gist of that. It parse text file mailman_force_settings as
input to the config_list utility. And the input file is thus the punch line:
[rick@linuxmafia]
/tmp/marc-merlin-mailman-scripts $ cat mailman_force_settings
reply_goes_to_list = 0
host_name = 'lists.svlug.org'
archive = 1
[rick@linuxmafia]
/tmp/marc-merlin-mailman-scripts $
Marc suspected well in advance that, eventually, SVLUG would put into
a position to do harm some officer or other volunteer who'd insist on
doing stupid things to the Mailman settings like force Reply-To munging,
or attempt to make lists.svlug.org forge some other FQDN, or have a
mailing list deliberately non-archived, so Marc set up a nightly cron job
to un-do the idiot's damage.
The idiot eventually did arrive in the person of raving loon and drunk
President Paul Reiber, who indeed did attempt to force Reply-To munging
and was reputedly driven to distraction by the fact that his change
kept reverting automatically shortly after he made it.
I'd be glad to give you a copy of the marc-merlin-mailman-scripts.tar.gz
collection, but the _real_ prize I wanted to give you is cleanarch,
a python script Marc had in /var/local/mailman/bin . File comments:
"""Clean up an .mbox archive file.
The archiver looks for Unix-From lines separating messages in an mbox archive
file. For compatibility, it specifically looks for lines that start with
"From " -- i.e. the letters capital-F, lowercase-r, o, m, space, ignoring
everything else on the line.
Normally, any lines that start "From " in the body of a message should be
escaped such that a > character is actually the first on a line. It is
possible though that body lines are not actually escaped. This script
attempts to fix these by doing a stricter test of the Unix-From lines. Any
lines that start "From " but do not pass this stricter test are escaped with a
> character.
Usage: cleanarch [options] < inputfile > outputfile
Options:
-s n
--status=n
Print a # character every n lines processed
-q / --quiet
Don't print changed line information to standard error.
-n / --dry-run
Don't actually output anything.
-h / --help
Print this message and exit
"""
You need this. I'm attaching a copy (GPLed by author).
There's upcoming scheduled sites outage due to some
scheduled electric utility work.
Scheduled outage window:
> =2017-09-26T09:00:00-0700-->~=2017-09-26T15:00:00-0700
sites impacted:
[*.]balug.org (note that it may or may not impact www.balug.org and
balug.org - depending where they are at that time, but will impact at
least all other *.balug.org sites/domains)
[*.]sf-lug.{org,com} (note that SF-LUG list isn't impacted)
To the extent feasible, I'll try to minimize the outage time, but
expect it may minimally be up to fair part of an hour, and
"worst case" might be 6 hours or more (notably also if I can
have things automagically recover easily enough unattended upon power
restoration - or not).
It's also possible this outage may be delayed or rescheduled (notably
depending what electric utility provider gets up to).
Status/tracking - DreamHost.com exodus
FYI, added a wiki-page to track at least high-level steps & status:
https://www.wiki.balug.org/wiki/doku.php?id=balug:dreamhost_exodus
I'm expecting to be fully off DreamHost.com by end of this month,
at least if all goes sufficiently and reasonably well.
Mostly just takes some available time for me to crank through the
remaining steps (and one step - or set of steps, for DreamHost.com to do).
Highly optimistically I'd say we'd be done by mid-month ... but more
realistically, within 7 to 10 days ... so ... that'd be by
2017-08-22. Let's see if we can make/beat that target! :-)
[adding balug-admin]
Quoting Glen Martin (glen(a)glen-martin.com):
> I expect the alterations are the insertion of a list footer in the
> message text. I expect similar insertion in the Subject header to be
> problematic as well, though this DMARC didn't complain of that.
My understanding is that the sending domain's DKIM policy (portion of
DMARC) declares which headers and text portions are attested by
cryptographic signing. Failing the DKIM aspect of DMARC means the
forwarder host (in this case the MLM host) has altered or inserted into
one of the signed areas of the message, and not stripped the DKIM
headers. (Some listadmins strip such headers as a way of avoiding the
perception of DMARC failure upon retransmission. I'm not sure this is
wise. Seems like there might be adverse consequences.)
> Here's a header from one of my own messages to the list, having come
> back to me through the list and evaluated using my own inbound
> testers (opendmarc and opendkim). The two "Authentication-results:"
> headers are the problem. For help understanding the headers, I'll
> point out that my primary domain on this MX is locutory.org, hence
> that name in the headers.
Downthread in that offlist conversation with Michael, I notice you said:
However, glen-martin.com publishes authorized sending hosts (in SPF),
and temp.balug.org (or any other balug.org) isn't on that list. Strike 1.
To the best of my understanding, MLM retransmission of messages to
subscribers is _not_ going to violate any sender's SPF policy, because
the retransmitted message to each subscriber bears a new, different envelope
citing the MLM's domain. Hence, on that retransmission, the relevant
SPF record that would be consulted upon arrival at the subscriber's MTA
is not the poster's domain but rather the MLM's (balug.org's).
The example test mail to balug-talk whose full headers you sent to
Michael had, as received by you as subscriber:
Received-SPF: pass (glen-martin.com: 50.196.148.122 is authorized to
use 'glen(a)glen-martin.com' in 'mfrom' identity (mechanism 'ip4:50.196.148.122'
matched) (glen-martin.com: 50.196.148.122 is authorized to use
'glen(a)glen-martin.com' in 'mfrom' identity (mechanism
'ip4:50.196.148.122' matched)))
If I'm reading that right, temp.balug.org's (IP 198.144.194.238's) MTA
software inserted that when it received your original posting, and
temp.balug.org considered all to be well at that point, as
50.196.148.122 was one of the declared authorised senders in your
domain's SPF RR.
temp.balug.org then turned around and retransmitted a fresh copy to each
balug-talk subscriber including, of course, you. It bore a totally new
SMTP envelope, for which the envelope sender was:
Return-Path: <balug-talk-bounces(a)temp.balug.org>
So, your receiving MTA, getting the subscriber copy, would have sought
to vet transmitting MTA IP 198.144.194.238 (temp.balug.org) against the
SPF RR not of _your_ domain (for that copy), but rather of domain
balug.org.
People get really confused about SPF and mailing list, I notice,
forgetting that the MLM's forward is a separate message with a different
SMTP envelope (from a different domain).
Let's look at the balug.org SPF RR:
:r! dig -t txt balug.org +short
[null return value]
Well, Michael, time to put an SPF RR in the balug.org DNS.
This one works for my domain:
:r! dig -t txt linuxmafia.com +short
"v=spf1 a mx -all"
Any questions, please just ask. It says 'use version 1 of SPF protocol,
consider the received mail legitimate if it arrives from an IP
corresponding with the linuxmafia.com A or MX RR, and please hardfail
as a forgery any message purporting to come from linuxmafia.com arriving
from any _other_ IP.'
> Authentication-Results: mail2.locutory.org (amavisd-new);
> dkim=fail (1024-bit key) reason="fail (body has been altered)"
> header.d=glen-martin.com
Well, could you please compare the message body as you sent it with the
message body as you received it in the retransmitted subscriber copy,
and tell us what you signed that got changed?
What I'm saying is that it's unclear on this end what is the scope of
your domain's DKIM cryptographic scrutiny. OK, the 'body has been
altered', but what specifically does that mean? Is your domain's DKIM
policy so twitchy that Mailman merely adding a Mailman footer breaks it?
Does Mailman adding List-Id, List-Subscribe, List-Help, and
List-Unsubscribe headers break it? You define and control that DKIM
policy for your domain, so perhaps you can tell us.
> Balug is also inserting headers/footers into Subject and Body, so the DKIM
> checksum fails (message is modified). Strike 2.
You cannot expect Mailman mailing lists to _not_ add additional SMTP
headers, add a footer, and insert a Subject header tag (like
'[BALUG-Talk]'), so it seems to me your DKIM policy ought _not_ to be set to
fail if such things are changed by standards-compliant forwarders such as
MLMs. Those are things that mailing lists _must_ do in some cases, and
traditionally do for excellent reasons (like adding a footer) in others.
If you expect mailing lists to not do normal mailing list things, then I
submit that IMO your domain's DKIM policy is broken and urgently needs
revision.
[and ... back on list ...]
> From: "Rick Moen" <rick(a)linuxmafia.com>
> Date: Thu, 17 Aug 2017 06:07:40 -0700
> Quoting Michael Paoli (Michael.Paoli(a)cal.berkeley.edu):
>
>> Rick,
>>
>> Please feel most graciously invited to, at your convenience (no
>> rush), add yourself as listadmin to BALUG's BALUG-{Announce,Talk}
>> list, if/presuming you still wish to do so.
>
> On the temp.balug.org host, I had already added myself as listadmin to:
> balug-admin
> balug-test
>
> I've just added myself to:
> BALUG-Talk
> BALUG-Announce
Much thanks! :-)
> 1. How do you feel about lowercasing the real_name and subject_prefix
> settings for those? I propose yes.
Since BALUG is acronym, rather than name, I prefer to keep it
with uppercase for that portion ... and the portion right after
the hyphen (-), and particularly in that context, probably makes
reasonable sense / better fit, with an initial cap there ... and
so it is, also has been that way a long time.
And yes, certainly debatable ... visibility vs. annoyance, how to
(not) handle an acronym, etc.
> 2. Just a note: The merger of the old mbox contents from Dreamhost and
> (potentially) other sources of backfill will create a one-time
> opportunity to select, if we wish, a different archive_volume_frequency
> setting than Monthly (Archiving Options page). BALUG has always used
> Monthly, but only because that's the Mailman default rather than as a
> conscious choice.
>
> For announce and test mailing lists, I recommend Yearly.
>
> For active discussion mailing lists like balug-talk, it's more arguable.
> Sometimes Quarterly is nice, sometimes Monthly.
>
> balug-admin is sufficiently low-traffic that I suggest Yearly.
>
> Under normal circumstances, there's a strong disincentive against
> changing, in that you end up assigning new URLs to existing archived
> postings, which breaks offsite links to the significant ones (if any
> ;-> ). But if you're merging stuff in, you're going to break most or
> all of the existing message URLs anyway.
Yes, good considerations. Unfortunately the logic runs contrary to
some of the list data. Notably BALUG-Announce and BALUG-Admin (I'll
have to review the data again, though, this is off-the-top-of-my-head)
have been the least screwed up by DreamHost.com - notably we may have
*all* the messages from those lists (or nearly so), and hence it would
be good to preserve backwards compatibility on the URLs. I do already
have the redirects in place for that - so as long as the sequence
numbers line up, the old URLs will redirect to the same
messages ... at least after reinjecting those messages, and after
lists.balug.org. has been moved over to the new host location.
In the meantime, for temp.balug.org, robots.txt tells the search
engines to *not* index the archives (less they get confused as numbers
will shift relative to what's presently on temp.balug.org), however
messages will be injected before lists.balug.org. is on this host
where the active lists are now ... and the robots.txt for
lists.balug.org on new host location, very much permits indexing
by search engines of all the archives.
And BALUG-Talk is the one DreamHost.com most thoroughly screwed up.
Do have much of that (at least in non-ideal form, but at least have
it) to be reinjected, and should soon have raw of the more recent
of that ... so, since BALUG-Talk is the one where the numbers and
alignment of messages are most likely to shift, as I/we work to try
and get as much of the missing messages reinjected, that would be
top candidate for potentially changing archive frequency. But it's
highest volume, so ought probably stay at monthly anyway. And
the other lists, lower volume, so by that quarter or yearly would
be good, but as those list (at least I think?) are mostly not missing
messages, changing that would break all the (individual message) linkage.
So, alas volume vs. archive frequency vs. clean and not-so-clean
reinjections and archive fixes to be done, are relatively
anti-correlated. :-/
So ... taken all together, I'm relatively inclined to go and
remain with monthly.
Now ... BALUG-Test on the other hand :-) - that one is a prime candidate
to change. I'd guestimate it's activity will be fairish bursts of
activity, but otherwise mostly a lot of nothing or nearly so. So,
quarterly, or even yearly, probably makes perfectly fine sense there.
Should decide before lists.balug.org. moves over. But other than that
I think either sounds fine for BALUG-Test. What say ye?
Preference/recommendation on BALUG-Test? I'm totally open on that one.
And, e.g., on the redirects:
Presently have, e.g.:
$ curl -I
http://lists.balug.org/pipermail/balug-admin-balug.org/2016-April/003013.ht…
HTTP/1.1 200 OK
Date: Fri, 18 Aug 2017 11:47:02 GMT
Server: Apache
Last-Modified: Mon, 18 Apr 2016 16:37:20 GMT
ETag: "10f5-530c4f972ce72"
Accept-Ranges: bytes
Content-Length: 4341
Vary: Accept-Encoding
Content-Type: text/html
Will later have for lists.balug.org (but already in place on
temp.balug.org):
$ curl -I
https://temp.balug.org/pipermail/balug-admin-balug.org/2016-April/003013.ht…
HTTP/1.1 301 Moved Permanently
Date: Fri, 18 Aug 2017 11:48:28 GMT
Server: Apache/2.4.10 (Debian)
Location: https://temp.balug.org/pipermail/balug-admin/2016-April/003013.html
Content-Type: text/html; charset=iso-8859-1
There's no there there yet, on the redirect target ... but will be after
reinjection of archives, and it will be so on lists.balug.org after
that's moved over.
There's somewhat different scheme on the paths for Debian, than
is set up on DreamHost.com. I did also note the redirect bits
on:
https://www.wiki.balug.org/wiki/doku.php?id=balug:mail_and_lists
(look for the strings redirect and/or rewrite - case insensitive,
or much of that is currently close to the bottom of that web page)
And to keep search engines reasonably happy (and optimize some other
bits too), generally better to have *one* canonical location for the
(same) content, and (permanent, e.g. 301) redirect to that location,
rather than have the exact same content at multiple paths and/or
hostnames (search engines tend to "dilute" and down-rank such, at
least some fair bit, whereas content at one unique location
tends to, in general rank better ... various exceptions, but
that's at least approximately the general case).