[Bcc: BALUG-Test list] So ... BALUG-Test list ... not an official announcement ... yet, but, it's up and running, and mostly functional ... with some noteworthy caveats/issues still to be addressed (and probably some I've not yet discovered).
Policy, etc. - haven't really written that up yet, but shall. In the meantime, it goes approximately as follows: Fairly similar to BALUG-Talk - generally need to be subscribed to post, no job postings (no reason to have exceptions on that on the Test list) It *is* a test list, so some things are and generally will remain different: Recommended convention: if you *do not* want a response to your posting to the list, include both of these two strings: test ignore in your Subject: header and they can be upper and/or lower case (testing should be done case insensitive). If you don't include both test and ignore, you may get human and/or automated responses, e.g. received your posting fine at ... and here's full headers as received. No guarantees you'll get such response(s), but if you don't include test and ignore, you may get such response(s).
The list may wink in and out of existence, but for the most part will be persistent, have preserved archives, etc. Archives may be reloaded and/or altered (posts added/dropped) - notably for testing - but for the most part the archives will generally contain what was posted to the list.
Anyway, that's at least approximation of the policy (will write it up more later, and have it on the list page).
Caveats - at present the archives are effectively broken. It's a web server configuration issue - they're otherwise fine and being archived (and it's not filesystem or file permissions or the like). I'll get that fixed (one of the relatively higher priorities).
At present, emails with an empty body are being rejected. I still need to fix that (as legitimate mailman commands may have command in Subject header and have empty body).
The mail server has a whole lot 'o anti-spam stuff on it. Fairly likely some of that may be too aggressive and/or annoying. I like how it's rejecting tons 'o cr*p spam :-) ... but I want to keep the nuisance factor sufficiently low on legitimate email. It will probably need some adjusting. If you get a "hard" / "permanent" failure (or the mail server automagically blacklist or bans you on what should be legitimate email), shoot me an email note: Michael.Paoli@cal.berkeley.edu - and include full headers and the complete bounce message contents. Some of the greylisting and flood protection may be a bit annoying - especially on first messages. E.g. a legitimate email doesn't necessarily get an "instant" response/acceptance - I've tuned some of that down so initial delays are less annoying (e.g. dropped to 2 minutes) while still effectively thwarting spammers. Anyway, much of that may yet be subject to further adjustments, etc., as reasonably appropriate. There are also a lot of block lists that are used ... at least some of those may be a bit too aggressive in their blocking and block some legitimate email. Anyway, let me know where issues are encountered.
Also, the site is neither high bandwidth nor high availability. So, sometimes it may not be available ... but for the most part it generally should be up and available.
So, where is it, how to get started? List page, here: https://temp.balug.org/cgi-bin/mailman/listinfo/balug-test It, and the other BALUG lists, will temporarily migrate from lists.balug.org to temp.balug.org - and later settle back on lists.balug.org (part of our migration away from DreamHost.com). BALUG-Test list first, then BALUG-Admin, then BALUG-Talk, and last, BALUG-Announce.
And how can you tell if your post was successful without being able to see the archives? Looks like with the default: Receive your own posts to the list? Yes One should be able to tell/distinguish by headers, etc., in received posting. Looks like also there's default: Avoid duplicate copies of messages? Yes One might possibly want to change to No to help ensure all copies are received.
And ... well, not exactly step-by-step, but tracking fair bit of migration and testing here: https://www.wiki.balug.org/wiki/doku.php?id=balug:mail_and_lists (it's not fully current ... but not that far from current either)
Quoting Michael Paoli (Michael.Paoli@cal.berkeley.edu):
Caveats - at present the archives are effectively broken. It's a web server configuration issue - they're otherwise fine and being archived (and it's not filesystem or file permissions or the like). I'll get that fixed (one of the relatively higher priorities).
You might want to check ownership/permissions in /var/lib/mailman/archives/private and /var/lib/mailman/archives/public .
Examples on linuxmafia.com:
linuxmafia:/var/lib/mailman/archives# ls -l private/ total 38 drwxrwsr-x 183 list list 13312 Jul 2 03:27 conspire drwxrwsr-x 2 list list 1024 Apr 16 15:53 conspire.mbox drwxrwsr-x 38 list list 3072 Apr 18 03:27 dvlug drwxrwsr-x 2 list list 1024 Jun 8 2009 dvlug.mbox drwxrwsr-x 12 list list 1024 Apr 5 03:27 friday-follies drwxrwsr-x 2 list list 1024 Sep 27 2010 friday-follies.mbox drwxrwsr-x 21 list list 1024 Jan 2 2017 linuxcare-alumni drwxrwsr-x 2 list list 1024 Dec 1 1999 linuxcare-alumni.mbox drwxrwsr-x 2 list list 1024 May 7 2003 mailman drwxrwsr-x 2 list list 1024 May 7 2003 mailman.mbox drwxrwsr-x 13 list list 1024 Apr 15 03:27 revival drwxrwsr-x 2 list list 1024 Dec 3 2007 revival.mbox drwxrwsr-x 52 list list 3072 Jul 3 03:27 sf-lug drwxrwsr-x 2 list list 1024 Dec 29 2016 sf-lug.mbox drwxrwsr-x 25 list list 2048 Jul 1 03:27 skeptic drwxr-sr-x 2 list list 1024 Mar 22 21:09 skeptic.mbox drwxrwsr-x 14 list list 1024 Oct 5 2016 test drwxrwsr-x 2 list list 1024 Jun 17 2010 test.mbox drwxrwsr-x 13 list list 1024 Apr 2 2006 weaselware drwxrwsr-x 2 list list 1024 May 13 2003 weaselware.mbox linuxmafia:/var/lib/mailman/archives# ls -l public/ total 0 lrwxrwxrwx 1 list list 52 Apr 3 2009 balug-talk-archive -> /var/lib/mailman/archives/private/balug-talk-archive lrwxrwxrwx 1 list list 57 Apr 3 2009 balug-talk-archive.mbox -> /var/lib/mailman/archives/private/balug-talk-archive.mbox lrwxrwxrwx 1 list list 42 Apr 3 2009 conspire -> /var/lib/mailman/archives/private/conspire lrwxrwxrwx 1 list list 47 Apr 3 2009 conspire.mbox -> /var/lib/mailman/archives/private/conspire.mbox lrwxrwxrwx 1 list list 39 Apr 3 2009 dvlug -> /var/lib/mailman/archives/private/dvlug lrwxrwxrwx 1 list list 44 Apr 3 2009 dvlug.mbox -> /var/lib/mailman/archives/private/dvlug.mbox lrwxrwxrwx 1 www-data list 48 May 19 2016 friday-follies -> /var/lib/mailman/archives/private/friday-follies lrwxrwxrwx 1 list list 44 Apr 3 2009 lg_mirrors -> /var/lib/mailman/archives/private/lg_mirrors lrwxrwxrwx 1 list list 48 Apr 3 2009 lg_translators -> /var/lib/mailman/archives/private/lg_translators lrwxrwxrwx 1 list list 41 Apr 3 2009 mailman -> /var/lib/mailman/archives/private/mailman lrwxrwxrwx 1 list list 46 Apr 3 2009 mailman.mbox -> /var/lib/mailman/archives/private/mailman.mbox lrwxrwxrwx 1 list list 40 Apr 3 2009 sf-lug -> /var/lib/mailman/archives/private/sf-lug lrwxrwxrwx 1 list list 45 Apr 3 2009 sf-lug.mbox -> /var/lib/mailman/archives/private/sf-lug.mbox lrwxrwxrwx 1 list list 38 Apr 3 2009 test -> /var/lib/mailman/archives/private/test lrwxrwxrwx 1 list list 43 Apr 3 2009 test.mbox -> /var/lib/mailman/archives/private/test.mbox linuxmafia:/var/lib/mailman/archives#
The mail server has a whole lot 'o anti-spam stuff on it.
Mad props!
And how can you tell if your post was successful without being able to see the archives?
It's a pity that GNU Mailman never added to the e-mail command interface (the -request address) the ability to fetch mail batches. For example, with Sympa, you can request that the command interface e-mail you a month's worth of old archived posts in gzipped mbox format.
Here are docs for the commands you can send to the e-mail command interface: http://www.list.org/mailman-member/node41.html#a:commands
That should help with one of your to-do items:
mailman commands should work via email: subscribe/unsubscribe/help (need more complete list)
And ... well, not exactly step-by-step, but tracking fair bit of migration and testing here: https://www.wiki.balug.org/wiki/doku.php?id=balug:mail_and_lists (it's not fully current ... but not that far from current either)
Coolness.
By the way, I had really good luck when I migrated the Skeptic mailing list from Johns Hopkins University's Sympa server to my Mailman server, in Octoble 2016, merging mail collected from the jhu.edu archives into Mailman's, even though the mailing list was operational at both locations for about a week or so during jhu.edu's shutdown of the mailing list. (The JHU people noticed that the listadmin no longer had a university connection, so told the subscribers they were going to shut it down imminently. I offered substitute hosting to prevent it from being lost.)
Of possible small interest: Skeptic list's FAQ that still reflects the Sympa setup (because I haven't yet had time to rewrite it): http://linuxmafia.com/pub/skeptic/skeptic_faq.html http://linuxmafia.com/pub/skeptic/list.html
The FAQ details various commands that can be sent to Sympa's e-mail command interface, including the GET command to fetch monthly archive bundles. (Again, GNU Mailman doesn't support that. Pity.)
Archives are now working. Relevant bit ... I ought (when I get around to it) check if there's bug filed (it may already be fixed even - but not yet to stable). And yes, I did say:
Caveats - at present the archives are effectively broken. It's a web server configuration issue - they're otherwise fine and being archived (and it's not filesystem or file permissions or the like). I'll get
The missing bit ... I'd (rather than redundantly copied/maintain) used: (relative to /etc/apache2): Include ../mailman/apache.conf in file sites-available/Include/temp.balug.org that was almost all well fine and good (I'd reviewed ./mailman/apache.conf earlier). But it left out one key needed bit, it has: <Directory /var/lib/mailman/archives/public/> Options FollowSymlinks AllowOverride None Order allow,deny Allow from all </Directory> but needs: <Directory /var/lib/mailman/archives/public/> Options FollowSymlinks AllowOverride None Order allow,deny Allow from all Require all granted </Directory> My relatively simple fix, add to file sites-available/Include/temp.balug.org <Directory /var/lib/mailman/archives/public/> Options FollowSymlinks AllowOverride None Order allow,deny Allow from all Require all granted </Directory> after: Include ../mailman/apache.conf ... Apache doesn't seem to care about the same <Directory /var/lib/mailman/archives/public/> appearing twice, and seems in that case to just use the latter fine, which keeps it a bit neater, by not needing to copy in all the other configuration contents from ../mailman/apache.conf
The .unroll script http://lists.balug.org/pipermail/balug-talk-balug.org/2017-July/000082.html came in handy :-) ... even though it turned out it wasn't an ordering dependency. It was very handy to pull all the config together (in a scratch vi session), toss out irrelevant bits, and look for relevant information and clues. That and having the Apache documentation handy (apache2-doc package - so versions all match 'n such). Useful clues also began to appear in some of the comments Debian included, notably with relevant context: # Sets the default security model of the Apache2 HTTPD server. It does # not allow access to the root filesystem outside of /usr/share and /var/www. # The former is used by web applications packaged in Debian, # the latter may be used for local directories served by the web server. If # your system is serving content from a sub-directory in /srv you must allow # access here, or in any related virtual host. <Directory /> Options FollowSymLinks AllowOverride None Require all denied </Directory> ... bit 'o research/refresher on Directory and Require, and then it was then quite clear why it wasn't working ... then just matter of try to determine best fix, apply and test ... and ... fixed. Works now.
So ... /etc/mailman/apache.conf should have included but failed to include, in it's section: <Directory /var/lib/mailman/archives/public/> the line: Require all granted So ... I think I'd call that a "bug" - even if it's documentation errata. Might be a Debian specific patch needed, as other distributions and/or Apache may have different defaults on that security. I don't know that anything in mailman causes /etc/mailman/apache.conf to be automagically included or copied in ... searching to see if any bits of that in the Apache config, other than what I put in ... nope, ... just what I put in ... but also very possible mailman might've added it if it were the case I had a default apache configuration or very close to that ... mine is quite a bit beyond default, but does also stick with the general Apache/Debian framework.
I had also fair bit earlier well gone over the permissions/ownerships, and had determined that was not the issue. By then clues were pointing in the direction of webserver configuration. I don't recall off-hand (maybe I ought test), but I think also Apache throws and/or logs slightly different error if it's from EPERM - file/directory permissions rather than Apache config. Could also potentially see that it wasn't EPERM by using strace(1) on the apache processes - but that probably woudln't be easiest/fastest way to get problem solved ... but it would be a relatively quick way to see if it was/wasn't an EPERM error and be able to confirm that.
From: "Rick Moen" rick@linuxmafia.com Subject: Re: [Balug-test] [BALUG-Admin] (pre-announce / "soft" open) BALUG-Test list Date: Mon, 10 Jul 2017 23:13:27 -0700
Quoting Michael Paoli (Michael.Paoli@cal.berkeley.edu):
Caveats - at present the archives are effectively broken. It's a web server configuration issue - they're otherwise fine and being archived (and it's not filesystem or file permissions or the like). I'll get that fixed (one of the relatively higher priorities).
You might want to check ownership/permissions in /var/lib/mailman/archives/private and /var/lib/mailman/archives/public .
Most relevant bit found among Debian bugs:
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=669813#36 The new apache security model requires adding this to the Directory stanza for mailman: Require all granted
But that's not particularly detailed, most notably omits mention of /etc/mailman/apache.conf and the <Directory /var/lib/mailman/archives/public/> section within.
Recommended to (mostly) fix mailman 1:2.1.18-2+deb8u1 amd64:
$ diff -U 5 etc/mailman/apache.conf.bug_669813 etc/mailman/apache.conf --- etc/mailman/apache.conf.bug_669813 2016-09-14 23:05:02.000000000 -0700 +++ etc/mailman/apache.conf 2017-07-11 07:01:29.116879436 -0700 @@ -26,10 +26,11 @@ <Directory /var/lib/mailman/archives/public/> Options FollowSymlinks AllowOverride None Order allow,deny Allow from all + Require all granted </Directory> <Directory /usr/share/images/mailman/> AllowOverride None Order allow,deny Allow from all $
At least that's the case for Jessie (presently oldstable) ( Debian GNU/Linux 8.8 (jessie) x86_64 mailman 1:2.1.18-2+deb8u1 amd64 apache2 2.4.10-10+deb8u9 amd64 )
I haven't (at least yet) checked to see if there's patch applied yet for newer than mailman 1:2.1.18-2+deb8u1 amd64 that may cover that fix.
In the meantime, for work-around for at least those versions, in Apache configuration, in addition to (which I added): Include ../mailman/apache.conf (or Include /etc/mailman/apache.conf or equivalent ) also add (and if the above is used via Include, use this *after* the above): <Directory /var/lib/mailman/archives/public/> Options FollowSymlinks AllowOverride None Order allow,deny Allow from all Require all granted </Directory>
From: "Michael Paoli" Michael.Paoli@cal.berkeley.edu Subject: Archives now working: BALUG-Test list Date: Tue, 11 Jul 2017 00:36:28 -0700
Archives are now working. Relevant bit ... I ought (when I get around to it) check if there's bug filed (it may already be fixed even - but not yet to stable).
The missing bit ... I'd (rather than redundantly copied/maintain) used: (relative to /etc/apache2): Include ../mailman/apache.conf in file sites-available/Include/temp.balug.org that was almost all well fine and good (I'd reviewed ./mailman/apache.conf earlier). But it left out one key needed bit, it has: <Directory /var/lib/mailman/archives/public/> Options FollowSymlinks AllowOverride None Order allow,deny Allow from all
</Directory> but needs: <Directory /var/lib/mailman/archives/public/> Options FollowSymlinks AllowOverride None Order allow,deny Allow from all Require all granted </Directory> My relatively simple fix, add to file sites-available/Include/temp.balug.org <Directory /var/lib/mailman/archives/public/> Options FollowSymlinks AllowOverride None Order allow,deny Allow from all Require all granted </Directory> after: Include ../mailman/apache.conf ... Apache doesn't seem to care about the same <Directory /var/lib/mailman/archives/public/> appearing twice, and seems in that case to just use the latter fine,
So ... /etc/mailman/apache.conf should have included but failed to include, in it's section: <Directory /var/lib/mailman/archives/public/> the line: Require all granted So ... I think I'd call that a "bug" - even if it's documentation errata. Might be a Debian specific patch needed, as other distributions and/or Apache may have different defaults on that security.
https://temp.balug.org/pipermail/balug-test/2017-July/000004.html temp.balug.org will in future be moved to lists.balug.org, so that will become: https://lists.balug.org/pipermail/balug-test/2017-July/000004.html
Quoting Michael Paoli (Michael.Paoli@cal.berkeley.edu):
And ... well, not exactly step-by-step, but tracking fair bit of migration and testing here: https://www.wiki.balug.org/wiki/doku.php?id=balug:mail_and_lists
Includes:
mailman admin commands should work via email (need more complete list)
This is woefully underdocumented (and underdeveloped?). I'm really not the extent of such functions. The standard doc for Mailman listadmins doesn't mention any, but it's known to be incomplete: https://www.gnu.org/software/mailman/mailman-admin/index.html Here's an example, showing how the listadmin can approve/reject/discard a held message entirely via e-mail: https://wiki.list.org/DOC/How%20do%20I%20respond%20to%20a%20held%20message%2...
That is in response to a held-message notice. In general, it _appears_ that Mailman doesn't really offer to listadmins the abilities it does to them via the WebUI.
In the Mailman conceptual model, docs are kept separately for users, listadmins, and site admins.
Maybe one reason for paucity of administrative functions via e-mail is that the command-line tools for site admins are so excellent -- from the Unix shell. Here is the list of the standard tools:
https://www.gnu.org/software/mailman/site.html
During the years that Marc Merlin administered the SVLUG mailing lists, he made me aware that there is also a robust and active community of people making additional command-line tools for Mailman that are not necessarily included in the Mailman standard package. I have a bunch of those that Merlin collected on linuxmafia.com in an archival replica of what Merlin put together.
Thanks, great pointers! :-)
Alas, GNU, they often put the "full" documentation in their info pages ... which ... of course since most folks interact with mailman without having mailman locally installed ... yeah, that wouldn't be the best place to document that. But yes, it ought be documented quite fully *somewhere*. Maybe they try not to overwhelm users/admins with "too much" detail/completeness? Seems there ought be commands (and probably web URL) that would give the full reference sets - for both users, and also admins. Ah well, note it on my wish/annoyance list. ;-)
Speaking of which ... GNU annoyances ... hit a couple of those in the last few months or so ... two of which I fine pretty dang annoying - nasty feature creep ... but alas, ... for a separate posting on some non-Test list. ;->
From: "Rick Moen" rick@linuxmafia.com Subject: Re: [Balug-test] [BALUG-Admin] (pre-announce / "soft" open) BALUG-Test list Date: Mon, 10 Jul 2017 23:51:40 -0700
Quoting Michael Paoli (Michael.Paoli@cal.berkeley.edu):
And ... well, not exactly step-by-step, but tracking fair bit of migration and testing here: https://www.wiki.balug.org/wiki/doku.php?id=balug:mail_and_lists
Includes:
mailman admin commands should work via email (need more complete list)
This is woefully underdocumented (and underdeveloped?). I'm really not the extent of such functions. The standard doc for Mailman listadmins doesn't mention any, but it's known to be incomplete: https://www.gnu.org/software/mailman/mailman-admin/index.html Here's an example, showing how the listadmin can approve/reject/discard a held message entirely via e-mail: https://wiki.list.org/DOC/How%20do%20I%20respond%20to%20a%20held%20message%2...
That is in response to a held-message notice. In general, it _appears_ that Mailman doesn't really offer to listadmins the abilities it does to them via the WebUI.
In the Mailman conceptual model, docs are kept separately for users, listadmins, and site admins.
Maybe one reason for paucity of administrative functions via e-mail is that the command-line tools for site admins are so excellent -- from the Unix shell. Here is the list of the standard tools:
https://www.gnu.org/software/mailman/site.html
During the years that Marc Merlin administered the SVLUG mailing lists, he made me aware that there is also a robust and active community of people making additional command-line tools for Mailman that are not necessarily included in the Mailman standard package. I have a bunch of those that Merlin collected on linuxmafia.com in an archival replica of what Merlin put together.