August 2024 - BALUG-Test - lists.balug.org

"Oops" ... lists down for a bit (back by the time you see this?)
by Michael Paoli 21 Aug '24

21 Aug '24

"Oops" ... lists down for a bit. Drats ... and hopefully all better by the time this mail makes it to list. So ... all was fine and dandy until ... managed to have a host booboo, basically locked up solid, and I power cycled it (this was physical host upon which the VM was running). Shouldn't be any biggie ... after that all up and fine again ... except the Mailman 3 lists. So, ... isolating and working towards fixing that. I'm guestimating maybe issue with some kind of lock file or the like that didn't get cleaned up upon (re)boot, or possibly, since I perhaps didn't do as many reboots as I ought to have made sure it would "always" come up clean, perhaps there was some misconfiguration or the like that somehow snuck in, that would be effectively a latent defect/issue, and that wouldn't show until a reboot or attempted restart of the relevant service(s). At this point still troubleshooting to narrow down the issue. As far as I can tell so far, looks like web server hits some kind of issue and generally gives or passes along a 500 response, that then ends up with the relevant web page(s) failing. When I did deeper into the backend service(s) ... notably postorius, looks like it gets requests okay and responds to them okay ... but something goes wrong somewhere between there and web server properly getting and passing that along to client browser. E.g.: https://lists.balug.org/mailman3/postorius/lists/ I see in postorius bit returning (from strace(1)) including: {\"display_name\": \"BALUG-Admin\", \"fqdn_listname\": \"balug-admin(a)lists.balug.org\", {\"display_name\": \"BALUG-Announce\", \"fqdn_listname\": \"balug-announce(a)lists.balug.org\", {\"display_name\": \"BALUG-Talk\", \"fqdn_listname\": \"balug-talk(a)lists.balug.org\", {\"display_name\": \"BALUG-Test\", \"fqdn_listname\": \"balug-test(a)lists.balug.org\", But somehow that doesn't make it to the web page the server serves ... so I'm guestimating there's some issue somewhere between postorius and Apache.

1 0

balug-test3@lists.balug.org --> balug-test@lists.balug.org
by Michael Paoli 20 Aug '24

20 Aug '24

The archives of balug-test3(a)lists.balug.org have been merged into those of balug-test(a)lists.balug.org and the balug-test3(a)lists.balug.org list has been removed.

1 0

test - ignore - db timezone testing
by Michael Paoli 20 Aug '24

20 Aug '24

db timezone testing 2024-08-20T10:15:15+00:00 Tue, 20 Aug 2024 10:15:15 +0000 Tue Aug 20 10:15:15 2024 2024-08-20T03:15:15-07:00 Tue, 20 Aug 2024 03:15:15 -0700 Tue Aug 20 03:15:15 2024

1 1

test - ignore db timezone testing
by Michael Paoli 20 Aug '24

20 Aug '24

db timezone testing 2024-08-20T10:15:15+00:00 Tue, 20 Aug 2024 10:15:15 +0000 Tue Aug 20 10:15:15 2024 2024-08-20T03:15:15-07:00 Tue, 20 Aug 2024 03:15:15 -0700 Tue Aug 20 03:15:15 2024

1 0

test - ignore Check database timezone behavior
by Michael Paoli 20 Aug '24

20 Aug '24

Check database timezone behavior 2024-08-20T09:35:01+00:00 Tue, 20 Aug 2024 09:35:01 +0000 Tue Aug 20 09:35:01 2024 2024-08-20T02:35:01-07:00 Tue, 20 Aug 2024 02:35:01 -0700 Tue Aug 20 02:35:01 2024

2 1

test - ignore Check database timezone behavior
by Michael Paoli 20 Aug '24

20 Aug '24

Check database timezone behavior 2024-08-20T09:35:01+00:00 Tue, 20 Aug 2024 09:35:01 +0000 Tue Aug 20 09:35:01 2024 2024-08-20T02:35:01-07:00 Tue, 20 Aug 2024 02:35:01 -0700 Tue Aug 20 02:35:01 2024

1 0

gravatar.com leakage
by Michael Paoli 19 Aug '24

19 Aug '24

Yeah, I notice at least by default on at least the Mailman 3 hyperkitty archive, it "leaks" information to gravatar.com using per-member URLs for images from gravatar.com ... ugh. Yeah, looks like I'm not the first to notice this: https://lists.mailman3.org/archives/list/mailman-users@mailman3.org/thread/… Anyway, one more item to add to migration checklist of things to do, and from the above, looks like it's probably not too difficult to manage to do. Michael Paoli wrote: > Mailman 2 --> Mailman 3 ... largely (but not entirely) done; holler if > one notes any issues. > > Summary on status/progres can be found here: > https://www.wiki.balug.org/wiki/doku.php?id=system:mailman3#mailman_2_--_... > > There were some glitches, notably with import21 ... > and did find a work-around(/fix?) for that, > and also had some issues with archive import+indexing, > but seem to have made it past all that now (though still have > some more validation checks to do, etc.). > > And note that thus far, the merging of archive from the > (temporary) balug-test3(a)lists.balug.org to > balug-test(a)lists.balug.org is still pending. > In the meantime, those archives are available here: > https://lists.balug.org/mailman3/hyperkitty/list/balug-test3@lists.balug.... > And the balug-test3(a)lists.balug.org is also mostly locked down > to prevent changes (no new postings or subscription/membership changes).

1 0

Mailman 2 --> Mailman 3 ... largely (but not entirely) done; holler if one notes any issues.
by Michael Paoli 19 Aug '24

19 Aug '24

Mailman 2 --> Mailman 3 ... largely (but not entirely) done; holler if one notes any issues. Summary on status/progres can be found here: https://www.wiki.balug.org/wiki/doku.php?id=system:mailman3#mailman_2_--_3_… There were some glitches, notably with import21 ... and did find a work-around(/fix?) for that, and also had some issues with archive import+indexing, but seem to have made it past all that now (though still have some more validation checks to do, etc.). And note that thus far, the merging of archive from the (temporary) balug-test3(a)lists.balug.org to balug-test(a)lists.balug.org is still pending. In the meantime, those archives are available here: https://lists.balug.org/mailman3/hyperkitty/list/balug-test3@lists.balug.or… And the balug-test3(a)lists.balug.org is also mostly locked down to prevent changes (no new postings or subscription/membership changes).

1 0

Re: Rats ... some hyperkitty_import issue(s) ...
by admin＠bclug.ca 19 Aug '24

19 Aug '24

Michael Paoli via BALUG-Test wrote on 2024-08-18 01:34: > So, seems it accepted and indexed them ... but for some reason isn't > yet showing them in hyperkitty - though it lets me download them > from the archive. You've probably tried this already, but: mailman-web update_index -h or mailman-web update_index_one_list -h rb

2 1

Re: And so it begins (well, only one teeny oops so far) Re: Welcome to the "BALUG-Test" mailing list
by admin＠bclug.ca 18 Aug '24

18 Aug '24

Michael Paoli via BALUG-Test wrote on 2024-08-18 01:30: > Just using the Debian packages. I think they include the venv setup, > but haven't poked into the details to see exactly how they function in > that regard. There should be approaching *zero* Debian packages installed! From my brain-addled notes: ## Use PG, not MySQL! Change next line per venv instructions online: apt install default-libmysqlclient-dev build-essential ## More Debian package(s): sudo apt install libpcre3 libpcre3-dev Various notes follow, the TL;DR is above regarding which repo packages to install. Everything gets installed inside a venv with pip! sudo useradd --create-home \ --home-dir /opt/mailman \ --shell /bin/bash \ mailman sudo chown -v mailman: /opt/mailman sudo chmod -v 755 /opt/mailman sudo -i -u mailman bash -c "python3 -m venv venv; echo 'source /opt/mailman/venv/bin/activate' >> /opt/mailman/.bashrc; exit" sudo -i -u mailman bash -c "pip install wheel mailman PyMySQL" ## Run as sudo: mkdir /etc/mailman3 touch /etc/mailman3/mailman3.service touch /etc/mailman3/mailman-hyperkitty.cfg ## This is about the only Debian / Ubuntu packages to install: ## And, screw messing with MySQL, PostgreSQL all the way: apt update ## Use PG, not MySQL, change next line per venv instructions online: apt install default-libmysqlclient-dev build-essential ## Is this run as mailman? I *think* so, inside the venv: pip install mailman-web mailman-hyperkitty ## As user mailman inside venv: MAILMAN_WEB_CONFIG=/etc/mailman3/mailman-web.py mailman-web migrate MAILMAN_WEB_CONFIG=/etc/mailman3/mailman-web.py mailman-web collectstatic MAILMAN_WEB_CONFIG=/etc/mailman3/mailman-web.py mailman-web compress MAILMAN_WEB_CONFIG=/etc/mailman3/mailman-web.py mailman-web compressmessages pip install uwsgi ## create & edit /etc/mailman3/uwsgi.ini: ## You can run test run uwsgi using the following command: ## ## (venv)$ uwsgi --ini /etc/mailman3/uwsgi.ini uwsgi --ini /etc/mailman3/uwsgi.ini ## NOTE: if /opt/mailman/web/logs/uwsgi-error.log (?) has this: ## !!! no internal routing support, rebuild with pcre support !!! ## then... ## pip uninstall uwsgi ## More Debian package(s): sudo apt install libpcre3 libpcre3-dev pip install --force-reinstall -U --no-deps --no-cache-dir uwsgi Next and last: Xapian full-text search. ## Xapian full-text search ################################### ## Xapian full-text search ################################### ## Xapian full-text search ################################### ## ## Latest version: 1.4.25 ## Inside (venv) as mailman: export XAPIAN_VERSION="1.4.25" mkdir /tmp/mailman/xapian -vp cd /tmp/mailman/xapian curl https://raw.githubusercontent.com/notanumber/xapian-haystack/master/xapian_w> bash xapian_wheel_builder.sh $XAPIAN_VERSION ls -l *.whl wheelFile=$(ls *.whl) pip install ${wheelFile} xapian-haystack unset wheelFile rm -fr /tmp/mailman/xapian > /dev/null echo "Edit /etc/mailman3/settings_local.py and add HAYSTACK_CONNECTIONS" ## DO NOT OVERLOOK XAPIAN-HAYSTACK IN LINES ABOVE!!! ## DO NOT OVERLOOK XAPIAN-HAYSTACK IN LINES ABOVE!!! ## DO NOT OVERLOOK XAPIAN-HAYSTACK IN LINES ABOVE!!!

1 0

2024

2023

2022

2021

2020

2019

2018

2017

BALUG-Test August 2024