Quoting Michael Paoli (Michael.Paoli@cal.berkeley.edu):
Yes, probably good to change it at least once in a while (like if it gets leaked/exposed, or someone ceases to be a listadmin, or probably at least quarterly if any usage forces it to go across the network in cleartext, and probably at least yearly at a minimum in any case).
I used to worry about this more than I do now. Quite a bit of mischief _can_ be done by too many people holding listadmin passwords (including summary deletion of the entire mailing list from its Web admin interface), but in practice it tends not to happen.