LWVWV Member Subscription Automation
Automated Docker container for subscribing LWVWV members to Mailman 3 mailing lists.
Overview
This automation coordinates:
- Downloading the latest member roster from the LWVWV portal
- Converting the roster to CSV format (email,first_name,last_name)
- Subscribing members to the Mailman list via the connector API
Architecture
┌─────────┐ ┌─────────────────────────────┐ ┌──────────────────┐
│ Ofelia │────>│ lwvwv-subscriber-cron │────>│ LWVWV Portal │
│Scheduler│cron │ (Docker Container) │ │ (download CSV) │
└─────────┘ └─────────────────────────────┘ └──────────────────┘
│
┌───────────────┼───────────────┐
▼ ▼ ▼
┌──────────┐ ┌──────────────┐ ┌──────────────┐
│download- │ │google-civic- │ │subscribe- │
│roster.js │──>│api.pl │──>│members.sh │
│(portal) │ │(email-csv) │ │(connector) │
└──────────┘ └──────────────┘ └──────────────┘
│
▼
┌──────────────┐
│mailman- │
│connector │
│(batch sub) │
└──────────────┘
Files
| File | Description |
|---|---|
Dockerfile |
Container image definition |
docker-compose.yml |
Service orchestration with Ofelia labels |
entrypoint.sh |
Parses ../env and exports variables |
run-subscription.sh |
Master orchestration script |
start-automation.sh |
Quick-start helper script |
README.md |
This file |
Environmental Variables
Single Source of Truth: ../env
All configuration is stored in one file: ../env (in the parent directory)
This file uses Perl-style syntax (which existing scripts expect):
$VARIABLE_NAME = "value";
How It Works
../envis the single source of truth for all configuration- The entrypoint.sh script parses
../envand exports variables as environment variables - All scripts can then access them via standard environment variable syntax (
$VARIABLE_NAME)
Configuration Variables
Required for Subscription
| Variable | Description | Example |
|---|---|---|
$CONNECTOR_URL |
URL of mailman-connector | https://mailman-connector.example.org |
$CONNECTOR_PASSWORD |
Secret password for connector API | your_secret_password |
$MAILMAN_LIST_ID |
Target mailing list ID | members.lists.example.org |
Required for Portal Download
| Variable | Description | Example |
|---|---|---|
$PORTAL_URL |
LWVWV portal URL | https://portal.lwv.org |
$MEMBERSHIP_URL |
Membership page URL | https://portal.lwv.org/groups/... |
Email Notifications (Optional)
| Variable | Description | Example |
|---|---|---|
$SMTP_HOST |
SMTP server | mail.example.org |
$SMTP_PORT |
SMTP port | 587 |
$SMTP_USER |
SMTP username | example@example.org |
$SMTP_PASSWORD |
SMTP password | your_smtp_password |
$EMAIL_FROM |
Sender address | example@example.org |
$EMAIL_TO |
Recipient for alerts | hello@example.org |
Google API (for district lookup)
| Variable | Description | Example |
|---|---|---|
$key |
Google Civic API key | AIzaSy... |
$STATE |
State code | WV |
$stateLeagueID |
State league ID | WV000 |
$localLeagueIDs |
Local league IDs | `WV102 |
Example ../env File
# Portal Configuration
$PORTAL_URL = "https://portal.lwv.org";
$MEMBERSHIP_URL = "https://portal.lwv.org/groups/53a93df1/league_membership_state_view";
# Connector Settings
$CONNECTOR_URL = "https://mailman-connector.example.org";
$CONNECTOR_PASSWORD = "your_secret_password";
$MAILMAN_LIST_ID = "members.lists.example.org";
# Email Notification Settings
$SMTP_HOST = "mail.example.org";
$SMTP_PORT = "587";
$SMTP_USER = "example@example.org";
$SMTP_PASSWORD = "your_smtp_password";
$EMAIL_FROM = "example@example.org";
$EMAIL_TO = "hello@example.org";
# Google API Configuration
$key = "LALKDdkdk_12LSL_RKFDL";
$STATE = "WV";
$stateLeagueID = "WV000";
$localLeagueIDs = "WV102|WV103|WV112";
Quick Start
1. Configure Environment
Ensure ../env exists with your settings:
cd automation
# The env file should already exist in the parent directory
ls ../env
2. Build and Start
docker-compose up -d
3. Verify Container is Running
docker ps | grep lwvwv-subscriber
Usage
Manual Trigger (for testing)
# Run the full workflow (downloads roster, converts, subscribes)
docker exec lwvwv-subscriber /app/run-subscription.sh
# With a specific CSV file (skips download)
docker exec lwvwv-subscriber /app/run-subscription.sh /path/to/members.csv
# Run subscribe-members.sh directly with env vars
docker exec lwvwv-subscriber /app/subscribe-members.sh /tmp/rosters/members.csv
Scheduled Execution (via Ofelia)
The container includes Ofelia labels for automatic scheduling:
- Default: 4x daily at 00:00, 06:00, 12:00, 18:00
- Format: Cron expression
0 0,6,12,18 * * *
To change schedule, edit docker-compose.yml:
labels:
ofelia.enabled: "true"
ofelia.job-exec.lwvwv-subscribe.schedule: "0 0,6,12,18 * * *"
Standalone Usage (without Docker)
The scripts also work outside Docker using positional arguments:
# subscribe-members.sh with all positional args
./subscribe-members.sh members.csv mypassword members.lists.example.org https://connector.example.com
# With some env vars, some positional
CONNECTOR_PASSWORD=mypassword ./subscribe-members.sh members.csv
Logs
# View container logs
docker logs lwvwv-subscriber
# Follow logs in real-time
docker logs -f lwvwv-subscriber
# View Ofelia scheduler logs
docker logs ofelia
Troubleshooting
Session Expired
If portal download fails with "Session expired":
# Run save-session.js on host to refresh session
cd ..
node save-session.js
# Then restart container
docker-compose restart
Missing Environment Variables
If you see errors like CONNECTOR_URL not set:
- Check that
../envexists and contains the variable - Verify the entrypoint is parsing it correctly:
docker exec lwvwv-subscriber env | grep CONNECTOR
Manual CSV Subscription
To subscribe members without downloading:
# Place CSV file in shared volume
cp members.csv automation/rosters/
# Run with specific file
docker exec lwvwv-subscriber /app/run-subscription.sh /tmp/rosters/members.csv
Network Requirements
The container needs access to:
- LWV Portal (
https://portal.lwv.org) - SMTP server (if using email alerts)
Security Notes
../envis mounted read-only (:ro) in the container- Sensitive data is never committed to git
- Session file (
.session.json) is also mounted read-only - All scripts use the single
../envfile - no duplicated configuration