bike/lwvwv
1
0
Fork 0
Code Issues 2 Pull Requests Projects Releases Wiki Activity

Adds automation in a containerized environment

Browse Source
This commit is contained in:
Jonathan Rosenbaum 2026-05-04 23:35:54 -04:00
parent 1dca98898e
commit 203bffbbf8
10 changed files with 733 additions and 29 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

5
.gitignore vendored
View File

@ -34,7 +34,6 @@ inc/
/MANIFEST.bak /MANIFEST.bak
/pm_to_blib /pm_to_blib
/*.zip /*.zip
env
*.csi *.csi
/node_modules /node_modules
*png *png
@ -42,3 +41,7 @@ env
dontadd dontadd
email email
.session.json .session.json
# Files and Directories
env
automation/rosters

46
Dockerfile Normal file
View File

@ -0,0 +1,46 @@
FROM node:20-bookworm
RUN apt-get update && apt-get install -y \
curl \
jq \
s-nail \
libwww-curl-perl \
cpanminus \
&& rm -rf /var/lib/apt/lists/*
RUN cpanm --notest WWW::Curl::Simple
RUN cpanm --notest JSON
# Set working directory
WORKDIR /app
# Copy scripts from parent directory
COPY download-roster.js ./
COPY google-civic-api.pl ./
COPY subscribe-members.sh ./
COPY env ./
COPY automation/run-subscription.sh ./
COPY automation/entrypoint.sh ./
# Make scripts executable
RUN chmod +x *.sh *.pl
# Install Node dependencies for download-roster.js
RUN npm install playwright
RUN npx playwright install chromium --with-deps
# Create work directory for temporary files
RUN mkdir -p /tmp/rosters
# Set environment variables
ENV NODE_ENV=production \
WORK_DIR=/tmp/rosters \
PLAYWRIGHT_BROWSERS_PATH=/root/.cache/ms-playwright
# Use entrypoint to parse env file
ENTRYPOINT ["/app/entrypoint.sh"]
# Keep container running for Ofelia to execute commands
CMD ["tail", "-f", "/dev/null"]

54
README.md
View File

@ -53,6 +53,9 @@ $localLeagueIDs = "WV102|WV103|WV112";
| `SSL_VERIFY_IGNORE` | No | SSL verify setting for SMTP (default: ignore) | | `SSL_VERIFY_IGNORE` | No | SSL verify setting for SMTP (default: ignore) |
| `EMAIL_FROM` | No | Sender email address for error notifications | | `EMAIL_FROM` | No | Sender email address for error notifications |
| `EMAIL_TO` | No | Recipient email address for error notifications | | `EMAIL_TO` | No | Recipient email address for error notifications |
| `MAILMAN_LIST_ID` | Yes | Target Mailman list ID(s) with optional League ID filter. Format: `list_id:league_id`. Examples: `members.lists.lwvwv.org:WV000` or `list1:WV000, list2:WV103`. If league omitted, defaults to ALL. Used by run-subscription.sh |
| `CONNECTOR_URL` | Yes | URL of mailman-connector API. Used by run-subscription.sh |
| `CONNECTOR_PASSWORD` | Yes | Secret password for connector API. Used by run-subscription.sh |
| `key` | Optional | Google Civic Information API key. See [Google API credentials](https://console.cloud.google.com/apis/credentials) - restrict key to the Google Civic Information API. Required for google-civic-api.pl. | | `key` | Optional | Google Civic Information API key. See [Google API credentials](https://console.cloud.google.com/apis/credentials) - restrict key to the Google Civic Information API. Required for google-civic-api.pl. |
| `STATE` | Required* | Two-letter state code (e.g., WV). Required for google-civic-api.pl. | | `STATE` | Required* | Two-letter state code (e.g., WV). Required for google-civic-api.pl. |
| `stateLeagueID` | Required* | State league ID (e.g., WV000). Required for google-civic-api.pl. | | `stateLeagueID` | Required* | State league ID (e.g., WV000). Required for google-civic-api.pl. |
@ -157,9 +160,50 @@ This may be freely redistributed under the terms of the GNU General Public Licen
## Automation ## Automation
### run-subscription.sh
Master orchestration script that coordinates the entire subscription workflow:
1. Downloads roster from LWVWV portal (via download-roster.js)
2. Filters roster by League ID for each list (via google-civic-api.pl)
3. Subscribes members to one or more Mailman lists
**Multi-List Support with League Filtering:**
`MAILMAN_LIST_ID` uses colon-separated format to associate each list with a League ID:
```bash
# Format: list_id:league_id
$MAILMAN_LIST_ID = "members.lists.lwvwv.org:WV000, morgantown.lists.lwvwv.org:WV103"
```
- **list_id**: The Mailman list identifier (e.g., `members.lists.lwvwv.org`)
- **league_id**: The LWV League ID to filter members (e.g., `WV000`, `WV103`, `ALL`)
If `:league_id` is omitted, defaults to `ALL` (all members).
**Examples:**
```bash
# Single list, all members
$MAILMAN_LIST_ID = "members.lists.lwvwv.org"
# Single list, only WV000 (state) members
$MAILMAN_LIST_ID = "members.lists.lwvwv.org:WV000"
# Multiple lists with different league filters
$MAILMAN_LIST_ID = "members.lists.lwvwv.org:WV000, morgantown.lists.lwvwv.org:WV103, huntington.lists.lwvwv.org:WV102"
```
**Usage:**
```bash
./run-subscription.sh [csv_file]
```
If no CSV file is provided, the script will download the latest roster from the portal.
### subscribe-members.sh ### subscribe-members.sh
Batch subscription example using [google-civic-api.pl](https://git.bikeshopi.dev/bike/lwvwv) to extract member data from LWVWV membership CSV files. This script processes membership exports and automatically subscribes members to mailing lists. Low-level script that subscribes members to a **single** Mailman 3 list via the connector API. Called by run-subscription.sh for each list when multiple lists are configured.
**Note:** This script handles ONE list per invocation. For multiple lists, use run-subscription.sh.
**Prerequisites:** **Prerequisites:**
- `google-civic-api.pl` must be in the same directory or PATH - `google-civic-api.pl` must be in the same directory or PATH
@ -167,7 +211,7 @@ Batch subscription example using [google-civic-api.pl](https://git.bikeshopi.dev
**Usage:** **Usage:**
```bash ```bash
./test-subscribe-members.sh <csv_file> <password> [list_id] [connector_url] ./subscribe-members.sh <csv_file> <password> [list_id] [connector_url]
``` ```
**Arguments:** **Arguments:**
@ -179,13 +223,13 @@ Batch subscription example using [google-civic-api.pl](https://git.bikeshopi.dev
**Examples:** **Examples:**
```bash ```bash
# Using default list and connector URL # Using default list and connector URL
./test-subscribe-members.sh members.csv mysecretpassword ./subscribe-members.sh members.csv mysecretpassword
# With specific list # With specific list
./test-subscribe-members.sh members.csv mysecretpassword test.lists.example.org ./subscribe-members.sh members.csv mysecretpassword test.lists.example.org
# With specific list and connector URL # With specific list and connector URL
./test-subscribe-members.sh members.csv mysecretpassword test.lists.example.org https://connector.example.com ./subscribe-members.sh members.csv mysecretpassword test.lists.example.org https://connector.example.com
``` ```
**How it works:** **How it works:**

245
automation/README.md Normal file
View File

@ -0,0 +1,245 @@
# LWVWV Member Subscription Automation
Automated Docker container for subscribing LWVWV members to Mailman 3 mailing lists.
## Overview
This automation coordinates:
1. **Downloading** the latest member roster from the LWVWV portal
2. **Converting** the roster to CSV format (email,first_name,last_name)
3. **Subscribing** members to the Mailman list via the connector API
## Architecture
```
┌─────────┐ ┌─────────────────────────────┐ ┌──────────────────┐
│ Ofelia │────>│ lwvwv-subscriber-cron │────>│ LWVWV Portal │
│Scheduler│cron │ (Docker Container) │ │ (download CSV) │
└─────────┘ └─────────────────────────────┘ └──────────────────┘
│
┌───────────────┼───────────────┐
▼ ▼ ▼
┌──────────┐ ┌──────────────┐ ┌──────────────┐
│download- │ │google-civic- │ │subscribe- │
│roster.js │──>│api.pl │──>│members.sh │
│(portal) │ │(email-csv) │ │(connector) │
└──────────┘ └──────────────┘ └──────────────┘
│
▼
┌──────────────┐
│mailman- │
│connector │
│(batch sub) │
└──────────────┘
```
## Files
| File | Description |
|------|-------------|
| `Dockerfile` | Container image definition |
| `docker-compose.yml` | Service orchestration with Ofelia labels |
| `entrypoint.sh` | Parses ../env and exports variables |
| `run-subscription.sh` | Master orchestration script |
| `start-automation.sh` | Quick-start helper script |
| `README.md` | This file |
## Environmental Variables
### Single Source of Truth: `../env`
All configuration is stored in **one file**: `../env` (in the parent directory)
This file uses **Perl-style syntax** (which existing scripts expect):
```perl
$VARIABLE_NAME = "value";
```
### How It Works
1. **`../env`** is the **single source of truth** for all configuration
2. The **entrypoint.sh** script parses `../env` and exports variables as environment variables
3. All scripts can then access them via standard environment variable syntax (`$VARIABLE_NAME`)
### Configuration Variables
#### Required for Subscription
| Variable | Description | Example |
|----------|-------------|---------|
| `$CONNECTOR_URL` | URL of mailman-connector | `https://mailman-connector.example.org` |
| `$CONNECTOR_PASSWORD` | Secret password for connector API | `your_secret_password` |
| `$MAILMAN_LIST_ID` | Target mailing list ID | `members.lists.example.org` |
#### Required for Portal Download
| Variable | Description | Example |
|----------|-------------|---------|
| `$PORTAL_URL` | LWVWV portal URL | `https://portal.lwv.org` |
| `$MEMBERSHIP_URL` | Membership page URL | `https://portal.lwv.org/groups/...` |
#### Email Notifications (Optional)
| Variable | Description | Example |
|----------|-------------|---------|
| `$SMTP_HOST` | SMTP server | `mail.example.org` |
| `$SMTP_PORT` | SMTP port | `587` |
| `$SMTP_USER` | SMTP username | `example@example.org` |
| `$SMTP_PASSWORD` | SMTP password | `your_smtp_password` |
| `$EMAIL_FROM` | Sender address | `example@example.org` |
| `$EMAIL_TO` | Recipient for alerts | `hello@example.org` |
#### Google API (for district lookup)
| Variable | Description | Example |
|----------|-------------|---------|
| `$key` | Google Civic API key | `AIzaSy...` |
| `$STATE` | State code | `WV` |
| `$stateLeagueID` | State league ID | `WV000` |
| `$localLeagueIDs` | Local league IDs | `WV102|WV103|WV112` |
### Example `../env` File
```perl
# Portal Configuration
$PORTAL_URL = "https://portal.lwv.org";
$MEMBERSHIP_URL = "https://portal.lwv.org/groups/53a93df1/league_membership_state_view";
# Connector Settings
$CONNECTOR_URL = "https://mailman-connector.example.org";
$CONNECTOR_PASSWORD = "your_secret_password";
$MAILMAN_LIST_ID = "members.lists.example.org";
# Email Notification Settings
$SMTP_HOST = "mail.example.org";
$SMTP_PORT = "587";
$SMTP_USER = "example@example.org";
$SMTP_PASSWORD = "your_smtp_password";
$EMAIL_FROM = "example@example.org";
$EMAIL_TO = "hello@example.org";
# Google API Configuration
$key = "LALKDdkdk_12LSL_RKFDL";
$STATE = "WV";
$stateLeagueID = "WV000";
$localLeagueIDs = "WV102|WV103|WV112";
```
## Quick Start
### 1. Configure Environment
Ensure `../env` exists with your settings:
```bash
cd automation
# The env file should already exist in the parent directory
ls ../env
```
### 2. Build and Start
```bash
docker-compose up -d
```
### 3. Verify Container is Running
```bash
docker ps | grep lwvwv-subscriber
```
## Usage
### Manual Trigger (for testing)
```bash
# Run the full workflow (downloads roster, converts, subscribes)
docker exec lwvwv-subscriber /app/run-subscription.sh
# With a specific CSV file (skips download)
docker exec lwvwv-subscriber /app/run-subscription.sh /path/to/members.csv
# Run subscribe-members.sh directly with env vars
docker exec lwvwv-subscriber /app/subscribe-members.sh /tmp/rosters/members.csv
```
### Scheduled Execution (via Ofelia)
The container includes Ofelia labels for automatic scheduling:
- **Default**: 4x daily at 00:00, 06:00, 12:00, 18:00
- **Format**: Cron expression `0 0,6,12,18 * * *`
To change schedule, edit `docker-compose.yml`:
```yaml
labels:
ofelia.enabled: "true"
ofelia.job-exec.lwvwv-subscribe.schedule: "0 0,6,12,18 * * *"
```
### Standalone Usage (without Docker)
The scripts also work outside Docker using positional arguments:
```bash
# subscribe-members.sh with all positional args
./subscribe-members.sh members.csv mypassword members.lists.example.org https://connector.example.com
# With some env vars, some positional
CONNECTOR_PASSWORD=mypassword ./subscribe-members.sh members.csv
```
## Logs
```bash
# View container logs
docker logs lwvwv-subscriber
# Follow logs in real-time
docker logs -f lwvwv-subscriber
# View Ofelia scheduler logs
docker logs ofelia
```
## Troubleshooting
### Session Expired
If portal download fails with "Session expired":
```bash
# Run save-session.js on host to refresh session
cd ..
node save-session.js
# Then restart container
docker-compose restart
```
### Missing Environment Variables
If you see errors like `CONNECTOR_URL not set`:
1. Check that `../env` exists and contains the variable
2. Verify the entrypoint is parsing it correctly:
```bash
docker exec lwvwv-subscriber env | grep CONNECTOR
```
### Manual CSV Subscription
To subscribe members without downloading:
```bash
# Place CSV file in shared volume
cp members.csv automation/rosters/
# Run with specific file
docker exec lwvwv-subscriber /app/run-subscription.sh /tmp/rosters/members.csv
```
## Network Requirements
The container needs access to:
- LWV Portal (`https://portal.lwv.org`)
- SMTP server (if using email alerts)
## Security Notes
- `../env` is mounted **read-only** (`:ro`) in the container
- Sensitive data is never committed to git
- Session file (`.session.json`) is also mounted read-only
- All scripts use the single `../env` file - no duplicated configuration

28
automation/docker-compose.yml Normal file
View File

@ -0,0 +1,28 @@
services:
lwvwv-subscriber:
build:
context: ..
dockerfile: Dockerfile
image: lwvwv-subscriber
container_name: lwvwv-subscriber
volumes:
# Mount parent directory env file (source of truth)
- ../env:/app/env:ro
# Mount session file for portal authentication
- ../.session.json:/app/.session.json:ro
# Optional: Persist roster files for debugging
- ./rosters:/tmp/rosters
networks:
- default
logging:
driver: "json-file"
options:
max-size: "10m"
max-file: "3"
# Ofelia labels for scheduling (4x daily)
labels:
ofelia.enabled: "true"
ofelia.job-exec.lwvwv-subscribe.schedule: "0 0,6,12,18 * * *"
ofelia.job-exec.lwvwv-subscribe.container: "lwvwv-subscriber"
ofelia.job-exec.lwvwv-subscribe.command: "/app/run-subscription.sh"
restart: always

27
automation/entrypoint.sh Executable file
View File

@ -0,0 +1,27 @@
#!/bin/bash
# Entrypoint script for LWVWV subscriber container
# Parses ../env (Perl format) and exports variables as environment variables
# Then executes the main command
ENV_FILE="/app/env"
if [ -f "$ENV_FILE" ]; then
echo "Loading configuration from $ENV_FILE..."
while IFS= read -r line || [ -n "$line" ]; do
# Skip empty lines and comments
[[ -z "$line" || "$line" =~ ^# ]] && continue
# Parse Perl format: $VAR = "value" or $VAR = "value";
if [[ $line =~ ^\$([A-Za-z_][A-Za-z0-9_]*)[[:space:]]*=[[:space:]]*\"([^\"]*)\"[[:space:]]*\;?$ ]]; then
varname="${BASH_REMATCH[1]}"
varvalue="${BASH_REMATCH[2]}"
export "$varname=$varvalue"
fi
done < "$ENV_FILE"
echo "Configuration loaded successfully"
else
echo "Warning: $ENV_FILE not found"
fi
# Execute the main command
exec "$@"

198
automation/run-subscription.sh Executable file
View File

@ -0,0 +1,198 @@
#!/bin/bash
source "$(dirname "$0")/entrypoint.sh" 2>/dev/null || true
# Master orchestration script for LWVWV member subscription automation
# This script coordinates the entire workflow:
# 1. Download roster from LWVWV portal (if download-roster.js is available)
# 2. Convert roster to CSV format, filtered by League ID per list
# 3. Subscribe members to Mailman list(s) via connector API
# 4. Cleanup temporary files
#
# Environment Variables (loaded from /app/env via entrypoint.sh):
# CONNECTOR_URL - URL of mailman-connector
# CONNECTOR_PASSWORD - Secret password for connector API
# MAILMAN_LIST_ID - Target mailing list ID(s). Format: "list_id:league_id, list_id:league_id"
# If :league_id is omitted, defaults to "ALL" (all members)
# EMAIL_TO - Email for failure notifications
# SMTP_HOST - SMTP server for notifications
#
# Examples:
# Single list, all members: "members.lists.lwvwv.org"
# Single list, WV000 only: "members.lists.lwvwv.org:WV000"
# Multiple lists: "members.lists.lwvwv.org:WV000, morgantown.lists.lwvwv.org:WV103"
set -e # Exit on error
# Configuration from environment
WORK_DIR="${WORK_DIR:-/tmp/rosters}"
TIMESTAMP=$(date +%Y%m%d_%H%M%S)
ROSTER_FILE="$WORK_DIR/roster_$TIMESTAMP.csv"
log() {
echo "[$(date '+%Y-%m-%d %H:%M:%S')] $1"
}
error_exit() {
echo "[ERROR] $1" >&2
# Send alert email if configured
if [ -n "$EMAIL_TO" ] && [ -n "$SMTP_HOST" ]; then
echo "$1" | s-nail -s "LWVWV Subscription Failed" "$EMAIL_TO" 2>/dev/null || true
fi
exit 1
}
# Create work directory
mkdir -p "$WORK_DIR"
# Determine CSV input source or download
if [ -n "$1" ]; then
# CSV file provided as argument
ROSTER_FILE="$1"
log "Using provided roster file: $ROSTER_FILE"
if [ ! -f "$ROSTER_FILE" ]; then
error_exit "Roster file not found: $ROSTER_FILE"
fi
else
# No CSV provided - try to download from portal
if [ ! -f "./download-roster.js" ]; then
error_exit "No CSV file provided and download-roster.js not found"
fi
# Step 1: Download roster
log "Downloading roster from LWVWV portal..."
if ! node download-roster.js; then
error_exit "Failed to download roster"
fi
# Find the downloaded file
ROSTER_FILE=$(ls -t league_membership_state_view_*.csv 2>/dev/null | head -1)
if [ -z "$ROSTER_FILE" ]; then
error_exit "No roster file found after download"
fi
log "Downloaded roster: $ROSTER_FILE"
fi
# Validate required environment variables
if [ -z "$CONNECTOR_URL" ]; then
error_exit "CONNECTOR_URL not set. Ensure /app/env is mounted and contains \$CONNECTOR_URL"
fi
if [ -z "$CONNECTOR_PASSWORD" ]; then
error_exit "CONNECTOR_PASSWORD not set. Ensure /app/env is mounted and contains \$CONNECTOR_PASSWORD"
fi
if [ -z "$MAILMAN_LIST_ID" ]; then
error_exit "MAILMAN_LIST_ID not set. Ensure /app/env is mounted and contains \$MAILMAN_LIST_ID"
fi
if [ ! -f "./google-civic-api.pl" ]; then
error_exit "google-civic-api.pl not found for roster conversion"
fi
if [ ! -f "./subscribe-members.sh" ]; then
error_exit "subscribe-members.sh not found"
fi
# Parse multiple list configurations (comma-separated)
# Format: list_id:league_id or just list_id (defaults to ALL)
IFS=',' read -ra LIST_CONFIGS <<< "$MAILMAN_LIST_ID"
log "=========================================="
log "Starting batch subscription"
log "=========================================="
log "Roster File: $ROSTER_FILE"
log "Connector URL: $CONNECTOR_URL"
log "Total Lists: ${#LIST_CONFIGS[@]}"
log "=========================================="
# Track success/failure per list
SUCCESS_COUNT=0
FAILED_LISTS=()
# Helper function to increment counter (avoids set -e issues with arithmetic)
increment_count() {
SUCCESS_COUNT=$((SUCCESS_COUNT + 1))
}
# Loop through each list configuration
for CONFIG in "${LIST_CONFIGS[@]}"; do
# Trim whitespace
CONFIG=$(echo "$CONFIG" | xargs)
# Parse list_id and league_id (format: list_id:league_id or just list_id)
if [[ "$CONFIG" =~ ^([^:]+):(.*)$ ]]; then
LIST_ID="${BASH_REMATCH[1]}"
LEAGUE_ID="${BASH_REMATCH[2]}"
else
LIST_ID="$CONFIG"
LEAGUE_ID="ALL"
fi
log ""
log "----------------------------------------"
log "Processing List: $LIST_ID"
log "League Filter: $LEAGUE_ID"
log "----------------------------------------"
# Generate filtered CSV for this list
FILTERED_CSV="$WORK_DIR/members_${LEAGUE_ID}_${TIMESTAMP}.csv"
log "Filtering roster for League ID: $LEAGUE_ID..."
if ! ./google-civic-api.pl "$LEAGUE_ID" "$ROSTER_FILE" email-csv > "$FILTERED_CSV" 2>/dev/null; then
log "✗ Failed to filter roster for $LEAGUE_ID"
FAILED_LISTS+=("$LIST_ID")
rm -f "$FILTERED_CSV"
continue
fi
# Check if any members were found
MEMBER_COUNT=$(wc -l < "$FILTERED_CSV" | tr -d ' ')
log "Found $MEMBER_COUNT members for $LEAGUE_ID"
if [ "$MEMBER_COUNT" -eq 0 ]; then
log "⚠ No members found for League ID $LEAGUE_ID, skipping $LIST_ID"
rm -f "$FILTERED_CSV"
continue
fi
# Subscribe members to this list
log "Subscribing $MEMBER_COUNT members to $LIST_ID..."
if ./subscribe-members.sh "$FILTERED_CSV" "$CONNECTOR_PASSWORD" "$LIST_ID" "$CONNECTOR_URL"; then
log "✓ Successfully subscribed to $LIST_ID"
increment_count
# Clean up filtered CSV on success
rm -f "$FILTERED_CSV"
else
log "✗ Failed to subscribe to $LIST_ID"
FAILED_LISTS+=("$LIST_ID")
# Keep filtered CSV for debugging on failure
log " Filtered CSV retained: $FILTERED_CSV"
fi
done
# Summary
log ""
log "=========================================="
log "Subscription Summary"
log "=========================================="
log "Total Lists: ${#LIST_CONFIGS[@]}"
log "Successful: $SUCCESS_COUNT"
log "Failed: ${#FAILED_LISTS[@]}"
log "=========================================="
# Cleanup original roster if we downloaded it
if [ -z "$1" ] && [ -f "$ROSTER_FILE" ]; then
log "Cleaning up roster file..."
rm -f "$ROSTER_FILE"
fi
# If any lists failed, report error
if [ ${#FAILED_LISTS[@]} -gt 0 ]; then
error_exit "Failed to subscribe to the following list(s): ${FAILED_LISTS[*]}"
fi
log "Subscription process completed successfully"
exit 0

93
automation/start-automation.sh Executable file
View File

@ -0,0 +1,93 @@
#!/bin/bash
# Quick start script for LWVWV automation
# Usage: ./start-automation.sh [command]
#
# Commands:
# start - Start the automation container
# stop - Stop the automation container
# restart - Restart the automation container
# logs - View container logs
# run-now - Trigger subscription immediately
# status - Check container status
set -e
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
cd "$SCRIPT_DIR"
show_help() {
cat << EOF
LWVWV Automation Quick Start
Usage: $0 [command]
Commands:
start Start the automation container
stop Stop the automation container
restart Restart the automation container
logs View container logs
run-now Trigger subscription immediately
status Check container status
build Rebuild the container image
help Show this help message
Examples:
$0 start # Start automation
$0 run-now # Run subscription now
$0 logs # View logs
$0 logs -f # Follow logs
EOF
}
case "${1:-help}" in
start)
echo "Starting LWVWV automation container..."
docker-compose up -d
echo "Container started. Use '$0 logs' to view logs."
;;
stop)
echo "Stopping LWVWV automation container..."
docker-compose down
echo "Container stopped."
;;
restart)
echo "Restarting LWVWV automation container..."
docker-compose restart
echo "Container restarted."
;;
logs)
echo "Viewing container logs..."
shift # Remove 'logs' from args
docker-compose logs "$@"
;;
run-now)
echo "Triggering subscription immediately..."
docker exec lwvwv-subscriber /app/run-subscription.sh
;;
status)
echo "Container status:"
docker-compose ps
;;
build)
echo "Rebuilding container image..."
docker-compose build --no-cache
echo "Build complete. Use '$0 start' to start."
;;
help|--help|-h)
show_help
;;
*)
echo "Unknown command: $1"
show_help
exit 1
;;
esac

2
download-roster.js
View File

@ -26,7 +26,7 @@ function loadConfig() {
const config = loadConfig(); const config = loadConfig();
const PORTAL_URL = config.PORTAL_URL || 'https://portal.lwv.org'; const PORTAL_URL = config.PORTAL_URL || 'https://portal.lwv.org';
const MEMBERSHIP_URL = config.MEMBERSHIP_URL || 'https://portal.lwv.org/groups/43a93df1-901a-4676-88c3-f4ea430d4884/league_membership_state_view'; const MEMBERSHIP_URL = config.MEMBERSHIP_URL || 'https://portal.lwv.org/groups/53a93df1-9d4884/league_membership_state_view';
// Email config // Email config
const SMTP_HOST = config.SMTP_HOST || ''; const SMTP_HOST = config.SMTP_HOST || '';

64
subscribe-members.sh
View File

@ -1,36 +1,56 @@
#!/bin/bash #!/bin/bash
# Production batch subscription script for LWVWV members # Production batch subscription script for LWVWV members
# Usage: ./subscribe-members.sh <csv_file> <password> [list_id] [connector_url] # Usage: ./subscribe-members.sh [csv_file] [password] [list_id] [connector_url]
# #
# This script reads a CSV file (email,first_name,last_name) and subscribes # This script reads a CSV file (email,first_name,last_name) and subscribes
# all members to the specified Mailman list via the connector API. # all members to the specified Mailman list via the connector API.
# #
# Arguments: # Arguments (positional, optional if env vars are set):
# csv_file Path to CSV file with format: email,first_name,last_name # csv_file Path to CSV file with format: email,first_name,last_name
# password Secret password for the mailman-connector (required) # password Secret password for the mailman-connector
# list_id Target mailing list (default: members.lists.example.org) # list_id Target mailing list
# connector_url URL of the mailman-connector (default: https://mailman-connector.example.com) # connector_url URL of the mailman-connector
# #
# Example: # Environment Variables (used as defaults if positional args not provided):
# CONNECTOR_URL - URL of the mailman-connector
# CONNECTOR_PASSWORD - Secret password for the connector
# MAILMAN_LIST_ID - Target mailing list ID
#
# Examples:
# # Using positional arguments:
# ./subscribe-members.sh members.csv mysecretpassword members.lists.lwvwv.org https://connector.example.com
#
# # Using environment variables (from ../env):
# ./subscribe-members.sh members.csv
#
# # With some positional, some from env:
# ./subscribe-members.sh members.csv mysecretpassword # ./subscribe-members.sh members.csv mysecretpassword
# ./subscribe-members.sh members.csv mysecretpassword test.lists.example.org
# ./subscribe-members.sh members.csv mysecretpassword test.lists.example.org https://connector.example.com
# Show help # Show help
show_help() { show_help() {
cat << EOF cat << EOF
Usage: $0 <csv_file> <password> [list_id] [connector_url] Usage: $0 [csv_file] [password] [list_id] [connector_url]
Arguments: Arguments (all optional if environment variables are set):
csv_file Path to CSV file with format: email,first_name,last_name csv_file Path to CSV file with format: email,first_name,last_name
password Secret password for the mailman-connector (required) password Secret password for the mailman-connector
list_id Target mailing list (default: members.lists.example.org) list_id Target mailing list
connector_url URL of the mailman-connector (default: https://mailman-connector.example.com) connector_url URL of the mailman-connector
Environment Variables (used as defaults):
CONNECTOR_URL URL of the mailman-connector
CONNECTOR_PASSWORD Secret password for the connector
MAILMAN_LIST_ID Target mailing list ID
Examples: Examples:
$0 members.csv mysecretpassword # All positional:
$0 members.csv mysecretpassword test.lists.example.org $0 members.csv mysecretpassword members.lists.lwvwv.org https://connector.example.com
$0 members.csv mysecretpassword test.lists.example.org https://connector.example.com
# Using env vars for connector settings:
$0 members.csv
# Mixed:
$0 members.csv mysecretpassword members.lists.lwvwv.org
CSV Format: CSV Format:
The CSV file should have one member per line: The CSV file should have one member per line:
@ -46,22 +66,22 @@ if [ "$1" = "-h" ] || [ "$1" = "--help" ]; then
exit 0 exit 0
fi fi
# Parse arguments # Parse arguments - use positional if provided, fallback to env vars
CSV_FILE="${1:-}" CSV_FILE="${1:-}"
CONNECTOR_PASSWORD="${2:-}" CONNECTOR_PASSWORD="${2:-${CONNECTOR_PASSWORD:-}}"
LIST_ID="${3:-members.lists.example.org}" LIST_ID="${3:-${MAILMAN_LIST_ID:-members.lists.example.org}}"
CONNECTOR_URL="${4:-https://mailman-connector.example.com}" CONNECTOR_URL="${4:-${CONNECTOR_URL:-https://mailman-connector.example.com}}"
# Validation # Validation
if [ -z "$CSV_FILE" ]; then if [ -z "$CSV_FILE" ]; then
echo "Error: CSV file argument is required" echo "Error: CSV file is required"
echo "" echo ""
show_help show_help
exit 1 exit 1
fi fi
if [ -z "$CONNECTOR_PASSWORD" ]; then if [ -z "$CONNECTOR_PASSWORD" ]; then
echo "Error: Password argument is required" echo "Error: Password is required (provide as argument or set CONNECTOR_PASSWORD)"
echo "" echo ""
show_help show_help
exit 1 exit 1