bike/lwvwv
1
0
Fork 0
Code Issues 2 Pull Requests Projects Releases Wiki Activity
36 Commits 1 Branch 0 Tags
Go to file
Clone
Open with VS Code Open with VSCodium Open with Intellij IDEA
Download ZIP Download TAR.GZ Download BUNDLE

README.md

League of Women Voters of West Virginia

Custom-developed software designed for the League of Women Voters of West Virginia, as well as any state or local League affiliated with the League of Women Voters.

IMPORTANT: Format for roster

NEW (December 2025): MAL are now included in the Local League Membership in Your State. This makes issues with the MAL‑specific list irrelevant, since it is no longer required.

NEW (July 2025): A new column has been added to the roster for MAL. To resolve this, remove the Middle Name column.

REQUIRED FILE

A file called env is required in the same directory as the programs. This file contains configuration for both the Perl and JavaScript scripts.

Complete env File Example

# Portal Configuration (for download-roster.js and save-session.js)
$PORTAL_URL = "https://portal.lwv.org";
$MEMBERSHIP_URL = "https://portal.lwv.org/groups/43a93df1-901a-4676-88c3-f4ea430d4884/league_membership_state_view";

# Email Notification Settings (for download-roster.js - error notifications via s-nail)
$SMTP_HOST = "mail.bikelover.org";
$SMTP_PORT = "587";
$SMTP_USE_STARTTLS = "true";
$SMTP_AUTH = "login";
$SMTP_USER = "your-email@lwv.org";
$SMTP_PASSWORD = "your-password";
$SSL_VERIFY_IGNORE = "ignore";
$EMAIL_FROM = "your-email@lwv.org";
$EMAIL_TO = "recipient@lwv.org";

# Google API Configuration (for google-civic-api.pl)
$key = "your-google-api-key";
$STATE = "WV";
$stateLeagueID = "WV000";
$localLeagueIDs = "WV102|WV103|WV112";

Configuration Details

Variable Required Description
PORTAL_URL Yes Base URL of the LWV portal (default: https://portal.lwv.org)
MEMBERSHIP_URL Yes Your league's membership page URL. Find this by navigating to your league's membership page in the portal and copying the URL
SMTP_HOST No SMTP server hostname for error email notifications (requires s-nail)
SMTP_PORT No SMTP server port (default: 587)
SMTP_USE_STARTTLS No Enable STARTTLS for SMTP (default: true)
SMTP_AUTH No SMTP auth type (default: login)
SMTP_USER No SMTP authentication username
SMTP_PASSWORD No SMTP authentication password
SSL_VERIFY_IGNORE No SSL verify setting for SMTP (default: ignore)
EMAIL_FROM No Sender email address for error notifications
EMAIL_TO No Recipient email address for error notifications
MAILMAN_LIST_ID Yes Target Mailman list ID(s) with optional League ID filter. Format: list_id:league_id. Examples: members.lists.lwvwv.org:WV000 or list1:WV000, list2:WV103. If league omitted, defaults to ALL. Used by run-subscription.sh
CONNECTOR_URL Yes URL of mailman-connector API. Used by run-subscription.sh
CONNECTOR_PASSWORD Yes Secret password for connector API. Used by run-subscription.sh
key Optional Google Civic Information API key. See Google API credentials - restrict key to the Google Civic Information API. Required for google-civic-api.pl.
STATE Required* Two-letter state code (e.g., WV). Required for google-civic-api.pl.
stateLeagueID Required* State league ID (e.g., WV000). Required for google-civic-api.pl.
localLeagueIDs Required* Pipe-separated list of local league IDs. Required for google-civic-api.pl.

*Required for google-civic-api.pl

JavaScript Scripts (download-roster.js and save-session.js)

These scripts automate downloading the membership roster CSV from the LWV portal.

Prerequisites

# Install Node.js dependencies
npm install playwright
npx playwright install chromium

# Install s-nail (required for error email notifications)
# On Debian/Ubuntu:
sudo apt-get install s-nail

First-Time Setup (Run Once)

Before running the download script for the first time, you need to save your browser session:

node save-session.js

This will:

  1. Open a browser window
  2. Prompt you to log in using the magic link method (email)
  3. Wait for you to complete login and press Enter
  4. Save your session to .session.json

The session is valid for 10 years (indefinite), but if it ever expires or fails, running save-session.js again will refresh it.

Downloading the Roster

To download the latest membership CSV:

node download-roster.js

This will:

  1. Load your saved session
  2. Navigate to the membership page
  3. Click Export → Download
  4. Save the CSV file with timestamp (e.g., league_membership_state_view_2026-04-15T05-00-24.csv)

Testing Error Email

To test that error emails work correctly:

# Temporarily rename the session file
mv .session.json .session.json.bak

# Run download - should fail and send error email
node download-roster.js

# Restore the session file
mv .session.json.bak .session.json

Error Notifications

When download-roster.js fails (due to missing/expired session, download error, etc.), it will send an error email via s-nail to the address specified in EMAIL_TO. The email will include:

  • The error message
  • Instructions to run save-session.js to refresh the session

Note: s-nail must be installed for error emails to work. See Prerequisites above.

Perl Script (google-civic-api.pl)

The original script for processing LWVWV membership data. It queries the Google Civic Information API for legislative district information and converts roster CSV files to various formats.

Description

Copyright (C) 2025 - by Jonathan Rosenbaum

This may be freely redistributed under the terms of the GNU General Public License

Usage

./google-civic-api.pl google ./roster-file  (queries Google Civic Api Delegate and Senate District for all LWVWV members)
./google-civic-api.pl WV000 '*.csv' (show all information for members at large, but do not query Google)
./google-civic-api.pl WV000 '*.csv' email (only show email addresses for members at large, but do not query Google)

Arguments

  1. Query Type:

    • google - All members with senate/delegate district query
    • ALL - All members without district query
    • League ID - Specific league (e.g., WV000, WV102, WV103, WV112)

  2. Roster file - Path to CSV file

  3. Output format (optional):

    • email - Output only email addresses
    • email-csv - Output CSV format: email,first_name,last_name

Output Formats

Full CSV (google type):

Name,Email,Phone,Address,Delegate District,Senate District,League ID,Join Date

Email only:

First Last <email@example.com>

Email CSV (for subscription):

email@example.com,FirstName,LastName

Examples

# Query Google API for all members with district info
./google-civic-api.pl google '*.csv' > 2023-districts.csv

# Get emails for state members only (WV000)
./google-civic-api.pl WV000 '*.csv' email > state-emails.txt

# Generate subscription CSV for all members
./google-civic-api.pl ALL '*.csv' email-csv > members.csv

Column Name Based Parsing

The script now uses column names from the CSV header rather than fixed positions, making it robust against column reordering. If required columns are missing, it will display an error showing which columns were expected and which are available.

Docker Automation

Copyright (C) 2025 - by Jonathan Rosenbaum

Automation

run-subscription.sh

Master orchestration script that coordinates the entire subscription workflow:

  1. Downloads roster from LWVWV portal (via download-roster.js)
  2. Filters roster by League ID for each list (via google-civic-api.pl)
  3. Subscribes members to one or more Mailman lists

Multi-List Support with League Filtering:

MAILMAN_LIST_ID uses colon-separated format to associate each list with a League ID:

# Format: list_id:league_id
$MAILMAN_LIST_ID = "members.lists.lwvwv.org:WV000, morgantown.lists.lwvwv.org:WV103"
  • list_id: The Mailman list identifier (e.g., members.lists.lwvwv.org)
  • league_id: The LWV League ID to filter members (e.g., WV000, WV103, ALL)

If :league_id is omitted, defaults to ALL (all members).

Examples:

# Single list, all members
$MAILMAN_LIST_ID = "members.lists.lwvwv.org"

# Single list, only WV000 (state) members  
$MAILMAN_LIST_ID = "members.lists.lwvwv.org:WV000"

# Multiple lists with different league filters
$MAILMAN_LIST_ID = "members.lists.lwvwv.org:WV000, morgantown.lists.lwvwv.org:WV103, huntington.lists.lwvwv.org:WV102"

Usage:

./run-subscription.sh [csv_file]

If no CSV file is provided, the script will download the latest roster from the portal.

subscribe-members.sh

Low-level script that subscribes members to a single Mailman 3 list via the connector API. Called by run-subscription.sh for each list when multiple lists are configured.

Note: This script handles ONE list per invocation. For multiple lists, use run-subscription.sh.

Prerequisites:

  • google-civic-api.pl must be in the same directory or PATH
  • Membership CSV file from LWVWV portal

Usage:

./subscribe-members.sh <csv_file> <password> [list_id] [connector_url]

Arguments:

  • csv_file - Path to CSV file with format: email,first_name,last_name
  • password - Secret password for the mailman-connector (required)
  • list_id - Target mailing list (default: members.lists.example.org)
  • connector_url - URL of the mailman-connector (default: https://mailman-connector.example.com)

Examples:

# Using default list and connector URL
./subscribe-members.sh members.csv mysecretpassword

# With specific list
./subscribe-members.sh members.csv mysecretpassword test.lists.example.org

# With specific list and connector URL
./subscribe-members.sh members.csv mysecretpassword test.lists.example.org https://connector.example.com

How it works:

  1. Runs ./google-civic-api.pl ALL <membership_file> email-csv to extract emails
  2. Converts output to CSV format: email,first_name,last_name
  3. Sends batch subscribe request to connector

Related Projects

  • mailman-connector - REST API service for batch-subscribing members to Mailman 3 lists (used by automation)
  • automation/ - Docker-based automation with Ofelia scheduler for hands-off operation. See automation/README.md for Docker setup and configuration.
Description
Custom-developed software designed for the League of Women Voters of West Virginia, as well as any state or local League affiliated with the League of Women Voters.
Readme GPL-3.0 195 KiB
Languages
JavaScript 34.4%
Shell 32.4%
Perl 30.9%
Dockerfile 2.3%