240 lines
9.1 KiB
Markdown
240 lines
9.1 KiB
Markdown
# League of Women Voters of West Virginia
|
||
|
||
Custom-developed software designed for the League of Women Voters of West Virginia, as well as any state or local League affiliated with the League of Women Voters.
|
||
|
||
## IMPORTANT: Format for roster
|
||
|
||
**NEW (December 2025):** MAL are now included in the Local League Membership in Your State. This makes issues with the MAL‑specific list irrelevant, since it is no longer required.
|
||
|
||
**NEW (July 2025):** A new column has been added to the roster for MAL. To resolve this, remove the *Middle Name* column.
|
||
|
||
|
||
## REQUIRED FILE
|
||
|
||
A file called `env` is required in the same directory as the programs. This file contains configuration for both the Perl and JavaScript scripts.
|
||
|
||
### Complete env File Example
|
||
|
||
```
|
||
# Portal Configuration (for download-roster.js and save-session.js)
|
||
$PORTAL_URL = "https://portal.lwv.org";
|
||
$MEMBERSHIP_URL = "https://portal.lwv.org/groups/43a93df1-901a-4676-88c3-f4ea430d4884/league_membership_state_view";
|
||
|
||
# Email Notification Settings (for download-roster.js - error notifications via s-nail)
|
||
$SMTP_HOST = "mail.bikelover.org";
|
||
$SMTP_PORT = "587";
|
||
$SMTP_USE_STARTTLS = "true";
|
||
$SMTP_AUTH = "login";
|
||
$SMTP_USER = "your-email@lwv.org";
|
||
$SMTP_PASSWORD = "your-password";
|
||
$SSL_VERIFY_IGNORE = "ignore";
|
||
$EMAIL_FROM = "your-email@lwv.org";
|
||
$EMAIL_TO = "recipient@lwv.org";
|
||
|
||
# Google API Configuration (for google-civic-api.pl)
|
||
$key = "your-google-api-key";
|
||
$STATE = "WV";
|
||
$stateLeagueID = "WV000";
|
||
$localLeagueIDs = "WV102|WV103|WV112";
|
||
```
|
||
|
||
### Configuration Details
|
||
|
||
| Variable | Required | Description |
|
||
|----------|----------|-------------|
|
||
| `PORTAL_URL` | Yes | Base URL of the LWV portal (default: https://portal.lwv.org) |
|
||
| `MEMBERSHIP_URL` | Yes | Your league's membership page URL. Find this by navigating to your league's membership page in the portal and copying the URL |
|
||
| `SMTP_HOST` | No | SMTP server hostname for error email notifications (requires s-nail) |
|
||
| `SMTP_PORT` | No | SMTP server port (default: 587) |
|
||
| `SMTP_USE_STARTTLS` | No | Enable STARTTLS for SMTP (default: true) |
|
||
| `SMTP_AUTH` | No | SMTP auth type (default: login) |
|
||
| `SMTP_USER` | No | SMTP authentication username |
|
||
| `SMTP_PASSWORD` | No | SMTP authentication password |
|
||
| `SSL_VERIFY_IGNORE` | No | SSL verify setting for SMTP (default: ignore) |
|
||
| `EMAIL_FROM` | No | Sender email address for error notifications |
|
||
| `EMAIL_TO` | No | Recipient email address for error notifications |
|
||
| `MAILMAN_LIST_ID` | Yes | Target Mailman list ID(s) with optional League ID filter. Format: `list_id:league_id`. Examples: `members.lists.lwvwv.org:WV000` or `list1:WV000, list2:WV103`. If league omitted, defaults to ALL. Used by run-subscription.sh |
|
||
| `CONNECTOR_URL` | Yes | URL of mailman-connector API. Used by run-subscription.sh |
|
||
| `CONNECTOR_PASSWORD` | Yes | Secret password for connector API. Used by run-subscription.sh |
|
||
| `key` | Optional | Google Civic Information API key. See [Google API credentials](https://console.cloud.google.com/apis/credentials) - restrict key to the Google Civic Information API. Required for google-civic-api.pl. |
|
||
| `STATE` | Required* | Two-letter state code (e.g., WV). Required for google-civic-api.pl. |
|
||
| `stateLeagueID` | Required* | State league ID (e.g., WV000). Required for google-civic-api.pl. |
|
||
| `localLeagueIDs` | Required* | Pipe-separated list of local league IDs. Required for google-civic-api.pl. |
|
||
|
||
*Required for google-civic-api.pl
|
||
|
||
|
||
## JavaScript Scripts (download-roster.js and save-session.js)
|
||
|
||
These scripts automate downloading the membership roster CSV from the LWV portal.
|
||
|
||
### Prerequisites
|
||
|
||
```bash
|
||
# Install Node.js dependencies
|
||
npm install playwright
|
||
npx playwright install chromium
|
||
|
||
# Install s-nail (required for error email notifications)
|
||
# On Debian/Ubuntu:
|
||
sudo apt-get install s-nail
|
||
```
|
||
|
||
### First-Time Setup (Run Once)
|
||
|
||
Before running the download script for the first time, you need to save your browser session:
|
||
|
||
```bash
|
||
node save-session.js
|
||
```
|
||
|
||
This will:
|
||
1. Open a browser window
|
||
2. Prompt you to log in using the **magic link** method (email)
|
||
3. Wait for you to complete login and press Enter
|
||
4. Save your session to `.session.json`
|
||
|
||
The session is valid for 10 years (indefinite), but if it ever expires or fails, running `save-session.js` again will refresh it.
|
||
|
||
### Downloading the Roster
|
||
|
||
To download the latest membership CSV:
|
||
|
||
```bash
|
||
node download-roster.js
|
||
```
|
||
|
||
This will:
|
||
1. Load your saved session
|
||
2. Navigate to the membership page
|
||
3. Click Export → Download
|
||
4. Save the CSV file with timestamp (e.g., `league_membership_state_view_2026-04-15T05-00-24.csv`)
|
||
|
||
### Testing Error Email
|
||
|
||
To test that error emails work correctly:
|
||
|
||
```bash
|
||
# Temporarily rename the session file
|
||
mv .session.json .session.json.bak
|
||
|
||
# Run download - should fail and send error email
|
||
node download-roster.js
|
||
|
||
# Restore the session file
|
||
mv .session.json.bak .session.json
|
||
```
|
||
|
||
### Error Notifications
|
||
|
||
When `download-roster.js` fails (due to missing/expired session, download error, etc.), it will send an error email via **s-nail** to the address specified in `EMAIL_TO`. The email will include:
|
||
- The error message
|
||
- Instructions to run `save-session.js` to refresh the session
|
||
|
||
**Note:** s-nail must be installed for error emails to work. See Prerequisites above.
|
||
|
||
|
||
## LLAW Google Civic Information API Query version 2
|
||
|
||
Copyright (C) 2025 - by Jonathan Rosenbaum
|
||
|
||
This may be freely redistributed under the terms of the GNU General Public License
|
||
|
||
Usage: ./google-civic-api.pl google ./roster-file (queries Google Civic Api Delegate and Senate District for all LWVWV members)
|
||
./google-civic-api.pl WV000 '*.csv' (show all information for members at large, but do not query Google)
|
||
./google-civic-api.pl WV000 '*.csv' email (only show email addresses for members at large, but do not query Google)
|
||
|
||
1st argument can be one of these types:
|
||
google (all members with senate/delegate district query) which prints out this csv data:
|
||
'Name,Email,Phone,Address,Delegate District,Senate District,League ID,Join Date'
|
||
ALL (all members without senate/delegate district query)
|
||
League ID: WV000 (members at large) WV102 (Huntington) WV103 (Morgantown-Monogalia) WV112 (Jefferson)
|
||
|
||
2nd argument must be the location of the LWVWV roster file, and prints out this csv information:
|
||
Name,Email,Phone,Address,Join Date,
|
||
|
||
3rd argument 'email' will only print out the email addresses, and only works with the ALL or League ID type argument
|
||
|
||
You will want to send results to a file.
|
||
Example: ./google-civic-api.pl google '*.csv' > 2023-districts
|
||
|
||
## Automation
|
||
|
||
### run-subscription.sh
|
||
|
||
Master orchestration script that coordinates the entire subscription workflow:
|
||
1. Downloads roster from LWVWV portal (via download-roster.js)
|
||
2. Filters roster by League ID for each list (via google-civic-api.pl)
|
||
3. Subscribes members to one or more Mailman lists
|
||
|
||
**Multi-List Support with League Filtering:**
|
||
|
||
`MAILMAN_LIST_ID` uses colon-separated format to associate each list with a League ID:
|
||
```bash
|
||
# Format: list_id:league_id
|
||
$MAILMAN_LIST_ID = "members.lists.lwvwv.org:WV000, morgantown.lists.lwvwv.org:WV103"
|
||
```
|
||
|
||
- **list_id**: The Mailman list identifier (e.g., `members.lists.lwvwv.org`)
|
||
- **league_id**: The LWV League ID to filter members (e.g., `WV000`, `WV103`, `ALL`)
|
||
|
||
If `:league_id` is omitted, defaults to `ALL` (all members).
|
||
|
||
**Examples:**
|
||
```bash
|
||
# Single list, all members
|
||
$MAILMAN_LIST_ID = "members.lists.lwvwv.org"
|
||
|
||
# Single list, only WV000 (state) members
|
||
$MAILMAN_LIST_ID = "members.lists.lwvwv.org:WV000"
|
||
|
||
# Multiple lists with different league filters
|
||
$MAILMAN_LIST_ID = "members.lists.lwvwv.org:WV000, morgantown.lists.lwvwv.org:WV103, huntington.lists.lwvwv.org:WV102"
|
||
```
|
||
|
||
**Usage:**
|
||
```bash
|
||
./run-subscription.sh [csv_file]
|
||
```
|
||
|
||
If no CSV file is provided, the script will download the latest roster from the portal.
|
||
|
||
### subscribe-members.sh
|
||
|
||
Low-level script that subscribes members to a **single** Mailman 3 list via the connector API. Called by run-subscription.sh for each list when multiple lists are configured.
|
||
|
||
**Note:** This script handles ONE list per invocation. For multiple lists, use run-subscription.sh.
|
||
|
||
**Prerequisites:**
|
||
- `google-civic-api.pl` must be in the same directory or PATH
|
||
- Membership CSV file from LWVWV portal
|
||
|
||
**Usage:**
|
||
```bash
|
||
./subscribe-members.sh <csv_file> <password> [list_id] [connector_url]
|
||
```
|
||
|
||
**Arguments:**
|
||
- `csv_file` - Path to CSV file with format: email,first_name,last_name
|
||
- `password` - Secret password for the mailman-connector (required)
|
||
- `list_id` - Target mailing list (default: members.lists.example.org)
|
||
- `connector_url` - URL of the mailman-connector (default: https://mailman-connector.example.com)
|
||
|
||
**Examples:**
|
||
```bash
|
||
# Using default list and connector URL
|
||
./subscribe-members.sh members.csv mysecretpassword
|
||
|
||
# With specific list
|
||
./subscribe-members.sh members.csv mysecretpassword test.lists.example.org
|
||
|
||
# With specific list and connector URL
|
||
./subscribe-members.sh members.csv mysecretpassword test.lists.example.org https://connector.example.com
|
||
```
|
||
|
||
**How it works:**
|
||
1. Runs `./google-civic-api.pl ALL <membership_file> email-csv` to extract emails
|
||
2. Converts output to CSV format: `email,first_name,last_name`
|
||
3. Sends batch subscribe request to connector
|
||
|