bryanb
/
Big-Link-Man
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Story 3.4 generated by scrum manager

main
PeninsulaInd 2025-10-21 14:58:11 -05:00
parent ee66d9e894
commit de9c015afd
5 changed files with 853 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

7
STORY_3.3_COMPLETE.md
View File

@ -294,10 +294,15 @@ Story 3.3 is complete as specified. Potential future improvements:
2. **Custom See Also Heading**: Make "See Also" heading configurable
3. **Link Position Strategy**: Preference for intro/body/conclusion placement
4. **Anchor Text Variety**: More sophisticated rotation strategies
5. **About/Privacy/Contact Pages**: Create pages to match nav menu links
5. ~~**About/Privacy/Contact Pages**: Create pages to match nav menu links~~ ✅ **PROMOTED TO STORY 3.4**
None of these are required for Story 3.3 completion.
### Story 3.4 Emerged from Story 3.3
During Story 3.3 implementation, we added navigation menus to all templates that link to `about.html`, `contact.html`, and `privacy.html`. However, these pages don't exist, creating broken links. This was identified as a high-priority issue and promoted to **Story 3.4: Boilerplate Site Pages**.
See: `docs/stories/story-3.4-boilerplate-site-pages.md`
---
## Sign-Off

277
STORY_3.4_CREATED.md 100644
View File

@ -0,0 +1,277 @@
# Story 3.4: Boilerplate Site Pages - CREATED
**Status**: Specification Complete, Ready for Implementation
**Date Created**: October 21, 2025
---
## Summary
Story 3.4 has been created to address the broken navigation menu links introduced in Story 3.3.
### The Problem
In Story 3.3, we added navigation menus to all HTML templates:
```html
<nav>
<ul>
<li><a href="/index.html">Home</a></li>
<li><a href="about.html">About</a></li>
<li><a href="privacy.html">Privacy</a></li>
<li><a href="contact.html">Contact</a></li>
</ul>
</nav>
```
However, we never created the `about.html`, `contact.html`, or `privacy.html` pages, resulting in broken links.
### The Solution
Story 3.4 will automatically generate these boilerplate pages for each site during batch generation.
---
## What Will Be Delivered
### 1. Three Boilerplate Pages Per Site (Heading Only)
- **About Page** (`about.html`) - `<h1>About Us</h1>` + template/navigation
- **Contact Page** (`contact.html`) - `<h1>Contact</h1>` + template/navigation
- **Privacy Policy** (`privacy.html`) - `<h1>Privacy Policy</h1>` + template/navigation
All pages have just a heading wrapped in the template structure. No other content text. User can add content manually later if desired.
### 2. Database Storage
New `site_pages` table stores pages separately from articles:
```sql
CREATE TABLE site_pages (
id INTEGER PRIMARY KEY AUTOINCREMENT,
site_deployment_id INTEGER NOT NULL,
page_type VARCHAR(20) NOT NULL, -- about, contact, privacy
content TEXT NOT NULL,
created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
updated_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
FOREIGN KEY (site_deployment_id) REFERENCES site_deployments(id),
UNIQUE (site_deployment_id, page_type)
);
```
### 3. Template Integration
- Pages use the same template as articles on the same site
- Professional, visually consistent with article content
- Navigation menu included (which links to these same pages)
### 4. Smart Generation
- Generated ONLY when new sites are created (not for existing sites)
- One-time backfill script for all existing imported sites
- Integrated into site creation workflow (not batch generation)
---
## Implementation Scope
### Effort Estimate
**15 story points** (reduced from 20, approximately 1.5-2 days of development)
Simplified due to empty pages - no complex content generation needed.
### Key Components
1. **Database Schema** (2 points)
- New `SitePage` model
- Migration script
- Repository layer
2. **Page Content Templates** (1 point - simplified)
- Heading-only page content
- Returns `<h1>About Us</h1>`, `<h1>Contact</h1>`, `<h1>Privacy Policy</h1>`
- No complex content generation
3. **Generation Logic** (2 points - simplified)
- Generate heading-only pages for each site
- Wrap heading in HTML template
- Store in database
4. **Site Creation Integration** (2 points)
- Hook into `site_provisioning.py`
- Generate pages when new sites are created
- Handle errors gracefully
5. **Backfill Script** (2 points)
- CLI script to generate pages for all existing sites
- Dry-run mode for safety
- Progress reporting and error handling
6. **Testing** (3 points - simplified)
- Unit tests for heading-only page generation
- Integration tests with site creation
- Backfill script testing
- Template application tests
**Total: 15 story points** (reduced from 20)
---
## Integration Point
Story 3.4 hooks into site creation, not batch generation:
### One-Time Setup (Existing Sites)
```bash
# Backfill all existing imported sites (hundreds of sites)
uv run python scripts/backfill_site_pages.py \
--username admin \
--password yourpass \
--template basic
# Output: Generated pages for 423 sites
```
### Ongoing (New Sites Only)
```
When creating new sites:
1. Create Storage Zone (bunny.net)
2. Create Pull Zone (bunny.net)
3. Save to database
4. ✨ Generate boilerplate pages (Story 3.4) ← NEW
5. Return site ready to use
Triggered by:
- provision-site CLI command
- auto_create_sites in job config
- create_sites_for_keywords in job config
```
### Batch Generation (Unchanged)
```
1. Generate articles (Epic 2)
2. Assign sites (Story 3.1) ← May use existing sites with pages
3. Generate URLs (Story 3.1)
4. Find tiered links (Story 3.2)
5. Inject interlinks (Story 3.3)
6. Apply templates (Story 2.4)
7. Deploy (Epic 4) ← Pages already exist on site
```
---
## Files Created
### Documentation
- `docs/stories/story-3.4-boilerplate-site-pages.md` (full specification)
- `docs/prd/epic-3-pre-deployment.md` (updated to include Story 3.4)
- `STORY_3.4_CREATED.md` (this summary)
### Implementation Files (To Be Created)
- `src/generation/page_templates.py` - Generic page content
- `src/generation/site_page_generator.py` - Page generation logic
- `src/database/models.py` - SitePage model (update)
- `scripts/migrate_add_site_pages.py` - Database migration
- `scripts/backfill_site_pages.py` - One-time backfill script
- `tests/unit/test_site_page_generator.py` - Unit tests
- `tests/integration/test_site_pages_integration.py` - Integration tests
- `tests/unit/test_backfill_script.py` - Backfill script tests
---
## Dependencies
### Requires (Already Complete)
- ✅ Story 3.1: Site assignment (need to know which sites are in use)
- ✅ Story 3.3: Navigation menus (these pages fulfill those links)
- ✅ Story 2.4: Template service (for applying HTML templates)
- ✅ Story 1.6: SiteDeployment table (for site relationships)
### Enables
- Story 4.1: Deployment (pages will be deployed along with articles)
- Complete, professional-looking sites with working navigation
---
## Example Output
### Site Structure After Story 3.4
```
https://example.com/
├── index.html (homepage - future/Epic 4)
├── about.html ← NEW (Story 3.4)
├── contact.html ← NEW (Story 3.4)
├── privacy.html ← NEW (Story 3.4)
├── how-to-fix-your-engine.html (article)
├── engine-maintenance-tips.html (article)
└── best-engine-oil-brands.html (article)
```
### About Page Preview (Heading Only)
```html
<!DOCTYPE html>
<html>
<head>
<title>About Us</title>
<!-- Same template/styling as articles -->
</head>
<body>
<nav>
<ul>
<li><a href="/index.html">Home</a></li>
<li><a href="about.html">About</a></li>
<li><a href="privacy.html">Privacy</a></li>
<li><a href="contact.html">Contact</a></li>
</ul>
</nav>
<main>
<h1>About Us</h1>
<!-- No other content - user can add manually later if desired -->
</main>
</body>
</html>
```
**Why heading-only pages?**
- Fixes broken navigation links (no 404 errors)
- Better UX than completely blank (user sees page title)
- Minimal implementation effort
- User can customize specific sites later if needed
- Deployment ready as-is
---
## Next Steps
### Option 1: Implement Now
- Start implementation of Story 3.4
- Fixes broken navigation links
- Makes sites look complete and professional
### Option 2: Defer to Later
- Add to backlog/technical debt
- Focus on Epic 4 deployment first
- Sites work but have broken nav links temporarily
### Option 3: Minimal Quick Fix
- Create simple placeholder pages without full story implementation
- Just enough to avoid 404 errors
- Come back later for full implementation
---
## Recommendation
**Implement Story 3.4 before Epic 4 deployment** because:
1. Sites look unprofessional with broken nav links
2. Fixes 404 errors on every deployed site
3. Only 15 story points (1.5-2 days) - simplified implementation
4. Empty pages are deployment-ready
5. User can add content to specific pages later if desired
The alternative is to deploy with broken links and fix later, but that creates technical debt and poor user experience.
**Simplified approach:** Pages have heading only (e.g., `<h1>About Us</h1>`), no body content. This makes implementation faster while still fixing the broken link issue and providing better UX than completely blank pages.
---
**Created by**: AI Code Assistant
**Created on**: October 21, 2025
**Next**: Decide when to implement Story 3.4 (now vs. later vs. minimal fix)

13
docs/prd/epic-3-pre-deployment.md
View File

@ -43,3 +43,16 @@ A script iterates through each new article in the batch.
* **If Tier 1:** It scans the HTML for anchor text from the T1 list and links the first instance to the `money_site_url`.
* **If Tier 2 or higher:** It scans the HTML for anchor text from the appropriate tier's list. For each of the 2-4 anchor texts found, it inserts a link to one of the `lower_tier_urls`.
* The script produces the **final, fully interlinked HTML content,** ready for deployment in Epic 4.
### Story 3.4: Generate Boilerplate Site Pages
**As a developer**, I want to automatically generate boilerplate `about.html`, `contact.html`, and `privacy.html` pages for each site in my batch, so that the navigation menu links from Story 3.3 work and the sites appear complete.
**Acceptance Criteria:**
* A function generates three boilerplate pages for each unique site in the batch.
* Pages created: `about.html`, `contact.html`, `privacy.html`
* Each page uses the same template as the articles for that site (basic/modern/classic/minimal).
* Pages contain professional, generic content suitable for any niche.
* Privacy policy is comprehensive and legally sound (generic template).
* Pages are stored in database and associated with the correct site.
* Pages are generated once per site (skip if already exist from previous batches).
* Navigation menu links from Story 3.3 now point to actual pages instead of being broken.

519
docs/stories/story-3.4-boilerplate-site-pages.md 100644
View File

@ -0,0 +1,519 @@
# Story 3.4: Generate Boilerplate Site Pages
## Status
Not Started
## Story
**As a developer**, I want to automatically generate boilerplate `about.html`, `contact.html`, and `privacy.html` pages for each site in my batch, so that the navigation menu links from Story 3.3 work and the sites appear complete.
## Context
- Story 3.3 added navigation menus to all HTML templates with links to:
- `/index.html` (homepage)
- `about.html` (about page)
- `privacy.html` (privacy policy)
- `contact.html` (contact page)
- Currently, these pages don't exist, creating broken links
- Each site needs its own set of these pages
- Pages should use the same template as the articles (basic/modern/classic/minimal)
- Content should be generic but professional enough for a real site
- Privacy policy needs to be comprehensive and legally sound (generic template)
## Acceptance Criteria
### Core Functionality
- A function generates the three boilerplate pages for a given site
- Pages are created AFTER articles are generated but BEFORE deployment
- Each page uses the same template as the articles for that site
- Pages are stored in the database for deployment
- Pages are associated with the correct site (via `site_deployment_id`)
### Page Content Requirements
#### About Page (`about.html`)
- Empty page with just the template applied
- No content text required (just template navigation/structure)
- User can add content later if needed
#### Contact Page (`contact.html`)
- Empty page with just the template applied
- No content text required (just template navigation/structure)
- User can add content later if needed
#### Privacy Policy (`privacy.html`)
- **Option 1 (Minimal):** Empty page like about/contact
- No content text required (just template navigation/structure)
- User can add content later if needed
**Decision:** Start with Option 1 (empty pages) for all three pages. Privacy policy content can be added later via backfill update or manual edit if needed.
### Template Integration
- Use same template engine as article content (`src/templating/service.py`)
- Apply the site's assigned template (basic/modern/classic/minimal)
- Pages should visually match the articles on the same site
- Include navigation menu (which will link to these same pages)
### Database Storage
- Create new `site_pages` table (clean separation from articles):
- `id`, `site_deployment_id`, `page_type`, `content`, `created_at`, `updated_at`
- Foreign key to `site_deployments` with CASCADE delete
- Unique constraint on (site_deployment_id, page_type)
- Indexes on site_deployment_id and page_type
- Each site can have one of each page type (about, contact, privacy)
- Pages are fundamentally different from articles, deserve own table
### URL Generation
- Pages use simple filenames: `about.html`, `contact.html`, `privacy.html`
- Full URLs: `https://{hostname}/about.html`
- No slug generation needed (fixed filenames)
- Pages tracked separately from article URLs
### Integration Point
- Hook into batch generation workflow in `src/generation/batch_processor.py`
- After site assignment (Story 3.1) and before deployment (Epic 4)
- Generate pages ONLY for newly created sites (not existing sites)
- One-time backfill script to add pages to all existing imported sites
### Two Use Cases
1. **One-time backfill**: Script to generate pages for all existing sites in database (hundreds of sites)
2. **Ongoing generation**: Automatically generate pages only when new sites are created (provision-site, auto_create_sites, etc.)
## Tasks / Subtasks
### 1. Create SitePage Database Table
**Effort:** 2 story points
- [ ] Create new `site_pages` table with schema:
- `id`, `site_deployment_id`, `page_type`, `content`, `created_at`, `updated_at`
- [ ] Add `SitePage` model to `src/database/models.py`
- [ ] Create migration script `scripts/migrate_add_site_pages.py`
- [ ] Add unique constraint on (site_deployment_id, page_type)
- [ ] Add indexes on site_deployment_id and page_type
- [ ] Add CASCADE delete (if site deleted, pages deleted)
- [ ] Test migration on development database
### 2. Create SitePage Repository
**Effort:** 2 story points
- [ ] Create `ISitePageRepository` interface in `src/database/interfaces.py`:
- `create(site_deployment_id, page_type, content) -> SitePage`
- `get_by_site(site_deployment_id) -> List[SitePage]`
- `get_by_site_and_type(site_deployment_id, page_type) -> Optional[SitePage]`
- `update_content(page_id, content) -> SitePage`
- `exists(site_deployment_id, page_type) -> bool`
- `delete(page_id) -> bool`
- [ ] Implement `SitePageRepository` in `src/database/repositories.py`
- [ ] Add to repository factory/dependency injection
### 3. Create Page Content Templates (SIMPLIFIED)
**Effort:** 1 story point (reduced from 3)
- [ ] Create `src/generation/page_templates.py` module
- [ ] Implement `get_page_content(page_type: str, domain: str) -> str`:
- Returns just a heading: `<h1>About Us</h1>`, `<h1>Contact</h1>`, `<h1>Privacy Policy</h1>`
- All three pages use same heading-only approach
- No other content text
- [ ] No need for extensive content generation
- [ ] Pages are just placeholders until user adds content manually
### 4. Implement Page Generation Logic (SIMPLIFIED)
**Effort:** 2 story points (reduced from 3)
- [ ] Create `src/generation/site_page_generator.py` module
- [ ] Implement `generate_site_pages(site_deployment: SiteDeployment, template_name: str, page_repo, template_service) -> List[SitePage]`:
- Get domain from site (custom_hostname or bcdn_hostname)
- For each page type (about, contact, privacy):
- Get heading-only content from `page_templates.py`
- Wrap heading in HTML template using `template_service`
- Store page in database
- Return list of created pages
- [ ] Pages have just heading (e.g., `<h1>About Us</h1>`) wrapped in template
- [ ] Log page generation at INFO level
### 5. Integrate with Site Creation (Not Batch Processor)
**Effort:** 2 story points
- [ ] Update `src/generation/site_provisioning.py`:
- After creating new site via bunny.net API, generate boilerplate pages
- Call `generate_site_pages()` immediately after site creation
- Log page generation results
- [ ] Update `provision-site` CLI command:
- Generate pages after site is provisioned
- [ ] Handle errors gracefully (log warning if page generation fails, continue with site creation)
- [ ] **DO NOT generate pages in batch processor** (only for new sites, not existing sites)
### 6. Update Template Service
**Effort:** 1 story point
- [ ] Verify `src/templating/service.py` can handle page content:
- Pages don't have titles/outlines like articles
- May need simpler template application for pages
- Ensure navigation menu is included
- [ ] Add helper method if needed: `apply_template_to_page(content, template_name, domain)`
### 7. Create Backfill Script for Existing Sites
**Effort:** 2 story points
- [ ] Create `scripts/backfill_site_pages.py`:
- Query all sites in database that don't have pages
- For each site: generate about, contact, privacy pages
- Use default template (or infer from site name if possible)
- Progress reporting (e.g., "Generating pages for site 50/400...")
- Dry-run mode to preview changes
- CLI arguments: `--dry-run`, `--template`, `--batch-size`
- [ ] Add error handling for individual site failures (continue with next site)
- [ ] Log results: successful, failed, skipped counts
### 8. Homepage Generation (Optional - Deferred)
**Effort:** 2 story points (if implemented)
- [ ] **DEFER to Epic 4 or later**
- [ ] Homepage (`index.html`) requires knowing all articles on the site
- [ ] Not needed for Story 3.4 (navigation menu links to `/index.html` can 404 for now)
- [ ] Document in technical notes
### 9. Unit Tests (SIMPLIFIED)
**Effort:** 2 story points (reduced from 3)
- [ ] Test heading-only page content generation
- [ ] Test domain extraction from SiteDeployment (custom vs bcdn hostname)
- [ ] Test page HTML wrapping with each template type
- [ ] Test SitePage repository CRUD operations
- [ ] Test duplicate page prevention (unique constraint)
- [ ] Test page generation for single site
- [ ] Test backfill script logic
- [ ] Mock template service and repositories
- [ ] Achieve >80% code coverage for new modules
### 10. Integration Tests (SIMPLIFIED)
**Effort:** 1 story point (reduced from 2)
- [ ] Test site creation triggers page generation
- [ ] Test with different template types (basic, modern, classic, minimal)
- [ ] Test with custom domain sites vs bunny.net-only sites
- [ ] Test pages stored correctly in database
- [ ] Test backfill script on real database
- [ ] Verify navigation menu links work (pages exist at expected paths)
## Technical Notes
### SitePage Model
```python
class SitePage(Base):
__tablename__ = "site_pages"
id: Mapped[int] = mapped_column(Integer, primary_key=True, autoincrement=True)
site_deployment_id: Mapped[int] = mapped_column(
Integer,
ForeignKey('site_deployments.id', ondelete='CASCADE'),
nullable=False
)
page_type: Mapped[str] = mapped_column(String(20), nullable=False) # about, contact, privacy, homepage
content: Mapped[str] = mapped_column(Text, nullable=False) # Full HTML
created_at: Mapped[datetime] = mapped_column(DateTime, default=datetime.utcnow, nullable=False)
updated_at: Mapped[datetime] = mapped_column(
DateTime,
default=datetime.utcnow,
onupdate=datetime.utcnow,
nullable=False
)
# Relationships
site_deployment: Mapped["SiteDeployment"] = relationship("SiteDeployment", back_populates="pages")
# Unique constraint
__table_args__ = (
UniqueConstraint('site_deployment_id', 'page_type', name='uq_site_page_type'),
)
```
### Database Migration
```sql
CREATE TABLE site_pages (
id INTEGER PRIMARY KEY AUTOINCREMENT,
site_deployment_id INTEGER NOT NULL,
page_type VARCHAR(20) NOT NULL,
content TEXT NOT NULL,
created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
updated_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
FOREIGN KEY (site_deployment_id) REFERENCES site_deployments(id) ON DELETE CASCADE,
UNIQUE (site_deployment_id, page_type)
);
CREATE INDEX idx_site_pages_site ON site_pages(site_deployment_id);
CREATE INDEX idx_site_pages_type ON site_pages(page_type);
```
### Page Content Template Examples (SIMPLIFIED)
#### Implementation - Heading Only
```python
# src/generation/page_templates.py
def get_page_content(page_type: str, domain: str) -> str:
"""
Generate minimal content for boilerplate pages.
Just a heading - no other content text.
"""
page_titles = {
"about": "About Us",
"contact": "Contact",
"privacy": "Privacy Policy"
}
return f"<h1>{page_titles.get(page_type, page_type.title())}</h1>"
```
#### Result - Heading Only Example
```html
<!DOCTYPE html>
<html>
<head>
<title>About Us</title>
<!-- Same template/styling as articles -->
</head>
<body>
<nav>
<ul>
<li><a href="/index.html">Home</a></li>
<li><a href="about.html">About</a></li>
<li><a href="privacy.html">Privacy</a></li>
<li><a href="contact.html">Contact</a></li>
</ul>
</nav>
<main>
<h1>About Us</h1>
<!-- No other content - user can add later if needed -->
</main>
</body>
</html>
```
#### Why Heading-Only Pages Work
1. **Fixes broken nav links** - Pages exist, no 404 errors
2. **Better UX than completely empty** - User sees something when they click the link
3. **User can customize** - Add content manually later for specific sites
4. **Minimal effort** - No need to generate/maintain generic content
5. **Deployment ready** - Pages can be deployed as-is
6. **Future enhancement** - Can add content generation later if needed
### Integration with Site Creation
```python
# In src/generation/site_provisioning.py
def create_bunnynet_site(name_prefix: str, region: str = "DE", template: str = "basic"):
# Step 1: Create Storage Zone
storage = bunny_client.create_storage_zone(...)
# Step 2: Create Pull Zone
pull = bunny_client.create_pull_zone(...)
# Step 3: Save to database
site = site_repo.create(...)
# Step 4: Generate boilerplate pages (NEW - Story 3.4)
logger.info(f"Generating boilerplate pages for new site {site.id}...")
try:
generate_site_pages(site, template, page_repo, template_service)
logger.info(f"Successfully created about, contact, privacy pages for site {site.id}")
except Exception as e:
logger.warning(f"Failed to generate pages for site {site.id}: {e}")
# Don't fail site creation if page generation fails
return site
```
### Backfill Script Usage
```bash
# One-time backfill for all existing sites (dry-run first)
uv run python scripts/backfill_site_pages.py \
--username admin \
--password yourpass \
--template basic \
--dry-run
# Output:
# Found 423 sites without boilerplate pages
# [DRY RUN] Would generate pages for site 1 (www.example.com)
# [DRY RUN] Would generate pages for site 2 (site123.b-cdn.net)
# ...
# [DRY RUN] Total: 423 sites would be updated
# Actually generate pages
uv run python scripts/backfill_site_pages.py \
--username admin \
--password yourpass \
--template basic
# Output:
# Generating pages for site 1/423 (www.example.com)... ✓
# Generating pages for site 2/423 (site123.b-cdn.net)... ✓
# ...
# Complete: 423 successful, 0 failed, 0 skipped
# Use different template per site (default: basic)
uv run python scripts/backfill_site_pages.py \
--username admin \
--password yourpass \
--template modern \
--batch-size 50 # Process 50 sites at a time
```
### Page URL Structure
```
Homepage: https://example.com/index.html
About: https://example.com/about.html
Contact: https://example.com/contact.html
Privacy: https://example.com/privacy.html
Article 1: https://example.com/how-to-fix-engines.html
Article 2: https://example.com/engine-maintenance-tips.html
```
### Template Application Example
```python
# For articles (existing)
template_service.apply_template(
content=article.content,
template_name="modern",
title=article.title,
meta_description=article.meta_description,
url=article_url
)
# For pages (new)
template_service.apply_template_to_page(
content=page_content, # Markdown or HTML from page_templates.py
template_name="modern",
page_title="About Us", # Static title
domain=site.custom_hostname or site.pull_zone_bcdn_hostname
)
```
### Backfill Script Implementation
```python
# scripts/backfill_site_pages.py
def backfill_site_pages(
page_repo,
site_repo,
template_service,
template: str = "basic",
dry_run: bool = False,
batch_size: int = 100
):
"""Generate boilerplate pages for all sites that don't have them"""
# Get all sites
all_sites = site_repo.get_all()
logger.info(f"Found {len(all_sites)} total sites in database")
# Filter to sites without pages
sites_needing_pages = []
for site in all_sites:
existing_pages = page_repo.get_by_site(site.id)
if len(existing_pages) < 3: # Should have about, contact, privacy
sites_needing_pages.append(site)
logger.info(f"Found {len(sites_needing_pages)} sites without boilerplate pages")
if dry_run:
for site in sites_needing_pages:
domain = site.custom_hostname or site.pull_zone_bcdn_hostname
logger.info(f"[DRY RUN] Would generate pages for site {site.id} ({domain})")
logger.info(f"[DRY RUN] Total: {len(sites_needing_pages)} sites would be updated")
return
# Generate pages for each site
successful = 0
failed = 0
for idx, site in enumerate(sites_needing_pages, 1):
domain = site.custom_hostname or site.pull_zone_bcdn_hostname
logger.info(f"Generating pages for site {idx}/{len(sites_needing_pages)} ({domain})...")
try:
generate_site_pages(site, template, page_repo, template_service)
successful += 1
except Exception as e:
logger.error(f"Failed to generate pages for site {site.id}: {e}")
failed += 1
# Progress checkpoint every batch_size sites
if idx % batch_size == 0:
logger.info(f"Progress: {idx}/{len(sites_needing_pages)} sites processed")
logger.info(f"Complete: {successful} successful, {failed} failed")
```
### Domain Extraction
```python
def get_domain_from_site(site_deployment: SiteDeployment) -> str:
"""Extract domain for use in page content (email addresses, etc.)"""
if site_deployment.custom_hostname:
return site_deployment.custom_hostname
else:
return site_deployment.pull_zone_bcdn_hostname
```
### Privacy Policy Legal Note
The privacy policy template should be:
- Generic enough to apply to blog/content sites
- Comprehensive enough to cover common scenarios (cookies, analytics, third-party links)
- NOT legal advice - users should consult a lawyer for specific requirements
- Include standard disclaimers
- Regularly reviewed and updated (document version/date)
Recommended approach: Use a well-tested generic template from a reputable source (e.g., Privacy Policy Generator) and adapt it to fit our template structure.
## Dependencies
- Story 3.1: Site assignment must be complete (need to know which sites are in use)
- Story 3.3: Navigation menu is already in templates (pages fulfill those links)
- Story 2.4: Template service exists and can apply HTML templates
- Story 1.6: SiteDeployment table exists
## Future Considerations
- Story 4.1 will deploy these pages along with articles
- Future: Custom page content per project (override generic templates)
- Future: Homepage generation with dynamic article listing
- Future: Allow users to edit boilerplate page content via CLI or web interface
- Future: Additional pages (terms of service, disclaimer, etc.)
- Future: Page templates with more customization options (site name, tagline, etc.)
## Deferred to Later
- **Homepage (`index.html`) generation** - Could be part of this story or deferred to Epic 4
- If generated here: Simple page listing all articles on the site
- If deferred: Epic 4 deployment could create a basic redirect or placeholder
- **Custom page content per project** - Allow projects to override default templates
- **Multi-language support** - Generate pages in different languages based on project settings
## Total Effort
15 story points (reduced from 20 due to heading-only simplification)
### Effort Breakdown
1. Database Schema (2 points)
2. Repository Layer (2 points)
3. Page Content Templates (1 point)
4. Generation Logic (2 points)
5. Site Creation Integration (2 points)
6. Template Service Updates (1 point)
7. Backfill Script (2 points)
8. Homepage Generation (deferred)
9. Unit Tests (2 points)
10. Integration Tests (1 point)
**Total: 15 story points**
### Effort Reduction
Original estimate: 20 story points (with full page content)
Simplified (heading-only pages): 15 story points
Savings: 5 story points (no complex content generation needed)
## Notes
- Pages should be visually consistent with articles (same template)
- **Pages have heading only** - just `<h1>` tag, no body content
- Better UX than completely empty (user sees page title when they click nav link)
- User can manually add content later for specific sites if desired
- Pages are generated once per site at creation time
- Future enhancement: Add content generation for privacy policy if legally required
- Future enhancement: CLI command to update page content for specific sites

38
docs/technical-debt.md
View File

@ -455,6 +455,44 @@ This would still provide value with much less complexity (2-3 story points inste
---
## Story 3.3: Content Interlinking Injection
### Boilerplate Site Pages (About, Contact, Privacy)
**Priority**: High
**Epic Suggestion**: Epic 3 (Pre-deployment) - Story 3.4
**Estimated Effort**: Medium (20 story points, 2-3 days)
**Status**: ✅ **PROMOTED TO STORY 3.4** (specification complete)
#### Problem
During Story 3.3 implementation, we added navigation menus to all HTML templates with links to:
- `about.html`
- `contact.html`
- `privacy.html`
- `/index.html`
However, these pages don't exist, creating broken links on every deployed site.
#### Impact
- Unprofessional appearance (404 errors on nav links)
- Poor user experience
- Privacy policy may be legally required for public sites
- No contact mechanism for users
#### Solution (Now Story 3.4)
See full specification: `docs/stories/story-3.4-boilerplate-site-pages.md`
**Summary:**
- Automatically generate boilerplate pages for each site during batch generation
- Store in new `site_pages` table
- Use same template as articles for visual consistency
- Generic but professional content suitable for any niche
- Generated once per site, skip if already exists
**Implementation tracked in Story 3.4.**
---
## Future Sections
Add new technical debt items below as they're identified during development.