11 KiB

Raw Blame History

Big Link Man - Content Automation & Syndication Platform

AI-powered content generation and multi-tier link building system with cloud deployment.

Quick Start

# Install dependencies
uv pip install -r requirements.txt

# Setup environment
cp env.example .env
# Edit .env with your credentials

# Initialize database
uv run python scripts/init_db.py

# Create first admin user
uv run python scripts/create_first_admin.py

# Run CLI
uv run python main.py --help

Environment Configuration

Required environment variables in .env:

DATABASE_URL=sqlite:///./content_automation.db
OPENROUTER_API_KEY=your_key_here
BUNNY_ACCOUNT_API_KEY=your_bunny_key_here

See env.example for full configuration options.

Database Management

Initialize Database

uv run python scripts/init_db.py

Reset Database (drops all data)

uv run python scripts/init_db.py reset

Create First Admin

uv run python scripts/create_first_admin.py

Database Migrations

# Story 3.1 - Site deployments
uv run python scripts/migrate_story_3.1_sqlite.py

# Story 3.2 - Anchor text
uv run python scripts/migrate_add_anchor_text.py

# Story 3.3 - Template fields
uv run python scripts/migrate_add_template_fields.py

# Story 3.4 - Site pages
uv run python scripts/migrate_add_site_pages.py

# Story 4.1 - Deployment fields
uv run python scripts/migrate_add_deployment_fields.py

# Backfill site pages after migration
uv run python scripts/backfill_site_pages.py

User Management

Add User

uv run python main.py add-user \
  --username newuser \
  --password password123 \
  --role Admin \
  --admin-user admin \
  --admin-password adminpass

List Users

uv run python main.py list-users \
  --admin-user admin \
  --admin-password adminpass

Delete User

uv run python main.py delete-user \
  --username olduser \
  --admin-user admin \
  --admin-password adminpass

Site Management

Provision New Site

uv run python main.py provision-site \
  --name "My Site" \
  --domain www.example.com \
  --storage-name my-storage-zone \
  --region DE \
  --admin-user admin \
  --admin-password adminpass

Regions: DE, NY, LA, SG, SYD

Attach Domain to Existing Storage

uv run python main.py attach-domain \
  --name "Another Site" \
  --domain www.another.com \
  --storage-name my-storage-zone \
  --admin-user admin \
  --admin-password adminpass

Sync Existing Bunny.net Sites

# Dry run
uv run python main.py sync-sites \
  --admin-user admin \
  --dry-run

# Actually import
uv run python main.py sync-sites \
  --admin-user admin

List Sites

uv run python main.py list-sites \
  --admin-user admin \
  --admin-password adminpass

Get Site Details

uv run python main.py get-site \
  --domain www.example.com \
  --admin-user admin \
  --admin-password adminpass

Remove Site

uv run python main.py remove-site \
  --domain www.example.com \
  --admin-user admin \
  --admin-password adminpass

Project Management

Ingest CORA Report

uv run python main.py ingest-cora \
  --file shaft_machining.xlsx \
  --name "Shaft Machining Project" \
  --custom-anchors "shaft repair,engine parts" \
  --username admin \
  --password adminpass

List Projects

uv run python main.py list-projects \
  --username admin \
  --password adminpass

Content Generation

Create Job Configuration

# Tier 1 only
uv run python create_job_config.py 1 tier1 15

# Multi-tier
uv run python create_job_config.py 1 multi 15 50 100

Generate Content Batch

uv run python main.py generate-batch \
  --job-file jobs/project_1_tier1_15articles.json \
  --username admin \
  --password adminpass

With options:

uv run python main.py generate-batch \
  --job-file jobs/my_job.json \
  --username admin \
  --password adminpass \
  --debug \
  --continue-on-error \
  --model gpt-4o-mini

Available models: gpt-4o-mini, claude-sonnet-4.5, - anything at openrouter

Note: If your job file contains a models config, it will override the --model flag and use different models for title, outline, and content generation stages.

Deployment

Deploy Batch

# Automatic deployment (runs after generation)
uv run python main.py generate-batch \
  --job-file jobs/my_job.json \
  --username admin \
  --password adminpass

# Manual deployment
uv run python main.py deploy-batch \
  --batch-id 123 \
  --admin-user admin \
  --admin-password adminpass

Dry Run Deployment

uv run python main.py deploy-batch \
  --batch-id 123 \
  --dry-run

Verify Deployment

# Check all URLs
uv run python main.py verify-deployment --batch-id 123

# Check random sample
uv run python main.py verify-deployment \
  --batch-id 123 \
  --sample 10 \
  --timeout 10

Link Export

Export Article URLs

# Tier 1 only
uv run python main.py get-links \
  --project-id 123 \
  --tier 1

# Tier 2 and above
uv run python main.py get-links \
  --project-id 123 \
  --tier 2+

# With anchor text and destinations
uv run python main.py get-links \
  --project-id 123 \
  --tier 2+ \
  --with-anchor-text \
  --with-destination-url

Output is CSV format to stdout. Redirect to save:

uv run python main.py get-links \
  --project-id 123 \
  --tier 1 > tier1_urls.csv

Utility Scripts

Check Last Generated Content

uv run python check_last_gen.py

List All Users (Direct DB Access)

uv run python scripts/list_users.py

Add Admin (Direct DB Access)

uv run python scripts/add_admin_direct.py

Check Migration Status

uv run python scripts/check_migration.py

Add Tier to Projects

uv run python scripts/add_tier_to_projects.py

Testing

Run All Tests

uv run pytest

Run Unit Tests

uv run pytest tests/unit/ -v

Run Integration Tests

uv run pytest tests/integration/ -v

Run Specific Test File

uv run pytest tests/unit/test_url_generator.py -v

Run Story 3.1 Tests

uv run pytest tests/unit/test_url_generator.py \
               tests/unit/test_site_provisioning.py \
               tests/unit/test_site_assignment.py \
               tests/unit/test_job_config_extensions.py \
               tests/integration/test_story_3_1_integration.py \
               -v

Run with Coverage

uv run pytest --cov=src --cov-report=html

System Information

Show Configuration

uv run python main.py config

Health Check

uv run python main.py health

List Available Models

uv run python main.py models

Directory Structure

Big-Link-Man/
├── main.py                 # CLI entry point
├── src/                    # Source code
│   ├── api/               # FastAPI endpoints
│   ├── auth/              # Authentication
│   ├── cli/               # CLI commands
│   ├── core/              # Configuration
│   ├── database/          # Models, repositories
│   ├── deployment/        # Cloud deployment
│   ├── generation/        # Content generation
│   ├── ingestion/         # CORA parsing
│   ├── interlinking/      # Link injection
│   └── templating/        # HTML templates
├── scripts/               # Database & utility scripts
├── tests/                 # Test suite
│   ├── unit/
│   └── integration/
├── jobs/                  # Job configuration files
├── docs/                  # Documentation
└── deployment_logs/       # Deployed URL logs

Job Configuration Format

Example job config (jobs/example.json):

{
  "job_name": "Multi-Tier Launch",
  "project_id": 1,
  "description": "Site build with 165 articles",
  "models": {
    "title": "openai/gpt-4o-mini",
    "outline": "anthropic/claude-3.5-sonnet",
    "content": "anthropic/claude-3.5-sonnet"
  },
  "tiers": [
    {
      "tier": 1,
      "article_count": 15,
      "validation_attempts": 3
    },
    {
      "tier": 2,
      "article_count": 50,
      "validation_attempts": 2
    }
  ],
  "failure_config": {
    "max_consecutive_failures": 10,
    "skip_on_failure": true
  },
  "interlinking": {
    "links_per_article_min": 2,
    "links_per_article_max": 4,
    "include_home_link": true
  },
  "deployment_targets": ["www.primary.com"],
  "tier1_preferred_sites": ["www.premium.com"],
  "auto_create_sites": true
}

Per-Stage Model Configuration

You can specify different AI models for each generation stage (title, outline, content):

{
  "models": {
    "title": "openai/gpt-4o-mini",
    "outline": "anthropic/claude-3.5-sonnet",
    "content": "openai/gpt-4o"
  }
}

Available models:

openai/gpt-4o-mini - Fast and cost-effective
openai/gpt-4o - Higher quality, more expensive
anthropic/claude-3.5-sonnet - Excellent for long-form content

If models is not specified in the job file, all stages use the model from the --model CLI flag (default: gpt-4o-mini).

Common Workflows

Initial Setup

uv pip install -r requirements.txt
cp env.example .env
# Edit .env
uv run python scripts/init_db.py
uv run python scripts/create_first_admin.py
uv run python main.py sync-sites --admin-user admin

New Project Workflow

# 1. Ingest CORA report
uv run python main.py ingest-cora \
  --file project.xlsx \
  --name "My Project" \
  --username admin \
  --password adminpass

# 2. Create job config
uv run python create_job_config.py 1 multi 15 50 100

# 3. Generate content (auto-deploys)
uv run python main.py generate-batch \
  --job-file jobs/project_1_multi_3tiers_165articles.json \
  --username admin \
  --password adminpass

# 4. Verify deployment
uv run python main.py verify-deployment --batch-id 1

# 5. Export URLs for link building
uv run python main.py get-links \
  --project-id 1 \
  --tier 1 > tier1_urls.csv

Re-deploy After Changes

uv run python main.py deploy-batch \
  --batch-id 123 \
  --admin-user admin \
  --admin-password adminpass

Troubleshooting

Database locked

# Stop any running processes, then:
uv run python scripts/init_db.py reset

Missing dependencies

uv pip install -r requirements.txt --force-reinstall

AI API errors

Check OPENROUTER_API_KEY in .env

Bunny.net authentication failed

Check BUNNY_ACCOUNT_API_KEY in .env

Storage upload failed

Verify storage_zone_password in database (set during site provisioning)

Documentation

Product Requirements: docs/prd.md
Architecture: docs/architecture/
Implementation Summaries: STORY_*.md files
Quick Start Guides: *_QUICKSTART.md files

11 KiB Raw Blame History