toolvana/README.md

Toolvana — Your free online media toolbox

A production-ready, SEO-first all-in-one media tools platform built on CodeIgniter 4.7 and PHP 8.2: YouTube tools, video/audio/image/GIF converters, PDF utilities, QR generation, programmatic SEO at scale (134+ root-level tool pages), a durable job queue with FFmpeg workers, anonymous analytics, and a complete admin panel.

Feature overview

Area	What ships
Tools	134 seeded tools across YouTube / video / audio / image / GIF / PDF / utility categories; URL-kind, upload-kind and text-kind flows
Processing	MySQL-backed queue (SELECT … FOR UPDATE SKIP LOCKED), driver architecture: Ffmpeg, Images (GD), Pdf (qpdf/Ghostscript/poppler), Youtube (thumbnails via i.ytimg.com; yt-dlp optional & policy-gated), Qr (server-side PNG)
Security	SSRF guard for URL fetches, extension and real-MIME validation, UUID storage names outside the web root, per-session/IP hourly rate limits, API-key auth with hashed lookup, admin bcrypt login, strict security headers, canonical 301 enforcement
SEO	Unique titles/H1s/descriptions generated per format pair, JSON-LD (WebApplication, BreadcrumbList, FAQPage, HowTo, Article), auto sitemap index split at 5 000 URLs, robots.txt blocking processing endpoints, noindex on search/API/admin, hreflang scaffolding, OG images rendered by GD
Admin (/admin)	Overview dashboard, tools CRUD with pre-publish SEO gate, categories, guides (Markdown), automated SEO audit across every page, analytics dashboard (success-rate per tool), job inspector with retry, system health (binaries/storage/worker heartbeat), feature-flag settings
API v1	/api/v1/tools, /api/v1/jobs (+ legacy /api/*), Bearer-token auth, per-key hourly quotas
PWA	Web manifest, offline-capable service worker, offline shell
Privacy	Anonymous session-hash analytics only, automatic media expiry (media:cleanup), DMCA/privacy/cookies pages

Requirements

• PHP 8.1+ (8.2 recommended) with intl, mbstring, zip, pdo_mysql, gd, exif
• MariaDB/MySQL 10.6+
• FFmpeg, ImageMagick (convert), poppler-utils (pdftoppm, pdftotext), qpdf, Ghostscript
• Optional: yt-dlp (disabled by policy until enabled in Admin → Settings)
• Composer 2

Quick start (development)

composer install
cp env .env            # then edit DB credentials
php spark key:generate

# create schema + seed registry/content
php spark migrate --all
php spark db:seed InitialSeeder

# run
php spark serve                    # http://127.0.0.1:8080 (falls back to 8081/8082 if busy)

# process jobs (keep one or more running)
php spark queue:work --loop        # add --sleep=2 to idle politely

# expire finished jobs + files (cron every 5 min in production)
php spark media:cleanup

Sign in to the admin panel at /admin/login. Set the password hash in .env:

php -r 'echo password_hash("your-password", PASSWORD_DEFAULT), "\n";'
# .env → site.adminPassword = '$2y$10$…'

Production deployment (Docker)

cp .env.docker.example .env.docker   # fill in secrets: encryption.key, site.adminPassword, DB password
docker compose up -d --build
docker compose exec web php spark migrate --all
docker compose exec web php spark db:seed InitialSeeder

Topology (docker-compose.yml):

browser ── nginx ──► web  (PHP-FPM, page rendering)
                └──► api  (PHP-FPM, /api + processing endpoints isolated)
workers  × N        (php spark queue:work --loop)      ← scale replicas freely
scheduler           (media:cleanup every 5 minutes)
db                  (MariaDB 11 + healthcheck)

• The single docker/Dockerfile image contains PHP-FPM and the full media binary stack, so web/worker versions can never drift.
• Media lives on named volumes (media-storage, incoming-storage); point them at shared NFS/EFS when running workers on separate hosts.
• Put TLS termination at your edge/load balancer and set app.forceGlobalSecureRequests = true; nginx config includes HSTS-ready headers.
• Recommended cron alternative to the scheduler container: */5 * * * * docker compose exec -T scheduler php spark media:cleanup.

Production checklist

• CI_ENVIRONMENT = production, strong encryption.key (php spark key:generate)
• site.adminPassword set to a fresh bcrypt hash
• site.ytDlpEnabled left false unless legal review is done
• Review retention: site.retentionHours (default 2 h) matches your privacy policy
• Point /sitemap.xml at Search Console; confirm canonical host in .env (app.baseURL)
• Run Admin → SEO Audit and clear critical issues before launch

Architecture notes

app/
├── Config/Site.php          # brand, binaries map, limits, policy flags
├── Controllers/             # Tools, Directory, Guides, Media (upload/start/download), Api/*, Admin/*
├── Libraries/
│   ├── Pipeline.php         # enqueue guards + driver routing (by operation/format family)
│   ├── Pipeline/{Ffmpeg,Images,Pdf,Youtube,Qr}.php
│   ├── CatalogBuilder.php   # generates the 134-tool registry with unique SEO copy
│   ├── Seo.php              # head tags, canonical, JSON-LD graphs
│   ├── UrlGuard.php         # SSRF protection for remote fetches
│   ├── ToolSearch.php       # weighted token search incl. aliases ("yt mp3")
│   └── RelatedTools.php     # internal-linking engine (format/category scoring)
├── Commands/                # queue:work, media:cleanup
└── Database/Migrations/     # categories, tools, guides, jobs, analytics_events, api_keys

Why a DB queue? Zero extra infrastructure, transactional claims via SKIP LOCKED, trivial horizontal scaling of identical worker containers, and a built-in audit trail surfaced in the admin Jobs screen.

Canonical URL policy: every non-canonical variant (uppercase, trailing slash, doubled slashes) 301-redirects before routing — including unmatched routes — via a required filter.

Testing

./vendor/bin/phpunit            # 19 tests, ~1 100 assertions

Suites:

• tests/unit/CatalogTest.php — catalog integrity (unique slugs, complete SEO fields, format-specific titles), YouTube ID extraction, SSRF guard behaviour.
• tests/feature/SeoPagesTest.php — the SEO contract: canonical/OG/Twitter tags, JSON-LD types present in first response, sitemaps contain only indexable URLs, robots.txt blocks processing endpoints, 301 canonicalisation, helpful 404s.

Useful commands

php spark queue:work [--loop] [--sleep=N] [--max-jobs=N]   # process jobs
php spark media:cleanup                                    # expire jobs/files past retention
php spark migrate --all && php spark db:seed InitialSeeder # rebuild registry (editorial fields preserved)