# Toolvana — Your free online media toolbox

A production-ready, SEO-first all-in-one media tools platform built on **CodeIgniter 4.7** and **PHP 8.2**: YouTube tools, video/audio/image/GIF converters, PDF utilities, QR generation, programmatic SEO at scale (134+ root-level tool pages), a durable job queue with FFmpeg workers, anonymous analytics, and a complete admin panel.

## Feature overview

| Area | What ships |
|---|---|
| **Tools** | 134 seeded tools across YouTube / video / audio / image / GIF / PDF / utility categories; URL-kind, upload-kind and text-kind flows |
| **Processing** | MySQL-backed queue (`SELECT … FOR UPDATE SKIP LOCKED`), driver architecture: `Ffmpeg`, `Images` (GD), `Pdf` (qpdf/Ghostscript/poppler), `Youtube` (thumbnails via i.ytimg.com; yt-dlp optional & policy-gated), `Qr` (server-side PNG) |
| **Security** | SSRF guard for URL fetches, extension **and** real-MIME validation, UUID storage names outside the web root, per-session/IP hourly rate limits, API-key auth with hashed lookup, admin bcrypt login, strict security headers, canonical 301 enforcement |
| **SEO** | Unique titles/H1s/descriptions generated per format pair, JSON-LD (WebApplication, BreadcrumbList, FAQPage, HowTo, Article), auto sitemap index split at 5 000 URLs, robots.txt blocking processing endpoints, noindex on search/API/admin, hreflang scaffolding, OG images rendered by GD |
| **Admin** (`/admin`) | Overview dashboard, tools CRUD with pre-publish SEO gate, categories, guides (Markdown), automated **SEO audit** across every page, analytics dashboard (success-rate per tool), job inspector with retry, system health (binaries/storage/worker heartbeat), feature-flag settings |
| **API v1** | `/api/v1/tools`, `/api/v1/jobs` (+ legacy `/api/*`), Bearer-token auth, per-key hourly quotas |
| **PWA** | Web manifest, offline-capable service worker, offline shell |
| **Privacy** | Anonymous session-hash analytics only, automatic media expiry (`media:cleanup`), DMCA/privacy/cookies pages |

## Requirements

- PHP 8.1+ (8.2 recommended) with `intl`, `mbstring`, `zip`, `pdo_mysql`, `gd`, `exif`
- MariaDB/MySQL 10.6+
- FFmpeg, ImageMagick (`convert`), poppler-utils (`pdftoppm`, `pdftotext`), qpdf, Ghostscript
- Optional: `yt-dlp` (disabled by policy until enabled in Admin → Settings)
- Composer 2

## Quick start (development)

```bash
composer install
cp env .env            # then edit DB credentials
php spark key:generate

# create schema + seed registry/content
php spark migrate --all
php spark db:seed InitialSeeder

# run
php spark serve                    # http://127.0.0.1:8080 (falls back to 8081/8082 if busy)

# process jobs (keep one or more running)
php spark queue:work --loop        # add --sleep=2 to idle politely

# expire finished jobs + files (cron every 5 min in production)
php spark media:cleanup
```

Sign in to the admin panel at `/admin/login`. Set the password hash in `.env`:

```bash
php -r 'echo password_hash("your-password", PASSWORD_DEFAULT), "\n";'
# .env → site.adminPassword = '$2y$10$…'
```

## Production deployment (Docker)

```bash
cp .env.docker.example .env.docker   # fill in secrets: encryption.key, site.adminPassword, DB password
docker compose up -d --build
docker compose exec web php spark migrate --all
docker compose exec web php spark db:seed InitialSeeder
```

Topology (`docker-compose.yml`):

```
browser ── nginx ──► web  (PHP-FPM, page rendering)
                └──► api  (PHP-FPM, /api + processing endpoints isolated)
workers  × N        (php spark queue:work --loop)      ← scale replicas freely
scheduler           (media:cleanup every 5 minutes)
db                  (MariaDB 11 + healthcheck)
```

- The single `docker/Dockerfile` image contains PHP-FPM **and** the full media binary stack, so web/worker versions can never drift.
- Media lives on named volumes (`media-storage`, `incoming-storage`); point them at shared NFS/EFS when running workers on separate hosts.
- Put TLS termination at your edge/load balancer and set `app.forceGlobalSecureRequests = true`; nginx config includes HSTS-ready headers.
- Recommended cron alternative to the scheduler container: `*/5 * * * * docker compose exec -T scheduler php spark media:cleanup`.

### Production checklist

- [ ] `CI_ENVIRONMENT = production`, strong `encryption.key` (`php spark key:generate`)
- [ ] `site.adminPassword` set to a fresh bcrypt hash
- [ ] `site.ytDlpEnabled` left `false` unless legal review is done
- [ ] Review retention: `site.retentionHours` (default 2 h) matches your privacy policy
- [ ] Point `/sitemap.xml` at Search Console; confirm canonical host in `.env` (`app.baseURL`)
- [ ] Run Admin → **SEO Audit** and clear critical issues before launch

## Architecture notes

```
app/
├── Config/Site.php          # brand, binaries map, limits, policy flags
├── Controllers/             # Tools, Directory, Guides, Media (upload/start/download), Api/*, Admin/*
├── Libraries/
│   ├── Pipeline.php         # enqueue guards + driver routing (by operation/format family)
│   ├── Pipeline/{Ffmpeg,Images,Pdf,Youtube,Qr}.php
│   ├── CatalogBuilder.php   # generates the 134-tool registry with unique SEO copy
│   ├── Seo.php              # head tags, canonical, JSON-LD graphs
│   ├── UrlGuard.php         # SSRF protection for remote fetches
│   ├── ToolSearch.php       # weighted token search incl. aliases ("yt mp3")
│   └── RelatedTools.php     # internal-linking engine (format/category scoring)
├── Commands/                # queue:work, media:cleanup
└── Database/Migrations/     # categories, tools, guides, jobs, analytics_events, api_keys
```

**Why a DB queue?** Zero extra infrastructure, transactional claims via `SKIP LOCKED`, trivial horizontal scaling of identical worker containers, and a built-in audit trail surfaced in the admin Jobs screen.

**Canonical URL policy:** every non-canonical variant (uppercase, trailing slash, doubled slashes) 301-redirects before routing — including unmatched routes — via a *required* filter.

## Testing

```bash
./vendor/bin/phpunit            # 19 tests, ~1 100 assertions
```

Suites:
- `tests/unit/CatalogTest.php` — catalog integrity (unique slugs, complete SEO fields, format-specific titles), YouTube ID extraction, SSRF guard behaviour.
- `tests/feature/SeoPagesTest.php` — the **SEO contract**: canonical/OG/Twitter tags, JSON-LD types present in first response, sitemaps contain only indexable URLs, robots.txt blocks processing endpoints, 301 canonicalisation, helpful 404s.

## Useful commands

```bash
php spark queue:work [--loop] [--sleep=N] [--max-jobs=N]   # process jobs
php spark media:cleanup                                    # expire jobs/files past retention
php spark migrate --all && php spark db:seed InitialSeeder # rebuild registry (editorial fields preserved)
```