=== Lens - AI Visibility Monitor ===
Contributors: jisanewebmarketing
Donate link: https://www.ewebmarketing.au/
Tags: aeo, answer engine optimization, ai seo, llms.txt, schema
Requires at least: 6.2
Tested up to: 7.1
Requires PHP: 7.4
Stable tag: 1.6.0
License: GPLv2 or later
License URI: https://www.gnu.org/licenses/gpl-2.0.html

First WordPress plugin scoring your site on Google Chrome Lighthouse’s new Agentic Browsing criteria. Plus per-page AI bot analytics. BYO OpenAI key.

== Description ==

**Lens is the first WordPress plugin to score your site on Google Chrome Lighthouse’s new *Agentic Browsing* readiness category** — the taxonomy Chrome uses to measure how ready your site is for AI agents to read and act on. Plus per-page AI bot analytics, crawl-to-citation correlation, and citation tracking against ChatGPT and Perplexity. All with your own OpenAI key. Zero data leaves your database. No SaaS subscription. Free forever.

Unlike traditional SEO plugins, Lens does not try to replace Yoast, Rank Math, AIOSEO or SEOPress. It runs alongside whichever SEO plugin you already use and adds the AI visibility layer none of them cover today.

= What makes Lens different =

Every other AI-SEO plugin on the market proxies your content through a paid SaaS. Lens is the only one that:

* **Logs actual AI bot hits per URL** — not inferred from Search Console, not simulated. Real user-agent-matched requests from 18 recognised AI bots (GPTBot, ClaudeBot, PerplexityBot, OAI-SearchBot, Claude-SearchBot, Google-Extended, Meta-ExternalAgent, Applebot-Extended, DeepSeekBot, CCBot, MistralAI-User and more).
* **Correlates crawls with citations** — cross-references the pages AI bots actually read with the pages the answer engines actually cite. Surfaces zero-efficiency pages that burn training-crawl budget without returning a citation.
* **Uses your own OpenAI API key** — you pay OpenAI directly. Nothing is proxied through Lens infrastructure. Zero data leaves your WordPress database. The only AI-SEO option regulated / enterprise / privacy-conscious buyers can adopt.
* **Understands the training vs retrieval vs assistant distinction** — the critical taxonomy most SEO plugins miss. Blocking GPTBot does NOT stop OpenAI from citing you (that's OAI-SearchBot's job). Lens is the only plugin that lets you make this decision per category.

= Seven pillars =

**1. AI Crawler Analytics (free, no API key required)**

Detects and logs visits from 18+ recognised AI bot families. Dashboard shows per-bot / per-day / per-page breakdowns, blind-spot pages, weekly trend, first-visit alerts.

**2. Crawl → Citation efficiency (v1.2.0)**

The diagnostic no other WordPress plugin offers. For every crawled URL, Lens shows how many times the answer engines cited it back. Zero-efficiency rows are the actionable priority list.

**3. Bot Access Control with training / retrieval / assistant taxonomy (v1.2.1)**

Three category cards on the Settings page let you block bots by their actual purpose. Block training crawlers to keep content out of model training data — WITHOUT blocking the retrieval crawlers that create citations. Advanced per-bot override table lets you force allow / force block any single bot.

**4. Chrome Lighthouse Agentic Browsing readiness (v1.4.0)**

The alignment claim no other WordPress plugin makes. Every AEO scan now runs the same-shape checks Google Chrome ships in its (experimental) Agentic Browsing Lighthouse category:

* **Accessibility for agents** — every interactive element must have a programmatic name (aria-label, associated <label for>, visible text) and must not be hidden from the accessibility tree while remaining focusable.
* **WebMCP integration** — detects declarative (`<script type="application/mcp+json">`) or imperative (`navigator.webmcp.registerTool`) tool registrations. Informational only — WebMCP is experimental in Chrome 150+ with origin trial.
* **Discoverability (llms.txt)** — checks for the machine-readable summary at your domain root.
* **Layout stability (CLS)** — informational + a link straight into PageSpeed Insights for the page’s real-user CLS.

Presented as a fractional pass ratio (X of Y checks passed), the same format Chrome uses. A sitewide "Agentic readiness" KPI appears on the Dashboard next to Citation rate.

**5. Per-page AEO scoring (bring your own AI key)**

Local checks always run for free: title, meta, headings, FAQ/HowTo/Article schema, image alt coverage, llms.txt entry, internal links, answer-shaped intro, and **extractable-passage detection** — the primary AEO signal after Google removed FAQ rich results in May 2026. AI-enriched scoring uses your OpenAI key to add a 0–100 citability score with three ranked fixes.

**6. Citation tracking (bring your own AI key, opt-in)**

Enter up to 10 seed queries. On a schedule you choose, Lens asks OpenAI whether your domain appears in the response. Trend charts of citation rate over time, per query and per provider.

**7. Site-wide baseline audit (v1.2.2)**

One-click HTML report aggregating every scan, every crawler hit, and every citation check on the site. Score distribution, worst-20 and best-20 pages, blind spots, and prioritised action list. Ideal for agency client onboarding.

= Automatic schema markup =

Lens emits JSON-LD on every post and page: WebSite, Organization, WebPage, BreadcrumbList, Article (or BlogPosting), plus auto-detected FAQPage, HowTo, ImageObject, VideoObject and SpeakableSpecification. Conflict-detects Yoast / Rank Math / AIOSEO / SEOPress / The SEO Framework — defaults OFF if any are active so you never get duplicate JSON-LD.

= Why Lens =

* **Free forever, no account.** Crawler analytics works the moment you activate. No signup, no licence key, no SaaS.
* **Your data stays on your site.** All logs, scans, citations and schema are generated locally in your WordPress database. Nothing is proxied through our servers.
* **BYO API key.** You pay OpenAI directly. No middleman markup, no per-scan credit resale.
* **Agency-ready reporting.** Baseline audit and weekly visibility reports designed for client delivery.
* **Open source under GPLv2.** Audit the code, fork it, hack on it.

= Who it's for =

* **Agencies** deploying AEO monitoring across many client sites.
* **SaaS, e-commerce and publishers** competing for AI citations.
* **Regulated / enterprise sites** (healthcare, legal, finance, government) where SaaS AI-SEO tools cannot be approved by procurement.
* **Local businesses** chasing AI Overview visibility.
* **Developers** who want raw crawler data and a REST API.

== External services ==

Lens connects to external services in the following ways. **All third-party calls are optional and require you to explicitly provide an API key.**

**OpenAI API** (https://api.openai.com)

* **Purpose:** Generate per-page AEO scores, suggest fixes, and check citation queries when OpenAI is selected as the citation provider.
* **When data is sent:** Only when you manually trigger an AI-enriched scan, or when a citation-tracking schedule you have explicitly enabled runs.
* **Data sent:** Post title and post content (for scans); user-supplied query strings and your site domain (for citation checks). No personal user data is transmitted.
* **Consent:** Required — you must enter your own OpenAI API key in Lens → Settings → AI Providers. Without a key, no data is sent.
* **Privacy policy:** https://openai.com/policies/privacy-policy
* **Terms of service:** https://openai.com/policies/terms-of-use

Lens does not transmit any data to any other external service. Crawler detection happens entirely inside your WordPress install — no external lookup is performed on visiting bots.

== Trademark notice ==

ChatGPT and OpenAI are trademarks of OpenAI. Perplexity is a trademark of Perplexity AI, Inc. Claude and Anthropic are trademarks of Anthropic PBC. Gemini is a trademark of Google LLC. Microsoft Copilot and Bing are trademarks of Microsoft Corporation. Meta AI and LLaMA are trademarks of Meta Platforms, Inc. Apple Intelligence is a trademark of Apple Inc.

Lens is an independent tool. It is not affiliated with, endorsed by, or sponsored by any of the above companies. All product names, logos, brands and trademarks mentioned in Lens are the property of their respective owners.

== Installation ==

1. Upload the `lens-ai-visibility-monitor` folder to `/wp-content/plugins/` or install through **Plugins → Add New** in the WordPress admin.
2. Activate the plugin via the **Plugins** screen.
3. You will be redirected to a short setup wizard.
4. Crawler analytics begin recording immediately — no further configuration required.
5. To enable AEO scoring and citation tracking, go to **Lens → Settings → AI Providers** and add your OpenAI API key. You can get one at https://platform.openai.com/api-keys.

== Frequently Asked Questions ==

= Does Lens replace my SEO plugin? =

No. Lens runs alongside Yoast SEO, Rank Math, AIOSEO and SEOPress. It does not write meta titles, meta descriptions, canonicals or sitemaps. It only adds the AI visibility layer those plugins do not currently cover.

= Is it really free? =

Yes. The crawler analytics dashboard, local AEO checks, settings, reporting, and the editor sidebar are free with no account required. AI-enriched features (per-page AI scoring and citation tracking) use your own OpenAI API key — you pay OpenAI directly. There is no Lens subscription.

= Do I need an OpenAI account? =

Only if you want the AI-enriched scoring and citation tracking. The crawler analytics dashboard works without one.

= Will it slow my site down? =

No. Crawler detection runs on a single regex match against the User-Agent header and writes one row per matched request. Frontend overhead is microseconds. Dashboard queries are paginated and indexed.

= How much does the OpenAI API cost? =

A typical per-page AEO scan uses around 500–1,500 tokens (≈ US $0.001–0.005 on gpt-4o-mini as of May 2026). A citation check uses around 300–800 tokens per query per run. Run the maths against your own usage at https://openai.com/pricing.

= What happens to my data on uninstall? =

By default, Lens preserves your crawler logs, scans, and settings on uninstall so a reinstall does not lose history. To wipe everything, enable **Lens → Settings → Data & Privacy → "Remove all data on uninstall"** before deactivating.

= Is the crawler log GDPR-compliant? =

Lens hashes every IP address with SHA-256 using a per-site salt before storage. Raw IPs are never written to the database. User-agent strings and request URIs are stored as-is for analytical purposes. You can configure log retention (default 90 days) and wipe logs at any time from **Settings → Data & Privacy**.

= Does Lens work with WooCommerce? =

Yes. Product pages are treated as standard post types for crawler logging and AEO scoring.

= Does it work in the Classic Editor? =

The crawler dashboard and Scans page work everywhere. The real-time scoring sidebar requires the block editor. In the Classic Editor, you trigger a scan from the Scans page instead.

= Is there a REST API? =

Yes. Read-only endpoints under `/wp-json/lens/v1/` expose crawler data, scans, and citation history. Authentication uses standard WordPress application passwords.

= Where do I report a security issue? =

Email jisan@ewebmarketing.com.au. Please do not file public GitHub issues for security reports.


= What is the Crawl → Citation efficiency view? =

For every URL that AI bots have crawled, Lens shows how many times that URL was cited in your citation checks and computes an efficiency ratio (citations ÷ crawls). Zero-efficiency rows are pages where AI bots consume crawl budget without ever citing you — the highest-priority pages to improve.

= What is the training vs retrieval vs assistant taxonomy? =

AI bots fall into three functional categories that most SEO tools lump together. Training crawlers (GPTBot, ClaudeBot, CCBot, Google-Extended, DeepSeekBot, Applebot-Extended) ingest content to train future AI models — blocking them removes you from future training corpora. Retrieval crawlers (OAI-SearchBot, Claude-SearchBot, PerplexityBot) fetch content at query time to include in AI answers with citations — blocking them means you will NOT be cited by that answer engine. Assistant browsers (ChatGPT-User, Claude-User, Perplexity-User, MistralAI-User) represent real users asking an AI to visit your page. Lens is the only plugin that lets you make these decisions per category.

= What is the site-wide baseline audit? =

A one-click HTML report aggregating every piece of data Lens has collected on your site — score distribution histogram, worst-20 and best-20 pages, blind spots, and a prioritised action list. Ideal for agencies onboarding a new client site or presenting month-over-month improvement to stakeholders.

= What is Chrome’s Agentic Browsing readiness category? =

Google Chrome introduced an experimental "Agentic Browsing" category in Lighthouse (May 2026 update) to measure how ready a site is for AI agents to read and act on it — separate from how ready a site is for humans. Instead of a weighted 0–100 score, it emits a fractional pass ratio across four buckets: Accessibility for agents, WebMCP integration, Layout stability (CLS), and Discoverability (llms.txt). Lens v1.4.0 runs the same-shape checks locally, presented in the same format, so the score you see in Lens tracks the score Chrome will surface in the browser. See [Chrome’s docs](https://developer.chrome.com/docs/lighthouse/agentic-browsing/scoring) for the reference implementation.

= What is the extractable-passage checker? =

Google removed FAQ rich results in May 2026. The primary AEO signal is now whether your content contains "extractable passages" — 40 to 100 word paragraphs that end with a terminator, start with something other than a pronoun, and contain a specific number, year or proper noun. Lens scores every page on this signal and it now carries the highest weight (15%) in the overall AEO score.

== Screenshots ==

1. Dashboard — AI bot activity over the last 30 days with the Crawl → Citation efficiency card and blind-spots list.
2. Crawl → Citation efficiency — per-URL breakdown showing which pages AI bots read and whether the answer engines cite them back.
3. Bot Access Control — training vs retrieval vs assistant category cards on the Settings page.
4. AEO Scans — sortable list of recent per-page scans with 0–100 scores. Includes extractable-passage counts.
5. Block editor sidebar — per-post AEO score, top three fixes, and extractable-passage warnings.
6. Citations — citation rate trend chart by provider with the tracked-queries table.
7. Site-wide baseline audit — one-page HTML report aggregating every signal Lens has collected. Agency-ready.
8. Settings — bundled view of AI Providers, Bot Access Control taxonomy, Schema markup and Reporting.

== Changelog ==

= 1.6.0 =
* New: **llms.txt + llms-full.txt generator.** Virtual `/llms.txt` and `/llms-full.txt` served from your site root so AI crawlers (ChatGPT, Claude, Perplexity, Gemini, and others) can discover your key content the way they discover a sitemap. Pick which post types to include from Settings → Agent readiness. No files written to disk, cached 15 minutes, purges automatically when you publish.
* New: **Markdown alternates.** Every page is available as clean markdown at `{permalink}.md` — e.g. `example.com/about.md`. Honours `Accept: text/markdown` on the normal URL too, so agents see markdown while humans still see your theme.
* New: **Agent discovery.** `/.well-known/agent-card.json` publishes your site name, description, contact, and key endpoints (llms.txt, sitemap, feed). An HTTP `Link` header advertises `llms.txt` on every request so autonomous agents auto-find it.
* New: **Score breakdown modal on the Dashboard.** Every KPI now has a "How is this calculated?" link that opens a plain-English explainer with the formula, weighted criteria, and how to improve the number. First covered: Citation rate and Agentic readiness.
* Compat: verified against WordPress 7.1 (no jQuery UI, no Gutenberg components, no admin bar / media processing hooks — nothing to change).

= 1.5.1 =
* Fix: **Dashboard tour re-appeared on every page load** for some users. Closing the tour with the X button now correctly marks it as seen (previously only the Finish button did). The tour is also marked seen the moment it starts, so a browser refresh mid-tour will not re-nag.
* Fix: **Getting Started checklist step 4 ("Review bot access control") never completed** for users who reviewed the section but chose to keep the safe defaults (i.e. block nothing). Now marked as done the moment the user opens the Bot Access Control section via the checklist link. Users can still change their access settings any time.

= 1.5.0 =
* New: **Educational welcome wizard.** Rewritten from procedural to plain-English. New 5-step flow teaches what AI visibility is, how AI bots find your website, and the critical training vs retrieval vs assistant distinction — no jargon.
* New: **Interactive dashboard tour.** First-visit spotlight overlay walks through each KPI card in plain English. Includes a "Take the tour" link to re-launch anytime.
* New: **Glossary tooltips everywhere.** 27 technical terms (AEO, GEO, crawler, citation, schema, llms.txt, agentic browsing, extractable passages, etc.) now have contextual ? icons across every admin page. Every explanation is in plain English, no jargon.
* New: **Getting Started checklist** on the Dashboard tracks four onboarding tasks — connect OpenAI key, run first scan, add citation query, review bot access. Progress bar disappears when complete.
* New: **Delayed review request.** Only shows after 7+ days of use AND at least one completed scan — never on install. Dismissible and postponable.
* Improvement: Every screen now surfaces plain-English explanations, making Lens usable by non-technical site owners.

= 1.4.0 =
* New: **Chrome Lighthouse Agentic Browsing alignment.** Lens now runs the same-shape checks Google Chrome ships in its (experimental) Agentic Browsing Lighthouse category: accessibility for agents, WebMCP integration detection, layout stability info, and llms.txt (already covered). Each AEO scan now emits a fractional pass ratio (X of Y checks passed) mirroring Chrome’s format.
* New: Sitewide **Agentic readiness** KPI card on the Dashboard, next to Citation rate.
* New: Score-weight rebalance — extractable-passages 15 → 13, accessibility-for-agents added at 10, llms.txt rebalanced to 7.
* New: `AgenticChecker` class + `compute_agentic_readiness()` aggregate. `compute_sitewide_agentic()` in the dashboard-data AJAX endpoint.
* Docs: New FAQ entry explaining Chrome’s Agentic Browsing category and how Lens aligns with it.

= 1.3.0 =
* New: **Extractable-passage checker** — a new local scan check that finds 40-100 word paragraphs, requires each to end with a terminator, avoid leading pronouns, and contain at least one specific fact (number, year, or proper noun). This is the primary AEO signal after Google removed FAQ rich results in May 2026.
* Change: Rebalanced scan-score weights — FAQ schema demoted (Google no longer surfaces FAQ rich results), extractable-passages promoted to the highest single weight (15%%).

= 1.2.2 =
* New: **Site-wide baseline audit** — one-click report aggregating every scan, every crawler hit, and every citation check on the site. Score distribution histogram, worst-20 and best-20 pages, blind spots, and prioritised action list. Ideal for agency client onboarding.
* New: Reports page has a new "Site-wide baseline audit" section with Preview + Email buttons.
* New: `BaselineReportBuilder` class + `admin_post_wplens_baseline_preview` route.

= 1.2.1 =
* New: **Bot access control** with the training vs retrieval vs assistant taxonomy. Three category cards on the Settings page let you block bots by their actual purpose — block training crawlers to protect content from being ingested for model training WITHOUT blocking retrieval crawlers that create citations.
* New: `RobotsWriter` class hooks the `robots_txt` filter to write Lens-managed directives. Detects a physical robots.txt file at the site root and warns the user (WordPress's virtual robots.txt is bypassed in that case).
* New: Advanced per-bot override table lets you force-allow or force-block any individual bot regardless of its category default.
* New: `BotRegistry` now includes a `category` field on every bot (training / retrieval / assistant / other) and exposes `categories()` with human-readable descriptions.

= 1.2.0 =
* New: **Crawl → Citation efficiency view** on the Dashboard. Cross-references which pages AI bots actually crawled with which pages the answer engines actually cited. Surfaces zero-efficiency pages that burn training-crawl budget without returning citations — a diagnostic no other WordPress SEO plugin offers.
* New: Range selector (7 / 30 / 90 days) on the correlation view.
* New: Updated tagline emphasising the unique combination of per-page bot analytics + citation correlation + BYO-key privacy.

= 1.1.5 =
* New: Plugin row on the WordPress Plugins list now shows "Settings", "OpenAI API key" (deep link to the key field), "View Details" (WP.org modal) and "Support" links.

= 1.1.4 =
* Fix: Plugin Check escape-output errors. The report-builder s() helper now returns the bare CSS string; every call site wraps it in esc_attr() inline, matching the pattern Plugin Check / WPCS recognises as proper output escaping. Functionally identical output, but the static analyser is satisfied.

= 1.1.3 =
* Fix: WordPress.org reviewer feedback. The weekly visibility report no longer uses a <style> block — every CSS rule is now applied as an inline style attribute on the element. This is also the canonical pattern for HTML emails (Outlook/Gmail/Apple Mail strip <style> blocks anyway).
* Refactor: Report builder split styling into a styles() table for maintainability.

= 1.1.2 =
* Fix: WordPress.org reviewer feedback. Inline crawler-filter script extracted to assets/js/crawlers.js and enqueued via wp_enqueue_script.
* Fix: Schema engine no longer passes JSON_UNESCAPED_SLASHES to wp_json_encode — forward slashes are now escaped, preventing any </script> breakout in JSON-LD output.
* Security: REST endpoints (/crawler/*, /scans, /citations) now require manage_options, matching the floor of the admin pages they back.
* Build: Updated Chart.js 4.4.0 → 4.5.1 (latest stable).
* Build: Excluded /assets/*.png + screenshots from the submission zip (those belong in SVN /assets/ per WordPress.org rules).
* Meta: Contributors line in readme corrected to WordPress.org owner username.

= 1.1.1 =
* Fix: Plugin-Check submission errors. Removed .DS_Store files. Escaped score output in posts-list column. Improved sanitisation of POST arrays.
* Internal: file-level phpcs:disable in admin/class-ajax.php with written justification — every endpoint is dispatched through a single nonce+capability gate that PHPCS cannot trace.

= 1.1.0 =
* New: Schema auto-generator. Lens now emits JSON-LD on every post and page — WebSite, Organization, WebPage, Article, BreadcrumbList, plus auto-detected FAQPage, HowTo, ImageObject, VideoObject and SpeakableSpecification.
* New: Schema settings panel with Organization name / logo / sameAs URLs and per-type toggles.
* New: SEO-plugin conflict detection. If Yoast / Rank Math / AIOSEO / SEOPress / The SEO Framework / Schema Pro is active on activation, the schema emitter defaults OFF to avoid duplicate output.
* Cache: generated JSON-LD is cached per-post and invalidated on save.

= 1.0.2 =
* Fix: Masked OpenAI API key no longer overflows the settings layout. Shortened to a fixed-length mask.

= 1.0.1 =
* Fix: Bulk-scan Start button now works without a separate Preview step. Live estimate auto-loads when filters change.
* Improvement: Better error messages and console diagnostics when bulk-scan calls fail.

= 1.0.0 =
* Initial public release.
* AI crawler analytics with detection for 18+ AI bots.
* Per-page AEO scoring (local checks free; AI-enriched scoring with BYO OpenAI key).
* Citation tracking against OpenAI / Perplexity (opt-in, BYO key).
* Weekly PDF and email visibility reports.
* Block editor sidebar panel.
* Read-only REST API.

== Upgrade Notice ==

= 1.6.0 =
Major: adds the llms.txt / llms-full.txt generator, markdown alternates for every page, /.well-known/agent-card.json, HTTP Link header advertising llms.txt, and a plain-English score-breakdown modal on the Dashboard KPIs. All features can be toggled from Settings → Agent readiness. No database schema changes.

= 1.5.1 =
Two bugfixes to the onboarding flow. Non-breaking.

= 1.5.0 =
UX polish release — educational welcome wizard, dashboard tour, glossary tooltips, and a Getting Started checklist. Non-breaking.

= 1.4.0 =
Adds Chrome Lighthouse Agentic Browsing alignment — the industry-standard taxonomy for AI-agent readiness. Non-breaking. Existing scans get the new checks on their next run.

= 1.3.0 =
Extractable-passage checker (post-FAQ-removal). Also bundles: bot access control taxonomy (1.2.1) and site-wide baseline audit (1.2.2). No breaking changes — no data migration needed.

= 1.2.0 =
New correlation view surfacing crawl-to-citation efficiency per URL. Non-breaking feature release.

= 1.1.5 =
Adds convenience links (Settings, OpenAI API key, View Details, Support) to the plugin row on the Plugins list.

= 1.1.4 =
Plugin Check compliance fix. Rebuild the submission zip and re-upload.

= 1.1.3 =
Reviewer-feedback fix. Rebuild the submission zip with build-zip.sh.

= 1.1.2 =
WordPress.org reviewer fixes. Rebuild the submission zip with build-zip.sh to ensure marketing assets are excluded.

= 1.1.1 =
Ready for WordPress.org Plugin Check. Hard-refresh any admin page to pick up updated assets.

= 1.1.0 =
Schema auto-generator. Visit Settings → Schema markup to configure your Organization details (logo, social profiles).

= 1.0.2 =
Cosmetic fix for the Settings page. Hard-refresh the admin to pick up the new CSS.

= 1.0.1 =
Bulk-scan UX fix. Hard-refresh the admin to pick up the updated JS.

= 1.0.0 =
First public release of Lens as an AI Visibility Monitor. If you used the unreleased internal builds (2.x), uninstall those first.
