=== Bottle ===
Contributors:           benrowland
Author URI:             https://www.tecture.com
Requires at least:      6.7
Requires PHP:           7.4
Tested up to:           7.0
Stable tag:             0.3.6
License:                GPLv3 or later
License URI:            https://www.gnu.org/licenses/gpl-3.0.html
Tags:                   bot, security, firewall, throttle

Bottle is a bot throttling plugin.

== Description ==

Bottle is a bot throttling plugin. It consists of a set of tools that can be used to reduce server load when excessive bot requests are made to a website.

It works by performing a variety of checks on each HTTP request made to WordPress, and throttling sessions or IPs when thresholds are exceeded. These include:

- **Too many sessions per IP.** Since some bots don't bother with cookies, Bottle sets a JWT session cookie for each request. If a single IP address creates too many new sessions within a given timeframe, it is likely a bot, and will be throttled.

- **Too many requests per session.** Bottle calculates a request rate for each session. Bots that exceed that rate will be throttled.

- **Too many requests per IP.** Bottle calculates a request rate per IP. For bots that don't process cookies (and therefore sessions), this can throttle abusive IPs.

- **Too many requests per bot.** Bottle calculates a request rate per bot (based on user agent reported). Since malicious bots forge user agents, this should be combined with other throttling mechanisms.

- **Bot verification check.** Bottle checks bots such as Google, Bing, Meta, etc. against authoritative lists of IP addresses, when possible. Bottle allows for a higher verified bot rate than non-verified rate with the idea that verified bots are likely better behaved.

- **Overall request rate.** Bottle can throttle specific sections of a website that are being abused by bots. (For instance, the courses catalog.) This is useful when a large website is being scraped by a bot network with rotating IPs.

- **Excessive 404s.** If bots are scanning a website, or are misconfigured, excessive 404s can be time-consuming for WordPress to process. Bottle can either throttle all 404 requests, or can block 404s for static files (i.e. images) only.

- **Exclusions.** Bottle can exclude certain URLs, IP addresses, or infrastructure (such as Stripe and Pingdom) from processing to prevent false positives.

When a client is throttled, Bottle can outright throttle it with a 429 response, or can serve a JavaScript-based cryptographic puzzle. If a client is legitimate, it will automatically compute the puzzle's solution and submit it to the server for verification. (No user input is required.) After it has proven its human-ness, Bottle will permit that user entry to the site. At the moment, some bots, but not all, will solve these puzzles.

When a throttle condition is reached, Bottle terminates WordPress processing, which substantially reduces server load when under attack. Throttles end automatically at the end of their respective measurement windows.

Bottle is not a firewall and will not block or sanitize malicious attacks. However, since such attacks are typically performed in large quantities, Bottle can be effective at slowing such attacks and reducing server load, keeping the website operational for legitimate users.

Overall, Bottle is a set of tools designed to reduce server load with minimal disruption to real users.

== Installation ==

Install and activate the plugin. It will create a mu-plugin automatically.

**Important:** We recommend the use of a persistent object cache such as Redis because Bottle uses cache/transients as a method of measuring bot activity. When a persistent object cache is available, Bottle will use its `wp_cache_incr` method. Otherwise, Bottle will fall back to use `get_transient` and `set_transient` functions, which reduces its performance.

After installation, all Bottle settings are initially disabled. Go to Settings -> Bottle Settings and configure the plugin as desired.

We recommend that you enable the "Request Logging Enabled" setting temporarily while configuring Bottle. This will show you a live view of all requests that come into WordPress, including the status of each test Bottle runs on each request. This can help you identify which ones need to be configured or adjusted.

== Frequently Asked Questions ==

= Does proof-of-work actually work? =

Yes. It's a somewhat controversial method, and we cannot claim credit for it; ours is a rather simple implementation inspired by Anubis. Is it ethical to offload the work to your users? We're not sure. Are there PoW-solving browser extensions that intercept and offload to GPU for faster processing? Sure. Does it stop all bots? Nope. We have found that some bots will solve this. However, at this time, and for the types of traffic we see on our sites, it significantly reduces server load while allowing users access to key sections of websites that would otherwise be blocked.

= Does Bottle work with Varnish? =

Yes, with proper configuration. We have provided Varnish VCL modifications that allow Varnish to continue caching pages while passing the JWT session cookie back and forth. When a page is cached by Varnish, it will never reach Bottle. This throttle is effective on the backend only. However, when a very large website is being scanned or attacked, and many of the pages are not cached, Bottle can be an effective deterrent without reducing the effectiveness of Varnish caching.

= Why did we create Bottle? =

We are a web software vendor that hosts many websites and applications for clients.

We observed many WordPress sites were being routinely attacked, and these attacks caused slowdowns and outages. Recently, with the uptick in AI bot traffic, sites began experiencing bot network attacks, in which clients rapidly rotate their IP addresses. One traffic pattern we identified was one in which large content sections of a website (such as /courses/*) were being scanned. Another involved dynamically contructed URLs (for instance, the following urls are all the same: /content/a-b-c /content/a-c-b /content/b-a-c /content/b-c-a /content/c-a-b /content/c-b-a). Such URls can be easily identified with regular expressions that could block this type of traffic, and the idea for Bottle was born.

Today, we run it on all of our sites. We find it protects them while minimizing user disruption. Your experience may be different. We think of Bottle as a toolkit we can configure based on the specific sites and situation.

== Screenshots ==

1. Bottle offers a range of configurable tools, stats, and logging.
2. A heatmap shows the level of throttle activity per throttle type, grouped hourly.
3. A bot heatmap indicates the level of website usage per bot. Note that some bots are verified by IP address. (Bots may forge their reported user agent.)
4. The Live Requests log shows all requests, including reverse DNS lookup, identified bot, bot IP verification status, and throttle action. Its purpose is to help you configure each of the throttle tools.
5. The Throttle log shows which requests were throttled and why, including IP, user agent, bot, and bot verification status. It also shows when the PoW is solved (which allows a client to bypass the throttle).
6. The User Agents log shows which user agents were throttled. This can be useful for identifying forged user agents, scrapers, security tools, and more.
7. The Paths log shows which URLs triggered throttles. This can be helpful if there are patterns of abuse, or if certain parts of the site are being scraped.
8. The IPs log shows which IPs triggered the throttles, and the number of throttles per IP.
9. Session throttle settings allow you to throttle IPs that create too many sessions too quickly, and throttle sessions that access too many pages too quickly.
10. Request rates by IP address and by bot type allow you to throttle abusers. Verified bots can be provided a higher rate.
11. Overall Page Matching Throttle is a tool that throttles a specific section of a website that is being abused by bots. (For instance, the courses catalog.) This is useful when a large website is being scraped by a bot network with rotating IPs.
12. 404s is a tool that can block IPs that make requests for too many non-existant static files. Or, it can throttle IPs that request too many non-existant pages.
13. Exclude is a tool that lets you configure overrides for the throttle tools (URL begins with, URL contains, and IP address). Known infrastructure services such as Stripe and Pingdom are verified by IP address and can be excluded from all throttles.
14. Proof-of-Work is an cryptographic puzzle displayed to users instead of an outright 429 HTTP code (throttle). Many bots do not process JavaScript and will not solve puzzles. The length is the complexity of the puzzle. A larger number generally takes longer to solve, so this is a measure of how long a delay your legitimate users will face when throttled.
15. Logging options includes log location and rotation settings. We recommend using the Request Logging tool only for setup or debugging and leave it disabled the rest of the time.
16. The Proof-of-Work presented to throttled clients.

== Changelog ==

= 0.3.6 =
* Output nocache header under throttle conditions so 429s are not mistakenly cached by varnish

= 0.3.5 =
* Fixed issue with log cleanup cron

= 0.3.4 =
* Improvements to indicate when bot IP verification was attempted and succeded or failed, both in logs and in the metrics heatmap
* Updates to bot list
* Minor log file handling improvements

= 0.3.3 =
* Improvements to display of bots on throttle log and user agents tabs
* Added more bots and verifications
* Improvements to Metrics bot chart labels
* Bug fixes

= 0.3.2 =
* Dashboard notice when Bottle is activated but not running
* Bug fix for missing throttle rate
* Performance improvement for Requests tab initial load time
* Requests tab table layout improvements
* Bug fixes

= 0.3.1 =
* Bug fix for logging bot when PoW is solved
* Added more user agents

= 0.3.0 =
* Added User Agents throttle log tab
* Bug fixes

= 0.2.9 =
* Added more user agents and IP lookups
* Added user agent to throttle log file
* Added user agent to Log tab
* Enhanced log displays with truncation of long FQDNs and user agents and tooltips
* Treat blank user agent as a bot
* Fixed metrics sorting
* Improved metrics colorscale to make small quantities more visible
* Improved metrics display by consolidating similar user agents with slightly different version numbers

= 0.2.8 =
* Added three new tests:
    * Added per-bot throttle rate feature
    * Added bot verification feature to look up bots against published lists of IP addresses to permit a higher request rate
    * Added exclusion for Stripe and Pingdom infrastructure IPs (including lookup against published lists of infrastructure IPs)
* Added new bot metrics heatmap
* Added reverse IP domain lookup and user agent display to requests screen for debugging purposes
* Changed behavior of PoW: will not serve puzzle to a request that identifies itself as a bot (will serve 429 throttled instead)
* Bug fix related to request rate per IP address measurement window
* Bug fix related to postpass wp-login.php action
* Bug fix related to timezone in throttle metrics
* Tested WordPress v7 compatibility

= 0.2.7 =
* Improved support for object caching by using wp_cache_incr method to increment counters

= 0.2.6 =
* Heatmap improvements and minor bug fixes / cleanup

= 0.2.4 =
* Added metrics tab and heatmap chart

= 0.2.3 =
* Created POT file for translators
* Enhancement: Requests log blocking display
* Enhancement: Added settings link to plugins list
* Bug: Fixed settings file upgrade error

= 0.2.2 =
* Initial release
