Thread Starter wp_kc

(@wp_kc)

Changes in Fail2Ban broke my custom filter that looked at the NinjaFirewall log files. Fail2Ban is expecting a log file to only be appended to, with a consistent end of file index. So the rolling log file nature of NinjaFirewall, where old events are removed from the start of the file, messes up NinjaFirewall. How about instead, making an option that disables the rolling log file feature?

The latest versions of Debian don’t have a syslog any more, everything goes to the journal. I assume the OS probably redirects software syslog writes to the journal. This is also problematic for Fail2Ban, since NinjaFirewall is not a systemd unit as Fail2Ban filters expect, and there is no way to redirect syslog writes to a text log file.

Thanks.

Thread Starter wp_kc

(@wp_kc)

This change to Fail2Ban happened a few years back, and I had put the problem on the back burner. As I recall, it has to do with the changing size of the log file. If the NinjaFirewall file size reaches it’s maximum for the month and starts trimming lines off the beginning, then Fail2Ban barfs with an error message saying the system clock is wrong, and then ignores the log file for the rest of the month!

I had a long back-and-forth with the developer about this. He had made some changes to solve a problem with some other stuff, and was unwilling to change it back just for Ninjafirewall logs (which he viewed as weird and non-standard, whatever that standard is). His suggestion was to use iNotify to capture changes to the log file and append those changes to a different log file. But when I looked into that idea, it also has issues. I also looked into systemd path units and a few other kernel level file monitoring methods, and none of them are ideal.

What I came up with in the meantime was a Must-Use plugin that loads before 0-ninjafirewall.php, and adds a register_shutdown_function callback, then checks for new lines at php shutdown. It’s ugly, but it works, sort of. It tends to reban things that were already banned even though I am keying off the incident ID number, so I end up setting the max number of days to 1 just to minimize rebanning IPs that haven’t come back.

Just looking for a way back to what I had before this change to Fail2Ban happened. Two of my customers are on VPS’ one of whom had the premium version of NinjaFirewall for a couple of years, then stopped paying their bill, the other never had a premium version. So I’ll have to do a sales pitch to get them to subscribe to the premium version.

All my other customers are on shared hosting, so syslog was always out of the question since I have no control over the host’s Fail2Ban. But it would be nice to have a script running that can take NinjaFirewall logs and do a ban through an API call to CloudFlare.

Thread Starter wp_kc

(@wp_kc)

Thread Starter wp_kc

(@wp_kc)

Works great with Fail2Ban. Thanks. Here is some sample config code for fail2ban with apache2…

/etc/fail2ban/jail.local
...
[ninjafirewall-logs]
port = http,https
filter = ninjafirewall-logs
enabled=true
maxretry = 1
logpath = /var/www/wp-content/.ht-ninjafirewall.log
banaction = %(banaction_allports)s
...

/etc/fail2ban/filter.d/ninjafirewall-logs.conf

[INCLUDES]
before = common.conf

[Definition]
_daemon = ninjafirewall-logs
failregex = \[(?:[\d\.]{1,8})\] \[(?:[\w\.\-]{4,80})\] \[\#(?:\d{5,9})\] \[(?:\d{1,6})\] \[[23]\] \[<HOST>\] \[(?:\d{3})\] \[

/var/www/.htninja
...
/**
 * Retrieve the line written to the firewall log.
 * Note: $line doesn't include the Line Feed (\n).
 */
function nfw_custom_user_log( $line ) {
   file_put_contents('/var/www/wp-content/.ht-ninjafirewall.log', "$line\n", FILE_APPEND);
}
...

/etc/logrotate.d/ninjafirewall-logs

/var/www/wp-content/.ht-ninjafirewall.log
{
rotate 0
weekly
create 640 www-data www-data
missingok
notifempty
}

Thread Starter wp_kc

(@wp_kc)

In a shared hosting environment, you can use the new hook to block and, after ten minutes, allow IPs on CloudFlare with the following .htninja script. You will need to create an API token with read and edit access to Account Access Rules. Use CloudFlare’s AI help to get the instructions on how to do this. Copy your account ID and token secret into the .htninja script where noted.

<?php

/**
* Use CloudFlare's reported connecting IP address.
*/
if ( !empty( $_SERVER[ "HTTP_CF_CONNECTING_IP" ] ) &&
filter_var( $_SERVER[ "HTTP_CF_CONNECTING_IP" ], FILTER_VALIDATE_IP ) ) {
$_SERVER[ "REMOTE_ADDR" ] = $_SERVER[ "HTTP_CF_CONNECTING_IP" ];
}

/**
* Retrieve the line written to the firewall log.
* Note: $line doesn't include the Line Feed (\n).
*
* @param string $line Log file line.
*/
function nfw_custom_user_log( $line ) {
/**
* Only act on severity levels: 1, 2, 3
*/
if ( 1 === preg_match( '/^\[(?:[\d]{10})\] \[(?:[\d\.]{1,8})\] \[(?:[\w\.\-]{4,80})\] \[\#(?:\d{5,9})\] \[(?:\d{1,6})\] \[([123])\] \[((?:(?:\d{1,3}\.){3}\d{1,3})|(?:(?:(?:[[:xdigit:]]{0,4}):){1,7}(?:[[:xdigit:]]{0,4})))\] \[(?:[\d]{3})\] \[(?:[a-zA-Z]{3,6})\] \[[^\]]{1,}\] \[([^\]]{1,})/', $line, $matches ) ) {
list(, $severity, $ip, $event) = $matches;
if ( '3' === $severity || '2' === $severity || ('1' === $severity && 'Blocked access to the login page' === $event) ) {
$cf = new NinjaCloudFlareConnector();
$cf->ip_block( $ip );
}
}
}

class NinjaCloudFlareConnector {

const BAN_TIME = 600; // 10 minutes.

private $api_token = '[PLACE TOKEN SECRET HERE]';
private $account_id = '[PLACE ACCOUNT ID HERE]';
private $db_file = __DIR__ . '/.ht-cf-rules.sqlite';

/**
* Nothing to do in public constructor.
*/
function __construct() {

}

/**
* Not cloneable.
*/
protected function __clone() {

}

/**
* Not serializeable.
* @throws \Exception
*/
public function __wakeup() {
throw new \Exception( 'Can not unserialize ' . __CLASS__ );
}

/**
* Send a command to block an IP address to CloudFlare.
*
* @param string $ip
*/
function ip_block( $ip ) {
/**
* @var string $endpoint CloudFlare API endpoint.
*/
$endpoint = "https://api.cloudflare.com/client/v4/accounts/{$this->account_id}/firewall/access_rules/rules";

/**
* @var array $payload CloudFlare block command data.
*/
$payload = [
'mode' => 'block',
'configuration' => [
'target' => (strlen( $ip ) < 16) ? 'ip' : 'ip6',
'value' => $ip
],
'notes' => 'Banned by NinjaFirewall'
];

$ch = curl_init( $endpoint );
curl_setopt_array( $ch, [
CURLOPT_RETURNTRANSFER => true,
CURLOPT_POST => true,
CURLOPT_POSTFIELDS => json_encode( $payload ),
CURLOPT_HTTPHEADER => [
'Authorization: Bearer ' . $this->api_token,
'Content-Type: application/json',
],
] );

$response = curl_exec( $ch );
$httpCode = curl_getinfo( $ch, CURLINFO_HTTP_CODE );
$result = json_decode( $response, true );

if ( $httpCode === 200 && $result[ 'success' ] ) {
$this->do_maintenance( $ip, $result );
}
}

/**
* Keep database and CloudFlare rules up-to-date.
*
* @param string $ip Newly banned IP address.
* @param array $json_results Results from ban on CloudFlare.
*/
private function do_maintenance( $ip, $json_result ) {
$db = null;
if ( file_exists( $this->db_file ) ) {
$db = new SQLite3( $this->db_file );
} else {
$db = $this->create_db( $this->db_file );
}
$this->save_access_rule( $db, $json_result );
$rule_ids = $this->get_old_access_rules( $db );
$rule_ids = $this->delete_access_rules( $rule_ids );
$this->remove_deleted_access_rules( $db, $rule_ids );
$db->close();
}

/**
* Create sqlite3 database.
*
* @param string $db_file Full path to database file.
* @return SQLite3 SQLite3 object.
*/
private function create_db( $db_file ) {
$db = new SQLite3( $db_file );
$db->exec( 'CREATE TABLE IF NOT EXISTS rules (rule_id TEXT NOT NULL, created INTEGER NOT NULL)' );
return $db;
}

/**
* Save the access rule ID and creation date in sqlite.
*
* @param SQLite3 $db SQLite3 object
* @param array $json_result
*/
function save_access_rule( $db, $json_result ) {
$rule_id = $json_result[ 'id' ];
$db->exec( "INSERT INTO rules (rule_id, created) VALUES ('{$rule_id}',unixepoch('now')" );
}

/**
* Find expired access rules.
*
* @param SQLite3 $db SQLiet3 object.
* @return array Array of expired rule ids.
*/
function get_old_access_rules( $db ) {
$rule_ids = [];

/**
* @var SQLite3Result $result Query results.
*/
$results = $db->query( 'SELECT rule_id FROM rules WHERE created < ' . time() - self::BAN_TIME );
while ( $row = $results->fetchArray() ) {
$rule_ids[] = $row[ 'rule_id' ];
}

return $rule_ids;
}

/**
* Delete list of rules.
* @param array $rule_ids Array of rule IDs to delete,
* @return array Array of successfully deleted rule IDs.
*/
function delete_access_rules( $rule_ids ) {

$deleted_rules = [];

foreach ( $rule_ids as $rule_id ) {
$ch = curl_init( "https://api.cloudflare.com/client/v4/accounts/{$this->account_id}/firewall/access_rules/rules/{$rule_id}" );

curl_setopt_array( $ch, [
CURLOPT_RETURNTRANSFER => true,
CURLOPT_CUSTOMREQUEST => 'DELETE',
CURLOPT_HTTPHEADER => [
'Authorization: Bearer ' . $this->api_token,
'Content-Type: application/json',
],
] );
$response = curl_exec( $ch );
$httpCode = curl_getinfo( $ch, CURLINFO_HTTP_CODE );
$result = json_decode( $response, true );

if ( $httpCode === 200 && !empty( $result[ 'success' ] ) ) {
$deleted_rules[] = $rule_id;
}
}

return $deleted_rules;
}

/**
* Delete a list of rule IDs from sqlite DB.
*
* @param SQLite3 $db SQLite3 object.
* @param array $deleted_rules List of rule IDs to delete from sqlite.
*/
function remove_deleted_access_rules( $db, $deleted_rules ) {
foreach ( $deleted_rules as $rule_id ) {
$db->exec( "DELETE FROM rules WHERE rule_id='{$rule_id}'" );
}
}
}

Thread Starter wp_kc

(@wp_kc)

The shared-hosting .htninja script above had a bug in it that prevented IP blocks from ever being unblocked. Here is the working code…

<?php

/**
* Use CloudFlare's reported connecting IP address.
*/
if ( !empty( $_SERVER[ "HTTP_CF_CONNECTING_IP" ] ) &&
filter_var( $_SERVER[ "HTTP_CF_CONNECTING_IP" ], FILTER_VALIDATE_IP ) ) {
$_SERVER[ "REMOTE_ADDR" ] = $_SERVER[ "HTTP_CF_CONNECTING_IP" ];
}

/**
* Retrieve the line written to the firewall log.
* Note: $line doesn't include the Line Feed (\n).
*
* @param string $line Log file line.
*/
function nfw_custom_user_log( $line ) {
/**
* Only act on severity levels: 1, 2, 3
*/
if ( 1 === preg_match( '/^\[(?:[\d]{10})\] \[(?:[\d\.]{1,8})\] \[(?:[\w\.\-]{4,80})\] \[\#(?:\d{5,9})\] \[(?:\d{1,6})\] \[([123])\] \[((?:(?:\d{1,3}\.){3}\d{1,3})|(?:(?:(?:[[:xdigit:]]{0,4}):){1,7}(?:[[:xdigit:]]{0,4})))\] \[(?:[\d]{3})\] \[(?:[a-zA-Z]{3,6})\] \[[^\]]{1,}\] \[([^\]]{1,})/', $line, $matches ) ) {
list(, $severity, $ip, $event) = $matches;
if ( '3' === $severity || '2' === $severity || ('1' === $severity && 'Blocked access to the login page' === $event) ) {
$cf = new NinjaCloudFlareConnector();
$cf->ip_block( $ip );
}
}
}

class NinjaCloudFlareConnector {

const BAN_TIME = 600; // 10 minutes.

private $api_token = '[ENTER API TOKEN HERE]';
private $account_id = '[ENTER ACCOUNT ID HERE]';
private $db_file = __DIR__ . '/.ht-cf-rules.sqlite';

/**
* Nothing to do in public constructor.
*/
function __construct() {

}

/**
* Not cloneable.
*/
protected function __clone() {

}

/**
* Not serializeable.
* @throws \Exception
*/
public function __wakeup() {
throw new \Exception( 'Can not unserialize ' . __CLASS__ );
}

/**
* Send a command to block an IP address to CloudFlare.
*
* @param string $ip
*/
function ip_block( $ip ) {
/**
* @var string $endpoint CloudFlare API endpoint.
*/
$endpoint = "https://api.cloudflare.com/client/v4/accounts/{$this->account_id}/firewall/access_rules/rules";

/**
* @var array $payload CloudFlare block command data.
*/
$payload = [
'mode' => 'block',
'configuration' => [
'target' => (false === strpos( $ip, ':' )) ? 'ip' : 'ip6',
'value' => $ip
],
'notes' => 'Banned by NinjaFirewall: ' . date( DATE_RFC2822 ),
];

$ch = curl_init( $endpoint );
curl_setopt_array( $ch, [
CURLOPT_RETURNTRANSFER => true,
CURLOPT_POST => true,
CURLOPT_POSTFIELDS => json_encode( $payload ),
CURLOPT_CONNECTTIMEOUT => 5,
CURLOPT_TIMEOUT => 10,
CURLOPT_HTTPHEADER => [
'Authorization: Bearer ' . $this->api_token,
'Content-Type: application/json',
],
] );

$response = curl_exec( $ch );
$httpCode = curl_getinfo( $ch, CURLINFO_HTTP_CODE );
$result = json_decode( $response, true );

if ( $httpCode === 200 && $result[ 'success' ] ) {
$this->do_maintenance( $ip, $result );
}
}

/**
* Keep database and CloudFlare rules up-to-date.
*
* @param string $ip Newly banned IP address.
* @param array $json_results Results from ban on CloudFlare.
*/
private function do_maintenance( $ip, $json_result ) {
$db = null;
try {
if ( file_exists( $this->db_file ) ) {
$db = new SQLite3( $this->db_file );
} else {
$db = $this->create_db( $this->db_file );
}
$db->busyTimeout( 5000 );
$this->save_access_rule( $db, $json_result );
$rule_ids = $this->get_old_access_rules( $db );
$rule_ids = $this->delete_access_rules( $rule_ids );
$this->remove_deleted_access_rules( $db, $rule_ids );
$db->close();
} catch ( Exception $e ) {
if ( $db ) {
$db->close();
}
}
}

/**
* Create sqlite3 database.
*
* @param string $db_file Full path to database file.
* @return SQLite3 SQLite3 object.
*/
private function create_db( $db_file ) {
$db = new SQLite3( $db_file );
$db->exec( 'CREATE TABLE IF NOT EXISTS rules (rule_id TEXT NOT NULL, created INTEGER NOT NULL);' );
return $db;
}

/**
* Save the access rule ID and creation date in sqlite.
*
* @param SQLite3 $db SQLite3 object
* @param array $json_result
*/
function save_access_rule( $db, $json_result ) {
$rule_id = $json_result[ 'id' ];
$stmt = $db->prepare( 'INSERT INTO rules (rule_id, created) VALUES (:rule_id, :created);' );
$stmt->bindValue( ':rule_id', $rule_id, SQLITE3_TEXT );
$stmt->bindValue( ':created', time(), SQLITE3_INTEGER );
$stmt->execute();
}

/**
* Find expired access rules.
*
* @param SQLite3 $db SQLiet3 object.
* @return array Array of expired rule ids.
*/
function get_old_access_rules( $db ) {
$rule_ids = [];

/**
* @var SQLite3Result $result Query results.
*/
$results = $db->query( 'SELECT rule_id FROM rules WHERE created < ' . (time() - self::BAN_TIME) . ';' );
while ( $row = $results->fetchArray() ) {
$rule_ids[] = $row[ 'rule_id' ];
}

return $rule_ids;
}

/**
* Delete
* @param array $rule_ids Array of rule IDs to delete,
* @return array Array of successfully deleted rule IDs.
*/
function delete_access_rules( $rule_ids ) {

$deleted_rules = [];

foreach ( $rule_ids as $rule_id ) {
$ch = curl_init( "https://api.cloudflare.com/client/v4/accounts/{$this->account_id}/firewall/access_rules/rules/{$rule_id}" );

curl_setopt_array( $ch, [
CURLOPT_RETURNTRANSFER => true,
CURLOPT_CUSTOMREQUEST => 'DELETE',
CURLOPT_CONNECTTIMEOUT => 5,
CURLOPT_TIMEOUT => 10,
CURLOPT_HTTPHEADER => [
'Authorization: Bearer ' . $this->api_token,
'Content-Type: application/json',
],
] );
$response = curl_exec( $ch );
$httpCode = curl_getinfo( $ch, CURLINFO_HTTP_CODE );
$result = json_decode( $response, true );

if ( $httpCode === 200 && !empty( $result[ 'success' ] ) ) {
$deleted_rules[] = $rule_id;
}
}

return $deleted_rules;
}

/**
* Delete a list of rule IDs from sqlite DB.
*
* @param SQLite3 $db SQLite3 object.
* @param array $deleted_rules List of rule IDs to delete from sqlite.
*/
function remove_deleted_access_rules( $db, $deleted_rules ) {
$stmt = $db->prepare( 'DELETE FROM rules WHERE rule_id = :rule_id;' );
foreach ( $deleted_rules as $rule_id ) {
$stmt->bindValue( ':rule_id', $rule_id, SQLITE3_TEXT );
$stmt->execute();
$stmt->reset();
}
}
}