and 7 for unknown-users (with max-bytes 5250 and 1500 respectively). This
allows pasting a short snippet of code, config file, text from a site, etc.
With multiline you have the guarantee that:
1) You will see the entire text with no delay between lines
2) You won't see another persons chat half-way through such a paste
3) For multiline supporting clients it is now clear that all the text
belongs to each other, which can make selecting/copying it easier.
This basically means short snippets/pastes like that can be completely on
IRC again. No need for a pastebin for it. Though, you may still need such
a service if you are pasting more lines.
Regarding the implementation in UnrealIRCd:
* Clients without multiline get individual fallback lines (concat lines
merged, blank lines skipped, as per spec). And we know that clients like
weechat - which does support multiline - also shows all lines and not
only a few plus snippet style "[.."]. That is another reason for only
allowing 15 lines by default and not something much more. Otherwise all
those clients would get a big wall of text, which just sucks.
* Spamfilter (also) runs on the full text of all lines together, so
splitting a phrase across lines does not evade spamfilter.
* Fakelag: a client can send the BATCH start+PRIVMSG (or NOTICE)+BATCH end
at full speed. We impose no fake lag there. Also, the multiline default
max-lines and max-bytes are lower than the example class::recvq of 8000,
so should be perfectly safe. If the entire BATCH is accepted then we
will impose fake-lag afterwards, with a cap of 15 seconds maximum.
If the BATCH is rejected, we impose half the fakelag plus 2sec.
* If the time between BATCH start and BATCH end is more than 15 seconds
then the BATCH is rejected (set::multiline::batch-timeout).
* The BATCH is atomic (either you see it all, or you see none of it):
* When the client sends it to server, it is buffered first.
* Only after the batch close the server indicates if it is accepted
or rejected. This has various reasons, two of them are: 1) The client
is going to send everything in one go anyway and not wait for a
response between each PRIVMSG, and 2) we can't do many checks in the
buffering stage and skip those after, that would cause a TOCTOU
problem (eg. a banned user still being able to speak).
* If any line gets rejected due to spamfilter or other case
(eg +c, +b ~text with block, etc etc), the entire batch is rejected
* Locally we deliver all or nothing (as said)
* S2S we buffer the batch as well, so if a server splits after having
received 10 lines out of 15, then clients will not see anything.
* We send max-lines and max-bytes, this is the hard upper limit.
* A multiline can still be limited more tight if:
* +f with 't' or 'm' restricts to fewer lines,
eg +f [5t]:15, which means max 5 lines per 15 seconds,
means the max accepted multiline is 5 for that channel.
* +F works the same, except that default +F normal does not
have a 't' at the moment and 'm' is very high (50) so
practically not limited by default.
* There will be a future +f flood subtype for some more control
TODO: we will send CAP NEW on unknown-users <-> known-users to
indicate the new max-lines value if you transition security groups
TODO: chat history does not yet include multiline batches.
the config file, without having to resort to things like mask %~asn:XXX;
Now you can just use:
ban user {
asn { 11111; 22222; 33333; 44444; }
soft yes;
reason "This ASN is not allowed. If you have an account you can still bypass";
}
Requested by nobody but sounds like a good idea :)
in the EFunction but not in the actual function. That's bad since it
means the "const guarantee" got lost. And one or two similar cases with
incorrect parameter types and mismatching return types. This was
found with some analyzer, we had no bugreports with regards to this.
This will return the number of characters that are in the unicode block
with that name.
spamfilter {
rule "unicode_count('Emoticons')>2";
target { private; channel; private-notice; channel-notice; }
action block;
reason "Too much emotion";
}
In this commit we also make it so we pass the ClientContext (including
clictx->textanalysis) in crule_context.
Example output:
*** SPAMINFO ***
This will show the original text and the deconfused text which can be used in a spamfilter block with input-conversion deconfused;
Original spam text: ẔŽŽẐ𝞕ȤℤΖℨℨ𝒁𝓩ẒŹƵᏃŻẒŽℨŹ𝒵𝛧Ż𝝛𝛧ℨℤ𝜡Ƶ𝞕𝘡ŹẐ𝑍ẔẐẐΖ𝜡Ẕ𝜡Ẕ𝞕ꓜ𝚭ᏃẐẔ𝙕
Deconfused spam text: ZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZ
AntiMixedUTF8 points: 64
Number of Unicode characters in total: 50
Number of different Unicode blocks used: 8
Unicode Block breakdown (name: bytes [capped at 255]):
- Latin Extended-A: 8
- Latin Extended-B: 3
- Greek and Coptic: 2
- Cherokee: 2
- Latin Extended Additional: 12
- Letterlike Symbols: 6
- Lisu: 1
- Mathematical Alphanumeric Symbols: 16
Make match_spamfilter use the clictx->textanalysis->deconfused rather than
calculating its own. The latter will probably disappear altogether.
Unrelated but also fixed: properly set e->unicode_blocks.
It now passes 'clictx' which at the moment only has clictx->cmd which
points to the command handler. So only useful in very few cases where
you have like a generic command handler and thus have no idea for which
command you are being called. In the future, with this new ClientContext
struct, we can simply add new fields to the struct without breaking
things in the core and in (third party) modules.
If you use the magic functions in your modules CMD_FUNC(cmd_mycmd),
OVERRIDE_FUNC(myoverride), CALL_NEXT_COMMAND_OVERRIDE() and such then
you shouldn't have any compile errors as these will use the correct
prototypes and variable names automatically. In a few cases you can't
use these, in which case you will need to update your modules.
Now this works like:
if the time param exists, even without a reason, it will be checked if it's a time param. if it's not a time param, it'll be considered to be the reason (or the first part of it anyway)
Reported by PeGaSuS in https://bugs.unrealircd.org/view.php?id=6105
ban ip {
mask { 1.1.1.1; 2.2.2.2; 3.3.3.3; }
reason "Go away";
}
Or the alternate form:
ban ip {
mask 1.1.1.1;
mask 2.2.2.2;
mask 3.3.3.3;
reason "Go away";
}
Suggested by magic000 in https://bugs.unrealircd.org/view.php?id=4599
Note that this is not a Mask item, these are special, hence the
special code.
This to replace the scattered IP setting. It is very important to always
use set_client_ip() from this point. Everywhere!
Also, in addition to client->ip, this adds client->rawip that contains
the IP in network byte order. In older UnrealIRCd versions we always had
the raw IP but not the IP as a string, so we moved to IP as a string,
but it can be useful to have both in terms of optimizations.
Of course, then the client->ip and client->rawip always need to 100% match,
hence the set_client_ip().
This also changes IsIPV6() to do A BUGFIX, it changes it from:
* if local user is the user connected over IPv6? Otherwise, does it have ':' in the IP?
To:
* check if the IPv6 flag is set (which is set if IP contains ':')
This may seem insignificant but it means that for spoofed IP addresses,
such as WEBIRC or transparant proxy, we use the correct transport.
Previously, if the proxy was IPv6 then even if the spoofed user was using
IPv4, the ident check would still be tried over IPv6. That sort of fun.
From now in, in such a situation client->local->socket_type will be
SOCKET_TYPE_IPV6 but since client->ip (and rawip) will contain IPv4
the IsIPV6() will actually return false, as it should be.
Also, in the HOOKTYPE_IP_CHANGE, enforce that if HOOK_DENY is returned,
the the user is killed by dead_link(). The user must be killed because
that is what we expect, and you cannot use exit_client() because from
some code paths that would be too much freed structures / hassle,
as a comment in src/modules/connect-flood.c correctly states:
/* There are two reasons why we can't use exit_client() here:
* 1) Because the HOOKTYPE_IP_CHANGE call may be too deep.
* Eg: read_packet -> webserver_packet_in ->
* webserver_handle_request_header -> webserver_handle_request ->
* RunHook().... and then returning without touching anything
* after an exit_client() would not be feasible.
* 2) Because in HOOKTYPE_ACCEPT we always need to use dead_socket
* if we want to print a friendly message to TLS users.
*/
at selected places (there needs to be explicit code in place to handle this).
At the moment it is supported at two places only:
* For spamfilters (was already possible via crules via ::rule with
a destination('xyz') but now non-crule destination "#xyz"; works as well, eg:
spamfilter {
...
except {
destination "#main";
}
}
Note that if you want to exempt a destination in all spamfilters,
we already have set::spamfilter::except for that!
* In restrict commands for like channel-message and such:
set {
restrict-commands {
channel-message {
except {
connect-time 600;
destination "#test";
}
}
}
}
Allow passing a crule_context via user_allowed_by_security_group_context()
and make user_allowed_by_security_group() call that.
Actually document spamfilter::except online in the docs (yeah you
won't see it in this commit, just mentioning...)
And yeah, by now i wonder if we should really call it crule_context
since it is more like a security group matching context, but.. whatever.
Hmm... we should probably use json_expand_tkl() differently for match items
instead of returning "<match-item>" literally. Consider this a TODO item :D
This only happens for config-based bans that can't be removed anyway, so..
For config-based spamfilters, the reason was not escaped, meaning that
spaces and underscores did not work as expected.
For example, in "STATS spamfilter" the spaces were displayed as-is
which means that the numeric output was not really parsable.
Apparently this bug exists since UnrealIRCd 5 already...
action { set REPUTATION--; } and similar.
Also enhancement to reputation S2S traffic, to support decreasing:
*
+ * Since UnrealIRCd 6.0.2+ there is now also asterisk-score-asterisk:
+ * :server REPUTATION 1.2.3.4 *2*
+ * The leading asterisk means no reply will be sent back, ever, and the
+ * trailing asterisk will mean it is a "FORCED SET", which means that
+ * servers should set the reputation to that value, even if it is lower.
+ * This way reputation can be reduced and the reducation can be synced
+ * across servers, which was not possible before 6.0.2.
+ *
So if you are actually decreasing reputation, you need all servers on
6.0.2 or higher for it to work properly, otherwise the other servers
don't decrease it, and next connect the highest wins again, etc.
Often you have default values for the config, and then a subsequent config
parsing run would overwrite the return value (= memory leak), merging/appending
would make no sense either, so it would force a free in all code before
calling us, well... let's just deal with it ourselves instead then ;)
The spamfilter::action stop ill prevent processing other spamfilters.
This would normally be a bit unusual, and potentially dangerous when you
do exclude things this way, but can be useful in some circumstances.
Stopping only affects the same type of spamfilters (general or central
spamfilters), so they don't interfere.
The tkldb write DB bug had to do with that it was processing
central spamfilters, which should be skipped just like config
based spamfilters were already skipped.
you can now configure to hide the message content in spamfilter hit
messages. Generally it is very useful to see if a spamfilter hit is
correct or not, so the default is 'always', but it also has privacy
implications so there is now this option to disable it.
Suggested by alice, quite a while ago.
https://www.unrealircd.org/docs/Set_block#set::spamfilter::show-message-content-on-hit
Also as mentioned there:
UnrealIRCd has the following spying countermeasure (for many years) to help
that spamfilters are not abused for spying. When a spamfilter hit happens
that has an action like gline or blocking, it is visible to the user that an
action was taken. There is also the action 'warn', which means: take no
action and only warn IRCOps, that one would be easy to use as a spy tool, so
when this happens and message content was revealed, numeric 659
(RPL_SPAMCMDFWD) is sent to the client to indicate that the message is
allowed through but IRCOps were informed.
With this new set::spamfilter::show-message-content-on-hit feature, when
the message content was hidden due to this setting (eg due to 'never' or
'channel-only'), the warn message will not be sent as there is no need to
inform the user in such a case.
to limit actions to limit-ban-action as the highest, and limit
ban times to limit-ban-time the highest, see
https://www.unrealircd.org/docs/Central_Spamfilter
This also changes highest_spamfilter_action() to highest_ban_action().
Bram Matthys