This so we can use fast(er) techniques here and there.
New functions are:
channel_has_invisible_users(client)
set_user_invisible(client, channel, 1|0)
Existing functions:
invisible_user_in_channel(client, channel)
user_can_see_member(user, target, channel)
user_can_see_member_fast()
This is work in progress, although the tests seem to pass atm.
things by making the keys with the most lookups first, e.g. "reputation",
"geoip", "certfp". This order is based on actual lookup counts during a
quick test with 250 clones doing some typical IRC traffic.
Key: Lookups: Position before: After split: After split+order:
"reputation" 20362 37 14 1
"geoip" 10555 44 15 2
"certfp" 9264 23 8 3
"webirc" 7407 27 10 4
"websocket" 7110 55 19 5
We could also consider going for a hash table, but this may be "good enough" for now.
up moddata_client_get() etc -> findmoddata_byname().
Apparently we have 52 moddata registrations (that is without 3rd party modules)
so otherwise it is a loooong linked list.
This function was added a short while ago, and well it seems to be
able to be possible in a module. Since the 'isupport' module is mandatory
and this is ISUPPORT related, it is the right place.
Can't move isupport_snapshot() because modules might not be loaded yet
or things are currently unloading, i think. Not important anyway.
Also, make things work if there are more changes than would fit
on one isupport line. Although I didn't really test this..
Ended up splitting things in 3 helper functions to avoid some
goto and/or duplicate code and stuff. The alternative was, surprisingly,
even more ugly.
* Calling from source is now in a separate function: int can_use_nick(Client *client, const char *nick)
* For hooks: don't free the reject reason, must use static storage like all other hooks
(TODO: clarify in all hooks?)
* Move it up a bit, right before find_qline
TODO (not necessarily me :D):
* Make it an efunc
* Also call it from some other places that do find_qline, like rpc/user.c
* You may want to prod 3rd party modules like SANICK
Example output:
*** SPAMINFO ***
This will show the original text and the deconfused text which can be used in a spamfilter block with input-conversion deconfused;
Original spam text: ẔŽŽẐ𝞕ȤℤΖℨℨ𝒁𝓩ẒŹƵᏃŻẒŽℨŹ𝒵𝛧Ż𝝛𝛧ℨℤ𝜡Ƶ𝞕𝘡ŹẐ𝑍ẔẐẐΖ𝜡Ẕ𝜡Ẕ𝞕ꓜ𝚭ᏃẐẔ𝙕
Deconfused spam text: ZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZ
AntiMixedUTF8 points: 64
Number of Unicode characters in total: 50
Number of different Unicode blocks used: 8
Unicode Block breakdown (name: bytes [capped at 255]):
- Latin Extended-A: 8
- Latin Extended-B: 3
- Greek and Coptic: 2
- Cherokee: 2
- Latin Extended Additional: 12
- Letterlike Symbols: 6
- Lisu: 1
- Mathematical Alphanumeric Symbols: 16
switches like antimixedutf8 did, and counts the number of characters
used per unicode block. Potentially more can be added later, this is
flexible and modules can add stuff (..well not yet.. the struct is
missing some members..).
Use it from antimixedutf8 so that it now uses the new code, which is
similar to what I made and then reverted in July 2023:
https://github.com/unrealircd/unrealircd/commit/3e2f668f10fccedfd035526d7b20d7ca6819a8ae
..except that it now calculated in src/modules/utf8functions.c.
But yeah, this needs more testing and possibly (default) score
adjustments to deal with false positives !! And a warning in release notes :D
Put the text analysis in ClientContext member textanalysis,
so typically accessed through clictx->textanalysis.
Note that this struct can (and often is) NULL, for example if it is
a remote client, if it is not a PRIVMSG/NOTICE (will improve later)
or if the utf8functions module is not loaded (to keep things optional).
BREAKING CHANGE is that ClientContext is now passed in the
HOOKTYPE_CAN_SEND_TO_CHANNEL and HOOKTYPE_CAN_SEND_TO_USER hooks.
So HOOKTYPE_CAN_SEND_TO_USER prototype changed from:
int hooktype_can_send_to_user(Client *client, Client *target, const char **text, const char **errmsg, SendType sendtype);
To:
int hooktype_can_send_to_user(Client *client, Client *target, const char **text, const char **errmsg, SendType sendtype, ClientContext *clictx);
And HOOKTYPE_CAN_SEND_TO_CHANNEL prototype changes from:
int hooktype_can_send_to_channel(Client *client, Channel *channel, Membership *member, const char **text, const char **errmsg, SendType sendtype);
To:
int hooktype_can_send_to_channel(Client *client, Channel *channel, Membership *member, const char **text, const char **errmsg, SendType sendtype, ClientContext *clictx);
A side-affect of this change for antimixedutf8 purposes is that,
while the analysis is only done once per line, the 'actions' are
performed for each target, so the action will run 4 times for
"PRIVMSG a,b,c,d :text" although that may not be important in
practice. Just mentioning.
I started work on this back then but didn't finalize it. Now I
have to figure out what was left to be done :D. Other than the
obvious case of seeing some debugging code that prints out for
every converted character. Not yet visible / usable by end-users!
Also fix documentation for ~10 hooks to mention the hook name.
Obviously, the maxperip module is loaded by default (in modules.default.conf)
but it is nice to have the 400+ lines contained in a separate module
rather than being in the nick module that does NICK/UID handling.
Will look at moving more later..
It now passes 'clictx' which at the moment only has clictx->cmd which
points to the command handler. So only useful in very few cases where
you have like a generic command handler and thus have no idea for which
command you are being called. In the future, with this new ClientContext
struct, we can simply add new fields to the struct without breaking
things in the core and in (third party) modules.
If you use the magic functions in your modules CMD_FUNC(cmd_mycmd),
OVERRIDE_FUNC(myoverride), CALL_NEXT_COMMAND_OVERRIDE() and such then
you shouldn't have any compile errors as these will use the correct
prototypes and variable names automatically. In a few cases you can't
use these, in which case you will need to update your modules.
Previously if a new history item was added (because someone sent a message)
we would always append at the end of chat history buffer of the channel.
Now we put the message at the position decided by the "time" message tag,
which could be at the end but also slightly before that.
* Upside: should result in a consistent chat history on all servers
* Downside: if your server time is off for several seconds then it
could look a little weird. Then again, it would already have looked weird
in real live chat with timestamps and when replaying chat history probably.
Also add some simple optimizations: in the log line object we now have direct
pointers to the msgid and time strings, so the code doesn't need to do a
find_mtag() all the time. This should lower CPU usage during log playback
and also makes things more simple in the source code.
I did some testing with various history injection variants but this needs
more extensive testing.
and use it not only from vhost { } block code but also for like
blacklist::reason.
This so the same variables with the same names are available at
those places.
Supported are:
$nick, $username, $realname, $ip, $hostname, $server, $account,
$operlogin, $operclass, $country_code (xx for unknown),
$asn (0 for unknown).
This so if there is ever an issue, we can hot-patch it. This affects
exit_client(), exit_client_fmt(), exit_client_ex(), banned_client(),
and various (internal) help functions.
This also means you cannot call these functions during TEST/INIT (eg
during REHASH) since the 'quit' module which provides these modules
may not be loaded yet. I don't think that's a situation/problem but
this needs some more testing.
This to replace the scattered IP setting. It is very important to always
use set_client_ip() from this point. Everywhere!
Also, in addition to client->ip, this adds client->rawip that contains
the IP in network byte order. In older UnrealIRCd versions we always had
the raw IP but not the IP as a string, so we moved to IP as a string,
but it can be useful to have both in terms of optimizations.
Of course, then the client->ip and client->rawip always need to 100% match,
hence the set_client_ip().
This also changes IsIPV6() to do A BUGFIX, it changes it from:
* if local user is the user connected over IPv6? Otherwise, does it have ':' in the IP?
To:
* check if the IPv6 flag is set (which is set if IP contains ':')
This may seem insignificant but it means that for spoofed IP addresses,
such as WEBIRC or transparant proxy, we use the correct transport.
Previously, if the proxy was IPv6 then even if the spoofed user was using
IPv4, the ident check would still be tried over IPv6. That sort of fun.
From now in, in such a situation client->local->socket_type will be
SOCKET_TYPE_IPV6 but since client->ip (and rawip) will contain IPv4
the IsIPV6() will actually return false, as it should be.
Also, in the HOOKTYPE_IP_CHANGE, enforce that if HOOK_DENY is returned,
the the user is killed by dead_link(). The user must be killed because
that is what we expect, and you cannot use exit_client() because from
some code paths that would be too much freed structures / hassle,
as a comment in src/modules/connect-flood.c correctly states:
/* There are two reasons why we can't use exit_client() here:
* 1) Because the HOOKTYPE_IP_CHANGE call may be too deep.
* Eg: read_packet -> webserver_packet_in ->
* webserver_handle_request_header -> webserver_handle_request ->
* RunHook().... and then returning without touching anything
* after an exit_client() would not be feasible.
* 2) Because in HOOKTYPE_ACCEPT we always need to use dead_socket
* if we want to print a friendly message to TLS users.
*/
Note that this is still a dumb interface and not a real proper
authentication framework.
This adds HOOKTYPE_SASL_AUTHENTICATE and HOOKTYPE_SASL_MECHS and
also provides 3 functions: sasl_succeeded(), sasl_failed() and
a helper function decode_authenticate_plain() for AUTHENTICATE PLAIN.
set::central-blocklist::spamreport and ::spamreport-enabled are now GONE.
We now require a normal spamreport block, just like for other spamreport
functionality. So, if you want to enable this feature, use:
spamreport unrealircd { type central-spamreport; }
See https://www.unrealircd.org/docs/Central_spamreport for all info.
You can use CBL with central spamreport or central spamreport without CBL.
All explained at that URL.
The `watch-check` function now has a new argument which can be used to pass data to watch_notify callbacks.
New `watch_add` and `watch_del` hooks are called whenever new entries are created or removed.
New `monitor_notification` hook is called whenever a RPL_MONONLINE or RPL_MONOFFLINE is being sent, so a module can add its own notification besides it.
The LoadPersistent*()/SavePersistent*() functions caused moddata to be
tagged with ->unloaded=1. Though it seems it caused no real issues this
is not good... we now properly tag them as 0 and the like. Also did a
code cleanup / overhaul on that system as well.
For other ModData we now handle the case where a module is loaded with
with a newer version and that newer version is no longer having certain
moddata, eg the name changed or it no longer needs it.
KNOWN ISSUE:
Unfortunately we cannot call the free function for the old moddata that
is no longer being handled by the newer version of the module, since the
module is already unloaded. So this will result in a memory leak, but
not in a crash.
KNOWN ISSUE:
Similarly, for SavePersistentPointer() there is a free function, again
this is called just fine if the module is permanently unloaded but NOT
if the module is reloaded with the same name and no longer is interested
in the persistent pointer object. Again, here too, that would result
in a memory leak but not in a crash.
Fortunately the "known issues" are rare. Fixing these is impossible
with the current module API because modules are unloaded after MOD_TEST
and before MOD_INIT, and only after MOD_INIT we know which moddata
is handled by the new version of the module. To change that we would
need to keep the old module around until after MOD_INIT of the new
module (so we can call free functions in the old module), but that
means delaying the MOD_UNLOAD for the old modules until after MOD_INIT
of the new modules, which changes the sequence too much that i don't
dare to do that. For example, it would mean a database save routine
in the old module would only be called after MOD_INIT finished in the
new module, which may be unexpected since right now MOD_UNLOAD is
called before MOD_INIT and maybe the db loading is done in MOD_INIT,
which would need to be moved to MOD_LOAD. That's just one example,
there may be others. I think such a change can only be done on a major
UnrealIRCd version change, so we will have to live this for now.
As said, fortunately it is a corner case.
action { set REPUTATION--; } and similar.
Also enhancement to reputation S2S traffic, to support decreasing:
*
+ * Since UnrealIRCd 6.0.2+ there is now also asterisk-score-asterisk:
+ * :server REPUTATION 1.2.3.4 *2*
+ * The leading asterisk means no reply will be sent back, ever, and the
+ * trailing asterisk will mean it is a "FORCED SET", which means that
+ * servers should set the reputation to that value, even if it is lower.
+ * This way reputation can be reduced and the reducation can be synced
+ * across servers, which was not possible before 6.0.2.
+ *
So if you are actually decreasing reputation, you need all servers on
6.0.2 or higher for it to work properly, otherwise the other servers
don't decrease it, and next connect the highest wins again, etc.
This is a mandatory module to load, and included in modules.default.conf.
This also meant that the crule_test() etc efunctions are available
before running config test routines, so we now have a flag for
early efuncs. I guess we could consider doing that for all efuncs
though, so not sure if this flag is really needed.
that was added late in 6.1.1 development to fix a crash with removing
websocket listeners. Now replaced with a generic HOOKTYPE_CONFIG_LISTENER
that is not only called for removed listeners, but for all listeners.
a function called start_dns_and_ident_lookup(). This can then
be easily called from other places as well, like the code k4be
did in src/modules/websocket.c to handle proxies.
Side-effect is that ident lookups would now be done, if we are
configured to do so, for forwarded webirc stuff (not that I
think many people use that feature at the moment...).
(or changing the port number). Reported by Nini.
Rather complex case: when the listen block is removed, obviously
the config hooks are not called for the (now non-existing) listen
block, and thus the websocket->request_handler and such are not
set to the new address of the websocket handler.
We now use a slightly silly workaround / new hook to fix this
corner case. Ideally there would be an extra layer in-between
like a handler lookup by name, or something like that.
(Or make the websocket module PERM but we don't want that!)
This so you can use throttling exceptions (eg in ELINE) on hostnames.
That is, the above is during normal circumstances. Similar to previous
commit we will turn this feature of during high connection rates.
That is a TODO item.
This adds user_can_see_member_fast() which is used in at least 3 places
now, more places may follow later. It has extra paramters for membership
and membership modes that is very likely already looked up by the caller
(or if not, it is worth doing so by the caller).
This is work in progress so if everything crashes or people mysteriously
seem not present in channels (or the other way around) i would not be
surprised :D.
When sending to channel members this will cache full IRC protocol
lines, including message tags and \r\n, for similar clients.
This avoid the need for many mtags_to_string() calls and also
entire parts of sendbuf_to_one() can be skipped as well.
The "Similar clients" cache entries are defined as clients that:
1) Are of the same type: normal local client, ircop local client
or remote client.
2) Have the same CAPs set, that is: we only look at CAPs that actually
have anything to do with message tags ('clicaps_affecting_mtag')
3) Optionally there can be an explicit line_opts. It is not used yet
but could be used when there are different type of lines sent
depending on other criteria, such as chanop status or something
else that doesn't fit in #1 and #2.
can be used from +f/+F as an action. You need to specify for which
flood type your mode is, eg `cmode.flood_type_action = 'j';` for joinflood.
Currently a mode can only choose one flood type action due to +f/+F
timer fights that could otherwise occur, but that shouldn't be too
much of an issue since we can live with that in core as well.
This will communicate the original issuer of a command.
For example an "SAMODE #test +s" results in a SAMODE coming from
:maintest.test.net MODE ....etc....
And with this feature, we will communicate the IRCOp who did it:
@unrealircd.org/issued-by=OPER:Syzop..etc....
This tag is only sent to servers and to IRCOps, not to ordinary users.
The plan is to support the following variants:
Services: unrealircd.org/issued-by=SERVICES:NickServ@services.test.net
IRCOp: unrealircd.org/issued-by=OPER:Syzop@maintest.test.net:Operblock_name
JSON-RPC: unrealircd.org/issued-by=RPC:adminpanel@irc1.test.net:Adminpanel_Actual_User
This first commit only adds SERVICES and OPER in the handlers of the
SVSNICK and SAMODE commands. The JSON-RPC variant and all of the other
commands have not been done yet.
This is checked for both local and remote services linking in.
Naturally, the list can be expanded to include more services that
really need ulines { }, and not statistical services or some other
purpose non-unrealircd servers, which is the reason why cannot
blindly assume all non-unrealircd servers require ulines.
This should hopefully help users a lot with "mysterious" issues
with services that we see too often in the support channel.
Suggested in https://bugs.unrealircd.org/view.php?id=5742
Note that this does require services to communicate their software
version via EAUTH. Anope does this for years already, but atheme only
does so since 10 days ago (git only, presumably not released yet)
after Valware filed a PR.
Bram Matthys