This reformats entire UnrealIRCd source code, according to
.clang-format which was previously commited.
This is reproducable using clang-format version 21.1.8.
These are only visual changes, they have no effect on code.
This commit will be added to .git-blame-ignore-revs so 'git blame'
and the similar blame view on GitHub will ignore this commit.
* Add a trailing comma after the last element. Without it clang-format
glues the closing brace onto the last element, like "NULL};".
* Write '= {' on the declaration line instead of a lone '{' on the next
line. clang-format keeps such a lone '{' as-is but gives it and the
whole body an extra continuation indent, which looks weird.
actually don't belong to a variable but are like instructions
if someone is adding new members, or they are for grouping,
move these to the top doxygen comment because otherwise future
clang-format would align them in separate groups which looks ugly
If we have this on "Always" then in .c files it will also align /* comments */
which is very shitty, you get things like:
n = callsomefunc(hfdshfsd, fdhshfsdhsfd, fdshsfdhfsd, fsdhsfdh); /* try this first */
if (n == 1) /* n is one! */
something we definately do NOT want.
But in include/ the results is quite pretty, in all the structs, enums, etc
So we now have different settings in src/ vs include/ and this means that
in src/ we loose it for structs, but manual alignment is already good there,
just something to keep in mind if adding new stuff.
In include/ it is a real benefit when we run make format, so keep it there..
* We will have #ifdef.. then space+#define etc.. indentation.. but
we obviously don't want that for include guards like
#ifndef __struct_include__ because then like 3000+ lines in
include/struct.h will start with a space. Now, clang-format has
detection for this, but it doesn't kick in at 2 files (including
include/struct.h) because of a minor thingy we fix here.
* Add clang-format off to a few tables, who were auto generated
or copied anyway.
When not using version 302, such as with "CAP LS", the specification does not
allow us to use continuation lines. This means all advertised caps must fit
into one line. That is no longer always the case, especially if you load 3rd
party capabilities. So we need to scratch advertising some capabilities to
<302 clients.
"CAP LS 302" is unaffected. Note that version 302 in the specification exists
since at least November 2017, so most clients use that one.
According to https://ircv3.net/software/clients the following clients are
affected by this change:
Desktop Clients
* KVIrc
* Circe
* catgirl
* BitchX
* Pidgin
* LimeChat
Mobile Clients
* IRC for Android
* LimeChat
And various older versions of other clients (obviously).
NOTE: The source is only that IRCv3 page. I did not check manually.
For this particular commit. We filter out various unrealircd.org informative
CAPs and the vendor specific json-log. So that isn't much of a problem.
However, in the future we may be forced to filter out more capabilities to
make room. It would be much better if all clients are on >=302. Also, I
should mention we are not the only IRCd out there, so I can't vouch on what
other IRCds (will) do when hitting this non-302-limit.
Reported by ProgVal in https://bugs.unrealircd.org/view.php?id=6630
The previous mechanism (from yesterday) was a bit too simple at the
chathistory.c where returned_lines < limit would set the end tag but
it would not deal with the situation where returned_lines == limit
which is ambigious. So we had to move up a layer (or is it down?),
don't handle this in chathistory.c but in the backend. A new struct
field r->reached_end was added for this (set by backend).
Signals to the client that it has reached the end of the history and
there are no more messages to fetch. The tag is attached to the BATCH
opener when the server returns the last page of results.
Only sent to clients that negotiated the draft/chathistory capability.
print copy-pastable config blocks. Previously we used add_name_list(),
which uses insert at beginning, which would reverse the order.
Also changed duplicate_name_list() to preserve order. Previously
it reversed the order of all items.
and "./unrealircd genlinkblock" outputs multiple password ".." { spkifp; }
lines in such a case.
Other than that some cleaning up of recently-added-functions that are
now no longer needed: we now create ctx_link_server and ctx_link_client
that represent set::server-linking::tls-options for incoming and outgoing
links. Which can be NULL, and then we use ctx_server / ctx_client (set::tls).
Also add proper documentation on this.
When using ./unrealircd spkifp, tell ./unrealircd genblock is cooler.
Nah.. it takes more factors into account, genlinkblock, so is preferred :D
`serversonly` (such as port 6900 in the example.conf) and link { } blocks
in a different way than regular listen { } blocks:
* If there are different certificates used in the serversonly listen block
vs link blocks, then this is almost always means server linking is broken,
so we now print a warning on boot and rehash.
* We also print an 'advice' if any of these are not using (long-lived)
self-signed certificate. This is because CA issued certificates are
typically not suitable because they typically rotate keys and thus change
the `spkifp`. Changing spkifp breaks server linking. We will now print
an advice along with command and config block instructions to fix it.
* We now use `set::server-linking::tls-options` for link { } blocks
and listen { } blocks that are `serversonly`. All the rest uses the
`set::tls` settings by default (eg the regular listen { } block on 6697).
* This means our guide on
[Using Let's Encrypt with UnrealIRCd](https://www.unrealircd.org/docs/Using_Let's_Encrypt_with_UnrealIRCd)
and generic usage is more intuitive. You just set both set settings
and then no longer need to use any tls-options in listen blocks or link
blocks. The example conf has also been updated with this.
* If `set::server-linking::tls-options` is not configured, it defaults
to `set::tls`, so there is no unexpected behavior change for anyone.
* In a future release we will make server linking with `spkifp` mandatory,
so all of this helps with getting people ready for that, making such
a future transition smooth.
TODO: Update wiki, better wording in release notes, etc.
This also changes the default example conf:
/* RECOMMENDED:
* Everyone should be using IRC over SSL/TLS on port 6697. However, to use
* it properly, you have to get a "real" certificate instead of the
* self-signed default certificate that was generated by the installer.
* The Let's Encrypt initiative allows you to get a free certificate that is
* issued by a trusted Certificate Authority. Instructions are at:
* https://www.unrealircd.org/docs/Using_Let's_Encrypt_with_UnrealIRCd
*
* When you follow that guide you will have a "dual certificate" setup:
* set::tls:
* Your trusted CA certificate, served to clients on port 6697.
* (key and certificate change and renew every xx days automatically)
* set::server-linking::tls-options
* A long-lived self-signed certificate for server linking, with
* a stable 'spkifp' signature that you use in link blocks.
* This certificate is used automatically in "serversonly" listen blocks
* (port 6900 in this configuration file) and automatically used for all
* link { } blocks.
*
*/
//set {
// tls {
// certificate "/etc/letsencrypt/live/irc.example.org/fullchain.pem";
// key "/etc/letsencrypt/live/irc.example.org/privkey.pem";
// }
// server-linking {
// tls-options {
// certificate "tls/server.cert.pem";
// key "tls/server.key.pem";
// }
// }
//}
and JSON-RPC.
This exposes the newly added flood counters from
4384f1127b and
029675f867 in JSON.
I didn't want to put it in every JSON log message. So right now it
is only in:
* JSON-RPC with object_detail_level >= 5.
* Central Spamreport
I may expand it later to one or a few other areas.
* `total_channel_flood_count('..setting..')` returns the number of
times `+f`/`+F` limits were exceeded by that user in all channels
the user is or was in. Available are: `nick`, `join`, `knock`, `msg`,
`ctcp`, `text`, `repeat` and `paste` (and `all` for the sum).
Suggested by westid in https://bugs.unrealircd.org/view.php?id=6477
* New [crule function](https://www.unrealircd.org/docs/Crule) that return
the number of times a flood was blocked for that user. For example,
`server_flood_count('away')` returns the number of time away-flood
was exceeded. Aslo available: `nick`, `join`, `invite`, `knock`,
`vhost` and `conversations`. Plus, there is `all` for a total of all.
* This can be used in a security-group::rule or spamfilter::rule.
Eg: `spamfilter { rule "server_flood_count('nick')>4"; action gline; }`
This also - internally - adds a mechanism to run spamfilter rule-only-
filters after the command handler, whenever a tag value or other thing
changed. That's part of this commit.
Basically if a $variable is empty, and there is a space before it in the
template string then we delete that space.
May seem (or is) a bit over the top but this way the template stays clean,
and it may be used/useful in other places as well.
This is a behavior change, but I think we can live with it. One can opt-
out via BUILDVARSTRING_KEEP_SPACE_FOR_EMPTY_VAR.
TKL_EXPIRE and SPAMFILTER_MATCH messages.
This uses the newly added functions log_data_optional_string() and
log_data_optional_name_value(). The first shows the optional string
like "abc" and the second expands to "[name: value]". What's also new
is that both of these will swallow a preceding space if there is no value.
This so you can just use "Something. $optional_string" and it will
expand to "Something." if $optional_string is empty. This makes things
less hacky and more human readable :)
Unlike non-config-based TKLs - which go through tkldb - they are still not
preserved through restarts. But at least they are not lost due to REHASH.
This is done via a save+restore, a bit complicated, but we have little
choice (other than not doing this at all).
This also moves remove_config_tkls() from conf.c to tkl.c
of the last hit, eg in `STATS gline` for GLINEs. These counts happen on
each individual server and are not network-wide. This allows IRCOps to see
which entries never get any hits and can potentially be removed.
* Important exception: config-based spamfilters/bans lose their counters
on `REHASH` and restart atm.
* For non-config TKLs, the hit count and last hit timestamp are preserved
across reboots (via tkldb).
* Again, see *Developers and protocol* for the exact STATS field.
The spamfilter hits already existed but all the rest is new.
Suggested by BlackBishop in https://bugs.unrealircd.org/view.php?id=6304
(in particular, time of the last hit)
By default - assuming you don't set set::reject-message things by yourself -
the *LINE id is appended at the end of the rejection that is shown to the
user, like: [ID: G7K2MP9WQX3].
Also new is spamfilter to *LINE mapping, so you can see which *LINE was
set by which SPAMFILTER. For this STATS gline and friends were enhanced.
In fact, multiple fields were added there, including some that are 0
(zero) placeholders at the moment. These will be set in a future commit.
Some things were combined here so we only have to break STATS and tkldb
database format once (unless i made a mistake, then the follow up commit
will correct that i guess :D).
This was requested by Hero in https://bugs.unrealircd.org/view.php?id=4397
in 2015. Again by musk in https://bugs.unrealircd.org/view.php?id=4397
in 2022. And on IRC by Chris and others.
As you can see it was not SUPER easy and a lot of thought went into this
(and in terms of S2S traffic it is part of something bigger too)
And the defines are more clear now (if .max_size is not set by caller.
DOWNLOAD_MAX_SIZE_MEMORY_BACKED: 1M
DOWNLOAD_MAX_SIZE_FILE_BACKED: 50M
The file-backed is mostly a defense-in-depth measure, so we don't
store infinite amounts of data in a download. Even though, in practice,
these - at least at the moment in unrealircd itself - all come from
trusted paths like remote includes.
In url_unreal.c we do the counting ourselves. In url_curl.c we use the
option CURLOPT_MAXFILESIZE_LARGE but this does not ensure it in all
cases so we still do our own counting as well in that file as well.
And ship with these by default (no need to copy this set block):
set {
log-throttle {
CONNTHROTTLE_IPV6_LIMIT 100:60;
MAXPERIP_LIMIT 100:60;
};
};
You can do the same for other events, or even override existing ones,
and use the special value "unlimited" to turn default set ratelimits off:
set {
log-throttle {
CONNTHROTTLE_IPV6_LIMIT 50:60;
MAXPERIP_LIMIT unlimited;
};
};
Suggested in 2020 at https://bugs.unrealircd.org/view.php?id=5523
(and keeping it simple)
When a client is rejected by maxperip (not new) or connthrottle
ipv6-unknown-users-limit (that one is new), a notice to +s +x will be sent.
maxperip ipv4 example:
*** Client testuser4 with IP 1.2.3.4 rejected: maxperip limit exceeded (4 global, max 3)
maxperip ipv6 with /64 example:
*** Client testuser4 with IP 2001:dbe:0:0:0:0:0:4 rejected: maxperip limit exceeded for 2001:dbe::/64 (4 local, max 3)
connthrottle example where /56 limit is exceeded:
*** Client testuser5 with IP 2001:db8:cafe:abcd:0:0:0:5 rejected:
connthrottle ipv6-unknown-users-limit (cidr-56, max 4) exceeded for
2001:db8:cafe::/56 (5 unknown / 0 excepted / 0 known)
Oh and this commit also fixes a typo in existing CONNTHROTTLE events,
which previously were CONNTHROTLE (a missing T).
Changed "Too many connections from your IP" to have "[maxperip]" at the end.
Also create new setting and swap it with existing-one-during-development.
Long story short, we now have 3 different messages for these limits:
set::reject-message::too-many-connections
"Too many connections from your IP [maxperip]"
set::reject-message::too-many-connections-ipv6-range
"Too many connections from your IPv6 range ($prefix_addr/$prefix_len) [maxperip]"
set::reject-message::too-many-new-connections-ipv6-range
"Too many new connections from this IPv6 range ($prefix_addr/$prefix_len) [connthrottle]"
So we explicitly mention whether it is maxperip or connthrottle limiting the
user, that should provide enough clue to the IRCOp if the user pastes the
message to them.
connect-flood and maxperip module. This so they actually take
set::default-ipv6-clone-mask into account.
This also changes the maxperip module to a more simple method of
just freeing all entries and rebuilding the hash table on load.
That's necessary since now set::default-ipv6-clone-mask can change.
The OOB write did not happen on file-backed downloads, such as remote
includes. It only happened for memory-backed requests, which are only
these 4 in standard UnrealIRCd: centralblocklist, central spam report,
other spamreport blocks (eg to dronebl) and the log block with
destination webhook. All those 4 cases are very likely to be trusted
web servers, given the nature of the data you are sending to them.
The fix was to extend the size fields everywhere to 64 bits. It was
applied to both URL backends: url_unreal.c and url_curl.c.
The new API feature is a 'max_size' in OutgoingWebRequest, which
defaults to 1MB. This is only used for memory-backed responses,
so not for real file downloads. This fixes not only the reported
bug but also the case where a rogue webserver was unbounded in
terms of what response it could send back, potentially filling
up gigabytes of server memory.
Reported by Link420.
There is no hard cap on batch reference length, so we had to make one up.
It is now a clear #define MAXBATCHREFLEN 48, which should be plenty.
No sane client is going to use like a 64 byte batch reference :D
So we did use 48, but we also accidentally used BATCHLEN at another
place. BATCHLEN is 22 and refers to how many bytes we generate, so
that is not appropritate.
Thanks to Valware for spotting this.
This wasn't done before, because optimizing stuff can always introduce
nice new issues. But is kinda necessary now since the previous way was
very inefficient. This now builds all the necessary buffers for multiline
clients and for non-multiline clients. And then iterates through both
types of clients, sending what they need. Instead of doing it the other
way around.
I had the dillema to either expose the linecache API and have everything
in multiline.c. Or, i do not expose linecache, and we do everything in
send.c. The downside of the latter is that if there is mistake then we
can't simply reload (or unload) the module to solve it. So, I have chosen
to expose the linecache API (sure, less clean) since that leaves us with
options if we screw up, plus it means everything related to multiline
sending is nicely in multiline.c, which is i guess just as good as an
argument as well ;)
Add a little fake lag based on history result: 400ms for 50 lines
under normal conditions where 50 lines = 50 lines. But this can go
up to 5000ms for worst-case amplification attacks where requesting
50 lines actually returns 50*15=750 lines when each line is a multiline
with max-lines, which gets you close to 350k+. This would only happen
if someone on the channel is doing evil stuff (with presumably consent
of the ops).
Also guard against hiting max sendq. If we are too close, then we
reject the CHATHISTORY request rather than quiting with "Max SendQ
exceeded". This protects against an attack where someone would be
tricked into joining a channel with amplified history (as explained
in previous paragraph), their client would do an automatic CHATHISTORY
request and then the victim would exceed max sendq and thus be killed.
And yes, this and maaaaany other multiline + history interactions
and many "buts" and security/flood concerns are why this implemtnation
took (and still takes) a lot of hours to get right :D.
For +H we now temporarily allow overshooting. This only matters for low limits.
Multiline batches are atomic so we have to choose to keep them as a whole
or remove the complete batch. So if +H 5:1h and the last message was a 15-line
multiline event, what do we do? We allow temporary overshooting to store the
15 lines. As said, the alternative would be to store 0 lines which would be
worse in terms of functionality, and the small overshoot is defensible.
For higher limits (where the +H line limit is bigger than multiline max-lines),
we always stay under the +H limit. Eg if all history in a channel consists
of 15 line multiline events and we have +H 100 then we will store 90, not 105.
It's only for +H linelimit < max-lines that this matters, because there the
zero-lines consequence sucks too much ;)
Bram Matthys