And the defines are more clear now (if .max_size is not set by caller.
DOWNLOAD_MAX_SIZE_MEMORY_BACKED: 1M
DOWNLOAD_MAX_SIZE_FILE_BACKED: 50M
The file-backed is mostly a defense-in-depth measure, so we don't
store infinite amounts of data in a download. Even though, in practice,
these - at least at the moment in unrealircd itself - all come from
trusted paths like remote includes.
In url_unreal.c we do the counting ourselves. In url_curl.c we use the
option CURLOPT_MAXFILESIZE_LARGE but this does not ensure it in all
cases so we still do our own counting as well in that file as well.
The OOB write did not happen on file-backed downloads, such as remote
includes. It only happened for memory-backed requests, which are only
these 4 in standard UnrealIRCd: centralblocklist, central spam report,
other spamreport blocks (eg to dronebl) and the log block with
destination webhook. All those 4 cases are very likely to be trusted
web servers, given the nature of the data you are sending to them.
The fix was to extend the size fields everywhere to 64 bits. It was
applied to both URL backends: url_unreal.c and url_curl.c.
The new API feature is a 'max_size' in OutgoingWebRequest, which
defaults to 1MB. This is only used for memory-backed responses,
so not for real file downloads. This fixes not only the reported
bug but also the case where a rogue webserver was unbounded in
terms of what response it could send back, potentially filling
up gigabytes of server memory.
Reported by Link420.
even if it costs an extra round-trip due to HRR (Hello Retry Request).
This is IRC after all, where connections live minutes, hours, days,
so that extra round trip is worth it if it means better security.
The TL;DR is: we try harder to use X25519MLKEM768.
The longer story is as follows:
In TLSv1.3, the client will indicate which groups it supports (eg
a list of 4 items) and which ones it speculates to be used (very
often just 2 items). Some TLS clients may not include X25519MLKEM768
in this initial speculation, but only f.e. X25519 and prime256v1
even though X25519MLKEM768 is communicated via their "supported" list.
Without this patch, we would then settle with one of those 2.
With this patch, we will send a Hello Retry Request, allowing to
use X25519MLKEM768.
This is rare, though, most TLS client implementations that have
X25519MLKEM768 will bet on it to be used (the 2 they bet on is
often X25519MLKEM768 & X25519). That's many browsers like Chrome,
OpenSSL, Go, etc.
GnuTLS usually will do this as well, but under some configurations
it may bet on 2 classic crypto to be used. For that specific (type
of) situation, this patch will help to use X25519MLKEM768.
This can be tested with OpenSSL to simulate such an implementation:
openssl s_client -connect 127.0.0.1:6697 -groups X25519MLKEM768:*X25519
Before this patch, it would result in X25519 (because that is the
speculated group, with the asterisk). After this patch it will
cause X25519MLKEM768 to be used.
The tuple syntax is in 3.5.0+ and our UNREALIRCD_DEFAULT_TLS_GROUPS_PRIMARY
with X25519MLKEM768 also requires 3.5.0+ so this is an easy change.
Oh and, this commit comment is rather long for a 1 byte change :D
I was dumb: with an RSA cert you need ECDHE-RSA-* and i had
only included ECDHE-ECDSA-*. Long story short: TLSv1.2 didn't work
if you had an RSA certificate. Reported by BlackBishop, and in
hindsight also by Mi_92. Thanks for the quick reports, this should
be a quick fix :-)
For reference, the established TLS connections at irc*.unrealircd.org
over the past 6 months were:
14379 TLSv1.3-TLS_CHACHA20_POLY1305_SHA256
368 TLSv1.2-ECDHE-ECDSA-AES256-GCM-SHA384
160 TLSv1.2-ECDHE-ECDSA-CHACHA20-POLY1305
3 TLSv1.3-TLS_AES_256_GCM_SHA384
There is nobody connecting with AES CBC in those statistics
(ECDHE-ECDSA-AES256-SHA256 and ECDHE-ECDSA-AES128-SHA384)
In config.h we had a:
#if OPENSSL_VERSION_NUMBER >= 0x10100000L
#define UNREALIRCD_DEFAULT_ECDH_CURVES "X25519:secp521r1:secp384r1:prime256v1"
#else
#define UNREALIRCD_DEFAULT_ECDH_CURVES "secp521r1:secp384r1:prime256v1"
#endif
...which is fine in theory, but openssl headers are not included at that point,
so OPENSSL_VERSION_NUMBER was not defined.
From now on, we have:
#define UNREALIRCD_DEFAULT_ECDH_CURVES_PRIMARY "X25519:secp521r1:secp384r1:prime256v1"
#define UNREALIRCD_DEFAULT_ECDH_CURVES_SECONDARY "secp521r1:secp384r1:prime256v1"
...and we try them in that order. If both fail, we exit with an error (like before).
This because X25519 is not available in OpenSSL before 1.1.0 (so really old)
and may also not be available when running in FIPS mode.
This fixes a bug where if you run ./Config with 'auto' file descriptors,
and then have an unusually low 'ulimit -n' of like 150, you would end up
with a negative amount of file descriptors available for use.
This fix moves it from compile-time setting of reserved fd's to runtime
setting.
All this is wrong, by the way, but that is for another major overhaul,
at least this bug is fixed now :D
* The [Central Spamfilter](https://www.unrealircd.org/docs/Central_Spamfilter),
which provides spamfilter { } blocks that are centrally managed, is
now fetched from a different URL if you have an Central API key set.
This way, we can later provide spamfilter { } blocks that build on
central blocklist scoring functionality, and also don't have to reveal
the central spamfilter blocks to 100% of the world.
Since 10k+ fd's available is the common situation, this means we then have
250 fd's reserved for non-clients, such as HTTPS callbacks and other things.
Previously:
<1024: reserve 4 fd's
1024+: reserve 8 fd's
Now:
<1024: reserve 8 fd's
1024-2047: reserve 16 fd's
2048-10000: reserve 32 fd's
10000+: reserve 250 fd's
This is quite a bit higher than client DNS lookups (1500ms first, on retry 3000ms)
and is because some DNSBL are reported to be quite a bit slower than ordinary DNS.
(Maybe just some, but.. the higher timeout does not hurt anyone anyway)
Note that all this has no effect on client handshake times, as DNSBL checks are
done in the background. Only side-effect is that if we do get a "late hit" then
you may now see a kill a few seconds after the client is online (which was actually
already possible before too for quick clients, but.. yeah...)
These settings can be overriden via set::dns, these are the defaults:
set {
dns {
client {
timeout 1500;
retry 2;
}
dnsbl {
timeout 3000;
retry 2;
}
}
}
When you REHASH we will check if the values are different than the current
c-ares settings and if so, reinitialize the resolver. Reinitializing the
resolver will destroy outstanding DNS requests, eg DNS lookups for clients
currently connecting, but so be it. Not a super-huge issue since changing
this is rare.
Requested by BlackBishop in https://bugs.unrealircd.org/view.php?id=6306
This also fixes a bug with OpenSSL 3.x where, when the ircd was
configured to still allow old TLSv1.0 / TLSv1.1, it would still
only allow TLSv1.2+.
But, as said, allowing TLSv1.0/TLSv1.1 is now no longer the default.
See release notes for more information or the documentation at
https://www.unrealircd.org/docs/TLS_Ciphers_and_protocols
That is, when in "auto" mode, which is like for 99% of the users.
NOTE: the sytem may still limit the actual number of FD's to
a lower value, depending on the value of "ulimit -n -H".
setting the default class::sendq that pretty much everyone overrides
in class (isn't this even required? ;D).
Rename to DEFAULT_SENDQ since we have DEFAULT_RECVQ too.
This also includes buffer modifications to have a larger read buffer
and IRCv3 implementations (partial or not) for:
labeled-response, msgid, server-time, batch and account-tag.
As said, it is the initial and partial implementation.
There are still various FIXME's and TODO's, the API of various
functions may still change (actually that is true for the next
months, even) and some stuff is currently in the core that will
be moved to modules.
usually the fast badwords system is used instead)
* Code deduplication in src/modules/{chanmodes,usermodes}/censor.c
to src/match.c -- which may be moved later again to efuncs.
* Add --without-tre:
This means USE_TRE will be enabled by default right now
but if using --without-tre it will be undef'ed. This so we
can prepare for the TRE phase-out in 2020.
* Remove include/badwords.h, put contents in include/struct.h
The new question in ./Config now defaults to 'auto' (both for new installs
and for upgrades). You can still specify a manual limit but it is no longer
recommended.
A MAXCONNECTIONS of 'auto' means - at present - that UnrealIRCd will try
to set a limit of 8192. This is quite a bump from the original 1024.
On systems where this is not possible we will simply use the highest amount
possible, such as 4096 on many systems, or 1024.
In fact, we now no longer error when MAXCONNECTIONS is higher than the
'ulimit -n' limit but will adjust ourselves to the limit.
Only if the effective limit is below 100 we will print out a fatal error
since running in such a scenario is highly discouraged.
The reason for this change is that nowadays with drone attacks we may need
to be able to handle more concurrent sockets. Also, many Linux distro's
have a default setting of unlimited or 4096 nowadays, out of the box.
For people packaging UnrealIRCd (not end-users):
The ./configure --with-fd-setsize=xx option was removed and the
optional(!!) --with-maxconnections=xx option has been added.
We recommend you NOT to pass this option. Not passing it means that
the previously mentioned 'auto' mode will be used, which is likely
best for most users.
Module coders:
Although it is unlikely you accessed the 'MAXCLIENTS' variable,
if you did, it is now called 'maxclients' (lowercase) since it is
adjusted at runtime and no longer a macro.
maximum number of conversations a user can have with other users at the
same time. Until now this was hardcoded at limiting /MSG and /INVITE to
20 different users in a 15 second period. The new default is 10 users,
which serves as a protection measure against spambots.
See https://www.unrealircd.org/docs/Set_block#maxcc for more details.
preferring AES-256 over AES-128 (in contrast to the Mozilla "intermediate"
profile which prefers AES-128). Again, this only affects non-PFS cases, as
all modern clients with PFS already had CHACHA20 and AES-256 negotiated.
The portion of non-PFS clients should only be few percent, if any.
I was actually considering removing non-PFS ciphersuites but it seems a bit
early to do so, at least not without more research on affected clients.
Bram Matthys