Fix OOB write on URL callback with 2GB+ response. Add new size limit.

The OOB write did not happen on file-backed downloads, such as remote
includes. It only happened for memory-backed requests, which are only
these 4 in standard UnrealIRCd: centralblocklist, central spam report,
other spamreport blocks (eg to dronebl) and the log block with
destination webhook. All those 4 cases are very likely to be trusted
web servers, given the nature of the data you are sending to them.

The fix was to extend the size fields everywhere to 64 bits. It was
applied to both URL backends: url_unreal.c and url_curl.c.

The new API feature is a 'max_size' in OutgoingWebRequest, which
defaults to 1MB. This is only used for memory-backed responses,
so not for real file downloads. This fixes not only the reported
bug but also the case where a rogue webserver was unbounded in
terms of what response it could send back, potentially filling
up gigabytes of server memory.

Reported by Link420.

include/config.h

@@ -210,6 +210,13 @@
  */
 #define DOWNLOAD_MAX_REDIRECTS 2
 
+/* Default maximum size (in bytes) for memory-backed HTTP responses
+ * (i.e. when OutgoingWebRequest.store_in_file == 0). Responses exceeding
+ * this are rejected and the transfer is aborted. Callers can override
+ * by setting OutgoingWebRequest.max_size before url_start_async().
+ */
+#define DOWNLOAD_MAX_SIZE 1048576
+
 /*
  * Max time from the nickname change that still causes KILL
  * automaticly to switch for the current nick of that user. (seconds)

include/h.h

@@ -1561,7 +1561,7 @@ extern int valid_operclass_name(const char *str);
 #define safe_free_outgoingwebrequest(x)	do { if (x) { free_outgoingwebrequest(x); x = NULL; } } while(0)
 extern void free_outgoingwebrequest(OutgoingWebRequest *r);
 extern OutgoingWebRequest *duplicate_outgoingwebrequest(OutgoingWebRequest *orig);
-extern void url_callback(OutgoingWebRequest *r, const char *file, const char *memory, int memory_len, const char *errorbuf, int cached, void *ptr);
+extern void url_callback(OutgoingWebRequest *r, const char *file, const char *memory, long long memory_len, const char *errorbuf, int cached, void *ptr);
 extern const char *synchronous_http_request(const char *url, int max_redirects, int connect_timeout, int transfer_timeout);
 extern int update_known_user_cache(Client *client);
 extern MODVAR SecurityGroup *known_users;

include/struct.h

@@ -1983,6 +1983,8 @@ struct OutgoingWebRequest
 	int connect_timeout; /**< How many seconds to wait for the (TLS) connect to succeed */
 	int transfer_timeout; /**< How many seconds the total transfer may take (connect+reading everything) */
 	int minimum_tls_version;
+	long long max_size; /**< Max response size for memory-backed downloads, in bytes.
+	                     *   0 = use DOWNLOAD_MAX_SIZE. Ignored for file-backed. */
 	// If you are adding fields here:
 	// 1) update duplicate_outgoingwebrequest() in src/misc.c
 	// 2) and update free_outgoingwebrequest() there as well (if something needs to be freed)
@@ -1993,7 +1995,7 @@ struct OutgoingWebResponse
 {
 	const char *file; /**< The temporary file of the download, or NULL. This is only set if OutgoingWebRequest had 'store_in_file' set to 1 and the download was succesful. */
 	const char *memory; /**< The memory buffer of the response, or NULL if an error occured (see errorbuf) */
-	int memory_len; /**< The length of 'memory', since the response may contain binary data. */
+	long long memory_len; /**< The length of 'memory', since the response may contain binary data. */
 	const char *errorbuf; /**< If this is non-NULL then an error occured and this is the error string. Check this member before checking any others! */
 	int cached; /**< Set to 1 if OutgoingWebRequest had 'cachetime' set and we have a cache hit on the webserver. The file and errobuf will be NULL since there was no data transfer. */
 	void *ptr; /**< The OutgoingWebRequest 'callback_data' */