Commit graph

1180 commits

Author SHA1 Message Date
Bill Currie
0dab26ce8f [cexpr] Allow untyped result values
If the result object type pointer is null, then the parsed result type
and value pointers are written directly to the result object rather than
testing the parsed result type against the object type and copying the
parsed result value data to the location of the object value. It is then
up to the caller to check the type and copy the value data.
2022-11-26 22:10:29 +09:00
Bill Currie
749a110b76 [util] Check share path too when filling file lists
This fixes maplist showing only those maps in the user directory.
However, no checking is done for duplicate files due to earlier search
paths overriding later paths.
2022-11-11 22:05:10 +09:00
Bill Currie
d9b231e2aa [util] Return null for missing wad lumps
Segfaults are not good when a lump can't be found as it makes error
checking difficult.
2022-11-10 14:44:08 +09:00
Bill Currie
7c06012383 [util] Support custom swap function for heapsort
Sorting ECS component pools needs to swap multiple chunks of data for
each element being swapped, so the standard swap function isn't enough.
2022-10-31 10:52:03 +09:00
Bill Currie
31dd419fde [util] Reject non-shortest encoded utf-8 sequences
While chatting about utf-8, I noticed that QF doesn't ensure the input
sequences are the shortest possible encodings. It turns out that the
check is easy in that only the second byte needs to be checked if the
first byte's data bits are 0, and the second byte must have a data value
larger than that representable by the next lower leading byte.
2022-10-15 22:54:24 +09:00
Bill Currie
5e154bf1b7 [cvar] Remove reliance on line number for developer parsing
I must have forgotten about the SYS_DeveloperID_... enum values, when I
wrote that code, because relying on the line number is not really for
the best.
2022-09-24 18:26:25 +09:00
Bill Currie
ed209c3aa4 [util] Add a comment to darray test
Linux libc has its own "remove" (POSIX, even).
2022-09-22 09:35:57 +09:00
Bill Currie
2a47a61bc3 [util] Add atomic ring buffer and automated tests
Just head and tail are atomic, but it seems to work nicely (at least on
intel). I actually had more trouble with gcc (due to accidentally
testing lock-free with the wrong ring buffer... oops, but yup, gcc will
happily optimize your loop to spin really really fast). Also served as a
nice test for C11 threading.
2022-09-22 09:35:57 +09:00
Bill Currie
1cd6a174fe [build] Correct cygwin system type
cygwin is not windows, so trying to build a win32 app on cygwin-native
doesn't work too well.
2022-09-19 16:36:29 +01:00
Bill Currie
c72b04a522 [cvar] Remove a debug print 2022-06-06 14:28:28 +09:00
Bill Currie
520371a3aa [zone] Fix bad suggested mem calculation
Because the calculation didn't take the hunk header size (which is not
included in the hunk size) into account, the conversion to MB was one
short and thus the rounding up to the next 8 MB boundary was giving the
current total hunk size (ie, the already given size). Most confusing to
a user ("But I already asked for 128MB!").
2022-06-06 14:05:52 +09:00
Bill Currie
ef64161835 [zone] Make Hunk_RawAlloc the core hunk allocator
It turns out that copying just "unknown" is a significant performance
hit when doing over 100M allocations. Making Hunk_RawAlloc the core and
initializing the name field with a single 0 shaved about a second off
`qfvis gmsp3v2.bsp` (from about 39s to about 38s).
2022-06-06 13:33:54 +09:00
Bill Currie
5c94c40985 [zone] Revert cache behavior to its original design
My reason for using Hunk_HighAlloc for allocating cache blocks was to
lock them down so they were safe for the sound mixer to access when
running in a real-time thread. However, I had never tested under tight
memory constraints, which proved that the design (or maybe just
implementation) just wasn't robust. However, now that sounds are loaded
into a completely separate region, it's safe to put the cache back to
its original behaviour (still with 64-byte alignment and such, of
course). This will even allow the high hunk to be used again, though it
effectively was anyway with Hunk_TempAlloc.
2022-06-06 13:33:15 +09:00
Bill Currie
11fee8571c [zone] Add functions to get and set the tag
Getting the tag is possibly useful in general and definitely in
debugging. Setting, I'm not so sure as it should be done when allocated,
but that's not always possible.

Also, correct the return type of z_block_size, though it affected only
Z_Print. While an allocation larger than 4GB is... big for zone, the
blocks do support it, so printing should too.
2022-06-06 12:39:54 +09:00
Bill Currie
4694ee693b [zone] Alight zone allocations to 64 bytes
Since Ruamoko got vector types, zone's 8-byte alignment was no longer
sufficient due to hardware-enforced alignment requirements of the
underlying vector operations.

Fixes #28.
2022-06-06 12:39:54 +09:00
Bill Currie
2bb0f0c104 [zone] Add support for reference counting to Z_*
And use it for Ruamoko object reference counts.

I need reference counts for dealing with block sound buffers since they
can be shared by many channels. I figured I take care of Ruamoko's
reference count location at the same time.

Fixes #27.
2022-06-06 12:39:54 +09:00
Bill Currie
94db9c8ab1 [quakefs] Add a data parameter to gamedir callbacks
While this didn't fix the problem for which I needed the data pointer,
having data pointers in callbacks is far too useful to throw out the
change.
2022-06-04 16:06:04 +09:00
Bill Currie
0c65d294ed [sys] Add a function to lock a region of memory
Locked memory is needed for real-time threads (which the sound mixer is
when using JACK).
2022-06-04 12:28:16 +09:00
Bill Currie
0ebb0717b0 [zone] Add failing test cases
The tests fail as they exercise how the cache *SHOULD* work rather than
how it does now.

The tests do currently pass for the pending work I've done on the cache
system, but while working on it, I remembered why I reworked cache
allocation...

The essential problem is that sounds are loaded into the cache, which is
fine for synchronous output targets, but has proven to be a minefield
for asynchronous output targets (JACK, ALSA).

The reason for the minefield is the hunk takes priority over the cache,
and is free to move cache blocks around, and *even dispose of them
entirely* in order to satisfy memory allocations from either end of the
hunk. Doing this in an entirely single-threaded process (as DOS Quake
was) is perfectly safe, as the users of the cache just reload the
pointer each time, and bail if it's null (meaning the block has been
freed), or even cause the data to be reloaded if possible (I'm a little
fuzzy on the details for that as I didn't write that code). However, in
multi-threaded code, especially real-time (JACK, possibly ALSA), it's a
recipe for disaster. The 4cab5b90e6 commit was a (mostly) successful
attempt to mitigate the problem by allocating the cache blocks from the
high-hunk (thus minimizing any movement caused by low-hunk allocations),
it resulted in cache allocates and regular high-hunk allocations somehow
getting intertwined: while investigating just how much memory ad_tears
needs (somewhere between 192MB and 256MB), I got "trashed sentinel"
errors and upon investigation, I found what looks very suspiciously like
audio data written across a hunk control block.

I've decided that the cache allocation *algorithm* should be reverted to
how it was originally designed by Id (details will remain "modern"), but
while working on the tests, I remembered why I had done the changes in
the first place (above story). Thus the work on reverting the cache
allocation can't go in until I get sound memory management independent
of the cache. The tests are going in now so I have a constant reminder :)
2022-06-03 12:52:59 +09:00
Bill Currie
3293fcaab0 [sys] Add some developer flags for cache/hunk/zone
And make Sys_MaskPrintf take the developer enum rather than just a raw
int.

It was actually getting some nasty hunk corruption errors when under
memory pressure that made it clear the sound system needs some work.
2022-06-03 12:04:27 +09:00
Bill Currie
dcfe0e33a3 [util] Allow zero-sized lumps to be at end of file
This fixes a crash when attempting to load a map with no texture data.
2022-05-26 12:35:50 +09:00
Bill Currie
7240d2dc80 [model] Move plane info into mnode_t, and visframe out
The main goal was to get visframe out of mnode_t to make it thread-safe
(each thread can have its own visframe array), but moving the plane info
into mnode_t made for better data access patters when traversing the bsp
tree as the plane is right there with the child indices. Nicely, the
size of mnode_t is the same as before (64 bytes due to alignment), with
4 bytes wasted.

Performance-wise, there seems to be very little difference. Maybe
slightly slower.

The unfortunate thing about the change is the plane distance is negated,
possibly leading to some confusion, particularly since the box and
sphere culling functions were affected. However, this is so point-plane
distance calculations can be done with a single 4d dot product.
2022-05-22 12:41:23 +09:00
Bill Currie
0dca1e1124 [plugin] Pull plugin from list before calling its shutdown
This ensures that the plugin's shutdown function won't get called twice
in the event of an error in the plugin's unload sequence triggering a
second Sys_Shutdown, especially if the plugin is being unloaded as a
part of another sub-system's shutdown sequence (which is probably in
itself a design mistake, need to look into that).
2022-05-13 09:50:24 +09:00
Bill Currie
bc674657cb [plugin] Ensure plugins get unloaded during shutdown
This included pre-registered (static) plugins.

Surprisingly, PI_Shutdown was never called.
2022-05-12 20:58:39 +09:00
Bill Currie
edf5c66321 [cvar] Plug a pile of memory leaks
Mostly in user cvars, but also any string vars.
2022-05-12 19:58:18 +09:00
Bill Currie
b7b6d4ad12 [hash] Clean up some duplicate code
Hash_Add and Hash_AddElement differed by only one line: the hash
calculation.
2022-05-12 18:52:36 +09:00
Bill Currie
f91167d74a [hash] Use uintptr_t for all hashes
This fixes the inconsistent hash types and removes all references to
"long", making for better portability.
2022-05-12 18:52:36 +09:00
Bill Currie
978d0306c0 [hash] Rename the publicly visible hashlink_t to hashctx_t
I think my biggest problem with the hashlink freelist parameter was how
much implementation it exposed in just the name.
2022-05-12 18:02:01 +09:00
Bill Currie
2ebefd7850 [wad] Return 0 if no wad file is loaded
This allows QF to load without a wad file as it will use the internal
character definition.
2022-05-08 23:56:11 +09:00
Bill Currie
ae9e8f8f4e [plist] Report line-relative character position in errors
Far better than string-relative.
2022-05-06 20:10:41 +09:00
Bill Currie
f6baa16084 [cvar] Remove the cvar lists
They're really redundant, and removing the next pointer makes for a
slightly smaller cvar struct. Cvar_Select was added to allow finding
lists of matching cvars.

The tab-completion and config saving code was reworked to use the hash
table DO functions. Comments removed since the code was completely
rewritten, but still many thanks to EvilTypeGuy and Fett.
2022-04-25 00:26:45 +09:00
Bill Currie
8bd626a3e4 [hash] Add a couple of data-oriented functions
Hash_Select returns a list of elements that match a given criterion
(select callback returning non-0).

Hash_ForEach simply calls a function for every element.
2022-04-25 00:18:22 +09:00
Bill Currie
90447a5d3b [quakefs] Ensure fs_sharepath and fs_userpath are never empty
Other parts of quakefs treat an empty path as an error, so fs_sharepath
and fs_userpath must never be empty or they will effectively be
rejected. While the user explicitly setting them to empty strings is one
way for them to become empty, another is QFS_CompressPath compressing
'.' to an empty path, which makes it rather difficult to set up the
traditional quake directory tree (ie, operate from the current
directory).
2022-04-24 19:15:22 +09:00
Bill Currie
12c84046f3 [cvar] Make cvars properly typed
This is an extremely extensive patch as it hits every cvar, and every
usage of the cvars. Cvars no longer store the value they control,
instead, they use a cexpr value object to reference the value and
specify the value's type (currently, a null type is used for strings).
Non-string cvars are passed through cexpr, allowing expressions in the
cvars' settings. Also, cvars have returned to an enhanced version of the
original (id quake) registration scheme.

As a minor benefit, relevant code having direct access to the
cvar-controlled variables is probably a slight optimization as it
removed a pointer dereference, and the variables can be located for data
locality.

The static cvar descriptors are made private as an additional safety
layer, though there's nothing stopping external modification via
Cvar_FindVar (which is needed for adding listeners).

While not used yet (partly due to working out the design), cvars can
have a validation function.

Registering a cvar allows a primary listener (and its data) to be
specified: it will always be called first when the cvar is modified. The
combination of proper listeners and direct access to the controlled
variable greatly simplifies the more complex cvar interactions as much
less null checking is required, and there's no need for one cvar's
callback to call another's.

nq-x11 is known to work at least well enough for the demos. More testing
will come.
2022-04-24 19:15:22 +09:00
Bill Currie
a52704e8ca [cexpr] Allow assignment of double to float
I didn't really want this, but it's needed for cvar support.
2022-04-24 19:15:22 +09:00
Bill Currie
aad4546c01 [cexpr] Expose cexpr assignment
This makes it much easier for other systems (in particular, cvar) to
copy values safely.
2022-04-24 19:15:22 +09:00
Bill Currie
021ec4962b [cexpr] Add optional error message prefix string
The prefix gives more context to the error messages, making the system a
lot easier to use (it was especially helpful when getting my cvar revamp
into shape).
2022-04-24 19:15:22 +09:00
Bill Currie
d68db74049 [cexpr] Add a flags type
Based on the flags type used in vkparse (difference is the lack of
support for plists). Having this will make supporting named flags in
cvars much easier (though setting up the enum type is a bit of a chore).
2022-04-24 19:15:22 +09:00
Bill Currie
392e032ce2 [cexpr] Add a get_string function to exprtype_t
This allows for easy (and safe) printing of cexpr values where the type
supports it. Types that don't support printing would be due to being too
complex or possibly write-only (eg, password strings, when strings are
supported directly).
2022-04-24 19:13:54 +09:00
Bill Currie
db5b77c838 [cexpr] Require designated initializers for exprtype_t
This will make expanding it much safer in the future.
2022-04-24 17:31:17 +09:00
Bill Currie
a29836cc2c [quakefs] Return QFile pointer from QFS_NextFile(name)
QFS_NextFilename was renamed to QFS_NextFile to reflect the fact it now
returns a QFile pointer for the newly created file (as well as the
name). This necessitated updating WritePNG to take a file pointer
instead of a file name, with the advantage that WritePNGqfs is no longer
necessary and callers have much more control over the creation of the
file.

This makes QFS_NextFile much more secure against file system race
conditions and attacks (at least in theory). If nothing else, it will
make it more robust in a multi-threaded environment.
2022-03-31 17:27:04 +09:00
Bill Currie
8cdabc8905 [quakefs] Reimplement QFS_NextFilename to be more secure
It's not there yet as it promptly closes the file and returns only the
filename (and then only the portion within the user's directory tree).
However, this worked nicely as a test for Sys_UniqueFile.
2022-03-31 16:44:19 +09:00
Bill Currie
a35bfef24c [sys] Add a function to safely create a unique file
QF currently uses unique file names for screenshots and server-side
demos (and remote snapshots), but they're generally useful.
QFS_NextFilename has been filling this role, but it is highly insecure
in its current implementation. This is the first step to taking care of
that.
2022-03-31 16:44:19 +09:00
Bill Currie
38319d01b2 Fix some null pointer shenanigans
clang doesn't like anything but a bare 0 as null (and in some of the
cases, it was quite right: '\0' should not be treated as a null
pointer). And the crashers were just for paranoia and probably aren't
needed any more (kept for now, though).
2022-03-31 00:25:22 +09:00
Bill Currie
939a73fb52 [sys] Override strdup for 32-bit windows
This fixes some nasty segfaults when calling free due to different
allocators being used.
2022-03-21 19:23:49 +09:00
Bill Currie
0f30f0a133 [mathlib] Remove suspicious IS_NAN
The implementation looks wrong (more like infinity). Where it was used
is currently disabled, but the usages were replaced with C99's isnan.
2022-03-19 12:50:08 +09:00
Bill Currie
65af7fb4a4 [mathlib] Remove frustum global
It should never have been there and is now in the refdef (not its final
home: it should probably be part of the camera).
2022-03-19 12:33:12 +09:00
Bill Currie
a4479f4840 [cvar] Ensure floats can round-trip when setting
The way Cvar_SetValue is used, floats need to be able to round-trip
reliably and thus need up to 9 digits of precision.
2022-03-18 11:42:14 +09:00
Bill Currie
16440bce2d [mathlib] Clean up AngleVectors comments a little
They're still slightly confusing, but the situation itself is confusing,
but the comments should be a little more helpful now as they are more
explicit about the orientation of the matrices and just which axis
points where.
2022-03-14 11:51:50 +09:00
Bill Currie
dce1a4d292 [util] Force 32-bit windows malloc to be 16-byte aligned
By replacing it :P (and its friends). This gets the non-sw renderers
working with recent scene changes.
2022-03-09 20:00:51 +09:00