Commit graph

3877 commits

Author SHA1 Message Date
Bill Currie
8433905015 [qfcc] Extend reachable to the label node's parents
While using just the label node's reachable set was sufficient for a
simple swap (t = a; a = b; b = t;), it is not sufficient for
read-before-write dependencies such as found in linked-list building:

    { o = array[ind]; o.next = obj; obj = o; }

The assignment to o.next uses obj, but that use is hidden because obj's
reachable nodes does not include o thus assigning o to obj causes the
array dereference to be assigned directly to obj and thus o.next winds
up pointing to o instead of whatever obj was. The parent nodes of obj's
node are its users, so any new assigned to obj must come after those
parents as well as any node reachable by obj's node.

Fixes a runaway loop error when adding a frikbot to the server.
2021-12-27 14:25:55 +09:00
Bill Currie
4ad84b3786 [qfcc] Add a test for use/write dependencies
I ran into this with frikbot causing an infinite loop due to incorrectly
linked objects.
2021-12-27 14:17:12 +09:00
Bill Currie
44dd183d55 [qfcc] Make it a little easier to see extra info
instead of having to find an #if 0, just uncomment the define.
2021-12-27 14:15:51 +09:00
Bill Currie
95991e0d77 [qfcc] Check switch test expression early
Avoids a segfault when the test expression has errors.
2021-12-27 00:47:35 +09:00
Bill Currie
3059aa7979 [qfcc] Preserve input qfo data in qfo_to_progs
qfo_to_progs was modifying the space data pointers in the input qfo,
making it impossible to reuse the qfo. However, qfo_relocate_refs needs
the updated pointers, thus do a shallow copy of the qfo and its spaces
(but not any of the data)
2021-12-27 00:27:15 +09:00
Bill Currie
dcf8beeccc [qfcc] Remove a union wart from qfo_mspace_t 2021-12-26 20:37:01 +09:00
Bill Currie
1d3970bc38 [qfcc] Allow extern builtin function declarations
build_builtin_function does the right thing, and it was only legacy
syntax functions that were affected anyway. Certainly, external
variables should not be initialized, but klik uses @extern { } wrapped
around several builtin functions and I had added the feature to allow
just this as it is rather convenient.
2021-12-26 15:01:38 +09:00
Bill Currie
22c67fc268 [qfcc] Use flow analysis for dealloc check
I decided that the check for whether control reaches the end of the
function without performing some necessary action (eg, invoking
[super dealoc] in a derived -dealoc) is conceptually the return
statement using a pseudo operand and the necessary action defining that
pseudo operand and thus is the same as checking for uninitialised
variables. Thus, add a pseudo operand type and use one to represent the
invocation of [super alloc], with a special function to call when the
"used" pseudo operand is "uninitialised".

While I currently don't know what else pseudo operands could be used
for, the system should be flexible enough to add any check.

Fixes #24
2021-12-25 17:04:26 +09:00
Bill Currie
f903211362 [qfcc] Make operand union anonymous
This one was easy enough and gets rid of that little o. wart.
2021-12-25 12:40:24 +09:00
Bill Currie
7c24f116b4 [qfcc] Rename tmpaddr to pseudo_addr
I want to use the function's pseudo address that was used for managing
aliased temporary variables for other pseudo operands as well. The new
name seems to better reflect the variable's purpose even without the
other pseudo operands as temporary variables are, effectively, pseudo
operands until they are properly allocated.
2021-12-25 12:21:59 +09:00
Bill Currie
79b760b1d0 [qfcc] Add failing test for multi-path dealloc
This test checks for control reaching the end of the function without
invoking [super dealloc] in all paths.
2021-12-24 22:45:43 +09:00
Bill Currie
8385046486 [qfcc] Warn when super dealloc invocation is missing
Forgetting to invoke [super dealloc] in a derived class's -dealloc
method has caused me to waste far too much time chasing down the
resulting memory leaks and crashes. This is actually the main focus of
issue #24, but I want to take care of multiple paths before I consider
the issue to be done.

However, as a bonus, four cases were found :)
2021-12-24 22:45:43 +09:00
Bill Currie
ff1cdb6f89 [qfcc] Give select expressions their own type
While get_selector does the job of getting a selector from a selector
reference expression, I have long considered lumping various expression
types under ex_expr to be a mistake. Not only is this a step towards
sorting that out, it will make working on #24 easier.
2021-12-24 22:45:43 +09:00
Bill Currie
bdd3870d2f [qfcc] Add failing test for dealloc warning
I have gotten tired of chasing memory leaks caused by me forgetting to
add [super dealloc] to my dealloc methods, so getting qfcc to chew me
out when I do seems to be a good idea (having such a warning would have
saved me many hours, just as missing return warnings have).
2021-12-24 22:45:43 +09:00
Bill Currie
bdc9e9c39f [qfcc] Unalias types before checking for equality
Fixes a bogus redefinition when going from struct declaration to
typedef.
2021-12-24 06:45:13 +09:00
Bill Currie
e0497c7fff [tools] Add a tool to dump sprite info 2021-12-17 08:31:28 +09:00
Bill Currie
ca9566a425 [qfbsp] Print the number of textures in the bsp
I needed to check what sort of numbers to expect for bsp texture counts.
It turns out id maps use only 81 max, but oum uses up to 173.
2021-12-17 08:28:02 +09:00
Bill Currie
3b1acf8a7b [qfcc] Correct some errors in the man page 2021-11-29 11:38:54 +09:00
Bill Currie
88b3965794 [vulkan] Pick up vulkan.h from correct location
Well... it could be done better, but this works for now assuming it's in
/usr/include (and it's correct for mxe builts). Does need proper
autoconfiscation, though.
2021-11-19 22:36:19 +09:00
Bill Currie
7ed12e2f37 [qfcc] Fix static function declarations
I'm surprised it took this long for static not working to cause a
problem.
2021-09-24 19:49:55 +09:00
Bill Currie
6144100d9b [qfcc] Check init exists before checking compound
Fixes a segfault that occurred during a parse error in a def
initializer.
2021-09-24 19:44:14 +09:00
Bill Currie
8a5c3c1ac1 [util] Add sys function to get cpu count
And use it in qfvis.
2021-08-13 21:26:48 +09:00
Bill Currie
d88bd96390 [qfvis] Use cmem for portal flow stack node allocation
The portal flow stack nodes contain a simd vector, which requires
16-byte alignment. However, on 32-bit Windows, malloc returns 8-byte
aligned memory, leading to eventual segfaults. Since pstack_t is 48
bytes on 32-bit systems, it fits nicely into a 64-byte aligned cache
line (or two on 64-bit systems due to being 80 bytes).
2021-08-13 11:33:03 +09:00
Bill Currie
6fb6885b88 [qfvis] Allocate only 128MB for the main hunk
Even ad_tears didn't really need 1GB, and 32-bit machines can't really
handle 1GB (at least on windows).
2021-08-13 11:33:03 +09:00
Bill Currie
2828500f04 [qfvis] Delay freeing of winding memory
If anything, this is probably a nano-optimization, depending on how
often portals are vis-rejected. I couldn't see any actual difference.
2021-08-08 12:34:18 +09:00
Bill Currie
9a93bf8d4a [qfvis] Make cluster reconstruction O(N)
For most (if not all) maps. The heapsort is needed only if the clustered
leafs are not contiguous, but most bsp compilers output contiguous leaf
clusters, so is just a bit of protection. The difference isn't really
noticeable on a fast machine, but no point in doing more work than
necessary.
2021-08-08 12:34:18 +09:00
Bill Currie
648ae3f877 [qfvis] Clean up the code and output a little
Dead code removed, and the job progress lines are now consistent and
have a job completion time when done.
2021-08-03 21:52:24 +09:00
Bill Currie
fe998f41b4 [qfvis] Convert leaf vis to cluster vis
Now that only 3852 clusters need to be checked for each cluster, fat-pvs
construction for ad_tears completes in about 0.7s, most of which seems
to be loading, conversion, compression and writing. O(N^3) cuts both
ways (hurts like crazy when N increases, does wonders when N decreases,
especially by a factor of 25). And then throw in improved cache
performance...

I suspect having an off-line compiler is still useful, but even if
qfvis's implementation never actually gets used, if cluster
reconstruction is put in the engine, large maps will be feasible even
for quakeworld. Just the reduced memory requirements alone will be a
huge benefit (~3GB down to 1.8MB).
2021-08-03 15:48:45 +09:00
Bill Currie
8da019e31c [qfvis] Reconstruct the leaf clusters in a bsp
This is only the first half (vertical) in that the vis bits are still
for the leafs rather than the clusters, but ad_tears goes from 500s to
7s for calculating the fat pvs (3852 clusters).
2021-08-03 11:37:24 +09:00
Bill Currie
421047328a [qfvis] Catch a missed winding mark stat 2021-08-02 23:17:55 +09:00
Bill Currie
ec54c54226 [build] Fix some windows bitrot 2021-08-02 14:02:41 +09:00
Bill Currie
f514345d77 [qfbsp] Show correct object counts for bsp29 files
Yes, this was the goal of that size_t change, but it made sense anyway.
2021-08-01 22:05:31 +09:00
Bill Currie
674ffa0941 [util] Make bsp_t counts size_t
and other bsp data counts unsigned, and clean up the resulting mess.
2021-08-01 21:54:05 +09:00
Bill Currie
40367e5bca [qfvis] Thread the ambient sounds computation 2021-08-01 18:14:28 +09:00
Bill Currie
e671b3f230 [qfvis] Thread the portal vis compaction
The compaction deals with merging all the portal visibility into cluster
visibility, expanding out to leafs, and final compression.
2021-08-01 17:06:13 +09:00
Bill Currie
523ab007d6 [qfvis] Produce more details base-vis stats
And nicely, things add up (after fixing 32-bit overflows :P)
2021-07-30 23:07:12 +09:00
Bill Currie
ca8dcf3fa9 [qfvis] Use cluster sphere culling for base vis
While this doesn't give as much of a boost as does basic sphere culling
(since it's just culling sphere tests), it took ad_tears' base vis from
1000s to 720s on my machine.
2021-07-30 18:52:47 +09:00
Bill Currie
9461779ba7 [qfvis] Remove the cluster portals limit
This removes the last of the arbitrary limits from qfvis. The goal is
not so much supporting crazy maps, but more about better data usage
(cluster_t is now 24 (or 16) bytes instead of 1048 (or 528). And
passages isn't used (yet?)...
2021-07-29 21:03:07 +09:00
Bill Currie
756214ca8e [qfvis] Use unsigned for the plane side tests
Doesn't make any difference to the number of instructions, but seeing
sar instead of shr bothers me when working with bit patterns.
2021-07-29 15:25:37 +09:00
Bill Currie
72a1fef714 [qfvis] Use hunk to manage winding memory
It turns out cmem is not so good for many large allocations (probably a
bug in handling the blocks), but was really meant for lots of little
churning allocations anyway. After an analysis of winding lifetimes, it
became clear that the hunk allocator would work very well. The base
windings are allocated from a global hunk (currently 1GB, plenty for
even ad_tears), and ephemeral windings are allocated from a per-thread
hunk of 1MB (seems to be way more than enough: gmsp3v2 uses a maximum of
only 56064 bytes, and ad_tears got through 30% before I gave up on it).
Any speed difference (for gmsp3v2) seems to be lost in the noise: still
completing in 38.4s on my machine.
2021-07-29 11:49:18 +09:00
Bill Currie
e39bc83a6a [qfvis] Optionally use utf8 to encode run lengths
Adds 50 bytes to marcher's fat-pvs, but removes about 4.7MB from
ad_tear's fat-pvs.
2021-07-27 23:29:14 +09:00
Bill Currie
b9d2882e02 [qfvis] Write out the fat-pvs file
The output fat-pvs data is the *difference* between the base pvs and fat
pvs. This currently makes for about 64kB savings for marcher.bsp, and
about 233MB savings for ad_tears.bsp (or about 50% (470.7MB->237.1MB)).
I expect using utf-8 encoding for the run lengths to make for even
bigger savings (the second output fat-pvs leaf of marcher.bsp is all 0s,
or 6 bytes in the file, which would reduce to 3 bytes using utf-8).
2021-07-27 20:04:19 +09:00
Bill Currie
49c3dacbbc [util] Rename set_size to set_count
After seeing set_size and thinking it redundant (thought it returned the
capacity of the set until I checked), I realized set_count would be a
much better name (set_count (node->successors) in qfcc does make much
more sense).
2021-07-27 11:52:21 +09:00
Bill Currie
946867c82e [qfvis] Start work on an off-line fat pvs compiler
Extremely large maps take a very long time to process their PVS sets for
PHS or shadows, so having an off-line compiler seems like a good idea.
The data isn't written out yet, and the fat pvs code may not be optimal
for cache access, but it gets through ad_tears in about 500s (12
threads, compared to 2100s single-threaded in the qw server).
2021-07-26 22:42:03 +09:00
Bill Currie
60d23bdc8f [qfbsp] Remove all arbitrary bsp limits
Planes, verts, etc can now all get crazy big.
2021-07-26 13:10:06 +09:00
Bill Currie
13e1682f5e [qfbsp] Add an option to dump info about a bsp file
Just the header plus some basics of model info.
2021-07-25 12:14:04 +09:00
Bill Currie
342ba65f57 [qfcc] Handle aliased field types
Fixes use of structs as entity fields. The test is currently
compile-only.
2021-07-24 18:09:54 +09:00
Bill Currie
8369a2206a [glstub] Add functions needed to get glsl working
Pity vulkan won't be so easy.
2021-07-22 10:12:41 +09:00
Bill Currie
6b38a17cf1 [gamecode] Clean up state imlementations
This makes the code easier to read. Also, yay for automated tests:
caught a mistyped time :)
2021-07-15 16:55:02 +09:00
Bill Currie
0a847f92f1 [util] Use mmap/munmap for cmem internal alloc/free
This reduces the overhead needed to manage the memory blocks as the
blocks are guaranteed to be page-aligned. Also, the superblock is now
alllocated from within one of the memory blocks it manages. While this
does slightly reduce the available cachelines within the first block (by
one or two depending on 32 vs 64 bit pointers), it removes the need for
an extra memory allocation (probably via malloc) for the superblock.
2021-07-12 16:33:47 +09:00