They're binned by powers of two (with in between sizes going to the
smaller bin should I make cache-line allocations NPOT (which I think
might be worthwhile). However, there seems to still be a bug somewhere
causing a nasty leak as now my hacked qfvis consumes 40G in less than a
minute.
The idea is to not search through blocks for an available allocation.
While the goal was to speed up allocation of cache lines of varying
cluster sizes, it's not enough due to fragmentation.
They take advantage of gcc's vector_size attribute and so only cross,
dot, qmul, qvmul and qrot (create rotation quaternion from two vectors)
are needed at this stage as basic (per-component) math is supported
natively by gcc.
The provided functions work on horizontal (array-of-structs) data, ie a
vec4d_t or vec4f_t represents a single vector, or traditional vector
layout. Vertical layout (struct-of-arrays) does not need any special
functions as the regular math can be used to operate on four vectors at
a time.
Functions are provided for loading a vec4 from a vec3 (4th element set
to 0) and storing a vec4 into a vec3 (discarding the 4th element).
With this, QF will require AVX2 support (needed for vec4d_t). Without
support for doubles, SSE is possible, but may not be worthwhile for
horizontal data.
Fused-multiply-add is NOT used because it alters the results between
unoptimized and optimized code, resulting in -mfma really meaning
-mfast-math-anyway. I really do not want to have to debug issues that
occur only in optimized code.
The problem is that I needed to support dynamic types on operators (for
bit-field enums), had things working, but a bad edit messed things up
and I had to rebuild that bit of code. Missed one bit :P
It is capable of parsing single expressions with fairly simple
operations. It current supports ints, enums, cvars and (external) data
structs. It is also thread-safe (in theory, needs proper testing) and
the memory it uses can be mass-freed.
This was inspired by
Hoard: A Scalable Memory Allocator
for Multithreaded Applications
Emery D. Berger, Kathryn S. McKinley, Robert D. Blumofe, Paul R.
Wilson,
It's not anywhere near the same implementation, but it did take a few
basic concepts. The idea is twofold:
1) A pool of memory from which blocks can be allocated and then freed
en-mass and is fairly efficient for small (4-16 byte) blocks
2) Tread safety for use with the Vulkan renderer (and any other
multi-threaded tasks).
However, based on the Hoard paper, small allocations are cache-line
aligned. On top of that, larger allocations are page aligned.
I suspect it would help qfvis somewhat if I ever get around to tweaking
qfvis to use cmem.
The calculation fails (produces NaN) if the vectors are anti-parallel,
but works for all other combinations. I came up with this implementation
when I discovered Unity's Quaternion.FromToRotation could did not work
with very small angles. This implementation will produce a usable
quaternion below 0.00255 degrees (though it will be slightly larger than
unit). Unity's failed such that I could see KSP's skybox snap while it
rotated around my test vessel.
The array has to be allocated using byte elements and thus the size of
the array is the number of bytes, but it needs to be the actual number
of elements in the array. Problem caused by not knowing the actual type
(and C not having type variables anyway).
PL_ParseDictionary itself does only one level, but it takes care of the
key-field mappings and property list item type checking leaving the
actual parsing to a helper specified by the field. That helper is free
to call PL_ParseDictionary recursively.
The first line of the parsed item is stored and can be retrieved using
PL_Line. Line numbers not stored for dictionary keys yet. Will be 0 for
any items generated by code rather than parsed from a file or string.
There's still some cleanup to do, but everything seems to be working
nicely: `make -j` works, `make distcheck` passes. There is probably
plenty of bitrot in the package directories (RPM, debian), though.
The vc project files have been removed since those versions are way out
of date and quakeforge is pretty much dependent on gcc now anyway.
Most of the old Makefile.am files are now Makemodule.am. This should
allow for new Makefile.am files that allow local building (to be added
on an as-needed bases). The current remaining Makefile.am files are for
standalone sub-projects.a
The installable bins are currently built in the top-level build
directory. This may change if the clutter gets to be too much.
While this does make a noticeable difference in build times, the main
reason for the switch was to take care of the growing dependency issues:
now it's possible to build tools for code generation (eg, using qfcc and
ruamoko programs for code-gen).
When I ported SEB to python, I discovered that I apparently didn't
really understand the paper's description of the end condition and the
usage of the affine and convex sets for center testing. This cleans up
the test and makes SEB more correct for the cases that have less than 4
supporting points (especially when there are less than 4 points total).
They take a pointer to a free-list used for hashlinks so the hashlink
pools can be per-thread. However, hash tables that are not updated are
always thread-safe, so this affects only updates. progs_t has been set
up such that it is easy for multiple progs within one thread can share
hashlinks.
Other than consistency with printf(), I'm not sure why we went with the
printed size as the return value; returning the resultant strings makes
much more sense as dsprintf() (etc) can then be used as a safe va()
The initial code was pretty much a port of the code in the editor I
wrote 25 years ago. Either I didn't think of the optimization back then,
or I tried to implement it, failed, and figured it wasn't worth it
(despite using it on a 386dx33). However, I noticed it now and it was
easy enough to get working, and it's always good to not do something
that's not needed.
The better accuracy is for specific cases (90 degree rotations around a
main axis: the matrix element for that axis is now 1 instead of
0.99999994). The speedup comes from doing fewer additions (multiply
seems to be faster than add for fp, at least in this situation).
After messing with SIMD stuff for a little, I think I now understand why
the industry went with xyzw instead of the mathematical wxyz. Anyway, this
will make for less pain in the future (assuming I got everything).
While scan-build wasn't what I was looking for, it has proven useful
anyway: many of the sizeof errors were just noise, but a few were actual
bugs (allocating too much or too little memory).
This fixes the problem of not finding files without a .gz extension when
gzip support is enabled (most of my quake data is compressed, so it took a
while for me to notice the problem).
_QFS_VOpenFile is actually _QFS_FOpenFile reimplemented to take vpath start
and end parameters so the search can be limited. QFS_VOpenFile,
_QFS_FOpenFile, and QFS_FOpenFile are all wrappers for _QFS_VOpenFile.
A vpath is the union of all locations searched for a file in a single
gamedir (eg, shadows, id1 etc). This is a necessary step to preventing
problems like id1/maps/start.lit being used for shadows/maps/start.bsp.
However, QFS_FilelistFill still needs to be reworked as it does not compile
yet (testing was done with a gutted QFS_FilelistFill).
The idea comes from The OpenGL Shader Wrangler
(http://prideout.net/blog/?p=11). Text files are broken up into chunks via
lines beginning with -- (^-- in regex). The chunks are optionally named
with tags of the form: [0-9A-Za-z._]+. Unnamed chunks cannot be found.
Searching for chunks looks for the longest tag that matches the beginning
of the search tag (eg, a chunk named "Vertex" will be found with a search
tag of "Vertex.foo"). Note that '.' forms the units for the searc
("Vertex.foo" will not find "Vertex.f").
Unlike glsw, this implementation does not have the concept of effects keys
as that will be separate. Also, this implementation takes strings rather
than file names (thus is more generally useful).
set_bits_t is now 64 bits for x86_64 machines (in linux, anyway). This gave
qfvis a huge speed boost: from ~815s to ~720s.
Also, expose some of the set internals so custom set operators can be
created.
Now we can get tight (<1e-6 * radius_squared error) bounding spheres. More
importantly (for qfvis, anyway) very quickly: 1.7Mspheres/second for a 5
point cloud on my 2.33GHz Core 2 :)
It "works" for lines, triangles and tetrahedrons. For lines and triangles,
it gives the barycentric coordinates of the perpendicular projection of the
point onto to features. Only tetrahedrons are guaranteed to reproduce the
original point.
Rather than prefixing free_ to the supplied name, suffix _freelist to the
supplied name. The biggest advantage of this is it allows the free-list to
be a structure member. It also cleans up the name-space a little.
Sys_LongTime returns time in microseconds as a 64-bit int. Sys_DoubleTime
uses Sys_LongTime, converts to double and offsets 0 time by 4G (2**32).
This gives us consistent sub-microsecond precision for a very long time.
See http://randomascii.wordpress.com/2012/02/13/dont-store-that-in-a-float/
The ~ gets expanded to CSIDL_LOCAL_APPDATA, $HOME, $USERPROFILE or just
".", whichever succeeds first. The usual location will be:
"C:\windows\profiles\<user>\Local Settings\Application Data".
"." is now the fallback for *nix systems too.
This gives QF a consistent qualilty PRNG on all platforms. The
implementation is slightly different from the standard, but gives the same
results for the same speed (details in mersenne.c).
It seems the code expected octal escapes to always start with 0. This is
not the case. Also, octal escapes are limited to 3 digits (and hex to 2).
This fixes the garbled bold text in ITS.
The bsp2 header is not necessarily correct (or even present), but the bsp29
header is: it was setup via set_bsp32_write. This fixes the bsp corruption
when vising a map (and, I expect, any problems with qfbsp on a big-endian
machine).
qfcc now does local common subexpression elimination. It seems to work, but
is optional (default off): use -O to enable. Also, uninitialized variable
detection is finally back :)
The progs engine now has very basic valgrind-like functionality for
checking pointer accesses. Enable with pr_boundscheck 2
Getting everything right with an enum proved to be too difficult if not
impossible. Also use better tests for equivalence and intersection.
Many more tests have been added. All pass :)
Also move the ALLOC/FREE macros from qfcc.h to QF/alloc.h (needed to for
set.c).
Both modules are more generally useful than just for qfcc (eg, set
builtins for ruamoko).
While accessing short foo[2][4]; as foo[0][0..7] should work in theory, who
knows what gcc does with it when optimizing. I don't know if this will fix
johnnyonflame's bsp loading problem, but no point in having rhinodemonic
code hanging around.
This necessitated disabling the id2 padding, but it's only commented out
incase there's more growth. Now the (compiler) error in -addObjectNoRetain
is caught ealier.
Need to subtract the size of the bsp_t/bsp29_t struct. Now old and new
qfbsp produce identical bsps (so long as they're both unoptimized, or
(probably) both optimized).
All of the nastiness is hidden in bspfile.c (including the old bsp29
specific data types). However, the conversions between bsp29 and bsp2 are
implemented but not yet hooked up properly. This commit just gets the data
structures in place and the obvious changes necessary to the rest of the
engine to get it to compile, plus a few obvious "make it work" changes.
I didn't like the way client/server code was poking around at the
implementation. Instead, provide a couple of accessor functions for the
same information.
And the tests really exercised VectorShear (first attempt had things
messed up when more than one shear value was non-zero). Also,
Mat4Decompose wasn't orthogonalizing the z axis row. Oops. Anyway,
Mat4Decompose is now known to work well, and the usage of its output is
understood :)
It seems (some versions of) windows vsnprintf don't count the terminating 0
when limiting the number of chars written to the buffer. Nor do they
guarantee the output string will be terminated.
I got the idea from blender when I discovered by accident that quat * vect
produces the same result as quat * qvect * quat* and looked up the code to
check what was going on. While matrix/vector multiplication still beats the
pants off quaternion/vector multiplication, QuatMultVec is a slight
optimization over quat * qvect * quat* (17+,24* vs 24+,32*, plus no need to
to generate quat*).
One's an actual bug, the other a bit of error checking (not sure how
necessary it is, but it's in code that we don't /want/ to run, so it can't
hurt :)
Thanks to spirit for pointing that QF wasn't compiling with zlib 1.2.6
(archlinux, not yet in debian).
I was using gzFile as "gzFile *gzfile", but gzFile is already a pointer. In
older versions of zlib (including the 1.2.3 that's in debian), gzFile is
declared as a void *, and it seems that gcc is happy with assigning void **
to void *. However, in recent zlib, gzFile is now struct gzFile_s *, which
gcc is most definitely unhappy about assigning to struct gzFile **.
I just hope that either I had misread the type back when I wrote quakeio,
or that nobody is using such an ancient zlib.
o All instances of LIBADD/LDADD have a corresponding DEPENDENCIES
specificatiion.
o libraries now use a lib_ldflags macro to keep things consistent
o duplication of source/lib names has been minimized (particularly in
the libraries; more work needs to be done for the executables)
o automake spec blocks have been organized (again, more work needs to be
done for the executables)
Buffer underflow and though strcpy has always been safe there, change to
memmove. Had the added benefit of helping me create more test cases for
better coverage.
Yay, Deek for finding another bug in qf :). He had an old dirconf that
didn't specify HudType and thus caused the status bar code to blow up. Set
the default to "id" if none is found.
I got rather tired of there being multiple definitions of mostly compatible
plane types (and I need a common type anyway). dplane_t still exists for
now because I want to be careful when messing with the actual bsp format.
I have no idea why I did that, but it crept in with the var substitution
fix, so I guess it might have been an attempt to fix a bug, but it looks
like it was broken anyway.
Currently only "id", "hipnotic" and "rogue" are supported (anything else is
treated as "id"). Has no effect in quakeworld (good thing too: changing
gamedirs is a little broken).
