It turned out they were always using floats for the source type (meaning
doubles were broken), and not shifting the component in the final sizzle
code meaning all swizzles were ?xxx (neglecting minus or 0). I'd make
tests, but I plan on modifying the instruction set a little bit.
Also, correct the handling of scalars in dot and wedge products: it
turns out s.v and s^v both scale. However, it seems the CSE code loses
things sometimes.
This has shown the need for more instructions, such as a 2d wedge
product and narrower swizzles. Also, making dot product produce a vector
instead of a scalar was a big mistake (works nicely in C, but not so
well in Ruamoko).
The current code is pretty broken when it comes to vector types (losing
the vector and bogus errors among other issues). The whole thing needs a
rework or even just to be tossed in favor of better DAG processing.
I guess Hamish's suggestion made sense at the time, but I found that
with the current instructions, the reversed bivector wasn't so nice to
implement it would need a swizzle as well as the cross-product.
By default. Conversion of quake strings needs to be requested (which is
done by nq and qw clients and servers, as well as qfprogs via an
option). I got tired of seeing mangled source code in the disassembly.
I'm not sure if that was a thinko, typo, or something else, but judging
by the relevant commit message, the use of quaternion and vector was
intended only for advanced progs (v6p).
This makes working with them much easier, and the type system reflects
what's in the multi-vector. Unfortunately, that does mean that large
algebras will wind up having a LOT of types, but it allows for efficient
storage of sparse multi-vectors:
auto v = 4*(e1 + e032 + e123);
results in:
0005 0213 1:0008<00000008>4:void 0:0000<00000000>?:invalid
0:0044<00000044>4:void assign (<void>), v
0006 0213 1:000c<0000000c>4:void 0:0000<00000000>?:invalid
0:0048<00000048>4:void assign (<void>), {v + 4}
Where the two source vectors are:
44:1 0 .imm float:18e [4, 0, 0, 0]
48:1 0 .imm float:1aa [4, 0, 0, 4]
They just happen to be adjacent, but don't need to be.
Scaling now works for multi-vector expressions, and always subtracting
even when addition is wanted doesn't work too well. However, now there's
the problem of multi-vectors very quickly becoming full algebra vectors,
which means certain things need a rethink.
This gets only some very basics working:
* Algebra (multi-vector) types: eg @algebra(float(3,0,1)).
* Algebra scopes (using either the above or @algebra(TYPE_NAME) where
the above was used in a typedef.
* Basis blades (eg, e12) done via procedural symbols that evaluate to
suitable constants based on the basis group for the blade.
* Addition and subtraction of multi-vectors (only partially tested).
* Assignment of sub-algebra multi-vectors to full-algebra multi-vectors
(missing elements zeroed).
There's still much work to be done, but I thought it time to get
something into git.
If a symbol is not found in the table and a callback is provided, the
callback will be used to check for a valid procedural symbol before
moving on to the next table in the chain. This allows for both tight
scoping of the procedural symbols and caching.
Due to joys of pointers and the like, it's a bit of a bolt-on for now,
but it works nicely for basic math ops which is what I wanted, and the
code is generated from the expression.
Only · (dot product) and × (cross product for vector, commutator product
for geometric algebra) have been tested so far, but that involved
fighting with cpp to get it to not convert the · to \U000000b7, which
was rather annoying.
Fixing a load of issues related to autoconf and some small source-level issues to re-add clang support.
autoconf feature detection probably needs some addressing - partially as -Werror is applied late.
I never liked it, but with C2x coming out, it's best to handle bools
properly. I haven't gone through all the uses of int as bool (I'll leave
that for fixing when I encounter them), but this gets QF working with
both c2x (really, gnu2x because of raw strings).
The warning flag check worked too well: it enabled the warning and
autoconf's default main wanted the const attribute. The bug has been
floating around for a while, it seems.
This uses ud-chains for function statements (call/return) to force their
arguments to be live (in particular, indirect references via pointers)
this fixes the arraylife test.
The ud- and du-chains include known side-effects of the instructions and
thus depict a more accurate view of what operands an instruction uses or
defines. Fixes the arraylife2 test.
Like defs, a partial write should not define the whole temp. Thus, copy
the "don't visit main" behavior recently added to def_visit_all. Fixes
missing ud-chains for component-by-component assignments to temporary
vectors.
I'm not certain this is correct, but it seems to me that du-chains are
the same information as ud-chains, but from the defining statement's
point of view instead of that of the using statement.
As certain statements (in particular, function calls) can use additional
variables via pointer parameters, it's necessary to iterate ud-chain
building until the count stabilizes. This should make live variable
analysis much easier.
I think the current build_element_chain implementation does a reasonable
job, but I'm in the process of getting designated initializers working,
thus it will become important to ensure uninitialized members get
initialized.
I never liked the various hacks I had come up with for representing
resource handles in Ruamoko. Structs with an int were awkward to test,
pointers and ints could be modified, etc etc. The new @handle keyword (@
used to keep handle free for use) works just like struct, union and
enum in syntax, but creates an opaque type suitable for a 32-bit handle.
The backing type is a function so v6 progs can use it without (all the
necessary opcodes exist) and no modifications were needed for
type-checking in binary expressions, but only assignment and comparisons
are supported, and (of course) nil. Tested using cbuf_t and QFile: seems
to work as desired.
I had considered 64-bit handles, but really, if more than 4G resource
objects are needed, I'm not sure QF can handle the game. However, that
limit is per resource manager, not total.
This takes advantage of the ud-chains to follow the trail of pointer
assignments looking for an address. This gets array element assignments
surviving across blocks when the array itself is passed to a function.
It doesn't help when the address of the element is taken though. I think
that's a dags problem and probably needs du-chains. Also, the ud-chain
creation should probably be done in two passes so the newly found
information can be recorded.
Def and kill are still handled in flow_analyze_statement, but this makes
call meta data more consistent between v6 and ruamoko progs, allowing
the statement use chain to be used for call argument analysis. It even
found a bug in the extraction of param counts from the call instruction.
I had missed the flowvar clearing for auxiliary use/def/kill operands.
It's possible it wasn't necessary at the time since the operands were
added just for dealloc checking, but there's every reason it could
become necessary.
The first use will be pointer analysis for function arguments where the
argument points to an array to mark the array as live, but I'm sure
there'll be plenty of other uses.
A partial write to a def should not define the whole def, thus
def_visit_all's overlap parameter now has a flag that prevents a visit
to the main def when accessing the def from an alias def. This prevents
a lot of spurious kills and defines in flow analysis.
The array access code was loading the vector, modifying the element,
then forgetting to write the modified vector back to whence it came.
However, that would be rather sub-optimal, so now when the vector is
accessed by a pointer, the array code switches to field access to get at
the vector element thus avoiding the need to copy the whole vector.
Needed for proper analysis (ud-chains etc). Of course, it was then
necessary to remove the parameter defs from the uninitialized defs.
Also, plug a couple of memory leaks (forgot to free some temporary
sets).
That is, `array + offset`. This actually works around the bug
highlighted by arraylife.r (because the array is explicitly used), but
is not a proper solution, so that test still fails of course. However,
with this, it's no longer necessary to use `&array[index]` instead of
`array + index`.
I could never remember what any of the numbers meant. While define is
still a little fuzzy (they're (pseudo)statement numbers), at least now
I'll always know that the numbers are the define set. Also, having the
flow address of the variable helps with understanding the reaching defs
output.
I had messed up the handling of declarators for combinations of pointer,
function, and array: the pointer would get lost (and presumably arrays
of functions etc). I think I had gotten confused and thought things were
a tree rather than a simple list, but Holub set me straight once again
(I've never regretted getting that book). Once I understood that, it was
just a matter of finding all the places that needed to be fixed. Nicely,
most of the duplicated code has been refactored and should be easier to
debug in the future.
The type system rewrite had lost some of the checks for function fields.
This puts the actual code in the one place and covers parameters as well
as globals.
Internally, * is not really a valid operator for vectors since it can
have many meanings. This didn't cause trouble until trying to build
everything in game-source (since there's still a lot of legacy code in
there).
The precedence check changes done in
63795e790b seem to have been incorrect
(game-source/ctf produced many false positives), so putting that check
against '=' back into the code seems like a good idea (no more false
positives). That sounds a bit cargo-cult, but I'm really not sure what I
was thinking when I did the changes (probably just tired).
This applies only to the top-level scope of the function. I'm not sure
if it's right for traditional quakec code, but that can be adjusted
easily enough.
The symtab code itself cares only about global/not global for the size
of the hash table, but other code can use the symtab type for various
checks (eg, parameter shadowing).
Along with QuakeC's, of course. This fixes type typeredef2 test (a lot
of work for one little syntax error). Unfortunately, it came at the cost
of requiring `>>` in front of state expressions on C-style functions
(QuakeC-style functions are unaffected). Also, there are now two
shift/reduce conflicts with structs and unions (but these same conflicts
are in gcc 3.4).
This has highlighted the need for having the equivalent of the
expression tree for the declaration system as there are now several
hacks to deal with the separation of types and declarators. But that's a
job for another week.
The grammar constructs for declarations come from gcc 3.4's parser (I
think it's the last version of gcc that used bison. Also, 3.4 is still
GPL 2, so no chance of an issue there).
This simplifies type type_specifier rule significantly as now TYPE_SPEC
(was TYPE) includes all types and their basic modifiers (long, short,
signed, unsigned). This should allow me to make the type system closer
to gcc's (as of 3.4 as that seems to be the last version that used a
bison parser) and thus fix typeredef2.
Once a unicode char (ie, > 127) was used, any ascii chars would get the
tail of the last unicode char resulting in broken utf-8 streams. The
resulting null glyph boxes were not very appealing.
This fixes the basic vecconst test (extending it to other types breaks
because long and ulong are not properly supported yet). The conversion
is done by the progs VM rather than writing another 256 conversions
(though loops could be used). This works nicely as a test for using the
VM to help with compiling.
While the option to make '*' mean dot product for vectors is important,
it breaks vector scaling in ruamoko progs as the resultant vector op
becomes a dot product instead of the indented hadamard product (ie,
component-wise).
The support for the new vector types broke compiling code using
--advanced. Thus it's necessary to ensure vector constants are
float-type and vec3 and vec4 are treated as vector and quaternion, which
meant resurrecting the old vector expression code for v6p progs.
The method is still held by known_methods, so freeing it causes grief.
However, it may cause a leak thus the free is only commented out. More
investigation is needed. I'm surprised the problem didn't show on linux,
but cygwin-native hit it and valgrind on linux found the spot :)
While it does get a bit cluttered currently, being able to see the
contents of structures makes a huge difference. Also highlights that
vector immediates do not get the correct type encodings.
This fixes the internal error generated by the likes of
`(sv_gravity * '0 0 1')` where sv_gravity is a float and `'0 0 1'` is an
ivec3: the vector is promoted to vec3 first so that expanding sv_gravity
is expanded to vec3 instead of ivec3 (which is not permitted for a
float: expansion requires the destination base type to be the same as
the source).
For now, anyway, as the generated code looks good. There might be
problems with actual pointer expressions, but it allows entity.field to
work as expected rather than generate an ICE.
The resultant unicode is encoded as utf-8, which does conflict with the
quake character map, but right now unicode is useful only with font
text, and those support only standard unicode (currently only as utf-8),
but something will need to be sorted out.
Arrays are passed as a pointer to the first element, so are always valid
parameters. Fixes a bogus "formal parameter N is too large to be passed
by value" error.
While swizzle does work, it requires the source to be properly aligned
and thus is not really the best choice. The extend instruction has no
alignment requirements (at all) and thus is much better suited to
converting a scalar to a vector type.
Fixes#30
It seems clang loses track of the usage of the referenced unions by the
time the code leaves the switch. Due to the misoptimization, "random"
values would get into the vector constants. This puts the usages in the
same blocks as the unions, causing clang to "get it right" (though I
strongly suspect I was running into UB).
While I might need to tighten up the rules later, this allows binary
operations between vector (the type) and explicitly typed vec3 constants
(and non-constants, about which I am undecided). The idea is that
explicit constants such as '1 2 3'f should be compatible with either
type.
This applies to quaternions as well.
As a class's ivars are built up by inheritance, but with only that
class's ivars in the symbol table, is is necessary to include an offset
based on the super class's ivars in order to ensure alignments are
respected. This is achieved via the new `base` parameter to
build_struct(), which is used to offset the current size while
calculating the aligned offset of the symbols. The parameter is ignored
for unions, as they always start at 0. The ivars for the current class
still have a base offset of 0 until they are actually added to the
class.
Fixes#29
The alignment is specified as a power of 2 (ie, actual alignment = 1 <<
alignment) allowing old object files to be compatible (as their
alignment is 0). This is necessary for (in part for #30) as it turned
out even global vectors were not aligned correctly.
Currently, only data spaces even vaguely respect alignment. This may
need to be fixed in the future.
Most were pretty easy and fairly logical, but gib's regex was a bit of a
pain until I figured out the real problem was the conditional
assignments.
However, libs/gamecode/test/test-conv4 fails when optimizing due to gcc
using vcvttps2dq (which is nice, actually) for vector forms, but not the
single equivalent other times. I haven't decided what to do with the
test (I might abandon it as it does seem to be UD).
