I don't remember why I couldn't get this to work last time I tried, but
it went well this time, and made a significant difference to compiling
vulkan.r (from 1.1s to 0.15s unoptimized or 0.8s optimized).
This fixes the problem with // comments after the file in #include and
the core problem the complicated // handling tried to fix with
suppressed directives. Funny how it's always the simpler code that works
better :/
I'm undecided about @ in macro names, but treating @id as one token in
the body is necessary with the single-pass tokenizing. Fixes an infinite
macro expansion loop in vecaddr.r (`#define dot @dot`), but that's
really only a bandaid for *that* issue as there are plenty of other
cases where macros will loop.
It turns out that the recursive lexing was over-complicated as the
tokens for nested macros need to come from the expanded stream, not the
raw input stream.
cpp does this for us so the double-generation is redundant (and
currently wrong anyway with things like <built-in> and <command-line>
getting into the list of dependencies).
Or at least mostly so. The __QFCC__ define isn't visible, and it seems
undef might not be working properly (ruamoko/lib/types.r doesn't
compile). Of course, there's still the issue of whether it's compiling
correctly.
If ID gets to the preprocessor parser in expressions, the ID is not
defined because if it was defined, it would have been expanded. Thus,
all IDs are 0.
In addition to cleaning up the old flex line rules, this improves
handling of the '# num "file" flags' from cpp to at least parse the
additional flags (support for the system header flag might come later,
but I doubt the extern-c flag will have much meaning).
QuakePascal has lost its line directive handling (no errors, but dead
rules) for now. Eventually the lexers will be merged.
I had wanted to do this earlier but shied away from the large edit. Now
it became more necessary (and will become even more necessary when I get
to the glsl front-end).
Really, function-type macros expand too, but incorrectly as the
parameters are not parsed and thus not expanded, but this gets the basic
handling implemented, including # and ## processing.
Converting ID and char constants too early resulted in poor handling of
keywords and spurious diagnostics about multi-byte character constants,
particularly with -E (preprocess-only)
Mostly white space, but a little more consistency in handling and remove
the use of REJECT (which does actually seem to make a difference, but it
could be just noise).
As far as I can tell, the preprocessor numbers conform with C23 except
for a couple of extensions (both ' and _ work for digit separators, and
d/D work for explicit doubles (since qfcc current defaults to float
instead of double)). This massively cleaned up the numeric rules and
even took care of some UB in the vector parsing code (I'm not sure which
is more surprising: that I didn't see it at the time, or that it was
blindingly obvious now).
This will be used for unifying preprocessing and parsing, the idea being
that the tokens will be recorded for later expansion via macros, without
the need to retokenize.
While my modified version is needed to actually avoid warnings (vs
upstream git flex), the files still work with debian's flex (with no
warnings). I needed to update (and fix) flex so the lexer line numbers
would be correct.
The improved location tracking isn't used yet, but was fairly invasive
on the bison-flex api.
It also cleans up some of the ancient workarounds for bad flex cores.
It turns out I need to create my own cpp in order to handle glsl's
directives. I've decided to make a unified lexer, and continuation lines
seemed a good place to start.
This fixes the really odd bug of certain string values getting swapped
in vkgen when DEBUG_QF_MEMORY was defined in expr.c. It will also
prevent a lot of fun with floats in the future, I imagine.
It's now meant only for ALLOC. Interestingly, when DEBUG_QF_MEMORY is
defined in expr.c, something breaks badly with vkgen (no sniffles out of
valgrind, though), but everything is fine with it not defined. It seems
there may be some unpleasant UB going on somewhere.
I'm not sure why this showed up now (I guess just not enough large
immediate values), but this fixes a segfault in the algtypes test (the
mystery is why it showed up this late).
This gets my `m * p * ~m` code as optimal as possible if my counting is
correct (this does not include the extra extends and add needed to merge
the values). Also, there might be a possibility of recombining some ops
into a vector op, but I'm happy with this.
That is, `x+x -> 2*x` (and similar for higher counts). Doesn't make much
difference for just 2, but it will make collecting scales easier and I
remember some testing showing that `2*x` is faster than `x+x` for
floating point.
Of course, motor-point keeps bouncing around numerically :/
This fixes the motor-point.r test (ie, the sub-type field selector works
on mono-group types now). Still need to sort out something for scalars
(but I imagine that can work only in an @algebra context).
This allows them to be matched with cancelling factors. My fancy zero
test is now just that: a fancy zero:
typedef @algebra(float(3,0,1)) PGA;
typedef PGA.group_mask(0xa) bivector_t;
typedef PGA.group_mask(0x1e) motor_t;
typedef PGA.tvec point_t;
typedef PGA.vec plane_t;
plane_t
apply_motor (motor_t m, point_t p)
{
return (m * p * ~m).vec;
}
0000 nop there were plums...
0001 adjstk 0, 0
apply_motor:
motor.r:32:{
0002 with 2, 0, 1
motor.r:33: return (m * p * ~m).vec;
0003 return (<void>)
The motor-point.r test fails because it uses (m * p * ~m).tvec to get
the value but the type system is slightly broken in that a mono-group
algebra type does not have a structure associated with it and thus the
"missing" field results in 0. Yes, I spent too long chasing that one,
too.
I'd missed this in the previous commit, which was a good thing, really,
as it turns out this was the trigger of the bug that causes my fancy
zero test to become non-zero. It seems the bug is in either
component_sum or in the extend merging.
Or really, the implementers. This gets my fancy zero test down to just
unrecognized permutations of commutative multiplies and dot products
(with the multiplies above the dot products).
While this does "explode" the instruction count (I'll have to collect
like terms later), it does allow for many more opportunities for things
to cancel out to 0 (once (pseudo)commutativity is taken care of).
That is, dot(scale(A,a),scale(B,b)) -> (a*b)*dot(A,B). Does the right
thing when only one side is a scale. No change to the instruction count
in my fancy zero, but it does open more opportunities when I distribute
products.
That is, scale(scale(A,b),c) becomes scale(A,b*c), thus giving the
expression dag more opportunities to find common sub-expressions. My
fancy zero test is down to 20 total instructions (including overhead, or
16 for the actual algebra).
While splitting up the scaled vector into scaled xyz and scaled w does
cost an extra instruction, it allows for other optimizations to be
applied. For one, extends get all the way to the top now, and there are
at most two (in my test cases), thus either break-even or even a slight
reduction in instruction count. However, in the initial implementation,
I forgot to do the actual scaling, and 12 instructions were removed from
my fancy zero case, but real tests failed :P It looks like it's just
distributivity and commutativity holding things back (eg,
omega*gamma*sigma - gamma*omega*sigma: should be 0, but not recognized
as that).
This fixes the motor test :) It turns out that every lead I had
previously was due to the disabling of that feature "breaking" dags
(such that expressions wouldn't be found) and it was the dagged
multi-vector components getting linked by expr->next that made a mess of
things.
Or at least mostly so (there are a few casts). This doesn't fix the
motor bug, but I've wanted to do this for over twenty years and at least
I know what's not causing the bug. However, disabling fold_constants in
expr_algebra.c does "fix" things, so it's still a good place to look.
This doesn't fix the motor bug, but it doesn't make it worse. However,
it does simplify the trees quite a bit, so it should be easier to debug.
It seems the problem has something to do with fold_constants messing up
dagged aliases: in particular, const-folding multiplication by e0123 in
3d PGA as fold_constants sees it as 1.
They should be treated as such only when merging vector components. This
fixes a bug that doesn't actually exist (it's in experimental code),
where the sum of two 3-component vectors was getting lost.
For cross products: remove any a from a×(...+/-a...)
For dot products: remove any a×b from a•(...+/-a×b...) (or b×a)
This removed another 2 instructions :)
They don't have much effect that I've noticed, but the expression dags
code does check for commutative expressions. The algebra code uses the
anticommutative flag for cross, wedge and subtract (unconditional at
this stage). Integer ops that are commutative are always commutative (or
anticommutative). Floating point ops can be controlled (default to non),
but no way to set the options currently.
This takes advantage of the expression dag to detect when an expression
is on both sides of a cross product (which always results in 0). This
removes 3 instructions from my motor test (28 to go).
Especially binary expressions. That expressions can now be reused is
what caused the need to make expression lists non-invasive: the reuse
resulted in loops in the lists. This doesn't directly affect code
generation at this stage but it will help with optimizing algebraic
expressions.
The dags are per sequence point (as per my reading of the C spec).
The removal of the e tag from expr_t necessitated making convert_name
return a new expressions which resulted in get_type no longer being
enough to both convert a name expression and get the type. This was just
another missed spot. With this, all of game-source except ctf builds.
Finally, that little e. is cleaned up. convert_name was a bit of a pain
(in that it relied on modifying the expression rather than returning a
new one, or more that such behavior was relied on).
Sum expressions pull the negation through extend expressions allowing
them to switch to subtraction when appropriate, and offset_cast reaches
past negation to check for extend expressions. This has eliminated all
negation-only instructions from the motor-point, shaving off more
instructions (now 27 not including the return).
I don't know why I didn't think to do it this way before, but simply
recursing into each operand for + or - expressions makes it much easier
to generate correct code. Fixes the motor-point test.
That is, passing int constants through ... in Ruamoko progs is no longer
a warning (still is for v6p and v6 progs). I got tired of getting the
warning for sizeof expressions when int through ... hasn't been a
problem for even most v6p progs, and was intended to not be a problem
for Ruamoko progs.
But really only for memset and memmove because they need to use an int
alias of the variable and it may be only that alias that sets a much
larger variable.
This removes all the special cases and thus it should be more robust. It
did show up some out-by-one (or a factor of two!) errors elsewhere in
the group mask calculations.
I'm uncertain about the names, but this makes it much easier to get at
specific subtypes (eg, PGA.tvec as a type) rather than having to know
the group mask.
This makes working with the plethora of types a little easier still. The
check for an algebra expression in field_expr needed to be moved up
because the struct code thought the algebra type was a normal vector.
And vis-versa.
I'm not sure what I was thinking, but I've decided that not being able
to cast the pseudo-scalar from float to double (for printf etc) was a
bug.
The merge_blocks function wasn't reporting whether it had done anything
so the thread/merge/dead blocks loop was bailing early. With this,
simple functions (ie, no control flow) are fully visible to the CSE
optimizer and it can get quite aggressive (removed 3 assignments and a
cross product from my barycenter test code).
Failing to promote ints to the algebra type results in a segfault in
assignment of a multi-vector due to the symbol pointer walking off the
end of the list of symbols.
And convert addition to subtraction when extend expressions are not
involved. This has taken my little test down to 56 instructions total
(21 for `l p ~l`), down from 74 (39).
This takes care of chained sums of extend expressions. Now `l p ~l` has
only four extend instructions which is expected for the code not
detecting the cross product that always produces 0.
This goes a ways towards minimizing extend expressions, even finding
zeros and thus eliminating whole branches of expressions, but shows the
need for better handling of chained sums of extends.
Any geometric algebra product of two negatives cancels out the negative,
and if the result is negative (because only one operand was negative),
the negation is migrated to above the operation. This resulted in
removing 2 instructions from one if my mini-tests (went from 74 to 78
with the addition/subtraction change, but this takes it back to 76
instructions).
Summed extend expressions are used for merging a sub-vector with a
scalar. Putting the vector first in the sum will simplify checks later
on (it really doesn't matter which is first so long as it's consistent).
Subtraction is implemented as adding a negative (with the plan of
optimizing it later). The idea is to give tree inspection and
manipulation a more consistent view without having to worry about
addition vs subtraction.
Negation is moved as high as possible in the expression, but is always
below an extend expression. The plane here is that the manipulation code
can bypass an alias-add-extend combo and see the negation.
This doesn't affect the generated code (aliases are free), but does
simplify the dag significantly, thus optimizing the compiler somewhat,
but also makes reading dags much easier and therefore optimizing the
debugging process.
Because the aliases were treated as live, every alias of a temp resulted
in an assignment, which proved to be quite significant (4-5 assignments
in some simple GA expressions). By using an alias node in the dag, the
unaliased temp can be marked live while the alias is treated as an
operation rather than an operand. Now my GA expressions have no
superfluous assignments (generally no assignments at all).
Simple k-vectors don't use structs for their layout since they're just
an array of scalars, but having the structs for group sets or full
multi-vectors makes the system alignment agnostic.
And geometric algebra vectors. This does break things a little in GA,
but it does bring qfcc's C closer to standard C in that sizeof respects
the alignment of the type (very important for arrays).
It's implemented as the Hodge dual, which is probably reasonable until
people complain. Both ⋆ and ! are supported, though the former is a
little hard to see in Consola.
That was surprisingly harder than expected due to recursion and a
not-so-good implementation in expr_negate (it went too high-level thus
resulting in multivec expressions getting to the code generator).
But only for scalar divisors. The simple method of AB†/(BB†) works only
if B is a versor and there's also the problem of left and right
division. Thanks to sudgy for making me stop and think before I actually
implemented anything (though he mentioned only that it doesn't work for
general mutli-vector divisors).
That was tedious. I can't say I'm looking forward to writing the tests
for 3d. And even though trivector . bivector and bivector . trivector
give the same answer, they're not really commutative when it comes to
the code.
Meaning vec3 is aligned to 4 components instead of 1. 2-component ops
use vec2 in the VM thus requiring alignment to boundaries of 2, but 4
seems better as it conforms with OpenGL and Vulkan (and, I imagine,
DirectX, but I doubt QF will ever use DirectX).
The singleton alias resulted in the adjusted swizzles being corrupted
when for the same def. Other than adding properly sized swizzles
(planned), the simplest solution is to (separately) allow alias that
stick out from from the def.
While the progs engine itself implements the instructions correctly, the
opcode specs (and thus qfcc) treated the results as 32-bit (which was,
really, a hidden fixme, it seems).
I didn't particularly like that solution due to the implied extra
bandwidth (probably should profile such sometime), but I think the
extend operations could be merged into simple assignments by the
optimizer at some stage (or further cleaned up when this stuff gets
moved to actual code gen where it should be).
Currently via only the group mask (which is really horrible to work
with: requires too much knowledge of implementation details, but does
the job for testing), but it got some basics working.
It turned out they were always using floats for the source type (meaning
doubles were broken), and not shifting the component in the final sizzle
code meaning all swizzles were ?xxx (neglecting minus or 0). I'd make
tests, but I plan on modifying the instruction set a little bit.
Also, correct the handling of scalars in dot and wedge products: it
turns out s.v and s^v both scale. However, it seems the CSE code loses
things sometimes.
This has shown the need for more instructions, such as a 2d wedge
product and narrower swizzles. Also, making dot product produce a vector
instead of a scalar was a big mistake (works nicely in C, but not so
well in Ruamoko).
The current code is pretty broken when it comes to vector types (losing
the vector and bogus errors among other issues). The whole thing needs a
rework or even just to be tossed in favor of better DAG processing.
I guess Hamish's suggestion made sense at the time, but I found that
with the current instructions, the reversed bivector wasn't so nice to
implement it would need a swizzle as well as the cross-product.
By default. Conversion of quake strings needs to be requested (which is
done by nq and qw clients and servers, as well as qfprogs via an
option). I got tired of seeing mangled source code in the disassembly.
I'm not sure if that was a thinko, typo, or something else, but judging
by the relevant commit message, the use of quaternion and vector was
intended only for advanced progs (v6p).
This makes working with them much easier, and the type system reflects
what's in the multi-vector. Unfortunately, that does mean that large
algebras will wind up having a LOT of types, but it allows for efficient
storage of sparse multi-vectors:
auto v = 4*(e1 + e032 + e123);
results in:
0005 0213 1:0008<00000008>4:void 0:0000<00000000>?:invalid
0:0044<00000044>4:void assign (<void>), v
0006 0213 1:000c<0000000c>4:void 0:0000<00000000>?:invalid
0:0048<00000048>4:void assign (<void>), {v + 4}
Where the two source vectors are:
44:1 0 .imm float:18e [4, 0, 0, 0]
48:1 0 .imm float:1aa [4, 0, 0, 4]
They just happen to be adjacent, but don't need to be.
Scaling now works for multi-vector expressions, and always subtracting
even when addition is wanted doesn't work too well. However, now there's
the problem of multi-vectors very quickly becoming full algebra vectors,
which means certain things need a rethink.
This gets only some very basics working:
* Algebra (multi-vector) types: eg @algebra(float(3,0,1)).
* Algebra scopes (using either the above or @algebra(TYPE_NAME) where
the above was used in a typedef.
* Basis blades (eg, e12) done via procedural symbols that evaluate to
suitable constants based on the basis group for the blade.
* Addition and subtraction of multi-vectors (only partially tested).
* Assignment of sub-algebra multi-vectors to full-algebra multi-vectors
(missing elements zeroed).
There's still much work to be done, but I thought it time to get
something into git.
If a symbol is not found in the table and a callback is provided, the
callback will be used to check for a valid procedural symbol before
moving on to the next table in the chain. This allows for both tight
scoping of the procedural symbols and caching.
Due to joys of pointers and the like, it's a bit of a bolt-on for now,
but it works nicely for basic math ops which is what I wanted, and the
code is generated from the expression.
Only · (dot product) and × (cross product for vector, commutator product
for geometric algebra) have been tested so far, but that involved
fighting with cpp to get it to not convert the · to \U000000b7, which
was rather annoying.
Fixing a load of issues related to autoconf and some small source-level issues to re-add clang support.
autoconf feature detection probably needs some addressing - partially as -Werror is applied late.
I never liked it, but with C2x coming out, it's best to handle bools
properly. I haven't gone through all the uses of int as bool (I'll leave
that for fixing when I encounter them), but this gets QF working with
both c2x (really, gnu2x because of raw strings).
The warning flag check worked too well: it enabled the warning and
autoconf's default main wanted the const attribute. The bug has been
floating around for a while, it seems.
This uses ud-chains for function statements (call/return) to force their
arguments to be live (in particular, indirect references via pointers)
this fixes the arraylife test.
The ud- and du-chains include known side-effects of the instructions and
thus depict a more accurate view of what operands an instruction uses or
defines. Fixes the arraylife2 test.
Like defs, a partial write should not define the whole temp. Thus, copy
the "don't visit main" behavior recently added to def_visit_all. Fixes
missing ud-chains for component-by-component assignments to temporary
vectors.
I'm not certain this is correct, but it seems to me that du-chains are
the same information as ud-chains, but from the defining statement's
point of view instead of that of the using statement.
As certain statements (in particular, function calls) can use additional
variables via pointer parameters, it's necessary to iterate ud-chain
building until the count stabilizes. This should make live variable
analysis much easier.
I think the current build_element_chain implementation does a reasonable
job, but I'm in the process of getting designated initializers working,
thus it will become important to ensure uninitialized members get
initialized.
I never liked the various hacks I had come up with for representing
resource handles in Ruamoko. Structs with an int were awkward to test,
pointers and ints could be modified, etc etc. The new @handle keyword (@
used to keep handle free for use) works just like struct, union and
enum in syntax, but creates an opaque type suitable for a 32-bit handle.
The backing type is a function so v6 progs can use it without (all the
necessary opcodes exist) and no modifications were needed for
type-checking in binary expressions, but only assignment and comparisons
are supported, and (of course) nil. Tested using cbuf_t and QFile: seems
to work as desired.
I had considered 64-bit handles, but really, if more than 4G resource
objects are needed, I'm not sure QF can handle the game. However, that
limit is per resource manager, not total.
This takes advantage of the ud-chains to follow the trail of pointer
assignments looking for an address. This gets array element assignments
surviving across blocks when the array itself is passed to a function.
It doesn't help when the address of the element is taken though. I think
that's a dags problem and probably needs du-chains. Also, the ud-chain
creation should probably be done in two passes so the newly found
information can be recorded.
Def and kill are still handled in flow_analyze_statement, but this makes
call meta data more consistent between v6 and ruamoko progs, allowing
the statement use chain to be used for call argument analysis. It even
found a bug in the extraction of param counts from the call instruction.
I had missed the flowvar clearing for auxiliary use/def/kill operands.
It's possible it wasn't necessary at the time since the operands were
added just for dealloc checking, but there's every reason it could
become necessary.
The first use will be pointer analysis for function arguments where the
argument points to an array to mark the array as live, but I'm sure
there'll be plenty of other uses.
A partial write to a def should not define the whole def, thus
def_visit_all's overlap parameter now has a flag that prevents a visit
to the main def when accessing the def from an alias def. This prevents
a lot of spurious kills and defines in flow analysis.
The array access code was loading the vector, modifying the element,
then forgetting to write the modified vector back to whence it came.
However, that would be rather sub-optimal, so now when the vector is
accessed by a pointer, the array code switches to field access to get at
the vector element thus avoiding the need to copy the whole vector.
Needed for proper analysis (ud-chains etc). Of course, it was then
necessary to remove the parameter defs from the uninitialized defs.
Also, plug a couple of memory leaks (forgot to free some temporary
sets).
That is, `array + offset`. This actually works around the bug
highlighted by arraylife.r (because the array is explicitly used), but
is not a proper solution, so that test still fails of course. However,
with this, it's no longer necessary to use `&array[index]` instead of
`array + index`.
I could never remember what any of the numbers meant. While define is
still a little fuzzy (they're (pseudo)statement numbers), at least now
I'll always know that the numbers are the define set. Also, having the
flow address of the variable helps with understanding the reaching defs
output.
I had messed up the handling of declarators for combinations of pointer,
function, and array: the pointer would get lost (and presumably arrays
of functions etc). I think I had gotten confused and thought things were
a tree rather than a simple list, but Holub set me straight once again
(I've never regretted getting that book). Once I understood that, it was
just a matter of finding all the places that needed to be fixed. Nicely,
most of the duplicated code has been refactored and should be easier to
debug in the future.
The type system rewrite had lost some of the checks for function fields.
This puts the actual code in the one place and covers parameters as well
as globals.
Internally, * is not really a valid operator for vectors since it can
have many meanings. This didn't cause trouble until trying to build
everything in game-source (since there's still a lot of legacy code in
there).
The precedence check changes done in
63795e790b seem to have been incorrect
(game-source/ctf produced many false positives), so putting that check
against '=' back into the code seems like a good idea (no more false
positives). That sounds a bit cargo-cult, but I'm really not sure what I
was thinking when I did the changes (probably just tired).
This applies only to the top-level scope of the function. I'm not sure
if it's right for traditional quakec code, but that can be adjusted
easily enough.
The symtab code itself cares only about global/not global for the size
of the hash table, but other code can use the symtab type for various
checks (eg, parameter shadowing).
Along with QuakeC's, of course. This fixes type typeredef2 test (a lot
of work for one little syntax error). Unfortunately, it came at the cost
of requiring `>>` in front of state expressions on C-style functions
(QuakeC-style functions are unaffected). Also, there are now two
shift/reduce conflicts with structs and unions (but these same conflicts
are in gcc 3.4).
This has highlighted the need for having the equivalent of the
expression tree for the declaration system as there are now several
hacks to deal with the separation of types and declarators. But that's a
job for another week.
The grammar constructs for declarations come from gcc 3.4's parser (I
think it's the last version of gcc that used bison. Also, 3.4 is still
GPL 2, so no chance of an issue there).
This simplifies type type_specifier rule significantly as now TYPE_SPEC
(was TYPE) includes all types and their basic modifiers (long, short,
signed, unsigned). This should allow me to make the type system closer
to gcc's (as of 3.4 as that seems to be the last version that used a
bison parser) and thus fix typeredef2.
Once a unicode char (ie, > 127) was used, any ascii chars would get the
tail of the last unicode char resulting in broken utf-8 streams. The
resulting null glyph boxes were not very appealing.
This fixes the basic vecconst test (extending it to other types breaks
because long and ulong are not properly supported yet). The conversion
is done by the progs VM rather than writing another 256 conversions
(though loops could be used). This works nicely as a test for using the
VM to help with compiling.
While the option to make '*' mean dot product for vectors is important,
it breaks vector scaling in ruamoko progs as the resultant vector op
becomes a dot product instead of the indented hadamard product (ie,
component-wise).
The support for the new vector types broke compiling code using
--advanced. Thus it's necessary to ensure vector constants are
float-type and vec3 and vec4 are treated as vector and quaternion, which
meant resurrecting the old vector expression code for v6p progs.
The method is still held by known_methods, so freeing it causes grief.
However, it may cause a leak thus the free is only commented out. More
investigation is needed. I'm surprised the problem didn't show on linux,
but cygwin-native hit it and valgrind on linux found the spot :)
While it does get a bit cluttered currently, being able to see the
contents of structures makes a huge difference. Also highlights that
vector immediates do not get the correct type encodings.
This fixes the internal error generated by the likes of
`(sv_gravity * '0 0 1')` where sv_gravity is a float and `'0 0 1'` is an
ivec3: the vector is promoted to vec3 first so that expanding sv_gravity
is expanded to vec3 instead of ivec3 (which is not permitted for a
float: expansion requires the destination base type to be the same as
the source).
For now, anyway, as the generated code looks good. There might be
problems with actual pointer expressions, but it allows entity.field to
work as expected rather than generate an ICE.
The resultant unicode is encoded as utf-8, which does conflict with the
quake character map, but right now unicode is useful only with font
text, and those support only standard unicode (currently only as utf-8),
but something will need to be sorted out.
Arrays are passed as a pointer to the first element, so are always valid
parameters. Fixes a bogus "formal parameter N is too large to be passed
by value" error.
While swizzle does work, it requires the source to be properly aligned
and thus is not really the best choice. The extend instruction has no
alignment requirements (at all) and thus is much better suited to
converting a scalar to a vector type.
Fixes#30
It seems clang loses track of the usage of the referenced unions by the
time the code leaves the switch. Due to the misoptimization, "random"
values would get into the vector constants. This puts the usages in the
same blocks as the unions, causing clang to "get it right" (though I
strongly suspect I was running into UB).
While I might need to tighten up the rules later, this allows binary
operations between vector (the type) and explicitly typed vec3 constants
(and non-constants, about which I am undecided). The idea is that
explicit constants such as '1 2 3'f should be compatible with either
type.
This applies to quaternions as well.
As a class's ivars are built up by inheritance, but with only that
class's ivars in the symbol table, is is necessary to include an offset
based on the super class's ivars in order to ensure alignments are
respected. This is achieved via the new `base` parameter to
build_struct(), which is used to offset the current size while
calculating the aligned offset of the symbols. The parameter is ignored
for unions, as they always start at 0. The ivars for the current class
still have a base offset of 0 until they are actually added to the
class.
Fixes#29
The alignment is specified as a power of 2 (ie, actual alignment = 1 <<
alignment) allowing old object files to be compatible (as their
alignment is 0). This is necessary for (in part for #30) as it turned
out even global vectors were not aligned correctly.
Currently, only data spaces even vaguely respect alignment. This may
need to be fixed in the future.
Most were pretty easy and fairly logical, but gib's regex was a bit of a
pain until I figured out the real problem was the conditional
assignments.
However, libs/gamecode/test/test-conv4 fails when optimizing due to gcc
using vcvttps2dq (which is nice, actually) for vector forms, but not the
single equivalent other times. I haven't decided what to do with the
test (I might abandon it as it does seem to be UD).
Defs and symbols benefit from swizzling as that's one instruction vs 2-3
for loading a scalar into a vector component by component. Constants are
ok because the result gets converted to a vector constant.
The destination operand must be a full four component vector, but the
source can be smaller and small sources do not need to be aligned: the
offset of the source operand and the swizzle indices are adjusted. The
adjustments are done during final statement emission in order to avoid
confusing the data flow analyser (and that's when def offsets are known).
This reverts commit 2904c619c1.
In order to support swizzle operations, I need to be able to alias defs
to larger types (eg, float to vec4), but alias_def rightly won't allow
this. However, as the plan is to do this in the final steps before
emitting the instruction, I plan on creating an alias to a float then
adjusting the type in the alias, but to do so without extra shenanigans,
I need alias_def to allow aliases to the same type. As a fringe benefit,
it makes the code agree with the comment in def.h :P
This came up when investigating an internal error from the line above.
It turned out the error was correct (problem with converting scalars to
vectors), but the break was not.
Currently, only vector/vec3 and quaternion/vec4 can be printed anyway,
but I plan on making explicit format strings for the types, so there
should be no need to promote any vector types (and really, any hidden
promotion is a bit of a pain, but standards...).
While the code would handle int vector types, there aren't any such
instructions, and the expression code shouldn't generate them, but all
float (32 and 64 bit) vector types do have a dot product instruction, so
check width rather than just vector/quaternion.
This fixes an error that's been lurking for over two years (since I made
parameters unlimited internally). The problem was the array was being
allocated on the stack and a simple struct copy was used to store type
type, resulting in a dangling pointer onto the stack. I'm surprised it
didn't cause more problems.
This allows all the tests to build and pass. I'll need to add tests to
ensure warnings happen when they should and that all vec operations are
correct (ouch, that'll be a lot of work), but vectors and quaternions
are working again.
With this, all vector widths and types are supported: 2, 3, 4 and int,
uint, long, ulong, float and double, along with support for suffixes to
make the type explicit: '1 2'd specifies a dvec2 constant, while '1 2 3'u
is a uivec3 constant. Default types are double (dvec2, dvec3, dvec4) for
literals with float-type components, and int (ivec2...) for those with
integer-type components.
Having three very similar sets of code for outputting values (just for
debug purposes even) got to be a tad annoying. Now there's only one, and
in the right place, too (with the other value code).
I'd created new_value_expr some time ago, but never used it...
Also, replace convert_* with cast_expr to the appropriate type (removes
a pile of value check and create code).
Use with quaternions and vectors is a little broken in that
vec4/quaternion and vec3/vector are not the same types (by design) and
thus a cast is needed (not what I want, though). However, creating
vectors (that happen to be int due to int constants) does seem to be
working nicely otherwise.
Nicely, I was able to reuse the generated conversion code used by the
progs engine to do the work in qfcc, just needed appropriate definitions
for the operand macros, and to set up the conversion code. Helped
greatly by the new value load/store functions.
pr_type_t now contains only the one "value" field, and all the access
macros now use their PACKED variant for base access, making access to
larger types more consistent with the smaller types.
This is an extremely extensive patch as it hits every cvar, and every
usage of the cvars. Cvars no longer store the value they control,
instead, they use a cexpr value object to reference the value and
specify the value's type (currently, a null type is used for strings).
Non-string cvars are passed through cexpr, allowing expressions in the
cvars' settings. Also, cvars have returned to an enhanced version of the
original (id quake) registration scheme.
As a minor benefit, relevant code having direct access to the
cvar-controlled variables is probably a slight optimization as it
removed a pointer dereference, and the variables can be located for data
locality.
The static cvar descriptors are made private as an additional safety
layer, though there's nothing stopping external modification via
Cvar_FindVar (which is needed for adding listeners).
While not used yet (partly due to working out the design), cvars can
have a validation function.
Registering a cvar allows a primary listener (and its data) to be
specified: it will always be called first when the cvar is modified. The
combination of proper listeners and direct access to the controlled
variable greatly simplifies the more complex cvar interactions as much
less null checking is required, and there's no need for one cvar's
callback to call another's.
nq-x11 is known to work at least well enough for the demos. More testing
will come.
The "not" because I'm pretty sure they're false positives due to when
the function is called, but clang doesn't know that (wonder why gcc was
ok with it).
It seems clang defaults to unsigned for enums. Interestingly, gcc was ok
with the checks being either way. I guess gcc treats enums that *can* be
unsigned as DWIM.
In working with vectors and matrices while testing the scene wrappers, I
found that there was a fair bit of confusion about how large something
could be. Return values can be up to 32 words (but qfcc wasn't aware of
that), parameters were limited to 4 words still (and possibly should be
for varargs), and temp defs were limited to 8 words (1 lvec4). Temps are
used for handling return values (at least when not optimizing) and thus
must be capable of holding a return value, and passing large arguments
through *formal* parameters should be allowed. It seems reasonable to
limit parameter sizes to return value sizes.
A temp and a move are still used for large return values (4x4 matrix),
but that's an optimization issue: the code itself is at least correct.
When the def can be found. This fixes direct assignments to arrays (and
probably structs) getting lost when the array is later read using a
variable index.
Float is not int, and Ruamoko has only int ifz/ifnz, which will fail for
-0.0 (0x80000000 when viewed as an int). And then there's nan, but I
haven't seen too many of those in quake.
I suspect this is an ancient bug that wasn't noticed due to not looking
at progs.src compiled code enough, but it makes the first statements of
the function point to the correct line instead of a forward declaration.
Currently only via pragma (not command line options), but I needed to
test the concept. Converting legacy code is just too error prone.
Telling the compiler how to treat the operator makes more sense. When *
acts as @dot with Ruamoko progs, the result is automatically aliased as
a float as this is the legacy meaning (ie, float result for dot
product).
This is a very tiny optimization, but there's no point in adjust the
stack if there's no actual adjustment. I didn't bother with it initially
because I thought it wouldn't happen (and I was more interested in
getting things working first), but it turns out that simple getters that
result in a zero adjustment are quite common (70/535 in qwaq-app.dat).
This is achieved by marking a void function with the void_return
attribute and then calling that function in an @return expression.
@return can be used only inside a void function and only with void
functions marked with the void_return attribute. As this is intended for
Objective-QC message forwarding, it is deliberately "difficult" to use
as returning a larger than expected value is unlikely to end well for
the calling function.
However, as a convenience, "@return nil" is allowed (in a void
function). It always returns an integer (which, of course,can be
interpreted as a pointer). This is safe because if the return value is
ignored, it will go into the progs return buffer, and if it is not
ignored, it is the smallest value that can be returned.
Having to remember to copy yet another specifier bit was getting
tedious, so use a union of a struct with the bitfields and an unsigned
int to access them in parallel. Makes for a tidier spec_merge, and one
less headache.
The command line option works the same way as
--advanced/traditional/extended, as does the pragma. As well, raumoko
(alternative spelling) can be used because both are legitimate and some
people may prefer one spelling over the other.
As always, use of the pragma is at one's own risk: its intended use is
forcing the target in the unit tests.
dvec4, lvec4 and ulvec4 need to be aligned to 8 words (32 bytes) in
order to avoid hardware exceptions. Rather than dealing with possibly
mixed alignment when a function has 8-word aligned locals but only
4-word aligned parameters, simply keep the stack frame 8-word aligned at
all times.
As for sizes, the temp def recycler was written before the Ruamoko ISA
was even a pipe dream and thus never expected temp def sizes over 4. At
least now any future adjustments can be done in one place.
My quick and dirty test program works :)
dvec4 xy = {1d, 2d, 0d, 0.5};
void printf(string fmt, ...) = #0;
int main()
{
dvec4 u = {3, 4, 3.14};
dvec4 v = {3, 4, 0, 1};
dvec4 w = v * xy + u;
printf ("[%g, %g, %g, %g]\n", w[0], w[1], w[2], w[3]);
return 0;
}
They're now properly part of the type system and can be used for
declaring variables, initialized (using {} block initializers), operated
on (=, *, + tested) though much work needs to be done on binary
expressions, and indexed. So far, only ivec2 has been tested.
When possible, of course. However, this tightens up struct and constant
index array accesses, and avoids issues with flow analysis losing track
of the def (such trucking is something I want to do, but haven't decided
out to get the information out to the right statements).
Since address expressions always product a pointer type, aliasing one to
another pointer type is redundant. Instead, simply return an address
expression with the desired type.
The FIXME was there because I couldn't remember why the test was
type_compatible but the internal error complains about the types being
the same size. The compatibility check is to see if the op can be used
directly or whether a temp is required. The offset check is because
types that are the same size (which they must be if they are
compatible) is because it is not possible to create an offset alias def
that escapes the bounds of the real def, which any non-zero offset will
do if the types are the same size.
This is the intended purpose of the offset field in address expressions,
and will make struct and array accesses more efficient when I sort out
the code generation side.
Ruamoko passes va_list (@args) through the ... parameter (as such), but
IMP uses ... to defeat parameter type and count checking and doesn't
want va_list. While possibly not the best solution, adding a no_va_list
flag to function types and skipping ex_args entirely does take care of
the problem without hard-coding anything specific to IMP.
The system currently just sets some bits in the type specifier (the
attribute list should probably be carried around with the specifier),
but it gets the job done for now, and at least gets things started.
This makes it much easier to check (and more robust to name changes),
allowing for effectively killing the node to which the variable being
addressed is attached. This fixes the incorrect address being used for
va_list, which is what caused double-alias to fail.
In order to not waste instructions, the Ruamoko ISA does not provide 1
and 2 component 64-bit load/store instructions since they can be
implemented using 2 and 4 component 32-bit instructions (load and store
are independent of the interpretation of the data). This fixes the
double test, and technically the double-alias test, but it fails due to
a problem with the optimizer causing lea to use the wrong reference for
the address. It also breaks the quaternion test due to what seems to be
a type error that may have been lurking for a while, further
investigation is needed there.
Since the call instruction in the Ruamoko ISA specifies the destination
of the return value of the called function, it is much like any
expression type instruction in that the def referenced by its c operand
is both defined and killed by the instruction. However, unlike other
instructions, it really has many pseudo-operands: the arguments placed
on the stack. The problem is that when one of the arguments is also the
destination of the return value, the dags code wants to use the stack
argument as it was the last use of the real argument. Thus, instead of
using the value of the child node for the result, use the value label
attached to the call node (there should be only one such label).
This fixes iterfunc, typedef, zerolinker and vkgen when optimizing. Now
all but the double tests and return postop tests pass (and the retun
postop test is not related to the Ruamoko ISA, so fails either way).
I really need to come up with a better way to get the result type into
the flow analyser. However, this fixes the aliasing ICE when optimizing
Ruamoko code that uses struct assignment.
