It turns out the sorting wasn't working properly and I've decided that
anything that actually needs the defs to be sorted by address (such as a
debugger searching for defs by address) can do the sorting itself. Fixes
a weird swapping of def names.
This is necessary to get statement disassembly working, and likely
debugging in general. locals is the total size of the stack frame and
thus reaches above the function-entry stack pointer, and params_start is
the local space relative start of the parameters. Thus, knowing the
function-entry stack pointer, the bottom of the locals space can be
found by subtracting params_start, and the top of the locals space by
adding (locals - params_start).
This gets all the sections of the progs file nicely aligned and the code
easier to read with the offset and size calculations not being spread
through the function. ivar-struct-return now works when compiled for
Ruamoko.
This cleans up dprograms_t, making it easier to read and see what chunks
are in it (I was surprised to see only 6, the explicit pairs made it
seem to have more).
While I think the reason the dags code moved an instruction before
adjstk and with was they shared a constant with that instruction (which
is a different bug), this ensures other instructions cannot get
reordered in front of adjstk and with, as doing so would cause any such
instructions to access incorrect data.
The goal was to get lea being used for locals in ruamoko progs because
lea takes the base registers into account while the constant pointer
defs used by v6p cannot. Pointer defs are still used for gobals as they
may be out of reach of 16-bit addressing.
address_expr() has been simplified in that it no longer takes an offset:
the vast majority of the callers never passed one, and the few that did
have been reworked to use other mechanisms. In particular,
offset_pointer_expr does the manipulations needed to add an offset
(unscaled by type size) to a pointer. High-level pointer offsets still
apply a scale, though.
Alias expressions now do a better job of hanling aliasing of aliases by
simply replacing the target type when possible.
It's possible I lost the child printing when creating the return
expressions, but dot diagrams are much more useful when they don't have
nodes with just pointer values.
The parameter defs are allocated from the parameter space using a
minimum alignment of 4, and varargs functions get a va_list struct in
place of the ...
An "args" expression is unconditionally injected into the call arguments
list at the place where ... is in the list, with arguments passed
through ... coming after the ...
Arguments get through to functions now, but there's problems with taking
the address of local variables: currently done using constant pointer
defs, which can't work for the base register addressing used in Ruamoko
progs.
With the update to test-bi's printf (and a hack to qfcc for lea),
triangle.r actually works, printing the expected results (but -1 instead
of 1 for equality, though that too is actually expected). qfcc will take
a bit longer because it seems there are some design issues in address
expressions (ambiguity, and a few other things) that have pretty much
always been there.
The aux use ops need to be counted and given nodes explicitly as they
may refer to defs that are not accessed by other statements other than
by aliases, and those aliases need to be marked live as well as the used
def.
This is part of the work for #26 (Record resource pointer with builtin
function data). Currently, the data pointer gets as far as the
per-instance VM function table (I don't feel like tackling the job of
converting all the builtin functions tonight). All the builtin modules
that register a resources data block pass that block on to
PR_RegisterBuiltins.
This will make it possible for the engine to set up their parameter
pointers when running Ruamoko progs. At this stage, it doesn't matter
*too* much, except for varargs functions, because no builtin yet takes
anything larger than a float quaternion, but it will be critical when
double or long vec3 and vec4 values are passed.
Storing a variable into a dereference pointer (*p = x) is not marking
the variable as used (due to a mistake while converting to Ruamoko
statement format) resulting in assignments to that variable being
dropped due to it being a dead assignment as the assignment to the
variable and the storing need to be in separate basic blocks (thus the
call in the test, though an if would have worked, I think) for the bug
to trigger.
The problem was a missed change when switching the internal statement
format to Ruamoko: I "used" the statement's operands directly rather
than the rotated ones when emitting v6p progs. Fixes a compile segfault
when NOT optimizing.
There was an out-by-one where attempting to run a program with only one
argument would result in the argument not being passed to the program
(two worked). This is actually the source of the error fixed in
9347e4f901 because test-harness.c was the
basis for qwaq's main.c
While all base registers can be used for any purpose at any time (this
is why the with instruction has hard-absolute modes: you can never get
permanently lost), qfcc currently uses the convention of register 0 for
globals and register 1 for stack locals (params, locals, function args).
The register used to access a def is stored in the def and that is used
to set the register bits in the instruction opcode.
The def code actually doesn't know anything about any conventions: it
assumes all defs are global for non-temp defs (the function code updates
the defs before emitting code) and the current function provides the
register to use for any temp defs allocated while emitting code.
Seems to work well, but debug is utterly messed up (not surprised, that
will be tricky).
Still need to get the base register index into the instructions, but I
think this is it for basic code generation. I should be able to start
testing Ruamoko properly fairly soon :)
Thanks to the use/def/kill lists attached to statements for pseudo-ops,
it turned out to be a lot easier to implement flow analysis (and thus
dags processing) than I expected. I suspect I should go back and make
the old call code use them too, and probably several other places, as
that will greatly simplify the edge setting.
The means that the actual call expression is not in the statement lint
of the enclosing block expression, but just its result, whether the call
is void or not. This actually simplifies several things, but most
importantly will make Ruamoko calls easier to implement.
The test is because I had some trouble with double-calls, and is how I
found the return-postop issue :P
Commit 76b3bedb72 broke more than just the
swap test, but at least I know I need to get an edge in the dag.
Currently, the following code is generated: return and add are reversed.
../tools/qfcc/test/return-postop.r:8: return counter++;
0001 store.i counter, .tmp0
0002 return .tmp0
0003 add.i .tmp0, (1), counter
However, I don't want to deal with it right now, so it's marked XFAIL.
Since Ruamoko now uses the stack for parameters and locals, parameters
need to come after locals in the address space (instead of before, as in
v6 progs). Thus use separate spaces for parameters and locals regardless
of the target, then stitch them together appropriately for the target.
The third space is used for allocating stack space for arguments to
called functions. It us not used for v6 progs, and comes before locals
in Ruamoko progs.
Other than the return value, and optimization (ice, not implemented)
calls in Ruamoko look like they'll work.
Thanks to me having done something right 20 years ago, that was pretty
easy :). The two boolean types aren't supported yet because I haven't
decided on just how to represent their types in qfcc.
This seems to be the most reasonable approach to allocating space for
function call parameters without using push and pop (or adding to the
stack pointer), though it's probably good even when using push and pop
to help keep things aligned.
My little test program now builds with the Ruamoko ISA :)
void cp (int *dst, int *src, int count)
{
while (count--) {
*dst++ = *src++;
}
}
Calls are broken (unimplemented), and non-void returns are not likely to
work either (only partially implemented).
Operand width is encoded in the instruction opcode, so the width needs
to be accounted for in order to select the correct instruction. With
this, my little test generates correct code for the ruamoko ISA (except
for return, still fails).
For the most part, it wasn't too bad as it's just a rotation of the
operands for some instructions (store, assign, branch), but dealing with
all the direct accesses to specific operands was a small pain. I am very
glad I made all those automated tests :)
This makes the v6p instruction table consistent with the ruamoko
instruction table, and clears up some of the ugliness with the load,
store, and assign instructions (. .= and = are now spelled out). I think
I'd still prefer an enum code (faster) but at least this is more
readable.
Missed this case in duplicate_type. Allows "short foo" and
"sizeof(short)" (even though qfcc and the engine have two ideas of the
size: I expect trouble later).
long is ignored for double, and v6p progs are stuck with 32 bits for
longs (don't feel like extending v6p any further), but the basics are
there for Ruamoko.
short is ignored for ints because the minimum size is 32, and signed is
just noise for ints anyway (and no chars, so...).
unsigned, however, is finally implemented properly (or at least seems to
be working correctly: tests pass after getting things compiling again,
and lt.u is used where it should be :)
Attempting to add ev_ushort caused ptraliasenc to break, but that was
because it was already broken: I had implemented the scan of the xdef
table incorrectly, thus adding only 1 ev type resulted in the walked
pointer being out of phase with its data due to it first passing over
the type encodings (which is why adding long and ulong didn't cause any
obvious trouble).
And other related fields so integer is now int (and uinteger is uint). I
really don't know why I went with integer in the first place, but this
will make using macros easier for dealing with types.
They are both gone, and pr_pointer_t is now pr_ptr_t (pointer may be a
little clearer than ptr, but ptr is consistent with things like intptr,
and keeps the type name short).
I don't know why they were ever signed (oversight at id and just
propagated?). Anyway, this resulted in "unsigned" spreading a bit, but
all to reasonable places.
This includes calls and unconditional jumps, relative and through a
table. The parameters are all lumped into the one object, with some
being unused by the different types (eg, args and ret_type used only by
call expressions). Just having nice names for the parameters (instead of
e1 and e2) makes it nice, even with all the sub-types lumped together.
No mysterious type aliasing bugs this time ;)
The move operator names are definitely obsolete (due to dropping the
expressions a year or two ago) and the precedence checks seem to be
handled elsewhere. Memset and state expressions went away a while back
too.
While this was a pain to get working, that pain only went to prove the
value of using proper "types" (even if only an enum) for different
expression types: just finding all the places to edit was a chore, and
easy to make mistakes (forgetting bits here and there).
Strangely enough, this exposed a pile of *type* aliasing bugs (next
commit).
v6 vs v6p are more or less as before, with ruamoko added in. qfcc will
now try (and fail, due to the opcode table opnames being wrong) to
create ruamoko progs when given the ruamoko target option.
At this stage, I doubt emit.c will need to know the details of the
target (v6, v6p, ruamoko) since the instruction formats are identical,
just different meanings for the opcode itself.
This allows v6, v6p (older QF VM) or ruamoko (new QF VM) to be targeted.
Currently defaults to v6p to allow QF to continue building without too
much hassle.
While qfcc dealing sensibly with mixed target VMs in the object files
has always been an outstanding issue, with the new instruction set it
has become a priority. Most importantly, this should allow QF to
continue building while I work on qfcc targeting the new IS.
It does little good for documentation to refer to fields that don't
exist (because a certain someone forgot to change the docs when changing
the field names, I wonder who :P).
And partial implementations in qfcc (most places will generate an
internal error (not implemented) or segfault, but some low-hanging fruit
has already been implemented).
This allows the VM to select the right execution loop and qfcc currently
still produces only the old IS (it doesn't know how to deal with the new
IS yet)
build_struct was unconditionally setting the type's alignment. This was
not a problem before because no types were requesting alignments larger
than those requested by their members (for structs). However, with the
upcoming new instruction set, quaternions need to be 4-word aligned.
The opcode table is a nightmare to maintain, but this does clean it up
and speed up opcode lookups since they can now be indexed. Of course, it
turns out I had missed adding several instructions, so had to fix that,
and qfcc needed a bit of a re-jigger to get the opcode out of the table.
The assignment to the node's variable must come after any uses of that
node, which the node's parent set indicates. In the swap test, this was
not a problem as the node had no parents, and in the link order test, it
just happened(?) to work.
While using just the label node's reachable set was sufficient for a
simple swap (t = a; a = b; b = t;), it is not sufficient for
read-before-write dependencies such as found in linked-list building:
{ o = array[ind]; o.next = obj; obj = o; }
The assignment to o.next uses obj, but that use is hidden because obj's
reachable nodes does not include o thus assigning o to obj causes the
array dereference to be assigned directly to obj and thus o.next winds
up pointing to o instead of whatever obj was. The parent nodes of obj's
node are its users, so any new assigned to obj must come after those
parents as well as any node reachable by obj's node.
Fixes a runaway loop error when adding a frikbot to the server.
qfo_to_progs was modifying the space data pointers in the input qfo,
making it impossible to reuse the qfo. However, qfo_relocate_refs needs
the updated pointers, thus do a shallow copy of the qfo and its spaces
(but not any of the data)
build_builtin_function does the right thing, and it was only legacy
syntax functions that were affected anyway. Certainly, external
variables should not be initialized, but klik uses @extern { } wrapped
around several builtin functions and I had added the feature to allow
just this as it is rather convenient.
I decided that the check for whether control reaches the end of the
function without performing some necessary action (eg, invoking
[super dealoc] in a derived -dealoc) is conceptually the return
statement using a pseudo operand and the necessary action defining that
pseudo operand and thus is the same as checking for uninitialised
variables. Thus, add a pseudo operand type and use one to represent the
invocation of [super alloc], with a special function to call when the
"used" pseudo operand is "uninitialised".
While I currently don't know what else pseudo operands could be used
for, the system should be flexible enough to add any check.
Fixes#24
I want to use the function's pseudo address that was used for managing
aliased temporary variables for other pseudo operands as well. The new
name seems to better reflect the variable's purpose even without the
other pseudo operands as temporary variables are, effectively, pseudo
operands until they are properly allocated.
Forgetting to invoke [super dealloc] in a derived class's -dealloc
method has caused me to waste far too much time chasing down the
resulting memory leaks and crashes. This is actually the main focus of
issue #24, but I want to take care of multiple paths before I consider
the issue to be done.
However, as a bonus, four cases were found :)
While get_selector does the job of getting a selector from a selector
reference expression, I have long considered lumping various expression
types under ex_expr to be a mistake. Not only is this a step towards
sorting that out, it will make working on #24 easier.
I have gotten tired of chasing memory leaks caused by me forgetting to
add [super dealloc] to my dealloc methods, so getting qfcc to chew me
out when I do seems to be a good idea (having such a warning would have
saved me many hours, just as missing return warnings have).
After seeing set_size and thinking it redundant (thought it returned the
capacity of the set until I checked), I realized set_count would be a
much better name (set_count (node->successors) in qfcc does make much
more sense).
When moving an identifier label from one node to another, the first node
must be evaluated before the second node, which the edge guarantees.
However, code for swapping two variables
t = a; a = b; b = t;
creates a dependency cycle. The solution is to create a new leaf node
for the source operand of the assignment. This fixes the swap.r test
without pessimizing postop code.
This takes care of the core problem in #3, but there is still room for
improvement in that the load/store can be combined into a move.
This reverts commit 2fcda44ab0.
Killing the node is not the correcgt answer as it blocks many
optimization opportunities. The correct answer is adding edges to
describe the temporal dependencies. Of course, this breaks the swap.r
test.
In order to correctly handle swap-style code
{ t = a; a = b; b = t; }
edges need to be created for each of the assignments moving an
identifier lable, but the dag must remain acyclic (the above example
wants to create a cycle). Having the reachable nodes recorded makes
checking for potential loops a quick operation.
Identifiers can be constants. I don't remember quite what it fixed other
than some bogus kill relations in the dags (which might have caused
issues later).
If the src type is not a class, there is no inheritance chain to walk.
Fixes a segfault when returning self after a syntax error in the
following:
+(EditStatus *)withRect:(Rect)rect
{
return [[[self alloc] initWithRect:rect]:
}
-setCursorMode:(CursorMode)mode
{
cursorMode = mode;
return self;
}
The server edict arrays are now stored outside of progs memory, only the
entity data itself (ie data accessible to progs via ent.fld) is stored in
progs memory. Many of the changes were due to code accessing edicts and
entity fields directly rather than through the provided macros.
It now takes a context pointer (opaque data) that holds the buffers it
uses for the temporary strings. If the context pointer is null, a static
context is used (making those uses of va NOT thread-safe). Most calls to
va use the static context, but all such calls have been formatted
consistently so they are easy to find when it comes time to do a full
audit.
Block expressions hide ex_error, but get_type() always returns null when
it finds one (which it does by recursing into block expression), so just
check the type itself.
When a global variable is accessed via only an alias in a function the
actual def's flowvar would remain in the state it was from the last
function that accessed the global normally. This would result in invalid
flowvar accesses which can be difficult to reproduce (thus no test
case).
There's still some cleanup to do, but everything seems to be working
nicely: `make -j` works, `make distcheck` passes. There is probably
plenty of bitrot in the package directories (RPM, debian), though.
The vc project files have been removed since those versions are way out
of date and quakeforge is pretty much dependent on gcc now anyway.
Most of the old Makefile.am files are now Makemodule.am. This should
allow for new Makefile.am files that allow local building (to be added
on an as-needed bases). The current remaining Makefile.am files are for
standalone sub-projects.a
The installable bins are currently built in the top-level build
directory. This may change if the clutter gets to be too much.
While this does make a noticeable difference in build times, the main
reason for the switch was to take care of the growing dependency issues:
now it's possible to build tools for code generation (eg, using qfcc and
ruamoko programs for code-gen).
This should keep things nicely extensible, since additional data can be
done in the data space and found using defs. This gets the compilation
units into the sym file.
They worked well if there was only one source file in the test, but
failed if there were two or more. While only typelinker needed the
enhanced macros, I got them all because I generally copy the nearest
block when adding a new test thus it's best if they're all "correct".
The compilation unit stores the directory from which qfcc was run and
any source files mentioned. This is similar to dwarf's compilation unit.
Right now, this is the only data in the new debug space, but more might
come in the future so it seems best to treat the debug space separately
in the object files.
getcwd is assumed to use malloc if its buff param is null. This may need
fixing in the future, but it's in one spot. The result in "saved" in the
non-progs pool.
It never really helped sort out the path issues when using build
directories. It worked well enough for single directory projects, but
things got messy very quickly, especially when mixing ruamoko libs with
external progs. A better method based on dwarf is coming.
Killed nodes can leave stray (dangling) edges that cause some confusion
in the dot graphs and may cause problems later on down the track, so
ensure there are no dangling edges.
The reason double-alias fails is when the double assignment occurs, the
int operands don't yet have leaf nodes and thus the nodes cannot be
killed. This doesn't fix the bug.
I'm not sure if it's due more to doubles or unions, but the bug was
found via double. It seems the dags code generator doesn't see that the
assignment to the union's double field kills the two int fields.
The test passes when NOT optimizing.
Because type aliases need to be unaliased, the type pointers in the type
encodings need to be correct when it comes to linking defs and
functions. This fixes the linking errors in ruamoko/game.
I was very uncertain about the validity of messing with the old type
encoding that way, but adding the check to ensure the type has been
processed never fired, so it seems ok. And the comments help me a lot :)
When aliasing a type that already has aliases, the top node needs to be
replaced if it is unnamed, or the alias-free branch of the new node
needs to reach around to the alias-free branch of the existing node.
This fixes the bogus param counts in qwaq.
This fixes the typelinker test, but not the linking error in
ruamoko/game that it was supposed to represent. I guess there's
something more going on (maybe type encoding relocation issues).
Fixes#6
It turned out that the problem with @zero was caused by initial type
chaining occurring before the structures had been initialized and thus
the linker's @zero type encoding string was incorrect: {?=} instead of
{tag @zero-}, thus when the actual type encoding supplied by an object
file came along (with the correct encoding string), it wasn't found.
It is now "consistent" with the rest of the type building in that it
uses find_type(append_type(return, params)) like the C version, thus
allowing append_type to do its thing with type aliases. This fixes the
overload test.
This is a bit of a weird one because it's a combination of the aliasing
code and mixing C prototypes with QuakeC function definitions, and the
function type rebuilding in qc-parse.y not being very "consistent" in
its abuse of the type system.
The full_type branch of an alias splitter (alias with null name) needs
to mirror the clean side up to the type alias. It is causing problems
with functions, but that's expected because parameters complicate
things.
It's not connected up yet because I'm unsure of just where to put things
(it gets messy fast), but just being able to see the structure of
complex types is nice.
This eases type unaliasing on functions a little.
Still more to to go, but this fixes a really hair-pulling bug: linux's
heap randomiser was making the typedef test fail randomly whenever
typedef.qfo was compiled.
When a type is aliased, the alias has two type chains: the simple type
chain with all other aliases stripped, and the full type chain. There
are still plenty of bugs in it, but having the clean type chain takes
care of the major issue that was in the previous attempt as only the
head of the type-chain needs to be skipped for type comparison.
Most of the bugs are in finding the locations where the head needs to be
skipped.
All simple type checks are now done using is_* helper functions. This
will help hide the implementation details of the type system from the
rest of the compiler (especially the changes needed for type aliasing).
They take a pointer to a free-list used for hashlinks so the hashlink
pools can be per-thread. However, hash tables that are not updated are
always thread-safe, so this affects only updates. progs_t has been set
up such that it is easy for multiple progs within one thread can share
hashlinks.
and its usage. The parts of flow_analyze_statement that use it know
where the returned operand needs to go. Unfortunately, this breaks dags
pretty hard, but that's because dags needs to learn about the fancy
assignment-type statements.
I noticed that pointer math is currently incorrect in qfcc, but it would
be nice for fixing it to not break anonstruct since it is testing
something else.
This fixes the technically correct but horrible mess of temps and
addressing when dealing with ivars, and the resulting uninitialized
temps due to the non-constant pointers (do need statement level constant
folding, though).
This is part of what messed up float_val in the encoding for @params.
The other part is something in the linker type encoding merge code: it
may be too aggressive. It's also what messed up the size of @params.
That is, those created by operand_address. The dag code needs the
expression that is attached to the statement to have the correct
expression type in order to do the right thing with the operands and
aliasing, especially when generating temps. This fixes assignchain when
optimizing (all tests pass again).
This reverts commit c78d15b331.
While a block expression's result may be an l-value, block expressions
are not (and their results may not be), thus taking the address of one
is not really correct. It seems the only place that tries to do so is
the assignment code when dealing with structures.
This reverts commit b49d90e769.
I suspect this was a workaround for the mess in assignment chains.
However, it caused compile errors with the new implementation, and is
just bogus anyway.
While I still hate ".=", at least it's more hidden, and the new
implementation is a fair bit cleaner (hah, goto a label in an if (0) {}
block).
Most importantly, the expression tree code knows nothing about it. Now
just to figure out what broke func-epxr. A bit of whack-a-mole, but yay
for automated tests.
Doing it in the expression trees was a big mistake for a several
reasons. For one, expression trees are meant to be target-agnostic, so
they're the wrong place for selecting instruction types. Also, the move
and memset expressions broke "a = b = c;" type expression chains.
This fixes most things (including the assignchain test) with -Werror
turned off (some issues in flow analysis uncovered by the nil
migration: memset target not extracted).
Now convert_nil only assigns the nil expression a type, and nil makes
its way down to the statement emission code (where it belongs, really).
Breaks even more things :)
This bug drove me nuts for several hours until I figured out what was
going on.
The assignment sub-tree is being generated, then lost. It works for
simple assignments because a = b = c -> (= a (= b c)), but for complex
assignments (those that require move or memset), a = b = c -> (b = c) (a
= c) but nothing points to (b = c). The cause is using binary
expressions to store assignments.
It's not possible to take the address of constants (at this stage) and
trying to use a move instruction with .zero as source would result in
the VM complaining about null pointer access when bounds checking is on.
Thus, don't convert a nil source expression until it is known to be
safe, and use memset when it is not.
This fixes the problem of using the return value of a function as an
element in a compound initializer. The cause of the problem is that
compound initializers were represented by block expressions, but
function calls are contained within block expressions, so def
initialization saw the block expression and thought it was a nested
compound initializer.
Technically, it was a bug in the nested element parsing code in that it
wasn't checking the result value of the block expression, but using a
whole new expression type makes things much cleaner and the work done
paves the way for labeled initializers and compound assignments.
Not that it really makes any difference for labels since they're
guaranteed unique, but it does remove the question of "why nva instead
of save_string?". Looking at history, save_string came after I changed
it from strdup (va()) to nva(), and then either didn't think to look for
nva or thought it wasn't worth changing.
Multi-line calls (especially messages) got rather confusing to read as
the lines jumped back and forth. Now the binding is better but the dags
code is reordering the parameters sometimes.
The server code is not yet ready for doubles, especially in its varargs
builtins: they expect only floats. When float promotion is enabled
(default for advanced code, disabled for traditional or v6only),
"@float_promoted@" is written to the prog's strings.
That was a fair bit trickier than I thought, but now .return and .paramN
are handled correctly, too, especially taking call instructions into
account (they can "kill" all 9 defs).
This reverts commit a2f203c840.
There is indeed a world of difference between "any" and "only", and it
helps if I read the rest of the docs AND the code :P.
As expected, this does not fix the mangled pointer problem in
struct-init-param.r, but it does improve the ud-chains. There's still a
problem with .return, but it's handling in flow_analyze_statement is a
bit "special" :P.
Doing the same thing at the end of two branches of an if/else seems off.
And doing an associative(?) set operation every time through a loop is
wasteful.
This fixes the ICE when attempting to compile address-cast without
optimization (just realized why, too: the assignment was optimized out
of existence).
This the fixes the incorrect flow analysis caused by the def being seen
to have the wrong size (structure field of structure def seen through a
constant pointer). Fixes the ICE, but the pointer constant is broken
somewhere in dags, presumably.
This fixes the problem of using nil for two different compound types
within the one expression. The problem is all compound types have the
same low-level type (ev_invalid) and this caused the two different nils
to have the same type when taken back up to expression level.
