quakeforge/doc/ideas/language.txt

I want `entities' to be a very lightweight object.  They would perform
the same tasks as structs in C, but with other things (optionally)
assosiated with them, such as physics interactions and
networking/communications addressing.  On a large scale they'd perform
the traditional role of players, interacting with physics, take input on
the client, and so forth.  But on a smaller scale an `entity' may just
be the spawning of a group of particle sparks.  Here we would only need
to transmit the number of particles and the location, and possible some
type of particle information (bright yellow vs deep red).  We would not
need to transmit the entity number, or even when the entity is removed,
since the client could determine that information itself.

To acomplish this we need a wide variety of entity types, and preferably
seperate entity indexes too.  When a server that sends something to
update a rocket it only needs to affect rockets, not players and other
seperate entities.  As well, the different fields with an entity should
be handled differently; some can be ignored altogether, some only need
partial information (eg the current position), while others need to
transmit every change of state.  Some may only need to send a small
range of data, eg 0 through 10, even though the field itself is a 32bit
integer.  (actually, it may be better to specify the field as a range at
the language level, and leave the implementation the job of making it a
32bit integer.  this avoids 32bit vs 64bit issues.) 

I like the idea of having seperate threads.  QuakeC does different
threads of a sort, by setting an entities think function and a nextthink
time, but it is extremely limited.  UnrealScript's method is somewhat
better, but I don't believe it allows real multithreading that would
take advantage of a SMP box, and thus I think it's still too limited. 
While real multithreading requires addressing issues such as locking and
concurent accesses, I don't think that's impossible to do in an
effective and efficient way, it'd just require alot of thought and
planning. 

UnrealScript also supports `mutators', which are in essence overriden
functions as you would find in C++.  While I can see the obvious benefit
of them, I believe they require much detailed examination of how they'd
interact, especially with networking, to be used effectively. 

A comment on enumerations.  They could be broken down into two types. 
The first, simpler form, is basically like in C.  It makes the
sourcecode more readable by converting the textual FOO_A into a number
when compiling.  It could also help debugging, by printing out the
reversed information.  But a second, more powerful form, would allow a
base module to define one form of the enumeration, while an extension
module adds some items to it.  The base module would see such additions
as FOO_UNKNOWN and be unable create them itself, but it would be able to
change the variable from FOO_UNKNOWN and copy the FOO_UNKNOWN into
another variable, which'd would have the value of the original variable
from the extension module's point of view. 

[a note on implimentation: the extension module would probably have a
global constant which defines the enumeration and it's textual form. 
When the server loads the module it'd fill in the global constant with
whatever value was convenient.  Thus, several modules could define
extensions, and they would not conflict (unless the textual versions
were the same).  For multiple servers to communicate it'd be
unreasonable to require one to change it's constants; instead they
should negotiate a mapping table when they connect, and convert them to
the form the other desires when transferring them.] 

I would like to have an explicit pointer type.  However, I would like
the operations available for it to be limited and safe.  You would not
be able to use pointer arithmetic (foo + 5), but array indexing would be
available (foo[5]).  You would not be able to convert pointers between
types, but perhaps would have pointers that can point to several types,
or "parent" types that are inherited into several types, and a withtype
language construct that'd allow the following block of code to treat the
pointer as a specific type, eg: 

class foo;
class bar inherits foo;
local class foo *a = spawn (class bar);
if (a.class == class foo)
	withtype (a, class bar) {
		// within this block a is treated as a class bar
	}
// a is now considered to be a class foo again

Of course this is just an idea, and the class semantics are pulled off
the top of my head.  It'd require refinement before practical use. 

Another matter is variable initialization.  Before a variable can be
used it must always be initialized.  But for a language to be guaranteed
safe the compiler must detect all situations where it might be used
uninitialized.  I propose that, since the compiler will detect the
usages anyway, we should make them all be initialized by default, and
save the programmer the trouble of doing it manually.  Then the
compiler's detection could work in reverse, and not initialize the
situations where it's sure they won't be used. 

As mentioned above, spawning a particle entity would entale sending some
data to the client.  This should be done through the fields of another
entity, and said field would probably be changed serially.  Or perhaps
an array that's not changed serially.  Either way though, it effectively
creates a heirarchy of objects. 

Serial state changes would be limited to one change per frame/tick.  To
perform more than one change per frame you could create an array,
which'd effective queue up the changes.  The entire array would then be
sent over the network each frame. 

The particle entity spawning would take place in such an array.  Each
frame would add some entities to the array, which'd be transmitted
serially (although without a strong requirement for reliability), and
would automatically be cleared at the end of the frame.  The client
would then keep track of the particles in it's own data structures. 

For bitfields you should use an array of numbers with a 0 to 1 range,
and allow the implimentation to fit them in to a more efficient data
structure. 

So far for fields transmition modes we have 'not transmitted', 'transmit
changes', and 'transmit current'.  Arrays (?) also have a 'automatically
clear' flag. 

It may be worth noting that mods would be expected to processes several
incoming frames within one of it's own frames.  This may mean that an
entity's callbacks could be called several times in one frame.  Therefor
you require an array/queue for serialized/'transmit changes' fields. 
Perhaps then that mode can only apply to arrays.  On second thought, no.
 For entities controlled by the server it'd have no issue limiting them
 to one change per frame.  And for ones it doesn't control it may want
 to apply such a limitation anyway. 

The server should use Just In Time compiling of the modules, and cache
this between runs to prevent unnecesary recompilation.  The client
should have a version string and crc (MD5 hash?) to identify each module
(of which it may have several), and save them per-server incase it
differs.  This'd allow a client to have modules from different servers
simultaneously, even if they're modified without properly updating the
version number.  It may require "garbage collection" though, to clean up
old unused versions. 

The engine would handle sending the data over the wire, resending
dropped packets, etc.  I'm not sure if what I've provided so far for
field modes is powerful enough, and in a sense it's tempting to provide
some sort of callback for the mod to add other methods, but I'm not sure
if that could be efficient/effective, or if it's even needed.  I
probably just need to go through some proper examples. 

Another aspect is that some entities may be visible to all players,
while others are only visible to some players, and still others aren't
visible to any.  Except for a few global pieces of data such as talking
and a global sound, most entities should be limited by PVS/PHS anyway. 
Perhaps each player should have it's own list of relevent entities,
through which it'd have links to the rest?  Eg, it'd have a link to
the/a world entity, which'd have have the PVS/PHS and the normal entity
lists.  A player could also have links to several world entities, either
on different servers or forming different sections of the map on one
server.  It's origin would then be relative to that world entity. 

Additionally to the pvs, though, we'd want the mod to be able to limit
an entity's visibility to only the players on a given team.  Perhaps
each player entity should have it's own entity list, to which everything
in the PVS would be added at the end of each frame, as well as whatever
other checks the mod wanted to do.  I'm not sure how much overhead
that'd add though.  (The PVS test itself would be done by the engine,
but the mod would call a function to do that). 

We'd want a very powerful find function.  It should have flags such as
PVS, PHS, radius (with a distance), and arc.  Ignoring certain entities
(such as your center point) could probably be done best by the mod. 
We'd also want to to control if it'd be sorted (closest first), and
partial vs full search (partial could be continued after, but may not
be.)  There's a fair bit of overlap with traceline here, not sure what
we'd want to do with it all. 

Re traceline: you should pass it pointers/references to the variables
you want it to modify, instead of it modifying globals. 

Since we lack pointer arithmetic we can't directly seperate a section of
an array.  Additionally, the array needs to have it's length assosiated
with it.  Perhaps we should have an "array slice" class that has the
base array, it's size, and an offset into it. 

Race conditions, including, but not limited to, deleting an entity while
something else is using it - we need to document programming techniques
to handle these situations.