I don't know why it didn't happen during the demo loop, but going from
the start map to e1m1 caused a segfault due to the efrags for a lava
ball getting double freed (however, I do think it might be because the
ball passed through at least two leafs, while entities in the demos did
not). The double free was because SCR_NewScene (indirectly) freed all
the efrags without removing them from entities, and then the client code
deleting the entities caused the visibility components to get deleted
and thus the efrags freed a second time. Using ECS_RemoveEntities on the
visibility component ensures the entities don't have a visibility
component to remove when they are later deleted.
While simple component pools can be cleared simply by zeroing their
counts, ones that have a delete function need that function to be called
for all the components in the pool otherwise leaks can happen.
It's currently used only by the vulkan renderer, as it's the only
renderer that can make good use of it for alias models, but now vulkan
show shirt/pants colors (finally).
This cuts down on the memory requirements for skins by 25%, and
simplifies the shader a bit more, too. While at it, I made alias skins
nominally compatible with bsp textures: layer 0 is color, 1 is emissive,
and 2 is the color map (emissive was on 3).
As the RGB curves for many of the color rows are not linearly related,
my idea of scaling the brightest color in the row just didn't work.
Using a masked palette lookup works much better as it allows any curves.
Also, because the palette is uploaded as a grid and the coordinates are
calculated on the CPU, the system is extendable beyond 8-bit palettes.
This isn't quite complete as the top and bottom colors are still in
separate layers but their indices and masks can fit in just one, but
this requires reworking the texture setup (for another commit).
For whatever reason, I had added an extra 4 bytes to the fragment
shader's push-constants. It took me a while to figure out why renderdoc
wouldn't stop complaining about me not writing enough data.
It turns out my approach to alias skin coloring just doesn't work for
the quake data due to the color curves not having a linear relationship,
especially the bottom colors.
It works on only one layer and one mip, and assumes the provided texture
data is compatible with the image, but does support sub-image updates
(x, y location as parameters, width and height in the texture data).
The bright end of the color map is actually twice the palette value, but
I didn't understand this when I came up with the shirt/pants color
scheme for vulkan. However, the skin texture can store only 0..1, so the
mapping to 0..2 needs to be done in the shader. It looks like it works
at least better: the gold key at the end of demo1 doesn't look as bleh,
though I do get some weird colors still on ogres etc.
Currently only for gl/glsl/vulkan. However, rather than futzing with
con_width and con_height (and trying to guess good values), con_scale
(currently an integer) gives consistent pixel scaling regardless of
window size.
Well, sort of: it's still really in the renderer, but now calling
R_AddEfrags automatically updates the visibility structure as necessary,
and deleting an entity cleans up the efrags automatically. I wanted this
over twenty years ago.
The support for the new vector types broke compiling code using
--advanced. Thus it's necessary to ensure vector constants are
float-type and vec3 and vec4 are treated as vector and quaternion, which
meant resurrecting the old vector expression code for v6p progs.
This fixes maplist showing only those maps in the user directory.
However, no checking is done for duplicate files due to earlier search
paths overriding later paths.
This involved disabling sigils for hipnotic and rogue (not used),
adjusting the number of items views, and moving the two keys views for
hipnotic. Rogue is not yet using the correct status bar pics.
The functionality of the hipnotic and rogue weapon power-ups is now done
by a various mappings instead of separate functions. In theory, this
should make things more flexible, but most importantly, there's a lot
less code duplication.
Sigils can't be flashed as they don't have any animations provided, and
they're not normally as critical. I don't know why items weren't
flashed, but since the pics are there, might as well use them (and the
flashing keys do look pretty good).
I think this makes the purpose of the functions more clear and makes the
protocol logic less dependent on the meaning of some of the updates.
Most of the update functions are not fully implemented yet.
I had forgotten that the cl structs in nq and qw were different layouts,
which resulted in qw's sbar/hud being quite broken. Rather than messing
with the structs, I decided it would be far better in the long run to
clean up sbar's access to the cl struct and the few other nq/qw specific
globals it used. There are still plenty of bugs to fix, but now almost
everything is in the one place.
In the end, it was removal of the old entries that corrupted the parent
indices. Very nicely, most of the fixes involved removing code. Taking
advantage of the ECS to debug the hierarchies was fun, and the resulting
colorized entity names helped no end.
Even 37 objects is a lot, but it's a whole lot better than 180. Most
importantly, it reproduces the problem, which seems to be not all parent
indices getting updated. The child indices seem to be working nice, as
do the reference object indices (ie, the entity components). I suspect
its the parent indices getting corrupted that cause problems on the
second switch of the hud/sbar cvar as the parent indices are used to
find the child indices that need to be updated.
This improves the behavior of hierarchies when self-inserting, but nq's
sbar still crashes when trying to do so. However, its tree is a fair bit
more complex than the test case (that does pass now), so I need to try
to replicate the important parts of the tree with fewer objects (180 is
too many to work with).
As expected, reparenting a sub-hierarchy such that it (and possibly its
children) move up the arrays fails (this is why sbar needs to first
remove the sub-hierarchy then insert it).
Since test_build_hierarchy2 already tested removal of a sub-hierarchy
(once fixed), it seems test_build_hierarchy3 testing parenting within
the same hierarchy would be a good idea. Reparenting such that
everything moves to later in the arrays works nicely (not very
surprising).
Its updates to the various indices were out, but this was missed due to
the tests being wrong. I wonder if I got interrupted while working on
them last and just assumed the removals were correct. This improves
sbar's behavior, but it's still wrong when pulling the armory view out
of the inventory. Very unsure what's going on, but the various indices
look ok, as do the view positions.
Ugh, things were quite bad, it turns out. It seems a lot of trouble
would have been saved if these tests had worked (however, something is
still not quite right as views are out of place).
This is the bug that sbar found when pulling a sub-hierarchy out of a
larger hierarchy: child indices not getting updated correctly for later
siblings and any niece objects.
The hierarchy-specific tests from the transform tests have been moved
into the ecs tests and the transform tests renamed appropriately. As
part of the process, hierarchies can now have a null type (ie, no
additional components maintained by the hierarchy). This should make
sorting out the issues highlighted by sbar a bit easier.
It should have always been here, but when I first created the hierarchy
and transform objects, I didn't know where things would go. Having two
chunks of code for setting an entity's parent was too already too much,
and I expect to have other hierarchy types. Doesn't fix the issues
encountered with sbar, of course.
The text object covering the whole passage was not being initialized,
thus center print tried to print rubbish when (incorrectly) printing the
entire message.
I'm not particularly happy with the way onresize is handled, but at this
stage a better way of dealing with resizing views and getting the child
views to flow correctly hasn't come to mind. However, the system should
at least be usable.
I'm not sure when things broke on my laptop (I thought I got warp and
fisheye working on my laptop), but it turns out things weren't quite
right, thus warp (and presumably fisheye) weren't working properly due
to GLSL errors that I only just noticed. This fixes water warp (and
probably fisheye).
This includes moving the related cvars from botn nq and qw into the
client hud code. In addition, the hud code supports update and
update-once function components. The update component is for updates
that occur every frame, but update-once components (not used yet) are
for one-shot updates (eg, when a value updates very infrequently).
Much of the nq/qw HUD system is quite broken, but the basic status bar
seems to be working nicely. As is the console (both client and server).
Possibly the biggest benefit is separating the rendering of HUD elements
from the updating of them, and much less traversing of invisible views
whose only purpose is to control the positioning of the visible views.
The view flow tests are currently disabled until I adapt the flow code
to ECS.
There seems to be a problem with view resizing in that some gravities
don't follow resizing correctly.
As the bookkeeping data is spread between three arrays, sorting a
component pool is not trivial and thus not something to duplicate around
the codebase.
It's not quite complete in that entities need to be created for the
objects, and passage text object might get additional components in the
hierarchy, but the direct use of views has been replaced by the use of a
hierarchy object with the same tree structure, and now has text objects
for paragraphs and the entire passage.
As an implementation detail, inserting null hierarchies (src hierarchy
is null) is supported for ease of hierarchy construction from data. No
component data is copied, only the child and parent indices and counts
are updated.
The resource functions assume the requested layers is correct (really,
the lighting code assumes that the resource functions assume such), but
QFV_CreateImage multiplies the layer count by 6 for cube maps (really,
the issue is in QFV_CreateImage, but I want to move away from it
anyway).
The check for the entity being the view model was checking only the
view model id, which is not sufficient when the view model is invalid by
never being set to other than 0s. A better system for dealing with the
view model is needed.
Another step towards moving all resource creation into the one place.
The motivation for doing the change was getting my test scene to work
with only ambient lights or no lights at all.
It seems this isn't needed any more (not sure why) as both glsl and
vulkan are happy without it. Also unsure why moving to ECS made gl and
sw change behavior regarding rendering the test models in my scene.
While the libraries are probably getting a little out of hand, the
separation into its own directory is probably a good thing as an ECS
should not be tied to scenes. This should make the ECS more generally
useful.
This fixes the segfault due to the world entity not actually existing,
without adding a world entity. It takes advantage of the ECS in that the
edge renderer needs only the world matrix, brush model pointer, and the
animation frame number (which is just 0/1 for brush models), thus the
inherent SOA of ECS helps out, though benchmarking is needed to see if
it made any real difference.
With this, all 4 renderers are working again.
Since entity_t has a pointer to the registry owning the entity, there's
no need to access a global to get at the registry. Also move component
getting closer to where it's used.
It no longer initializes the new component. For that, use
Ent_SetComponent which will copy in the provided data or call the
component's create function if the data pointer is null (in which case,
Ent_SetComponent acts as Ent_SetComponent used to).
I realized I should check that the entity owns the component before
treating it as existing when adding an existing component, then noticed
that Ent_RemoveComponent didn't check before removing the component. I
imagine that would have been a fun debug session :P
While checking on how Ent_AddComponent behaved (I don't remember what I
was looking for, though), I realized that instead of treating adding the
same component to an entity as an error, Ent_AddComponent should just
return the existing component.
It was being set to -1 unconditionally due to forgetting to use id.
However, I decided I didn't like reusing the id var and did some
renaming while I was at it.
This puts the hierarchy (transform) reference, animation, visibility,
renderer, active, and old_origin data in separate components. There are
a few bugs (crashes on grenade explosions in gl/glsl/vulkan, immediately
in sw, reasons known, missing brush models in vulkan).
While quake doesn't really need an ECS, the direction I want to take QF
does, and it does seem to have improved memory bandwidth a little
(uncertain). However, there's a lot more work to go (especially fixing
the above bugs), but this seems to be a good start.
Hierarchies are now much closer to being more general in that they are
not tied to 3d transforms. This is a major step to moving the whole
entity/transform system into an ECS.
Not that anything is actually rendered yet, but the validation layers
don't like the null render pass. Came up now because ctf1 seems to make
the first light an ambient light.
It didn't really add anything of value as the glyph bitmap rects and the
bearings were never used together, and the rest of the fields were
entirely redundant. A small step towards using a component system for
text.
That does feel a little redundant, but I think the System in ECS is
referring to the systems that run on the components, while the other
system is the support code for the ECS. Anyway...
This is based heavily on the information provided by @skipjack in his
github blog about EnTT. Currently, entity recycling and sparse arrays
for component pools have been implemented, and adding components to an
entity has been tested.
The inconsistencies in clang's handling of casts was bad enough, and its
silliness with certain extensions, but discovering that it doesn't
support raw strings was just too much. Yes, it gives a 3s boost to qfvis
on gmsp3v2.bsp, but it's not worth the hassle.
While chatting about utf-8, I noticed that QF doesn't ensure the input
sequences are the shortest possible encodings. It turns out that the
check is easy in that only the second byte needs to be checked if the
first byte's data bits are 0, and the second byte must have a data value
larger than that representable by the next lower leading byte.
While the base of a memory object was aligned when calculating the
memory block size, the top end was not, which could result in the memory
block not getting enough bytes allocated to satisfy alignment
requirements (eg, for flushing).
While fixing that, I noticed the offsets of objects were not being
aligned when binding, so that is fixed as well.
Fixes Mr Fixit on my VersaPro.
This is the beginning of adding ECS to QF. While the previous iteration
of hierarchies was a start in the direction towards ECS, this pulls most
of the 3d-specific transform stuff out of the hierarchy "objects",
making all the matrices and vectors/quaternions actual components (in
the ECS sense). There's more work to be done with respect to the
transform and entity members of hierarchy_t (entity should probably go
away entirely, and transform should become hierref_t (or whatever its
final name becomes), but I wanted to get things working sooner than
later.
The motivation for the effort was to allow views to use hierarchy_t,
which should be possible once I get entity and transform sorted out.
I am really glad I already had automated tests for hierarchies, as
things proved to be a little tricky to get working due to forgetting why
certain things were there.
With the improved atlas allocation, 2x is no longer needed and 1.2x
seems to be sufficient. Most importantly, it reduced the texture for
amiri-regular.ttf at 72 pix height from 8x to 4x (the staging buffer
isn't big enough for 8k textures).
Currently, only 16 fonts can be loaded (I need to sort out descriptor
set pools), but glyphs are grouped into batches of the same font. While
not quite optimal as it can result in bouncing between descriptor sets a
lot, it's still reasonably efficient.
Line rendering now has its own pipeline (removing the texture issue).
Glyph rendering (for fonts) has been reworked to use instanced quad
rendering, with the geometry (position and texture coords) in a static
buffer (uniform texture buffer), and each instance has a glyph index,
color, and 2d base position.
Multiple fonts can be loaded, but aren't used yet: still just the one
(work needs to be done on the queues to support multiple
textures/fonts).
Quads haven't changed much, but buffer creation and destruction has been
cleaned up to use the resource functions.
Its value on input is ignored. QFV_CreateResource writes the resource
object's offset relative to the beginning of the shared memory block.
Needed for the Draw overhaul.
I got tired of writing the same 13 or so lines of code over and over (it
actually put me off experimenting with Vulkan). Thus...
QFV_PacketCopyBuffer does the work of handling barriers and a (full
packet) copy from the staging buffer to a GPU buffer.
QFV_PacketCopyImage does a similar job, but for images. However, it
still needs a lot of work, but it does make getting a basic texture onto
the GPU much less of a hassle.
Both functions should make staging data much less error-prone.
This moves the qfv_resobj_t image initialization code from the IQM
loader into the resource management code. This will allow me to reuse
the code for setting up glyph data. As a bonus, it cleans up the IQM
code nicely.
A passage object has a list of all the text objects in the given string,
where the objects represent either white space or "words", as well as a
view_t object representing the entire passage, with paragraphs split
into child views of the passage view, and each paragraph has a child
view for every text/space object in the paragraph.
Paragraphs are split by '\n' (not included in any object).
White space is grouped into clumps such that multiple adjacent spaces
form a single object. The standard ASCII space (0x20) and all of the
Unicode characters marked "WS;<compat> 0020" are counted as white space.
Unless a white space object is the first in the paragraph, its view is
marked for suppression by the view flow code.
Contiguous non-white space characters are grouped into single objects,
and their views are not suppressed.
All text object views (both white space and "word") have their data
pointer set to the psg_text_t object representing the text for that
view. This should be suitable for simple text-mode unattributed display.
More advanced rendering would probably want to create suitable objects
and set the view data pointers to those objects.
No assumption is made about text direction.
Passage and paragraph views need to have their primary axis sizes set
appropriately, as well as their resize flags. Their xlen and ylen are
both set to 10, and xpos,ypos is 0,0. Paragraph views need their
setgeometry pointer set to the appropriate view_flow_* function.
However, they are set up to have their secondary axis set automatically
when flowed.
Text object views are set up for automatic flowing: grav_flow, 0,0 for
xpos,ypos. However, xlen and ylen are also both 0, so need to be set by
the renderer before attempting to flow the text.
Adjusting the size of the parent (container) view to the views it
contains will be useful for automatic layout and knowing how large the
view is for scrolling. New tests added so testing both with and without
the option is still possible.
This should be suitable for laying out text objects with word-wrap,
where each view is a "word" or break between "words". This should be
useful for any other objects that could benefit from similar layout
rules. All eight flows are supported left-right-top-down (English and
most European languages), right-left-top-down (Arabic and similar),
top-down-right-left (Chinese, Japanese, Korean), top-down-left-right,
as well as bottom-up variants of those four.
More work is needed for support of things like views being centered on
the flow line rather than on one edge (depends on flow direction),
offset views, and others. Suppression of "spaces" at the beginning of a
line is supported but not tested.
Due to the changes related to console views, the console was either
fully visible or not at all visible, so it took several seconds to
disappear whenever closed.
Taking the screen data from the event fixes the console size being out
due to screen_view updating after the app_window event fires. Really,
this makes it independent of the order.
UVs being 0 meant that lines were picking up the upper left pixel of
char 0 of conchars. With quake data, this meant a transparent pixel.
Fixes invisible debug lines :P.
It turns out that using the swapchain image for the size requirements is
unreliable: when running under renderdoc, vkGetImageMemoryRequirements
sets the memory requirements fields to 0, leading eventually to a null
memory object being passed to vkMapMemory, which does not end well.
I had missed that vkCmdCopyImage requires the source and destination
images to have exactly the same size, and I guess assumed that the
swapchain images would always be the size they said they were, but this
is not the case for tiled-optimal images. However,
vkCmdCopyImageToBuffer does the right thing regardless of the source
image size.
This fixes the skewed screenshots when the window size is not a multiple
of 8 (for me, might differ for others).
There's a problem with screenshot capture in that the image is sheared
after window resize, but the screen view looks good, and vulkan is happy
with the state changes.
As gbuf_base derives from the base pipeline, it inherits base's dynamic
setting, and thus doesn't need its own. I had a FIXME there as I wasn't
sure why I had a redundant setting, but I really can't see why I'd want
it different from any of the other main renderpass pipelines.
I've found and mostly isolated the parts of the code that will be
affected by window resizing, minus pipelines but they use dynamic
viewport and scissor settings and thus shouldn't be affected so long as
the swapchain format doesn't change (how does that happen?)
Finally, the model_funcs and render_funcs struts use designated
initializers. Not only are they good for ensuring correct
initialization, they're great for the programmer finding the right
initializer.
I must have forgotten about the SYS_DeveloperID_... enum values, when I
wrote that code, because relying on the line number is not really for
the best.
Due to design issues in the console API that I don't feel like
addressing at this stage, the console view is not a child of the
client's screen view (not even sure it should be in the first place), so
it won't get resized automatically when the client's screen view
resizes. However, ie_app_window is sent when the screen size changes,
and the console has to process input events anyway, so it's quite
reasonable to handle the event.
With the addition of dependencies on freetype and harfbuzz, it became
clear that the renderer plugins need to be explicitly linked against
external dependencies (and that I need to do more installed testing,
rather than just my static local builds). This fixes the unresolved
symbols when attempting to load any of the plugins.
qwaq doesn't supply a backtile pic, so Draw_TileClear in the gl and glsl
renderers would segfault when qwaq's window width changed due to some
back-tile being drawn.
As of a recent nvidia driver update, it became necessary to enable the
feature. I guess older drivers (or vulkan validation layers?) were a bit
slack in their checking (or perhaps I didn't actually get those lighting
changes working at the time despite having committed them).
This did involve changing some field names and a little bit of cleanup,
but I've got a better handle on what's going on (I think I was in one of
those coding trances where I quickly forget how things work).
Just head and tail are atomic, but it seems to work nicely (at least on
intel). I actually had more trouble with gcc (due to accidentally
testing lock-free with the wrong ring buffer... oops, but yup, gcc will
happily optimize your loop to spin really really fast). Also served as a
nice test for C11 threading.
This makes bsp traversal even more re-entrant (needed for shadows).
Everything needed for a "pass" is taken from bsp_pass_t (or indirectly
through bspctx_t (though that too might need some revising)).
Ambient lights are represented by a point at infinity and a zero
direction vector (spherical lights have a non-zero direction vector but
the cone angle is 360 degrees). This fixes what appeared to be mangled
light renderers (it was actually just an ambient light being treated as
a directional light (point at infinity, but non-zero direction vector).
There are some issues with the light renderers getting mangled, and only
the first light is even touched (just begin and end of render pass), but
this gets a lot of the framework into place.
Sounds odd, but it's part of the problem with calling two different
things with essentially the same name. The "high level" render pass in
question may be a compute pass, or a complex series of (Vulkan) render
passes and so won't create a Vulkan render pass for the "high level"
render pass (I do need to come up with a better name for it).
I really don't remember why I made it separate, though it may have been
to do with r_ent_queue. However, putting it together with the rest is
needed for the "render pass" rework.
