This is a special case, the bug was already present in Vanilla Q2: If a
server is offering assets for download but is missing some files the
USP download code runs in an endless loop. CL_ParseDownload() detects
that something is wrong and calls CL_RequestNextDownload() which tries
to download the same file again... Work around this by skippig over that
file.
This closes#552.
The fallback logic is: r1q2 style URLs -> q2pro style URLs -> UDP. In
case that an r1q2 or q2pro server is missing files, we may fall through
to UDP and download files over abysmal slow UDP that are available over
HTTP. To work around this problem we reset the precacher iterations back
to 0 and start over with r1q2 style URLs. This works as long as we're
able to download all files. But if one or more files are unavailable
over all three download options we'll run into an endless loop. Mitigate
this by starting over only once. This comes with the drawback that on
servers that miss more than one file downloads might done over UDP were
HTTP is available. In that case the server should be fixed.
This closes#429.
Working with getter and setters was a good idea as long as we had one or
two quirks. Now we're at three with maybe more to come so it's easier to
use a struct to communicate quirks between the precacher and the HTTP
download code.
The r1q2 URL generator was, like everything in this game, buggy. It took
cl.gamedir into account when generating the URLs, but overlooked that it
is only set when 'gamedir != BASEDIR'. So baseq2 assets ended up with
/maps/foo.bsp and mod assets with /mod/maps/foo.bsp. q2pro fixed that
to always include the gamedir...
Work around this by refactoring the HTTP -> UDP fallback logic to be
more generic: Count the number of iterations and depending on the
iteration set the gamedir to be used by the URL generator or force
UDP downloads.
Cleaning the download queue as soon as a file finished downloading leads
in combination with only one parallel download and multiple precacher
runs to an ugly problem: The generic filelist is requested several time
which can lead to cycles. Hack around this by rembering if we already
requested it and reset set reminder as soon as the precacher finished.
Yes, I'm feeling dirty.
This looks easy, but is rather hacky... Downloading is implemented
through the precacher. The server sends an asset list, while loading
the map another one is generated. CL_RequestNextDownload() goes
through this list, in the order models / maps -> sounds -> images,
calls CL_CheckOrDownloadFile() for each file. CL_CheckOrDownloadFile()
checks if the file is already there, return true if it is and false
if not. If the return code is false CL_RequestNextDownload() itself
returns, it's called again by CL_ParseDownload() as soon as the just
queued file finished downloading. This way all missing files are
downloaded one after the other, when CL_RequestNextDownload() finally
reaches it's end (all files are there) it send 'begin' to the server,
thus putting the client into the game.
HTTP downloads are parallel, so CL_RequestNextDownload() cannot track
which files are there and which are missing. The work around for that is
to queue the file but have CL_CheckOrDownloadFile() return true. So
CL_RequestNextDownload() thinks the file is already there, continues
with the next one, until all missing files are queued. After that it
polls CL_PendingHTTPDownloads() and sends the 'begin' as soon as all
HTTP downloads are finished.
If a HTTP download fails we cannot just queue it as UDP download,
because the precacher things that the file is already there. And we
can't tell the precacher that it's not because the precacher tracks
files only by the number of downloaded files per asste type and not
their name. Just decreasing the number of downloaded files isn't
possible since the precacher may have progressed to the next asset
type.
So: On the HTTP side it's tracked if there was an error or not. After
CL_RequestNextDownload() has queued all files and waited for all HTTP
downloads to finish it checks the HTTP error status. If there was an
error the precacher state is reset and CL_RequestNextDownload() recurses
into itself to take another run. All files that couldn't be downloaded
are queued again, this time as UDP downloads.
This is a very first cut:
* It compiles
* It doesn't crash
What's missing:
* cmake integration
* CURL should be loaded dynamically
* Integration between download code and filesystem
* Likely UTF-8 stuff
* cl_http.c needs cleanup
* Windows support
On Unix platforms unicode is implemented through UTF-8 which is
transparent for applications. But on Windows a UTF-16 dialect is
used which needs alteration at application side. This wrapper is
another step to unicode support on Windows, now we can replace
fopen() by a function that converts our internal UTF-8 pathes to
Windows UTF-16 dialect.
This is a noop for Unix platforms. The Windows build is broken,
the compiler errors out in shared.h. This will be fixed in a
later commit.
Caveats:
* fopen() calls in 3rd party code (std_* and unzip) are not replaced.
This may become a problem. We need to check that.
* In the Unix specific code fopen() isn't replaced since it's not
necessayry.
This is largely based upon the cl_async 1 mode from KMQuake2, which in
turn is based upon r1q2. The origins of this code may be even older...
Different to KMQuake2 the asynchonous mode is not optional, the client
is always asynchonous. Since we're mainly integrating this rather
fundamental change to simplify the complex internal timing between
client, server and refresh, there's no point in keeping it optional.
The old cl_maxfps cvar controls the network frames. 30 frames should be
enough, even Q3A hasn't more. The new gl_maxfps cvar controls the render
frames. It's set to 95 fps by default to avoid possible remnant of the
famous 125hz bug.
Moving the check under the "do we have the file localy" check prevents
spurious "Refusing to download a path with .." messages with some game
data. The tank commander skin is one example. This change has no
security impact since FS_LoadFile() just opens and closes the file.
While at it tighten the condition to prevent pathes with colons (this
condition is added at the server side, too) and pathes starting with
slashes and dots.
sounds easy, right?
Except some genius decided to save CVAR_LATCH cvars in buffers of
MAX_OSPATH length into savegames.. so just changing MAX_OSPATH
breaks savegame compatibility.
Fortunately, this assumption only matters in SV_WriteServerFile() and
SV_ReadServerFile() so I worked around it by introducing a
platform-specific LATCH_CVAR_SAVELENGTH (because MAX_OSPATH was 256 on
Windows but 128 on other systems..)
Revert "change several strcat calls to Q_strlcat calls"
This reverts commit ab879f1bc7.
Revert "change (v)sprintf calls to (v)snprintf calls"
This reverts commit b46e210d76.
