Commit graph

16096 commits

Author SHA1 Message Date
alexey.lysiuk
b0398273e2 - fixed compilation with GCC and Clang
src/g_mapinfo.cpp:840:58: error: cannot pass objects of non-trivially-copyable type ‘class FString’ through ‘...’
2019-05-14 20:13:16 +02:00
drfrag
369233c2cf - fixed 'I' to 'i' upper-to-lower mapping
https://forum.zdoom.org/viewtopic.php?t=64526
(patch by _mental_)
2019-05-14 20:13:13 +02:00
drfrag
59633d8c72 - Changed deprecations. 2019-05-14 20:10:58 +02:00
alexey.lysiuk
1577da4186 - fixed deprecation warnings in core scripts
Actually no warnings here (drfrag)

# Conflicts:
#	wadsrc/static/zscript/ui/statscreen/statscreen.zs
2019-05-14 20:07:55 +02:00
Christoph Oelckers
6da5f579e0 - updated UFMF-spec for UTF-8. 2019-05-14 19:58:17 +02:00
Player701
f55ad14663 Added support for monospacing alignment modes to HUDFont / BaseStatusBar.DrawString (#810)
* - Added support for monospacing alignment modes to HUDFont / BaseStatusBar.DrawString

* - added underlying type declaration for EMonospacing

* - replaced "#include v_video.h" with a declaration of EMonospacing
2019-05-14 19:50:43 +02:00
Christoph Oelckers
a6491c71b5 - added monospacing support to Screen.DrawText and its native counterparts.
# Conflicts:
#	src/g_statusbar/shared_sbar.cpp
#	src/v_draw.cpp
#	src/v_video.h

Without the HUD modifications. (drfrag)

# Conflicts:
#	src/v_draw.cpp
#	wadsrc/static/zscript/base.zs
2019-05-14 19:50:25 +02:00
Christoph Oelckers
2887bc6afc - do proper checks for chat string length.
The counter variable was not only used incorrectly, it was completely redundant.
This still has a byte limit due to how the network code works so for non-Latin languages may result in shorter strings.
2019-05-14 19:47:12 +02:00
Player701
405ece578a - Added "const" qualifiers to the new UTF-8-aware string methods 2019-05-14 19:39:09 +02:00
Christoph Oelckers
0c36ab0651 - added CharUpper and CharLower functions to ZScript.
These, like MakeUpper and MakeLower, use the internal Unicode case conversion tables.

# Conflicts:
#	src/scripting/vmthunks.cpp
2019-05-14 19:37:49 +02:00
Christoph Oelckers
1e0228c13a - added Unicode aware MakeUpper/MakeLower functions to FString and ZScript's String and deprecated ToUpper/ToLower because their semantics did not allow fixing them. 2019-05-14 19:37:47 +02:00
drfrag
f2a43353c4 - Exported some types to fontinternals.h. 2019-05-14 19:37:44 +02:00
Christoph Oelckers
6caf1f60c6 - added some UTF-8 utilities to FString.
This deprecated CharAt and CharCodeAt for being unsuitable for text processing and in the case of CharCodeAt also for being buggy.
A new replacement, ByteAt has been added that reads a string byte by byte, as well as CodePointCount, which counts the amount of Unicode code points in a string and GetNextCodePoint which reads the string code point by code point.
Note that while this woll work as intended with the currently supported languages as a means to read single characters, there is no guarantee that this will remain so if Unicode support gets extended to things which break the "one code point == one character" assumption.
2019-05-14 19:35:21 +02:00
drfrag
77f2497ecf - added a few more character fallbacks and fixed the StringWidth function
This checked character size without substituting missing characters, resulting in bad return values.
(patch by Graf)
2019-05-14 19:34:12 +02:00
Christoph Oelckers
344c6a380b - fixed: The WideString conversion function was missing a null check. 2019-05-14 19:10:27 +02:00
Christoph Oelckers
52b8c02fdf - use the Unicode version of GetCommandLine for the crash log. 2019-05-14 19:09:24 +02:00
alexey.lysiuk
d85fb785eb - fixed missing command line in crash reports
https://forum.zdoom.org/viewtopic.php?t=64149
2019-05-14 19:09:22 +02:00
Alexander
73a0275cf6 option search improvements
1. Top-level menu names are now properly handled.
2. Changing "Any or All terms" option now immediately updates the results.
3. Reformatted menu.zs to have tabs instead of spaces.
2019-05-14 19:01:25 +02:00
drfrag
e290cb77e4 - Added missing language definitions for option search.
# Conflicts:
#	wadsrc/static/language.rus
2019-05-14 18:59:34 +02:00
Alexander
dc666203b7 added option search menu
# Conflicts:
#	src/g_cvars.cpp
#	wadsrc/static/menudef.txt

# Conflicts:
#	src/menu/menu.cpp
2019-05-14 18:59:10 +02:00
Christoph Oelckers
f026198a8d - set a Unicode capable font for the Windows console.
The default raster font only contains the OEM 437 code page which is quite useless.
2019-05-14 18:42:33 +02:00
drfrag
a31c9273dd - do not use iswspace to check raw UTF-8.
At least under MSVC this function reports 0x85 as whitespace, but it is a continuation byte for UTF-8 sequences and may not be treated as whitespace.
(patch by Graf)
2019-05-14 18:41:40 +02:00
drfrag
8a9ffb3031 - added a fallback mapping for the typographic upper quotation mark to the generic form.
- added lowercase->uppercase mapping for ß.
(patch by Graf)
2019-05-14 18:39:00 +02:00
Christoph Oelckers
d47d1f248b - fixed: The command line parser of the console was incapable of handling non-ASCII characters. 2019-05-14 18:31:02 +02:00
Christoph Oelckers
182846c9a0 - fixed character counting in TypeOn HUD message.
# Conflicts:
#	src/g_shared/hudmessages.cpp
2019-05-14 18:29:11 +02:00
Christoph Oelckers
c7101b6b32 - reformat the intermission screen if the text is very long.
This will ensure better visibility on widescreen displays, at least.

# Conflicts:
#	src/intermission/intermission.cpp
2019-05-14 18:04:22 +02:00
Christoph Oelckers
5bd6b6758c - fixed utf8_decode.
This function was written for already validated UTF-8 but not for text that can be mixed with ISO-8859-1.
To handle that properly it needs to do a bit more validation to avoid mangling its output and instead reject invalid input.
2019-05-14 18:01:46 +02:00
Christoph Oelckers
2ea80429a7 - fixed the status bar string drawers which weren't UTF-8 capable yet.
# Conflicts:
#	src/g_statusbar/sbarinfo.cpp
#	src/g_statusbar/shared_sbar.cpp
2019-05-14 18:00:54 +02:00
Christoph Oelckers
d2959760dc - fixed chat input.
# Conflicts:
#	src/ct_chat.cpp
2019-05-14 17:49:05 +02:00
Christoph Oelckers
4561eaf093 - Fixed the character substitution logic.
For pure uppercase fonts it makes no sense to try a lowercase substitution as a first step.

# Conflicts:
#	src/gamedata/fonts/font.cpp
#	src/v_font.h
2019-05-14 17:46:42 +02:00
Christoph Oelckers
3127a28e35 - fixed reading of exit text replacements and display of intermission text screens for Unicode. 2019-05-14 17:40:12 +02:00
Christoph Oelckers
cf818fbe10 - changed zstrformat to allow %c to emit non-ASCII characters as UTF-8. 2019-05-14 17:38:45 +02:00
Christoph Oelckers
0153a42240 - fixed issues with text entering
* entering a savegame description did not work anymore
* the length check was too restrictive and always underestimated the available space
* use the console font for entering a savegame description. This has more characters and better contrast for this content.
* the interface to the text enterer used bad measurements.

# Conflicts:
#	wadsrc/static/zscript/ui/menu/optionmenuitems.zs
2019-05-14 17:38:21 +02:00
drfrag
4c7eb79d72 - Fixed MinGW compilation. 2019-05-14 17:30:58 +02:00
Christoph Oelckers
d51728c7c3 - did a bit of fine tuning to the character replacement mappings:
* prefer accent-less lower case over uppercase letters if an accented lower case letter cannot be found.
* added accent-less mappings for Latin Extended 1 (0x100-0x17f) and some easy to handle characters between 0x200 and 0x220. This should allow to display all Eastern European text without empty gaps for missing letters.

# Conflicts:
#	src/v_font.cpp
2019-05-14 17:28:16 +02:00
Christoph Oelckers
b14a48d2dc - fixed Windows startup.
* the window class name was still ASCII, thanks to some totally pointless and ultimately dangerous type cast to LPCTSTR which rendered all type checks ineffective.
* use wWinMain instead of WinMain so that a Unicode argv gets created. For whatever reason, the ANSI startup leaves this variable empty.
* added a 'disablecrashlog' CCMD for Windows. It is a lot more useful with a debugger present to get the standard crash notification from the system which allows opening a debugger than the crash log and no option to open a debugger.
2019-05-14 17:20:30 +02:00
Christoph Oelckers
163d3395b8 - fixed crash log display.
This still contained pieces where a multibyte string was passed through SendMessage and WM_SETTEXT. All these have been replaced with SetWindowTextW.
This commit also removes the never used crash log upload code and all associated assets because it is extremely unlikely that such a feature will ever be implemented.
2019-05-14 17:20:17 +02:00
Christoph Oelckers
1e9a65e673 - fixed the player name display.
This was broken by several small unicode-incompatible code fragments.
This commit also removes the input limit for the player name and the savegame description. With multibyte encoding, limiting them to a fixed length did not work right.
Currently these will just overflow the fields if the text becomes too long, this needs some additional work.
2019-05-14 17:19:34 +02:00
Christoph Oelckers
f790d48041 - made the menu's text input handler Unicode capable.
Also make sure that the savegame description remains readable. Unlike in-game text this can be done without double-encoding existing UTF-8.
2019-05-14 17:19:19 +02:00
Christoph Oelckers
d067c55cff - made the chat input Unicode-capable.
Also changed this to use the console font because it is far better equipped with special characters than the small font.

# Conflicts:
#	src/ct_chat.cpp

# Conflicts:
#	src/ct_chat.cpp
2019-05-14 17:17:41 +02:00
Rachael Alexanderson
9b8ec98f78 - fix building in 32 bit windows 2019-05-14 17:16:15 +02:00
Christoph Oelckers
c7877388a9 - made the console Unicode-capable.
This also necessitated some character remapping in the console font to move the Windows-1252 extra characters to their proper Unicode code points.

# Conflicts:
#	src/CMakeLists.txt
#	src/posix/sdl/i_input.cpp
#	src/scripting/backend/codegen.cpp
#	src/v_font.cpp

# Conflicts:
#	src/c_console.cpp
2019-05-14 17:12:18 +02:00
Christoph Oelckers
9030289e1e - use the Unicode version of Windows's clipboard functions. 2019-05-14 17:10:00 +02:00
Christoph Oelckers
4c9e6853b1 - made Windows backend parts of the console Unicode capable.
# Conflicts:
#	src/c_console.cpp
#	src/v_font.cpp
#	src/win32/i_system.cpp
2019-05-14 17:05:40 +02:00
drfrag
77498e2740 Revert "- Applied some Blzut3 patches to compile with MinGW."
This reverts commit ad5959b827.

Compilation with vanilla MinGW is broken since the changes to the HQ resize modes anyway.
2019-05-14 16:32:39 +02:00
Christoph Oelckers
8d157f3ced - switched the Windows backend to use the Windows Unicode API.
With localization for non-Latin languages on the support list the multibyte API doesn't cut it anymore. It neither can handle system text output outside the local code page nor can an ANSI window receive text input outside its own code page.
Similar problems exist for file names. With the multibyte API it is impossible to handle any file containing characters outside the active local code page.

So as of now, everything that may pass along some Unicode text will use the Unicode API with some text conversion functions. The only places where calls to the multibyte API were left are those where known string literals are passed or where the information is not used for anything but comparing it to other return values from the same API.

# Conflicts:
#	src/rendering/hwrenderer/postprocessing/hw_postprocess.h
#	src/win32/base_sysfb.cpp
#	src/win32/i_main.cpp
#	src/win32/win32basevideo.cpp
#	src/win32/win32glvideo.cpp

# Conflicts:
#	src/version.h
#	src/win32/i_main.cpp
#	src/win32/i_system.cpp
#	src/win32/optwin32.h
#	src/win32/win32gliface.cpp
#	wadsrc/static/language.enu
2019-05-14 16:17:05 +02:00
Christoph Oelckers
e27efa5a2e - fixed DHudMessageTypeOnFadeOut's character counter to be UTF-8 compatible.
This was reading the string by byte and not by character and could end up printing incomplete UTF-8 data.
2019-05-14 13:31:23 +02:00
Christoph Oelckers
18253b5203 - added some character counting utilities to FString. 2019-05-14 13:31:07 +02:00
Christoph Oelckers
8626dd665c pass full Unicode characters to EV_GUI_Char for Linux and macOS.
The Linux backend looked like it didn't handle anything non-ASCII at all, but this all needs to be tested.
Windows will be a bit more work because it requires using the Unicode API for creating the main window.
2019-05-14 13:30:52 +02:00
drfrag
b041e629ac - fixed lowercase handling of non-basic-latin characters.
The added table may be overkill but this way the font engine is prepared for things to come.
Currently the text placement in the menu seems a bit broken, that's a task for later.
(patch by Graf)

# Conflicts:
#	src/v_font.cpp
2019-05-14 13:11:10 +02:00