Recent Commits to HX:master (5)

Merge branch 'unxed-dkrnl32-cp-utf8'

hdpmi: add int 31h, ax=507h extension (set PCD/PWT flags); XMS hook improved for A20 enable

a20gate: comment added

HX#76: move UTF-8 conversions into WC2UTF8.ASM

Assisted-by: OpenAI GPT-6 Luna

DKRNL32: CP_UTF8 in MultiByteToWideChar, WideCharToMultiByte, GetCPInfo

First of the small PRs proposed in Baron-von-Riedesel/HX#75.

Until now, MultiByteToWideChar and WideCharToMultiByte ignored the code page: bytes were zero-extended to WCHARs, and WCHARs were truncated to their low byte. A Win32 program that keeps its text in UTF-8 (CP_UTF8, 65001) got garbage for anything outside ASCII.

With this change, CP_UTF8 is converted for real (WC2MB.ASM):

- UTF-8 <-> UTF-16, with surrogate pairs for U+10000..U+10FFFF;
- ill-formed input (overlong forms, encoded surrogates, values above U+10FFFF, truncated sequences, stray continuation bytes, unpaired surrogates) is replaced by U+FFFD exactly as Windows does it, including how many U+FFFD it gives. With MB_ERR_INVALID_CHARS / WC_ERR_INVALID_CHARS the call fails with ERROR_NO_UNICODE_TRANSLATION, and a too small buffer is reported first, also as on Windows;
- length -1, size queries (buffer size 0) and ERROR_INSUFFICIENT_BUFFER behave as on Windows.

GetCPInfo(CP_UTF8) now reports MaxCharSize = 4 (GETCPINF.ASM).

All other code pages (CP_ACP, CP_OEMCP, ...) behave exactly as before. Mapping them through the DOS code page table is left for a separate PR.

Testing: a differential test on GitHub Actions (windows-latest). WC2MB.ASM and GETCPINF.ASM are assembled with JWasm 2.20 using the flags of SrcEmu/DKRNL32/MAKEFILE (release and _DEBUG), then linked into a Win32 test program. Every case runs through both DKRNL32 and the kernel32 of Windows, and the result, the output and the last error must match. The test also checks that other code pages are unchanged. Test and workflow: https://github.com/unxed/sandbox/tree/main/hx-utf8. The full DKRNL32.DLL build and a run under HX itself were not done.

DKRNLHIS.TXT is not touched, so that you can number the version yourself.

AI disclosure: this change (code, test and this text) was written by the Claude Opus 5.5 model (Anthropic), working under the Lunobot instructions (https://github.com/unxed/lunobot) on the go2dos project (https://github.com/unxed/go2dos), at the request of the fork owner (@unxed).

*Lunobot-1 (node 19d368003c6c048e43dbec70; LNX; Claude Opus 5.5)*

Assisted-by: Claude Opus 5.5 (Anthropic)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>