Home Dashboard Directory Help

Unicode issues with WriteFile and in the CRT by Nico Rieck


 as Won't Fix Help for as Won't Fix

Sign in
to vote
Type: Bug
ID: 543801
Opened: 3/22/2010 2:28:38 PM
Access Restriction: Public
User(s) can reproduce this bug


I believe there is an issue with WriteFile [1]. Either in it's implementation, or its documentation is lacking.

WriteFile is supposed to return the number of _bytes_ written. Taking the same input, the returned value for number of bytes written can be different for different codepages. And when writing UTF-8 data with multibyte characters to stdout it seems to return the number of _characters_ written.

For example when writing a char* buffer with value { 0xC3, 0xA4, 0 } (character ä encoded as UTF-8) WriteFile returns the following values:

With console output codepage 850:
- Output: +ñ
- Bytes written: 2 (correct, and output is obviously broken)

With console output codepage 65001:
- Output: ä
- Bytes written: 1 (incorrect since "ä" in UTF-8 takes 2 bytes, but output is correct)

This behavior results in issues in Microsoft's C Runtime implementation. Several functions (like fflush) are required to verify that the number of bytes written is equal to the number of bytes in the input. The behavior of WriteFile triggers those checks and output streams are flagged with _IOERR, thus breaking programs that verify streams with ferror().

All tests were done on a German Windows 7 Professional (64-bit) using 32- and 64-bit test programs.

MSVCR90.DLL: 9.0.30729.4926
KERNEL32.DLL: 6.1.7600.16385
cmd.exe: 6.1.7600.16385

Also please refer to this thread [2] in the Visual C++ General forum where the CRT behavior was confirmed for VS 2010 RC.

[1]: http://msdn.microsoft.com/en-us/library/aa365747%28VS.85%29.aspx
[2]: http://social.msdn.microsoft.com/Forums/en-US/vcgeneral/thread/e4b91f49-6f60-4ffe-887a-e18e39250905
Sign in to post a comment.
Posted by Microsoft on 3/27/2011 at 6:02 PM

Thanks for reporting this issue. We appreciate your feedback and are constantly trying to improve our products based on it. However at this stage we are unable to address this issue due to other high priority items for the team.

Best Regards,
Visual C++ Team
Posted by Microsoft on 3/22/2010 at 11:21 PM
Thanks for your feedback.

We are rerouting this issue to the appropriate group within the Visual Studio Product Team for triage and resolution. These specialized experts will follow-up with your issue.

Thank you
Sign in to post a workaround.