Unicode issues with WriteFile and in the CRT - by Nico Rieck

Status : 

  Won't Fix<br /><br />
		Due to several factors the product team decided to focus its efforts on other items.<br /><br />
		A more detailed explanation for the resolution of this particular item may have been provided in the comments section.

Sign in
to vote
ID 543801 Comments
Status Resolved Workarounds
Type Bug Repros 6
Opened 3/22/2010 2:28:38 PM
Access Restriction Public


I believe there is an issue with WriteFile [1]. Either in it's implementation, or its documentation is lacking.

WriteFile is supposed to return the number of _bytes_ written. Taking the same input, the returned value for number of bytes written can be different for different codepages. And when writing UTF-8 data with multibyte characters to stdout it seems to return the number of _characters_ written.

For example when writing a char* buffer with value { 0xC3, 0xA4, 0 } (character ä encoded as UTF-8) WriteFile returns the following values:

With console output codepage 850:
- Output: +ñ
- Bytes written: 2 (correct, and output is obviously broken)

With console output codepage 65001:
- Output: ä
- Bytes written: 1 (incorrect since "ä" in UTF-8 takes 2 bytes, but output is correct)

This behavior results in issues in Microsoft's C Runtime implementation. Several functions (like fflush) are required to verify that the number of bytes written is equal to the number of bytes in the input. The behavior of WriteFile triggers those checks and output streams are flagged with _IOERR, thus breaking programs that verify streams with ferror().

All tests were done on a German Windows 7 Professional (64-bit) using 32- and 64-bit test programs.

MSVCR90.DLL: 9.0.30729.4926
KERNEL32.DLL: 6.1.7600.16385
cmd.exe: 6.1.7600.16385

Also please refer to this thread [2] in the Visual C++ General forum where the CRT behavior was confirmed for VS 2010 RC.

  [1]: http://msdn.microsoft.com/en-us/library/aa365747%28VS.85%29.aspx
  [2]: http://social.msdn.microsoft.com/Forums/en-US/vcgeneral/thread/e4b91f49-6f60-4ffe-887a-e18e39250905
Sign in to post a comment.
Posted by Microsoft on 3/27/2011 at 6:02 PM

Thanks for reporting this issue. We appreciate your feedback and are constantly trying to improve our products based on it. However at this stage we are unable to address this issue due to other high priority items for the team.

Best Regards,
Visual C++ Team
Posted by Microsoft on 3/22/2010 at 11:21 PM
Thanks for your feedback.

We are rerouting this issue to the appropriate group within the Visual Studio Product Team for triage and resolution. These specialized experts will follow-up with your issue.

Thank you