std::endl template specialization for wchar_t writes a single-byte newline instead of Unicode newline - by Fran Litterio

Status : 

  Fixed<br /><br />
		This item has been fixed in the current or upcoming version of this product.<br /><br />
		A more detailed explanation for the resolution of this particular item may have been provided in the comments section.


1
0
Sign in
to vote
ID 730916 Comments
Status Closed Workarounds
Type Bug Repros 0
Opened 3/13/2012 9:23:59 AM
Access Restriction Public
Moderator Decision Sent to Engineering Team for consideration

Description

At line 1016 of file C:\Program Files (x86)\Microsoft Visual Studio 10.0\VC\crt\src\ostream is this definition of std::endl:

_CRTIMP2_PURE inline basic_ostream<wchar_t, char_traits<wchar_t> >&
	__CLRCALL_OR_CDECL endl(basic_ostream<wchar_t,
		char_traits<wchar_t> >& _Ostr)
	{	// insert newline and flush wide stream
	_Ostr.put('\n');
	_Ostr.flush();
	return (_Ostr);
	}

The call to _Ostr.put('\n') should be _Ostr.put(_Ostr.widen('\n')), because this is a template specialization for type wchar_t, and thus should always write wide-characters to the stream. This may not matter for output to consoles, where mixing wide and single-byte characters is probably not an issue, but when used to output to a file (via fstream, ofstream, or even just via console output redirection), this will result in a file containing a mixture of wide and single-byte characters, which can be problematic when reading the file.
Sign in to post a comment.
Posted by Microsoft on 4/29/2014 at 12:24 PM
Thank you for reporting this issue. This issue has been fixed in Visual Studio 2013. You can install a trial version of Visual Studio 2013 with the fix from: http://go.microsoft.com/?linkid=9832436
Posted by Microsoft on 12/17/2012 at 4:51 PM
Hi again,

We've eliminated these unnecessary overloads, which you'll be able to see in the next release of our C++ Standard Library implementation.

Note: Connect doesn't notify me about comments. If you have any further questions, please E-mail me.

Stephan T. Lavavej
Senior Developer - Visual C++ Libraries
stl@microsoft.com
Posted by Microsoft on 3/14/2012 at 7:21 PM
Hi,

Thanks for reporting this issue. It's currently assigned to me, and I'm keeping it active because I believe these specializations should simply be removed - they aren't mandated by the Standard, and they appear to be relics of the pre-VC10 era when lots of iostreams stuff was separately compiled. However, iostreams is notoriously tricky, and it's too late to mess with this for VC11, so I'll look at it for VC12.

In the meantime, I don't believe that this causes actual harm. By default, writing to wcout actually truncates 16-bit characters to 8-bit characters, which doesn't affect ASCII but is obviously undesirable for Unicode. Using _O_U16TEXT as described by http://msdn.microsoft.com/en-us/library/tw4k6df8.aspx allows wcout and wprintf() to write UTF-16 to the console. When this magic flag is used, endl works properly:

C:\Temp>type meow.cpp
#include <fcntl.h>
#include <io.h>
#include <stdio.h>
#include <iostream>
#include <ostream>
using namespace std;

int main() {
    _setmode(_fileno(stdout), _O_U16TEXT);
    wprintf( L"wprintf: \x043a\x043e\x0448\x043a\x0430\n");
    wcout << L" wcout: \x043a\x043e\x0448\x043a\x0430" << endl;
    wprintf(L"Meow!\n");
}

C:\Temp>cl /EHsc /nologo /W4 /MTd meow.cpp
meow.cpp

C:\Temp>meow > foo.bar

C:\Temp>type purr.cpp
#include <stddef.h>
#include <stdio.h>
#include <fstream>
#include <ios>
#include <vector>
using namespace std;

vector<unsigned char> read_all(ifstream& f) {
    vector<unsigned char> ret;

    const int N = 64 * 1024;

    vector<char> buf;

    while (f) {
        buf.resize(N);
        f.read(&buf[0], N);
        buf.resize(static_cast<size_t>(f.gcount()));
        ret.insert(ret.end(), buf.begin(), buf.end());
    }

    return ret;
}

int main() {
    ifstream f("foo.bar", ios_base::binary);

    const vector<unsigned char> v = read_all(f);

    for (vector<unsigned char>::size_type i = 0; i < v.size(); ++i) {
        if (i > 0 && i % 32 == 0) {
            printf("\n");
        } else if (i > 0 && i % 2 == 0) {
            printf(" ");
        }

        printf("%02X", v[i]);
    }

    printf("\n");
}

C:\Temp>cl /EHsc /nologo /W4 /MTd purr.cpp
purr.cpp

C:\Temp>purr
7700 7000 7200 6900 6E00 7400 6600 3A00 2000 3A04 3E04 4804 3A04 3004 0D00 0A00
2000 2000 7700 6300 6F00 7500 7400 3A00 2000 3A04 3E04 4804 3A04 3004 0D00 0A00
4D00 6500 6F00 7700 2100 0D00 0A00

Here, the "\n" given to wprintf() and the endl given to wcout both emit "0D00 0A00", which is correct.

If you have any further questions, feel free to E-mail me at stl@microsoft.com .

Stephan T. Lavavej
Visual C++ Libraries Developer
Posted by MS-Moderator10 [Feedback Moderator] on 3/13/2012 at 8:19 PM
Thank you for submitting feedback on Visual Studio and .NET Framework. Your issue has been routed to the appropriate VS development team for investigation. We will contact you if we require any additional information.
Posted by MS-Moderator01 on 3/13/2012 at 9:53 AM
Thank you for your feedback, we are currently reviewing the issue you have submitted. If this issue is urgent, please contact support directly(http://support.microsoft.com)