extended ascii characters in filenames not working with windows

I'm stumped with this one. It seems like something to do with Unicode conversion or some other character conversion, but I could be wrong. I couldn't find any specific code that filters out "bad ASCII characters" as such...?

Help required! :)

The DELE command is failing at line 1690 in wzd_file.c (SVN 1656) due to:

  ret = unlink(filename);
  if (ret==-1) {

The value of filename in my test was: C:/wzdftpd-dbg/ftproot/öooo.txt" (const char *)

unlink() is returning -1 when called with non-ASCII values in the filename.

errno is being set to ENOENT (0x2) which means the file or path is not found or the path specified a directory.

Note that my last message is wrong... the problem is not with unlink() but rather with the directory listing showing öooo instead of öooo.

I'll try to keep tracking this bug down to its source!

Ok traced this problem back to:

  ret = WideCharToMultiByte(CP_UTF8, 0, dir->entry.cFileName, wcslen(dir->entry.cFileName)+1, filename, sz, NULL, NULL);

Line 224 of fs_dir_read in wzd_fs.c (SVN 1656).

This command is converting öooo to öooo which is bad!

I'm not sure why it is converting codepages/Unicode characters/etc? What is in the directory is what you need to send in the directory listing as-is... so why is this function even needed?

Some more info at:

• http://blogs.msdn.com/michkap/archive/2005/04/18/409095.aspx
• http://msdn.microsoft.com/library/default.asp?url=/library/en-us/intl/unicode_2bj9.asp

This is needed, because the server must be able to speak UTF-8.

The fix is to wrap the unlink() function, so the windows part of the function use the wide version of unlink (something like _wunlink() )

See http://msdn2.microsoft.com/en-us/library/1c3tczd6(VS.80).aspx

The problem I'm talking about is directory listings. The directory listing transfer is actually sending the wrong filenames to the client, which thinks it is a normal 8 bit character set with no fancy encoding.

For example, FlashFXP shows öooo as the filename instead of what it is on the disk as öooo.

So I still believe the WideCharToMultiByte?() function is wrong in its current state (?)

You have to check if the client asks for UTF-8 (sends the command OPTS UTF8 ON during login sequence). The sequence 'öooo' is a valid UTF-8 encoding, displayed in iso-8859

There are 2 options:

• the string may be double-encoded (easy to check)
• flashfxp thinks it is iso-8859, while the server sent UTF-8

As discussed on IRC, this is a major architectural issue in wzdftpd, I'm rolling this milestone waaayyyy forward :)

Also see #90 which is a similar issue, merged into this ticket now.