Some 3D format, problem when importing/exporting process if contains Non-latin

[tkbd]

In current Wings3D v2.2.2.
I found some problems when import non latin file-name/material-name/texture-name.
These caused by a string that has Non-Latin... diacritical-marks (ex:ÁắǼąĄǽĈ) and/or CJK(ex:汉书한어漢字あ) ... characters.
Diacritical-marks are using French ,Türkçe,Türkçe,Deutsch,Español,Svenska and Italiano.

I checkd the problem happen Windows10 and MacOS.
(Sorry,I couldn't check it on Linux,but they probably will occur.)

I attached zip file that contains 2 wings test files and screenshots.
,and intentionally named various items to clarify encoding bugs. See screenshot in zip file.
(Data is set in Object/Light/NormalMap/RooughnessMap/BaseColorMap/Material/Texure/fileName)

These can be categorized into 3 types. Crash when file export,Texture-file-linking is broken, Mojibake
The former 2 are both diacritcal-mark and CJK characters. Mojibake is occur only when encode CJK characters.

( NOTE: To recreate the troubles , Use the test files and Each 3D formats files that exported from them. )

List of the troubles
--Crash---
*Open wings file,and immediately crash-log output for pick material name with Non Latin characters.
*Lightwave/modo exporter can't work and output crash log
*VRML2.0 exporter can't export file (3D data isn't export)

--Texture linking broken--
*3ds and obj importers can't with texture image files that contains Non-Latin string(It isn't appear in OutLiner,and not display in 3D geometry)
--Texture linking broken for sub folder(obj importer only) --
*obj importer can't link texture image files that reference to subfolder path (It isn't appear in OutLiner,and not display in 3D geometry)

--Mojibake--
*imported 3ds data is mojibake-ed (The file saved with UTF16 encoding by Wings3D ,but Wings3D dosen't import the decode it perfectly)
*imported obj data is mojibake-ed

FYIiacritical mark ,CJK characters, Mojibake
https://en.wikipedia.org/wiki/CJK_characters
https://en.wikipedia.org/wiki/Diacritic
https://en.wikipedia.org/wiki/Mojibake

FYI:3DS Max help -internal data encoding is utf16-
https://help.autodesk.com/view/3DSMAX/20...8E89F5_htm

This verification in many file formats takes a lot of time and effort.
So I can provide the files and information that was resource exported at this time for developer.
Please contact me if you are interest this problem.
(I have crash-logs and sample-data exported to each 3D format by Wings3D.)


---------------------------------------------------------------------------
Best regards

  0_original_wings_file.zip (Size: 111.78 KB / Downloads: 2)

[dgud]

3ds can't be utf-16, if wings expect that all old files will fail to load,
we would get Mojibake for every old-file which uses ascii or utf-8.
3ds Max may use it internally now but that doesn't help with an old file-format that expects ASCII.

The encoding for strings in 3ds, obj and lwo is latin-1/ascii encoded not UTF-8.
You may get it work with UTF-8 because UTF-8 is designed to work for ASCII,
but then you will break Latin-1 encoded files.

Wings could guess on import which encoding was used but that is still just a guess,
and when doing export the only correct way is to encode in Latin-1 for old formats that have not specified the encoding format.

So if you want to use old file-formats stay with ASCII characters, that is the only thing that will work.
If you need to Unicode characters use newer formats with a specified encoding.

[tkbd]

dgud,Thank you for reply.
Especially,The encoding for file formats is useful for me.
I surprised that obj file format latin only support.
I have met terrible utf8 data in an obj 3D file saved by Photoshop(Japanese ver) before.
OK,Basically to treat for 3D data,It only use ascii character ,but it use newer format that Collada and/or glTF if using utf8 string need.
I agree for your advice that how to naming of files and treat about Mojibake.

By the way,Is Texture file linking problem also deeply relation to the file format encoding?
I wish that at least the texture link will be solve if possibility...
Even Mojibake occurring, if the texture linking successfully, it will be solved by an User could renaming the texture name to the latin string on Outliner.

[dgud]

obj and 3ds are really old formats made before Unicode was used anywhere.

It is just that I don't want to spend several hours/days to fix things that never was supposed to work.
It is hard to test and get every corner case correct on all OS'es, and I do have more fun features to do.

Wings saving and loading should of course work.