Subversion contains filenames with utf-8 encoding

Continua CI Bugs
1

ContinuaCI cannot handle UTF-8 characters in filenames.

Is there a way to set command options on hg.exe ?


Regards
Hans

2

Hi Hans,

Can you post the steps to reproduce this? I just added a file named Völlerei lässt grüßen.txt to our test SVN repository and the build ran without error. Which version of ContinuaCI are you running? Are there any errors in the event log? It will also help us to investigate this quicker if you send a debug log to support@finalbuilder.com

3

Hi 
Inside the attached zip-File you will find the kind of files we have in the repository.
Could you make a try with the cyrillic filename, this kind of files causes errors when ContinuaCI wants to retrieve the repository.

Regards
Hans

__32773__0__Files.zip (470 B)

4

Hi Hans,

Thanks for sending these examples. We’ve reproduced the issue here. There is no simple way to get Mercurial to accept the Cyrillic characters in the filename. We have however worked out a possible solution based on an extension called fixutf8. We’ll need to do some work to integrate this with Continua and ensure that it does not break existing repositories, so it may be a few days before we can release an update.

5

Hi

After more than 2 years I retried again to migrate our FinalBuilder Server projects to Continua with the latest version.

I get still the same issues with utf-8 encoded filenames.

Is there a solution under way ?

Regards,
Hans

6

Hi Hans

We are acutely aware of this. The issue is with our use of Mercurial as a repository cache and tranport mechanism (between server and agents), mercurial just doesn’t deal with file name encoding properly on windows (because windows uses ansi or utf-16). There is some work underway by the mercurial dev team - https://www.mercurial-scm.org/wiki/WindowsUTF8Plan but it’s progress is slow. I have had a look at the mercurial source to see if we could contribute, however our python experience here is rather limited, and the mercuial code is quite complex.

I notice that we are still shipping 3.9.2 and the latest is 4.2.1 so we’ll do some testing and see if anything has improved.