rsync: File name too long (91)

Hi,

I have the same problème... But it's very important in my project to find a solution for those files...  What are the way to solve this limitation ? I need to rebuild a new version of the cygwin dll, there is something else to do ?

Thanks in advance for your help/advice ! 

Teddy

I have tried upgrading to the newest cygwin (1.5.24-2) to no avail.  Has anyone devised a workaround for this problem?  I am also stuck on this as the windows machine has 1000s of very long paths and file names that rsync will give "rsync: readlink  (in test) failed: File name too long (91)"

For cygwin to use longer path names, the work is partially done.  Cygwin needs to use the unicode win api calls.  See here for a patched version does that:

  http://www.okisoft.co.jp/esc/utf8-cygwin/

Then, a couple of other things need to be changed.  Here is a quote from an email conversation with the author of the cygwin-utf8 patches:

(12/7/2006 7:11 PM)
 Wayne wrote:

    Thanks for your work on the UTF-8 patches.  I am using your cygwin1.dll for rsync backups.  I hoped it would bring the advantage of longer paths, but I am getting ENAMETOOLONG - "File name too long (91)" errors when using Rsync.  Using the wide (unicode) win32 api calls brings another advantage besides supporting special characters.  Many of the unicode (wide) functions support a 32000 character path length instead of 260 characters.

Note what the following page says:
  https://msdn.microsoft.com/en-gb/library/aa365247.aspx

<BEGIN QUOTE>

Maximum Path Length

In the Windows API, the maximum length for a path is MAX_PATH, which is defined as 260 characters. A path is structured in the following order: drive letter, colon, backslash, components separated by backslashes, and a null-terminating character, for example, the maximum path on the D drive is D:\<256 chars>NUL.

The Unicode versions of several functions permit a maximum path length of approximately 32,000 characters composed of components up to 255 characters in length. To specify that kind of path, use the "\\?\" prefix.

Note  The maximum path of 32,000 characters is approximate, because the "\\?\" prefix can be expanded to a longer string, and the expansion applies to the total length.

For example, "\\?\D:\<path>". To specify such a UNC path, use the "\\?\UNC\" prefix. For example, "\\?\UNC\<server>\<share>". These prefixes are not used as part of the path itself. They indicate that the path should be passed to the system with minimal modification, which means that you cannot use forward slashes to represent path separators, or a period to represent the current directory. Also, you cannot use the "\\?\" prefix with a relative path. Relative paths are limited to MAX_PATH characters.

When using the API to create a directory, the specified path cannot be so long that you cannot not append an 8.3 file name.

The shell and the file system may have different requirements. It is possible to create a path with the API that the shell UI cannot handle.

<END QUOTE>

It would be nice if you include in your patch the advantage of longer path names.  Items that would have to be considered:

    1) Where does cygwin use the windows MAX_PATH constant
    2) How is CYG_MAX_PATH used?  Can it be changed to something longer?
          (rsync uses #define MAXPATHLEN 1024)
    3) If there is the possibility of handling relative paths, these must be transformed to absolute.
    4) prepend "\\?\" to the path names before you call the wide api calls (such as before calling CreateFileW)

What do you think of this idea?

It is very interesting.

    I am sorry for my late response.  As for handling path
names in Cygwin, you should look into "class path_conv" in
winsup/cygwin/path.h first.  Its private member "char path"
holds the path name to be processed.  It represents both a posix (Unix)
path name and a Windows path name, which is why it is difficult to
incorporate Unicode into the middle of cygwin1 and I have taken a wrapper
approach.

    Just redefining CYG_MAX_PATH and modifying the wrapper functions
may be sufficient to handle long path names.  But I do not have much time
to try it out for now.  I am hoping you will try it for yourself.  The
patch at http://www.oki-osk.jp/close/osk/index.html is open for everyone.
To compile it:

$ tar xf cygwin-1.5.22-1-src.tr.bz2
$ cd cygwin-1.5.22-1/winsup
$ bzcat ~/winsup-utf8-patch-08-09.diff.bz2 | patch -p0
$ cd ..
$ ./configure
$ make

You will get new-cygwin1.dll in i686-pc-cygwin/winsup/cygwin.  This is
explained in Japanese at
http://www.oki-osk.jp/close/osk/index.html
and I hope you read the command lines there.

Best Regards,

Wayne 

Hi *all

I'm French, so my english is not very good :-(

Thx for this very good work...

I have the same problem but I don't understant how to resolv it.

 I use cwRsync_2_1_0 on WinXp pro.

I tried to replace cygwin1.dll by http://www.okisoft.co.jp/esc/utf8-cygwin/ version but no result :(.

 does it exist a full win32 install which take into account this patch ?

Thx for your help

Regards 

Hi,

I use cwRsync every day and I have a lot of "File name too long" and "File has vanished" errors. I need to find a solution, but I don't have suifisents skills to solve the problem... So, I propose to pay something to the person he solve this problem ! It's not a joke !

Who has a solution ?

Ps: sorry for my english...

Teddy

Hi. Just wondering before I spend time on this solution if it works? I think this is a serious issue that
needs an easy solution because it renders rsync useless if you have lots of long filenames because your backup wont be right.

My biggest issue is I am using rsync to backup my CPanel folders and the mail files are too long. Its important I make a backup of this mail. I can download these files from my Linux to Windows server using FTP and the filenames so not an issue, it must be with rsync.

thats a shame. I have had to buy a linux box and use built in linux rsync for my backups due to some files not being copied by cwrsync. I'll keep an eye for a new version that may address this issue though. thanks.

One can get a Beta Version of Cygwin 1.7.0 with rSync 3.0.2 from my Server.

Only a beta with no warrenty - but supports long file names!

Best regards

Michael Wallner

I have downloaded the aeler version, I have substituted all the dll files into cwrsync folder, nor it work for long name but i have problem about some particular chars like à ò ù.

I have tried to set --iconv version option in some mode but I can't resolve this .

i use windows xp german as client and ubuntu 8.04 as server without any problem. (à,ù and so on)

remember - this is only the rsync client for windows - no server! (as i use only linux and no windows backup server) 

rsync = plain Rsync 3.0.2 with default iconv setup (--enable-iconv=yes)

there is a 'bug' in the rsync.bat file:

change: set PATH=%PATH%;C:\rsync\bin

to: set PATH=C.\rsync\bin;%PATH%

and use the path to your installation!

Here is my batch file:

set CYGWIN=nontsec nodosfilewarning
set PATH=D:\rsync_new\bin;%path%
pause
del rlog.txt 
rsync -rthvR --progress --stats --delete-after --modify-window=5 --log-file=./rlog.txt --files-from=./copy.txt --exclude-from=./nocopy.txt -e  'ssh -i X:/id_rsa' / desktop@192.168.1.65:incoming/ 

My backup configuration is backuppc software on linux debian and some windows machine that i want to backup, so my linux machine call by rsync client the windows machine to cwrsync service and copy all the file to linux server.

The two problem are the long files name and special character like è à,  how can i resolve this problem ? I have tried the binary patch but the problems remain.

Very thanks

my configuration is:

Server: Ubuntu 8.04 with Charset: en_US.UTF-8

Client: Windows XP PRO with Charset: Unicode

Filename: ĐURO ĐAKOVIC...

rsync works with this bat-file

set CYGWIN=nontsec codepage:utf8
set LC_CTYPE="C-UTF-8"
set PATH=d:\rsync_new\bin;%PATH%
rsync -rvcRe 'ssh -i x:/id_rsa_test' /cygdrive/d/test test@192.168.1.5:incoming/
pause 

File-Names will look a little bit strange in Linux (ÄURO ÄAKOVIC ....) - but one can use rsnc in both directions and the filename will be correct in windows!
So if you use rsync for backup and restore, i see no problem.

best regards,

michael 

yes , it works , but I should run rsync client on the windows machine , instead I need to call from linux to windows because I have backup software on linux machine that call rsync service on windows , so It start all from linux and windows should be only listen

Can I do this rsync client as windows service ?

Try to replace the current cygwin dlls in your /bin directory with the new one.

Replace your rsync.exe with the new one (should work as demon also)

try it out and

tell us about your experience...

(i use copSHH with new cygwin dlls and the new rsync.exe (client mode) without any Problem.)

Best Regards,

Michael

p.s. set puTTY / windows/Translation to UTF-8 and all Filenames will be displayed correct 

It seems to works ..., about first test I updated only dll files and It worked not very well, now I have updated rsync.exe too and it seems works. I will confirm you by some next backup.

For charset I have set into rsyncd.conf on windows machine, charset = iso-8859-1 and on the linux machine I have set rsync iconv option as --iconv=utf-8,iso-8859-1

Michael, I tried

to solve long file names and it worked. Many thanks for solving this intractable problem

Can I just get your advice on also solving transmission of chinese characters. Do I just add

set CYGWIN=nontsec codepage:utf8
set LC_CTYPE="C-UTF-8"
to the bat file?

Many thanks

Denys

PS Any problems reported with your long file name version?