cwRsync - Rsync for Windows

cwRsync - Rsync for Windows

cwRsync is a packaging of Rsync for Windows with a client GUI. You can use cwRsync for fast remote file backup and synchronization.

Rsync uses the Rsync algorithm which provides a very fast method for bringing remote files into sync. It does this by sending just the differences in the files across the link, without requiring that both sets of files are present at one of the ends of the link beforehand. At first glance this may seem impossible because the calculation of diffs between two files normally requires local access to both files.

Rsync normally uses ssh for communication. It requires no special privileges for installation. You must, however, have a working ssh system. Alternatively, rsync can run in `daemon' mode, listening on a socket. This is generally used for public file distribution, although authentication and access control are available.  

Cygwin is a Linux-like environment for Windows. It consists of a DLL (cygwin1.dll), which emulates substantial Linux API functionality, and a collection of tools.

 

Itefix provides a virtual test lab in where you can try our products with full functionality. At the moment we have 7 hosts (Windows 7, Windows 2008 R2, Windows 2012 R2, Windows 10, Windows 2016, Windows 2019 and Lubuntu 16.04 for Unix scenarios). Our lab has no Internet access. All lab computers are refreshed every three hours.

Itefix software is available via a URL on the desktop. You have also access to some popular 3rd party software to test client scenarios. Download the software to lab computers which share the same network, so it is possible to experiment with multiple hosts. A network diagram showing the connectivity is available via the URL.

Our lab has also a Lubuntu 16.04 desktop to test Unix/X windows scenarios. In the case you need to login via ssh from an other test computer, you can use following credentials: user lab1user, pwd user1lab

cwRsync - Rsync for Windows
USD 15.00

The product contains client (with a GUI) or server installers (64-bit). The rsync binary provided has following convenient patches: transliterate, timelimit, ignore case, no strict check of password file permissions (unofficial). You may consider our kit cwRsync server + Copssh if you want to set up secure Rsync servers.

Quantity is the number of computers to install the product.

If you want to initiate rsync connections from your computer, you need cwRsync client. If you want to accept rsync connections from other computers, you need cwRsync server.

Installation

Supported platforms: Vista/2008(R2)/7/8/2012(R2)/10/2016

cwRsync comes as a zip archive containing a Nullsoft Installer package. Unzip downloaded file and run cwRsync_x.x.x_Installer.exe or cwRsync_Server_x.x.x_Installer.exe (server version) :

  1. Click Next at Welcome-page
  2. View license agreement.
  3. Select components that varies depending on package type: Client package has an optional component (Secure Channel Wrapper & Wizard). It makes creation of secure channels to cwrsync servers an easy task.
  4. Specify an installation location.
  5. (cwRsyncServer only) Specify a service account.
  6. Installation starts. By clicking 'Details' button, you can get more detailed information about installation. Check if everything seems ok.

You're DONE! cwRsync is installed on your machine.

cwRsync server only

You should update the configuration file according to your needs. If you have defined uploadable modules (read only = false), you have to make sure that Rsync Server has correct permissions on directories behind those modules. You can use Prep a Dir for Upload wizard from the start menu for that purpose.

cwRsync installs rsync server as a service (RsyncServer) with following parameters :

--config rsyncd.conf --daemon --no-detach

Default start mode is manual.

 

Usage

a client GUI with the following features is available from the start menu:

  • Specify sources and destination (local, remote ssh and daemon modes)
  • Specify filters and a subset of options
  • Create rsync command according to your specification
  • Run rsync on the fly both in production and test mode
  • Create batch files for task scheduling
  • Save and load your favorite rsync settings as profiles
  • Language localization

Client installation includes also a batch file starting rsync in a right environment. You can customize it according to your needs. Man pages in html format are also available from start menu. cwRsync FAQs from this site can also be helpful.

In addition, as rsync people suggest, you can try Google for resources :-) I give also some pointers to rsync website :

cwRsync client is too slow!

Try to remove /etc/fstab and use the option --no-perms  option instead. The problem can also be related to real-time anti-virus scanning. Consider file/folder exclusion if possible.

I can't use long path names

FAQ

From Windows Developer network:

“In the Windows API (with some exceptions discussed in the following paragraphs), the maximum length for a path is MAX_PATH, which is defined as 260 characters. A local path is structured in the following order: drive letter, colon, backslash, name components separated by backslashes, and a terminating null character. For example, the maximum path on drive D is "D:\some 256-character path string<NUL>" where "<NUL>" represents the invisible terminating null character for the current system codepage. (The characters < > are used here for visual clarity and cannot be part of a valid path string.)

The Windows API has many functions that also have Unicode versions to permit an extended-length path for a maximum total path length of 32,767 characters. This type of path is composed of components separated by backslashes, each up to the value returned in the lpMaximumComponentLength parameter of the GetVolumeInformation function (this value is commonly 255 characters). To specify an extended-length path, use the "\\?\" prefix. For example, "\\?\D:\very long path".”

 

Even if the underlying APIs may have support for really long paths, the problem occurs due to user interface tools like Windows Explorer, Cmd prompt and PowerShell which don't support Unicode variants of the Windows APIs. The result is then path names up to 256 chars. As of version 1.7, Cygwin uses unicode Windows APIs, you can use tools like bash shell or nano editor for management of files/paths with long names.

Log example for cwRsync transfer with long path names:

...
2014/03/25 22:08:42 [1784] >f+++++++++ backup/0123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789
/0123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789
/0123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789
/0123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789/testfile
...

How can I pull files from windows to linux/unix within a secure network ?

Assuming that you want to pull directory c:\backup on Windows machine to /var/backup on Linux machine:

On Windows machine :

  • Install Cwrsync Server
  • Start RsyncServer service
  • Add a new module to the configuration file rsyncd.conf :

[backup]
path = /cygdrive/c/backup
read only = true
transfer logging = yes


On Linux machine :

  • Use the command below to initiate rsync:

rsync -vrt windows_machine::backup /var/backup


NB! Keep in mind that the recipe above is suitable for operations within a secure network.

How can I make pre-exec and post-exec work ?

As of version 2.6.7, rsync server can kick scripts before and after transfers. You can use the recipe below to implement this very handy functionality in cwrsync server:

  1. Install Copssh and Cwrsync Server.
  2. Start a windows command shell from cwrsync start menu and make a copy of bash.exe as sh.exe in the bin directory.

    copy bin\bash.exe bin\sh.exe
  3. Create two script files in the bin directory for pre-exec operations:

    c:\program files\ICW\bin\pre-exec.sh



    # Shell script for your pre-exec operations
    cmd /c "c:\program files\ICW\bin\pre-exec.cmd"
    exit 0




    c:\program files\ICW\bin\pre-exec.cmd




    @ECHO OFF
    .....
  4. Create two script files for post-exec operations as described above.
  5. Update your rsyncd.conf file:




    # Module definitions
    [test]
    path = .....
    ......
    pre-xfer exec = /bin/pre-exec.sh
    post-xfer exec = /bin/post-exec.sh

Can cwRsync copy files in use ?

NB! The recipe below applies to Windows 2003 Server only.
 

I recently had the issue of running rsync to backup up files that are "in use". A good example was backing up documents and settings and the ntuser.dat file would always fail.along with several other system locked files.

Resolution was to download the VSS SDK from Microsoft's web site. The SDK has an executable vshadow.exe which will allow you to create volume shadow copies that are mapped to a drive, allowing rsync to use the VSS drive where no files are locked.

 

An example command set would be:


rem create a permanent shadow copy
vshadow.exe -p -nw -script=vars.bat
call vars.bat
rem drive H: is assigned to the permanent shadow copy created
vshadow.exe -el=%SHADOW_ID_1%,H:
rem perform your rsync operation, use drive H: (shadow drive) as source
rsync.exe -ruztiv '/cygdrive/h/xxxxxxx' myrsyncserver::myrsyncuser
rem remove the permanent shadow copy
vshadow.exe -ds=%SHADOW_ID_1%

 

-- Contributed by Rob Bosch


You may also visit this link for more detailed information.

How can I install cwRsync silently ?

Use switch /S during installation:

 

cwRsync_x.x.x_Installer.exe /S

cwRsync_Server_x.x.x_Installer.exe /u=user /p=password /S

 

where user/password specify the service account credentials

 

You can also specify a new installation directory:


cwRsync_x.x.x_Installer.exe /S /D=C:\test\cwrsync

cwRsync_Server_x.x.x_Installer.exe /u=user /p=password /S /D=C:\test\cwrsync_server

Why is it called cwRsync?

Use of vocals would be inappropriate (!) as the word rsync has no vocals in it, and this package is also dependent on cygwin:

 

CYGWIN + RSYNC

How can I secure connections between windows rsync clients and cwRsync servers ?

**UPDATED 17.03.2014 **

Assuming that you want to synchronize c:\documents and settings\user on WINCLIENT to c:\backup on WINSERVER :

On WINSERVER:
 

  • Install Copssh and cwRsync server. Use the same service account.
  • Start RsyncServer and OpenSSHD services
  • Activate a user and create a PKA key pair via Copssh control panel:

Create a PKA key pair with empty passphrase via Copssh Control Panel

 

  • Transfer user's private key file to WINCLIENT.
  • Make sure that the user and the service account has write access to c:\backup
  • Add a new module to rsyncd.conf :

[backup]
path = /cygdrive/c/backup
read only = false
transfer logging = yes

 

on WINCLIENT :

  • Install cwRsync with Secure Channel Wrapper
  • Start Secure Channel Wizard and specify private key file location, WINSERVER and rsync command:

cwRsync Secure Channel Wizard - screen 1

 

  • Click next and replace SvcwRsync by the user activated at the server side via Copssh Control Panel:

cwRsync Secure Channel Wizard - screen 2

  • Save contents as a batch for further operations and/or scheduling.


NB! You must forward ssh port 22 to WINSERVER at the edge of your secure network (your router/firewall). I recommend strongly to tighten security further by using options available in rsync and openssh (host limitations, secrets file ...).

Can rsync transfer security/ownership information between two windows machines ?

cwRsync itself has no support for it. However, after an rsync operation, you can use Robocopy (standard in more recent Windows versions), to transfer all windows specific file information (NTFS security, timestamps, attributes, ownership, auditing info):


ROBOCOPY source destination /XO /XN /XC /E /COPY:ATSOU

 

options /XO (exclude older), /XN (exclude newer) and /XC (exclude changed) assure that only existing files are targeted. Option /E is for recursive directory operation including empty ones. Option /COPY:ATSOU instructs Robocopy to copy only attributes (A), timestamps (T), NTFS security information (S), Ownership O) and auditing (U).

How can I use cwRsync between two windows machines within a secure network ?

Assuming that you want to synchronize from directory c:\srcdir on machine WINCLIENT to directory c:\destdir on machine WINSERVER :

On WINSERVER :

  • Install cwRsync Server
  • Start RsyncServer service
  • Make sure that the service account has write access to c:\destdir. You can use Prep a Dir for Upload wizard from start menu.
  • Add a new module to rsyncd.conf :

[wintest]
path = /cygdrive/c/destdir
read only = false
transfer logging = yes

 

On WINCLIENT :

  • Install cwRsync client.
  • Use the command below to initiate rsync communication:

"c:\Program Files\cwRsync\bin\rsync" -av /cygdrive/c/srcdir/ WINSERVER::wintest


NB! Keep in mind that the recipe above is suitable for operations within a secure network.

How can I secure connections between linux/unix rsync clients and cwRsync servers ?

Assuming that you want to synchronize from directory /home/user on *nix machine to directory c:\backup on Windows machine :

On Windows:
 

  • Install Copssh and cwRsync server. Use the same service account.
  • Start RsyncServer and OpenSSHD services
  • Activate a user and create a PKA key pair via Copssh control panel:

Create a PKA key pair with empty passphrase via Copssh Control Panel

 

  • Transfer user's private key file to Linux client.
  • Make sure that the user and the service account has write access to c:\backup
  • Add a new module to rsyncd.conf :

[backup]
path = /cygdrive/c/backup
read only = false
transfer logging = yes

On Linux:

  • Use shell script below after having it updated according to your needs:
#!/bin/bash
# Secure Channel Wrapper for connection to cwRsync servers from Linux/Unix clients
# v1.0 - Initial version, Sep 2005, Tevfik K., http://itefix.no

# Customize variables below according to your needs

# identity: private key file for user winuser
identity='winuser.key'

# localport: local port for forwarding
localport=9119

# remoteport: termination port (this should be the port rsync daemon listens to)
remoteport=873

# remotehost: cwRsync Server name/ip-adress
remotehost=192.168.2.26

# your rsync module at server side
rsyncmodule=backup

# Function to terminate secure tunnel processes
TerminateTunnel ()
{
ps ax | grep "ssh -i $identity -L $localport" | awk '{print $1}' | xargs -i kill {} 2&>/dev/null
}

# Clean up ... terminate zombie tunnels
TerminateTunnel

echo Establishing secure channel ...
ssh -i $identity -L $localport:127.0.0.1:$remoteport winuser@$remotehost -T -N -f

# your rsync command, you can edit:
# - rsync options (-av)
# - source files (temp/)
# - exchange source and destination
# - and more :-) Try and tell me!
rsync -vrt /home/user/ rsync://winuser@localhost:$localport/$rsyncmodule

echo Terminating secure channel ...
TerminateTunnel


NB! You must forward ssh port 22 to Windows machine at the edge of your secure network (your router/firewall). I recommend strongly to tighten security further by using options available in rsync and openssh (host limitations, secrets file ...).

How can I change port number used by rsync server ?

Rsync server uses port 873 by default. You can add port directive to the configuration file rsyncd.conf to specify an alternative port.

 

port new-port-number

 

Don't forget to restart the service.

I want to set up ssh communication without passwords !!

Here is a simple recipe ( NB! This is not a very secure solution, since we use null passphrase during generation of public key pairs, use it at your own risk):

  • cwRsync client. Generate key pairs :

ssh-keygen -q -t rsa -f cwrsync -N ""


Files cwrsync and cwrsync.pub will be generated.

  • Transfer cwrsync.pub to your *nix machine.
  • *nix machine. Run following commands:

mkdir -p ~$user/.ssh
cat cwrsync.pub > ~$user/.ssh/authorized_keys
chmod 755 ~$user ~$user/.ssh
chmod 644 ~$user/.ssh/authorized_keys
chown $user ~$user ~$user/.ssh ~$user/.ssh/authorized_keys

  • cwRsync client. Update your rsync commands in your batch file by specifying option -e "ssh -i cwrsync".

How can I secure connections between cwrsync clients and unix/linux servers ?

cwRsync has all ssh binaries you need to establish ssh communication.


Use batch file example in the cwRsync client package to set up communication correctly.

Why does my cwrsync try to load ssh?

It's trying to fire up ssh because your target has a single colon in it.

server:/path/to/stuff means "fire up ssh or rsh and make me a tunnel to
server"

server::module/path/to/stuff means "try to access an rsync daemon on
server and access path/to/stuff on module"

/path/to/stuff means "access /path/to/stuff on the local machine"

Hope that helps.

by Jim Salter

Rsync does not recognize windows paths in a correct manner !

Windows paths may contain a colon (:) as a part of drive designation and backslashes (example c:\, g:\). However, in rsync syntax, a colon in a path means searching for a remote host.

Solution: use absolute path 'a la unix', replace backslashes (\) with slashes (/) and put -/cygdrive/- in front of the drive letter.

Example :

 

C:\WORK\*

    • -->

/cygdrive/c/work/*

Problem with secrets file

If you're setting up cwrsync to run as a Service on a Windows machine ensure the .conf file has the line:

strict modes = false

otherwise rsync passwords will not work in Windows.

Can cwRsync co-exist if cygwin is already installed ?

YES: Cygwin 1.7 binaries used by Cwrsync allow co-existence of multiple Cygwin implementations.

Permissions on files/directories are cluttered/mixed up !

If you run into permissions problems or your directories' security ACLs are populated by some unwanted groups/users, you need to make sure that:

  • File ../etc/fstab  exists with at least the content below (with Unix line endings even if the file has only one line!):

none /cygdrive cygdrive binary,posix=0,user,noacl 0 0
  

That will instruct Cygwin not to touch permissions.

 

NB! You may experience very slow transfer rates and/or building the file list may take long time. You can remove /etc/fstab and use the option --no-perms  option instead. The problem can also be related to real-time anti-virus scanning. Consider file/folder exclusion if possible.

 

More information:

The Cygwin Mount Table

The cygdrive path prefix

How can I use rsync to pull files from windows to linux/unix within a secure network ?

Assuming that you want to pull directory c:\backup on machine WIN_SOURCE to directory /var/backup on machine LINUX_DEST :

On WIN_SOURCE :

- Install cwRsync Server (without openssh component)
- Start RsyncServer service
- Add a new module to rsyncd.conf :

[backup]

 path = /cygdrive/c/backup

 read only = true

 transfer logging = yes


On LINUX_DEST :

- Use the command below to initiate rsync:

  • rsync -av WIN_SOURCE::backup /var/backup


NB! Keep in mind that the recipe above is suitable for operations within a secure network.

This package contains components with following terms of licensing:

Component Version Licensing
Rsync

3.1.3

Rsync license (GPL) Source code for Rsync is available here.
Cygwin and GNU tools 3.0.7 CYGWIN license (GPL) Source code for Cygwin and GNU Tools are available here.
OpenSSH 8.0p OpenSSH license
SSL Library LibreSSL 2.9.1 Apache license 1.0, 4-clause BSD LicenseISC license, and some are public domain
cwRsync

6.0.0

Itefix EULA

cwRsync client free edition contains a barebone distribution of Rsync for Windows. Unzip archive contents to a directory and update the supplied batch file cwrsync.cmd. That's all you need to be able to initiate rsync from your computer. cwRsync FAQs or user forum archive are available for help.

 

Check our product edition if you want rsync server, useful patches, 64-bit binaries, client GUI, extended installer features, upgrade protection and support