linux – rsync with –remove-sent-files选项和打开文件教程

我需要每分钟将记录的文件从3台服务器复制到一台数据存储器.我不需要保存原始文件 – 数据处理不在所有文件中.

但是当我使用选项–remove-sent-files时,rsync会发送并删除未完成(未关闭)的文件.

我试图阻止使用lsof和–exclude-from发送这些打开的文件,但似乎rsync并没有在exlude列表中取消完整路径：

--exclude-from=FILE     read exclude >>patterns<< from FILE

lsof | grep /projects/recordings/.\\+\\.\\S\\+ -o | sort | uniq
/projects/recordings/<uid>/<path>/2012-07-16 13:24:32.646970-<id>.WAV

所以,脚本看起来像：

# get open files in src dir and put them into rsync.exclude file
lsof | grep /projects/recordings/.\\+\\.\\S\\+ -o | sort | uniq > /tmp/rsync.exclude
# sync without these files
/usr/bin/rsync -raz --progress --size-only --remove-sent-files --exclude-files=/tmp/rsync.excldude /projects/recordings/ site.com:/var/www/storage/recordings/
# change owner
ssh [email protected] chown -hR storage:storage /var/www/storage/recordings

那么,我可能会尝试另一种工具吗？或者为什么rsync不听exlude？

解决方法:

我不确定这是否对您有所帮助,但这是我的解决方案,只有当前没有写入的rsync文件.我用它来进行tshark捕获,每隔N秒使用-a标志写入一个新文件(例如tshark -i eth0 -a duration：30 -w / foo / bar / caps).注意那个棘手的rsync,包含和排除的顺序很重要,如果我们想要子目录,我们需要包含“* /”.

-G

$save_path=/foo/bar/
$delay_between_syncs=30
while true;
do
 sleep $delay_between_syncs

 # Calculate which files are currently open (i.e. the ones currently being written to)
 # and avoid uploading it. This is to ensure that when we process files on the server, they
 # are complete.
 echo "" > /tmp/include_list.txt
 for i in `find $save_path/ -type f`
  do
    op=`fuser $i`
    if [ "$op" == "" ]
            then
                    #echo [+] $i is good for upload, will add it list.
                    c=`echo $i | sed 's/.*\///g'`
                    echo $c >> /tmp/include_list.txt
    fi
  done

 echo [+] Syncing...
 rsync -rzt --include-from=/tmp/include_list.txt --include="*/" --exclude \* $save_path user@server:/home/backup/foo/
 echo [+] Sunk... 

done