Hi,
To be honest, I’m not sure that FTPMirror is the culprit as we have been having issues with the FTP service of a supplier. However, I just want to check how FTP Mirror action should handle mirroring into a folder tree that contains many files - at the moment, I am seeing just under 1.9 MILLION files in the tree, coming to a total size of 109Gb. Theoretically, if the action had to deal with that many files, should it be able to cope OK?
Obviously, pruning the files would be ideal but I wouldn’t know where to start!
Hi,
We have tested internally with a million files, and a million files with max length filenames. In both cases the mirror was completed successfully. It does however take time. We have found that the time taken is a factor of the ping time between the server and the client.
We have found that the action can do quite a large number of files with recent updates to it, however it is memory bound. We have found that the only reason the action will not complete is due the list of files to copy gets beyond the memory available for the process. In these cases a insufficient memory error will be reported.
Another approach is to setup multiple mirror actions. One way to split these up is on a sub-directory of the main directory. A directory iteration would work to achieve this on a changing file/folder structure.
Thanks for the update Jason. The FTP site ping is fine and I’ve got them to prune their file list. Also I set up a scheduled task to archive files more than 30 days old with Robocopy. The weekend run worked smoothly.