Sunday, May 9, 2010

Mirror a site

Mirroring a site is the act of taking the exact copy of a site, copying them in such a way that even the hieracrchy of the files on the remote server is maintained. Mirroring can be used if you want to have a backup of the directories on your remote server. Sites such as sourceforge.net which experience heavy traffic have "mirrored" links so that users can download from the mirrored links if they are unable to do so from the original website.

Anyone can mirror a site but it has do be done after taking permission from the owner, and is illegal if done without prior permission as it would be a violation copyright

Some servers implement a feature in which if too much of data is being downloaded very quickly, it can block those users.

Also mirroring a site sometimes causes a problem if it contains dynamic elements such as php ,javascripts and other dnamic elements.

The most famous software used is Wget, it was initially made for unix systems but was ported to windows, links are provided:-

For Windows:-

1.Wget command line

2.WinWGet gui

3.VisualWin (gui)

For Linux:-
1.GNU wget


Basic commands to use the wget command line (Windows)

wget -m [site name] Would mirror the site and save it on the local hard drive with the directory name as the site name

wget -r [site name] Would save all the pages and all the links

wget -h show all the options available


I was not able to change the default directory therefore I used the WinWGet gui version, its pretty simple.

Important links:-

1.Wiki on mirroring

2 comments:

  1. Some Nice Posts in this blog.

    Can wireshark be used to detect stealth port scans in IDS setup in organization?

    Some other topics at:
    http://shellscriptsguide.blogspot.com

    ReplyDelete
  2. of course...any packet intended for your machine will be captured...what makes a port scan stealthy is the nature in which scanning is done..so packets which may appear to be harmless might actually carry some valuable information to the attacker...ef icmp messages even though completely harmless to a user can give away information to an attacker...thorough details in some future article... :D

    ReplyDelete