Use wget to download large directories recursively
Have you ever wanted to recursively get a web directory? Or how about a web site?
If your a linux user, you can use wget (man wget for more info). Wget is a non-interactive network downloader. Mac users can try to get a binary <a href=”http://www.statusq.org/archives/2008/07/30/1954/”>here</a>. Wget is a very powerful tool for many reasons, but it can be quite useful if you want to download multiple files from a site.
A few days ago, I complained that I didn’t have many fonts for use in designing, so a friend linked me to <a href=”http://www.fonts500.com”>fonts500.com</a>. My first thought was “Wow, I want all of these fonts, but I don’t want to click all of those links, spread over 5 pages.” Enter wget!
wget -r --level=2 -A zip --quiet http://www.fonts500.com/A little explanation of the command:
The -r is to be recursive.
The –level=2 sets the maximum depth (we don’t want to go too far off the page)
The “-A zip” sets the file inclusion parameter, which means we only want the zip files.
–quiet is just to squelch output, just a personal preference.
And lastly, the URL you want to use.
Hope it helps!