Do you find your self ever wishing you could use an ftp client against an http server for quickly traversing directories and setting up large downloads? I do, quite often. This is why I was very pleased to come across lftp's ability to connect to http servers, and act like a ftp client!! Case in point: [jkeating@yoda jkeating]$ lftp http://download.fedoralegacy.org lftp download.fedoralegacy.org:/> ls -rw-r--r-- -- redhat/7.2 -rw-r--r-- -- redhat/7.3 -rw-r--r-- -- redhat/8.0 lftp download.fedoralegacy.org:/> Well, now wait a sec, if you visit http://download.fedoralegacy.org in a browser, there is an actual html page at the top, whats going on here? Well lftp is parsing that html page, and showing you the links that are available. We can choose to follow a link, or we can go our own way, if we know the directory before hand. lftp download.fedoralegacy.org:/> pwd http://download.fedoralegacy.org/ lftp download.fedoralegacy.org:/> cd redhat lftp download.fedoralegacy.org:/redhat> pwd http://download.fedoralegacy.org/redhat lftp download.fedoralegacy.org:/redhat> ls drwxr-xr-x - 2004-01-22 20:52 / drwxr-xr-x - 2004-01-20 14:26 7.2 drwxr-xr-x - 2004-01-20 14:26 7.3 drwxr-xr-x - 2004-01-20 14:26 8.0 lftp download.fedoralegacy.org:/redhat> now we don't see links anymore, as we're into a directory structure that allows for directory indexing. From here we can go all over the place if we wish. A fun command on a server like this is the "mirror" command. The mirror command will basically mirror the remote to the local. It takes 2 main arguments, the remote directory, and the local directory. This is handy when keeping things in sync, as it will examine the local directory and compare it to the remote directory (or vice versa), and do the right thing to keep them in sync. The -R flag will make the remote directory equal the local directory. "help mirror" inside lftp will give you a nice summary of the command. Now, what about servers that don't allow for directory indexing? Well, you can use lftp on them too, due to it's html parsing. Check this out: [jkeating@yoda jkeating]$ lftp http://www.sco.com lftp www.sco.com:/> ls -rw-r--r-- -- htmlf/newnav.js -rw-r--r-- -- htmlf/menu03/sco_menu_data.js -rw-r--r-- -- htmlf/menu03/dqm_script.js drwxr-xr-x -- / -rw-r--r-- -- images/sco_logo_blue.gif drwxr-xr-x -- howtobuy drwxr-xr-x -- worldwide -rw-r--r-- -- images/yellow_bg.gif drwxr-xr-x -- products -rw-r--r-- -- images/menu/products_services2_off.gif drwxr-xr-x -- information_for -rw-r--r-- -- images/menu/information_for2_off.gif drwxr-xr-x -- resources -rw-r--r-- -- images/menu/resources2_off.gif drwxr-xr-x -- company -rw-r--r-- -- images/menu/company2_off.gif drwxr-xr-x -- partners -rw-r--r-- -- images/menu/mysco2_off.gif -rw-r--r-- -- \"/images/flash/homepage.swf\" -rw-r--r-- -- images/flash/alt.gif -rw-r--r-- -- images/dot.gif drwxr-xr-x -- products/authentication -rw-r--r-- -- images/homepage/auth_banner.jpg drwxr-xr-x -- scosource -rw-r--r-- -- images/bulletins/sub_item14_off.gif -rw-r--r-- -- scosource/unix_licensee_letter_20031218.pdf -rw-r--r-- -- scosource/abi_files_letter_20031219.pdf -rw-r--r-- -- images/company/2003fast500winnerbooklet.pdf -rw-r--r-- -- images/company/fast500.png -rw-r--r-- -- images/bulletins/bulletin233_off.gif drwxr-xr-x -- company/news -rw-r--r-- -- images/bulletins/sub_item3_off.gif drwxr-xr-x -- solution_builder -rw-r--r-- -- images/buttons/build_custom_solution_off.gif -rw-r--r-- -- company/feedback/index.html -rw-r--r-- -- images/line_vert.gif drwxr-xr-x -- company/legal drwxr-xr-x -- company/privacy drwxr-xr-x -- download drwxr-xr-x -- register -rw-r--r-- -- tarantella_jump.html -rw-r--r-- -- images/tarentella_gray.gif -rw-r--r-- -- calderagraphics_jump.html -rw-r--r-- -- images/caldera_graphics_gray.gif lftp www.sco.com:/> Neat huh! Some other fun things that can be done with lftp: 1) bash like history searches: [jkeating@yoda jkeating]$ lftp http://download.fedoralegacy.org (reverse-i-search)`mirror': help mirror 2) sh like piping lftp download.fedoralegacy.org:/redhat/7.3/updates-testing/i386> ls drwxr-xr-x - 2004-01-20 14:24 .. -rw-r--r-- 93k 2004-01-21 20:06 arpwatch-2.1a11-17.7.3.4.legacy.i386.rpm -rw-r--r-- 1.0M 2004-01-24 00:55 cvs-1.11.1p1-9.7.legacy.i386.rpm -rw-r--r-- 3.6M 2004-01-19 22:56 ethereal-0.9.16-0.73.2.legacy.i386.rpm -rw-r--r-- 2.6M 2004-01-19 23:00 ethereal-gnome-0.9.16-0.73.2.legacy.i386.rpm drwxr-xr-x - 2004-01-24 01:15 headers -rw-r--r-- 316k 2004-01-21 20:06 libpcap-0.6.2-17.7.3.4.legacy.i386.rpm -rw-r--r-- 433k 2004-01-19 20:22 screen-3.9.11-4.legacy.i386.rpm -rw-r--r-- 266k 2004-01-21 20:06 tcpdump-3.6.3-17.7.3.4.legacy.i386.rpm lftp download.fedoralegacy.org:/redhat/7.3/updates-testing/i386> ls |grep tcp -rw-r--r-- 266k 2004-01-21 20:06 tcpdump-3.6.3-17.7.3.4.legacy.i386.rpm lftp download.fedoralegacy.org:/redhat/7.3/updates-testing/i386> 3) redirection lftp download.fedoralegacy.org:/redhat/7.3/updates-testing/i386> ls >list lftp download.fedoralegacy.org:/redhat/7.3/updates-testing/i386> by [jkeating@yoda jkeating]$ cat list drwxr-xr-x - 2004-01-20 14:24 .. -rw-r--r-- 93k 2004-01-21 20:06 arpwatch-2.1a11-17.7.3.4.legacy.i386.rpm -rw-r--r-- 1.0M 2004-01-24 00:55 cvs-1.11.1p1-9.7.legacy.i386.rpm -rw-r--r-- 3.6M 2004-01-19 22:56 ethereal-0.9.16-0.73.2.legacy.i386.rpm -rw-r--r-- 2.6M 2004-01-19 23:00 ethereal-gnome-0.9.16-0.73.2.legacy.i386.rpm drwxr-xr-x - 2004-01-24 01:15 headers -rw-r--r-- 316k 2004-01-21 20:06 libpcap-0.6.2-17.7.3.4.legacy.i386.rpm -rw-r--r-- 433k 2004-01-19 20:22 screen-3.9.11-4.legacy.i386.rpm -rw-r--r-- 266k 2004-01-21 20:06 tcpdump-3.6.3-17.7.3.4.legacy.i386.rpm 4) use of cat lftp geek.j2solutions.net:/rpms/afio> ls drwxr-xr-x - 2004-01-23 10:37 .. -rw-r--r-- 92k 2003-10-04 14:12 afio-2.4.7-2.src.rpm drwxr-xr-x - 2003-10-04 15:19 fedora -rw-r--r-- 1k 2003-10-04 15:21 md5sum drwxr-xr-x - 2003-10-04 15:18 rhl9 lftp geek.j2solutions.net:/rpms/afio> cat md5sum b726f38a0dfcf6953800b0ee7978a52c afio-2.4.7-2.src.rpm 55 bytes transferred lftp geek.j2solutions.net:/rpms/afio> and all kinds of fun stuff! In summary, lftp is one hot little app. In fact, I think it is one of the killer apps for the command line. Cheers!