Wednesday, June 18, 2008

Rapidshare downloader - Perl+Bash

Update: RS redesigned their site and this script no longer works. (July 3, 2008)

Hi frndz,

Today I wrote a Perl script to download a set of files from at "Happy Hours" for free users. This works in Linux/Unix only and only if you have a pppoe connection. It uses WWW::Mechanize module which is not available with standard Perl library. You will need to install it separately. This script works with current design of rapidshare site. If they make changes in site, this may fail. The usage is:

perl [-d download_directory] links.txt

This program will wait till "Happy Hours" (will check every 10 mins) and will start downloading rapidshare links in links.txt. The program will ignore blank lines and lines starting with a '#' in the input file. If -d flag is omitted, files will be downloaded to current working directory. You can also group your downloads simply by putting a name in one line and all following links till next name will be saved to a directory with that name. eg:


Here is the code...

#!/usr/bin/perl -w

use WWW::Mechanize;
use Getopt::Std;

my $mech = WWW::Mechanize->new();
my $pwd = `pwd`;

if (getopts('d:')) {
for (my $i = 0; $i <= $#ARGV; $i++) {
$ARGV[$i] = $pwd."/".$ARGV[$i] unless ($ARGV[$i] =~ m{^/});
chdir "$opt_d" or die "Couldn't cd to $opt_d: $!";
$pwd = `pwd`;
else {
die "Error in option? $!";

while (<>) {
# if input line is not a url, treat it as a directory name to create and cd.
unless (/http/) {
chdir "$pwd";
mkdir "$_", 0755 or warn "Failed to create directory $_: $!";
chdir "$_";

print "Trying to download $_\n";


if ($mech->response()->content() =~ m{h1>Error./h1>}) {
print "Some server error. Skipping file: $_\n";

$mech->submit_form("form_number", 2); # click the "Free" button
print "Selected Free download.\n";
$data = $mech->response()->content();
if ($data =~ /Happy Hour/) {
print "It\'s Happy Hour!\n";
if ($data =~ /form name="dl" action="(.*?)"/) { # get file url
print "Got free link.\n";
`wget $1`;
print "Reconnecting pppoe after 1 minutes.\n";
`poff -a;sleep 1m;pon dsl-provider;sleep 4`;
elsif ($data =~ /reached the limit/) { # didnt get new ip. try again.
print "Reached free user's limit.\n";
print "Reconnecting pppoe after 2 minutes.\n";
`poff -a;sleep 2m;pon dsl-provider;sleep 4`;
else {
print "Not happy hour :-(\nRetrying after 10 minutes\n";
for ($i = 1; $i <= 10; $i++) {
`sleep 1m`;
print " $i";
print "\n";

Wednesday, June 11, 2008

GNU/Linux package installation without Internet

Now, I found out a better simple solution for this. You can use an online tool described here:

Updated on July 4, 2008.

Hi friends,

Many of you may be in a situation that, you installed GNU/Linux in your computer but couldn't setup internet in it and many of your favorite packages are not available in the installation media. Package installation in GNU/Linux system is a bit difficult if you don't have Internet and don't have complete local repository, because you will have to resolve all dependencies yourself.

This was a major complaint of many of my friends who are new to GNU/Linux. MP3 support, video codecs etc. are not usually available in the installation media. This is also the same with Ubuntu. "Ubuntu looks good but it can't play my MP3s or videos". :-) I recently got a nice solution for this (from different sources in internet). Local update and install for APT. This trick will work on all distributions with APT as its package manager (Debian, Ubuntu etc.). You will need to work in command line a bit. But it's easy. Here is how to do it in Ubuntu 8.04:

1) You need to enable repositories and get packages list. Usually you do this by opening System->Administration->Synaptic Package Manager and selecting Settings->Repositories and then enabling universe, restricted, multiverse etc.

The package list is located in /var/lib/apt/lists/ directory. So, you can simply copy your friend's(who have internet in Linux) package list(all files in the above directory) to your system. Make sure that your friend also uses the same version og Ubuntu as yours. To paste files into this directory, you need root privileges. To get a file browser with root privileges, type:
sudo nautilus /var/lib/apt/lists
in a terminal.

If you have to download the package list yourself from another system or from MS Windows, then you can do this:
Enable the repositories you need. Then type:
sudo apt-get update --print-uris > ~/Desktop/update.txt
in a terminal.
This will give you the list of files to be downloaded in the file update.txt in your desktop. Here is the format of each line in that file:
'download_location' filename 0
You need to download each file and rename it to corresponding "filename" and then copy them to /var/lib/apt/lists/ .

I haven't tested this offline APT update. But I hope that it will work.

2) Now you can download packages. Use synaptic package manager to find out the name of package you need to download. Here, I am gonna install "cheese" (a software to capture video). Use the --print-uris option to get the list of packages to be downloaded.
sudo apt-get install cheese --print-uris > ~/Desktop/packages.txt
The packages.txt will contain the packages to be downloaded. Download all those packages and then copy them to /var/cache/apt/archives/

Then, we can install the package without Internet.
sudo apt-get install cheese

That's all.
Have fun with GNU/Linux. If you have any doubt, leave a comment.