Wednesday, June 18, 2008

Rapidshare downloader - Perl+Bash

Update: RS redesigned their site and this script no longer works. (July 3, 2008)

Hi frndz,

Today I wrote a Perl script to download a set of files from at "Happy Hours" for free users. This works in Linux/Unix only and only if you have a pppoe connection. It uses WWW::Mechanize module which is not available with standard Perl library. You will need to install it separately. This script works with current design of rapidshare site. If they make changes in site, this may fail. The usage is:

perl [-d download_directory] links.txt

This program will wait till "Happy Hours" (will check every 10 mins) and will start downloading rapidshare links in links.txt. The program will ignore blank lines and lines starting with a '#' in the input file. If -d flag is omitted, files will be downloaded to current working directory. You can also group your downloads simply by putting a name in one line and all following links till next name will be saved to a directory with that name. eg:


Here is the code...

#!/usr/bin/perl -w

use WWW::Mechanize;
use Getopt::Std;

my $mech = WWW::Mechanize->new();
my $pwd = `pwd`;

if (getopts('d:')) {
for (my $i = 0; $i <= $#ARGV; $i++) {
$ARGV[$i] = $pwd."/".$ARGV[$i] unless ($ARGV[$i] =~ m{^/});
chdir "$opt_d" or die "Couldn't cd to $opt_d: $!";
$pwd = `pwd`;
else {
die "Error in option? $!";

while (<>) {
# if input line is not a url, treat it as a directory name to create and cd.
unless (/http/) {
chdir "$pwd";
mkdir "$_", 0755 or warn "Failed to create directory $_: $!";
chdir "$_";

print "Trying to download $_\n";


if ($mech->response()->content() =~ m{h1>Error./h1>}) {
print "Some server error. Skipping file: $_\n";

$mech->submit_form("form_number", 2); # click the "Free" button
print "Selected Free download.\n";
$data = $mech->response()->content();
if ($data =~ /Happy Hour/) {
print "It\'s Happy Hour!\n";
if ($data =~ /form name="dl" action="(.*?)"/) { # get file url
print "Got free link.\n";
`wget $1`;
print "Reconnecting pppoe after 1 minutes.\n";
`poff -a;sleep 1m;pon dsl-provider;sleep 4`;
elsif ($data =~ /reached the limit/) { # didnt get new ip. try again.
print "Reached free user's limit.\n";
print "Reconnecting pppoe after 2 minutes.\n";
`poff -a;sleep 2m;pon dsl-provider;sleep 4`;
else {
print "Not happy hour :-(\nRetrying after 10 minutes\n";
for ($i = 1; $i <= 10; $i++) {
`sleep 1m`;
print " $i";
print "\n";

Wednesday, June 11, 2008

GNU/Linux package installation without Internet

Now, I found out a better simple solution for this. You can use an online tool described here:

Updated on July 4, 2008.

Hi friends,

Many of you may be in a situation that, you installed GNU/Linux in your computer but couldn't setup internet in it and many of your favorite packages are not available in the installation media. Package installation in GNU/Linux system is a bit difficult if you don't have Internet and don't have complete local repository, because you will have to resolve all dependencies yourself.

This was a major complaint of many of my friends who are new to GNU/Linux. MP3 support, video codecs etc. are not usually available in the installation media. This is also the same with Ubuntu. "Ubuntu looks good but it can't play my MP3s or videos". :-) I recently got a nice solution for this (from different sources in internet). Local update and install for APT. This trick will work on all distributions with APT as its package manager (Debian, Ubuntu etc.). You will need to work in command line a bit. But it's easy. Here is how to do it in Ubuntu 8.04:

1) You need to enable repositories and get packages list. Usually you do this by opening System->Administration->Synaptic Package Manager and selecting Settings->Repositories and then enabling universe, restricted, multiverse etc.

The package list is located in /var/lib/apt/lists/ directory. So, you can simply copy your friend's(who have internet in Linux) package list(all files in the above directory) to your system. Make sure that your friend also uses the same version og Ubuntu as yours. To paste files into this directory, you need root privileges. To get a file browser with root privileges, type:
sudo nautilus /var/lib/apt/lists
in a terminal.

If you have to download the package list yourself from another system or from MS Windows, then you can do this:
Enable the repositories you need. Then type:
sudo apt-get update --print-uris > ~/Desktop/update.txt
in a terminal.
This will give you the list of files to be downloaded in the file update.txt in your desktop. Here is the format of each line in that file:
'download_location' filename 0
You need to download each file and rename it to corresponding "filename" and then copy them to /var/lib/apt/lists/ .

I haven't tested this offline APT update. But I hope that it will work.

2) Now you can download packages. Use synaptic package manager to find out the name of package you need to download. Here, I am gonna install "cheese" (a software to capture video). Use the --print-uris option to get the list of packages to be downloaded.
sudo apt-get install cheese --print-uris > ~/Desktop/packages.txt
The packages.txt will contain the packages to be downloaded. Download all those packages and then copy them to /var/cache/apt/archives/

Then, we can install the package without Internet.
sudo apt-get install cheese

That's all.
Have fun with GNU/Linux. If you have any doubt, leave a comment.

Sunday, May 11, 2008

Best of luck to Yahoo! Interns

Jithin and Vineeth reached Bengaluru(I too started using the new name) yesterday and their internship will start tomorrow. Best of luck to them. And I am sitting here having nothing to do. No... I have things to do... I started downloading movies(after reading IMDB reviews as Syam used to do) :-) I am gonna be very busy from tomorrow onwards.

I will be joining Yahoo! on 28th July(most probably) as Kurian also selected that date. So, we have about 3 months. I know I wont get such a long vacation in the near future. Have to utilize it wisely. I resumed working in RentACoder and now taking only Perl projects. Best way to get used to Perl. I need to learn Bash scripting too. Now I know only some bits and pieces of shell scripting. Jemshad bhayya used to say "read 'man bash'". Yeah, I am gonna read it for the first time.

I am also interested in Python. But no use of it in Y!. So not stepping into it now.

Eagerly waiting to hear Y! experiences of both Jithin and Vineeth... :-)

Tuesday, May 6, 2008

Back from dead

Hi friends...

It's been a long time since my last post. I was very busy with exams and some personal matters. So many tensions in my head and I was not in a mood to write anything. :(

Now, everything cleared up and I am back to action. :) S8 exams are over, except of course the back papers. :) So theoretically, I am now a BTech. But Practically, not! lol. Being a graduate is a great feeling. But leaving college is not. I didn't make many friends; I made a few very good friends. Leaving them is unbearable. Hmm... have to leave, hoping to bump into each other (in Kiran's words) again.

I have 3 more months before joining Yahoo!. What should I do? Take care of my body (:D)? Sharpen my Linux skills? I have no idea. Btw, I installed Ubuntu hardy Heron and PC-BSD 1.5.1 in my system. Will talk about it later... bye...