...making Linux just a little more fun!
MNZ [mnzaki at gmail.com]
Hi, Whenever I try to get to linuxgazette.net:80 from a perl script I get a redirect to gentikayos.com What can be causing this?
-- //MNZ\\ "We all need mirrors to remind ourselves who we are" -- Leonard Shelby
Francis Daly [francis at daoine.org]
On Wed, Mar 07, 2007 at 06:25:55PM +0400, MNZ wrote:
Hi there,
> Whenever I try to get to linuxgazette.net:80 from a perl script I get a > redirect to gentikayos.com
When I do it and leave out the http Host: header, I get the content for genetikayos.com, but I don't get a redirect.
It might just be a terminology thing.
> What can be causing this?
Your script isn't doing what it should.
What you should do is send lines like
""" GET / HTTP/1.0 Host: linuxgazette.netto the tcp port 80. If your client doesn't send the Host: header, it can't really expect to work well on the current web. (There are other headers that probably should be sent too, as per RFC. And if you're a spider, there are other things to be aware of.)"""
Module authors hopefully wrote correct clients, if you're in a rush to "have a client" rather than "write a client", that might be a useful path to take.
All the best,
f
-- Francis Daly francis at daoine.org
MNZ [mnzaki at gmail.com]
On Wed, Mar 07, 2007 at 01:25:13PM +0000, Francis Daly wrote:
> On Wed, Mar 07, 2007 at 06:25:55PM +0400, MNZ wrote: > > Hi there, > > > Whenever I try to get to linuxgazette.net:80 from a perl script I get a > > redirect to gentikayos.com > > When I do it and leave out the http Host: header, I get the content for > genetikayos.com, but I don't get a redirect.
For me, I get a "301 Moved Permanently"
> It might just be a terminology thing. > > > What can be causing this? > > Your script isn't doing what it should. > > What you should do is send lines like > > """ > GET / HTTP/1.0 > Host: linuxgazette.net > > """
Thanks. I did read the RFC but I guess the "Host: " part was in one of the
sections I skipped 
> to the tcp port 80. If your client doesn't send the Host: header, it > can't really expect to work well on the current web. (There are other > headers that probably should be sent too, as per RFC. And if you're a > spider, there are other things to be aware of.) > > Module authors hopefully wrote correct clients, if you're in a rush to > "have a client" rather than "write a client", that might be a useful > path to take. > > All the best,
I'm actually writing a tiny client to just check that links ar ok (2xx). Now don't tell me there's a header to do that instead of GETing
Thanks again.
-- //MNZ\\ "We all need mirrors to remind ourselves who we are" -- Leonard Shelby
Ben Okopnik [ben at linuxgazette.net]
On Wed, Mar 07, 2007 at 06:17:55PM +0400, MNZ wrote:
> > I'm actually writing a tiny client to just check that links ar ok (2xx). Now > don't tell me there's a header to do that instead of GETing
[grin] Of course there is.
ben at Tyr:/$ HEAD linuxgazette.net 200 OK Connection: close'HEAD' is one of the aliases optionally installed during the installation of the LWP module. If you have it installed, but don't have those aliases, simply rerun the installation ('force install LWP') in the CPAN shell and choose the right thing when you're prompted.Date: Wed, 07 Mar 2007 14:38:42 GMTAccept-Ranges: bytes ETag: "7fc1ed-2634-67f763c0" Server: Apache/2.0.46 (Red Hat) DAV/2 SVN/1.2.1 mod_ssl/2.0.46 OpenSSL/0.9.7a Content-Length: 9780 Content-Type: text/html Last-Modified: Sat, 03 Mar 2007 14:19:03 GMT Client-Date: Wed, 07 Mar 2007 14:38:48 GMT Client-Peer: 64.246.26.120:80 Client-Response-Num: 1
-- * Ben Okopnik * Editor-in-Chief, Linux Gazette * http://LinuxGazette.NET *
MNZ [mnzaki at gmail.com]
On Wed, Mar 07, 2007 at 09:42:39AM -0500, Ben Okopnik wrote:
> On Wed, Mar 07, 2007 at 06:17:55PM +0400, MNZ wrote:
> >
> > I'm actually writing a tiny client to just check that links ar ok (2xx). Now
> > don't tell me there's a header to do that instead of GETing
>
> [grin] Of course there is.
>
> ```
> ben at Tyr:/$ HEAD linuxgazette.net
> 200 OK
> Connection: close
> Date: Wed, 07 Mar 2007 14:38:42 GMT
> Accept-Ranges: bytes
> ETag: "7fc1ed-2634-67f763c0"
> Server: Apache/2.0.46 (Red Hat) DAV/2 SVN/1.2.1 mod_ssl/2.0.46 OpenSSL/0.9.7a
> Content-Length: 9780
> Content-Type: text/html
> Last-Modified: Sat, 03 Mar 2007 14:19:03 GMT
> Client-Date: Wed, 07 Mar 2007 14:38:48 GMT
> Client-Peer: 64.246.26.120:80
> Client-Response-Num: 1
> '''
>
> 'HEAD' is one of the aliases optionally installed during the
> installation of the LWP module. If you have it installed, but don't have
> those aliases, simply rerun the installation ('force install LWP') in
> the CPAN shell and choose the right thing when you're prompted.
Well, I don't have LWP, infact I don't have anything but the core modules and have never run the CPAN shell before. Now that I did, the first thing was 'install LWP'..... it has been downloading/installing dependencies since These perl guys think of everything, don't they? There's a link tester module on CPAN already.......
-- //MNZ\\ "We all need mirrors to remind ourselves who we are" -- Leonard Shelby
Ben Okopnik [ben at linuxgazette.net]
On Wed, Mar 07, 2007 at 08:06:55PM +0400, MNZ wrote:
> On Wed, Mar 07, 2007 at 09:42:39AM -0500, Ben Okopnik wrote:
> >
> > 'HEAD' is one of the aliases optionally installed during the
> > installation of the LWP module. If you have it installed, but don't have
> > those aliases, simply rerun the installation ('force install LWP') in
> > the CPAN shell and choose the right thing when you're prompted.
>
> Well, I don't have LWP, infact I don't have anything but the core modules and
> have never run the CPAN shell before. Now that I did, the first thing was
> 'install LWP'..... it has been downloading/installing dependencies since
> These perl guys think of everything, don't they? There's a link tester module
> on CPAN already.......
[blink] I guess I'm not surprised. Not very surprised, anyway. If you want to script it, we've already got a volunteer who's willing to do the actual link testing. Or you two could team up - your choice, since you were first.
LWP is a really good kit of stuff to have, anyway - it drags in stuff like MD5::Hash and Readline, etc. All good, useful things that come up again and again.
-- * Ben Okopnik * Editor-in-Chief, Linux Gazette * http://LinuxGazette.NET *
MNZ [mnzaki at gmail.com]
On Thu, Mar 08, 2007 at 03:33:26PM -0500, Ben Okopnik wrote:
> [blink] I guess I'm not surprised. Not very surprised, anyway. If you
err..not surprised about what? The fact that I'm module-less?
> want to script it, we've already got a volunteer who's willing to do the > actual link testing. Or you two could team up - your choice, since you > were first.
If there's someone already doing it, then let them continue. I was just trying it out for a bit of fun (and experience). I have a "skeleton" for the script though and I could send it in to the other volunteer if they haven't started on it yet. Who is (s)he?
> LWP is a really good kit of stuff to have, anyway - it drags in stuff > like MD5::Hash and Readline, etc. All good, useful things that come up > again and again.
Well yeah, it dragged in quite some stuff.
-- //MNZ\\ "We all need mirrors to remind ourselves who we are" -- Leonard Shelby
Ben Okopnik [ben at linuxgazette.net]
On Fri, Mar 09, 2007 at 05:32:23PM +0400, MNZ wrote:
> On Thu, Mar 08, 2007 at 03:33:26PM -0500, Ben Okopnik wrote: > > [blink] I guess I'm not surprised. Not very surprised, anyway. If you > > err..not surprised about what? The fact that I'm module-less?
No, that a module for this already exists. There's probably a module for picking the nose of an ambidextrous spotted wombat, too (with options for specific colors, degrees of furriness, etc.); there are modules for just about anything you can name.
http://cpan.org/modules/01modules.index.html
> > want to script it, we've already got a volunteer who's willing to do the > > actual link testing. Or you two could team up - your choice, since you > > were first. > > If there's someone already doing it, then let them continue. I was just trying > it out for a bit of fun (and experience). I have a "skeleton" for the script > though and I could send it in to the other volunteer if they haven't started > on it yet. Who is (s)he?
It's not really that someone was already doing it; you volunteered to write the script, and Suramya Tomar volunteered to do the actual link-checking. If you two want to get together and work out a good way to do this, that would be great. I'll ping you off-list with his address.
-- * Ben Okopnik * Editor-in-Chief, Linux Gazette * http://LinuxGazette.NET *