The Idiot's Guide to Solving Perl CGI Problems

If you're like most people on the newsgroup, you've presented us with the clasiquísimo CGI problem: ``it doesn't work'', without providing us sufficient background to divine what the real problem is. This is infuriating, because is makes you come off like some drooling idiot begging for free consulting help without having done any investigation on your own part. This drives people away who could otherwise help you and impoverishes the Net by exacerbating the already annoyingly low signal-to-noise ratio.

For a start, you need to look through the checklist below for a few simple questions that need answering. While some sample answers are provided, you'll have to figure them out on your own.

[Note from johnpc: The answers have been modified such that they apply to professional and advanced servers on XS4ALL.]

Good luck.


Q: Who owns the script?
A: I do.

Q: What are the script's permissions?
A: They were 0600, but I just realized that's not executable. I'd better make it 0755 instead.

Q: Is your script in the right directory?
A: No, I forgot to put it in /usr/local/WWW/my.server.name/cgi-bin or in /home/u/user/WWW.cgi/ if I have an advanced server

Q: Is CGI execution enabled in your server for that directory and/or file suffix?
A: Yes, the server admin enabled CGI scripts (provided the script is in the right directory!)

Q: Under what uid does the server runs its CGI programs?
A: nobody (Oops, it can't write my files or directories.)

Q: Can the server's uid write any files you're trying to write?
A: Nope - I own the files, but it doesn't run as me and the permissions are 0600 instead of 0666. I guess that's why it can't open my own files.

Q: What happens when you run it interactively?
A: I didn't know you could run these interactively because I never bothered to read the documentation for the CGI.pm library.

Q: What's CGI.pm? I use "cgi-lib.pl" or I just copied some code from another script.
A: You're using ancient outdated programming habits. Use modern tools like CGI.pm that give you lots of advantages.

Q: Did you remember to upload the script in ASCII mode? In binary mode, DOS text will confuse unix machines.
A: I never knew there was a difference, that unix machines don't like the additional ^M characters that DOS boxes put in. So either I transfer the scripts again, using ftp, in ASCII mode, or I run: 
       % dos2unix < script > script.tmp
       % mv script.tmp script
       % chmod 0755 script
(but of course you knew you had to issue that last command again, didn't you?)
(Note: if dos2unix doesn't work, try dos2bsd)

Q: What's in the server error log?
A: I never thought to look. Oh, there it is - never mind.

Q: Error log file? I have an advanced website, so I don't have access to the actual cgi machine?
A: Yes you have. Telnet to cgi.yourdomain.nl and login with your regular username and password. You'll then see the error log file over the last few hours (for your system only)

Q: Where's the server log for professional websites?
A: /usr/local/log/httpd.error

Q: What's the Perl version and OS version?
A: It depends on what type of website you have:

Q: What's the library version?
A: grep -i version in the library, or for CGI.pm, do this: 
$ perl -le 'use CGI; print $CGI::VERSION'
For advanced websites, 2.42 is currently installed. For professional websites, it depends on the host on which your site is hosted: 
	lyrix:~ $ perl5 -MCGI -le 'print $CGI::VERSION'
	2.36
	geriatrix:~ $ perl5 -MCGI -le 'print $CGI::VERSION'
	2.36
	nestorix:~ % perl5 -MCGI -le 'print $CGI::VERSION'
	2.21

Q: What's the path to your perl executable on the server?
A: /usr/local/bin/perl5
(this is ALWAYS perl5 no matter what server you're on. It might be different versions on different hosts, though. See 2 questions up)

Q: What's the path to your perl executable specified in your script?
A: /usr/bin/perl (oops, of course it can't find it)
(/usr/bin/perl is correct on most xs4all machines too!)

Q: What's the http daemon server's version number so people who might help you have a better clue about your local environment?
A: Apache. For advanced websites, apache/1.3.3. For professional websites, it depends on the host. Lyrix runs apache/1.3, geriatrix and nestorix run apache/1.20

Q: What happens when you add Perl's -w flag?
A: It tells me about my silly mistakes that are listed in detail in the perldiag manpage where I diligently looked them up.

Q: What happens when you add Perl's -T flag?
A: It tells me about security problems as described in the perlsec manpage, which I've carefully read and understood. I even read the CGI Security FAQ.

Q: What happens when you add use strict?
A: It makes me declare my variables and quote my strings and finds all these silly errors which I've carefully corrected using my() declarations, use vars, and quotes.

Q: Did you remember to output the MIME type before any other non-header output (other headers might be Location: or Set-Cookie:)
A: Oh, right. It has to be a valid header and then a valid body. I guess I need to say this stuff earlier than I was doing: and make sure I actually put two newlines out, not just one. 
         print "Set-cookie: GroversDelight\n";
         print "Content-Type: text/html\n\n";        # <-- two newlines
         print "<TITLE>Sample Titlel</TITLE>\n";

Q: Did you remember to flush STDOUT at the top of your script so the MIME type gets out before any errors?
A: Nope - gosh, no wonder it doesn't get out before it bombs! I guess I better add this to the top: 
          $| = 1
In fact, to be perfectly safe, start your script with the following: 
     #!/usr/bin/perl -Tw
     $|++;
     use strict;
     use CGI::Carp qw(fatalsToBrowser);
     use CGI;
(oops! ``fatalsToBrowser'' is only available for advanced websites and for professional websites on lyrix.xs4all.nl. It's NOT available on geriatrix or nestorix. Ask your site to be moved if this bothers you!)

NOTE: Please ``use CGI::Carp qw(fatalsToBrowser);'' while developing. It saves a LOT of development time, by sending errors to your browser instead of burying them in the errorlog file.


Q: What happens when you check return values of each and every one of your syscalls?
A: That seems like way too much work, but sure enough, just as soon as I added something like 
          open(FILE, ">some_file")
               || die("can't write some_file: $!");
the error log showed that $! contains ``Permission denied'' or ``No such file or directory'', and everything became clear.

Q: Did you use the standard CGI.pm module to do this for you instead of parsing by hand (which would be a really idiotic idea) or use the more limited cgi-lib.pl library?
A: Gee, you mean somebody else has actually done this kind of thing before? I had no idea I didn't know I didn't have to do it all myself, and and that I could get the latest version of the neat CGI module from http://www.perl.com/cgi-bin/cgi_mod?modules=CGI
(See above for the version of the installed CGI.pm)

Q: If you used a library, did you type make install to place it in the proper system directory (somewhere in @INC, probably a lot like /usr/local/lib/perl/site_perl/CGI.pm) so it can be properly found and have its set permissions to 0644, or did you just blindly do a manual copy and screw up the path, permissions, or both?
A: What's make? Look, like I just (mis)installed it myself by a silly copy to cgi-bin and then I made it mode 0700. I guess that doesn't work at all, does it? (CGI.pm is in the default @INC path on all xs4all machines)

Q: Did you remember to post to comp.infosystems.www.authoring.misc, instead of blasting the comp.lang.perl.* groups with questions that aren't related to Perl in the least?
A: Nope - is that why I don't get any useful answers but just a bunch of flames instead?

Q: If you're running some brain-dead, bondage-and-discipline pseudo-operating system from The Evil Empire, did you bother to read what the Windoze Perl FAQ says about the Web?
A: No, I didn't know about http://www.perl.hip.com/PerlFaq.htm. (and you don't have to, since we only use Real OSes at XS4ALL)

Q: Did you avoid placing an interpreter in your cgi-bin the way you're warned about at http://www.perl.com/perl/news/latro-announce.html and from CERT?
A: No, what's wrong with that? (noise of disk grinding to dust after corporate secrets stolen heard in the background)
Odds are that if you'd merely answer these questions and carry out the specified steps, you yourself would solve your very own problem without having to come begging for help and pissing off the Net.

Once you have thoroughly investigated ALL of these matters on your own and then asked your buddy in the next cubicle whether he's seen anything like it, should you find that it's still a problem, then by all means do go ahead and post, but make certain you provide us with plenty of details, including your own answers to all (or at least most) of the questions listed above.

On the other hand, if most of those questions don't even begin to make the least bit of sense to you, then you're already in way over your head. Should this be the case, you'd better hold off on CGI programming for a spell. Go find yourself a local guru and check out some of the many books listed at http://wwwiz.com/books/cgi-perl.html

IM!HO, the Addison-Wesley and O'Reilly books tend to be the most thorough, but there are so many out there that you'd do well to look them over yourself. Remember that thickness needn't imply clarity, completeness, or correctness.

This page was based upon a posting by: 
-- 
Tom Christiansen      Perl Consultant, Gamer, Hiker      tchrist@mox.perl.com

    #define SIGILL 6         /* blech */
        --Larry Wall in perl.c from the 4.0 perl source code
Converted to HTML by JohnPC