POD servers for Perl

You want to browse Perl embedded documentation (POD)

You have some software that was designed for browsing information: a web browser

You need a server to turn the POD into web pages and serve it to the browser.

You try one of these modules:

• https://metacpan.org/pod/Pod::Browser
• Uses Catalyst
• https://metacpan.org/pod/Pod::Server
• ?
• https://metacpan.org/pod/Pod::Webserver
• Very basic
• Parsing much better than Pod::POM::Web, but still fails in places.
• Contains all the Perl built-in modules, which I don't want to see.
• How to run: podwebserver
• https://metacpan.org/pod/Pod::POM::Web
• Advanced design
• Fails to parse nested tags
• Contains all the Perl built-in modules, which I don't want to see.
• How to run: perl -I $DOC_PATH -I $POD_POM_WEB_PATH -MPod::POM::Web -e server

Questions:

• What is the reference implementation of a POD interpreter? `perldoc` just removes links and it is plain text only. So in what viewer do the links actually link to something?

Recording HTTP responses with Perl

There are two main ways. If your HTTP requests are being done in a different way, you will need to build your own solution.

The LWP way:

use LWP::UserAgent::Mockable;

BEGIN {
    # Options: record|playback|passthrough
    $ENV{LWP_UA_MOCK}      ||= 'playback';
    $ENV{LWP_UA_MOCK_FILE} ||= "$0-lwp-mock.out";
}

END {
    LWP::UserAgent::Mockable->finished();
}

# You might be able to normalise the requests using the callbacks

The Mojo way:

use Mojo::UserAgent::Mockable;

my $mock_ua = Mojo::UserAgent::Mockable->new(
    mode        => $ENV{MOJO_UA_MOCK},
    file        => "$0-mojo-ua-mock.out",
#    ignore_body => 1,
# or:
    request_normalizer => sub {
        my ( $req, $recorded_req ) = @_; 

        my ($time) = $recorded_req->body =~ m{(.*)};
        my ($hash) = $recorded_req->body =~ m{(.*)};

        my $body = $req->body;
        $body =~ s{.*}{$time};
        $body =~ s{.*}{$hash};

        $req->body($body);
    },  

);

END {
    $mock_ua->save if $ENV{MOJO_UA_MOCK} eq 'record';
}

ok my $t = Test::Mojo->new("My::App");

$t->app->mock( useragent => sub { $mock_ua } );

# Configure the proxy in record/passthrough mode only

if ( $MODE =~ /record|passthrough/ ) { 

    local $ENV{HTTPS_PROXY} = $t->app->config->{http_proxy};

    local $ENV{HTTP_PROXY}  = $t->app->config->{http_proxy};

    my $proxy = Mojo::UserAgent::Proxy->new;

    $proxy->detect;

    $mock_ua->proxy( $proxy );

}

Add headers or cookies to request in Mojolicious tests

When using Test::Mojo, sometimes you want to modify the request to add a cookie or custom headers.

.

401 vs 403 HTTP codes

Explanation:

401 'Unauthorized' should actually be 401 'Unauthenticated'
It should be used for missing or bad authentication.

403 'Forbidden' is the server telling you, "I know who you are, and you don't have permission to access this resource".

(source)

Perl one-liner to serve a directory over HTTP

plackup -e 'use Plack::App::Directory; Plack::App::Directory->new({ root => "/opt/dweb/packages" })->to_app;'

(source)

Simplest easiest quickest web server script

cd ~
python -m SimpleHTTPServer 8080

(source)

SSH tunnel

ssh -N -R 5555:1.2.3.4:666 user@hostname

• 5555 is a port you make up
• 666 is the port to which you are forwarding
• 1.2.3.4 is the IP of the target host to which you are fowarding
• hostname is the host where port 5555 will be made available
• you run this command on a third machine (e.g. your local host)

This results in  hostname:5555  getting forwarded to  1.2.3.4:666

Web developer browser plugins

Chrome:

• Chrome itself doesn't require admin privileges to install
• Chrome has dev tools built in - type F12 and select a tab e.g. Network, or right-click on the page and choose 'Inspect element'
• chrome://net-internals/
• Web developer plugin
• Modify headers

Internet Explorer:

• Internet Explorer should be installed by default
• It has some dev tools built in - type F12 and then:
• Menu: Find | Select element by click
• Menu: Cache | View cookie information
• Tab: Script | Console
• Tab: Network (IE9 and later) | Go to detailed view | Double-click an item for headers and cookies

Standalone:

• Fiddler2 - inspect and modify HTTP traffic (requires admin)
• DebugBar (requires admin)
• Proxomitron - inspect and modify headers,  (does not require admin rights)
• Proximodo a clone of Proxomitron (requires admin)
• Charles Proxy
• Other proxies

Non-GUI:

• curl -D -
• wget -S

(source, source, source, source, source)

Use Firefox as if it were WWW::Mechanize::Shell

Depends on plugin Mozrepl: https://github.com/bard/mozrepl/wiki
Use WWW::Mechanize::Firefox

HTTPS in Perl

After Crypt::SSLeay or IO::Socket::SSL is installed, LWP will automatically work for URLs starting https://

POST contents of a file

#!/usr/bin/perl

use strict;
use warnings;
require LWP::UserAgent;
use HTTP::Request;

my $ua = LWP::UserAgent->new;
$ua->timeout(10);
$ua->env_proxy;

my $url = $ARGV[0];
my $file = $ARGV[1];
my $head = $ARGV[2];

my $text = do { local( @ARGV, $/ ) = $file or die $!; <> } ;

my $headers;
if ($head) {
$headers = HTTP::Headers->new(Accept=> $head);
}

my $req = HTTP::Request->new('POST', $url, $headers);
$req->content($text);
my $response = $ua->request($req);

print $response->content;

Stop XML::LibXSLT and XML::LibXML connecting to w3.org

If you have this line in your XML/XSL:

<!DOCTYPE xml PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">

Then your parser may connect to w3.org for validation. If w3 blocks you for hammering their server, then you may get this error message: http error : Operation in progress

http error : Operation in progress

If you don't want/need validation against the DTD (and you probably don't), you can turn off that feature in XML::LibXML:

1.70
$parser = XML::LibXML->new(load_ext_dtd => 0);

< 1.70
$parser = XML::LibXML->new();
$parser->load_ext_dtd(0);

Ruby Net::HTTP cheat sheet

http://august.lilleaas.net/blog/ruby-net-http-cheat-sheet

XSL: http requests

Getting the hostname given in the current http request can be done using XSP.