Using your bastion server as a socks proxy

How to access web services via a jump host:
​
ssh -v -N -C -D 9090 username@bastion.example.com
​
* `-v` verbose, see any issues as they arise
* `-N` don't execute a remote command
* `-C` turn on compression
* `-D #` open a SOCKS proxy on this local port
​
Browser setup: FoxyProxy (Configure to "Use Enabled Proxies By Patterns and Order")

NOTE: It's a bad idea to route all web traffic through your company's bastion server / jump host. Only route the hosts you absolutely need to.

Or for command line stuff:

ssh -v -N -C -D 9090 mad@bastard.example.com
hpts -s 127.0.0.1:9090 -p 8989 --level info
export http_proxy=http://127.0.0.1:8989/
export https_proxy=http://127.0.0.1:8989/

Recording HTTP responses with Perl

There are two main ways. If your HTTP requests are being done in a different way, you will need to build your own solution.

The LWP way:

use LWP::UserAgent::Mockable;

BEGIN {
    # Options: record|playback|passthrough
    $ENV{LWP_UA_MOCK}      ||= 'playback';
    $ENV{LWP_UA_MOCK_FILE} ||= "$0-lwp-mock.out";
}

END {
    LWP::UserAgent::Mockable->finished();
}

# You might be able to normalise the requests using the callbacks

The Mojo way:

use Mojo::UserAgent::Mockable;

my $mock_ua = Mojo::UserAgent::Mockable->new(
    mode        => $ENV{MOJO_UA_MOCK},
    file        => "$0-mojo-ua-mock.out",
#    ignore_body => 1,
# or:
    request_normalizer => sub {
        my ( $req, $recorded_req ) = @_; 

        my ($time) = $recorded_req->body =~ m{(.*)};
        my ($hash) = $recorded_req->body =~ m{(.*)};

        my $body = $req->body;
        $body =~ s{.*}{$time};
        $body =~ s{.*}{$hash};

        $req->body($body);
    },  

);

END {
    $mock_ua->save if $ENV{MOJO_UA_MOCK} eq 'record';
}

ok my $t = Test::Mojo->new("My::App");

$t->app->mock( useragent => sub { $mock_ua } );

# Configure the proxy in record/passthrough mode only

if ( $MODE =~ /record|passthrough/ ) { 

    local $ENV{HTTPS_PROXY} = $t->app->config->{http_proxy};

    local $ENV{HTTP_PROXY}  = $t->app->config->{http_proxy};

    my $proxy = Mojo::UserAgent::Proxy->new;

    $proxy->detect;

    $mock_ua->proxy( $proxy );

}

SSH tunnel using a jump host

Access a service on a remote machine via an intermediary

ssh -v -L 4444:app.example.com:5000 $USER@jump.example.com -nNT

Now you can access the service running on app.example.com:5000 by going to localhost:4444 in your browser.

Explanation of the command

• from the host machine (where you are running the command)
• connect to jump.example.com as user $USER
• once there, access service app.example.com on port 5000
• then make that service available on the host machine on port 4444

Advanced usage - Two jumps

ssh -J user@jump.example.com user@app.example.com -L 1111:database.example.com:3306 -nNT -vvv

Notes:

-J jumps to another host

-L makes a tunnel to a service that's already running

Now you can do:

mysql --protocol=tcp --host=127.0.0.1 --port=1111

Notes:

- you must specify protocol because of the tunnel

- specifying 127.0.0.1 (instead of "localhost") prevents MySQL trying to use a local socket and failing

Use a proxy on a remote machine via an intermediary

If there's a proxy you need to use: proxy.example.com:8888 -- but you can only access it from jump.example.com -- then set up a tunnel like this:

ssh -A -L 4444:proxy.example.com:8888 $USER@jump.example.com -nNTv

Now you can use http://localhost:4444 as your proxy server, instead of http://proxy.example.com:8888

How to set network proxy from command line in Ubuntu

You might need to do this if your network settings dialog box crashes as I've seen on Ubuntu 13.10:

gsettings set org.gnome.system.proxy autoconfig-url http://myserver/myconfig.pac

gsettings set org.gnome.system.proxy mode auto

This is how I found out the settings:

$ for key in $(gsettings list-keys org.gnome.system.proxy); do echo $key = $(gsettings get org.gnome.system.proxy $key); done

autoconfig-url = 'http://myserver/myconfig.pac'
ignore-hosts = ['localhost', '127.0.0.0/8']
mode = 'auto'
use-same-proxy = false

It doesn't have to be a .pac file, ours is .dat.

(source)

SSH port forwarding / tunnelling

Command for a tunnel:

ssh -D 9999 username@remote-host

• -D means "dynamic application-level port forwarding"
• 9999 is a port you make up
• username@remote-host is the account you're relaying through
• A prompt on the remote host will appear, ignore it (or use -N to avoid)

In your web browser, set the SOCKS5 proxy to localhost:9999

Use lighttpd as a proxy-pass server

$HTTP["host"] =~ "made.up.hostname" { # hostname you want to create
    $SERVER["socket"] == ":1234" { # port you want to create

        server.errorlog      = "/var/logs/madeup/lighttpd-error.log"
        accesslog.filename   = "/var/logs/madeup/lighttpd-access.log"

        proxy.server = ( "" =>
                           ( ( 
                               "host" => "123.456.789.012", # where you want to proxy to
                               "port" => 5678, # port you want to proxy to
                             ) ) 
                         )   
    }   
}

# Now go to http://made.up.hostname:1234 and you will get directed through to http://123.456.789.012:5678

Configure CNTLM

0) Download cntlm
./configure
make
sudo make install

1) Generate the password hash
cntlm -u username -d DOMAIN -H

2) paste the output into the config file, and add this:
Auth NTLM

3) Run:
cntlm

4) export http_proxy=http://localhost:3128

5) You can now access web pages


See also Charles proxy

wget through a proxy

Edit /etc/wgetrc or ~/.wgetrc

Add these lines:

https_proxy = http://username:password@host:port/
http_proxy = http://username:password@host:port/
ftp_proxy = http://username:password@host:port/

Make apt-get and aptitude work through a proxy

1) sudo vi /etc/apt/apt.conf.d/70debconf
2) It should already have: DPkg::Pre-Install-Pkgs {"/usr/sbin/dpkg-preconfigure --apt || true";};
3) Add this after what's there (no blank lines):
Acquire {
        http {
                Proxy "http://user:pass@host:port";
                No-Cache "false";
                Max-Age "86400";
                No-Store "false";
        };
};
Thanks