Amazon VPC IPSec + BGP with ipsec-tools
10/06/2013 10:04:00 PM
Posted by johnhomer
First time to use github's gists to embed code here. This little weekend project will connect your Ubuntu 12.04 server to Amazon VPC through IPSec.
Usage:
./vpcstart.sh [amazon-generic-config-file.txt]
Where the parameter is the "Generic" and "Vendor Agnostic" config file downloaded from the Amazon Console. You also need to change REMOTE_NET and WAN_INT variables to suit your needs.
You do need the following for IPSec to work:
- Public and static IP Address
- Open ports for UDP 500, protocol AH, protocol ESP, TCP 179 for BGP
EC2 Autoscaling Dynamic De/Registration for HAProxy
4/29/2013 08:32:00 PM
Posted by johnhomer
This is a work-in-progress script I whipped up over the weekend. This assumes the following architecture:
- For a 3-tier architecture
- Autoscaling Group may or may not run on multi-AZ
- Each AZ will have a software loadbalancer
- SNS and SQS are utilized for notification of scaling activities (Starting and Terminating of instances)
- Script runs through through cron and most be installed in between the Web and App servers
- Uses the string "# Begin" to mark where new HAProxy configs will go. So do an "echo '# Begin' >> /etc/haproxy/haproxy.cnf" before running this script
- Step-by-step procedure on how to setup autoscaling is documented here.
#!/usr/bin/perl # EC2 Autoscaling dynamic registration script for HAProxy # Requirements: SNS topic, SQS subscription # Notes: Supposed to be run as a cronjob # John Homer H Alvero # April 29, 2013 # # Install pre-reqs by # yum install perl-Amazon-SQS-Simple perl-Net-Amazon-EC2 --enablerepo=epel use Amazon::SQS::Simple; use Net::Amazon::EC2; my $access_key = ''; my $secret_key = ''; my $queue_endpoint = 'https://sqs.us-east-1.amazonaws.com/123456789012/yourqueue'; my $haproxy_file = '/etc/haproxy/haproxy.cfg'; my $my_az = `wget -qO- http://169.254.169.254/latest/meta-data/placement/availability-zone`; # Create an SQS object my $sqs = new Amazon::SQS::Simple($access_key, $secret_key); # Connect to an existing queue my $q = $sqs->GetQueue($queue_endpoint); my $ec2 = Net::Amazon::EC2->new(AWSAccessKeyId => $access_key, SecretAccessKey => $secret_key); # Retrieve a message while (my $msg = $q->ReceiveMessage()) { $sqs_msg = $msg->MessageBody(); # parse message, get instance id (my $action = $1, $instance_id = $2) if $sqs_msg =~ /(Terminating|Launching).+EC2InstanceId\\\"\:\\\"(i-.{8})/; # do action my $running_instances = $ec2->describe_instances(InstanceId => $instance_id); foreach my $reservation (@$running_instances) { foreach my $instance ($reservation->instances_set) { $pdns_name = $instance->private_dns_name; $instance_az = $reservation->instances_set->[0]->placement->availability_zone; } } if ($my_az eq $instance_az) { if ($action eq "Launching") { print "adding instance id $instance_id $pdns_name\n"; # Get last app number $lastapp = `grep '\# Begin' $haproxy_file -A1000 | grep server | sort -k1 | cut -f6 -d' ' | tail -1`; chomp($lastapp); $lastapp = "app000" if $lastapp eq ""; $lastapp++; # Update haproxy config file system("/bin/echo \" server $lastapp $pdns_name:80 check # $instance_id\" >> $haproxy_file | service haproxy reload"); } elsif ($action eq "Terminating") { print "removing instance id $instance_id\n"; system("sed -i \"/$instance_id/d\" $haproxy_file" | service haproxy reload); } else { die("unhandled exception. exiting.\n"); } # delete from queue $q->DeleteMessage($msg->ReceiptHandle()); } else { print "$instance_id $instance_az does not belong to this AZ.\n"; } # unset variables $instance_id = ""; $launch_id = ""; $action = ""; $pdns_name = ""; $instance_az = ""; }
nginx + python + uwsgi + django + virtualenv + virtualenvwrapper
3/30/2013 11:01:00 AM
Posted by johnhomer
For Systems Engineers coming in from PHP world, installing and configuring the software stack needed to run python + django applications can be a daunting task specially when dealing with multiple python versions. Or, when the operating system python version is not compatible with what is required by the python web application. Here's how I did it with Ubuntu:
- Set locale and timezone
locale-gen en_US.UTF-8 echo "Asia/Singapore" > /etc/timezone dpkg-reconfigure --frontend noninteractive tzdata
- Update packages
apt-get update && apt-get upgrade -y
- Limits
ulimit -n 20000 echo 'fs.file-max = 200000' >> /etc/sysctl.d/20_nginx.conf