Redmine on EC2 Cloud using Alami 2012.03

3/31/2012 02:52:00 PM

Posted by johnhomer

0 comments

Redmine is a flexible project management web application. Written using the Ruby on Rails framework, it is cross-platform and cross-database.


OS: Alami 2012.03

Install Procedure

  • Install packages 
    yum install ruby-devel gcc-c++ openssl-devel httpd httpd-devel mysql-server  mysql-devel make ruby-rdoc libcurl-devel rubygem-rake
  • Rubygems. Version 1.6.2 is the current sweet spot. Using the latest version will result to “depreciated errors in apache error logs”. Using an older version will prevent you from installing bundle 
    cd /tmp/
wget http://production.cf.rubygems.org/rubygems/rubygems-1.6.2.tgz
tar xvfz rubygems-1.6.2.tgz
cd rubygems-1.6.2
ruby setup.rb
  • Passenger 
    gem install passenger
passenger-install-apache2-module
  • Load the passenger apache module. Add the following config in /etc/httpd/conf/httpd.conf
    LoadModule passenger_module /usr/lib/ruby/gems/1.8/gems/passenger-3.0.11/ext/apache2/mod_passenger.so
PassengerRoot /usr/lib/ruby/gems/1.8/gems/passenger-3.0.11
PassengerRuby /usr/bin/ruby
  • Setup apache vhost 
    cat <<'EOF' > /etc/httpd/conf.d/redmine.conf
<VirtualHost *:80>
 ServerName redmine.local
 DocumentRoot /var/www/redmine/public/
 
 <Directory "/var/www/redmine/public/">
  Options Indexes ExecCGI FollowSymLinks
  AllowOverride all
  Order allow,deny
  Allow from all
 </Directory>
</VirtualHost>
EOF
    Note that redmine.local is a local domain and is a manual entry in my workstation's /etc/hosts file. This is done for testing purposes only. For production systems, this may very well be a subdomain under you company's domain name e.g. redmine.acme.com
  • Download and extract redmine 
    cd /tmp
wget http://rubyforge.org/frs/download.php/75910/redmine-1.3.2.tar.gz
tar xvfz redmine-1.3.2.tar.gz
mkdir /var/www/redmine
cp -a redmine-1.3.2/* /var/www/redmine
 
chown -R apache.apache /var/www/redmine && chmod -R 755 /var/www/redmine
  
    touch /var/www/redmine/log/production.log
    chown root.apache /var/www/redmine/log/production.log
chmod 664 /var/www/redmine/log/production.log
  • Prep Gemfile dependency 
    cat <<EOF> /var/www/redmine/Gemfile
source "http://rubygems.org" 
gem "rake", "0.8.3" 
gem "rack", "1.1.0" 
gem "i18n", "0.4.2" 
gem "rubytree", "0.5.2", :require => "tree" 
gem "RedCloth", "~>4.2.3", :require => "redcloth" # for CodeRay
gem "mysql" 
gem "coderay", "~>0.9.7" 
EOF
  • Bundle 
    gem install bundler
cd /var/www/redmine/
bundle install
  • Move CGI files 
    cd /var/www/redmine/public/
mv dispatch.cgi.example dispatch.cgi
mv dispatch.fcgi.example dispatch.fcgi
mv dispatch.rb.example dispatch.rb
mv htaccess.fcgi.example .htaccess
  • Set rails to production environment in /var/www/redmine/config/environment.rb 
    ENV['RAILS_ENV'] ||= 'production'
  • Setup MySQL DB 
    service mysqld start
chkconfig mysqld on
/usr/bin/mysql_secure_installation
mysql -uroot -p -e 'create database redmine character set utf8; grant all on redmine.* to [email protected] identified by "my_passwd";flush privileges';
  • Setup redmine database connection 
    mv /var/www/redmine/config/database.yml.example /var/www/redmine/config/database.yml
vi /var/www/redmine/config/database.yml
 
# In the production section, update username, password and other parameters accordingly like so:
 
production:
  adapter: mysql
  database: redmine
  host: localhost
  username: redmine
  password: my_passwd
  encoding: utf8
  • Create session store 
    cd /var/www/redmine
RAILS_ENV=production bundle exec rake generate_session_store
  • Migrate database models 
    RAILS_ENV=production bundle exec rake db:migrate
  • Load MySQL database schema and default data 
    RAILS_ENV=production bundle exec rake redmine:load_default_data
  • and finally, start Apache 
    service httpd start
chkconfig httpd on
  • you may now open and point your browser to http://redmine.local and login as admin/admin

LDAP Server Installation for openssh-lpk clients

3/30/2012 05:22:00 PM

Posted by johnhomer

1 comments

Since OpenLDAP version 2.3, configuration through cn=config is supported. It is also known as run-time configuration (RTC) or zero downtime configuration.

In accomplishing this task, we will use a cn=config type of configuration since by default, Amazon's Official Linux Ami (ALAMI 2012.03) uses this type.


OS: Alami 2012.03 / CentOS 6.2

Objectives

  • Centralize the administration of linux accounts
  • Centralize the administration of sudo access
  • Use public keys

OpenLDAP Config

  1. Update the system. Fix timezone
    yum -y update
echo -e "ZONE=Asia/Singapore\nUTC=false" > /etc/sysconfig/clock
ln -sf /usr/share/zoneinfo/Asia/Singapore /etc/localtime
  2. Install LDAP packages
    yum install openldap-servers openldap-clients -y
  3. Generate the admin password
    $ slappasswd -s mysecret
{SSHA}IwmKUosglAO6RpcjGDYm04HUu0VgWP0Y
    Note: mysecret will now be your Manager password. You will use this password to execute administrative commands. Displayed after is the corresponding hash. Use the hash in succeeding steps.
  4. TLS settings
    sed -i 's/dc=my-domain,dc=com/dc=johnalvero,dc=com/g' /etc/openldap/slapd.d/cn\=config/olcDatabase\=\{2\}bdb.ldif
 
# Also, add the password and TLS settings in the file
cat <<'EOF'>> /etc/openldap/slapd.d/cn\=config/olcDatabase\=\{2\}bdb.ldif
olcRootPW: {SSHA}IwmKUosglAO6RpcjGDYm04HUu0VgWP0Y
olcTLSCertificateFile: /etc/pki/tls/certs/slapdcert.pem
olcTLSCertificateKeyFile: /etc/pki/tls/certs/slapdkey.pem
EOF
  5. Also add a password for “cn=admin,cn=config” user 
    cat <<'EOF'>> /etc/openldap/slapd.d/cn\=config/olcDatabase\=\{0\}config.ldif
olcRootPW: {SSHA}IwmKUosglAO6RpcjGDYm04HUu0VgWP0Y
EOF
  6. Monitor configuration 
    sed -i 's/cn=manager,dc=my-domain,dc=com/cn=Manager,dc=johnalvero,dc=com/g' /etc/openldap/slapd.d/cn\=config/olcDatabase\=\{1\}monitor.ldif
  7. DB config
    cp /usr/share/openldap-servers/DB_CONFIG.example /var/lib/ldap/DB_CONFIG
chown -R ldap:ldap /var/lib/ldap/
  8. Generate SSL keys 
    openssl req -new -x509 -nodes -out /etc/pki/tls/certs/slapdcert.pem -keyout /etc/pki/tls/certs/slapdkey.pem -days 365
chown -Rf root.ldap /etc/pki/tls/certs/slapdcert.pem 
chown -Rf root.ldap /etc/pki/tls/certs/slapdkey.pem

Schemas

  1. Add openssh-lpk shema 
    cat <<'EOF'> /etc/openldap/slapd.d/cn=config/cn=schema/cn={21}openssh-lpk.ldif
dn: cn={21}openssh-lpk
objectClass: olcSchemaConfig
cn: {21}openssh-lpk
olcAttributeTypes: {0}( 1.3.6.1.4.1.24552.500.1.1.1.13 NAME 'sshPublicKey' DES
 C 'MANDATORY: OpenSSH Public key' EQUALITY octetStringMatch SYNTAX 1.3.6.1.4.
 1.1466.115.121.1.40 )
olcObjectClasses: {0}( 1.3.6.1.4.1.24552.500.1.1.2.0 NAME 'ldapPublicKey' DESC
  'MANDATORY: OpenSSH LPK objectclass' SUP top AUXILIARY MAY ( sshPublicKey $ 
 uid ) )
structuralObjectClass: olcSchemaConfig
entryUUID: 135574f4-bda0-102f-9362-0b01757f31d8
creatorsName: cn=config
createTimestamp: 20110126135819Z
entryCSN: 20110126135819.712350Z#000000#000#000000
modifiersName: cn=config
modifyTimestamp: 20110126135819Z
EOF
  2. Add the sudoers schema 
    cat<<'EOF'> /etc/openldap/slapd.d/cn=config/cn=schema/cn={23}sudo.ldif
dn: cn={23}sudo
objectClass: olcSchemaConfig
cn: {23}sudo
olcAttributeTypes: {0}( 1.3.6.1.4.1.15953.9.1.1 NAME 'sudoUser' DESC 'User(s) 
 who may  run sudo' EQUALITY caseExactIA5Match SUBSTR caseExactIA5SubstringsMa
 tch SYNTAX 1.3.6.1.4.1.1466.115.121.1.26 )
olcAttributeTypes: {1}( 1.3.6.1.4.1.15953.9.1.2 NAME 'sudoHost' DESC 'Host(s) 
 who may run sudo' EQUALITY caseExactIA5Match SUBSTR caseExactIA5SubstringsMat
 ch SYNTAX 1.3.6.1.4.1.1466.115.121.1.26 )
olcAttributeTypes: {2}( 1.3.6.1.4.1.15953.9.1.3 NAME 'sudoCommand' DESC 'Comma
 nd(s) to be executed by sudo' EQUALITY caseExactIA5Match SYNTAX 1.3.6.1.4.1.1
 466.115.121.1.26 )
olcAttributeTypes: {3}( 1.3.6.1.4.1.15953.9.1.4 NAME 'sudoRunAs' DESC 'User(s)
  impersonated by sudo' EQUALITY caseExactIA5Match SYNTAX 1.3.6.1.4.1.1466.115
 .121.1.26 )
olcAttributeTypes: {4}( 1.3.6.1.4.1.15953.9.1.5 NAME 'sudoOption' DESC 'Option
 s(s) followed by sudo' EQUALITY caseExactIA5Match SYNTAX 1.3.6.1.4.1.1466.115
 .121.1.26 )
olcObjectClasses: {0}( 1.3.6.1.4.1.15953.9.2.1 NAME 'sudoRole' DESC 'Sudoer En
 tries' SUP top STRUCTURAL MUST cn MAY ( sudoUser $ sudoHost $ sudoCommand $ s
 udoRunAs $ sudoOption $ description ) )
structuralObjectClass: olcSchemaConfig
entryUUID: 13557a62-bda0-102f-9364-0b01757f31d8
creatorsName: cn=config
createTimestamp: 20110126135819Z
entryCSN: 20110126135819.712350Z#000000#000#000000
modifiersName: cn=config
modifyTimestamp: 20110126135819Z
EOF
  3. Make initial files for base, group, people and sudoers

    base.ldif
    dn: dc=johnalvero,dc=com
dc: johnalvero
objectClass: top
objectClass: domain

dn: ou=People,dc=johnalvero,dc=com
ou: People
objectClass: top
objectClass: organizationalUnit

dn: ou=Group,dc=johnalvero,dc=com
ou: Group
objectClass: top
objectClass: organizationalUnit
    newgroup.ldif
    dn: cn=phstaff,ou=Group,dc=johnalvero,dc=com
objectClass: posixGroup
objectClass: top
cn: phstaff
userPassword: {crypt}x
gidNumber: 1000
    newpeople.ldif
    dn: uid=john,ou=People,dc=johnalvero,dc=com
uid: john
cn: John Alvero
objectClass: account
objectClass: posixAccount
objectClass: top
objectClass: shadowAccount
objectClass: ldapPublicKey
userPassword: {CRYPT}cr5y5J6F67Ci2
shadowLastChange: 15140
shadowMin: 0
shadowMax: 99999
shadowWarning: 7
loginShell: /bin/bash
uidNumber: 1000
gidNumber: 1000
homeDirectory: /home/john
sshPublicKey: myrsakeyhere_changeme
    newsudoers.ldif
    dn: ou=sudoers,dc=johnalvero,dc=com
objectclass: organizationalUnit
ou: sudoers

dn: cn=defaults,ou=sudoers,dc=johnalvero,dc=com
objectClass: top
objectClass: sudoRole
cn: defaults
description: Default sudoOption's go here
sudoOption: logfile=/var/log/sudolog

dn: cn=root,ou=sudoers,dc=johnalvero,dc=com
objectClass: top
objectClass: sudoRole
cn: root
sudoUser: root
sudoHost: ALL
sudoCommand: ALL

# Sample sudo user
dn: cn=john,ou=sudoers,dc=johnalvero,dc=com
objectClass: top
objectClass: sudoRole
cn: john
sudoUser: john
sudoHost: ALL
sudoCommand: ALL
sudoOption: !authenticate
  4. We can now start the services and add the entries: 
    chkconfig slapd on
service slapd start
ldapadd -x -W -D "cn=Manager,dc=johnalvero,dc=com" -f base.ldif
ldapadd -x -W -D "cn=Manager,dc=johnalvero,dc=com" -f newgroup.ldif
ldapadd -x -W -D "cn=Manager,dc=johnalvero,dc=com" -f newpeople.ldif
ldapadd -x -W -D "cn=Manager,dc=johnalvero,dc=com" -f newsudoers.ldif
  5. And try searching 
    ldapsearch -x -b "dc=johnalvero,dc=com"
ldapsearch -H "ldap://johnalvero.com:389" -x -b "dc=johnalvero,dc=com"

Configuring ssh-lpk Clients

  • Install the packages
    yum install openssh-ldap nss-pam-ldapd
  • Setup LDAP config. This will modify various LDAP files including that of PAM 
    authconfig --disablenis --enablemkhomedir --enableshadow --enablelocauthorize --enableldap --ldapserver=johnalvero.com --enablemd5 --ldapbasedn=dc=johnalvero,dc=com --updateall
 
# Or, you can use a curses-based application. Enable necessary options based on the above command but --enablemkhomedir is not available in authconfig-tui 
 
authconfig-tui
  • Allow SSH public-key login 
    cat <<'EOF'> /etc/ssh/ldap.conf
uri ldap://johnalvero.com/
base dc=johnalvero,dc=com
ssl no
EOF
 
cat <<'EOF'>> /etc/ssh/sshd_config
AuthorizedKeysCommand /usr/libexec/openssh/ssh-ldap-wrapper
AuthorizedKeysCommandRunAs nobody
EOF
  • Tell system to lookup sudoers info from ldap or files respectively
    echo 'sudoers: ldap files' >> /etc/nsswitch.conf
 
cat <<'EOF'>> /etc/nslcd.conf
ou=sudoers,dc=johnalvero,dc=com
sudoers_base ou=sudoers,dc=johnalvero,dc=com
EOF
  • Restart sshd 
    service sshd restart

nslcd start/restart hack

Since, Alami's nss-pam-ldapd suffers from the same bug described in https://bugzilla.redhat.com/show_bug.cgi?id=760843. I have made a patch for /etc/init.d/nslcd. This will make nss-pam-ldapd play nicely with sudo. Essentially, what is does is comment out “sudo-ldap”-related config in /etc/nslcd.conf just before starting the daemon and uncommenting these configs right after.

If you dont apply this patch, you will get errors in restarting/starting nslcd.

There's another option though, instead of installing nss-pam-ldapd from the default amzn-main repo, you can install the one in http://danielhall.me/shared/rpms/nss-pam-ldapd/ and forget about this patch. 
*** /etc/init.d/nslcd 2012-03-30 13:42:53.859493505 +0800
--- /root/nslcd 2012-03-30 13:28:08.120237533 +0800
***************
*** 29,35 ****
--- 29,39 ----
 
  start() {
      echo -n $"Starting $prog: "
+     sed -i 's/^ou/#ou/' /etc/nslcd.conf
+     sed -i 's/^sudoers_base/#sudoers_base/' /etc/nslcd.conf
      daemon $program
+     sed -i 's/^#ou/ou/' /etc/nslcd.conf
+     sed -i 's/#sudoers_base/sudoers_base/' /etc/nslcd.conf
      RETVAL=$?
      echo
      [ $RETVAL -eq 0 ] && touch /var/lock/subsys/$prog
and then, patch by: 
cd /etc/init.d/
patch -i /path/to/patch/nslcd.patch

MySQL Cluster 7.2 with MySQL Cluster Management (MCM)

3/26/2012 04:58:00 PM

Posted by johnhomer

3 comments

This guide describes the step-by-step procedure on setting up a test MySQL Cluster using the MySQL Cluster Management Console (MCM).

Architecture

  • A total of four physical or virtual servers (known as Cluster Nodes in MySQL Cluster term)
    • Two cluster nodes will serve as Data Node (ndb, this is where our data reside)
    • Two other servers will serve as both SQL Nodes (mysqld) and Management Nodes (ndb_mgmd)
  • All four servers will only need to have the MySQL Cluster Management Agent installed
  • Each Data Node will have a pair of ndbd processs to maintain the replica assigned to it
  • Clients (PHP web application) will connect to SQL Nodes (mysqld)
  • This setup is supposed to survive a single Cluster Node failure
     

Requirements

  • VMware workstation, Hyper-V, Xen or even Amazon AWS
  • Four server instances
  • Each instance should have at least 1GB RAM (although Management Nodes/API Nodes can have lesser RAM. more about it during the steps)
  • CentOS 6.2
  • SELinux disabled
  • iptables disabled
  • MySQL Cluster Manager 1.1.4+Cluster for Red Hat and Oracle Linux 5 x86 (64-bit) - from oracle edelivery site. This package includes the MCM Agent and MySQL Cluster software

Installing the MCM Agent with MySQL Cluster

All four server instances should have this management agent. This is the only manual process that needs to be done on individual nodes. All other activities can be done through the MCM commandline console
  • for configuration simplicity, register all nodes in the hosts file
    cat <<EOF>> /etc/hosts
192.168.0.10 site1
192.168.0.11 site2
192.168.0.12 site3
192.168.0.13 site4
EOF
  • copy the MCM Agent to /tmp
  • prepare the MCM agent files
    cd /tmp
unzip V31807-01-MCM-Cluster.zip
mkdir /opt/mcm
tar xvz --directory=/opt/mcm/ --strip-components=1 -f mcm-1.1.5_64-cluster-7.2.5_64-linux-rhel5-x86.tar.gz
  • add users and fix directory permissions
    groupadd clustermanager && useradd -M -d /opt/mcm/ -g clustermanager clustermanager
chown -R clustermanager.clustermanager /opt/mcm/
  • start the MCM daemon
    sudo -u clustermanager /opt/mcm/bin/mcmd &
At this point, we are done with the rest of the Cluster Node instances, all of the steps from this point forward can be done at the first or second server instances (the management servers).

Firing the First Cluster

We are now ready to create our first cluster. The main steps are: create a site, add a package, create a cluster and finally start the cluster.
  • connect to MCM command-line console. The default password is super
    /opt/mcm/cluster/bin/mysql -h127.0.0.1 -P1862 -uadmin -psuper --prompt='mcm> '
  • create a site
    mcm> create site --hosts=site1,site2,site3,site4 mysite;
  • create a package. A package is like a MySQL instance composed of MySQL binaries, libraries and configuration files. The name of the package we are going to create is 7.2
    mcm> add package --basedir=/opt/mcm/cluster 7.2;
  • create a cluster
    ndb_mgmd - Cluster management node on site1 & site2
    ndbd - Single threaded Data node on site3 & site4 (twice. Each machine hold couple of data nodes for our demo)
    mysqld - MySQL interface node on site1 & site2
    ndbapi - for API interface
    ndbmtd - for the multi-threaded NDB engine
    mcm> create cluster --package=7.2 [email protected],[email protected],[email protected],[email protected],[email protected],[email protected],[email protected],[email protected] mycluster;
  • if you do not have 1GB RAM for Data Node instances, you may need to modify innodb_buffer_pool_size here so that MySQL will start. This is also the right time to make other MySQL tuning
    get -d innodb_buffer_pool_size:mysqld mycluster;

# This enables me to run cluster nodes with only 2GB RAM for testing purposes
set innodb_buffer_pool_size:mysqld:51=16777216 mycluster;
set innodb_buffer_pool_size:mysqld:52=16777216 mycluster;

# Do this of you plan on storing large datasets
set DataMemory:ndbd=3145728000 mycluster;
set IndexMemory:ndbd=536870912 mycluster;
  • Start the cluster
    mcm> start cluster -B mycluster;
  • See the status of the cluster
    mcm> show status -r mycluster;
  • Connecting through MySQL Client 
    mkdir  /var/lib/mysql/
ln -s /tmp/mysql.mycluster.51.sock /var/lib/mysql/mysql.sock
mysql -uroot

Other tasks

  • Changing from a single-threaded cluster node to multi-threaded
    mcm> change process ndbd:3=ndbmtd mycluster;
  • You don't normally need to manually do a rolling restart since MySQL cluster will take care of it if you make changes that requires a restart. But if you need it, here's how it's done
    mcm> restart cluster -B mycluster;
  • Here's how to do an online upgrade of cluster software. We call the new package as 7.3
    mcm> add package --basedir=/usr/local/mysql_7_3 7.3;
mcm> upgrade cluster --package=7.3 mycluster;
  • Adding new hosts



    # Initialize the new hosts. Also take note that you need to add necessary entries in /etc/hosts for the new hosts
 
mcm> add hosts --hosts=site5,site6 mysite;
mcm> add package --basedir=/opt/mcm/cluster --hosts=site5,site6 7.2;
 
# Finally, add it to the cluster.
# Note that the we are also adding API instances on site1 and site2. Also, as pointed out by Andrew Morgan, we have to guess the node-id's of the the new mysqld's. In our case, the will be node-id's are 53 and 54 following the output in show statur -r mycluster
mcm> add process [email protected],[email protected],[email protected],[email protected],[email protected],[email protected] -s port:mysqld:53=3307,port:mysqld:54=3307 mycluster;
mcm> start process --added mycluster; 
 
# On any of the API servers, do the following commands to repartition the 
# existing cluster and use the new data nodes
 
mysql> ALTER ONLINE TABLE [table-name] REORGANIZE PARTITION;
mysql> OPTIMIZE TABLE [table-name];

Deleting the cluster

stop cluster -B mycluster;
 
delete cluster mycluster;
delete package 7.2;
delete site mysite;

other useful commands

list clusters mysite;
list packages mysite;
list sites;


Credits to Andrew Morgan for the write-up and images.

AWS Autoscaling How To

3/23/2012 05:26:00 PM

Posted by johnhomer

3 comments

  • Setup autoscaling and cloudwatch CLI
    cd /home/john && mkdir ec2 && cd ec2
 
wget http://ec2-downloads.s3.amazonaws.com/AutoScaling-2011-01-01.zip
unzip AutoScaling-2011-01-01.zip
wget http://ec2-downloads.s3.amazonaws.com/CloudWatch-2010-08-01.zip
unzip CloudWatch-2010-08-01.zip
 
export EC2_HOME=/home/john/ec2
export PATH=$PATH:$EC2_HOME/bin  
export JAVA_HOME=/usr
export EC2_PRIVATE_KEY=/home/john/pk.pem # You need to get this file from your AWS Credentials
export EC2_CERT=/home/john/cert.pem      # You need to get this file from your AWS Credentials
 
export AWS_AUTO_SCALING_HOME=$EC2_HOME/AutoScaling-1.0.49.1
export AWS_AUTO_SCALING_URL=https://autoscaling.us-east-1.amazonaws.com
export PATH=$PATH:$AWS_AUTO_SCALING_HOME/bin
 
export AWS_CLOUDWATCH_HOME=$EC2_HOME/CloudWatch-1.0.12.1
export PATH=$PATH:$AWS_CLOUDWATCH_HOME/bin
  • Setup variables
    EC2_REGION="us-east-1"
ZONE="us-east-1d"
SECURITY_GROUP="default"
INSTANCE_SIZE="t1.micro"
LB_NAME="autoscalelb"
LC_NAME="autoscalelc"
LC_IMAGE_ID="ami-31814f58" # Could be any AMI of choice
LC_KEY="john-east"  # You need to create this key in the AWS console
SG_NAME="autoscalesg"
 
UP_POLICY_NAME="MyScaleUpPolicy"
DOWN_POLICY_NAME="MyScaleDownPolicy"
HIGH_CPU_ALRM_NAME="MyHighCPUAlarm"
LOW_CPU_ALRM_NAME="MyLowCPUAlarm"
MIN_SIZE=1
MAX_SIZE=4  # For testing purposes, set to 1
DOWN_THRESHOLD=40  # scale down when average CPU load is 40% or below 
UP_THRESHOLD=80  # scale up when average CPU load reaches 80%
  • Create Launch Config
    as-create-launch-config $LC_NAME --image-id $LC_IMAGE_ID --instance-type $INSTANCE_SIZE --group $SECURITY_GROUP --key $LC_KEY --block-device-mapping '/dev/sda2=ephemeral0' --user-data-file ud.txt
  • Create Autoscaling Group
    as-create-auto-scaling-group $SG_NAME --availability-zones $ZONE --launch-configuration $LC_NAME --min-size $MIN_SIZE --max-size $MAX_SIZE --load-balancers $LB_NAME
  • Trigger scaling up
    ARN_HIGH=`as-put-scaling-policy $UP_POLICY_NAME --auto-scaling-group $SG_NAME --adjustment=1 --type ChangeInCapacity --cooldown 300`
mon-put-metric-alarm $HIGH_CPU_ALRM_NAME --comparison-operator GreaterThanThreshold --evaluation-periods 1 --metric-name CPUUtilization --namespace "AWS/EC2" --period 600 --statistic Average --threshold $UP_THRESHOLD --alarm-actions $ARN_HIGH --dimensions "AutoScalingGroupName=$SG_NAME"
  • Trigger scaling down
    ARN_LOW=`as-put-scaling-policy $DOWN_POLICY_NAME --auto-scaling-group $SG_NAME --adjustment=-1 --type ChangeInCapacity --cooldown 300`
mon-put-metric-alarm $LOW_CPU_ALRM_NAME --comparison-operator LessThanThreshold --evaluation-periods 1 --metric-name CPUUtilization --namespace "AWS/EC2" --period 600 --statistic Average --threshold $DOWN_THRESHOLD --alarm-actions $ARN_LOW --dimensions "AutoScalingGroupName=$SG_NAME"

#Post notifications to SNS (needed for dynamic registration)
as-put-notification-configuration $SG_NAME --topic-arn arn:aws:sns:us-east-1:123456789012:topic01 --notification-types autoscaling:EC2_INSTANCE_LAUNCH, autoscaling:EC2_INSTANCE_TERMINATE
  • Pausing and Restarting autoscaling activities
    as-suspend-processes $SG_NAME
as-resume-processes $SG_NAME
  • Expand to other Availability Zones
    as-update-auto-scaling-group $SG_NAME --availability-zones us-east-1a, us-east-1b, us-east-1c --min-size 3
elb-describe-instance-health  $LB_NAME
elb-enable-zones-for-lb  $LB_NAME  --headers --availability-zones us-east-1c
  • Clean up
    as-update-auto-scaling-group $SG_NAME --min-size 0 --max-size 0
as-delete-auto-scaling-group $SG_NAME
as-delete-launch-config $LC_NAME
 
mon-delete-alarms $HIGH_CPU_ALRM_NAME $LOW_CPU_ALRM_NAME

References

http://docs.amazonwebservices.com/AutoScaling/latest/DeveloperGuide/US_SetUpASLBApp.html

Apache + PHP-FPM + mod_fastcgi

3/22/2012 05:14:00 PM

Posted by johnhomer

0 comments

OS: ALAMI 2011.09

  1. Install pre-req
    yum -y install make libtool httpd-devel apr-devel apr
  2. Install Apache and PHP-FPM
    yum -y install httpd php-fpm php-cli
  3. Install mod_fastcgi
    mkdir /root/files ; cd /root/files
wget http://www.fastcgi.com/dist/mod_fastcgi-current.tar.gz
tar -zxvf mod_fastcgi-current.tar.gz
cd mod_fastcgi-2.4.6/
cp Makefile.AP2 Makefile
make top_dir=/usr/lib/httpd
make install top_dir=/usr/lib/httpd
  4. Setup fastcgi folder
    mkdir /var/www/fcgi-bin
cp $(which php-cgi) /var/www/fcgi-bin/
chown -R apache: /var/www/fcgi-bin
chmod -R 755 /var/www/fcgi-bin
  5. Load the module and setup php handler in /etc/httpd/conf.d/php-fpm.conf
    LoadModule fastcgi_module modules/mod_fastcgi.so
LoadModule actions_module modules/mod_actions.so
 
<IfModule mod_fastcgi.c>
        ScriptAlias /fcgi-bin/ "/var/www/fcgi-bin/"
        FastCGIExternalServer /var/www/fcgi-bin/php-cgi -host 127.0.0.1:9000 -pass-header Authorization
        AddHandler php-fastcgi .php
        Action php-fastcgi /fcgi-bin/php-cgi
</IfModule>
  6. Start the servers
    chkconfig php-fpm on
chkconfig httpd on
service php-fpm start
service httpd start

Scalr + Ubuntu 11.10 Installation

3/21/2012 03:18:00 PM

Posted by johnhomer

0 comments

Environment

OSUbuntu Server 11.10 Oneiric Ocelot
Scalr Ver.scalr-2.5.r6086
Application Folder/var/www/app
Application VHostscalr.local

Installation

  1. Install required packages
    apt-get install apache2-mpm-prefork php5 php5-mysql php5-curl php5-mcrypt php5-snmp php-pear rrdtool librrd-dev libcurl4-openssl-dev mysql-server snmp libssh2-php apparmor-utils
  2. Unpack scalr application files. This assumes that the scalr package is at /tmp folder
    cd /tmp
tar xvfz scalr-2.5.r6086.tar.gz
mv scalr-2.5.r6086/app /var/www
chown root.www-data /var/www/app -R
chmod g+w /var/www/app/etc/.cryptokey
chmod g+w /var/www/app/cache -R
  3. Before we proceed. Let's fix some code. This will resolve Bind DNS issues. Comment out the following code in /var/www/app/src/Scalr/Net/Dns/Bind/RemoteBind.php (line 36-37). The commented code should look like:
    //                        if (count($this->zonesConfig) == 0)
//                                throw new Exception("Zones config is empty");
  4. Setup MySQL
    mysql -uroot -p -e 'create database scalr; grant all on scalr.* to [email protected] identified by "<scalrpassword>";flush privileges;'
 
cat /tmp/scalr-2.5.r6086/sql/structure.sql | mysql -uscalr -p scalr
cat /tmp/scalr-2.5.r6086/sql/data.sql | mysql -uscalr -p scalr
  5. Tell scalr how to connect to MySQL by modifying /var/www/app/etc/config.ini. The [db] part of that file should look similar to:
    driver=mysqli
host = "localhost"
name = "scalr"
user = "scalr"
pass = "<scalrpassword>"

    Note: The pass parameter should reflect the same password stated in the previous step (step 4)
  6. Setup and enable the Apache VHost
    cat <<EOF> /etc/apache2/sites-available/scalr 
<VirtualHost *:80>
        ServerAdmin [email protected]
        ServerName scalr.local
        DocumentRoot /var/www/app/www
        <Directory /var/www/app/www>
                Options Indexes FollowSymLinks MultiViews
                AllowOverride All
        </Directory>
</VirtualHost>
EOF
 
a2ensite scalr
  7. Install additional PHP modules
    pecl install rrd
echo 'extension=rrd.so' >  /etc/php5/apache2/conf.d/rrd.ini
pecl install pecl_http
echo 'extension=http.so' >  /etc/php5/apache2/conf.d/http.ini
a2enmod rewrite
service apache2 restart
  8. At this point, we can now check if our environment has all the Apache and PHP modules required to run scalr. Open and point your browswer to http://scalr.local/testenvironment.php. Note that scalr.local is a local domain so make necessary changes in your own DNS resolvers or your workstations /etc/hosts.
  9. Cron jobs
    cat <<EOF> /etc/cron.d/scalr 
*/2 * * * *  root /usr/bin/php -q  /var/www/app/cron-ng/cron.php --Poller
* * * * *  root /usr/bin/php -q /var/www/app/cron/cron.php --Scheduler
*/10 * * * *  root /usr/bin/php -q  /var/www/app/cron/cron.php --MySQLMaintenance
* * * * *  root /usr/bin/php -q  /var/www/app/cron/cron.php --DNSManagerPoll
17 5 * * * root /usr/bin/php -q  /var/www/app/cron/cron.php --RotateLogs
*/2 * * * *  root /usr/bin/php -q  /var/www/app/cron/cron.php --EBSManager
*/20 * * * *  root /usr/bin/php -q  /var/www/app/cron/cron.php --RolesQueue
*/5 * * * *  root /usr/bin/php -q  /var/www/app/cron-ng/cron.php --DbMsrMaintenance
*/2 * * * *  root /usr/bin/php -q  /var/www/app/cron-ng/cron.php --Scaling
*/5 * * * *  root /usr/bin/php -q  /var/www/app/cron/cron.php --DBQueueEvent
*/2 * * * *  root /usr/bin/php -q  /var/www/app/cron/cron.php --SzrMessaging
*/4 * * * *  root /usr/bin/php -q  /var/www/app/cron/cron.php --RDSMaintenance
*/2 * * * *  root /usr/bin/php -q  /var/www/app/cron/cron.php --BundleTasksManager
* * * * *  root /usr/bin/php -q  /var/www/app/cron-ng/cron.php --ScalarizrMessaging
* * * * *  root /usr/bin/php -q  /var/www/app/cron-ng/cron.php --MessagingQueue
*/2 * * * *  root /usr/bin/php -q  /var/www/app/cron-ng/cron.php --DeployManager
EOF
  10. Bind
    apt-get install bind9
chmod g+w /etc/bind/named.conf
echo 'include "/var/named/etc/namedb/client_zones/zones.include";' >> /etc/bind/named.conf
mkdir -p /var/named/etc/namedb/client_zones
chown root.bind /var/named/etc/namedb/client_zones
chmod 2775 /var/named/etc/namedb/client_zones
 
# New domains will go to this file
echo ' ' > /var/named/etc/namedb/client_zones/zones.include
chown root.bind /var/named/etc/namedb/client_zones/zones.include
chmod g+w /var/named/etc/namedb/client_zones/zones.include
 
# Put Bind in apparmor complain mode. This will allow Bind to include **zones.include** as mentioned above. May need to setup a more secure configuration
aa-complain /usr/sbin/named
 
# Restart
service bind9 restart

Next Steps

  1. Login as Scalr Admin
    http://scalr.local
 
Email: admin
Password: admin
    Note: When logging in as admin, you may see an “Insufficient permissions” error message. I have no idea how to fix that, but you may ignore that error message.
  2. Change Admin password (upper right corner of the screen)
    admin->Profile
  3. Change Core settings
    Settings->Core settings
  4. Create a scalr user. Then login as that user to create your first server farm
    Accounts->Manage
  5. Create your first server farm as described in the Getting Started Guide

Subscribe to: Posts (Atom)