Since the last update of VPS on Dreamhost, that my ‘machine’ is f*cked up…
Till then, i’v managed to make it stable on nginx + php-fpm or something…

Now that 502 and 504 nightmare is back.

In the journey to solve my PHP-CGI problems on dreamhost, i’v installed monit.
This is how I made it and the tweaks that i’v gathered from other guys.

Monit is easiest to install through apt-get.

apt-get install monit

BUT on dreamhost this will install us the version 5.1.1, a older one, currently the latest version is 5.6, and for my needs I need at least the 5.3.

I will explain how I’v upgraded from 5.1.1 to 5.6.
Credits go to Luc Castera.

Update monit 5.1.1 to monit 5.6

My “/home/utils/webserver_response.sh”

#!/bin/bash
curl -sL -w "%{http_code}\\n" "http://www.mydomain.com" -o /dev/null -m 20  --connect-timeout 20
exit $?

My “/etc/monit/conf.d/webserverrequest.cfg”

check program script with path "/home/utils/webserver_response.sh"
if status == 502 then exec "/home/utils/nginx_cron.sh"
if status == 504 then exec "/home/utils/nginx_cron.sh"
if status == 000 then exec "/home/utils/nginx_cron.sh"

My nginx_cron.sh

Got from http://files.gimmesoda.com/nginx_cron.sh

#!/bin/bash

# PHP FastCGI BandAid on DH
# Version 4, 10-28-2011
# By JuanJose Galvez (DH Tech Support)
# Set this up as a cron job under root
# I normally save this under /root/php_fastcgi.sh
# *     *       *       *       *       /bin/sh /root/php_fastcgi.sh;
# - Added support for PHP 5.3
# - Removed lsof check
# - Writing to log
# - Remove while loops
# - Added user name to log and fixed date -Tyler

#USERS=`ls -la /home/ | grep -v root | awk {'print $3'} | sed '/^$/d'`
USERS=`grep pragma /dh/nginx/servers/*/nginx.conf | awk {'print $3'} | sort | uniq`
for user in $USERS
do
        echo "User: $user"

        PHPCOUNT=`ps aux | grep '.p*.sock' | grep $user | wc -l`
        echo "PHP Processes: $PHPCOUNT"

        if [ "$PHPCOUNT" -lt 3 ]
        then
            DATE=`date`
            echo "$DATE: Only $PHPCOUNT PHP Processes for $user" >> /root/cron_log.txt
            pkill -9 -f .p*.sock -u $user
            /etc/init.d/nginx startphp
            PHPCOUNT=`ps aux | grep '.p*.sock' | grep $user | wc -l`
        fi
done

Once monit is installed we can edit the configuration file:

vi /home/utils/monit-5.6/monitrc

this is what I actually have on my monitrc

set daemon 10
set logfile /var/log/monit.log
set mailserver 127.0.0.1
set mail-format {from:[email protected]}
set alert [email protected] only on {timeout, nonexist}
set httpd port 2812 and
use address localhost
allow localhost
include /etc/monit/conf.d/*.cfg

all the rest (default) is there commented…

I had to edit  /etc/default/monit

and change value from startup from 0 to 1

I tried to start monit and I got the following error…

Starting daemon monitor: monit/etc/monit/monitrc:246: Warning: include files not found '/etc/monit/conf.d/*'

Some people say that we just need to create a dummy.conf file on /etc/monit/conf.d/ and move one… but wasn’t my case, in fact I need a .conf there.

The /etc/monit/conf.d/nginx.conf

check process nginx
with pidfile /dh/nginx/servers/httpd-xxxxxxx/var/logs/nginx.pid
start program = "/etc/init.d/nginx start"
stop program = "/etc/init.d/nginx stop"
if failed host mydomain.com port 80 protocol HTTP then restart #set your server IP that runs nginx
if failed host mydomain.com port 80 protocol HTTP then alert
if 5 restarts with 5 cycles then alert
check program webserverresponse with path /root/webserver_response.sh
if match "502" then exec "/root/nginx_cron.sh"
if match "504" then exec "/root/nginx_cron.sh"
if match "502" then alert
depends on nginx

Please notice that check program was only introduced in monit 5.3.
I guess that apt-get install monit on dreamhost install us the monit 5.1.1.

My webserver_response.sh
Got from http://lists.gnu.org/archive/html/monit-general/2012-11/msg00032.html

#!/bin/bash
curl -sL -w "%{http_code}\\n" "http://localhost" -o /dev/null --connect-timeout 5 --max-time 5
exit $?

I’v added –connect-timeout 5 –max-time 5 at the end to get it faster and avoid endless waitings… 

My nginx_cron.sh
Got from http://files.gimmesoda.com/nginx_cron.sh

#!/bin/bash

# PHP FastCGI BandAid on DH
# Version 4, 10-28-2011
# By JuanJose Galvez (DH Tech Support)
# Set this up as a cron job under root
# I normally save this under /root/php_fastcgi.sh
# *     *       *       *       *       /bin/sh /root/php_fastcgi.sh;
# - Added support for PHP 5.3
# - Removed lsof check
# - Writing to log
# - Remove while loops
# - Added user name to log and fixed date -Tyler

#USERS=`ls -la /home/ | grep -v root | awk {'print $3'} | sed '/^$/d'`
USERS=`grep pragma /dh/nginx/servers/*/nginx.conf | awk {'print $3'} | sort | uniq`
for user in $USERS
do
        echo "User: $user"

        PHPCOUNT=`ps aux | grep '.p*.sock' | grep $user | wc -l`
        echo "PHP Processes: $PHPCOUNT"

        if [ "$PHPCOUNT" -lt 3 ]
        then
            DATE=`date`
            echo "$DATE: Only $PHPCOUNT PHP Processes for $user" >> /root/cron_log.txt
            pkill -9 -f .p*.sock -u $user
            /etc/init.d/nginx startphp
            PHPCOUNT=`ps aux | grep '.p*.sock' | grep $user | wc -l`
        fi
done

Solving monit: error connecting to the monit daemon

Monit can present an HTTP interface which I didn’t enable as I thought it was just for me, it turns out it’s also for the command line tools!

It’s really easy to enable, in /etc/monit.conf or wherever your conf file is located just add

set httpd port 2812 and
use address localhost
allow localhost

and restart monit with

service monit restart

[ps154868]$ sudo ./monit start
monit: The control file ‘/home/utils/monit-5.6/monitrc’ must be owned by you.
[ps154868]$ ls -la
total 3364
drwxr-xr-x 8 501 staff 4096 Set 5 03:03 .
drwxrwxrwx 3 root root 113 Set 5 02:44 ..
drwxr-xr-x 2 root root 10 Set 5 02:55 .libs
-rw-r–r– 1 501 staff 36073 Set 3 09:03 CHANGES
-rw-r–r– 1 501 staff 34940 Ago 20 00:37 COPYING
-rw-r–r– 1 root root 43265 Set 5 02:45 Makefile
-rw-r–r– 1 501 staff 3784 Set 3 05:55 Makefile.am
-rw-r–r– 1 501 staff 42849 Set 3 05:55 Makefile.in
-rw-r–r– 1 501 staff 3986 Ago 20 00:37 README
-rw-r–r– 1 501 staff 344252 Set 3 05:51 aclocal.m4
-rwxr-xr-x 1 501 staff 501 Mai 10 2012 bootstrap
drwxr-xr-x 2 501 staff 113 Set 5 02:44 config
-rw-r–r– 1 root root 257210 Set 5 02:45 config.log
-rwxr-xr-x 1 root root 64542 Set 5 02:45 config.status
-rwxr-xr-x 1 501 staff 495482 Set 3 05:55 configure
-rw-r–r– 1 501 staff 25097 Set 3 05:50 configure.ac
drwxr-xr-x 3 501 staff 4096 Set 5 02:45 contrib
drwxr-xr-x 2 501 staff 50 Set 5 02:44 doc
drwxr-xr-x 7 501 staff 4096 Set 5 02:54 libmonit
-rwxr-xr-x 1 root root 293314 Set 5 02:45 libtool
-rwxr-xr-x 1 root root 1565950 Set 5 02:55 monit
-rw-r–r– 1 501 staff 171084 Set 3 05:48 monit.1
-rwxrwxrwx 1 501 staff 10898 Set 5 03:03 monitrc
drwxr-xr-x 6 501 staff 4096 Set 5 02:55 src
[ps154868]$ chown root monitrc
[ps154868]$ sudo ./monit start
monit: The control file ‘/home/utils/monit-5.6/monitrc’ must have permissions no more than -rwx—— (0700); right now permissions are -rwxrwxrwx (0777).
[ps154868]$ chmod 0700 monitrc
[ps154868]$ sudo ./monit start
monit: please specify the configured service name or ‘all’ after start
[ps154868]$

Some credits
Houssan A. Hijazi, Analista de Sistemas www.lojasnoparaguai.com.br / www.libanovivo.org
http://lists.gnu.org/archive/html/monit-general/2012-11/msg00032.html