Web International Awards

payday loan

3

MAY 2011 4

Start passenger standalone on server restart or on service failure

We've had this problem with our own servers and rails apps running on Passenger standalone and turns out there are a lot of you experiencing the same problems from time to time. This short tutorial will help you solve the following problems:

  1. How can I keep Passenger Standalone up even after a server restart?
  2. How do I run health checks on Passenger Standalone?
  3. How do I automatically restart and spawn a rails app if it crashes under Passenger Standalone?

There are a number of fixes for the above issues, but there isn't a simple one to fix all in one. Or at least I haven't come across any of them. One fix for the first problem was placing a script in /etc/init so that passenger would start when the server is rebooted. This however didn't fix the other two issues as it would only start Passenger when the entire server would start, meaning Passenger wouldn't be started if for example the server ran out of memory, something crashed the rails apps etc.

The best fix I could think of was creating a script that checks whether Passenger standalone is running every minute and if it isn't, then start it. This fix would work for all scenarios: server reboot, application crash. Moreover, as the script would run as a cronjob every minute, it would also perform a health check on all running apps. Here's the code for the script.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
#!/bin/bash
 
/usr/sbin/lsof -i :13007 > /home/{USER}/rails_apps/{APP1}test
 
if [ ! -s /home/{USER}/rails_apps/{APP1}test ];
  then
    # passenger down
    date
    . "/usr/local/rvm/scripts/rvm"
    rvm use 1.9.2
    cd /home/{USER}/rails_apps/{APP1}/
    passenger start -a 127.0.0.1 -p 13007 -d -e production --user {USER}
fi
 
/usr/sbin/lsof -i :13008 > /home/{USER}/rails_apps/{APP2}test
 
if [ ! -s /home/{USER}/rails_apps/{APP2}test ];
  then
    # passenger down
    date
    . "/usr/local/rvm/scripts/rvm"
    rvm use 1.9.2
    cd /home/{USER}/rails_apps/{APP2}/
    passenger start -a 127.0.0.1 -p 13008 -d -e production --user {USER}
fi

The above example is for a server running two apps on Passenger Standalone behind apache on ports 13007 and 13008. Replace {USER} , {APP1}, {APP2} with your own configuration. Once you've written your file, save it to /home/{USER}/rails_apps/health_check.sh and then add a cronjob to run it every minute, 5 minutes or whenever you want.

The code above checks whether there are any processes listening on the ports defined for each app. If there is a single process, the output will be written to a file which will have a file size greater than 0. If there are no processes listening on that port, the file size will be 0. The next step is to check the file size. If it is 0, our app is not running and we can start it. Simple as that. And it works for reboots, crashes, integrity and health checks and more.

If you're wondering what's with the date command, here's the answer. In my cronjob I've set the output of the command to /home/{USER}/rails_apps/health_check.log. If any app is started (meaning it was down for some reason) I will have the info and the timestamp in the log. I can then look throughout the apache / nginx / rails logs at that specific timestamp and identify the problem that caused the app to crash, if there were any bugs.

Published on Tuesday, May 3rd, 2011 at 5:00 pm in code, tutorials.

About Bogdan Pop

Bogdan Pop is a young Romanian entrepreneur who runs WebRaptor. He is a web developer with awesome design skills, who enjoys writing about everyday's work and usability. He relaxes by taking photos every once in a while and by mixing french electronic music. Connect with him via Twitter.
 
  1. Avin Tokade says: August 9th, 2011 at 1:50 pm

    Good Post… There is small mistake you accidentally written wrong port number (12008) for second App.

    I have some query.
    What should be path for app1 ? My application located at /var/www/apps/railsapp1

    I got following error while executing script
    ip_health.sh: line 3: /var/www/apps/railsapp1: Is a directory

  2. Bogdan Pop says: August 9th, 2011 at 5:43 pm

    Hi Avin, thanks for pointing out the typo on the port number. I’ve slightly modified the script because when I added the {app1} etc vars I missed that the check files would get the same name as the app directory itself.

    Everything should work fine now.

  3. Avin Tokade says: August 10th, 2011 at 1:55 pm

    Thank you Bogdan.

    I tried your new script work like a charm.

    Sometime rails application has directory with name ‘test’. So It is giving error. like
    ip_health.sh: line 3: /var/www/apps/railsapp/test: Is a directory

    So I just changed ‘test’ to ‘health’

    Thanks again for your help..
    This is proper and smart script for my requirement.

  4. Bogdan Pop says: August 11th, 2011 at 10:17 am

    You’re welcome!





Save time next time! You won't have to fill out all these fields again. Register in just a few clicks and then login.


If you do not have a username, you can register in just a few clicks.