Gearman 0.28 Setup on Centos 6

Gearman Setup on Centos

Was told about a great tool the other day for processing queues called Gearman. It has the ability to cluster both on the server side and on the worker side which allows you to grow your server without having to change too much once the queue manager is configured. The best part is that it keeps language independence on both sides of the fence. If you want to write your client using PHP and your worker as C, you can do that. Currently we are looking at moving all our emails to this queue manager so that we can offload them to a secondary server and the emails can be sent when the server has free cycles. So the point of this post is just to document some of the steps I had to go through inorder to setup Gearman on the server using PHP for the client and Worker.

This was started with a basic install of CentOS 6, the Repos (Remi, EPEL, and RPM Forge)

rpm -Uvh http://dl.fedoraproject.org/pub/epel/6/x86_64/epel-release-6-5.noarch.rpm \
http://packages.sw.be/rpmforge-release/rpmforge-release-0.5.2-2.el6.rf.x86_64.rpm \
http://rpms.famillecollet.com/enterprise/remi-release-6.rpm

Basic Setup

(PHP)

yum install -y php php-cli php-common php-devel php-gd php-mbstring php-mysql php-pdo php-pear \
php-soap php-xml php-mcrypt pcre-devel libssh2-devel

(Needed for compiling the Gearman source code)

yum install kernel sources kernel-devel libevent-devel wget gcc gcc-c++ make autoconf automake \
boost boost-devel libuuid libuuid-devel kernel-devel

Make sure all the libraries are up to date

yum update -y

At this point I usually like to reboot the server to make sure that any services required are started up when the server is rebooted in the future, because god knows that all this setup work is forgotten the moment you start working on the next project.

reboot

Gearman & Gearmand Setup

Get the source code for Gearmand and install the libgearman. This is required for the next step. If you try to install pecl gearman it will fail saying that you need libgearman. This next step will give you that library.

wget https://launchpad.net/gearmand/trunk/0.29/+download/gearmand-0.29.tar.gz
tar zxvf gearmand-0.29.tar.gz
cd gearmand-0.29
./configure
make
make install

At this point you will now have both gearmand (Server) and gearman (Client/Worker) installed.

(Server) - gearmand &
(Worker) - gearman -w -f test_fn wc
(Client) - gearman -f test_fn "Hello World"

A break down of what is happening above.
Server: starts up the Gearman Server and runs it in the background. (Would require three command windows to run this if we didn’t push all this to the background)
Worker: gearman (without the d) and the option -w tells it that it is a worker, -f is the name of the function (test)
Client: gearman and the option -f names the function the worker will run and “Hello World” is the parameter being passed into the function

Installing Gearman PHP Extension

pecl install gearman

You should add “extension=gearman.so” to php.ini

vi /etc/php.ini
extension=gearman.so

Verify that the module has been add with:

php -modules

PHP Gearman Test

gearmand daemon must be running in the background at least for these to work. Start the Worker first php worker.php, then in a separate terminal window run php client.php.

Worker – worker.php

<?php
$worker = new GearmanWorker();
$worker->addServer(); //localhost by default
$worker->addFunction("reverse", "my_reverse_function"); //register reverse with the server
while ($worker->work());

function my_reverse_function($job)
{
return strrev($job->workload());
}
?>

Client – client.php

<?php
$client= new GearmanClient();
$client->addServer(); //localhost by default
print $client->do("reverse", "Hello World!");
?>

PHP Test Results

PHP Library for accessing Gearman
For more links and documentation check out Gearman.org