deployment - Which is the most elegant way to start celeryd in a Django project? -


i'm starting out celery in django project , couldn't wonder: elegant way start start celery's workers project?

let me explain reasoning question.

currently, recommended way start celery seems python manage.py celeryd in simpler setups , along lines of /etc/init.d/celeryd start in more complex ones. however, in first case, process feels fragile since process wouldn't start automatically, while second require quite bit of project-specific configuration (virtualenv, settings etc.) latter demonstrates general feeling celery worker tied codebase it's part of , it's tied main project process, since celery worker without create tasks in practially useless (with 1 exception being celerybeat). problem init.d scripts they'd need advanced logic handling several projects per server (with separate virtual environments, settings, paths etc.)

so figured, might quite elegant configuration-wise start celeryd main process, e.g. spawn mod_wsgi withing apache (similar other setup options) , kill when main process goes down (/etc/init.d/apache2 stop). however, i'm not quite sure whether there technical traps considering performance and/or security in reasoning -- might case since i've tried googling , found virtually nothing.

  1. is reasoning flawed considering celery architecture?
  2. can somehow spawn celeryd somewhere within mod_wsgi , sensible?
  3. how start celery workers in projects?

i start celery using manage.py celeryd , managing supervisor. on each deployment modifies task restart celery after i've deployed , restarted apache.\

edit:

we use chef manage our configurations supervisor , other processes, might bit on kill. have 1 master supervisord.conf includes separate configuration file each site run. 1 of might this:

[program:celery_newproject] command = /srv/www/.virtualenvs/newprojectenv/bin/python /srv/www/projects/newproject/manage.py celeryd --concurrency=2 --settings=settings.alpha --pidfile=/var/run/celery/celery_newproject.pid user = wsgiuser environment = python_egg_cache="/tmp/.python-eggs" 

on deploy run

sudo supervisorctl restart celery_newproject 

this restarts supervisor process celery , picks new tasks you've defined.

there other non elegant ways well. on personal sites have cron job running checks existence of .pid file , restarts celery if doens't exist. not elegant @ all, works low reliability sites.


Comments

Popular posts from this blog

delphi - How to convert bitmaps to video? -

jasper reports - Fixed header in Excel using JasperReports -

python - ('The SQL contains 0 parameter markers, but 50 parameters were supplied', 'HY000') or TypeError: 'tuple' object is not callable -