deployment - Which is the most elegant way to start celeryd in a Django project? -
i'm starting out celery in django project , couldn't wonder: elegant way start start celery's workers project?
let me explain reasoning question.
currently, recommended way start celery seems python manage.py celeryd
in simpler setups , along lines of /etc/init.d/celeryd start
in more complex ones. however, in first case, process feels fragile since process wouldn't start automatically, while second require quite bit of project-specific configuration (virtualenv
, settings
etc.) latter demonstrates general feeling celery worker tied codebase it's part of , it's tied main project process, since celery worker without create tasks in practially useless (with 1 exception being celerybeat
). problem init.d
scripts they'd need advanced logic handling several projects per server (with separate virtual environments, settings, paths etc.)
so figured, might quite elegant configuration-wise start celeryd
main process, e.g. spawn mod_wsgi
withing apache (similar other setup options) , kill when main process goes down (/etc/init.d/apache2 stop
). however, i'm not quite sure whether there technical traps considering performance and/or security in reasoning -- might case since i've tried googling , found virtually nothing.
- is reasoning flawed considering celery architecture?
- can somehow spawn
celeryd
somewhere withinmod_wsgi
, sensible? - how start celery workers in projects?
i start celery using manage.py celeryd , managing supervisor. on each deployment modifies task restart celery after i've deployed , restarted apache.\
edit:
we use chef manage our configurations supervisor , other processes, might bit on kill. have 1 master supervisord.conf includes separate configuration file each site run. 1 of might this:
[program:celery_newproject] command = /srv/www/.virtualenvs/newprojectenv/bin/python /srv/www/projects/newproject/manage.py celeryd --concurrency=2 --settings=settings.alpha --pidfile=/var/run/celery/celery_newproject.pid user = wsgiuser environment = python_egg_cache="/tmp/.python-eggs"
on deploy run
sudo supervisorctl restart celery_newproject
this restarts supervisor process celery , picks new tasks you've defined.
there other non elegant ways well. on personal sites have cron job running checks existence of .pid file , restarts celery if doens't exist. not elegant @ all, works low reliability sites.
Comments
Post a Comment