python - Gunicorn async workers (gevent) not working with Django-Oscar: 500 Internal server error -
brief
django-oscar web application running in production , staging environment, hosted in virtual environment, running systemd daemon using gunicorn.
nginx used proxy, gunicorn working left default "worker-class" "sync", not suitable our case, need change async, "gevent", done adding following parameters execstart
--worker-class gevent --worker-connections 768 but workers did not start , got following error
[2017-08-15 08:42:35 +0000] [25425] [info] starting gunicorn 19.7.1 [2017-08-15 08:42:35 +0000] [25425] [debug] arbiter booted [2017-08-15 08:42:35 +0000] [25425] [info] listening at: unix:/tmp/myapp.sock (25425) [2017-08-15 08:42:35 +0000] [25425] [info] using worker: gevent [2017-08-15 08:42:35 +0000] [25430] [info] booting worker pid: 25430 [2017-08-15 08:42:35 +0000] [25425] [debug] 1 workers [2017-08-15 11:42:37 +0000] [25430] [error] exception in worker process traceback (most recent call last): file "/home/ubuntu/.virtualenv/myapp-env/local/lib/python2.7/site-packages/gunicorn/arbiter.py", line 578, in spawn_worker worker.init_process() file "/home/ubuntu/.virtualenv/myapp-env/local/lib/python2.7/site-packages/gunicorn/workers/ggevent.py", line 190, in init_process super(geventworker, self).init_process() file "/home/ubuntu/.virtualenv/myapp-env/local/lib/python2.7/site-packages/gunicorn/workers/base.py", line 126, in init_process self.load_wsgi() file "/home/ubuntu/.virtualenv/myapp-env/local/lib/python2.7/site-packages/gunicorn/workers/base.py", line 135, in load_wsgi self.wsgi = self.app.wsgi() file "/home/ubuntu/.virtualenv/myapp-env/local/lib/python2.7/site-packages/gunicorn/app/base.py", line 67, in wsgi self.callable = self.load() file "/home/ubuntu/.virtualenv/myapp-env/local/lib/python2.7/site-packages/gunicorn/app/wsgiapp.py", line 65, in load return self.load_wsgiapp() file "/home/ubuntu/.virtualenv/myapp-env/local/lib/python2.7/site-packages/gunicorn/app/wsgiapp.py", line 52, in load_wsgiapp return util.import_app(self.app_uri) file "/home/ubuntu/.virtualenv/myapp-env/local/lib/python2.7/site-packages/gunicorn/util.py", line 352, in import_app __import__(module) file "/home/ubuntu/.virtualenv/myapp-env/local/lib/python2.7/site-packages/gevent/builtins.py", line 93, in __import__ result = _import(*args, **kwargs) file "/home/ubuntu/myapp/myapp/wsgi.py", line 16, in <module> application = get_wsgi_application() file "/home/ubuntu/.virtualenv/myapp-env/local/lib/python2.7/site-packages/django/core/wsgi.py", line 14, in get_wsgi_application django.setup() file "/home/ubuntu/.virtualenv/myapp-env/local/lib/python2.7/site-packages/django/__init__.py", line 18, in setup apps.populate(settings.installed_apps) file "/home/ubuntu/.virtualenv/myapp-env/local/lib/python2.7/site-packages/django/apps/registry.py", line 108, in populate app_config.import_models(all_models) file "/home/ubuntu/.virtualenv/myapp-env/local/lib/python2.7/site-packages/django/apps/config.py", line 198, in import_models self.models_module = import_module(models_module_name) file "/usr/lib/python2.7/importlib/__init__.py", line 37, in import_module __import__(name) file "/home/ubuntu/.virtualenv/myapp-env/local/lib/python2.7/site-packages/gevent/builtins.py", line 93, in __import__ result = _import(*args, **kwargs) file "/home/ubuntu/myapp/main/checkout/models.py", line 5, in <module> main.shipping.models import shippingperiod file "/home/ubuntu/.virtualenv/myapp-env/local/lib/python2.7/site-packages/gevent/builtins.py", line 93, in __import__ result = _import(*args, **kwargs) file "/home/ubuntu/myapp/main/shipping/models.py", line 57, in <module> oscar.apps.shipping.models import * # noqa file "/home/ubuntu/.virtualenv/myapp-env/local/lib/python2.7/site-packages/gevent/builtins.py", line 93, in __import__ result = _import(*args, **kwargs) file "/home/ubuntu/.virtualenv/myapp-env/local/lib/python2.7/site-packages/oscar/apps/shipping/models.py", line 1, in <module> oscar.apps.shipping import abstract_models file "/home/ubuntu/.virtualenv/myapp-env/local/lib/python2.7/site-packages/gevent/builtins.py", line 93, in __import__ result = _import(*args, **kwargs) file "/home/ubuntu/.virtualenv/myapp-env/local/lib/python2.7/site-packages/oscar/apps/shipping/abstract_models.py", line 12, in <module> scale = loading.get_class('shipping.scales', 'scale') file "/home/ubuntu/.virtualenv/myapp-env/local/lib/python2.7/site-packages/oscar/core/loading.py", line 29, in get_class return get_classes(module_label, [classname], module_prefix)[0] file "/home/ubuntu/.virtualenv/myapp-env/local/lib/python2.7/site-packages/oscar/core/loading.py", line 100, in get_classes local_module = _import_module(local_module_label, classnames) file "/home/ubuntu/.virtualenv/myapp-env/local/lib/python2.7/site-packages/oscar/core/loading.py", line 121, in _import_module return __import__(module_label, fromlist=classnames) file "/home/ubuntu/.virtualenv/myapp-env/local/lib/python2.7/site-packages/gevent/builtins.py", line 93, in __import__ result = _import(*args, **kwargs) importerror: no module named scales [2017-08-15 11:42:37 +0000] [25430] [info] worker exiting (pid: 25430) [2017-08-15 08:42:37 +0000] [25425] [info] shutting down: master [2017-08-15 08:42:37 +0000] [25425] [info] reason: worker failed boot. after reading , trying many solutions, tried --preload in mix, , got workers up, unfortunately reason causing application return internal server error (500)
[2017-08-15 14:44:51 +0000] [26323] [debug] / [2017-08-15 14:44:51 +0000] [26323] [error] error handling request / traceback (most recent call last): file "/home/ubuntu/.virtualenv/myapp-env/local/lib/python2.7/site-packages/gunicorn/workers/async.py", line 56, in handle self.handle_request(listener_name, req, client, addr) file "/home/ubuntu/.virtualenv/myapp-env/local/lib/python2.7/site-packages/gunicorn/workers/ggevent.py", line 152, in handle_request super(geventworker, self).handle_request(*args) file "/home/ubuntu/.virtualenv/myapp-env/local/lib/python2.7/site-packages/gunicorn/workers/async.py", line 107, in handle_request respiter = self.wsgi(environ, resp.start_response) file "/home/ubuntu/.virtualenv/myapp-env/local/lib/python2.7/site-packages/django/core/handlers/wsgi.py", line 170, in __call__ self.load_middleware() file "/home/ubuntu/.virtualenv/myapp-env/local/lib/python2.7/site-packages/django/core/handlers/base.py", line 50, in load_middleware mw_class = import_string(middleware_path) file "/home/ubuntu/.virtualenv/myapp-env/local/lib/python2.7/site-packages/django/utils/module_loading.py", line 26, in import_string module = import_module(module_path) file "/usr/lib/python2.7/importlib/__init__.py", line 37, in import_module __import__(name) file "/home/ubuntu/.virtualenv/myapp-env/local/lib/python2.7/site-packages/gevent/builtins.py", line 93, in __import__ result = _import(*args, **kwargs) file "/home/ubuntu/myapp/oscarapi/middleware.py", line 16, in <module> oscarapi.basket.operations import ( file "/home/ubuntu/.virtualenv/myapp-env/local/lib/python2.7/site-packages/gevent/builtins.py", line 93, in __import__ result = _import(*args, **kwargs) file "/home/ubuntu/myapp/oscarapi/basket/operations.py", line 22, in <module> applicator = get_class('offer.applicator', 'applicator') file "/home/ubuntu/.virtualenv/myapp-env/local/lib/python2.7/site-packages/oscar/core/loading.py", line 29, in get_class return get_classes(module_label, [classname], module_prefix)[0] file "/home/ubuntu/.virtualenv/myapp-env/local/lib/python2.7/site-packages/oscar/core/loading.py", line 100, in get_classes local_module = _import_module(local_module_label, classnames) file "/home/ubuntu/.virtualenv/myapp-env/local/lib/python2.7/site-packages/oscar/core/loading.py", line 121, in _import_module return __import__(module_label, fromlist=classnames) file "/home/ubuntu/.virtualenv/myapp-env/local/lib/python2.7/site-packages/gevent/builtins.py", line 93, in __import__ result = _import(*args, **kwargs) importerror: no module named applicator ==> /var/log/nginx/access.log <== 10.0.52.154 - - [15/aug/2017:11:44:51 +0000] "get / http/1.1" 500 141 "-" "mozilla/5.0 (x11; linux x86_64) applewebkit/537.36 (khtml, gecko) chrome/56.0.2924.87 safari/537.36" can kindly point me in right direction :) in advance everyone!
update tried eventlet since gevent not working, , this
[2017-08-16 07:19:30 +0000] [7481] [debug] current configuration: proxy_protocol: false worker_connections: 768 statsd_host: none max_requests_jitter: 0 post_fork: <function post_fork @ 0x7f1637e18938> errorlog: /tmp/logging.log enable_stdio_inheritance: false worker_class: eventlet ssl_version: 2 suppress_ragged_eofs: true syslog: false syslog_facility: user when_ready: <function when_ready @ 0x7f1637e18668> pre_fork: <function pre_fork @ 0x7f1637e187d0> cert_reqs: 0 preload_app: false keepalive: 2 accesslog: none group: 1000 graceful_timeout: 30 do_handshake_on_connect: false spew: false workers: 3 proc_name: none sendfile: none pidfile: none umask: 0 on_reload: <function on_reload @ 0x7f1637e18500> pre_exec: <function pre_exec @ 0x7f1637e18ed8> worker_tmp_dir: none limit_request_fields: 100 pythonpath: none on_exit: <function on_exit @ 0x7f1637e1e758> config: none logconfig: none check_config: false statsd_prefix: secure_scheme_headers: {'x-forwarded-protocol': 'ssl', 'x-forwarded-proto': 'https', 'x-forwarded-ssl': 'on'} reload_engine: auto proxy_allow_ips: ['127.0.0.1'] pre_request: <function pre_request @ 0x7f1637e1e0c8> post_request: <function post_request @ 0x7f1637e1e1b8> forwarded_allow_ips: ['127.0.0.1'] worker_int: <function worker_int @ 0x7f1637e18c08> raw_paste_global_conf: [] threads: 1 max_requests: 0 chdir: /home/ubuntu daemon: false user: 1000 limit_request_line: 4094 access_log_format: %(h)s %(l)s %(u)s %(t)s "%(r)s" %(s)s %(b)s "%(f)s" "%(a)s" certfile: none on_starting: <function on_starting @ 0x7f1637e18398> post_worker_init: <function post_worker_init @ 0x7f1637e18aa0> child_exit: <function child_exit @ 0x7f1637e1e320> worker_exit: <function worker_exit @ 0x7f1637e1e488> paste: none default_proc_name: myapp.wsgi:application syslog_addr: udp://localhost:514 syslog_prefix: none ciphers: tlsv1 worker_abort: <function worker_abort @ 0x7f1637e18d70> loglevel: debug bind: ['unix:/tmp/myapp.sock'] raw_env: [] initgroups: false capture_output: false reload: false limit_request_field_size: 8190 nworkers_changed: <function nworkers_changed @ 0x7f1637e1e5f0> timeout: 30 keyfile: none ca_certs: none tmp_upload_dir: none backlog: 2048 logger_class: gunicorn.glogging.logger [2017-08-16 07:19:30 +0000] [7481] [info] starting gunicorn 19.7.1 [2017-08-16 07:19:30 +0000] [7481] [debug] arbiter booted [2017-08-16 07:19:30 +0000] [7481] [info] listening at: unix:/tmp/myapp.sock (7481) [2017-08-16 07:19:30 +0000] [7481] [info] using worker: eventlet [2017-08-16 07:19:30 +0000] [7486] [info] booting worker pid: 7486 [2017-08-16 07:19:30 +0000] [7487] [info] booting worker pid: 7487 [2017-08-16 07:19:30 +0000] [7487] [error] exception in worker process traceback (most recent call last): file "/home/ubuntu/.virtualenv/myapp-env/local/lib/python2.7/site-packages/gunicorn/arbiter.py", line 578, in spawn_worker worker.init_process() file "/home/ubuntu/.virtualenv/myapp-env/local/lib/python2.7/site-packages/gunicorn/workers/geventlet.py", line 102, in init_process self.patch() file "/home/ubuntu/.virtualenv/myapp-env/local/lib/python2.7/site-packages/gunicorn/workers/geventlet.py", line 91, in patch hubs.use_hub() file "/home/ubuntu/.virtualenv/myapp-env/local/lib/python2.7/site-packages/eventlet/hubs/__init__.py", line 70, in use_hub mod = get_default_hub() file "/home/ubuntu/.virtualenv/myapp-env/local/lib/python2.7/site-packages/eventlet/hubs/__init__.py", line 38, in get_default_hub import eventlet.hubs.epolls file "/home/ubuntu/.virtualenv/myapp-env/local/lib/python2.7/site-packages/eventlet/hubs/epolls.py", line 27, in <module> eventlet.hubs.hub import basehub file "/home/ubuntu/.virtualenv/myapp-env/local/lib/python2.7/site-packages/eventlet/hubs/hub.py", line 23, in <module> eventlet.support import greenlets greenlet, clear_sys_exc_info, monotonic, 6 file "/home/ubuntu/.virtualenv/myapp-env/local/lib/python2.7/site-packages/eventlet/support/monotonic.py", line 167, in <module> raise runtimeerror('no suitable implementation system') runtimeerror: no suitable implementation system [2017-08-16 07:19:30 +0000] [7486] [error] exception in worker process traceback (most recent call last): file "/home/ubuntu/.virtualenv/myapp-env/local/lib/python2.7/site-packages/gunicorn/arbiter.py", line 578, in spawn_worker worker.init_process() file "/home/ubuntu/.virtualenv/myapp-env/local/lib/python2.7/site-packages/gunicorn/workers/geventlet.py", line 102, in init_process self.patch() file "/home/ubuntu/.virtualenv/myapp-env/local/lib/python2.7/site-packages/gunicorn/workers/geventlet.py", line 91, in patch hubs.use_hub() file "/home/ubuntu/.virtualenv/myapp-env/local/lib/python2.7/site-packages/eventlet/hubs/__init__.py", line 70, in use_hub mod = get_default_hub() file "/home/ubuntu/.virtualenv/myapp-env/local/lib/python2.7/site-packages/eventlet/hubs/__init__.py", line 38, in get_default_hub import eventlet.hubs.epolls file "/home/ubuntu/.virtualenv/myapp-env/local/lib/python2.7/site-packages/eventlet/hubs/epolls.py", line 27, in <module> eventlet.hubs.hub import basehub file "/home/ubuntu/.virtualenv/myapp-env/local/lib/python2.7/site-packages/eventlet/hubs/hub.py", line 23, in <module> eventlet.support import greenlets greenlet, clear_sys_exc_info, monotonic, 6 file "/home/ubuntu/.virtualenv/myapp-env/local/lib/python2.7/site-packages/eventlet/support/monotonic.py", line 167, in <module> raise runtimeerror('no suitable implementation system') runtimeerror: no suitable implementation system [2017-08-16 07:19:30 +0000] [7486] [info] worker exiting (pid: 7486) [2017-08-16 07:19:30 +0000] [7487] [info] worker exiting (pid: 7487) [2017-08-16 07:19:30 +0000] [7481] [info] shutting down: master [2017-08-16 07:19:30 +0000] [7481] [info] reason: worker failed boot.
Comments
Post a Comment