Fix: Celery Beat Not Working — Scheduled Tasks Not Running or Beat Not Starting

The Problem

Celery Beat is running but scheduled tasks never execute:

celery -A myapp beat -l info
# [2026-03-26 10:00:00,000: INFO/MainProcess] beat: Starting...
# [2026-03-26 10:00:00,001: INFO/MainProcess] Scheduler: Sending due task send-weekly-report (myapp.tasks.send_weekly_report)
# But the task never appears in worker logs

Or Beat starts but immediately exits:

ERROR/MainProcess] beat: ERROR: Another beat is already running!
ValueError: not enough values to unpack

Or tasks run at the wrong time despite setting a schedule:

CELERY_BEAT_SCHEDULE = {
    'daily-report': {
        'task': 'myapp.tasks.send_report',
        'schedule': crontab(hour=9, minute=0),
        # Expected: 9:00 AM — but runs at a different time
    },
}

Why This Happens

Celery Beat is a scheduler that sends tasks to the queue — it doesn’t execute them itself. This split is the single most important fact about Beat and the reason for most “scheduled task not running” tickets. Beat publishes a task message to the broker (Redis, RabbitMQ, SQS) at the scheduled time; a worker subscribed to that broker picks up the message and runs the task. If Beat publishes and no worker is listening, the message lands in the queue and sits there. If the worker is listening but Beat isn’t running, nothing is ever scheduled.

The second source of confusion is state. Beat needs to remember when each task last ran, so it doesn’t fire the same daily-9am task ten times in a row when restarted at 9:05. The default scheduler stores this state in a celerybeat-schedule file (a Python shelve database). Lose the file and Beat re-fires every overdue task. Run two Beat processes against the same file and one corrupts the other’s state — that is what the “Another beat is already running” message means.

The third pitfall is timezones. Beat reads crontab(hour=9, minute=0) and asks “is it now 9am in my configured timezone?” If your server runs in UTC, Django says USE_TZ = True, and Celery has no explicit timezone setting, the answer can drift between releases of Celery itself. Pin both CELERY_TIMEZONE and CELERY_ENABLE_UTC to make the behavior deterministic.

• No worker running — Beat sends tasks to the broker queue, but if no Celery worker is consuming that queue, tasks pile up unexecuted.
• Beat and worker running in the same process — running celery worker -B starts both, but this is only recommended for development. In production, they should be separate processes.
• celerybeat-schedule file conflict — Beat stores its schedule state in a file (celerybeat-schedule). If two Beat processes start, the second fails with “Another beat is already running.”
• Timezone mismatch — if CELERY_TIMEZONE doesn’t match the server’s system timezone, crontab schedules run at unexpected times.
• Task not registered — if the task module isn’t imported when Beat starts, the task won’t be found. Celery must discover the task before Beat can schedule it.

Fix 1: Run Beat and Worker as Separate Processes

Beat schedules tasks; workers execute them. Both must be running:

# Terminal 1 — start the worker (executes tasks)
celery -A myapp worker -l info

# Terminal 2 — start beat (schedules tasks)
celery -A myapp beat -l info

# Development shortcut — run both in one process (not for production)
celery -A myapp worker --beat -l info
# Or:
celery -A myapp worker -B -l info

Production setup with Supervisor:

; /etc/supervisor/conf.d/celery.conf

[program:celery-worker]
command=/venv/bin/celery -A myapp worker -l info --concurrency=4
directory=/app
user=celery
autostart=true
autorestart=true
stderr_logfile=/var/log/celery/worker.err.log
stdout_logfile=/var/log/celery/worker.out.log

[program:celery-beat]
command=/venv/bin/celery -A myapp beat -l info --scheduler django_celery_beat.schedulers:DatabaseScheduler
directory=/app
user=celery
autostart=true
autorestart=true
stderr_logfile=/var/log/celery/beat.err.log
stdout_logfile=/var/log/celery/beat.out.log

Docker Compose:

# docker-compose.yml
services:
  redis:
    image: redis:7-alpine

  worker:
    build: .
    command: celery -A myapp worker -l info
    environment:
      - CELERY_BROKER_URL=redis://redis:6379/0
    depends_on:
      - redis

  beat:
    build: .
    command: celery -A myapp beat -l info --scheduler django_celery_beat.schedulers:DatabaseScheduler
    environment:
      - CELERY_BROKER_URL=redis://redis:6379/0
    depends_on:
      - redis
      - worker

Warning: Only run ONE Beat instance at a time. Running multiple Beat instances causes duplicate task execution and the “Another beat is already running” error.

Fix 2: Configure CELERY_BEAT_SCHEDULE Correctly

Define the schedule in your Celery config:

# celery.py
from celery import Celery
from celery.schedules import crontab

app = Celery('myapp')

app.config_from_object('django.conf:settings', namespace='CELERY')

# Load tasks from all registered apps
app.autodiscover_tasks()

# Define the beat schedule
app.conf.beat_schedule = {
    # Run every 30 seconds
    'check-new-orders': {
        'task': 'orders.tasks.check_new_orders',
        'schedule': 30.0,
    },

    # Run every 5 minutes
    'sync-inventory': {
        'task': 'inventory.tasks.sync_inventory',
        'schedule': crontab(minute='*/5'),
    },

    # Run daily at 9:00 AM
    'send-daily-report': {
        'task': 'reports.tasks.send_daily_report',
        'schedule': crontab(hour=9, minute=0),
        'args': (),
        'kwargs': {'format': 'pdf'},
    },

    # Run on weekdays at 8:00 AM
    'morning-standup': {
        'task': 'notifications.tasks.send_standup',
        'schedule': crontab(hour=8, minute=0, day_of_week='mon-fri'),
    },

    # Run on the 1st of every month at midnight
    'monthly-billing': {
        'task': 'billing.tasks.process_monthly_billing',
        'schedule': crontab(day_of_month=1, hour=0, minute=0),
    },
}

crontab parameter reference:

from celery.schedules import crontab

crontab(minute=0, hour=0)           # Midnight daily
crontab(minute='*/15')              # Every 15 minutes
crontab(hour='*/2', minute=0)       # Every 2 hours on the hour
crontab(day_of_week='monday')       # Every Monday at midnight
crontab(day_of_week='0,6')          # Saturday and Sunday
crontab(day_of_month='1,15')        # 1st and 15th of each month
crontab(month_of_year='*/3')        # Every quarter (Jan, Apr, Jul, Oct)

# Complex: 9am–5pm, Monday–Friday, every 30 min
crontab(hour='9-17', minute='*/30', day_of_week='mon-fri')

Fix 3: Fix Timezone Issues

Beat uses UTC by default. If your crontab should run in a local timezone:

# settings.py (Django) or celeryconfig.py

# Correct — set timezone explicitly
CELERY_TIMEZONE = 'America/New_York'  # Or 'Europe/London', 'Asia/Tokyo', etc.
CELERY_ENABLE_UTC = True  # Keep UTC internally, display in CELERY_TIMEZONE

# Django's TIME_ZONE must also be set for Django-Celery-Beat
TIME_ZONE = 'America/New_York'
USE_TZ = True

# celery.py
app.conf.update(
    timezone='America/New_York',
    enable_utc=True,
)

Verify timezone handling:

# Check what timezone Beat thinks it is
celery -A myapp inspect clock

# Check worker timezone
celery -A myapp inspect conf | grep timezone

Note: When in doubt, schedule everything in UTC and convert in the application code. This avoids daylight saving time bugs entirely.

Fix 4: Use django-celery-beat for Dynamic Schedules

The default file-based scheduler requires a code deploy to change schedules. django-celery-beat stores schedules in the database, allowing changes without restarts:

pip install django-celery-beat

# settings.py
INSTALLED_APPS = [
    ...
    'django_celery_beat',
]

python manage.py migrate  # Creates the beat schedule tables

# Start beat with the database scheduler
celery -A myapp beat -l info \
  --scheduler django_celery_beat.schedulers:DatabaseScheduler

Manage schedules via Django admin or programmatically:

from django_celery_beat.models import PeriodicTask, CrontabSchedule
import json

# Create a crontab schedule
schedule, created = CrontabSchedule.objects.get_or_create(
    hour=9,
    minute=0,
    day_of_week='*',
    day_of_month='*',
    month_of_year='*',
    timezone='America/New_York',
)

# Create the periodic task
PeriodicTask.objects.update_or_create(
    name='Daily Report',
    defaults={
        'task': 'reports.tasks.send_daily_report',
        'crontab': schedule,
        'args': json.dumps([]),
        'kwargs': json.dumps({'format': 'pdf'}),
        'enabled': True,
    },
)

Fix 5: Ensure Tasks Are Discoverable

Beat can only schedule tasks that Celery has discovered and registered:

# myapp/__init__.py — ensures Celery app is loaded with Django
from .celery import app as celery_app

__all__ = ('celery_app',)

# celery.py
from celery import Celery

app = Celery('myapp')
app.config_from_object('django.conf:settings', namespace='CELERY')

# Auto-discover tasks in all INSTALLED_APPS
app.autodiscover_tasks()  # Looks for tasks.py in each installed app

# tasks.py — task must use @shared_task or @app.task
from celery import shared_task

@shared_task
def send_daily_report(format='pdf'):
    # Task logic here
    pass

Verify tasks are registered:

# List all registered tasks
celery -A myapp inspect registered

# Or start a worker and check
celery -A myapp worker -l debug 2>&1 | grep "Registered tasks"

Fix 6: Monitor Beat with Flower

Use Flower to monitor task execution and verify Beat is sending tasks:

pip install flower

# Start Flower
celery -A myapp flower --port=5555

# Open http://localhost:5555
# Tasks tab: see queued, active, completed tasks
# Monitor tab: see when tasks were received vs executed

Check Beat is producing tasks from logs:

# Beat log shows tasks being sent
celery -A myapp beat -l info
# INFO: Scheduler: Sending due task send-daily-report (reports.tasks.send_daily_report)

# Worker log shows tasks being received and executed
celery -A myapp worker -l info
# INFO: Received task: reports.tasks.send_daily_report[uuid]
# INFO: Task reports.tasks.send_daily_report[uuid] succeeded in 1.23s: None

# If Beat sends but worker doesn't receive: check broker connectivity
# If worker receives but task fails: check task code and logs

Version History: Celery Beat Across Releases

Celery Beat has shipped with the main Celery distribution since Celery 1.0 (2009). The scheduling primitive (crontab schedules and interval schedules) is essentially unchanged from the 3.x line. What has moved are the operational defaults, the supported Python versions, the schedulers that ship with the project, and the django-celery-beat integration.

Celery 4.0 (November 2016) → 4.4 (December 2019) was the long-running 4.x line. The headline change versus 3.x was the move from celery beat (lowercase, separate binary) to celery -A app beat as the canonical invocation. The old binary still works in 4.x but emits a deprecation warning. Configuration migrated from uppercase to lowercase keys: CELERY_BEAT_SCHEDULE became beat_schedule when set via app.conf.update(), though Django settings retain the uppercase namespaced form (CELERY_BEAT_SCHEDULE becomes beat_schedule after Celery strips the CELERY_ prefix). Mixing the two styles in the same project still trips people up.

Celery 5.0 (November 2020) was a breaking release that dropped Python 2 entirely. Minimum Python version became 3.6. The CLI was rewritten on top of Click, which changed flag parsing in subtle ways — celery -A myapp worker --beat still works, but some shell completions broke and pre-5.0 supervisor configs that relied on argument ordering need review. The celery worker -B shortcut still works for development as shown in Fix 1.

Celery 5.1 (May 2021) dropped Python 3.6 and made Python 3.7 the minimum. The celery shell command got a few quality-of-life improvements but Beat semantics did not change.

Celery 5.2 (November 2021) added official Python 3.10 support. This release also made the default app.conf.broker_connection_retry_on_startup important — Beat now respects this setting on startup, and in environments where the broker takes longer to come up than Beat (Docker Compose without proper depends_on: condition: service_healthy), the default of retrying for 100 seconds is critical. Set it explicitly:

app.conf.broker_connection_retry_on_startup = True

Celery 5.3 (June 2023) added Python 3.11 support and improved Redis cluster compatibility. If you use Redis as a broker behind a cluster (AWS ElastiCache cluster mode enabled, or Redis Cluster on Kubernetes), 5.3 is the first version where Beat and worker handle the MOVED redirects without periodic disconnects. Pre-5.3, Beat would log “Connection lost” every few minutes on a cluster broker even though the application kept working. Upgrade to 5.3 if you see that pattern.

Celery 5.4 (April 2024) added Python 3.12 support and tightened the beat scheduler shutdown sequence. The 5.4 scheduler flushes its schedule file atomically on shutdown, which makes Kubernetes rolling restarts cleaner — the new pod no longer reads a partially-written celerybeat-schedule file. If you saw “Another beat is already running” errors immediately after deploys on a single-replica Beat deployment, 5.4 fixes that race.

django-celery-beat tracks Celery closely. Version 2.5 (May 2023) added Python 3.11 support and last_run_at precision improvements. Version 2.6 (early 2024) made the clocked schedule type more reliable for one-shot scheduled tasks. If you store schedules in the database (Fix 4), upgrade django-celery-beat in lockstep with Celery — mismatches between Celery 5.3+ and django-celery-beat <2.5 cause migration warnings on Django startup.

The practical guidance: if you are on Celery 4.x, 5.4 is the right upgrade target as of 2026. The migration is mostly mechanical (rename uppercase config keys, replace celery beat with celery -A app beat) and the operational improvements — cleaner shutdown, Redis cluster support, modern Python — are significant.

Still Not Working?

celerybeat-schedule file is stale — delete the celerybeat-schedule file (or celerybeat-schedule.db for the shelve scheduler) and restart Beat. This file stores the last run time for each task. A corrupted or outdated file can prevent tasks from running.

Tasks queued but not executed — Beat puts tasks on the queue. If the queue fills up (broker storage full, worker crashed), tasks pile up. Check the queue depth: celery -A myapp inspect active_queues and celery -A myapp inspect reserved.

Beat runs tasks at startup then stops — this happens when Beat’s schedule file is empty or the task has never run before. Beat calculates the next run time from the last run. On first start, it runs immediately, then waits for the next scheduled time. This is expected behavior.

Multiple Beat instances in Kubernetes — if your Beat pod restarts or you run multiple replicas, multiple Beat instances will run simultaneously, causing duplicate task execution. Always run exactly one Beat pod: set replicas: 1 and avoid HorizontalPodAutoscaler for the Beat deployment.

PeriodicTask.enabled = False is the silent killer — if you use django-celery-beat and a task suddenly stops, check PeriodicTask.objects.filter(name='your-task')[0].enabled in the Django shell. The admin UI has a small checkbox that anyone can toggle by mistake. Disabled tasks remain in the database but Beat skips them entirely; there is no error message because nothing went wrong from Beat’s perspective.

Worker prefetch hides slow tasks — the worker prefetches up to worker_prefetch_multiplier * concurrency tasks. If Beat schedules a 5-minute task every minute and the prefetch multiplier is 4 with concurrency of 4, the worker pulls 16 tasks ahead, and new Beat-scheduled tasks queue behind them. Lower worker_prefetch_multiplier to 1 for long-running periodic tasks, or use a dedicated queue for scheduled work.

DST transitions skip or repeat tasks — crontab(hour=2, minute=30) in a timezone with DST will be skipped entirely on the spring-forward day and run twice on the fall-back day. If timing matters, schedule everything in UTC and convert in application code, or move daily jobs out of the 2:00-3:00 window where DST transitions happen in most regions.

For related Celery issues, see Fix: Celery Task Not Executing, Fix: Celery Task Not Received, Fix: Django Migration Conflict, and Fix: Redis Connection Refused.