Troubleshooting

The errors people actually hit, and what each one is telling you.

The problems people actually hit with Skyline, and what each one is telling you. Most of them are authentication against the private Composer registry, or a feature quietly declining to run because its requirements are not met — Skyline prefers to say so, with a status code or a banner, rather than pretend.

Composer returns 401 for laravel-skyline.composer.sh#

The machine running Composer has no valid credentials for the registry. The username is the licensee email tied to the purchase and the license key is the password — a common cause is putting the key in both fields.

composer config --global --auth http-basic.laravel-skyline.composer.sh \
  you@example.com \
  YOUR-LICENSE-KEY

Every machine that pulls the package needs this, not just your laptop: CI runners, build containers and deploy hosts included. On those, prefer the COMPOSER_AUTH environment variable over a committed auth.json:

export COMPOSER_AUTH='{"http-basic":{"laravel-skyline.composer.sh":{"username":"you@example.com","password":"YOUR-LICENSE-KEY"}}}'

If you are asking for a release published after your upgrade year ended, this is also what you will see — and only here. Already-installed versions keep running; an expired upgrade year is a build-time failure, never a runtime one.

Dependabot's pull requests fail to install#

Almost always the Actions/Dependabot secret split. ${{ secrets.* }} inside .github/dependabot.yml resolves against Dependabot secrets, stored under Settings → Secrets and variables → Dependabot. A secret of the same name under Actions is invisible there, which is why an otherwise-correct config still 401s.

The same split catches the CI runs on the pull requests Dependabot opens: those workflow runs are given the Dependabot secrets, not the Actions ones. If your build runs composer install, the credentials it reads must also exist as Dependabot secrets — otherwise Skyline's own update PRs will be the only ones whose CI fails.

And if Dependabot skips your Composer manifest entirely rather than failing on one package, check that the update entry names the registry: without registries: [skyline] it resolves boring-o11y/laravel-skyline against Packagist, does not find it, and gives up on the whole file. See Installation.

Composer refuses to install Skyline alongside laravel/horizon#

That is intended. Skyline declares replace: laravel/horizon, which is what makes every package that depends on Horizon resolve against Skyline instead. The two cannot coexist. Remove laravel/horizon from your require block — see Migrating from Laravel Horizon.

A package requires a newer laravel/horizon than Skyline replaces#

Skyline's replace entry names the single upstream Horizon tag it is rebased onto, so it satisfies requirements up to that version and no further. A dependency asking for a release published after it — a package requiring laravel/horizon: ^5.51 while your Skyline replaces 5.50.0 — has nothing to resolve against, and Composer reports the constraint as unsatisfiable.

First run composer update boring-o11y/laravel-skyline: upstream releases are merged within 14 days, so the usual fix is simply that a newer Skyline exists. If you are already on the latest and the constraint still fails, the rebase has not shipped yet — relax the other package's requirement, or email tech@boring-observability.dev and we will tell you when that upstream tag lands.

Do not work around it by adding laravel/horizon back to require. The two cannot coexist, which is the section above.

Pausing a queue returns 409 Conflict#

Per-queue pausing is built on Laravel's native queue-pause API and needs a shared cache store. Skyline responds 409 — and hides the per-queue Pause buttons behind an explanatory banner — when either requirement is unmet:

• The framework does not provide the queue-pause API. It landed in Laravel 12.40 and was never backported to 11.x. Below that the feature hides itself rather than half-working.
• The cache store is array or null. Neither can carry state between the dashboard process and the worker processes, so a pause set in one would be invisible to the other. They are rejected rather than silently dropped.

Global and per-supervisor pause (horizon:pause, horizon:pause-supervisor) work regardless, because they signal the master process rather than storing state. See Pausing & resuming queues.

The metrics page is empty#

horizon:snapshot is not scheduled. The per-job and per-queue metrics tables are driven by Horizon's snapshot command exactly as upstream, and without it there is nothing to draw:

use Illuminate\Support\Facades\Schedule;

Schedule::command('horizon:snapshot')->everyFiveMinutes();

This catches people migrating from a Horizon install that never had it either — the metrics page was empty before the swap too, and Skyline is simply the first thing that made you look.

The trend charts have gaps#

Workload and wait samples are taken from the php artisan horizon master supervisor loop, not from a scheduled command. A gap in the chart means the master process was not running for that period — a deploy, a crashed supervisor, a scaled-to-zero worker dyno. Nothing else is affected, and the series resumes on its own when the master comes back.

If the charts are empty rather than gappy, check that the retention window has not outrun the data: with the default trends.interval of 15 minutes, a freshly started process has one bucket to draw.

A ShouldBeUnique job has silently stopped dispatching#

Its unique lock is almost certainly still held, and Laravel discards every dispatch while it is. With the default uniqueFor of 0 the lock never expires on its own. Open Locks & Limits in the dashboard: a lock whose job has completed, failed or disappeared is flagged, the Skipped Dispatches table shows how many dispatches it has eaten, and the Release button frees it.

The usual ways it got there:

• The job was deleted from Redis or its queue was cleared by a tool that doesn't release locks, such as queue:clear or Horizon before the swap. Skyline's Delete, Empty and horizon:clear release them.
• A ShouldBeUniqueUntilProcessing job failed before it started, usually past its retryUntil() deadline. Skyline releases that lock and logs unique_lock.released_after_failure, unless the job was queued before you upgraded and has no recorded owner.
• A worker was killed while holding the lock. Nothing can release that except a TTL or a person, so set uniqueFor.
• The job has no uniqueId(), so every instance of the class shares one lock and a dispatch for one record blocks all the others. The lock is behaving as designed, and the fix is the key.

If the lock doesn't show on the screen, it is kept on a store other than the default one through uniqueVia(), and has to be removed from that store by hand. See Unique job locks.

A ShouldBeUnique job was queued twice#

Laravel only checks uniqueness on dispatch(). A copy queued by a chain, a batch, Queue::push() or a retry is queued regardless, and Skyline does not change that. The Locks & Limits screen is how you tell this case apart from the others: if the job's lock is listed and held while copies keep running, the duplicates came through one of those paths.

If no lock is held at all, check uniqueFor. A TTL shorter than the job's time in the queue plus its retries lets the lock expire while the first copy is still around. See Chains, batches, Queue::push and retries for the workarounds.

A job ran twice at the same time#

Look for job.reservation_expired in the log. A Redis queue hands a reserved job to another worker once retry_after has passed since it was picked up, whether or not the first worker is still running it. The line is logged when that happens, and it fires for one of two reasons:

• The job runs longer than retry_after. Keep every job's timeout, including a $timeout set on the job class, below the connection's retry_after. php artisan horizon warns at startup for any supervisor whose timeout is not.
• The worker died from an OOM kill or a deploy that didn't wait for workers. That is the queue recovering as intended, and the job only ran once.

Redis alone can't tell those apart, so the message names both. A job stopped from the dashboard is excluded, and so is the retry of a job that timed out below retry_after, which is logged as job.migrated. With alerts on, the reservation_expired check sends the same news as a critical alert, once per queue.

A job failed with "attempted too many times" but never ran#

Laravel checks retryUntil() and tries when a worker picks a job up, before handle(), and reports both as MaxAttemptsExceededException. Two things cause it without the job ever running:

• The retryUntil() deadline passed while the job waited. The deadline is fixed at dispatch, so a delay or a backlog can use it up. Skyline reports these as RetryWindowExpired, with a message such as never ran: its retryUntil() allowed 60s from dispatch, but a worker only picked it up 600s after dispatch, and warns at dispatch when the delay alone is longer than the window.
• Middleware released it until its tries ran out. WithoutOverlapping, RateLimited and ThrottlesExceptions release a job without running it, and each release spends an attempt. Previous Attempts on the job's page shows the release reasons when you use Skyline's drop-in middleware. Use retryUntil() instead of a small tries for these jobs.

Either failure on a ShouldBeUniqueUntilProcessing job also strands its lock, and the job then stops dispatching unless Skyline releases it.

Alerts never arrive#

Start with php artisan horizon:alert:test. It sends a test alert to every configured channel and ignores the enabled flag, the sustain windows, the cooldown and deploy suppression, so if nothing arrives the problem is the destination: a revoked Slack webhook, a mailer that is not configured, or no channel set at all. Add --severity=critical to test one route. If the test arrives and real alerts do not, check these in order:

• HORIZON_ALERTS=true is set where the supervisors run. Alerts are off by default.
• The condition has not held for the check's for window yet. php artisan horizon:alerts shows what is firing right now, and --history shows what was sent.
• A deploy happened in the last five minutes. Notifications are withheld for suppress_after_deploy seconds, while the checks keep counting.
• For "Horizon is down", horizon:check is not scheduled. That check cannot run inside the fleet it watches, so it needs Schedule::command('horizon:check')->everyMinute() wherever your scheduler runs.
• For worker_crash_loop, HORIZON_INSIGHTS is off. Insights record the restarts that check counts.

horizon:alerts also warns about a severity route that names a channel with no destination. See Alerts.

The dashboard looks exactly like Horizon's#

You are serving stale published assets. Skyline ships its own compiled dashboard assets; if your deploy publishes Horizon's into public/, it needs to re-run the publish step after the swap:

php artisan horizon:publish

The tell is a dashboard that works but has no Scheduled and Retries tabs: old JavaScript, talking to Skyline's API.

onFront() appears to do nothing#

Three possibilities, in the order they are usually the cause:

• The job class is missing the trait. PendingDispatch is not macroable, so onFront() reaches the job through its __call proxy — without use InteractsWithFrontOfQueue on the job, the call has nowhere to land.
• The job is delayed. A ->delay(...) job lives in a sorted set ordered by its available-at time, not on the ready list, so there is no head to jump to yet.
• The connection is not Redis. The behaviour relies on Redis list semantics and does not apply to the sync or database drivers.

See Front-of-queue dispatching.

Provisioning throws about queueWeights and balance#

The [supervisor-1.queueWeights] option only applies when [balance] is false;
each queue gets its own pool when balancing.

Weights order the queues a single worker checks. Under simple or auto balancing each queue already gets its own process pool, so there is no ordering left to weight and the two settings contradict each other. Skyline throws rather than silently ignoring one of them. Either set 'balance' => false on that supervisor, or drop the queueWeights map. See Weighted queues.