Queues Are the Backbone
In any fintech platform, queues handle the heavy lifting: commission calculations, payment processing, email notifications, report generation, and data synchronization. A well-architected queue system is the difference between a platform that handles 1,000 transactions per minute and one that chokes at 100.
Dedicated Queues by Priority
Never put all jobs on a single queue. Separate by priority and concern:
// config/horizon.php
'environments' => [
'production' => [
'payments' => [
'connection' => 'redis',
'queue' => ['payments'],
'balance' => 'simple',
'processes' => 5,
'tries' => 3,
'timeout' => 30,
],
'commissions' => [
'connection' => 'redis',
'queue' => ['commissions'],
'balance' => 'auto',
'processes' => 10,
'tries' => 3,
'timeout' => 60,
],
'notifications' => [
'connection' => 'redis',
'queue' => ['notifications'],
'balance' => 'auto',
'processes' => 3,
'tries' => 2,
'timeout' => 15,
],
'reports' => [
'connection' => 'redis',
'queue' => ['reports'],
'balance' => 'simple',
'processes' => 2,
'tries' => 1,
'timeout' => 300,
],
],
],
This ensures a spike in commission calculations never delays payment webhook processing.
Job Batching for Bulk Operations
Monthly commission payouts process thousands of IBs. Job batching handles this cleanly:
class ProcessMonthlyPayouts
{
public function handle(): void
{
$ibs = IB::where('status', 'active')
->where('pending_commission', '>', 0)
->get();
$jobs = $ibs->map(fn ($ib) => new ProcessIBPayout($ib));
Bus::batch($jobs)
->name('Monthly Payouts - ' . now()->format('Y-m'))
->onQueue('commissions')
->allowFailures()
->then(fn (Batch $batch) => $this->onComplete($batch))
->catch(fn (Batch $batch, Throwable $e) => $this->onFailure($batch, $e))
->dispatch();
}
private function onComplete(Batch $batch): void
{
Notification::send(
User::role('finance')->get(),
new PayoutBatchCompleted($batch)
);
}
}
Batches give you progress tracking, failure handling, and the ability to cancel remaining jobs if something goes wrong.
Rate Limiting External APIs
PSP APIs have rate limits. Respect them with job middleware:
class RateLimitedPSPCall
{
public function handle($job, $next): void
{
Redis::throttle('psp-api')
->block(0)
->allow(30)
->every(60)
->then(function () use ($job, $next) {
$next($job);
}, function () use ($job) {
$job->release(30); // Retry in 30 seconds
});
}
}
class CheckPaymentStatus implements ShouldQueue
{
public function middleware(): array
{
return [new RateLimitedPSPCall];
}
public function handle(): void
{
$status = $this->psp->checkStatus($this->transactionId);
// Process status...
}
}
Unique Jobs
Prevent duplicate processing. If a trade event fires twice (common with webhooks), don't calculate commissions twice:
class CalculateTradeCommission implements ShouldQueue, ShouldBeUnique
{
public function __construct(
private int $tradeId
) {}
public function uniqueId(): string
{
return "commission-{$this->tradeId}";
}
// Lock expires after 60 seconds
public int $uniqueFor = 60;
}
Dead Letter Handling
Jobs that fail after all retries need attention, not silence:
class ProcessPaymentWebhook implements ShouldQueue
{
public int $tries = 3;
public array $backoff = [10, 60, 300]; // 10s, 1m, 5m
public function failed(Throwable $exception): void
{
// Log to dedicated failed payments table
FailedPayment::create([
'transaction_id' => $this->transactionId,
'error' => $exception->getMessage(),
'payload' => $this->payload,
'failed_at' => now(),
]);
// Alert finance team immediately
Notification::send(
User::role('finance')->get(),
new PaymentProcessingFailed($this->transactionId, $exception)
);
}
}
Build an admin interface to review and retry failed jobs. The finance team should be able to see what failed, why, and trigger a retry without developer intervention.
Monitoring with Horizon
Laravel Horizon is essential for queue visibility:
- Dashboard — real-time view of all queues, throughput, and wait times
- Metrics — job completion times, failure rates over time
- Tags — tag jobs by client, IB, or transaction type for filtering
- Alerts — configure notifications when queues back up
class CalculateTradeCommission implements ShouldQueue
{
public function tags(): array
{
return [
"trade:{$this->tradeId}",
"ib:{$this->ibId}",
"type:commission",
];
}
}
Key metrics to watch:
- Queue wait time — if jobs wait more than 30 seconds, add workers
- Failure rate — anything above 0.1% on payment queues needs investigation
- Throughput — track jobs/minute to capacity plan
Key Takeaways
- Separate queues by priority — payments never wait behind report generation
- Use job batching for bulk operations like monthly payouts
- Rate limit external API calls to avoid being throttled or banned
- Implement unique jobs to prevent duplicate processing from webhook retries
- Handle failures explicitly — dead letters need alerting and admin retry capability
- Monitor with Horizon — queue health is as important as application health
Queues are invisible to users when they work well and catastrophic when they don't. Invest in getting them right, and your platform will handle growth gracefully.