Distributed Asynchronous Translation Workflows in Laravel

Crafting a Scalable Translation Engine

In modern global applications, providing content in multiple languages is no longer a luxury—it's a core requirement. However, translating complex datasets like examination questions in real-time presents a unique set of challenges: high latency from external APIs, cost management, and ensuring a smooth user experience.

Today, we delve into the architecture of our distributed translation process, designed to handle these challenges through asynchronous workflows and smart orchestration.

The Architecture at a Glance

Our system follows an event-driven pattern that separates the user-facing request from the resource-heavy translation logic.

graph TD
    A[UI Request] --> B[TranslationController]
    B -->|Check Cache/DB| C{Exists?}
    C -->|Yes| D[Broadcast Event]
    C -->|No| E[Dispatch TranslateTextJob]
    E --> F[TranslationService]
    F --> G[External API: Google/LLM]
    G --> H[Create Translation Record]
    H --> I[Broadcast TranslationComplete]
    D --> J[Frontend UI Update]
    I --> J

1. The Entry Point: Orchestrating State

The process begins with the TranslationController. Instead of executing the translation immediately, it manages the state of the request. By using hashing and caching, we ensure that redundant requests for the same content-language pair are handled efficiently.

1
public function __invoke(Request $request)
2
{
3
    $hash = hash('sha256', "$source:$target:$questionId");
4
    $translation = Translation::where('hash', $hash)->first();
5

6
    if ($translation) {
7
        // Instant feedback if already translated
8
        event(new TranslationComplete($translation, $userId));
9
        return response()->json($translation);
10
    }
11

12
    // Mark as pending and offload to the background
13
    Cache::put("translation:pending:$hash", true, now()->addMinutes(5));
14
    TranslateTextJob::dispatch($hash, $question, $source, $target, $userId);
15

16
    return response()->json(['status' => 'queued']);
17
}

2. Background Processing: The Distributed Worker

The TranslateTextJob acts as the bridge between the request and the service layer. Running in the background, it ensures the main application remains responsive while waiting for external translation services.

1
public function handle(): void
2
{
3
    if (Translation::where('hash', $this->hash)->exists()) {
4
        Cache::forget("translation:pending:$this->hash");
5
        return;
6
    }
7

8
    $translationService = new TranslationService;
9
    $translation = $translationService->translateQuestion($this->question, $this->target, $this->source);
10

11
    TranslationComplete::dispatch($translation, $this->userId);
12
    Cache::forget("translation:pending:$this->hash");
13
}

3. Smart Batching: Optimizing the Payload

The core logic resides in the TranslationService. To minimize costs and latency, we implement "Smart Batching"—collecting all translatable parts (question text, descriptions, answers) and sending them in a single payload to the translation provider.

1
private function translateWithGoogle(Question $question, string $target, string $source): Translation
2
{
3
    $textsToTranslate = [];
4
    $keyMap = [];
5

6
    // Collect all parts into a flat array for batching
7
    $textsToTranslate[] = (string) $question->question;
8
    $keyMap[] = ['type' => 'question'];
9

10
    foreach ($question->answers as $answer) {
11
        $textsToTranslate[] = $answer->answer;
12
        $keyMap[] = ['type' => 'answer', 'id' => $answer->id];
13
    }
14

15
    // Single Batch Call
16
    $translatedTexts = $this->client->translate($textsToTranslate, $target, $source);
17

18
    // Reconstruct the structured data
19
    return $this->createTranslationModel($question, $source, $target, $translatedData);
20
}

4. Real-time Synchronization: Broadcasting

Once the translation is complete, the system needs to notify the user. We use Laravel's broadcasting capabilities to push the results directly to the user's browser over a private websocket channel.

1
class TranslationComplete implements ShouldBroadcast
2
{
3
    public function broadcastOn()
4
    {
5
        return [new PrivateChannel('App.Models.User.' . $this->userId)];
6
    }
7
}

Conclusion

By decoupling the translation logic from the request cycle, we achieved a system that is both resilient and highly performant. The combination of asynchronous jobs, intelligent caching, and batch processing allow us to provide a premium, multi-lingual experience without compromising on speed or reliability.