Universal GPU Support via Vulkan

Why Vulkan

Proposed Architecture

GPU Memory Model

Tensors that are moved to the GPU will have two representations:

Key design decisions:

• Lazy transfer — data is uploaded to GPU only when first needed by a GPU kernel. PHP code never triggers a transfer explicitly.
• Unified weight memory for LLMs — model weights reside permanently on GPU after the first forward pass. No round-trip per token.
• Activation tensors — intermediate activations are created on GPU, freed after the backward pass. Never touch CPU RAM.
• Gradient synchronization — optimizer step runs on GPU. Updated weights stay on GPU. Only loss scalar and metrics cross the PCIe bus.

SPIR-V Compute Kernels

Each C tensor operator (tensor_matmul, tensor_softmax, etc.) will have a corresponding GLSL compute shader that compiles to SPIR-V via glslc. SPIR-V binaries are embedded into libgpu.so at build time.

Example: GEMM kernel (matmul.comp)

#version 450
// Tiled GEMM: each workgroup handles a 16×16 output tile
layout(local_size_x = 16, local_size_y = 16) in;

layout(set = 0, binding = 0) readonly buffer MatA { float A[]; };
layout(set = 0, binding = 1) readonly buffer MatB { float B[]; };
layout(set = 0, binding = 2) writeonly buffer MatC { float C[]; };

layout(push_constant) uniform PushConstants {
    uint M; uint N; uint K;
    float alpha; float beta;
};

shared float tileA[16][16];
shared float tileB[16][16];

void main() {
    uint row = gl_GlobalInvocationID.x;
    uint col = gl_GlobalInvocationID.y;
    float acc = 0.0;

    for (uint t = 0; t < (K + 15) / 16; t++) {
        tileA[gl_LocalInvocationID.x][gl_LocalInvocationID.y] =
            (row < M && t*16+gl_LocalInvocationID.y < K)
            ? A[row * K + t*16 + gl_LocalInvocationID.y] : 0.0;
        tileB[gl_LocalInvocationID.x][gl_LocalInvocationID.y] =
            (t*16+gl_LocalInvocationID.x < K && col < N)
            ? B[(t*16+gl_LocalInvocationID.x) * N + col] : 0.0;
        barrier();
        for (uint k = 0; k < 16; k++)
            acc += tileA[gl_LocalInvocationID.x][k]
                 * tileB[k][gl_LocalInvocationID.y];
        barrier();
    }
    if (row < M && col < N)
        C[row * N + col] = alpha * acc + beta * C[row * N + col];
}

Priority kernel list

Dispatch Strategy

Vulkan compute shaders are dispatched via vkCmdDispatch(). The workgroup size is tuned per operator:

For batched GEMM (LLM prefill), workgroups are dispatched in a 3-D grid: dispatchX = ceil(M/16), dispatchY = ceil(N/16), dispatchZ = batch. This maps batch → Z and keeps the tile loop identical to single-batch.

PHP API — Unchanged

The entire PHP API is 100% backward-compatible with the CPU backend. The GPU backend is selected at TensorEngine initialization time by an environment variable or config flag. All Tensor, Sequential, InferenceSession, etc. calls remain identical.

// No PHP changes required — backend selected by env var
// PML_BACKEND=vulkan php train.php
// PML_BACKEND=cpu php train.php  (default)

$model->train($dataset, epochs: 20);  // same call — GPU if available
$preds = $model->predict($test);      // same call

Optionally, explicit device placement will be exposed:

// Future API (not yet implemented)
Tensor::setDefaultDevice('gpu');
$t = Tensor::randomNormal([4096, 4096]);  // allocated on GPU VRAM
$t->to('cpu');                                // explicit download

Implementation Phases

Expected Performance

Estimates based on operator FLOP counts and hardware peak throughput (TFLOPS). Actual results depend on memory bandwidth, kernel launch overhead, and occupancy.

Current Limitations to Resolve First

Before the Vulkan backend can be integrated, these CPU-side items should be addressed: