Training System

Trainer

Pml\Training\Trainer is the high-level training orchestrator. It wraps a Sequential model and a TrainingArguments config, runs the epoch loop, calls LR scheduler, handles early stopping, saves checkpoints, and fires callbacks at each lifecycle event.

$args = new TrainingArguments(
    epochs:       30,
    batchSize:    128,
    learningRate: 3e-4,
    lrSchedule:   'cosine',
    patience:     5,
    outputDir:    'ckpt/run1',
);

$trainer = new Trainer($model, $args);
$trainer->addCallback(new MyCallback());
$result = $trainer->train($trainDs, $valDs);

TrainingArguments

Value object holding all training hyperparameters. All properties are readonly.

$args = new TrainingArguments(
    epochs:        100,
    batchSize:     64,
    learningRate:  1e-3,
    lrSchedule:    'cosine',
    warmupEpochs:  5,
    patience:      10,
    outputDir:     'ckpt/mymodel',
);

LRScheduler

Pml\Training\LRScheduler adjusts the optimizer's learning rate each epoch based on the schedule set in TrainingArguments. Used internally by Trainer; you can also use it directly in custom loops.

Schedule formulas

All schedules are preceded by a linear warm-up if warmupEpochs > 0.

GradScaler

Pml\Training\GradScaler implements dynamic loss scaling for mixed-precision training. Currently a scaffold — useful for fp32 training on systems that benefit from gradient scaling to prevent underflow. Full fp16/bf16 support requires GPU backend.

$scaler = new GradScaler(enabled: true);

foreach ($loader->batches() as $batch) {
    $preds = $model->forward($batch->samples());
    $grad  = $model->getLoss()->differentiate($preds, $batch->labels());
    $grad  = $scaler->scale($grad);
    $model->backward($grad);
    $scaler->unscaleAndStep($model->getOptimizer(), $model->getLayers());
    $scaler->update();
}

EarlyStopping

Pml\NeuralNetwork\EarlyStopping monitors a metric (typically validation loss) and signals when training should stop. Used internally by Sequential::train() and Trainer. Can also be used standalone in custom loops.

Callbacks

Implement Pml\Training\TrainerCallback to hook into training lifecycle events:

interface TrainerCallback
{
    public function onTrainBegin(TrainingArguments $args, int $steps): void;
    public function onEpochBegin(int $epoch, int $epochs): void;
    public function onBatchEnd(int $step, float $batchLoss): void;
    public function onEpochEnd(int $epoch, float $trainLoss, ?float $valLoss): void;
    public function onTrainEnd(TrainingResult $result): void;
}

class WandbCallback implements TrainerCallback
{
    public function onEpochEnd(int $epoch, float $trainLoss, ?float $valLoss): void
    {
        // Log to W&B, TensorBoard, Redis, etc.
        file_put_contents('logs/loss.ndjson',
            json_encode(['epoch' => $epoch, 'train' => $trainLoss, 'val' => $valLoss]) . "\n",
            FILE_APPEND
        );
    }

    public function onTrainBegin(TrainingArguments $args, int $steps): void {}
    public function onEpochBegin(int $e, int $t): void {}
    public function onBatchEnd(int $s, float $l): void {}
    public function onTrainEnd(TrainingResult $r): void {}
}

TrainingResult

Returned by Trainer::train().

Full Example

use Pml\NeuralNetwork\Sequential;
use Pml\NeuralNetwork\Layers\{Dense, LayerNorm, Gelu, Dropout};
use Pml\NeuralNetwork\Optimizers\AdamW;
use Pml\Losses\CategoricalCrossEntropy;
use Pml\Training\{Trainer, TrainingArguments};
use Pml\Dataset;

$model = new Sequential(
    layers: [
        new Dense(512, 256),
        new LayerNorm(256),
        new Gelu(),
        new Dropout(0.1),
        new Dense(256, 10),
    ],
    lossFn:    new CategoricalCrossEntropy(),
    optimizer: new AdamW(learningRate: 3e-4, weightDecay: 0.01),
);

$args = new TrainingArguments(
    epochs:       50,
    batchSize:    256,
    learningRate: 3e-4,
    lrSchedule:   'cosine',
    warmupEpochs: 3,
    patience:     8,
    outputDir:    'ckpt/mnist',
);

$trainer = new Trainer($model, $args);
$trainer->addCallback(new WandbCallback());

$result = $trainer->train($trainDs, $valDs);

printf("Best epoch: %d, Best val loss: %.4f, Stopped early: %s\n",
    $result->bestEpoch,
    $result->bestValLoss,
    $result->stoppedEarly ? 'yes' : 'no'
);