OptimizerState

class vulkpy.nn.OptimizerState

Bases: object

Abstract base class for Optimizer State

See also

vulkpy.nn.Optimizer

Optimizer

vulkpy.nn.SGDState

OptimizerState subclass for SGD

vulkpy.nn.AdamState

OptimizerState subclass for Adam

Notes

Mutable per-parameter values are stored at this class instance, although static global parameters (e.g. learning rate) are stored at Optimizer class.

Subclass of OptimizerState should implement Optimizer.grad2diff(), which takes accumulated gradients and returns update difference.

In standard design, OptimizerState holds a reference to its parent Optimizer in order to access global parameters.

Methods Summary

grad2diff(grad)

Compute update diff from gradient

Methods Documentation

grad2diff(grad: Array) → Array

Compute update diff from gradient

Parameters:

grad (vulkpy.Array) – Accumulated gradient

Returns:

diff – Update diff. (v += opt_state.grad2diff(grad))

Return type:

vulkpy.Array

Notes

Subclass must implement this method.

__init__()