Optimizer

class vulkpy.nn.Optimizer

Bases: object

Abstract base class for Optimizer

See also

vulkpy.nn.OptimizerState

Optimizer State

vulkpy.nn.SGD

Optimizer subclass for SGD

vulkpy.nn.Adam

Optimizer subclass for Adam

Notes

Optimizer class is designed to pass to Parameter constructor through Module constructor. Inside Parameter constructor, Optimizer.init_state() is called and corresponding OptimizerState are stored at the Parameter instance.

Mutable per-parameter values are stored at OptimizerState class instance, although static global parameters (e.g. learning rate) are stored at this class.

To implement specific optimizer, Subclass of Optimizer should implement Optimizer.init_state() method, which returns corresponding subclass of OptimizerState.

Examples

>>> import vulkpy as vk
>>> gpu = vk.GPU()
>>>
>>> adam = vk.nn.Adam(gpu) # Optimizer
>>> dense = vk.nn.Dense(gpu, 1, 1, w_opt=adam, b_opt=adam) # Module

Methods Summary

init_state(shape)

Create OptimizerState

Methods Documentation

init_state(shape: Iterable[int]) → OptimizerState

Create OptimizerState

Parameters:

shape (iterable of ints) – Parameter Shape

Returns:

opt_state – Optimizer State

Return type:

vulkpy.nn.OptimizerState

Notes

Subclass must implement this method.

__init__()