Dense

class vulkpy.nn.Dense

Bases: Module

Fully connected Dense Layer

Methods Summary

__call__(x)	Call Module
backward(dy)	Backward
forward(x)	Forward
update()	Update values with accumulated gradients
zero_grad()	Clear accumulated gradients

Methods Documentation

__call__(x: Array) → Array

Call Module

Parameters:

x (vulkpy.Array) – Input

Returns:

y – Output

Return type:

vulkpy.Array

Raises:

ValueError – If input (x) shape doesn’t have at least 2-dimensions.

Notes

This function stores input (x) and output (y) for training.

backward(dy: Array) → Array

Backward

Parameters:

dy (vulkpy.Array) – Batch grad

Returns:

Batch grad

Return type:

vulkpy.Array

Notes

\[\begin{split}dx = dy W\\ dW = dy ^T \cdot x\\ db = dy\end{split}\]

forward(x: Array) → Array

Forward

Parameters:

x (vulkpy.Array) – Batch input

Returns:

Batch output

Return type:

vulkpy.Array

Notes

\[y = Wx + b\]

Warning

Generally, users should not call this method directly. Use __call__ instead, where input / output are stored for training.

update()

Update values with accumulated gradients

zero_grad()

Clear accumulated gradients

__init__(gpu: GPU, input_dim: int, output_dim: int, *, w_init: Callable[[GPU, Iterable[int]], Array] | None = None, b_init: Callable[[GPU, Iterable[int]], Array] | None = None, w_opt: Optimizer | None = None, b_opt: Optimizer | None = None, w_reg: Regularizer | None = None, b_reg: Regularizer | None = None)

Initialize Dense

Parameters:

• gpu (vulkpy.GPU) – GPU

• input_dim (int) – Input dimension

• output_dim (int) – Output dimension

• Callable (b_init) – Weight initializer. If None (default), vulkpy.nn.HeNormal is used.

• optional – Weight initializer. If None (default), vulkpy.nn.HeNormal is used.

• Callable – Bias initializer. If None (default), bias is initialized with 0.

• optional – Bias initializer. If None (default), bias is initialized with 0.

• w_opt (vulkpy.nn.Optimizer, optional) – Weight Optimizer. If None (default), vulkpy.nn.Adam is used.

• b_opt (vulkpy.nn.Optimizer, optional) – Bias Optimizer. If None (default), vulkpy.nn.Adam is used.

• w_reg (vulkpy.nn.Regularizer, optional) – Weight Regularizer.

• b_reg (vulkpy.nn.Regularizer, optional) – Bias Regularizer