Parametric functions

nnabla_rl.parametric_functions.noisy_net(inp: nnabla._variable.Variable, n_outmap: int, base_axis: int = 1, w_init: Optional[Callable[[Tuple[int, ...]], numpy.ndarray]] = None, b_init: Optional[Callable[[Tuple[int, ...]], numpy.ndarray]] = None, noisy_w_init: Optional[Callable[[Tuple[int, ...]], numpy.ndarray]] = None, noisy_b_init: Optional[Callable[[Tuple[int, ...]], numpy.ndarray]] = None, fix_parameters: bool = False, rng: Optional[numpy.random.mtrand.RandomState] = None, with_bias: bool = True, with_noisy_bias: bool = True, apply_w: Optional[Callable[[nnabla._variable.Variable], nnabla._variable.Variable]] = None, apply_b: Optional[Callable[[nnabla._variable.Variable], nnabla._variable.Variable]] = None, apply_noisy_w: Optional[Callable[[nnabla._variable.Variable], nnabla._variable.Variable]] = None, apply_noisy_b: Optional[Callable[[nnabla._variable.Variable], nnabla._variable.Variable]] = None, seed: int = - 1) nnabla._variable.Variable[source]

Noisy linear layer with factorized gaussian noise proposed by Fortunato et al. in the paper “Noisy networks for exploration”. See: https://arxiv.org/abs/1706.10295 for details.

Parameters
  • inp (nn.Variable) – Input of the layer n_outmaps (int): output dimension of the layer.

  • n_outmap (int) – Output dimension of the layer.

  • base_axis (int) – Axis of the input to treat as sample dimensions. Dimensions up to base_axis will be treated as sample dimensions. Defaults to 1.

  • w_init (None or Callable[[Tuple[int, ...]], np.ndarray]) – Initializer of weights used in deterministic stream. Defaults to None. If None, will be initialized with Uniform distribution \((-\frac{1}{\sqrt{fanin}},\frac{1}{\sqrt{fanin}})\).

  • b_init (None or Callable[[Tuple[int, ...]], np.ndarray]) – Initializer of bias used in deterministic stream. Defaults to None. If None, will be initialized with Uniform distribution \((-\frac{1}{\sqrt{fanin}},\frac{1}{\sqrt{fanin}})\).

  • noisy_w_init (None or Callable[[Tuple[int, ...]], np.ndarray]) – Initializer of weights used in noisy stream. Defaults to None. If None, will be initialized to a constant value of \(\frac{0.5}{\sqrt{fanin}}\).

  • noisy_b_init (None or Callable[[Tuple[int, ...]], np.ndarray]) – Initializer of bias used in noisy stream. Defaults to None. If None, will be initialized to a constant value of \(\frac{0.5}{\sqrt{fanin}}\).

  • fix_parameters (bool) – If True, underlying weight and bias parameters will Not be updated during training. Default to False.

  • rng (None or np.random.RandomState) – Random number generator for parameter initializer. Defaults to None.

  • with_bias (bool) – If True, deterministic bias term is included in the computation. Defaults to True.

  • with_noisy_bias (bool) – If True, noisy bias term is included in the computation. Defaults to True.

  • apply_w (None or Callable[[nn.Variable], nn.Variable]) – Callable object to apply to the weights on initialization. Defaults to None.

  • apply_b (None or Callable[[nn.Variable], nn.Variable]) – Callable object to apply to the bias on initialization. Defaults to None.

  • apply_noisy_w (None or Callable[[nn.Variable], nn.Variable]) – Callable object to apply to the noisy weight on initialization. Defaults to None.

  • apply_noisy_b (None or Callable[[nn.Variable], nn.Variable]) – Callable object to apply to the noisy bias on initialization. Defaults to None.

  • seed (int) – Random seed. If -1, seed will be sampled from global random number generator. Defaults to -1.

Returns

Linearly transformed input with noisy weights

Return type

nn.Variable

nnabla_rl.parametric_functions.spatial_softmax(inp: nnabla._variable.Variable, alpha_init: float = 1.0, fix_alpha: bool = False) nnabla._variable.Variable[source]

Spatial softmax layer proposed in https://arxiv.org/abs/1509.06113. Computes

\[ \begin{align}\begin{aligned}s_{cij} &= \frac{\exp(x_{cij} / \alpha)}{\sum_{i'j'} \exp(x_{ci'j'} / \alpha)}\\f_{cx} &= \sum_{ij} s_{cij}px_{ij}, f_{cy} = \sum_{ij} s_{cij}py_{ij}\\y_{c} &= (f_{cx}, f_{cy})\end{aligned}\end{align} \]

where \(x, y, \\alpha\) are the input, output and parameter respectively, and \(c, i, j\) are the number of channels, heights and widths respectively. \((px_{ij}, py_{ij})\) is the image-space position of the point (i, j) in the response map.

Parameters
  • inp (nn.Variables) – Input of the layer. Shape should be (batch_size, C, H, W)

  • alpha_init (float) – Initial temperature value. Defaults to 1.

  • fix_alpha (bool) – If True, underlying alpha will Not be updated during training. Defaults to False.

Returns

Feature points, Shape is (batch_size, C*2)

Return type

nn.Variables