Actor Networks

Actor Networks#

This module provides neural network architectures for policy learning in reinforcement learning. It includes both deterministic and probabilistic actor models.

Descriptions#

  • ActorNetProbabilistic: Outputs a squashed Gaussian distribution over actions. Useful in stochastic policies like Soft Actor-Critic (SAC).

  • ActorNet: Outputs deterministic actions, typically used in deterministic policy gradients or for evaluation.

Classes#

class objectrl.nets.actor_nets.ActorNetProbabilistic(dim_state: int, dim_act: int, n_heads: int = 1, depth: int = 3, width: int = 256, act: Literal['relu', 'crelu'] = 'relu', has_norm: bool = False, upper_clamp: float = -2.0)[source]#

Bases: Module

__init__(dim_state: int, dim_act: int, n_heads: int = 1, depth: int = 3, width: int = 256, act: Literal['relu', 'crelu'] = 'relu', has_norm: bool = False, upper_clamp: float = -2.0) → None[source]#

Probabilistic Actor Network that outputs a Gaussian distribution over actions using a squashed Gaussian head.

Args: dim_state (int): Observation space dimension (assumed 1D tuple). dim_act (int): Action space dimension (assumed 1D tuple). n_heads (int): Number of policy heads (useful for ensemble methods). depth (int): Number of hidden layers. width (int): Width of each hidden layer. act (str): Activation function to use. has_norm (bool): Whether to include normalization layers. upper_clamp (float): Upper clamp value for log-variance in Squashed Gaussian.

forward(x: Tensor, is_training: bool = True) → dict[str, Any][source]#

Forward pass to generate a squashed Gaussian action distribution.

Parameters:

  • x (Tensor) – Input observation tensor.

  • is_training (bool) – Whether to sample actions stochastically.

class objectrl.nets.actor_nets.ActorNet(dim_state: int, dim_act: int, n_heads: int = 1, depth: int = 3, width: int = 256, act: Literal['crelu', 'relu'] = 'relu', has_norm: bool = False)[source]#

Bases: Module

__init__(dim_state: int, dim_act: int, n_heads: int = 1, depth: int = 3, width: int = 256, act: Literal['crelu', 'relu'] = 'relu', has_norm: bool = False) → None[source]#

Deterministic Actor Network that outputs continuous actions.

Args: dim_state (int): Observation space dimension. dim_act (int): Action space dimension. n_heads (int): Number of parallel output heads. depth (int): Number of hidden layers. width (int): Width of each hidden layer. act (str): Activation function name. has_norm (bool): Whether to use normalization layers.

forward(x: Tensor, is_training: bool | None = None) → dict[str, Tensor][source]#

Forward pass for deterministic action prediction.

Parameters:

  • x (Tensor) – Input observation tensor.

  • is_training (Optional[bool]) – Unused; included for interface compatibility.

Note

Both classes support multiple output heads (n_heads) for ensemble learning or multi-policy settings.

Contents