Getting Started#

Installation Guide#

ObjectRL is tested with Python 3.12. We recommend using a virtual environment to keep dependencies isolated and manageable.

Create and activate a virtual environment using conda:

conda create -n objectrl python=3.12 -y
conda activate objectrl

Alternatively, you can use venv or other virtual environment tools.

Install ObjectRL

You have two options for installing ObjectRL: from PyPI or directly from the GitHub repository for the latest development version.

Install the latest stable release:

pip install objectrl

Install the latest development version from source:

git clone https://github.com/adinlab/objectrl.git
cd objectrl
pip install -e .

Install optional dependencies

For additional features such as documentation generation, install optional dependencies:

pip install objectrl['docs']

pip install -e .[docs]

Running Your First Experiment#

You can launch your first RL experiment directly from the command line.

Run SAC on the default cheetah environment:

If installed from PyPI, use:
```
python -m objectrl.main --model.name sac
```
If installed from GitHub, use:
```
python objectrl/main.py --model.name sac
```
Other examples will assume the GitHub installation.

Run DDPG on the hopper environment:

python objectrl/main.py --model.name ddpg --env.name hopper

Customize training parameters (e.g., train SAC on hopper for 100,000 steps and evaluate every 5 episodes):

python objectrl/main.py\
   --model.name sac\
   --env.name hopper\
   --training.max_steps 100_000\
   --training.eval_episodes 5

Configuration Options#

For simple changes, CLI arguments are usually sufficient. However, for more complex configurations or reproducibility, you can create custom YAML file, e.g.,

ppo.yaml#

model:
   name: ppo
training:
   warmup_steps: 0
   learn_frequency: 2048
   batch_size: 64
   n_epochs: 10

and use it at runtime via

python objectrl/main.py --config ppo.yaml

This system enables clean separation of experiment configurations and code.

Command line arguments are able to overwrite these parameters which allows for a quick iteration of parameter tuning. Assume you want to keep your ppo.yaml file, but test whether a different batch size improves performance.

python objectrl/main.py --config ppo.yaml --training.batch_size 32

To get an overview of all non-model specific parameters run

python objectrl/main.py --help

Model specific parameters, e.g., SAC’s, are available via

python objectrl/main.py --help_model sac

See API for further details on the configuration.

Next Steps#

Once you have your environment set up and can run a simple experiment, explore the following documentation sections for deeper understanding:

Examples: Hands-on tutorials and case studies for customizing and extending ObjectRL.
API: Detailed reference documentation for classes, modules, and configuration options.

Troubleshooting#

If you encounter errors related to environment dependencies (e.g., MuJoCo), verify your installation and environment variables.
Use pip list or conda list to check installed packages.
Run experiments with --help to see available CLI options.
Open an issue on our GitHub repository.

Getting Started

Contents