# pyRDDLGym-rl

**Repository Path**: xthaf/pyRDDLGym-rl

## Basic Information

- **Project Name**: pyRDDLGym-rl
- **Description**: No description available
- **Primary Language**: Unknown
- **License**: MIT
- **Default Branch**: main
- **Homepage**: None
- **GVP Project**: No

## Statistics

- **Stars**: 0
- **Forks**: 0
- **Created**: 2025-02-12
- **Last Updated**: 2025-02-13

## Categories & Tags

**Categories**: Uncategorized

**Tags**: None

## README

# pyRDDLGym-rl

![Python Version](https://img.shields.io/badge/python-3.8%2B-blue)
[![PyPI Version](https://img.shields.io/pypi/v/pyRDDLGym-rl.svg)](https://pypi.org/project/pyRDDLGym-rl/)
[![Documentation Status](https://readthedocs.org/projects/pyrddlgym/badge/?version=latest)](https://pyrddlgym.readthedocs.io/en/latest/sb.html)
![License: MIT](https://img.shields.io/badge/License-MIT-blue.svg)
[![Cumulative PyPI Downloads](https://img.shields.io/pypi/dm/pyrddlgym-rl)](https://pypistats.org/packages/pyrddlgym-rl)

[Installation](#installation) | [Examples](#running-the-basic-examples) | [Creating Environments](#creating-an-environment)

Provides wrappers for deep reinforcement learning algorithms (i.e. Stable Baselines 3 and RLlib) to work with pyRDDLGym.

> [!NOTE]  
> If your environment has differentiable dynamics and a differentiable reward, try the gradient-based [JAX planner](https://github.com/pyrddlgym-project/pyRDDLGym-jax).

## Installation

To run the basic examples you will need ``pyRDDLGym>=2.1``, ``rddlrepository>=2.0`` and one of the supported reinforcement learning frameworks:
- ``stable-baselines3>=2.2.1``
- ``ray[rllib]>=2.9.2``

You can install this package, together with all of its requirements via pip:

```shell
pip install stable-baselines3  # need one of these two
pip install -U "ray[rllib]"
pip install rddlrepository pyRDDLGym-rl
```

## Running the Basic Examples

### Stable Baselines 3

To run the stable-baselines3 example, navigate to the install directory of pyRDDLGym-rl, and type:

```shell
python -m pyRDDLGym_rl.examples.run_stable_baselines <domain> <instance> <method> <steps> <learning_rate>
```

where:
- ``<domain>`` is the name of the domain in rddlrepository, or a path pointing to a ``domain.rddl`` file
- ``<instance>`` is the name of the instance in rddlrepository, or a path pointing to an ``instance.rddl`` file
- ``<method>`` is the RL algorithm to use [a2c, ddpg, dqn, ppo, sac, td3]
- ``<steps>`` is the (optional) number of samples to generate from the environment for training, and
- ``<learning_rate>`` is the (optional) learning rate to specify for the algorithm.

### RLLib

To run the RLlib example, from the install directory of pyRDDLGym-rl, type:

```shell
python -m pyRDDLGym_rl.examples.run_rllib <domain> <instance> <method> <iters>
```

where:
- ``<domain>`` is the name of the domain in rddlrepository, or a path pointing to a ``domain.rddl`` file
- ``<instance>`` is the name of the instance in rddlrepository, or a path pointing to an ``instance.rddl`` file
- ``<method>`` is the RL algorithm to use [dqn, ppo, sac]
- ``<iters>`` is the (optional) number of iterations of training

## Creating an Environment

You can create an environment wrapper to use with your own RL implementations, or a package that is not currently supported by us:

```python
import pyRDDLGym
from pyRDDLGym_rl.core.env import SimplifiedActionRDDLEnv
env = pyRDDLGym.make("domain name", "instance name", base_class=SimplifiedActionRDDLEnv)
```

This creates an instance of ``gymnasium.Env`` in which the action space is simplified by concatenating all continuous, discrete and boolean action fluents into single tensors.
You can then use this environment as you would in [pyRDDLGym](https://github.com/pyrddlgym-project/pyRDDLGym).