BulletArm Simulator

This is the main package for the BulletArm library. It contains the PyBullet simulation environments, robots, and task definitions. The simulation workspace consists of a robot arm mounted on the floor of the simulation environment, a workspace where objects are generated, and sensor(s). There are three key terms within BulletArm: Enviornment, Configuration, and Episode. An environment is an instance ofthe PyBullet simulator in which the robot interacts with objects while trying to solve some task. This include the initial environment state, the reward function, and termination requirements. A configuration contains additional specifications for the task such as the robot arm, the size of the workspace, the physics mode, etc. A full list of parameters can be found here. Episodes are generated by taking actions (steps) within an environment until the episode ends.

The simulation environment containing a robot arm, a camera, and a workspace.

---

Tasks

Our set of tasks are seperated into two categories based on the action space: open-loop control and closed-loop control. Future work is planned to allow all tasks to use either control-type but for now the below table shows current avaliable control type for each task. For each avaliable task, the name required by the EnvFactory to initalize that task is also shown.

Task	Open-Loop	Closed-Loop
Block Reaching	☐	☑ close_loop_block_reaching
Block Pushing	☐	☑ close_loop_block_pushing
Block Pulling	☐	☑ close_loop_block_pulling
Block Picking	☑ block_picking	☑ close_loop_block_picking
Block Stacking	☑ block_stacking	☑ close_loop_block_stacking
Block In Bowl	☐	☑ close_loop_block_in_bowl
House Building 1	☑ house_building_1	☑ close_loop_house_building_1
House Building 2	☑ house_building_2	☐
House Building 3	☑ house_building_3	☐
House Building 4	☑ house_building_4	☐
Improvise House Building 2	☑ improvise_house_building_2	☐
Improvise House Building 3	☑ improvise_house_building_3	☐
Bin Packing	☑ block_bin_packing	☐
Bottle Arrangement	☑ bottle_tray	☐
Box Palletizing	☑ box_palletizing	☐
Covid Test	☑ covid_test	☐
Corner Picking	☐	☑ close_loop_block_picking_corner
Drawer Opening	☐	☑ close_loop_drawer_opening
Object Grasping	☑ clutter_picking	☑ close_loop_clutter_picking

---

EnvFactory & EnvRunner

Interaction with BulletArm is done through the EnvFactory and EnvRunner classes. The EnvFactory is the entry point and creates the environment specified by the configuration passed as input. The EnvFactory can create either a single environment (SingleRunner) or multiple environments (MultiRunner) to be run in parallel. The EnvRunner provides the API which interacts with the environment. This API is modeled after the typical agent-environment RL setup popularized by OpenAI Gym.

EnvFactory

createEnvs(num_processes, env_type, env_config={}, planner_config={})

Wrapper function to create either a single env the the main process or some number of envs each in their own seperate process.

Parameters

• num_processes (int) – Number of envs to create

• env_type (str) – The type of environment to create

• env_config (dict) – Intialization arguments for the env

• planner_config (dict) – Intialization arguments for the planner

Returns

SingleRunner or MultiRunner containing the environment

Return type

EnvRunner

createMultiprocessEnvs(num_processes, env_type, env_config={}, planner_config={})

Create a number of environments on different processes to run in parralel

Parameters

• num_processes (int) – Number of envs to create

• env_type (str) – The type of environment to create

• env_config (dict) – Intialization arguments for the env

• planner_config (dict) – Intialization arguments for the planner

Returns

MultiRunner containing all environments

Return type

MultiRunner

createSingleProcessEnv(env_type, env_config={}, planner_config={})

Create a single environment

Parameters

• env_type (str) – The type of environment to create

• env_config (dict) – Intialization arguments for the env

• planner_config (dict) – Intialization arguments for the planner

Returns

SingleRunner containing the environment

Return type

SingleRunner

getEnvFn(env_type)

Get the env creation function of the given type.

Parameters

env_type (str) – The type of environment to create

Returns

A function which creates the env when run

Return type

function

EnvRunner

class MultiRunner(env_fns, planner_fns)

Runner which runs mulitple environemnts in parallel in subprocesses and communicates with them via pipe.

Parameters

• env_fns (list[function]) – Env creation functions

• planner_fns (list[function]) – Planner creation functions

close()

Close all worker processes.

gatherDeconstructTransitions(planner_episode)

Gather deconstruction transitions and reverse them for construction

Parameters

planner_episode (-) – The number of expert episodes to gather

Returns: list of transitions. Each transition is in the form of ((state, in_hand, obs), action, reward, done, (next_state, next_in_hand, next_obs))

getEmptyInHand()

static getEnvGitHash()

getNextAction()

Get the next action from the planner for each environment.

Returns

Actions

Return type

numpy.array

getNumObj()

Get the number of objects in the environment Returns: int: number of objects

loadFromFile(path)

Loads the state of the environments from a file.

Parameters

path (str) – The path to the environment states to load

Returns

Flag indicating if the loading succeeded for all environments

Return type

bool

reset()

Reset each environment.

Returns

Observations

Return type

numpy.array

reset_envs(env_nums)

Resets the specified environments.

Parameters

env_nums (list[int]) – The environments to be reset

Returns

Observations

Return type

numpy.array

restore()

Restores the locally saved state of the environments.

save()

Locally saves the current state of the environments.

saveToFile(path)

Saves the current state of the environments to file.

Parameters

path (str) – The path to save the enviornment states to

step(actions, auto_reset=False)

Step the environments synchronously.

Parameters

• actions (numpy.array) – Actions to take in each environment

• auto_reset (bool) – Reset environments automatically after an episode ends

Returns

(observations, rewards, done flags)

Return type

(numpy.array, numpy.array, numpy.array)

stepAsync(actions, auto_reset=False)

Step each environment in a async fashion.

Parameters

• actions (numpy.array) – Actions to take in each environment

• auto_reset (bool) – Reset environments automatically after an episode ends

stepWait()

Wait until each environment has completed its next step.

Returns

(observations, rewards, done flags)

Return type

(numpy.array, numpy.array, numpy.array)

class SingleRunner(env, planner=None)

RL environment runner which runs a single environment.

Parameters

• env (BaseEnv) – Environment

• planner (BasePlanner) – Planner

getNextAction()

loadFromFile(path)

Loads the state of the environment from a file.

Parameters

path (str) – The path to the environment state to load

Returns

Flag indicating if the loading succeeded

Return type

bool

reset()

Reset the environment.

Returns

Observation

Return type

numpy.array

restore()

Restores the locally saved state of the environment.

save()

Locally saves the current state of the environment.

saveToFile(path)

Saves the current state of the environment to file.

Parameters

path (str) – The path to save the enviornment state to

step(action, auto_reset=True)

Step the environment.

Parameters

• action (numpy.array) – Action to take in the environment

• auto_reset (bool) – Reset the environment after an episode ends

Returns

(observations, rewards, done flags)

Return type

(numpy.array, numpy.array, numpy.array)

---

Configuration Parameters

A task in BulletArm is defined as the combination of the environment and the configuration passed to that environemnt. The environment details the high level details for the task i.e. the initial state, goal state, invalid states, etc. The configuration contains lower level details which can add to task variety. The default config can be found here. We briefly detail the various config parameters below but additional information can be found in the default config.

Parameter	Example	Description
render	False/True	Render the PyBullet GUI or run headless
max_steps	10	The maximum number of steps per episode.
workspace	[[0, 0.2], [0, 0.2], [0, 0.2]]	The workspace in terms of the range in x, y, and z.
workspace_check	point/bounding_box	How to determine if a object is within the workspace by either check the COM of the object of the bounding box.
robot	kuka	The type of the robot to use in the simulator.
action_sequence	pxyzr	The action space the agent will act within. ‘pxyzr’ defines a 5-vector action including the gripper action (p), the position of the gripper (x,y,z), and its top-down rotation (r).
obs_size	128	The pixel size of the heightmap.
in_hand_size	24	The pixel size of the in-hand image.
random_orientation	True	Whether or not to generate objects with a random orientation
object_scale_range	[0.6, 0.7]	The scale applied to objects in the workspace. Addes variation and helps with generalization.
fast_mode	False/True	If True, teleports the arm when possible to speed up simulation. Bad for more dynamical domains & closed-loop control.
phyics_mode	fast, slow	The type of physics to use during simulation. Can be run faster with less accurate physics or slower with more accurate physics.
num_objects	3	Specifies the number of objects to generate in domains which have a variable number of objects.

---

Robots

Different robotic arms can lead to drastically different policies due to the differing kinematics between arms. We provide four robotic arms: KUKA IIWA, Frane Emika Panda, Universal Robots UR5 with a parallel jaw gripper, and Universal Robots UR5 with Robotiq 2F-85 gripper. To add a new manipulator see the new robot tutorial.

Currently supported robotic arms.

---

Objects

We provide a number of objects to use out-of-the-box when creating new tasks. The list of avaliable objects can be seen below. Objects are generating within the workspace by using the _generateShapes function in the base enviornment. Objects are identified by their ID which is located in bulletarm/pybullet/utils/constants.py. To add a new object see the new object tutorial.

Object
Cube
Sphere
Cylinder
Cone
Brick
Triangle
Roof
Teapot
Teapot Lid
Cup
Bowl
Plate
Spoon
Bottle
Box
Pallet
Test Tube
Swab
Flat Block
Random Household 200
Grasp Net

_generateShapes(self, shape_type=0, num_shapes=1, scale=None, pos=None, rot=None, min_distance=None, padding=None, random_orientation=False, z_scale=1, model_id=1)

Generate objects within the workspace.

Generate a number of objects specified by the shape type within the workspace.

Parameters

• shape_type (int) – The shape to generate. Defaults to 0.

• num_shapes (int) – The number of objects to generate. Deafults to 1.

• scale (double) – The scale for the entire object. Defaults to None.

• pos (list[double]) – Generate object at specified position. Defalts to None.

• rot (list[double]) – Generate object at specified orientation. Defaults to None,

• min_distance (double) – Minimum distance objects must be from other objects. Defaults to None.

• padding (double) – Distance from edge of workspace that objects can be generated in. Defaults to None.

• random_orientation (bool) – Generates objects with a random orientation if True. Defaults to False.

• z_scale (double) – The scale for the objects Z axis. Defaults to 1.

• model_id (int) – Used to specify the specific object within a class of object shapes. Defaults to 1.

Returns

PyBullet IDs for the generated objects

Return type

list[int]