Public Interfaces¶
ROSBridgeLib¶
We use the following template from github for the ROSBridge: https://github.com/michaeljenkin/unityros.
Basically the ROSBridge consists of three different parts:
- ROSBridgeWebSocketConnection
- ROSBridgeMsg
- ROSBridge Actor aka Subscriber, Publisher and Service
ROSBridgeWebSocketConnection¶
-
class
ROSBridgeLib::
ROSBridgeWebSocketConnection
¶ This class handles the connection with the external ROS world, deserializing json messages into appropriate instances of packets and messages.
This class also provides a mechanism for having the callback’s exectued on the rendering thread. (Remember, Unity has a single rendering thread, so we want to do all of the communications stuff away from that.
The one other clever thing that is done here is that we only keep 1 (the most recent!) copy of each message type that comes along.
Version History 3.1 - changed methods to start with an upper case letter to be more consistent with c# style. 3.0 - modification from hand crafted version 2.0
- Author
- Michael Jenkin, Robert Codd-Downey and Andrew Speers
- Version
- 3.1
Public Functions
-
ROSBridgeLib.ROSBridgeWebSocketConnection.ROSBridgeWebSocketConnection(string host, int port)
Make a connection to a host/port.
This does not actually start the connection, use Connect to do that.
-
void ROSBridgeLib.ROSBridgeWebSocketConnection.AddServiceResponse(Type serviceResponse)
Add a service response callback to this connection.
-
void ROSBridgeLib.ROSBridgeWebSocketConnection.RemoveServiceResponse(Type serviceResponse)
Not Implemented!
-
void ROSBridgeLib.ROSBridgeWebSocketConnection.AddSubscriber(Type subscriber)
Add a subscriber callback to this connection.
There can be many subscribers.
-
void ROSBridgeLib.ROSBridgeWebSocketConnection.RemoveSubscriber(Type subscriber)
Removes a subscriber and disconnects him if the connection is already running.
-
void ROSBridgeLib.ROSBridgeWebSocketConnection.AddPublisher(Type publisher)
Add a publisher to this connection.
There can be many publishers.
-
void ROSBridgeLib.ROSBridgeWebSocketConnection.RemovePublisher(Type publisher)
Removes a publisher from the connection.
Disconnects the publisher if connection is already running.
-
void ROSBridgeLib.ROSBridgeWebSocketConnection.Connect()
Connect to the remote ros environment.
-
void ROSBridgeLib.ROSBridgeWebSocketConnection.Disconnect()
Disconnect from the remote ros environment.
Private Functions
-
void ROSBridgeLib.ROSBridgeWebSocketConnection.CheckConnection()
Checks if the connection is open and prints a message in that case.
Does not do anything in the other case as Connect() has an exception in this case and it wont be printed anyway.
-
void ROSBridgeLib.ROSBridgeWebSocketConnection.OnError(object sender, WebSocketSharp.ErrorEventArgs e)
Error handler method which simply prints the error message.
ROSBridgeMsg¶
-
class
ROSBridgeLib::
ROSBridgeMsg
¶ This (mostly empty) class is the parent class for all RosBridgeMsg’s (the actual message) from ROS.
As the message can be empty....
This could be omitted I suppose, but it is retained here as (i) it nicely parallels the ROSBRidgePacket class which encapsulates the top of the ROSBridge messages which are not empty, and (ii) someday ROS may actually define a minimal message.
Version History 3.1 - changed methods to start with an upper case letter to be more consistent with c# style. 3.0 - modification from hand crafted version 2.0
- Author
- Michael Jenkin, Robert Codd-Downey and Andrew Speers
- Version
- 3.1
Subclassed by ROSBridgeLib.custom_msgs.DebugMsg, ROSBridgeLib.custom_msgs.DurationMsg, ROSBridgeLib.custom_msgs.ErrorMsg, ROSBridgeLib.custom_msgs.ExternalForceMsg, ROSBridgeLib.custom_msgs.ExternalJointMsg, ROSBridgeLib.custom_msgs.FloatArrayMsg, ROSBridgeLib.custom_msgs.ForceMsg, ROSBridgeLib.custom_msgs.InfoMsg, ROSBridgeLib.custom_msgs.LinkMsg, ROSBridgeLib.custom_msgs.ModelMsg, ROSBridgeLib.custom_msgs.PositionCustomMsg, ROSBridgeLib.custom_msgs.RoboyPoseMsg, ROSBridgeLib.custom_msgs.RoboyPositionMsg, ROSBridgeLib.custom_msgs.StringArrayMsg, ROSBridgeLib.custom_msgs.TendonInitializationMsg, ROSBridgeLib.custom_msgs.TendonUpdateMsg, ROSBridgeLib.custom_msgs.WarningMsg, ROSBridgeLib.geometry_msgs.PointMsg, ROSBridgeLib.geometry_msgs.PoseMsg, ROSBridgeLib.geometry_msgs.QuaternionMsg, ROSBridgeLib.geometry_msgs.TwistMsg, ROSBridgeLib.geometry_msgs.Vector3Msg, ROSBridgeLib.sensor_msgs.CompressedImageMsg, ROSBridgeLib.sensor_msgs.ImageMsg, ROSBridgeLib.std_msgs.BoolMsg, ROSBridgeLib.std_msgs.ColorRGBAMsg, ROSBridgeLib.std_msgs.HeaderMsg, ROSBridgeLib.std_msgs.Int32Msg, ROSBridgeLib.std_msgs.Int32MultiArrayMsg, ROSBridgeLib.std_msgs.Int64Msg, ROSBridgeLib.std_msgs.Int64MultiArrayMsg, ROSBridgeLib.std_msgs.Int8Msg, ROSBridgeLib.std_msgs.Int8MultiArrayMsg, ROSBridgeLib.std_msgs.MultiArrayDimensionMsg, ROSBridgeLib.std_msgs.MultiArrayLayoutMsg, ROSBridgeLib.std_msgs.StringMsg, ROSBridgeLib.std_msgs.TimeMsg, ROSBridgeLib.std_msgs.UInt16Msg, ROSBridgeLib.std_msgs.UInt16MultiArrayMsg, ROSBridgeLib.std_msgs.UInt32Msg, ROSBridgeLib.std_msgs.UInt32MultiArrayMsg, ROSBridgeLib.std_msgs.Uint64Msg, ROSBridgeLib.std_msgs.UInt64MultiArrayMsg, ROSBridgeLib.std_msgs.UInt8Msg, ROSBridgeLib.std_msgs.UInt8MultiArrayMsg, ROSBridgeLib.turtlesim.ColorMsg, ROSBridgeLib.turtlesim.PoseMsg, ROSBridgeLib.turtlesim.VelocityMsg
As every type of ROSBridgeMsg should derive from this class, here is an example how an actual implementation looks like.
-
class
ROSBridgeLib::turtlesim::
PoseMsg
¶ Define a turtle pose message.
This has been hand-crafted from the corresponding turtle message file.
Version History 3.1 - changed methods to start with an upper case letter to be more consistent with c# style. 3.0 - modification from hand crafted version 2.0
Inherits from ROSBridgeLib.ROSBridgeMsg
Public Functions
-
ROSBridgeLib.turtlesim.PoseMsg.PoseMsg(JSONNode msg)
This constructor is called when you receive a message from the ROSBridge.
- Parameters
msg
-
ROSBridgeLib.turtlesim.PoseMsg.PoseMsg(float x, float y, float theta, float linear_velocity, float angular_velocity)
This constuctor can be used to construct a message in Unity and send it over the ROSBridge.
- Parameters
x
y
theta
linear_velocity
angular_velocity
-
override string ROSBridgeLib.turtlesim.PoseMsg.ToYAMLString()
You need this function to send a message over the ROSBridge to the desired ROS node as YAML is the standard format for this.
- Return
Public Static Functions
-
static string ROSBridgeLib.turtlesim.PoseMsg.GetMessageType()
This is called when you send the message over the ROSBridge.
It must be equal to the type of the input of the receiving node.
- Return
-
Managers¶
RoboyManager¶
-
class
RoboyManager
¶ Roboymanager has the task to adjust roboys state depending on the ROS messages.
In summary it does the following:
-# receive pose messages to adjust roboy pose. -# subscribe to the external force event and forward the message to the simulation. -# send a service call for a world reset. -# FUTURE: receive motor msg and forward it to the according motors.
Inherits from Singleton< RoboyManager >
Public Functions
-
void RoboyManager.InitializeRoboyParts()
Initializes the roboy parts with a random count of motors => WILL BE CHANGED IN THE FUTURE, for now just a template
-
void RoboyManager.ReceiveMessage(RoboyPoseMsg msg)
Main function to receive messages from ROSBridge.
Adjusts the roboy pose and the motors values (future).
- Parameters
msg
: JSON msg containing roboy pose.
-
void RoboyManager.ReceiveExternalForce(RoboyPart roboyPart, Vector3 position, Vector3 force, int duration)
Sends a message to the simulation to apply an external force at a certain position.
- Parameters
roboyPart
: The roboypart where the force should be applied.position
: The relative position of the force to the roboypart.force
: The direction and the amount of force relative to roboypart.duration
: The duration for which the force should be applied.
Property
-
property
RoboyManager::Roboy
Public variable so that all classes can access the roboy object.
-
property
RoboyManager::RoboyParts
Public variable for the dictionary with all roboyparts, used to adjust pose and motor values
Private Functions
-
void RoboyManager.Awake()
Initialize ROSBridge and roboy parts
-
void RoboyManager.Update()
Run ROSBridge
-
void RoboyManager.drawTendons()
Test function to draw tendons.
For now draws only random lines. TEMPLATE!
-
void RoboyManager.adjustPose(string name, RoboyPoseMsg msg)
Adjusts roboy pose for all parts with the values from the simulation.
- Parameters
msg
: JSON msg containing the roboy pose.
-
void RoboyManager.getRoboy()
Searches for roboy via the “Roboy” tag.
-
void RoboyManager.getRoboyParts(string name)
Searches for roboy and all roboy parts.
Private Members
-
Transform RoboyManager.m_Roboy
Transform of roboy with all roboy parts as child objects
-
RoboyPoseMsg RoboyManager.m_RoboyPoseMessage
Pose message of roboy in ou r build in class
-
Dictionary<string, RoboyPart> RoboyManager.m_RoboyParts
= new Dictionary<string, RoboyPart>() Dictionary with all roboyparts, used to adjust pose and motor values
-
InputManager¶
-
class
InputManager
¶ InputManager holds a reference of every tool.
On top of that it listens to button events from these tools and forwards touchpad input to the respective classes.
Inherits from Singleton< InputManager >
Public Types
Public Functions
-
void InputManager.Initialize(List< ControllerTool > toolList)
Initialize all tools.
-
void InputManager.OnChangeGUITool(object sender, ClickedEventArgs e)
Changes view mode when the user presses the side button on the controller.
- Parameters
sender
e
-
void InputManager.OnChangeTool(object sender, ClickedEventArgs e)
Changes the tool when the user presses the side button on the controller.
- Parameters
sender
e
-
void InputManager.GetTouchpadInput(object sender, ClickedEventArgs e)
Retrives the touchpad input of the tool controller and updates the values.
- Parameters
sender
e
Property
-
property
InputManager::GUI_Controller
Public GUIController reference.
-
property
InputManager::View_Controller
Public ViewController reference.
-
property
InputManager::ModelSpawn_Controller
Public ModelSpawnController reference.
-
property
InputManager::Selector_Tool
Public SelectorTool reference.
-
property
InputManager::ShootingTool
Public ShootingTool reference.
-
property
InputManager::TimeTool
Public TimeTool reference.
-
property
InputManager::SelectorTool_TouchpadStatus
Touchpad status of the controller where selector tool is attached to.
-
property
InputManager::GUIController_TouchpadStatus
Touchpad status of the controller where gui controller tool is attached to.
Private Functions
-
void InputManager.Update()
Calls the ray cast from the selector tool if it is active.
-
void InputManager.setTools(List< ControllerTool > toolList)
Set all tools depending on their type to the respective variable.
- Parameters
toolList
-
IEnumerator InputManager.initControllersCoroutine()
Initializes all controllers and tools.
- Return
Private Members
-
SelectorTool InputManager.m_SelectorTool
Private SelectorTool reference.
Is serialized so it can be dragged in the editor.
-
ShootingTool InputManager.m_ShootingTool
Private ShootingTool reference.
Is serialized so it can be dragged in the editor.
-
TimeTool InputManager.m_TimeTool
Private TimeTool reference.
Is serialized so it can be dragged in the editor.
-
HandTool InputManager.m_HandTool
Private HandTool reference.
Is serialized so it can be dragged in the editor.
-
GUIController InputManager.m_GUIController
Private GUIController reference.
Is serialized so it can be dragged in the editor.
-
ViewController InputManager.m_ViewController
Private GUIController reference.
Is serialized so it can be dragged in the editor.
-
ModelSpawnController InputManager.m_ModelSpawnController
Private ModelSpawn reference.
Is serialized so it can be dragged in the editor.
-
SelectionWheel InputManager.m_ToolWheel
Selection wheel to select tools.
-
SelectionWheel InputManager.m_GUIWheel
Selection wheel to select different GUI modes.
-
bool InputManager.m_Initialized
= false Controllers initialized or not.
-
ModeManager¶
-
class
ModeManager
¶ ModeManager holds a reference of every active mode and provides function to switch between them.
This includes:
- Current tool: ShootingTool, SelectionTool etc.
- Current view mode: singe vs. comparison
- Current GUI mode: selection vs. GUI panels
- Current panel mode: motorforce, motorvoltage etc.
Inherits from Singleton< ModeManager >
Public Types
-
enum
Viewmode
¶ We change between Single view where we can choose only one objet at a time and comparison view with three maximum objects at a time.
Values:
-
Single
¶
-
Comparison
¶
-
-
enum
Panelmode
¶ Describes the different modes for panel visualization.
Values:
-
Motor_Force
¶
-
Motor_Voltage
¶
-
Motor_Current
¶
-
Energy_Consumption
¶
-
Tendon_Forces
¶
-
Public Functions
-
void ModeManager.ChangeViewMode()
Changes between single and comparison view.
-
void ModeManager.ChangeGUIViewerMode()
Switches between selection and panels GUI mode.
-
void ModeManager.ChangeToolMode()
Switches between all tools.
-
void ModeManager.ChangeToolMode(ToolMode mode)
Changes the tool mode based on the enum to the new one and turns off the old tool.
- Parameters
mode
-
void ModeManager.ChangePanelModeNext()
Changes the panel mode to the next one based on the order in the enum defintion.
-
void ModeManager.ChangePanelModePrevious()
Changes the panel mode to the previous one based on the order in the enum defintion.
-
void ModeManager.ResetPanelMode()
Resets current panel mode to MotorForce.
Property
-
property
ModeManager::CurrentViewmode
Current view mode, READ ONLY.
-
property
ModeManager::CurrentPanelmode
Current panel mode, READ ONLY.
-
property
ModeManager::CurrentGUIViewerMode
Current GUIViewer mode, READ ONLY.
-
property
ModeManager::CurrentToolMode
Current Tool mode, READ ONLY.
-
property
ModeManager::CurrentGUIMode
Current GUI Mode.
READ ONLY.
Private Functions
-
void ModeManager.changeToolStatus(ToolMode tool, bool state)
Changes the tool based on the enum to the new state.
- Parameters
tool
state
Private Members
-
Viewmode ModeManager.m_CurrentViewmode
= Viewmode.Comparison Private variable for current view mode.
-
Panelmode ModeManager.m_CurrentPanelmode
= Panelmode.Motor_Force Private variable for current panel mode.
-
GUIViewerMode ModeManager.m_CurrentGUIViewerMode
= GUIViewerMode.Selection Private variable for current GUIViewer mode.
-
SpawnViewerMode ModeManager.m_CurrentSpawnViewerMode
= SpawnViewerMode.Insert Private variable for the current mode of the model spawn controller.
-
ToolMode ModeManager.m_CurrentToolMode
= ToolMode.SelectorTool Private variable for current Tool mode.
-
GUIMode ModeManager.m_CurrentGUIMode
= GUIMode.GUIViewer Current view mode of the GUI tools
SelectorManager¶
-
class
SelectorManager
¶ SelectorManager is responsible to hold references of all selected roboy parts and the corresponding UI elements.
Inherits from Singleton< SelectorManager >
Public Functions
-
void SelectorManager.AddSelectedObject(SelectableObject obj)
Adds the roboy part to selected objects.
- Parameters
obj
: SelectableObject component of the roboy part.
-
void SelectorManager.RemoveSelectedObject(SelectableObject obj)
Removes the roboy part from the selected objects.
- Parameters
obj
: SelectableObject component of the roboy part.
-
void SelectorManager.ResetSelectedObjects()
Resets all roboy parts to default state and empties the selected objects list.
Public Members
-
int SelectorManager.RoboyUIElementsCount
= 13 TEMPORARY VARIABLE TO CHECK HOW MANY UI ELEMENTS ARE INITIALIZED
Property
-
property
SelectorManager::UI_Elements
Property which returns a dictionary of all UI elements in the SelectionPanel.
-
property
SelectorManager::SelectedParts
Reference of all currently selected roboy parts.
-
property
SelectorManager::MaximumSelectableObjects
Integer to switch between single mode selection and normal mode collection.
Private Functions
-
IEnumerator SelectorManager.Start()
Initializes all variables.
- Return
Private Members
-
Transform SelectorManager.m_Roboy
Transform of roboy model.
-
List<SelectableObject> SelectorManager.m_RoboyParts
= new List<SelectableObject>() List of SelectableObject components of all roboy parts.
-
List<SelectableObject> SelectorManager.m_SelectedParts
= new List<SelectableObject>() List of SelectableObject components of all selected parts.
-
int SelectorManager.m_MaximumSelectableObjects
= 3 Maximum cound of selectable objects in multiple selection mode.
-
int SelectorManager.m_CurrentMaximumSelectedObjects
= 3 Current count of maximum selectable objects.
-
Dictionary<string, GameObject> SelectorManager.m_UI_Elements
= new Dictionary<string, GameObject>() Private reference to all UI elements.
-
BeRoboyManager¶
-
class
BeRoboyManager
¶ BeRoboymanager has different tasks to do:
- Keep track of user movement and translate roboy when in specific view modes
- Convert received images into textures which can then be rendered on screen
- Send tracking messages over the rosbridge to gazebo/ real roboy
Inherits from Singleton< BeRoboyManager >
Public Functions
-
void BeRoboyManager.ReceiveZedMessage(ImageMsg image)
Primary function to receive image (zed) messages from ROSBridge.
Renders the received images.
- Parameters
msg
: JSON msg containing roboy pose.
-
void BeRoboyManager.ReceiveSimMessage(ImageMsg image)
Primary function to receive image (simulation) messages from ROSBridge.
Renders the received images.
- Parameters
msg
: JSON msg containing roboy pose.
-
void BeRoboyManager.ReceiveExternalJoint(List< string > jointNames, List< float > angles)
Function to publish ExternalJoint messages via ROS.
- Parameters
jointNames
angles
-
void BeRoboyManager.ReceivePosition(Vector3 pos)
Function to publish Position messages via ROS.
- Parameters
pos
Public Members
-
bool BeRoboyManager.TrackingEnabled
= false Set whether head movement should be tracked or not.
-
RenderTexture BeRoboyManager.RT_Zed
Reference to the render texture in which the Zed feed gets pushed into.
-
RenderTexture BeRoboyManager.RT_Simulation
Reference to the render texture in which the Simulation feed gets pushed into.
Private Functions
-
void BeRoboyManager.Awake()
Initialize textures.
-
void BeRoboyManager.RefreshZedImage(ImageMsg image)
Renders the received images from the zed camera
- Parameters
msg
: JSON msg containing the roboy pose.
-
void BeRoboyManager.RefreshSimImage(ImageMsg image)
Renders the received images from the simulation.
- Parameters
msg
: JSON msg containing the roboy pose.
-
void BeRoboyManager.translateRoboy()
Turn Roboy with the movement of the HMD.
-
void BeRoboyManager.tryInitializeCamera()
Looking for the main camera in the scene, which can be attached to Roboy.
Private Members
-
GameObject BeRoboyManager.m_Cam
The HMD main camera.
-
Texture2D BeRoboyManager.m_TexSim
Texture in which the received simulation images get drawn.
-
Texture2D BeRoboyManager.m_TexZed
Texture in which the received zed images get drawn.
-
bool BeRoboyManager.m_CamInitialized
= false Is the main camera initialized or not.
-
float BeRoboyManager.m_CurrentAngleX
= 0.0f Variable to determine if headset was rotated.
-
float BeRoboyManager.m_CurrentAngleY
= 0.0f Variable to determine if headset was rotated.
-
Color [] BeRoboyManager.m_ColorArraySim
= new Color[640 * 480] Color array for the simulation image conversion.
-
Color [] BeRoboyManager.m_ColorArrayZed
= new Color[1280 * 720] Color array for the zed image conversion.
-
SteamVR_TrackedObject BeRoboyManager.m_Controller
Reference to the tools controller (right).
ViewSelectionManager¶
-
class
ViewSelectionManager
¶ ViewSelectionManager handles the transition between various view scenarios.
Inherits from Singleton< ViewSelectionManager >
Public Functions
-
void ViewSelectionManager.TurnTrackingOn()
Turn head tracking for BeRoboy on.
-
void ViewSelectionManager.TurnTrackingOff()
Turn head tracking for BeRoboy off.
-
void ViewSelectionManager.SwitchToSimulationView()
Switches the view to the simulation view.
-
void ViewSelectionManager.SwitchToZEDView()
Switches the view to the ZED(real roboy camera in the head) view.
-
void ViewSelectionManager.SwitchToObserverView()
Switches the view to the observer view.
-
void ViewSelectionManager.SwitchToBeRoboyView()
Switches the view to the beroboy view.
Public Members
-
Canvas ViewSelectionManager.InstructionCanvas
Reference to the Canvas that is placed on the Camera plane(HMD).
-
Image ViewSelectionManager.BackgroundImage
Reference to the image where intructive text can be displayed.
-
RawImage ViewSelectionManager.GazeboImage
Reference to the image where the simulation feed can be displayed.
-
Image ViewSelectionManager.HtcImage
Reference to the image where the htc feed can be displayed.
-
RawImage ViewSelectionManager.ZedImage
Reference to the image where the zed feed can be displayed.
-
GameObject ViewSelectionManager.Cave
Reference to Roboy’s surrounding.
-
GameObject ViewSelectionManager.Roboy
Reference to Roboy itself.
-
GameObject ViewSelectionManager.Pointer
Reference to the teleporter ray.
-