Archive Notes

Week 10. Human-Robot Interaction

2 min read

Human-Robot Interaction (HRI) is the study of interactions between humans and robots. There are different types of HRI-related robotics:

  • Collaborative Robotics: human and robot share a goal, cooperate to achieve it
  • Social Robotics: robots that communicate socially with humans in their environment
  • Assistive Robotics: provide assistance to users in need
    • Physically Assistive Robots perform physical tasks that involve physical contact
    • Socially Assistive Robots provide assistance to human users through non-physical social interactions
  • Teleoperated Robots: low autonomy, remote controlled robots which are usually controlled from a distance; e.g. telepresence, rescue, bomb-defusal, nuclear decommissioning

Required skills for HRI include:

  • vision processing: recognition of user, body pose, gesture, expressions
  • audio processing: speech synthesis and recognition
  • physical interaction: tactile sensors

Speech Recognition

Speech recognition is the process of extracting the textual representation of a human’s speech from voice.

For the following, must allow container access to audio:

singularity run --bind /run/user itr.sif

Install and use a package:

# dependencies
paru -S python-vosk-bin python-pyttsx3 python-sounddevice
 
cd ros_ws
pushd src
 
# clone vosk
git clone https://github.com/alphacep/ros-vosk
 
# or clone hmi_robin
git clone https://github.com/gerardcanal/hmi_robin.git --branch=kinetic-devel
 
# build everything
popd
catkin build

Using vosk:

# start vosk node
roslaunch ros_vosk ros_vosk.launch
 
# read recognised text
rostopic echo /vosk_engine/final_result
rostopic echo /vosk_engine/partial_result
 
# final result includes both at once:
rostopic echo /vosk_engine/final_result

Speech Synthesis

Speech synthesis is the process of converting text into speech that a human user can understand.

vosk also supports TTS:

# ensure vosk is running first
rostopic pub /tts/phrase std_msgs/String "data: 'this is a test'"

Or use Google Translate API using hmi_robin:

# launch db
roslaunch speech_database speech.launch
 
# publish TTS data
rostopic pub /speech std_msgs/String "data: 'this is a test'"

Graph View