Would you like to hear what we've been working on? Then you've come to the right place!

We are developing acoustic sensors for soft actuators, that use sound from within the actuator to detect, localize and overall characterize contact and other interactions.

We believe this to be a very powerful sensorization approach and want to share it with you. So on this page, we want to provide you with all the necessary tools to get started yourself.

The idea of acoustic sensing is to use information contained in sound, to sense the physical state of the sensorized object. We use it, for example, to measure the contact location on soft pneumatic actuators. But we have also used acoustic sensing to determine the force of a contact or to identify the material of the touched object.

This is possible, because sound traveling through any structure, e.g. soft actuators, is modulated slightly depending on the physical state of that structure. We can detect this small change and learn to infer from it the corresponding physical state, e.g. a contact at a specific location.

The great thing about this is that all you need is a microphone and speaker and you're ready to go! In the following, we present what hardware and software we are useing and provide you with simple scripts to get you started. In a few minutes, you can use acoustic sensing yourself to measure contact on a soft actuator. Or you can get more creative and try to sense how full a glass of water is, recognize the type of shoe you're wearing, or attempt to determine the ripeness of a watermelon by tapping it. Get creative!

If you want to get started right away, all you need is microphone and a speaker. You can even use the built-in ones from you laptop. (For example, I used the provided scripts to detect which part of my laptop I was tapping against, using only my laptop's microphone.)

If you want to sensorize objects other than your laptop, you will, however, need external microphones and speakers. Here is the list of components we use to sensorize the PneuFlex actuator:

• Microphone
  • You'll want to use a small microphone that can be easily embedded into whatever structure you want to sensorize. It is very useful if the mic already includes a voltage regulator and returns "line level" signal. Otherwise you will need to add your own voltage regulator and amplifier. Make sure the microphone has a good range, e.g. 100Hz - 10KHz.
  • We use this one: Adafruit Silicon MEMS Microphone Breakout - SPW2430
  • You can get it here: https://www.adafruit.com/product/2716 

• Speaker
  • Like the microphone, the speaker should also be small enough to be embedded. Its operating range should be similar to that of the mic.
  • We use this one: Knowles RAB-32063-000
  • You can get it here: https://mouser.com/Search/Refine?Keyword=RAB-32063-000

• USB Audio Interface
  • The audio interface is used to connect microphone and speaker to the computer. It should have enough inputs and outputs for the number of sensors you want to use in parallel. Optionally, it can be useful if it has a headphone output to check or demonstrate the audio signal.
  • We use this one: ESI Maya 44 USB+
  • You can get it here: https://www.esi-audio.com/products/maya44usb+
  • or here (German shop): https://www.thomann.de/intl/esi_maya_44_usb.htm

We provide example code for the recording of training data, training of the simple sensor model, and live sensing using the trained model.

The code can be found here: https://git.tu-berlin.de/rbo/robotics/acoustic_sensing_starter_kit

If you have any questions or would like to tell us about the cool acoustic sensor application you created, please don't hesitate to contact us!

Vincent Wall

• Gabriel Zöller and Vincent Wall and Oliver Brock. Active Acoustic Contact Sensing for Soft Pneumatic Actuators. Proceedings of the IEEE International Conference on Robotics and Automation (ICRA), 2020.
• Gabriel Zöller and Vincent Wall and Oliver Brock. Acoustic Sensing for Soft Pneumatic Actuators. IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 6986–6991, 2018.