First Time Using a Visuotactile Sensor: Getting Grasp Tests to Run
Community Discussion · General

First Time Using a Visuotactile Sensor: Getting Grasp Tests to Run

Old Ye from BCGOld Ye from BCGOct 42026/10/04 302 views

Spent the weekend tinkering with a customer's gripper test rig, stepped in plenty of pits. The star is Yimu Technology's biomimetic visuotactile sensor, less than 3mm thick, stuck on the gripper fingertip, used to judge whether the robot has actually gripped something securely.

First, what it is. An ordinary force sensor can only tell you "how much force was used," like how far a spring got compressed. Visuotactile takes another path: a layer of soft gel on the surface, with rows of small dots embedded in the gel, and a tiny camera underneath continuously filming this gel layer. When something presses on it, the gel deforms, the dots shift position, the camera captures the movement, and then an algorithm back-calculates to get pressure, lateral force, and surface texture. You can roughly think of it as fitting the robot finger with an eye hidden under the skin. The lateral force mentioned here is called shear force in the industry—the sideways push when an object is about to slip.

My task this time was simple: have the gripper pinch three kinds of objects—soft, slippery, and textured—and see whether I could read "about to slip" directly from the contact image.

1. Mounting. Optical face outward, facing the object being gripped, adhesive backing against the fingertip. Thin, so it basically doesn't affect the gripper's closing stroke. Wipe the fingertip clean before mounting, push from one side to the other, don't let air get trapped in the middle.

2. Connect the host computer to see the live view. The host computer is the PC connected to the sensor and its software. After opening, you first see a grayscale image with a regularly arranged dot array on it. Don't rush to put anything on at this step. If the image is too dark or the dots blur into a blob, adjust focus and fill light until the dot edges are clear.

3. Capture a no-load baseline image. Gripper empty, gripping nothing, take one and save it. This image is the zero point; all subsequent deformation is compared against it. Change the gripper, change the lighting, overnight temperature changes—you have to retake it. Last time I forgot to retake it and the whole dataset was wasted; rerunning took another half day.

4. Start the press test. Put the object in the gripper, close slowly, watch the dot array. Pressed straight on, the dots expand uniformly outward; with lateral force, the dots shift in one direction. This is what an ordinary force sensor can't see.

5. Collect data, run the solver. Save the images of the whole contact process, hand them to the algorithm to output pressure distribution and shear direction. For the solving part I used the bundled tool, parameters basically default, good enough to see trends first.

The results for the three objects were clearly separated. The slippery one: small contact area, shear reading rises then falls, and that inflection point before the drop is visible to the naked eye. The soft one: contact area spreads fast, dot displacement is also large. The textured one: slight irregular disturbance appears on the dot array, which for now can only be viewed qualitatively.

There's a review that breaks the visuotactile perception pipeline into four steps: contact, deformation, image capture, data solving. After running through it once, I think this breakdown is quite solid. The first two steps are about hardware and environment; the last two are where the real time goes into tuning. There are also papers saying contact surface reconstruction error can be pressed below 100 microns, localized to sub-millimeter level. I don't have the conditions to verify that finely here, but judging from the directionality of dot displacement, it's enough to detect slip.

The pitfalls are mainly in three places. Air bubbles trapped during mounting make the soft gel's local stiffness inconsistent, producing a fake deformation zone on the image that looks like it's under force but isn't. Ambient light drift changes dot contrast, and the solving drifts with it, so try to fix the light source. The third is the most easily overlooked and most expensive: not retaking the baseline image. The first two are hardware and environment; this one is process discipline.

After getting single-finger working, the next step can be trying symmetric dual-finger capture, feeding the contact signal into the grasping process for closed loop—add force as soon as the slip inflection point appears. Or compare real-machine data on hand against simulated contact data to see where the difference lies. For the simulation gap in the contact area, from what I've seen recently, it mainly gets stuck on modeling the soft gel.

2 replies

?
Ctrl + Enter to reply
I am Dapeng

I think the thing about the slip inflection point being visible to the naked eye is that if you make it into a product, you have to let the customer see it with their own eyes, otherwise they won't believe it.

Long Ji
Long JiOct 5
Reply to I am Dapeng

I actually tried it out. In the grayscale image, the dot offset direction is visible to the naked eye, but for a customer-facing interface, you have to mark the inflection point of the shear reading directly, otherwise they really won't believe it.