Backflips are easy, stairs are hard: Robots still struggle with simple human movements, experts say

Yet speakers at the Fortune Brainstorm AI conference on Tuesday warned against getting too dazzled by the acrobatic feats. A robot doing a backflip–something difficult for a person–looks impressive. But ask a robot to perform seemingly easy tasks, say, climbing up stairs or grabbing a glass of water, and many of todays droids still struggle.

“What looks hard is easy, but what looks easy is really hard,” Stephanie Zhan, a partner at Sequoia Capital, explained, paraphrasing an observation from computer scientist Hans Moravec. In the late Eighties, Moravec and other computer scientists noted that it was easier for computers to perform well on tests of intelligence, yet failed at tasks that even young children could do.

Deepak Pathak, CEO of robotics startup Skild AI, explained that robots, and computers in general, were good at doing complex tasks when operating in a controlled environment. Showing a video of a Skild robot skipping down a sidewalk, Pathak noted that “apart from the ground, the robot is not interacting with anything.”

Yet for tasks like picking up a bottle or walking up stairs, a person is using vision to “continuously correct” what he or she is doing, Pathak explains. “That interaction is the root reason for human general intelligence, which you don’t appreciate because almost every human has it.”

Zhan explained that viral videos of humanoid robots don’t show how the product was trained, nor whether it can operate in an uncontrolled environment. “The challenge for you as a consumer of all these videos is to really discern what’s real and what’s not,” she said.

The next step for robots

Still, both speakers were optimistic that advances in general intelligence will soon lead to more advanced and flexible robots.

“Robots used to be driven more by human intelligence. Somebody super smart would look at [a task], and…pre-program the robot mathematically to do it,” Pathak said. 

But now, the robotics field is shifting from “programming something to learning from experience,” he explained. This allows for new robots that handle more complex tasks in more uncontrolled environments, and which can easily be adapted for other tasks without the cost of reprogramming and retooling them. 

Today’s robotics firms are “still constrained by having robots that are only built for specific things,” Zhan argued. A robotics platform with more general intelligence can open up “possibilities that are otherwise not possible for us to achieve,” including tasks that are currently dangerous for human workers.

Consumers could benefit too. “You see all these household robots, but they’re only capable of doing one thing,” Zhan said. “But if we succeed at building general intelligent robots, you will finally have consumer robots that can tackle the whole host of household tasks that you now have.” A similar point was made earlier at Brainstorm AI by Qualcomm CEO Rene Haas, who said that the general adaptability of humanoid robots will make them much better suited for factory jobs than the robotics arms used today.

There are social repercussions to a robotics boom, dislodging jobs that, as of now, still needed to be done by humans. Yet Pathak was sanguine about the social benefits of spreading automation. One is safety, as robots remove the need for humans to do jobs that are hazardous or unhealthy in the long-run. Another benefit is filling the massive labor shortage for blue-collar and manufacturing jobs. (That shortfall has been a barrier to U.S. efforts to re-shore much of its advanced manufacturing from Asian economies.)

Yet Pathak also envisioned a future where robots free humans from the drudgery of everyday work, even as he admitted that societies needed to figure out how to spread the gains from automation. “There lies a scenario, a good scenario, where everybody is doing things that they like,” Pathak said. “Work is more optional, and they are doing things that they enjoy.”