Amazon stow, pick and pack roles are ‘the last bits’ to automate
Just outside Luton, an Amazon fulfilment centre the size of four football pitches moves yellow shelving units along conveyor belts. Tubs holding a Lego Spider-Man kit, dishwasher tablets and a pouch of muscle-building protein powder travel along the line until they reach a person tasked with stacking the unit that has stopped in front of them. Amazon calls the arrangement the “symphony of people and technology” to which it owes its success, Jenny Kleeman reported in a Guardian feature.
Amazon began opening its fulfilment centres to carefully controlled free public tours a decade ago after a barrage of negative publicity about working practices. A guide told visitors about the competitive rates of pay Amazon workers receive, their pensions, dental cover and options for flexible working. “You will see robots and humans working in harmony,” she said, describing what robots are good at as “working without getting tired.”
Almost every judgment call on the floor appeared to be made by a computer, Kleeman wrote: the system shines a light on the specific spot on a specific shelf where a worker should place each item, chooses which size box an order is packed in and determines how much tape seals it. “The human beings are here to provide the hands, not the brains,” she wrote. From what she could see, it seemed that if a task could be done more efficiently by a machine, Amazon would rather leave it to a robot.
No hardware or software yet rivals a human hand, according to the feature. Humanoid robots, often called “embodied AI,” have made great strides over the past decade, as evidenced in China’s recent five-day robot Olympics, but producing a machine that can pack a shelf, tighten a bolt and also pick a raspberry has proved surprisingly difficult. Replicating the breadth of function of the human hand is one of the biggest remaining challenges in robotics, Kleeman wrote, and until robots are dexterous, human workers will always be necessary.
Investors are now gambling that humanoid robots will soon become as much a part of everyday life as large language models. China’s government has pledged to invest more than £100bn in robotics and high-tech industries over the next two decades. Earlier this year, Tesla halted production of its Model X and Model S cars so that factories could focus instead on producing Optimus humanoid robots. “Human hands have evolved to be this incredibly sophisticated machine. Your hand is actually a remarkable thing,” Tesla chief executive Elon Musk told a conference last year. “In order to create a robot that can be a generalised humanoid, you must solve the hands problem.”
Amazon employs more than 1.5 million people worldwide today. Kleeman framed the question the feature pursues: what happens to those workers, and to the rest of us, when the hands problem is solved.
Alistair Hunter, a hand and wrist surgeon at University College London hospitals, told Kleeman that the difficulty starts with anatomy. “The hand is our conduit between the brain and the world,” he said. “The hand is our way of turning thought into action.” There are 27 bones in the hand, plus the nerves and muscles that control it, and behind them a brain interacting millisecond by millisecond with what the hand senses. Hunter argued that humanity’s edge comes from the combination of cognitive power and dexterity rather than intelligence alone. The joint at the bottom of the thumb, the trapeziometacarpal joint, is one of evolution’s main advances, enabling a fine pincer grip between thumb, index and middle finger alongside an powerful grasp — the capability that let humans manipulate stones and rocks and, three million years ago, start developing tools.
The brain coordinates larger muscles in the forearm, which supply power and a strong grip, with smaller muscles in the hand that give fine control, and it receives enormous amounts of data in return. “The density of nerve fibres and sensory endings in the hand is enormous,” Hunter said. Specialist receptors and different corpuscle cells tell the brain about vibration, roughness and smoothness, and about how fingers stretch and conform around objects. “It’s that interplay between the sensory side, the motor side and the brain that makes the hand wonderful – and so difficult to replicate.”
The feedback loop is fast. Picking up a raspberry, the fingers register how soft it is, whether it is slippery and how the skin conforms around it; the brain compares that against what it expected and eases the compression if the fruit seems to be crushing. “That feedback loop takes about a tenth of a second,” Hunter said. The brain also predicts the pressure to apply before the grasp — a feed-forward mechanism — which Hunter said explains why a baby makes a mess before learning it. That combination of feedback and feed-forward, learned from years of interaction with the world, is what makes the hand so hard to copy, and why robot engineers historically built task-specific machines instead.
“I think at some point they’ll be able to do it. We’ve got many, many years ahead of us, and you can see that already robots are able to walk and run,” Hunter said. “But building a computer that is able to predict the outside world and also deliver the right amount of sensory information back – that’s going to be difficult.”
Shadow Robot, Europe’s premier manufacturer of dexterous hands, began in the 1980s as a collective of hobbyists and robot enthusiasts meeting in a north London attic to tinker with scavenged materials, hoping one day to build a full humanoid. The company was founded after it received its first commercial order, for a leg, in 1997, and for roughly the last decade its focus has been hands. Its early biped could stand and right itself when shoved and now sits in the Science Museum; Walker estimated that Honda had spent in the order of $100m on the programme that produced its P1 humanoid, which could walk. Shadow decided a useful robot would need to pick up and interact with the same objects a person can.
In the demo room, Kleeman handled Shadow Dexterous Hands mounted on two motorised arms — made of MDF and nylon, with textured magnetic sensors on the fingertips, and a grip she found gentle and natural. A hand starts at £20,000 without any software or operating system; the full system costs £100,000. Each hand weighs roughly 4.5kg. Liem Tran, Shadow’s robotics deployment and support manager, operates them wearing gloves with sensors that track his movements, which the hands copy in real time. The teleoperation doubles as training, giving the software a reference point for what a successful motion looks like so machine learning can later reproduce it without a human. Shadow builds the hardware and basic systems; customers in industry, manufacturing and academic research typically develop their own software.
“We don’t have anything that’s industry standardised, or anything that’s fully developed and ready for use,” Tran said. “It’s exciting, because we’re essentially dictating what happens next with the tech. I like the idea of replicating a human in machine form and seeing where it takes us … Do you want a grape?” The left hand pulled a bunch of white grapes from a brown paper bag and held the stalk in a pincer movement between thumb and index finger; the right hand plucked a grape and dropped it into Kleeman’s palm, unsquashed. The hands also stacked plastic cups into a pyramid, built a tower seven blocks high on a wooden cube held in the other hand, and sorted stars, cylinders and triangles into a children’s shape sorter.
The demonstration also exposed limits. The Shadow Dexterous Hands cannot generate anything like the amount of sensory data a human hand produces, and their learning model depends on a human showing them what to do. Shadow’s DEX-EE, created for Google DeepMind and first unveiled in 2024, takes a different approach: three digits that function like thumbs, no wrist and optical sensors on each fingertip, giving it a clawed appearance and little facility with fine objects. It can absorb a baseball bat, and at £60,000 per hand Kleeman did not swing hard. “It can take a beating,” Tran said. “That’s its whole purpose: it can go through hours and hours of collisions with the work surface.” The design is meant to withstand the repeated mistakes of machine learning, “the same way that a baby learns to use their hands and fingers.” After experimenting with the DEX-EE, DeepMind has developed robots that can tie shoelaces, put a shirt on a hanger and perform repairs on other robots.
Tran disputed the suggestion that manual labour is safe for long. “Around the world, so many companies are now making all these dexterous hands. There’s a huge race for the smoothest and most human-like. They’re getting quite impressive,” he said, noting that some hands in China cost less than £2,000 — cheap enough that “you don’t have to worry about repair, just buy another hand.” Of the timeline, he said: “I don’t think we’re too far off.”
Walker, Shadow’s director, said the company never set out to compete in that fight. “We never set up this organisation to make money out of building robots. It was much more, ‘Let’s build robots! How do we do that?’” he said. “Making some money on the way turns out to be a good way to make sure that you can build the robots.” Developing the first prototypes, the team pored over anatomy textbooks, envious of natural hardware. “If we had those materials, it would be great,” he said. “Self-lubricating joints. Skin that regrows. We don’t have those, so we have to use other mechanisms and solutions.”
The commercial focus narrowed to hands that could fulfil specific tasks. “We’re working with a wide range of people, everything from applications in nuclear decommissioning through to medical technology and care robotics and production line stuff. A really wide range of areas where people have said, ‘Could we use a robot here?’ or, ‘Could we improve our robots with things like human hands?’” Asked what he wants the robots used for, Walker paused before answering. “We want to build robots for good. At the moment, we’re looking at robots that can do tedious things that get in the way of doing something else,” he said, giving the example of cleaning up the workbench of a highly skilled engineer. The company has turned down clients and jobs: it will not work for arms manufacturers, and after discussion the team declined a project that would debone carcasses and another that involved filling sausage casings with meat. “It’s just the ick factor sometimes.” Walker is vegan.
The limit on that restraint is the size of the field. “Ten years ago, there were four or maybe five companies making robot hands, and you knew all of them, and you knew everything about all of them. And now it’s just …” Walker said, throwing his hands up. “I had a Chinese analyst firm contact me a year and a half ago, and they were like, ‘We have 110 robot hands on our list.’” San Diego-based Psyonic, which specialises in prosthetic hands for amputees, has branched into robot hands, billing its products as the “future of automation” with “human-level stability.” Singapore-based Sharpa boasts “human-like dexterity” for complex tasks such as “screwing, clipping [and] fastening.” Boston Dynamics’ Atlas humanoid is not yet on the market but can use nail guns and drills, deftly changing a fine drill bit in a demonstration video. Chinese state media valued the domestic dexterous hand industry last year at more than 50bn yuan, or £5.6bn.
Walker resists the language of competition, seeing the field as a shared effort. “There will be breakthroughs that suddenly just move everything forward. That might be a better motor, it might be a polymer that makes a skin that is better, it might be a way of sensing the world that lets the robot understand things in a way that we haven’t been able to do before at scale in a cheap robot,” he said. “It’s exciting.”
Lepora, of the University of Bristol, described a far more crowded market. “Shadow is in a difficult position,” he said. “The Shadow Hand was best in class for 20 years, but it’s the price of a Lamborghini. Now there are a lot of very capable hands for the price of a Vauxhall. China is going through a massive expansion of hand development, and at the beginning of this decade you would never have predicted the level of dexterity that we’re seeing today.” On how close the field is to a hand that rivals a human’s, he said that is not the point: “We don’t necessarily need the robots to do all that hands can do. We want them to be able to pick up and manipulate objects and tools.” The current crop is close, he added. “We’ve got a good chance of seeing general tool use by the end of this decade.”
For the Amazon workers Kleeman saw, that timeline is short. The fulfilment-centre staff stow, pick and pack a wide variety of objects and are not required to use tools, which is where Lepora expects automation to land. “Their days are numbered,” he said. “They are the last bits that haven’t yet been automated.” Amazon must already be working on replacing them, he added: “I’m sure they are close to it.” Kleeman noted that China already runs fully automated “dark factories,” production lines that operate around the clock with the lights off.
Skilled trades draw divergent forecasts. Asked about retraining as a plumber, Lepora said: “Maybe for 10 or 15 years. Plumbing’s not going to be done by a robot this decade, that’s for sure. But after then?” Walker argued that plumbers are secure for the foreseeable future because the work demands adaptability across unusual spaces and problems rather than a fixed procedure, and a dexterous hand alone will not take the job. “If you’ve ever seen a plumber get under your bath and do something to the tap at the back, you’ll realise that’s going to be a job for a humanoid robot at best.”
Walker also pushed back on the idea that his work points toward human obsolescence. If robots mastered every task on Shadow’s stated list — jobs too dull, dirty, difficult, dexterous, dangerous, and/or dear for people — he said it would take “several hundred years’ work for several hundred organisations and several hundred major breakthroughs.” Asked whether the work could lead to the doom of humanity, he said: “That sounds like a marketing campaign. If I wanted to increase my share value, I’d probably say, yes, we’re going to lead to the doom of humanity – invest in us.”
Hunter, the surgeon, doubted a machine could match his profession’s demands. “When I’m operating, there are times where I’m at the very limit of my manual dexterity and my understanding. Is it possible for a robot to learn that amount of experience that I’ve developed over many years and then action it autonomously? I think that would be very difficult,” he said. “I don’t think you can ever say ‘impossible’, because 50 years ago I wouldn’t have anticipated anything of what’s possible now. But knowing what I do about the hand, and knowing what I do about surgery, I think that would be a very difficult thing to do.”
Lepora expects the next stage to matter as much as the first: robot hands making robot hands. “I don’t think any company is at that pivot point yet, but that’s where we’re ending up,” he said. “You could produce them at scale, but you could also automate the improvement of them as well. Once they can build them themselves, you iterate it. That is the possible tipping point for when the hands will become really, really capable.” It is also the point, Kleeman wrote, where the robot engineers themselves might be out of a job.