Smarter tomato-picking robots learn to judge each fruit before harvest

Tomato vines can look calm from a distance. Up close, they feel like a crowded maze. Fruit hangs in clusters. Stems twist in odd angles. Leaves hide what you need to see. For farmers facing labor shortages, that clutter has become a growing problem, because harvesting still depends heavily on human hands.

Robots could help, but tomatoes are not an easy test. A machine must pick ripe fruit and leave the rest. It also needs to avoid damaging the plant. That requires more than spotting tomatoes. It requires judgment.

Osaka Metropolitan University Assistant Professor Takuya Fujinaga thinks the key is teaching robots to pause and assess. Instead of asking whether a robot can grab a tomato, he wants the robot to estimate how likely a good pick will be, before it commits.

Tomatoes rarely grow one by one. They often form clusters, with fruit tucked behind stems or other tomatoes. A robot might see a ripe tomato, yet fail to reach it cleanly. The gripper may bump a stem. The arm may collide with leaves. A front approach may look safe, then fail at the last second.

For a robot, a failed attempt costs time. It also risks bruising fruit or stressing the plant. That is why decision-making matters. Picking is not only about detecting red fruit. It is also about choosing the best angle.

Fujinaga focused on a practical question: how can a robot judge the “ease of harvesting” for each tomato? His approach treats harvesting like a probability problem, not a simple yes or no task.

Fujinaga programmed robots to evaluate how easy each fruit would be to harvest. His model combines image recognition with statistical analysis. Together, they guide the robot toward the smartest approach direction.

The system starts with image processing. It looks at the tomato, the stems and nearby leaves. It also checks whether the fruit is hidden behind another part of the plant. Those details matter, because obstacles can block a gripper even when fruit looks reachable.

The model then estimates the best direction to approach the tomato. Instead of forcing the robot into a single path, it encourages the robot to choose the path with the highest chance of success.

Fujinaga calls this shift “harvest-ease estimation.” “This moves beyond simply asking ‘can a robot pick a tomato?’ to thinking about ‘how likely is a successful pick?’, which is more meaningful for real-world farming,” he said.

When Fujinaga tested the model, the results stood out. The robot achieved an 81% success rate. That was far higher than the predictions suggested.

The most telling detail came from the “second chance” picks. About a quarter of the successful harvests involved tomatoes that the robot collected from the right or left side after a front approach failed. That pattern suggested the robot did not just try harder. It changed its plan.

"In real terms, it acted more like a person. If you reach for a tomato and a stem blocks your hand, you shift your wrist and come in from the side. Our system pushed robots to do something similar, using visual cues and probability to guide the next move," Fujinaga told The Brighter Side of News.

"Those choices mattered because obstacles vary from fruit to fruit. One tomato might sit clear in open space. Another might hide behind a leaf. Another might sit close to a thick stem. A single fixed approach will not work for all of them," he continued.

Fujinaga’s work highlights how many factors shape robotic harvesting. Fruit clustering matters. Stem geometry matters. Leaves in the background matter. Occlusion matters. Each one can raise or lower the odds of a clean pick.

“This research establishes ‘ease of harvesting’ as a quantitatively evaluable metric, bringing us one step closer to the realization of agricultural robots that can make informed decisions and act intelligently,” he said.

That metric could also change how engineers build future harvesters. If a robot can score a fruit as easy or hard, it can prioritize its time. It can pick the best fruit first. It can avoid repeated failures. It can work faster with less damage.

Fujinaga sees a near-term future where robots and humans split the job. Robots would handle tomatoes that score as easy picks. Humans would focus on the difficult fruit that require flexible hands and quick judgment.

“This is expected to usher in a new form of agriculture where robots and humans collaborate,” he explained. “Robots will automatically harvest tomatoes that are easy to pick, while humans will handle the more challenging fruits.”

That vision speaks to the reality many farms face. Labor shortages keep rising. Growing demand keeps pressure high. Any system that reduces the most repetitive work could help farms stay productive.

For now, the breakthrough is not a robot that replaces people. It is a robot that makes better decisions, fruit by fruit, inside a messy real plant.

By turning “ease of harvesting” into a measurable score, this work gives agricultural robots a clearer way to decide what to pick and how to approach it. That can reduce wasted time from failed attempts and lower the risk of damaging plants and fruit. It can also help farms respond to labor shortages by automating the easiest portion of harvest work.

For researchers, the approach offers a new target for improvement. Instead of only building better tomato detection, teams can build better decision systems that choose angles, avoid obstacles and adapt after failure. Over time, this could lead to smarter robotic harvesters that work across more crops, more growing styles and more greenhouse layouts.

For society, better harvesting automation could support steadier food supply and reduce losses in the field. It may also improve working conditions by shifting people away from the most repetitive picking tasks and toward higher-skill oversight and problem-solving roles.

Research findings are available online in the journal Smart Agricultural Technology.

Like these kind of feel good stories? Get The Brighter Side of News' newsletter.