Geometry
Where a blemish lands
The surface that carries each view into the map determines where a blemish lands. Real fruit deviate from a spheroid, so FruitMap measures the surface of each fruit.
Machine vision for postharvest fruit inspection
FruitMap
Pixel-traceable and illumination-normalised skin maps from multi-view RGB photographs.
FruitMap converts a fruit's RGB photographs into one readable skin map for sampling inspection, quality audits, claim handling and training AI graders. Each coloured map pixel records its 3D point, source photograph and pixel, and unobserved skin stays uncoloured and marked.
01 The problem
One fruit, many photographs.
Grading lines photograph fruit turning under fixed cameras and lamps, so the skin's lighting changes. A photograph shows less than half of the skin, and the same blemish appears in several photographs. Per-photograph inspection can count a blemish several times; fusing photographs onto spheroids can misplace it.
Seen in
5 of 16 photographs
On the skin map
1 position
A brown blemish on another red apple from the Fruits-360 turntable sequences. Five of its 16 photographs show it away from the fruit's rim; the skin map shows it once, with its source photograph and pixel.
Assembly
Values with their origin
An inspection map must keep its photographed values together with their origin. Texturing splits the skin into many atlas fragments and levels their colours.
Illumination
Comparable colour
The lamps stay fixed to the camera while the fruit turns, so a map assembled from photographs inherits a patchwork of shading stripes that raise false defect alarms.
02 How it works
From photographs to an inspection record.
The record has four linked parts: the fruit's photographed surface, a planar skin map whose pixels are called texels, the provenance of every photographed texel and a support state for every texel with the three values photographed, surface without photograph and no surface. Scroll to follow the apple above through the pipeline.
-
01
Photographs
Ordinary RGB photographs of one fruit with foreground masks. Here, 16 turntable photographs of an apple from Fruits-360.
-
02
Cameras
FruitMap accepts three camera sources: cameras supplied with the capture, structure-from-motion cameras and the feed-forward network VGGT-Ω, which also predicts per-view depth for the initial surface.
-
03
Dense matches
Consistency-gated correspondence pooling distils millions of dense matches into a compact, spatially balanced set. Each match shows where one skin point appears in two photographs.
-
04
Refined surface
Anchored first-hit refinement fits the surface to these matches where their pixel rays first hit it, so that a blemish lands on the same map position from one photograph to the next. The same matches correct network-predicted cameras.
-
05
One photograph per region
A provenance-preserving labelling gives each observed skin region exactly one photograph and records that source, so provenance holds by construction.
-
06
Readable map
A gated radial unwrapping carries each texel, together with its source pixel, into one readable map. Its texels still inherit the lamp shading of their photographs as vertical stripes.
-
07
Illumination correction
Frame-aware illumination factorisation (FAIF) estimates exposure and shading from skin points seen in at least two photographs, then divides each coloured texel by the lighting of its source photograph.
-
08
Final map
The illumination-normalised map keeps the skin's own colour pattern and detail, and every coloured texel still records its source photograph and pixel.
Three pretrained networks take part: a dense matcher, a feed-forward geometry network and a single-image intrinsic network. Each runs with its published weights, without fine-tuning on fruit, and only in the roles described on this page.
03 Pixel-traceable
Point at the map. Get the pixel.
Every coloured texel, a pixel of the map, is linked to its 3D surface point and to its source photograph and pixel. Move over the map of this apple to follow a coloured texel back to its source photograph and pixel.
Longitude and latitude about the rotation axis. Click to pin a texel; use the arrow keys to move it.
Source photograph· / 16
- Map position
- ·
- Support
- ·
- Source photograph
- ·
- Source pixel
- ·
Source photographs · point at one to show the skin it colours
Grid: uncoloured skin that no photograph shows. How much of a fruit a map covers depends on the capture.
04 Illumination
Divide out the lamps. Keep the skin.
On conveyors and turntables the lamps stay fixed to the camera while the fruit turns. FAIF estimates the lighting from skin points seen in at least two photographs and divides each coloured texel by the lighting of its source photograph, so that texels taken from different photographs can be compared directly.
Single-photograph mapFruitMap
Per-texel correction, log₂(FruitMap / single-photograph map)
Frame test
Which frame do the lamps follow?
A test on held-out photographs decides between lamps fixed to the camera and lamps fixed to the object. It chose correctly for all 71 rotating items and 42 rendered scenes.
Held-out residual of this apple, in grey levels: photographs as taken, the object-frame model, which corrects exposure only, and the camera-frame model, which is selected.
Proposition 1
What a rotation leaves unidentified
We prove that the zonal shading, which depends on latitude alone, is unidentified under a single-axis rotation. FAIF fits the rest of the shading from the photographs, and a learned intrinsic prior supplies only this latitude profile.
Camera-frame shading of the apple over the normals facing the camera; dashed: rotation axis; blue to red: darker to brighter.
Latitude completion
A single latitude profile
The learned prior completes the zonal part with one gain per latitude about the rotation axis.
Latitude gain of this apple against latitude about the rotation axis.
05 Results
112 specimens, 31 species, four sources.
The evaluation covers roller-conveyor sequences with human-annotated skin points, laser-scanned sweet peppers, reflectance-referenced renders of scanned fruit and Fruits-360 turntable sequences. No parameter is set on a test object.
Skin-point transfer, held-out conveyor fruit
97.8%
of annotated skin points transferred between photographs land within 3% of fruit diameter.
Laser reference, held-out peppers
3.0 mm
median distance from laser points on photographed skin to the surface.
Reflectance, camera-fixed lamps, rendered scans
½ the error
FruitMap has the highest fine-detail correlation and half the reflectance error of the next best, learned intrinsic decomposition.
Defect inspection, maps of rendered scans
96%
of scored defects, planted at known positions, are found by a detector, which counts 93% exactly once, whereas per-photograph inspection counts a defect 3.8 times on average.
Lighting frame, held-out photographs
71 + 42
rotating items and rendered scenes, for all of which the frame test chooses correctly between camera-fixed and object-fixed lighting.
Conveyor rotation
36 fruit · 3 species
Roller conveyor, lamps on camera. Reference: annotated skin points.
Fruits-360 turntable
35 items · 24 species
Motorised shaft, lamps on camera. Reference-free: held-out photographs.
Bonn peppers
29 sweet peppers
Camera moving around the fruit. Reference: laser scans.
Scanned fruit renders
12 fruit · 11 species · 7 lightings
16 renders of textured scans. Reference: geometry and reflectance.
06 Gallery
Conveyor and turntable fruit, mapped.
A photograph, the map on the reconstructed surface and the readable skin map. The maps keep real skin marks sharp, such as the lesion and suture of the peach, the lenticels of the pear, the russet of the golden apple and the areoles of the cactus fruit.
07 Paper
FruitMap: Pixel-traceable and illumination-normalised skin maps from multi-view RGB photographs for postharvest fruit inspection
Paper · link to follow
j7du@uwaterloo.ca
Abstract
Skin appearance influences the grade and price of fresh produce. Grading lines photograph fruit turning under fixed cameras and lamps, so the skin's lighting changes. Per-photograph inspection can count a blemish several times; fusing photographs onto spheroids can misplace it. FruitMap converts a fruit's RGB photographs into one readable skin map for sampling inspection, quality audits, claim handling and training AI graders. Each observed region comes from one photograph; each coloured map pixel records its 3D point, source photograph and pixel; unobserved skin stays uncoloured and marked. FruitMap fits the surface and any network-predicted cameras to cycle-consistent matches from a pretrained matcher. A frame-aware illumination factorisation chooses between camera-fixed and object-fixed lighting from held-out photographs, correctly for all 71 rotating items and 42 rendered scenes, and divides each pixel by its photograph's lighting. A learned prior supplies only the latitude-dependent shading that rotation provably leaves unidentified. Evaluation covers 112 specimens of 31 species. On held-out conveyor fruit, 97.8% of annotated skin points transferred between photographs land within 3% of fruit diameter, against at most 83.5% for spheroid and Gaussian-splatting baselines. On held-out peppers, laser points on photographed skin lie a median of 3.0 mm from the surface. Under camera-fixed lamps on rendered scans, FruitMap has the highest fine-detail correlation and half the reflectance error of the next best, learned intrinsic decomposition. On maps of these scans with planted defects, a detector finds 96% of scored defects, counting 93% exactly once, whereas per-photograph inspection counts a defect 3.8 times on average.
BibTeX
@misc{du2026fruitmap,
title = {{FruitMap}: Pixel-traceable and
illumination-normalised skin maps
from multi-view {RGB} photographs
for postharvest fruit inspection},
author = {Du, Jing},
year = {2026},
note = {Manuscript}
}