GrapeSAM

ViViD-5K: Vineyard vision dataset for field-based berry detection and segmentation and grape cluster closure estimation

arXiv preprint · May 2026

Xiangzhi Tong, Chengrui Zhang, Mac Flaherty, Andre Matteo Garcia, Dominic Gorman, Jonathan Jaramillo, Justine E. Vanden Heuvel, Yu Jiang

Cornell University

Three stages on the same vineyard photo: grape clusters segmented as whole instance masks; every berry segmented in a different color; and the extracted berry masks alone on a gray background.
The GrapeSAM pipeline on a field image. Cluster instance segmentation (left), SAM-prompted per-berry segmentation (center), and the extracted berry masks (right) from which visual cluster closure is estimated.

Abstract

Cluster closure, defined as the progressive filling of gaps between the berries in a grape bunch, is a key trait in vineyard management, impacting disease risk. However, traditional visual scoring methods are labor-intensive, subjective, and lack temporal resolution. Existing datasets rarely support fine-grained berry-level analysis, limiting the development of robust deep learning models. In this work, we present ViViD-5k, a large-scale in-field Vineyard Vision Dataset containing 5,000 images with dense annotations, including over 648,000 berry centroids and cluster segmentation masks spanning 13 grape varieties. Building on this dataset, we introduce GrapeSAM, a two-stage visual pipeline that combines point-based berry localization with prompt-based segmentation using Segment Anything, followed by transformer-based cluster segmentation. The pipeline enables automated, in-field estimation of cluster closure with minimal supervision. Quantitative results demonstrate strong segmentation and counting accuracy across diverse conditions, while visualizations confirm robustness on both in-domain and out-of-domain samples. This work provides a scalable and objective alternative to manual compactness scoring and supports high-throughput grape phenotyping with enhanced spatial detail.

How GrapeSAM works

  1. Locate every berry. A point-localization network predicts one centroid per berry, giving a count and a set of spatial prompts.
  2. Segment berries with SAM. The predicted points prompt the Segment Anything Model, turning each centroid into a precise berry mask.
  3. Segment the cluster. A Mask2Former model outlines each grape cluster as a whole, and comparing berry coverage against the cluster outline yields a visual cluster-closure estimate.
A grape cluster on the vine covered by a single blue instance segmentation mask labeled 0.
Cluster segmentation. Mask2Former predicts one instance mask per cluster.
The same grape cluster with every individual berry segmented in a different bright color.
Berry segmentation & counting. Point prompts turn every berry into its own SAM mask.

Results across varieties

Trained once on ViViD-5K, the pipeline holds up across berry colors, cluster shapes, and lighting — from tight dark-skinned bunches to loose green ones.

Fourteen vineyard photos across seven grape varieties: the top row shows whole grape clusters segmented as single colored instance masks; the bottom row shows every individual berry segmented in a different color.
Qualitative results on seven of the thirteen ViViD-5K grape varieties: cluster instance masks (top row) and per-berry segmentation (bottom row) in commercial vineyards.

The ViViD-5K dataset

5,000Field images
648,000+Berry centroids
13Grape varieties

ViViD-5K pairs in-field vineyard photos with dense annotations — berry centroids and cluster instance masks — collected across growing conditions and varieties. It is openly available on Hugging Face and Google Drive under CC-BY-4.0, and the full pipeline code is on GitHub under Apache-2.0.

Citation

@article{tong2026vivid5k,
  title={ViViD-5K: Vineyard vision dataset for field-based berry detection
         and segmentation and grape cluster closure estimation},
  author={Tong, Xiangzhi and Zhang, Chengrui and Flaherty, Mac and
          Garcia, Andre Matteo and Gorman, Dominic and Jaramillo, Jonathan
          and Vanden Heuvel, Justine E. and Jiang, Yu},
  journal={arXiv preprint arXiv:2605.24353},
  year={2026}
}