Frequent drought and heatwaves due to climate change pose significant threats to global wheat yields. Number of grains per spike is an important yield component that shows potential for optimization. A way to increase the number of grains is to select for high fruiting efficiency (number of grains per gram of spike dry weight at anthesis). Currently, there is a lack of high-throughput phenotyping (HTP) methods that estimate fruiting efficiency. With our new sensor head, containing a rigid setup of 13 RGB cameras that captures multiple top-view images from hundreds of different genotypes, we are able to detect and to count spikes.

While spike dry weight is difficult to measure directly, the volume at flowering, the so-called fruiting capacity may be used as proxy for spike weight at anthesis and determines the capacity of the reproductive organ to be filled. Estimating fruiting capacity and efficiency and elaborating their relationship will allow to dissect genotype-specific differences in yield and in their underlying causes. Such insights are crucial to mitigate the impacts of heat and drought under projected climate conditions.

Developing deep learning models requires a solid data set for training and tests. For this work, approximately 1000 spikes from a diverse set of genotypes were labelled in the field, imaged both individually in the field with a smartphone, and at the plot level with the 13 cameras. After harvest, their volume was measured with a 3D scanner. As a proof of concept, volume estimation from RGB images was first tested on the close-up smartphone images using deep learning models and yielded an MAPE of 9.7%.

Results indicate that neural networks show a higher prediction accuracy compared to the baseline model, which fits smoothing splines along the axis and integrates over the splines to get the volume. Furthermore, augmenting the dataset with artificial images can both reduce the prediction error of the neural network compared to using real images only by 3.51 %, and reduce overfitting on the training set.

As an outlook, next steps will involve applying the developed models to predict in-field volumes directly with the multi-view sensor head. The extracted volume estimations over time will allow identifying genotypes with high fruiting capacity that can efficiently fill their potential volume even under stress conditions.

Ultimately, this will allow to achieve high yields despite heat and drought events throughout the season.