A machine vision algorithm to recognize eggplant fruit and estimate its size for selective robotic harvesting was developed and tested. Images acquired in both outdoor and indoor conditions were examined. The algorithm was designed for a two-step operation. The first step was a simple color segmentation to distinguish low intensity for speeding and the second step was a vertical-dividing operation using two templates to eliminate noise such as stems and leaves. The vertical-dividing operation could exclude 60-pixels-length object and cope with the fruit angle up to approximately 20°.The algorithm could recognize eggplant fruits accurately at a high ratio from 80.0 to 97.5% with the operating time of approximately l sec. The fruit length and the fruit maximal diameter, which would be indices of selective harvesting, were estimated from geometric features of the fruit binary image. Estimation accuracy of the fruit maximal diameter was varying from 16.0 to 21.5% and higher than the one of the fruit length.