	In Browser
	StumbleUpon
	del.icio.us
	Google
	Google Buzz
	reddit
	LinkedIn

	Facebook
	Twitter
	Linkedin
	E-Mail

Generative AI > Google Gemini API > Gemini API Image Understanding and Analysis

Gemini API Image Understanding and Analysis

Author: Venkata Sudhakar

The Gemini API processes images natively alongside text, enabling a new class of automation in retail and manufacturing. You can analyse product photos for quality defects, extract text from labels and packaging, compare product images for consistency, and generate detailed product descriptions automatically - all without any computer vision model training.

ShopMax India uses Gemini image analysis at two key points in its workflow. Sellers uploading products have their images automatically checked for quality and compliance before listing. The warehouse team photographs inbound stock and Gemini detects damaged packaging, verifies quantity counts, and reads barcode labels even when partially obscured. This has eliminated two full-time manual inspection roles.

The below example shows how to analyse a product image using the Gemini API and extract structured metadata for the product catalogue.

import vertexai
from vertexai.generative_models import GenerativeModel, Part, Image
import json

vertexai.init(project="shopmax-india", location="asia-south1")
model = GenerativeModel("gemini-2.0-flash")

def analyse_product_image(image_path: str, category: str) -> dict:
    image = Image.load_from_file(image_path)

prompt = f"""Analyse this {category} product image for ShopMax India.
Return ONLY a JSON object with:
- product_name: best guess at product name and model
- brand: brand name if visible
- color: primary color(s)
- condition: new / used / damaged
- image_quality: good / acceptable / poor
- quality_issues: list of issues (e.g. blurry, poor_lighting, background_cluttered)
- visible_text: list of text visible on product or packaging
- description: 2-sentence product description for listing
- listing_approved: true if image meets quality standards, false if not
- rejection_reason: reason if listing_approved is false, else null
"""

response = model.generate_content(
        [image, prompt],
        generation_config={"temperature": 0.1, "response_mime_type": "application/json"}
    )
    return json.loads(response.text)

# Analyse a seller-uploaded product image
result = analyse_product_image("product_upload.jpg", "headphones")

print(f"Product: {result['product_name']}")
print(f"Brand: {result['brand']}")
print(f"Condition: {result['condition']}")
print(f"Image quality: {result['image_quality']}")
print(f"Approved: {result['listing_approved']}")
if not result["listing_approved"]:
    print(f"Rejection reason: {result['rejection_reason']}")
print(f"Description: {result['description']}")

It gives the following output,

Product: Sony WH-1000XM5 Wireless Noise Cancelling Headphones
Brand: Sony
Condition: new
Image quality: good
Approved: True
Description: Sony WH-1000XM5 over-ear wireless headphones in black with
  premium noise cancellation. Shown with original packaging and accessories.

The below example shows how to use Gemini to compare two product images and detect whether they show the same item - useful for duplicate listing detection.

def compare_product_images(image_path1: str, image_path2: str) -> dict:
    img1 = Image.load_from_file(image_path1)
    img2 = Image.load_from_file(image_path2)

prompt = """Compare these two product images.
Return ONLY a JSON object with:
- same_product: true if they appear to show the same product model
- confidence: high / medium / low
- differences: list of observed differences between the two images
- duplicate_risk: high / medium / low (risk these are duplicate listings)
- recommendation: approve_both / flag_for_review / reject_duplicate
"""

response = model.generate_content(
        [img1, img2, prompt],
        generation_config={"temperature": 0.1, "response_mime_type": "application/json"}
    )
    return json.loads(response.text)

comparison = compare_product_images("listing_a.jpg", "listing_b.jpg")
print(f"Same product: {comparison['same_product']} ({comparison['confidence']} confidence)")
print(f"Duplicate risk: {comparison['duplicate_risk']}")
print(f"Recommendation: {comparison['recommendation']}")
if comparison["differences"]:
    print(f"Differences: {comparison['differences']}")

It gives the following output,

Same product: True (high confidence)
Duplicate risk: high
Recommendation: flag_for_review
Differences: ['Different background color', 'Image 2 shows side angle']

Gemini multimodal analysis eliminates the need for separate computer vision pipelines for product image quality checks. ShopMax India runs all seller image uploads through the analyse_product_image function before they enter the catalogue pipeline - images failing quality standards are automatically returned to the seller with specific feedback on what to fix. The duplicate detection function runs nightly across all active listings, catching sellers who attempt to list the same product multiple times to game search ranking.

Send your comments, suggestions or queries regarding this site to [email protected].