ArtScan

PWA - Progressive Web App

June, 2025

Google Gemini Vision APINext.jsTypescript

ArtScan is a personalized inventory management tool for art collectors. It automates the process of organizing artwork by identifying images and logging them into a digital collection—eliminating manual data entry and making collection tracking effortless.

How it works:

  1. Upload or capture an image of the artwork.
  2. Gemini Vision API analyzes the image to identify artist and title.
  3. Automatically logs artwork info into a Google Sheet.
  4. Leaves price blank to preserve privacy of acquisition cost.
  5. Accessible as a lightweight PWA for quick mobile use.
ArtScan - How it Works

Implementation Details:

I experimented with multiple image recognition APIs, including SerpAPI and reverse image search tools, before finalizing Google’s Gemini Vision API for its reliability and flexibility. The identified artwork metadata—such as artist name and title—is automatically written into a connected Google Sheet using the Sheets API.

To maintain flexibility in pricing data, the price field is intentionally left blank, allowing collectors to optionally input it later. The entire tool is packaged as a Progressive Web App (PWA) with offline caching and mobile responsiveness for easy access on-the-go.

Insights & Future Improvements:

Through building ArtScan, I gained a deeper appreciation for the challenges of visual recognition in domain-specific contexts like art. While Gemini Vision API performed well for many well-known works, it occasionally struggled with private collections or lesser-known pieces—highlighting the need for fallback strategies and user confirmation flows.

In future iterations, I plan to:

  • Train my own model which learns from updated, recent art sales so that the app performs better on emerging artists
  • Add manual correction and review features for uncertain predictions.
  • Support batch image uploads and metadata editing via spreadsheet sync.
  • Integrate a lightweight provenance tracker to log acquisition history.
  • Explore local model inference for privacy-conscious collectors.
  • Experiment with embeddings for similarity-based artwork grouping.