Remember when we thought AI chatbots were just fancy Q&A machines? Google's Gemini just threw that idea out the window. Their new computer use feature can take control of your browser, click links, fill out forms, and complete tasks for you.
I spent time testing this capability from New Zealand, and the results were eye-opening. Here's what I discovered.
What's Actually Happening Here?
Released recently, the computer use model works like this: Gemini takes a screenshot of your screen, analyses it with multimodal AI, decides the next action, then performs it in your browser. The loop is straightforward:
- Screenshot → AI analysis → Decision → Action
- Repeat until the task is done
Real-World Testing: Price Comparison
Prompt: "Find me which is cheaper - a DJI Action 5 or GoPro Hero 13 on PB Tech." Gemini:
- Navigated to PB Tech and searched for both cameras
- Recorded current pricing
- Returned a clear comparison
In about 3 minutes, it reported GoPro Hero 13 at $569 versus DJI Action 5 at $759 - GoPro wins. The key difference from static chat answers: this used live, current web data.
The Form-Filling Test
Next, I asked it to "Go to the Harkness AI website and submit the contact form for help with my plumbing business." Gemini:
- Found the site via Google
- Navigated to the contact section
- Filled the form with appropriate details
- Wrote a professional message requesting AI help
- Successfully submitted the form
Five minutes later, a legitimate inquiry appeared in my inbox.
The Developer Advantage
Unlike some closed agent modes, Gemini's computer use is developer-friendly. You can integrate this capability directly via API, with Python examples and documentation available, making it a building block for next-gen automated tools.
The Performance Numbers
Google's benchmarks suggest Gemini outperforms comparable models on many tasks and is notably faster. The shorter perception→action cycle makes it practical for real-world use.
The Bigger Picture (And the Concerns)
We're moving from "AI that answers questions" to "AI that takes actions." The benefits - delegating research, submissions, bookings, and comparisons - are obvious, but risks grow as systems act with our credentials. Expect adversarial sites designed to trick agents; security must evolve alongside capability.
My Take
The tech is impressive and the potential massive, but we're entering uncharted territory for AI agency and security. We're living through the moment AI transitions from helper to agent.
