Google DeepMind has unveiled experimental prototypes for an AI-powered mouse pointer that lets users direct Gemini across applications by pointing, speaking, and using natural shorthand like "fix this" or "move that here." The research is detailed in a blog post from DeepMind's Adrien Baranes and Rob Marchant.
The pointer captures the visual and semantic context around the cursor, letting Gemini understand both what the user is referring to and what they want done with it. Demos are live in Google AI Studio for image editing and map navigation, the technology is rolling into Gemini in Chrome, and a "Magic Pointer" feature is scheduled for Googlebook laptops later in 2026.
Beyond the AI Detour
The redesign aims to remove the friction of jumping between an application and a separate AI window.
Most AI tools live in their own windows, requiring users to copy or drag content into them. DeepMind's prototype reverses that pattern. Users can point at a PDF and request a summary, hover over a table and ask for a chart, or highlight a recipe and ask for doubled ingredients. The pointer's goal, according to DeepMind, is to let Gemini meet users across all tools without interrupting flow.
Show, Tell, and Point
The system reads the word, paragraph, image region, or code block sitting under the cursor and pairs that context with brief spoken commands.
DeepMind organizes the prototype around four design principles. "Maintain the flow" is the no-detour philosophy. "Show and tell" is the underlying capability that lets the computer "see" what's important on screen without the user describing it in text. "The power of this and that" replaces text-heavy prompts with pronouns: saying "fix this" or "move that here" while pointing collapses a paragraph of instructions into a few words. The fourth, turning pixels into actionable entities, extends the idea further: scribbled notes become interactive elements, and paused video frames turn into booking links.
From Demo to Product
Two interactive prototypes are live in Google AI Studio, with a built-in version arriving on a Google laptop platform.
Image editing and map navigation demos run in Google AI Studio, both using the same point-and-speak input pattern. Gemini in Chrome supports pointer-based queries about specific elements on a webpage. The Magic Pointer feature, built jointly by the DeepMind and Googlebook teams, ships on Googlebook laptops later in 2026.
Cursor Cuts a New Take
Pointer-driven AI shifts routine tasks away from prompt-writing toward direct manipulation.
The implication of point-and-speak control is that everyday AI requests bypass the prompt entirely: users point at the thing, name the change, and the system handles the rest. The open question is whether Gemini consistently interprets ambiguous references, since "fix this" is only useful if the AI agrees on what "this" refers to. DeepMind's public framing of the work points to a design philosophy intended to scale across Google's product surfaces.
