Microsoft announced the Fall Moments update for the Windows 11 operating system. As part of the update, every supported PC will receive two main features that turn it into an AI tool: Copilot Voice and Copilot Vision. Copilot Voice allows voice interaction with the computer and natural conversation, similar to how we interact today with ChatGPT (which, by the way, comes from the same house). Copilot Vision enables the computer to see and understand its real-world environment through the camera lens, or to see and understand what is happening on the user’s screen and perform actions on the content. Another feature is the expansion of the Copilot Actions feature, which allows the browser to perform tasks on behalf of the user also in the local operating system, as described further below.
With Copilot Voice, now available on every Windows 11 PC, users can activate the AI assistant Copilot simply by saying “Hey, Copilot.” You can ask it questions, search for information, request it to perform tasks, and more.
With Copilot Vision, the main part of the update, Windows 11 users can fully share their desktop or applications with Copilot, and the AI can provide insights, answer questions, and more.
Another significant part is the guidance feature. Users who don’t know how to perform certain tasks can point to the AI on the screen and ask how to do something, and the AI will guide them. This feature is particularly useful for older users or those with low technological familiarity who don’t always know how to do everything on a computer. Microsoft calls this feature Highlights, and it works within applications as well.
In this context, Copilot Vision can fully understand the context within Microsoft 365 (formerly Office) applications like PowerPoint, Excel, and Word. With user permission, the AI can see what’s on the screen and assist with tasks such as designing presentations or analyzing data.
The Copilot typing interface is now integrated directly into the taskbar with a redesigned look, making it easier to interact with the AI assistant directly from the taskbar.
The Expansion of Copilot Actions
A feature Microsoft introduced over the past year is Copilot Actions, which allows an intelligent agent to perform tasks on behalf of the user within the browser, such as booking a hotel, finding flights, and more, without user involvement. Think of it as a human assistant doing all the browser work for you. This feature is now expanding to the Windows 11 Insiders program, including tasks outside the browser and within the operating system, such as in File Explorer, searching for files on the PC, and more.
All you need to do is describe to the computer what you want to do, and it will execute it for you. This capability ties into another Microsoft announcement: connecting AI to services like OneDrive and others (including external services such as Google Gmail) and performing actions on them. The results of the tasks given to the computer can be exported to other applications, for example, for schoolwork.
Microsoft is also collaborating with the task agent tool Manus, allowing it to perform complex tasks on your local files, with permission. The foundation for this is a shared infrastructure Microsoft created with Manus’ parent company, using Azure servers enhanced with AI capabilities. This is a surprising move, considering Manus’ parent company is a third-party firm based in Shanghai.
Security and Privacy Settings
Of course, everything described comes with new security and privacy levels, including new “security fences.”
Microsoft strives to address these concerns (and we all remember the recall fiasco): this time, control is fully in the user’s hands, and you can disable or suspend Copilot Actions at any time.
Microsoft also states that users will be provided full transparency about actions, with updates at every stage. Regarding security, Microsoft asserts that comprehensive checks and organizational feedback collection are continuously performed to improve security and privacy mechanisms.