In recent months, Apple has been quietly working on a series of artificial intelligence projects that are meant to radically change the way we interact with our devices. From new tools for modifying images and creating videos to significant improvements in Spotlight and Xcode. But what exactly do we know about these innovations and what effect will they have?
It seems that the launch of iOS 18 is going to make history. Tim Cook himself has confirmed that Apple is investing a "tremendous" amount of time and effort in the development of artificial intelligence technology, and they are excited to share the details "later this year 2024".
And we are not just talking about iOS 18 being inspired by visionOS, or about bringing more power to Siri and the Messages app with its own AI. We are talking about an operating system that will mark a historic change and will be characterized by the integration of one or several AIs deeply rooted in the core of Apple devices. All this probably backed by the new improved Neural Engine in the upcoming iPhone 16 and the existing ones in previous models.
So far, two of these new AI tools have been confirmed and presented, and three more are rumored. Let's take a look at all of them.
Editing images with Artificial Intelligence by asking Siri
The "MGIE" tool (Guided Image Editing by MLLM) is one of the first surprises, allowing complex editions and modifications in images through simple instructions to Siri.
Developed in collaboration with academics from the University of California, MGIE uses Large Multimodal Language Models (MLLM) to interpret verbal requests and execute precise pixel-level editions. This includes global adjustments like changing lighting, sharpness, and contrast, to specific edits like altering the color of individual objects or adding elements not present in the original image.
Additionally, it allows modifying the shape, size, color, or texture of specific regions or objects in the image, and even making Photoshop-style modifications like cropping, resizing, rotating, adding filters, changing backgrounds, and merging images.
The ability of MGIE to understand and act on complex instructions like "make the sky bluer" or "add a dog to the right side of the image" greatly simplifies photo editing, making advanced image manipulation accessible to a wider audience. In fact, the model is already available on GitHub, including the code, data, and pre-trained models, so if we have the necessary equipment, we can already test the system.
Creating videos from text with Keyframer
On the other hand, "Keyframer" is a very significant innovation in the field of image animation. This tool allows us to convert static images into dynamic animations using simple text commands.
When uploading an image, for example, of a landscape, we can request "create a sequence where the sun sets and the stars begin to shine," and Keyframer will automatically generate the necessary CSS code to carry out this animation. The tool works by transforming images into scalable vector graphics (SVG) and uses instructions based on large language models (LLMs), from simple text indications, to design complex animations that until now would require advanced skills in graphic design and animation.
This ability to generate animations from textual descriptions opens new horizons in the creation of visual content, for example in the Keynote app, where, when integrated, it will allow the animation of slides in a way never seen before. A service that brings our ideas to life intuitively and without the need to master professional animation software.
Spotlight powered by Artificial Intelligence
Entering the realm of rumors, as 9to5 Mac reports, Apple is also exploring improving Spotlight with generative AI, which could make it a much more powerful search and organization tool. By integrating large language models, Spotlight could offer answers to complex queries, execute actions within applications, and provide information in a broader context, surpassing its current capabilities limited to basic searches.
For example, it would be possible for us to ask how to perform a specific task in a productivity application and receive detailed instructions directly in Spotlight. In addition to interact more deeply with the device's content, such as accessing specific events in the calendar or initiating calls in communication applications with just a verbal or written request.
Xcode: Automation of Software Development
In the realm of software development, rumors (via macrumors) indicate that Apple is also preparing a new AI tool in Xcode that promises to streamline code generation. Inspired by solutions like GitHub Copilot, this tool could predict code and complete blocks based on natural language descriptions (text), greatly facilitating the application development process.
In addition, this tool is expected to have the capability to convert code from one programming language to another, thus optimizing the workflow of multi-platform application development. The integration of these AI capabilities in Xcode has the potential to accelerate software development, reduce errors, and make programming accessible to an even wider audience, including those with less development experience.
iWork: Artificial Intelligence in Office Software
Finally, we know of the acquisition of the iWork.ai domain. And with it, we believe that Apple hints at the integration of AI into its office suite, suggesting that applications like Pages, Numbers, and Keynote could receive significant improvements. An AI could offer design suggestions based on the content of the document, advanced data analysis in Numbers, and dynamic assistance in creating presentations in Keynote.
In addition, it is speculated that an AI could facilitate writing and editing text through automatic content generation or paragraph reformulation to improve clarity and cohesion. It could also integrate the ability to perform complex tasks, such as automatically scheduling events based on the content of a document or generating executive summaries for long reports.
Although Apple has introduced these AI technologies without the media hype of other launches, the potential it has right now to transform our interaction with technology is immense. We are on the verge of a new era where AI will be integrated into our digital lives, facilitating creativity and improving efficiency in everyday tasks. With all hopes placed on iOS 18 and the statements from Tim Cook, the future of technology, marked by advances in artificial intelligence, has the potential to be revolutionary.
On Hanaringo | 4 Safari extensions that will change the way we browse