The industry and not least the market is eagerly awaiting what Apple will reveal of AI functions in iOS 18, which will be shown during WWDC 2024 in June.
Apple has two secret weapons in its AI arsenal for September
Rumors have swirled that Apple is internally testing ChatGPT-like services. It goes without saying that they have been doing it for several years, but it must be reassuring for investors to hear AI leaks from a company that has been completely absent from the AI race. As usual, Apple deals with its planned press conferences and developer roadmaps for testing new software, AI or not.
In addition to the reassuring leak that Apple knows what is happening out there in the AI world, two research reports have been published. You may have heard of the first one already. AI costs money if the device constantly has to query a cloud service – the companies that can deliver local LLMs on the device will have a big advantage.
We wrote about the research in December :
"For the great good news is that the company's researchers have found a way to deliver LLM models locally also on iPhones effectively on older models as well. How has Apple solved this? By using the storage chips on iPhones, which usually have 128GB and more, in stark contrast to many iPhones that have 4GB and maybe 6GB of RAM.”
This is report number two
If Apple can manage to run LLMs locally on older iPhones, this alone will strengthen Apple's AI ecosystem as soon as it becomes possible with iOS 18. The new disclosure ( read the document ) focuses on how Apple will probably let users manipulate images with Siri and text commands. In other words, Apple has researched how to manipulate images only by describing what is desired using text and "MLLM-Guided Image Editing (MGIE)."
The technique uses multimodal LLMs to understand what the user wants to do with images by analyzing sentences. As you can see from the example below, an AI can make a number of changes, including adding more tomatoes and herbs if the user writes that he or she wants a healthier pizza. It is also possible to remove unwanted objects.
In addition to pure image manipulation, it is also possible to rotate and resize images, in addition to manipulating brightness, contrast and image balance. Everything is possible with text commands.