#61. What I learned from Apple’s Developer Conference (WWDC 2024) - Apple Intelligence
Dear Reader,
Last few weeks have been interesting. I attended a great event at London Business School with Dr. Robert Cialdini on his successful book Influence, written over 40 years ago. I also attended the London Tech Week, and would be writing about both events soon.
In our last issue on Nero’s Weekly, I mentioned the different ways big tech companies have positioned themselves strategically for AI, and how Apple could compete using Siri, and we shall see at Apple’s WWDC 2024 event.
Well, Apple WWDC 2024 was last week and Apple did not disappoint. Today I’ll be reviewing all the announcements at the event, especially Apple Intelligence. Enjoy😊.
The event started with the latest software update for the Apple Vision Pro device, iPhone, the Apple home devices (Air Pods, Apple TV, and Apple Watch), the iPad, and the Mac.
visionOS
It seems like only yesterday (Feb. 2024) that the Apple Vision Pro VR headset was released, but now we are getting a platform update: visionOS 2.0.
The key updates were immersive media features such as Spatial photos, and Mac mirroring. Spatial photos enable you convert your photos into those optimised with more natural depth.
Mac mirroring means that you can mirror your Mac display in Vision Pro showing an ultrawide display of your screen. Also Included in this update was a new wrist turn gesture to see the time and battery percentage of your device.
I also thought the final comment on “letting you take a private movie theatre wherever you go” was directed at Meta’s VR headset’s Travel Mode which was demoed by Mark Zuckerberg.
iOS 18
The iOS18 update introduced the ability to personalize and rearrange your home screen. Although this was already on android, it got a cheer from the crowd. It also introduced the ability to Lock and Hide an App to prevent someone using your iPhone to find these apps (Tinder?).
Across Messaging: there were new updates which included the ability to schedule your messages, more emoji support, satellite emergency messaging, and finally, rich content messaging support.
Other updates include a redesigned photo app, automatic categorization for emails in the Mail app, and a tap to cash feature for easy money transfer in Apple Wallet.
Apple Home: AirPods, AppleTV, and WatchOS
For the Apple Air Pods: Say you are receiving a call on a crowded train, the new update for the Apple Air Pods includes the ability to respond to a prompt by nodding or shaking your head. It also includes the new voice isolation which removes background noise to deliver clear speech whilst on a call.
For Apple TV: using the controls, Apple TV now shows the character’s name and song in a scene that you are watching. You can then add this song to your Apple Music playlist.
For WatchOS: the biggest update was the new Training mode which translates your workout data into an effort rating. This should help users optimise their workout effort level.
iPadOS 18
The presentation for the latest updates for the iPadOS started with a new way to personalize your home screen, and customize the tab bar. This was in addition to the new personalization features we saw earlier for iOS18. But that was not the main event.
The first major update was ‘Screen sharing’ where you can help someone on their iPad/iPhone by tapping or pointing on your screen and it shows on theirs, you can also ask for permission to control their device from yours. Pretty useful for family tech support 😊.
The second major feature is a new calculator app for the iPad known as Math Notes. Initially it looks like just the calculator app on the iPod transcribed to the iPad, but its more.
Using the Apple Pencil, you can write down math expressions and once you add an equal sign, the calculator solves the expression for you (in your handwriting). This feature also works in the Notes app.
This was obviously powered by some Artificial Intelligence (AI), but till this point AI hadn’t been mentioned in the presentation.
I was thinking about why Apple would go these lengths to release a calculator app, and then I figured it out. OpenAI recently released a math tutor functionality with their latest model. This was Apple’s response.
macOS Sequoia
The main update for macOS was iPhone Mirroring feature (within the Continuity app) which lets you access your iPhone from your Mac, and receive your iPhone notifications.
Another update was when working with different screens the Mac automatically suggest a tiled arrangement to split your screen, like you have on Windows.
Finally, for when you about to present from your Mac on a zoom call, a new Presenter preview has been introduced to prevent you from sharing the wrong screen.
This brings us to the main event of Apple’s WWDC conference: Apple Intelligence.
Apple Intelligence
The presentation began with Apple making a distinction with other GenAI tools – its distinction being having a personal context, and focusing on security of your data:
“With iOS 18, iPadOS 18, and macOS Sequoia we are embarking on a new journey to bring you intelligence that understands you. Apple Intelligence is the personal intelligence system that puts powerful generative models right at the core of your iPhone, iPad, and Mac. It draws on your personal context to give you intelligence that’s most helpful and relevant for you. It protects your privacy at every step. And it is deeply integrated into our platforms and throughout the apps you rely on to communicate, work, and express yourself.” – Craig Federighi
The presentation on Apple Intelligence was structured around its - Capabilities, Architecture, and Experiences.
Capabilities
This was focused on how Apple intelligence makes your device experience delightful with a suite of new features. It can understand language, images, text, your personal context, and take action. This includes:
Writing tools that can help you summarise, proofread, or rewrite a piece of text across your apps.
Images – AI helps you create original images. Since it understands your personal context and photos, you can create a personalized image of your friend while in conversation with them. “The next time you tell mum she is your hero; you can send an image of her in a superhero cape to land your point”.
Action – Apple intelligence can now take action across your devices e.g. “Play the podcast my wife sent to me”.
“Apple powerful intelligence goes hand in hand with powerful privacy” – Craig Federighi
This brings us to the architecture of Apple Intelligence that is built for privacy and security.
Architecture
The key aspect of Apple Intelligence is On-device processing. This means that all the processing is done on the device, and hence Apple does not collect your personal data.
This is achieved through Apple’s unique integration of hardware and software, powerful advance silicon chips, and capable Large Language Models (LLMs) and diffusion models for everyday task. This makes it easy to organize, find, and surface the relevant information across your apps. It leverages Apple cloud built with Apple silicon chips, and running the secure swift programming language.
But there are times when you require models that are complex or too big for your device. The cloud (servers) can help with this but they usually store your data and may misuse your data. Apple claims it has solved this, and extended the privacy and security on your device into the cloud by creating Private Cloud Compute. This is the 2nd key aspect of its architecture.
“When you make a request Apple Intelligence checks if it can be processed on the device, and if needs more compute it draws on Private Cloud Compute to send only the data required – your data is never stored and used only for your request” – Craig Federighi
This brings us to the experience of Apple Intelligence.
Experience
As expected, the key experience of Apple Intelligence was an updated and rebranded Siri that is meant to be more natural, relevant, and personal.
You can ask Siri questions from across your apps e.g. the weather, a photo on your device, and your mail and calendar – “What time do I pick mum from the airport?” You also have the option for a text prompt known as ‘Type to Siri’, and switch back to voice seamlessly.
Another key experience (as mentioned earlier) is better writing tools to rewrite, proofread, and summarize text and emails. When crafting an email reply, Apple intelligence enables you rewrite your draft based on a specific tone.
Apple intelligence also enables you create delightful images and emojis. You can sketch an image using Image Wand in the Notes app and it would create a well-drawn image. You can also use the new Image playground app to create an emoji or new image (sketch, illustration, or animation).
This brings us to the final (and according to Elon Musk, controversial) aspect of Apple Intelligence: a ChatGPT integration.
ChatGPT Integration
“There are other Artificial Intelligence tools that can be useful for task that draw on broad world knowledge or specialized domain knowledge, we are starting out with the best of these, the pioneer and market leader ChatGPT from OpenAI powered by GPT4o” - Craig Federighi
If there are complex task that ChatGPT’s model can better help with, Apple Intelligence would ask the user if it should use ChatGPT.
According to Apple, ChatGPT can’t store a user’s request, and they would also mask the user’s IP address. This should make it difficult to identify or create a search profile of a particular user.
Thoughts on Apple Intelligence
It is interesting how Apple has positioned the message of Apple Intelligence (AI) around Personalisation, Privacy, and Security. They have taken a genuine fear users have around their data and AI, and made it a unique selling point that differentiates them from other big tech AI providers. That is the first strategic level. This could also bolster Apple’s case with the EU’s Digital Market Act (which demanded Apple open up its app store to 3rd party payment providers, which Apple argues compromises security).
The second strategic level: Apple is leveraging its control of the hardware, software, Siri, the powerful M1 family of chips on the latest Apple devices and iCloud, in addition to access to personal data across 2 billion devices. They even branded AI: Apple Intelligence. Thirdly, by integrating ChatGPT, Apple has commoditized AI models, making them a feature and not a product.
It would be interesting how Apple Intelligence evolves. But then again, no product gets it right the first time. You couldn’t change the home screen of the first iPhone, or perform copy and paste for 3 years.
The pieces are all there for the first AI phone… The AI iPhone.
There you have it.
For more around Apple’s strategy read my 2022 book review of Strategy Rules.
How did you like this article on a scale 0-5? Is there a topic you currently struggle with? Let me know. You can message me at notesbynero@gmail.com.
If you’re finding this newsletter valuable, consider sharing it with friends, or subscribing at nerookwa.substack.com to receive all my deep dive and weekly articles.
Thanks for reading and bye for now.
Nero
Go Further