At WWDC this year, Apple announced a team-up with OpenAI.  ChatGPT will be embedded in many places in the next iOS version, serving up writing assistance, creating custom memojis, and making Siri smarter.

And it costs you nothing!

However, does it cost Apple nothing?

According to reports, Apple is not paying OpenAI one thin dime for this integration.  So why would OpenAI do this?  After all, it requires enormous processing power, management time, development work, etc.  It’s easy to understand Apple’s motivations: it gets leading-edge AI integrated into its platforms.

But what does OpenAI get?

The Data Wall

In the landmark series of essays, Situational Awareness, the “data wall” is explained:

There is a potentially important source of variance for all of this: we’re running out of internet data. That could mean that, very soon, the naive approach to pretraining larger language models on more scraped data could start hitting serious bottlenecks.

Frontier models are already trained on much of the internet. Llama 3, for example, was trained on over 15T tokens. Common Crawl, a dump of much of the internet used for LLM training, is >100T  tokens raw, though much of that is spam and duplication (e.g., a relatively simple deduplication leads to 30T tokens, implying Llama 3 would already be using basically all the data). Moreover, for more specific domains like code, there are many fewer tokens still, e.g. public github repos are estimated to be in low trillions of tokens.

Training LLMs requires vast amounts of data.  In fact, we’re out of data.  All the large publicly available datasets are already trained to death, and doing clever things like sucking in every Reddit post or every Tweet has already been done.  There’s ebooks and YouTube and other things, but the core problem is that there is only so much data.

Now Apple is presenting OpenAI with a vast new storehouse of data: everything its hundreds of millions of customers do on their iPhones, iPads, Watches, and Macs.

Remember, Sam Altman is a Liar

Now, of course, they won’t be training on Apple user data.  Of course not.

Also remember that at every turn, Sam Altman has ignored safety, rules, and limits.  That’s why he was canned.

If you honestly believe that your Apple-originated data is not going to be used to improve ChatGPT’s models, I have a bridge in New York I’d like to sell you.

The official story is that Apple is acting as a brand ambassador for OpenAI…wink, wink.

• Author
• Recent Posts

raindog308

Dread Lord of LowEnd Content at LowEndBox

I'm raindog308, techno polymath and long-time LowEndTalk community Administrator. My technical interests include all things Unix, perl, python, golang, shell scripting, vintage operating systems such as MVS, and relational database systems such as Oracle, PostgreSQL, and MySQL.

I also enjoy using LEBs! I have an empire of little guys for VPNs, larger guys for apps, and big guys for databases and storage. When I'm not in front a screen I'm into German Shepherd dogs, quality knives, target shooting, theology, tabletop roleplaying games, and forest hiking.

I enjoy writing technical articles here on LowEndBox to help people get more out of their systems. You can find me on LowEndTalk @raindog308

Latest posts by raindog308 (see all)

• Hey Providers – Want Some FREE Advertising During the SuperBowl? - November 14, 2024
• Inception Hosting is Closing Its Doors - November 12, 2024
• How Will the 2024 Election Results Affect Tech? - November 11, 2024