Your GPT-4 Cheat Sheet

Everything You Need to Know About OpenAI's Next Gen LLM

Background:

1. GPT-4 is the latest release from OpenAI, the company responsible for the first widespread public release of an AI that can chat convincingly with people, ChatGPT.
2. GPT stands for "generative pre-trained transformer," a family of language models created by OpenAI based in part on technology first developed by Google in 2017 (Transformer).
3. Like its predecessors, GPT-4 is what's known as a Large Language Model (LLM). LLMs are algorithms produced by feeding a computer massive amounts of text, usually scraped from the public Internet, and other data, then training it to reliably detect linguistic patterns from word fragments. It then uses statistics to predict what new combinations of fragments in a given context will make sense to people who understand the language. It can also handle other content, such as math and computer code.
4. LLMs are said to be "unsupervised" when the code mostly runs by itself without human intervention, except maybe some fine tuning at the very end, when people might observe the results and provide feedback to correct the model.
5. They are said to be "supervised" when people review the results and provide feedback to help tune the model throughout the process. (I participated in this type of work at Twitter leading a team using GPT-3 in 2021 to train a supervised model to summarize Twitter Sports Trends, it was pretty successful, more so than attempts to do it completely unsupervised).
6. Before GPT-4 there was GPT in 2018, GPT-2 in 2019, and GPT-3 in 2020. All three use mostly unsupervised training.
7. In 2022, OpenAI took a different tack and developed a mostly supervised training model through a fork called InstructGPT. This produced GPT-3.5, the basis of its breakout hit, ChatGPT.
8. GPT-4 is probably a combination of unsupervised and supervised learning, although whether the latter goes beyond fine tuning is unknown. OpenAI is not saying much about how it was developed or works (see below), but it did say it incorporated significant human feedback to improve safety, so we know that much.
9. After the insane reaction to ChatGPT, OpenAI CEO Sam Altman said the much anticipated update to GPT-4 was sure to disappoint. He may have missed on that.

Highlights:

1. OpenAI smoked Google. Again. Or did Google smoke itself? One hour before the GPT-4 announcement was scheduled to go live, Google dropped a public API for its PaLM LLM and integrations with Google Docs, Gmail and other products – news that got completely buried about 60 minutes later. (A Google search on each reveals about 5,500 links related to PaLM and 19.5 million related to GPT-4.)
2. GPT-4 is "multimodal," meaning it can interpret both text and images, although it only outputs text, so no prompt-to-image capabilities are included. (OpenAI has another product that does this, DALL-E). Instead it can produce a detailed text description from an image prompt, which has other valuable uses.
3. OpenAI is testing the image to text capability exclusively for now with Be My Eyes, a tech company that makes accessibility solutions for blind and visually impaired people. The plan is to create a mobile app that can describe the world in real time to assist in the basic logistics of getting around sightless.
4. It can explain memes! (This seems almost as urgent as helping people who can't see well – the Internet needs a cure for cultural blindness. Example below.)
5. OpenClosedAI disclosed almost no technical details about its new model, citing a need to "weigh the competitive, above the scientific, value of further transparency." Many experts described this as a shocking break with academic norms in the space, and an about face for the company, which was founded as a nonprofit open research foundation. (We wrote approvingly about open source AI yesterday, related to a leak of Meta's LLaMA model.)
6. GPT-4 is BIG. It trained on 100 trillion parameters! (See below)
7. The raw numbers sound impressive – many people have pointed out that's more than 500 times the number used for GPT-3.5, the model used to train ChatGPT – but that's not a meaningful way to compare the two, experts caution.... Also OpenAI did not actually publish the params in its research paper.... Also bigger is not always better. Smaller models are cheaper to run and can be supported on less powerful and less expensive hardware (see LLaMA above.)
8. Microsoft, OpenAI's biggest investor and customer, built a custom supercomputer to develop the model, which uses some very cutting edge computer design techniques never before seen.
9. OpenAI said the improvements over ChatGPT are not always obvious for simpler tasks, but emerge when faced with more complex and nuanced problems. For example, GPT-4 outperforms ChatGPT dramatically in common standard tests like the Bar Exam, and is 82% less likely to respond to requests for disallowed content and 40% more likely to produce factual responses than GPT-3.5 in internal evaluations.

How Can You Get Access:

1. Subscribers to OpenAI's paid ChatGPT-Plus service should be able to access it now. It costs is $20/month, is available only in the U.S. at the moment and is waitlisted for new signups. Here's the form to join.
2. If you are a developer, you can sign up for the waitlist for the API. No word on how long before you'll get accepted though.

GPT-4 Struts Its Stuff

Create

Some of our favorite digital art that's on auction right now.

"Azura Carnivale" by Belgian artist Els Casaert at Foundation.

"The axis of creation" by Data Velvet on Foundation.

"Grand Sante" by Glitch of Mind has just sold on SuperRare.

Quick Hits

Microsoft Cuts One of Its Ethical AI Teams
Was it ethical to have more than one? Did they know about each other?

Here's Some GPT-4 You Can Get Today
Just sign up for Microsoft Bing. It's in there!

Quiz. What Do Chicken Nuggets and GPT-4 Have in Common?
That's it. Follow the link if you don't already know the answer, it's worth it.

You May Have Missed the 70s, Don't Miss This
Getty Images is auctioning photos of some iconic music artists from a legendary era in its first NFT drop March 21. Springsteen, Elvis, Iggy Pop and more.

Cool Tools

So, AI Chat, That's Totally Free, Now, Right?
Step-by-step tutorial to install and run Meta's LLaMA on Google's free Colab cloud service

Meet Claude, the AI Chatbot You Can't Social Engineer
SMH. The Internet loves a good challenge

GPT-4 Saves the World
And Fixes Your Dumb ETH Smart Contracts

Why Take One, When You Can Have Both?
Quora's Poe AI integration offers GPT-4 and Claude, together. How long before they get into an argument?