Why does GPT-3 work for ad copies?

The hype for GPT-3 has come down a little the last couple of months. Also, since the start of this month, OpenAI has made the API access a paid product. So most probably we will either hear of very specific business cases or not hear them at all as businesses may start using GPT-3 behind the scenes.

But thanks to OpenAI providing access to more than 1000 people(I am guessing based on the slack users), we were able to see lots of innovative and fun ideas using the GPT3 API. In this blog itself I have covered lots of interesting use cases which I have built.

I think now we have a fair idea of what GPT3 is capable of and what its not. Though there have been arguments about how GPT-n approach is the path to AGI, I will not go down that road. What I will do though, is to look at the successful GPT3 examples and make some (bold?) predictions.

Thanks to reddit, we have a list of free sites/programs powered by GPT3 which can be used without a waitlist.

Going through the list, we can see a pattern of what works on GPT3.

Startup idea generator, Tagline generator, Blog idea generator, Job description generator, Idea generator, Philosopher AI and AI Dungeon.

We have also had some mind blowing demos like text to html/react, text to figma design etc. But we have not yet had a public launch of any of these.

So what does work well on GPT3?

Its actually pretty easy to figure out.

The main job of the GPT3 API is to predict the next word. That’s pretty much it in a nutshell.

So lets try to break it down and see how it works.

Lets pick a startup idea generator.

Lets say I am giving you the following priming example:

Generate startup ideas like below:

-> A web-based social seating check-in platform to help air travelers see who is on board their flight and use Facebook and Linked in to assign all flight seats with one click.

-> An online and mobile platform to help socially-conscious businesses reward high school students for good grades and good attendance with deals and local programs that inspire a community-wide concern for education.

-> A gift recommendation engine to help men foster better relationships through periodic and thoughtful romantic gestures, utilizing behavioral science, models of preference and social trends.

-> A college-based mobile social……………

Can you fill up the remaining sentence with a startup idea? Each reader would have come up with their own startup idea(please leave them in the comments below :)).

Now if a system exists which has been trained on a lot of startup ideas, then it will most probably predict the next word to be “network” followed by “for” followed by a variety of nouns with equal probabilities like “gamers”, “high school students” etc etc. and so on.

And that’s exactly what GPT3 does. It keeps on finding statistical best fits and outputs the sentences. And for us, rather than think up all possible startup ideas, its easier to read through generated startup ideas and select ones which make sense, this works out.

You can apply the same logic for any generator ideas above. In fact for a startup generator we can templatize it more.

“my company, _(insert name of company)_, is developing _(a defined offering)_ to help _(a defined audience)_ _(solve a problem)_ with _(secret sauce)_”.

So we have understood why GPT3 works for these generator ideas.

What about the code completion ideas then?

This is a little bit more iffy. My experience is that these are more brittle. We need to do a lot more tooling on top of GPT3 API to make it production ready. I am not sure if anyone has cracked it yet. So far only video demos with sign up forms for waitlist are available.

And I think it makes sense. As long as GPT3 has seen examples of code for adding two numbers in python or react examples of creating buttons, it will predict the next tokens one after another in the correct way and you may end up with the right program if you are lucky. But if the input is opened up and people can ask for anything then it will predict different tokens and the output will not work. The output will still look like react code or python code. But it will not be correct.

I am not saying code completion is not a good use case for GPT3. I am just saying that it will need a lot more tooling in addition to just GPT3.

Final Say:

If you want to use GPT3 in your products to have the extra edge, I would suggest you stick with 1-2 sentence generative text ideas. It works really well in that department. For any other thing, plan your pipeline properly and see how GPT3 can be helped to not make mistakes.