Taking on OpenAI
Steps to build foundational models which Sama seems to think is hopeless to do.
Saw a tweet from Rajan Anandan:
https://twitter.com/RajanAnandan/status/1666641010284449792?s=20
So, Sama thinks that building foundational models that take on ChatGPT is hopeless. Of course he has to think that since he is now the incumbent.
Sama has now clarified that he meant it is hopeless to build foundational models spending only $10 million.
Whatever his thoughts, his belief stems from a couple of reasons.
He believes that the current approaches are the only way we can build a system like ChatGPT.
The current approaches depend on data and compute and Open AI most probably has the most amount of data and compute.
When someone says “hopeless”, thats when my eyes light up. So, time to take up the challenge.
So far we have been working on varied AI problems with quite a good success. But from today I am making it my personal goal to build a foundational model out of India within 2 years.
And it’s not like I am just making something out of thin air. I am talking because I have exposure to models using the Alpes algorithm which have shown lots of promise. We have already built our own embeddings.
We have started building foundational models for ASR which have shown good promise.
We have started to build text foundational models. So far we some results for 4 window context sentences(sort of like 4-grams).
But we know that we are building on top of a mathematically proven model. So it’s just a matter of time and some investment and we will have our models out in the world soon.
This post is an off the cuff post I am writing after I saw the tweet. So I have not thought it out fully. But all I know is that
I will not be using deep learning or any of the known methods(there is no way we can win using their methods). I will most probably be using KESieve algorithm from Alpes.
I will be using students a lot. With some help from Swecha and FSF if possible.
I will be using maths a lot. Need to identify better algorithms which do not need GPUs.
And most probably I will not be using any funding(no Indian VC will be follish enough to fund this :))
Stay tuned for more updates on this very soon.