2003

predicting the next word that can come after a sequence of words.

2013 Google made it work. word2vec

this could use the self supervision