Scale Virtual Events
timezone
+00:00 GMT
SIGN IN
Sign in or Join the community to continue

Introduction to Text and Code Embeddings in the OpenAI API

Posted Oct 18, 2022 | Views 213
# TransformX 2022
# Breakout Session
# GPT-3
# Large Language Models (LLMs)
Share
SPEAKER
Arvind Neelakantan
Arvind Neelakantan
Arvind Neelakantan
Research Lead & Manager @ OpenAI

Arvind Neelakantan is a Research Lead and Manager at OpenAI working on deep learning research for real-world applications. He got his PhD from UMass Amherst where he was also a Google PhD Fellow. His work has received best paper awards at NeurIPS and at Automated Knowledge Base Construction workshop.

+ Read More

Arvind Neelakantan is a Research Lead and Manager at OpenAI working on deep learning research for real-world applications. He got his PhD from UMass Amherst where he was also a Google PhD Fellow. His work has received best paper awards at NeurIPS and at Automated Knowledge Base Construction workshop.

+ Read More
SUMMARY

Text embeddings are useful features in many applications including semantic search, predicting code completion, natural language, topic modeling, classification, and computing text similarity. Arvind Neelakantan, Research Lead and Manager at OpenAI, introduces the concept of embeddings, a new terminus in the OpenAI API.

When OpenAI originally introduced the API two years ago, it was based on the GPT-3 model, which was useful for many tasks. But, as Neelakantan explains, GPT-3 is not explicitly optimized to produce a single vector or embedding of the input. This ability, to have a condensed representation of the input, would be helpful for programmers and others to use as features for downstream applications, the OpenAI team determined. They set about building an unsupervised model that is good at getting this kind of single embedding, and created a contrastive pre-training model, which Neelakantan will describe. He covers use cases for embeddings, and how the API is used in the real world, including at JetBrains Research for astronomical research and at FineTune Learning, which builds education systems. FineTune is using text embeddings to more accurately find textbook content based on learning objectives.

+ Read More

Watch More

42
Posted Sep 09, 2022 | Views 42.1K
# Large Language Models (LLMs)
# Natural Language Processing (NLP)
55:22
Posted Oct 06, 2021 | Views 18.8K
# TransformX 2021
# Fireside Chat
47
Posted Oct 20, 2022 | Views 3.2K
# TransformX 2022
# Fireside Chat
# Natural Language Processing (NLP)
# Foundation Models
# Large Language Models (LLMs)
# Generative Models