[API调用] ChatGPT4-O large model launch: full video replay and interpretation, real-time speech translation, video image interpretation, emotion recognition

https://v.douyin.com/i293cBrv/
This livestream lasted 20 minutes and mainly introduced several new features of GPT-4o, including real-time voice conversation, real-time video conversation, and sentiment analysis, which gives ChatGPT eyes, ears, a mouth, and emotional intelligence. After the livestream ended, some accounts had already received trial access. The moderator promptly provided an interpretation of the content for everyone; please see below.



I. Main Updates

1. Expanded Context Window
GPT-4o supports a context window of up to 128,000 tokens, equivalent to being able to process 300 pages of text. This enables the model to process and understand complex tasks over a larger scope, greatly improving processing efficiency.

2. Faster Speed and Lower Cost
GPT-4o has significantly improved speed (the moderator has experienced its extremely fast speed), and its cost is lower. Compared with GPT-4, input tokens are 3 times cheaper and output tokens are 2 times cheaper. The message limit per three hours has also been increased fivefold, greatly improving processing efficiency.https://openai.com/api/pricing/



3. Multimodal Capabilities
GPT-4o integrates visual and audio processing capabilities and can process images, text, and speech. This includes real-time conversational voice functionality, enabling natural voice interaction and supporting real-time translation and emotion recognition.

4. Visual Processing
The model can perform image analysis and interaction by uploading screenshots, photos, and documents containing text. It can also recognize and describe content in images, making it suitable for complex applications that require image and text processing.

5. Improved User Interface and User Experience
The new user interface is cleaner and more natural, making it easier for users to interact with the model. It also supports multi-file uploads and longer login durations, optimizing the user experience.

6. Broad User Coverage
GPT-4o is now available not only to paid users but also to free users, enabling more people to use advanced AI tools.


II. Hands-on Demos and Demonstrations


Real-time Conversational Voice Demo
At the launch event, Mark Chen conducted a real-time conversational voice demo with ChatGPT, demonstrating GPT-4o's efficiency and naturalness. The demo process is as follows:



At 9:33 in the video

Step 1: Mark had a voice conversation with ChatGPT via his phone. He told ChatGPT that he was a little nervous, and ChatGPT offered breathing relaxation suggestions and provided real-time feedback.
Step 2: Mark asked ChatGPT to tell his friend Barrett a bedtime story about a robot in love. ChatGPT not only generated the story but also adjusted the emotion and style of its voice as requested, demonstrating its rich dynamic range.
Step 3: Mark further asked ChatGPT to tell the story using a dramatic, robotic voice and in a singing manner; ChatGPT successfully completed all tasks, demonstrating its diverse voice generation capabilities.

Visual Processing Demo
Barrett Zoph demonstrated GPT-4o's visual processing capabilities, showing how the model processes and understands images. The specific process is as follows:


At 14:14 in the video

Step 1: Barrett wrote a linear equation on paper and uploaded it to ChatGPT via a photo.
Step 2: ChatGPT successfully recognized the equation and guided Barrett to solve it step by step. ChatGPT first suggested that Barrett move the constants in the equation to one side, then perform subtraction.
Step 3: During the problem-solving process, ChatGPT provided useful hints that helped Barrett complete solving the linear equation.

Real-time Translation Demo
An audience member requested real-time translation, and Mark Chen and ChatGPT conducted a real-time translation conversation between English and Italian. The demo process is as follows:



At 22:40 in the video

Step 1: Mark asked ChatGPT to provide real-time translation between him and his friend who only speaks Italian.
Step 2: Mira Murati asked ChatGPT a question in Italian, and ChatGPT translated it into English; after Mark answered, ChatGPT translated the answer back into Italian. The entire process was smooth and natural, demonstrating GPT-4o's multilingual processing capabilities.

Emotion Recognition Demo
Barrett Zoph also demonstrated GPT-4o's emotion recognition function. The specific process is as follows:



At 23:30 in the video

Step 1: Barrett uploaded a selfie and asked ChatGPT to recognize his emotion.
Step 2: ChatGPT successfully determined Barrett's emotion in the photo, noting that he looked happy and excited.
Step 3: ChatGPT also humorously responded to Barrett's praise of its performance, demonstrating its ability to interact and understand emotions.