OpenAI GPT-4 Omni model can interpret audio, video, and text in real time
The latest iteration of ChatGPT promises to be the most advanced one yet.
OpenAI has issued an update for its ChatGPT bot. The GPT-4o update promises greater ease of use for all users, as well as increased speed across the board.
"GPT-4o (“o” for “omni”) is a step towards much more natural human-computer interaction—it accepts as input any combination of text, audio, and image and generates any combination of text, audio, and image outputs," reads the OpenAI website. "It can respond to audio inputs in as little as 232 milliseconds, with an average of 320 milliseconds, which is similar to human response time(opens in a new window) in a conversation. It matches GPT-4 Turbo performance on text in English and code, with significant improvement on text in non-English languages, while also being much faster and 50% cheaper in the API. GPT-4o is especially better at vision and audio understanding compared to existing models."
OpenAI technology chief Mira Murati spoke during a livestream on Monday about the latest ChatGPT additions. She demonstrated some of its capabilities, including some new translation features. With the latest update, ChatGPT can now operate across 50 different languages.
OpenAI GPT-4 Omni x BeMyEyes demo #OpenAI #GPT4 #ComputerVision @BeMyEyes pic.twitter.com/Yr0wpFyFin
— Shacknews (@shacknews) May 13, 2024
As noted by CNBC, Murati made sure to thank NVIDIA CEO Jensen Huang for helping power OpenAI's technology. NVIDIA has a significant amount of money invested in the AI sector, which has helped power that company to better-than-expected earnings.
-
Shacknews
replyOzzie Mejia posted a new article, OpenAI GPT-4 Omni model can interpret audio, video, and text in real time
-
https://openai.com/index/spring-update/
No OpenAI thread?
"Any sufficiently advanced technology is indistinguishable from magic"
Yep, this ChatGPT demo.
https://x.com/BenBajarin/status/1790070846473523390
OpenAI announces Her.-
-
Yeah, I see a post and nothing else.
-
-
is there a demo or anything? OP is just same text as this post.
-
Watch the video, there's a demo
https://www.youtube.com/live/DQacCB9tDaw?si=J-Stu5dDZ4dMXMhw&t=555
-
-
Spring update here:
https://www.youtube.com/live/DQacCB9tDaw?si=xEEwr0UMiDBnGLin-
About 9m30s in is where it starts to get... Wild.
-
this is both amazing and also annoying as fuck when it pretends to be human by sighing when responding to rapidly changing instructions, or claiming "i got too excited" when it did something wrong and is then corrected
the future of people falling in love with this is shit is closer than ever-
-
-
Honestly, I'm very hopeful for that future. I can see a few situations where this is useful:
- People who didn't have positive socialization skills growing up or never learned how to connect safely.
- Poly/Mono couples.
- Recovery from an abusive relationship.
- People who just don't want to have a relationship with another human.
Like all things outside assistance will be needed, but this bridges the gap between "I need someone to confide in" and "I can't face a therapist / cohort directly."
-
-
-
you can obviously just ask it to have a completely flat affect and speak in a robot voice if you want an autistic computer assistant but they wanted to show off how much more massively human it's capable of sounding now which is useful for all kinds of stuff
-
Uh. That's... Not a great usage of the word autistic there, bud.
-
-
-
-
faking AI demos is more of Google’s thing
-
-
just like people
-
Sounds like MY WIFE. kidding not really
-
-
-
-
I'm unclear whether this is going to hasten the erosion of communication between people online, or help the loneliness problem with an always on friend that can do instant research & feedback for you.
This thing is a few generations away from ushering us in to the post-people era where ya'll are obsolete and my assistant loves me more than my parents did.-
-
I'm concerned because we saw the floodgates open with social media with barely any consideration towards the mental well-being of it users.
It sounds like the ol' move fast & break things is back on the menu which bothers me.
Her is totally a great story about giving up on people and leaning in to a digital companion.
-
-
I think cellphonea have done or are doing that more than chat gpt. My kids, to my parents, all just pull out their phones whenever at a family gathering to browse shit. There’s no social norms anymore it’s fine to just completely zone out of wherever you are as a person. When it becomes just glasses or eye contacts people will just drop out even more
-
-
I'm looking forward to the point when the tech is mature enough to be a full-on digital assistant. The Voice conversations with ChatGPT are already remarkably helpful for me when it comes to brainstorming and organization, but if I could tie it into my calendar, all my documents, my home automation stuff, etc., that'd be awesome.
-
Awesome? Do you remember how that movie ended?
-
-
Man. That’s good stuff.
-
-
Probably a reference to the movie with the same name - about a female AI that the main character fell in love with. Great movie.
-
I was going more for this https://www.youtube.com/watch?v=pf9L-pPkKCY
-
-
Yeah but correct me if I'm wrong but doesn't joaquin choose physical connection over digital because the AI runs off with the other AI or something?
I mean I thought it was an allegory about how real life connection is superior because you can understand nuance and emotion better from a real person rather than a disembodied voice.
-
check out whos on the hog in the rearview mirror
-
-
There is a little bit of delay in responses that you don't see in this demo, but it's totally manageable. I do enjoy it. Would like to have it running on some little piece of hardware in my living room.
-
Looks like there is a mac app, so that problem might be solved.
-
-
-
On the app or just on the website? Because when I updated the app earlier I didn't see the new model available.
-
Hmm, so now the new model is available but it doesn't have any of the functionality they were showing off. I'm guessing that will come with an app update.
-
-
Ah, okay.
-
-
-
-
-
-
It’s there but timing out for me at the moment.
-
-
-
-
Feel like 1/3 of my prompts to ChatGPT are "please be more concise/less verbose"
-
Damn I just watched an episode of TNG where Picard said this verbatim to Data.
(S1 “Justice,” truly an awful episode.)
-
You can coax it into the type of responses you need via prompts and memory. Tell it you're rarely interested in verbose responses and prefer to the point and will ask when you need more assistance.
-
they haven't actually released the new voice mode, they're afraid to.
"In the future, improvements will allow for more natural, real-time voice conversation and the ability to converse with ChatGPT via real-time video. For example, you could show ChatGPT a live sports game and ask it to explain the rules to you. We plan to launch a new Voice Mode with these new capabilities in an alpha in the coming weeks, with early access for Plus users as we roll out more broadly"
"Today we are publicly releasing text and image inputs and text outputs. Over the upcoming weeks and months, we’ll be working on the technical infrastructure, usability via post-training, and safety necessary to release the other modalities."
-
That demo was creepy as fuck. Burn it with fire.
-
“All it’s doing” is vocalizing what it can already do with text.
-
It just feels like it's tipped over into uncanny valley territory for me. Listening to it seriously gave me super weird and uncomfortable feelings.
-
That’s not the case. This version is significantly more capable with text, but more importantly it’s natively multimodal. And the voice responses are massively faster and with more human affect and capabilities.
-
I get that, that’s why I put it quotes. I think what I said is true, conceptually. It shouldn’t freak anyone out or blow their mind if they’ve been using ChatGPT thus far.
-
-
-
-
maybe google will show something similar tomorrow, might be why this presentation was thrown together and released today but they haven't actually released the functionality... maybe they don't have a good way to stop legions of halfwits from falling in love with it, or using it to make zany tiktoks that will cause reputational harm (*starts camera* "hey chatgpt sing me a song about deez nuts")
-
-
2004. Search.
-
Maps. When it first came out we had Mapquest before that. Zooming in the way you could on Google maps was awesome.
-
You're thinking of their public products and I get where you're at with that.
If you look at the tech ology built at Google, they have been instrumental in moving internet technology forward. For example, they are the ones who published the paper that lead to all this gen ai craziness
-
The timing does feel very intentional. Google has clearly been behind in the race up until now, but they do have some intrinsic advantages that I think are starting to come online now. Like the fact that they've already had mature infrastructure for in-house ML chips (TPUs) for many years.
-
releasing in a few weeks after an unveil isn’t some weird schedule and is in line with the usual rollout speed when they announce new stuff (excluding SoRa)
-
-
It’s pretty rare for WWDC to have anything for broad release. Any potential iOS integration surely does not become available until the fall with the new iPhone launch.
-
-
Adds more color to stories of frustration on the Apple Vision Pro team that Siri is so useless as a tool or interface for the device. Likewise can see why Zuck is so bearish on the Meta glasses given where they are on AI research
-
-
-
-
-
This will be bard tomorrow
https://youtu.be/ksHw6ybWJHg?si=RaQGay9l8KOa8M9k
-
-
ChatGPT är en fantastisk teknik
Du kan använda den på: https://chatgptsv.se/
-
