OpenAI has recently released an extraordinary image analysis update for its chatbot, which evokes a sense of both admiration and uneasiness. This guide will provide instructions on how to utilize this innovative feature effectively, along with valuable advice for conducting experiments using it.
OPENAI RECENTLY ANNOUNCED an upgrade to ChatGPT (Apple, Android) that introduces two enhancements: AI voice options that allow users to hear the chatbot's responses to their prompts, and image analysis capabilities. The image functionality closely resembles the existing image search feature provided by Google's Bard chatbot at no charge.
Even after extensive testing of ChatGPT's boundaries and capabilities, I find OpenAI's chatbot both astonishing and disconcerting. Although I was genuinely impressed by the web browsing beta offered through ChatGPT Plus, I couldn't shake my concerns about the potential implications of this tool for online writers who depend on earning income. The introduction of the new image feature for OpenAI's subscribers has left me with conflicting emotions.
While I have personally not yet had the opportunity to experiment with the newly added audio capabilities (although other exceptional reporters on staff have), I have been able to evaluate the upcoming image features. Here is a guide on how to utilize the forthcoming image search in ChatGPT, along with some useful tips to get you started.
How to Access ChatGPT’s Image Features
The update is set to be released before the year's end, but the specific rollout date for the image and voice features to the general public remains uncertain. As per OpenAI's usual practice, paying subscribers are given priority access, similar to the introduction of ChatGPT's GPT-4 version.
Additionally, you have the choice to select and upload photos from your device, picking files stored on your phone. Desktop browser users of ChatGPT can conveniently upload saved photos from their computer. While video uploading functionality is not yet available for the chatbot, you can still submit multiple images in a single prompt.
Advice for Experimenting With the New AI Tools
This isn’t the first time a kind of “computer vision” has been made available to the public, but the accessible user interface paired with a powerful chatbot leaves the impression that there might be something distinct, and potentially transformative, happening here. Before we go any further, remember to never upload personal, sensitive photos to ChatGPT as you try out the image feature.
Want to limit how long OpenAI stores your data and AI interactions to train its chatbot? Go into Settings, then Data Controls, and turn off Chat History & Training. With this disabled, your information is deleted after one month. This process must be done individually for every browser that you use to access ChatGPT, for both PC and mobile.
I received the best results from ChatGPT when uploading images that were clear and well lit. It fumbled a few guesses, but the chatbot was able to label many of the objects in my apartment: from an orchid plant and international coins to a stray charging cable and Steve Irwin Funko Pop.
Despite the tool's power for searching through information, refrain from immediately trusting its answers. ChatGPT mislabeled my daily multivitamin as a pill for treating erectile dysfunction.
ChatGPT definitely has its limits. When given a random photo of a mural, it couldn’t identify the artist or location; however, ChatGPT easily clocked where images of multiple San Francisco landmarks were taken, like Dolores Park and the Salesforce Tower. Although it may still feel a bit gimmicky, anyone out on an adventure in a new city or country (or just a different neighborhood) might have fun playing around with the visual aspect of ChatGPT.
One of the major guardrails OpenAI put around this new feature is a limit on the chatbot’s ability to answer questions that identify humans. “I’m programmed to prioritize user privacy and safety. Identifying real people based on images, even if they are famous, is restricted in order to maintain these priorities,” ChatGPT told me. While it didn’t refuse to answer every question when shown pornography, the chatbot did hesitate to make any specific descriptions of the adult performers, beyond explaining their tattoos.
It’s worth noting that one conversation I had with the early version of ChatGPT’s image feature seemed to skirt around part of the guardrails put in place by OpenAI. At first, the chatbot refused to identify a meme of Bill Hader. Then ChatGPT guessed that an image of Brendan Fraser in George of the Jungle was actually a photo of Brian Krause in Charmed. When asked if it was certain, the chatbot switched over to the correct response.
In this same conversation, ChatGPT went wild trying to describe an image from RuPaul’s Drag Race. I shared a screenshot of Kylie Sonique Love, one of the drag queen contestants, and ChatGPT guessed that it was Brooke Lynn Hytes, a different contestant. I questioned the chatbot’s answer, and it proceeded to guess Laganja Estranja, then India Ferrah, then Blair St. Clair, then Alexis Mateo.
This isn't the first time a form of "computer vision" has been accessible to the public. However, the combination of a user-friendly interface and a powerful chatbot gives the impression that something distinct and potentially transformative is taking place here. Before proceeding, it's important to remember to never upload personal, sensitive photos to ChatGPT while experimenting with the image feature.
Interested in limiting how long OpenAI retains your data and AI interactions for training the chatbot? Simply go to Settings, then Data Controls, and disable Chat History & Training. By doing this, your information will be deleted after one month. Remember that this process must be done individually for each browser you use to access ChatGPT, whether on a PC or mobile device.
When I uploaded clear and well-lit images, I obtained the best results from ChatGPT. While the chatbot made a few errors, it was able to correctly identify many objects in my apartment, ranging from an orchid plant and international coins to a misplaced charging cable and a Steve Irwin Funko Pop.
Despite the tool's capability to search for information, it's important not to immediately trust its answers. For instance, ChatGPT mislabeled my daily multivitamin as a pill for treating erectile dysfunction.
ChatGPT does have limitations. It struggled to identify the artist or location in a random photo of a mural. However, it had no trouble identifying various landmarks in San Francisco, such as Dolores Park and the Salesforce Tower. Although it might still seem a bit gimmicky, individuals exploring new cities, countries, or neighborhoods might enjoy playing around with the visual aspect of ChatGPT.
One of the primary safeguards imposed by OpenAI on this new feature is a limitation on the chatbot's ability to answer questions that identify individuals. "I prioritize user privacy and safety. Identifying real people based on images, even if they are famous, is restricted to maintain these priorities," ChatGPT informed me. While it didn't refuse to answer all questions related to pornography, the chatbot hesitated to provide specific descriptions of adult performers beyond mentioning their tattoos.
It's worth noting that during one conversation I had with the early version of ChatGPT's image feature, it seemed to partially bypass some of the safeguards set by OpenAI. Initially, the chatbot refused to identify a meme featuring Bill Hader. However, it later guessed that an image of Brendan Fraser in "George of the Jungle" was actually a photo of Brian Krause in "Charmed." When questioned about its certainty, the chatbot corrected itself.
In the same conversation, ChatGPT struggled to describe an image from "RuPaul's Drag Race." I shared a screenshot of Kylie Sonique Love, one of the drag queen contestants, and ChatGPT mistakenly identified her as Brooke Lynn Hytes, a different contestant. When I challenged the chatbot's response, it proceeded to guess Laganja Estranja, India Ferrah, Blair St. Clair, and Alexis Mateo.
"I apologize for the oversight and incorrect identifications," ChatGPT responded when I pointed out the repetitive errors. As the conversation continued and I uploaded a photo of Jared Kushner, ChatGPT declined to identify him.
If the safeguards were removed, either through a jailbroken ChatGPT or an open-source model released in the future, the privacy implications could be unsettling. What if every picture of you posted online could be easily linked to your identity with just a few clicks? What if someone could take a photo of you in public without consent and instantly find your LinkedIn profile? Without proper privacy protections for these new image features, women and other marginalized groups are likely to face increased abuse from individuals using chatbots for stalking and harassment.
0 Comments