It’s been just two weeks since IBM showed off the power and future of Watson in Las Vegas, and the company is at it again. This time in San Francisco, and this time with a deep developer focus.
Taking place at the Innovation Hangar in downtown San Francisco, IBM’s Watson Developer Conference was home to thousands of developers, coming from all over the world to learn about the power of cognitive technology.
Opening the event, IBM CEO Ginni Rometty set the tone for what’s to come in the world of tech.
https://twitter.com/TechPortfolio/status/796414331667841024
Get the TechPortfolio briefing
The month’s fintech and funding news, plus new tools for founders, in one short email.
IBM Chief Digital Officer Bob Lord made a major announcement, impacting designers, developers, data scientists, and programmers.
https://twitter.com/TechPortfolio/status/796427089444536324
Why Bots Were the Talk of 2016
The excitement didn’t come out of nowhere. In 2016, the biggest names in tech were all racing to give developers tools for building chatbots. IBM had just launched its Watson Conversation service at World of Watson, and around the same time, SiliconANGLE noted that Google had bought Api.ai, a startup with a toolkit for building virtual agents, while Microsoft had introduced its own bot development framework that May.
Messaging apps like Slack had become the place where a lot of work actually happened, and bots promised to bring answers, tasks and services straight into those conversations. For developers, that meant a new kind of interface to design for, one built on words instead of buttons and screens.
While the conversation centered around cognitive technology, attendees were especially interested in the topic of bots.
You had me at bots.#WatsonDevCon
— annebot 💯 (@annebot) November 9, 2016
To say people were excited would be an understatement.
Then again, judging by how far language and tone analysis has come with IBM Watson, “excited” might be just the word needed to appropriately generate a positive sentiment data set.
The top bot-related takeaways from #WatsonDevCon
1) Developers who are building bots need to keep it simple. Don’t throw the kitchen sink at the bot dev process.
If you're a #developer, don't build a #bot to solve all company problems, says @mattmakai of @Twilio. Solve one problem first. #WatsonDevCon pic.twitter.com/NaBbsWDBix
— Tech Ed Canada (@TechEdCA) November 9, 2016
Consider the language, tone, and sentiment of users to build-in appropriate responses.
https://twitter.com/TechPortfolio/status/796470374645059585
2) What kinds of user experiences should bots create? Conversational or intelligent conversation? These questions generated a few different perspectives from speakers at #WatsonDevCon.
First, developers need to think about how and where someone will use a bot. Some conversations don’t need to appear to be driven by intelligence if they’re just functional.
https://twitter.com/TechPortfolio/status/796482357998010368
There are a few things that are important to consider in making a bot feel conversational.
https://twitter.com/TechPortfolio/status/796473192873394176
https://twitter.com/TechPortfolio/status/796467621654831105
Or maybe it’s more simple than that. Amir Shevat, Director of Developer Relations at Slack, said bots should pass the beer test, not the Turing test.
https://twitter.com/TechPortfolio/status/796485167875575813
3) The bot revolution is just beginning.
While bots can already do a lot, panelists on the “Intelligent User Engagement with Chat Bots” panel agreed that there is much opportunity ahead in totally untouched areas and industries.
Where are we in evolution of #bots? "We are like cave people talking about spaceships. It's very early."—@ashevat @SlackHQ #WatsonDevCon #AI pic.twitter.com/6VJIeRbhdo
— Tech Ed Canada (@TechEdCA) November 9, 2016
For businesses that are transitioning to use bots and cognitive technology, the opportunity for developers has become clear.
You, the #developers, are the builders of the next century, says @GinniRometty #WatsonDevCon pic.twitter.com/CxgzdwlPvA
— Tech Ed Canada (@TechEdCA) November 9, 2016
But opportunity doesn’t stop there.
The final takeaway from #WatsonDevCon: bots are poised to change more than just the world of tech, they’ll soon change tertiary industries and professions as well.
Great insights by the amazing @chrismessina of @uber at #WatsonDevCon https://t.co/bhAeKvf6xm
— Brian Roemmele (@BrianRoemmele) November 9, 2016
The Beer Test Versus the Turing Test
Shevat’s line is worth unpacking. The Turing test, proposed by computer scientist Alan Turing, asks whether a machine can hold a conversation well enough that a person can’t tell it apart from a human. It’s a test of imitation.
The beer test is a much more down-to-earth standard. It asks whether the bot is something you’d actually enjoy dealing with: helpful, pleasant and easy to talk to. A bot doesn’t need to fool anyone into thinking it’s human to pass. It just needs to do its job without being annoying, which ties straight back to the first takeaway about keeping things simple.
What Happened Next
Most chatbots of that era relied on scripted conversation flows and intent matching, so they worked best on narrow, well-defined tasks. Step outside the script, and they tended to fall back on “Sorry, I didn’t understand that.”
The panelists who said the bot revolution was just beginning were proved right, although the biggest leap came several years later. OpenAI released ChatGPT on November 30, 2022, and large language models made open-ended conversation with software feel normal almost overnight. IBM followed in May 2023 by announcing watsonx, a platform built around foundation models and generative AI.
Slack, whose developer relations director gave us the beer test, went down the same path. In October 2025, Engadget reported that Slack was testing a redesigned, AI-powered Slackbot that people can talk to in everyday language to draft project plans, flag priorities and find information.
The leadership at IBM changed too. Ginni Rometty, who opened the conference, stepped down as IBM’s CEO on April 1, 2020 and retired from the company at the end of that year.
What It Means for Bot Builders
The advice from #WatsonDevCon has aged surprisingly well. Today’s AI models can say almost anything, which makes “keep it simple” more important, not less. A bot that tries to do everything is harder to test and easier to break.
Thinking about tone, and about where and how someone will use a bot, still separates good assistants from frustrating ones. And the beer test is still a useful gut check: if you wouldn’t want to spend a few minutes chatting with your bot, your users probably won’t either.
Frequently Asked Questions
Where was IBM's Watson Developer Conference held?
The conference took place at the Innovation Hangar in downtown San Francisco, about two weeks after IBM's World of Watson event in Las Vegas. Thousands of developers attended.
What was the main topic at #WatsonDevCon?
The conversation centered on cognitive technology, but attendees were especially interested in bots, including how to design them and what kinds of user experiences they should create.
What is the beer test for bots?
Amir Shevat, Director of Developer Relations at Slack, said bots should pass the beer test, not the Turing test. Rather than trying to pass as human, a bot should simply be helpful and pleasant enough that people enjoy dealing with it.
What advice did speakers give developers building bots?
Speakers said developers should keep bots simple, consider the language, tone and sentiment of users, and think about how and where someone will use a bot before deciding how intelligent it needs to seem.
What happened to chatbots after 2016?
Most bots of that era relied on scripted conversation flows. OpenAI released ChatGPT on November 30, 2022, IBM announced its watsonx platform in May 2023, and in October 2025 Slack began testing a redesigned, AI-powered Slackbot.