Disclaimer: AI Chatbot Answers are Subject to Error

Generative AI is in its infant stages. Sometimes AI, including AI chatbots, provides inaccurate answers.

• Business Decisions should not be made based on AI answers provided.
• Actions that may cost time or money should not be taken based on AI answers provided. 

Need to verify a chatbot answer?

Contact our Customer Success team and at [email protected] and we will be happy to help you verify an answer.

Document Content

The following information is to give you a better understanding of the steps you can take to protect your organization from inaccurate generative AI answers from a chatbot or any source, should they occur.

Sections

Section 1: Common (generative AI) errors a chatbot might make

Section 2: What you can do to protect yourself against inaccurate AI answers

Section 3: Steps we have taken to ensure our AI Chatbot is providing accurate answers

Section 4: Our commitment to you

Section 1: Common (generative AI) errors a chatbot might make

Here is a list of common errors to be aware of as you incorporate generative AI answers into your tech stack.

• Irrelevant or out of context answers that often stem from a chat bot that has access to external source data that is not relative to the subject(s) discussed with the chatbot.
• Confusion interpreting the question, leading to inaccurate answers.
• Lack of knowledge related to industry terms, product/software terms, and vocabulary used.
• Although rare, some chatbots have been known to hallucinate (make stuff up – tell stories). This often stems from the Large Language Model settings, chatbot architecture, multi-purpose bots, external source data, lack of advanced engineered prompts designed to prevent hallucinations and more.
• Bad answers advising you to take steps or actions that you should not take.
  
  

Section 2: What you can do to protect yourself against inaccurate AI answers

• Use common sense. The best results come from a combination of both Artificial Intelligence and Human Intelligence. Incorporate both for the best answers.
• Train employees in your organization to never make a business decision solely based on an answer generated by AI. Especially business decisions that may cost money, time, or impact your organization in any way.
• If in doubt of an answer, you can do one or more of the following
  • Ask the question another way and compare answers.
  • Just like you would with a human, verify by repeating back what you heard the answer to be.
  • Ask the bot for bullet point steps that can be quickly reviewed and tested without wasting valuable time.
  • Ask the bot to be transferred to a human. It may take a little more time to receive your answer, however, it might be the best option if the answer will impact your organization.
    
    

Section 3: Steps we have already taken for you to providing the highest quality and most accurate answers possible

Generative AI | Chatbot Integrity Steps Taken:

– Our Chatbot, Kosmo was built and is maintained by industry professionals that have experience building chatbots and specialize in AI, not an internal team.

– Our Chatbot uses the latest version of OpenAI. An advanced / paid version with better language and reasoning capabilities.

– Our Chatbot is built on Enterprise platforms like Intercom & Salesforce

– Our chatbot does not have access to the ‘open data’ and/or uncontrolled data on the internet.

– Our chatbot only uses the source data we have provided.

– We provide multiple data sources for the same topic (cross education) to give our AI Chatbot multiple sources of context to pull from to be used on every answer.

– We have provided Prompt engineering & additional backend source data to improve the accuracy of answers.

– We proactively guard against security, privacy, governance, bias, and toxicity.

– We have extensively tested and continue to spot check answers to identify inaccurate answers to be improved, helping our chatbot deliver the best answers possible.

– We have an End User feedback feature in place which allows visitors to submit any wrong answers. We use AI Engineering to quickly fix and train our chatbots to provide accurate answers.

– We put a human layer between AI and directions given that might have an impact on an organization. One example would be asking the chatbot to update your payment information. Even if the Chatbot Agent updates the payment information, a Finance team member must verify before changes are final.

– We adopt the security firewalls of each integrated platform our chatbot is built on and/or access for source data.

– Chatbot Answer history is reviewed by our Support Team who has button labeled “Improve This Answer”. This empowers our Customer Success staff to instantly train the chatbot by providing a better answer that the Chatbot then incorporates into the source data for all future responses.

Section 4: Our commitment to you

As we continue expanding, integrating, and improving our AI technology to assist our customers, we commit to continue upholding these same lofty standards to ensure we are doing everything we can to protect our customers, while delivering the best and most efficient AI experience we can.