Cody Terminologies 101

A beginner’s guide to understanding Cody Terminologies in a simplified manner.

Before you get started with building your own GPT-based bot for your business using Cody, it is essential that you understand the core terminologies so that the flexibility our bot-builder offers does not intimidate you. You do not need to be an AI expert or have prior knowledge in this domain to understand these terminologies, as they will be explained in the most simplified way. Think of this as a beginner’s guide or ‘Cody For Dummies‘.

This blog will not cover setting up Cody or importing data into Cody since we have already posted a few blogs that will help you with the setup. In the upcoming blogs, we will delve deeper into those subjects. Without any further ado, let’s understand the different terminologies you will come across while building your bot.


Cody offers two ways to define your bot’s personality: Template Mode and Advanced Mode. The personality of your bot establishes the context and behaviour of your bot to ensure it generates helpful responses that align with your expectations and prevent any unforeseen hallucinations.

❓Hallucination refers to the phenomenon where a language model generates responses that may be imaginative, speculative, or not grounded in factual information.

Template Mode

This mode is useful for those businesses who want a plug-and-play solution to building a bot. Our developers have created bot templates catering to different business domains such as Support, Training, HR and Marketing.

Cody Template Mode Personality

Select any domain from the following and Cody will handle the rest. For added flexibility, Cody also provides settings that can be used to customize the personality to some extent.

For eg. If you choose to go ahead with IT Support as your bot personality, you can change your bot’s nickname, and decide how it will be handling unanswerable queries (a default case) and the languages you need the bot to converse in.

Cody Template Mode Settings

This mode is still under development and will be soon getting updated to be able to cater to multiple other business domains with a host of customization features.

Need more flexibility? Don’t worry, we have got you covered with our Advanced Mode.

Advanced Mode

The Advanced Bot Builder gives you the freedom to customize multiple parameters and lets you build the perfect bot that suits your use case. Currently, you can customize the following parameters:

  1. Prompt
  2. Relevance Score
  3. Token Distribution
  4. Persistent Prompt
  5. Reverse Vector Search


The prompt will define the personality of your bot. To simplify the prompting process, consider the bot as an employee at your business. Although there is no specific structure for writing a personality prompt, we have prepared a list of parameters for your reference.

A sample prompt:

“You are the waiter at a pizza joint. Maintain a professional and friendly demeanor throughout all interactions, ensuring that users feel comfortable and supported. Remember to convey a sense of expertise and confidence in your responses. Additionally, I encourage you to actively promote our premium pizzas whenever appropriate. Do not refer to any menu sources other than the ones provided in the knowledge base. While recommending pizzas, state their prices and any offers that are applicable for them, too.”

Relevance Score

The Relevance Score reflects the degree of similarity between the user’s query and Cody’s response. Using semantic search, Cody compares the user’s query with the data present in the knowledge base. A higher relevance score will result in a precise answer but will compromise on understanding the overall context of the query and vice versa. In simple words, the relevance score is the degree to which the AI fears making mistakes and taking risks while responding.

Relevance Score Slider

Token Distribution

The token is the computational currency for large language models like the GPT family. The query (input statement) asked by the user is broken down into blocks of characters known as ‘tokens’. As AI models are really resource-intensive, in order to address the computational constraints and memory limitations, these models have a certain limit to the input data that can be processed and generated. This limit is the ‘context window’.

Context Window Illustration

Cody uses the GPT family of models and the number of tokens available are limited. The token distribution feature helps in micro-managing the usage of tokens for different purposes.

They are mainly divided into Context, History and Response Generation.

  1. Context: The tokens required for understanding the user query and knowledge base context.
  2. History: The tokens required for adding context to the user query using the chat history.
  3. Response Generation: The tokens required for assessing the coherence, grammar, and semantic validity of the generated text.

Token Distribution Slider

For the highest accuracy is important that the context makes up a large portion of the Token Distribution.

Persistent Prompt

By continuously reinforcing the prompt (personality of the bot), you create a form of conversational context and constraint that keeps the AI on track and helps maintain compliance with the desired outcomes. It acts as a reminder for the AI to stay within the predefined boundaries and provide responses that are relevant, accurate, and aligned with your goals.

Reverse Vector Search

Reverse Vector Search is a feature specifically designed for businesses that utilize Cody for employee training purposes. It seamlessly integrates previously generated AI responses with user queries, creating a comprehensive contextual understanding that enhances the quality and relevance of subsequent responses. It is especially valuable when using Cody for step-by-step guides and instructional purposes.


This image describes how reverse vector search works.


To Wrap It Up

When you have a good grasp of these terminologies, you’ll be able to refine your bot’s responses and make them even more personalized. Stay tuned for upcoming blogs offering further guidance, updates, and additional information to unleash your Cody AI bot’s full potential. For more resources, you can also check out our Help Center and join our Discord community.



More From Our Blog

Anthropic's Claude 3.5 Sonnet Released: Better Than GPT-4o?

Anthropic's Claude 3.5 Sonnet Released: Better Than GPT-4o?

Claude 3.5 Sonnet is the latest model in the Claude 3.5 family of large language models (LLMs). Introduced by Anthropic in March 2024, it marks a significant leap forward. This model surpasses its predecessors and notable competitors like GPT-4o and ...

Read More
RAG-as-a-Service: Unlock Generative AI for Your Business

RAG-as-a-Service: Unlock Generative AI for Your Business

With the rise of Large Language Models (LLMs) and generative AI trends, integrating generative AI solutions in your business can supercharge workflow efficiency. If you’re new to generative AI, the plethora of jargon can be intimidating. This b...

Read More

Build Your Own Business AI

Get Started Free