Artificial intelligence (AI) has the potential to revolutionize many aspects of our lives, and one area where it is already making a big impact is in knowledge management. In organizations of all sizes, there is a vast amount of weekly documentation and meeting notes that can be difficult for human beings to keep up with. This is where AI can provide significant value.
Large language models, in particular, are well-suited to addressing this challenge. They can read and understand a wide range of data types, and retrieve answers to user questions on the fly. This is why there has been so much discussion about the potential for search engines like Google to be disrupted by large language models. Platforms like ChatGPT and Plexity are already being used by many people to answer their day-to-day questions, and there are also platforms like Link that are focused specifically on knowledge management for corporate data.
Building an AI chatbot that can interact with PDFs, PowerPoints, and spreadsheets is actually quite easy, but building one that can answer even basic questions accurately is much more challenging. There is a significant gap between what people think AI is capable of today and what it is actually capable of.
Over the past few months, I have been experimenting with different ways of building AI applications for various business use cases. One of the most common ways to give a large language model your private knowledge is through fine-tuning or training your own model. This involves baking your knowledge into the model weights, which can provide precise knowledge with fast inference. However, it is not always easy to effectively fine-tune a model, as it requires a good understanding of how to prepare the training data.
Another way to give a large language model your private knowledge is through retrieval augmented generation (RAG). This involves retrieving relevant information and documents from your private database and inserting them into the prompt given to the language model. This can be a simpler and more effective way to provide a language model with the knowledge it needs to answer user questions.
To set up a proper RAG pipeline, you need to start with data preparation. This involves extracting information from your real data sources and converting them into a vector database, which is a special type of database that can understand the semantic relationship between different data points. When a user asks a question, the RAG pipeline will retrieve relevant information from the vector database and send it to the language model.
One of the challenges of RAG is that real-world data can be messy and may not be in a format that is easy for a language model to process. For example, data may include images, diagrams, charts, and tables, which can be difficult to extract and understand. Additionally, different types of data and documentation may require different retrieval methods. For example, spreadsheets and SQL databases may be best searched using vector search, while keyword search or SQL search may be more effective for other types of data.
To mitigate these risks, there are several advanced RAG tactics that you can use. These include techniques like better prompt chunking, which can help to improve the accuracy of the language model, and genetic behavior, which can be used to improve the relevancy of the documents that are retrieved.
As a builder of AI applications, I am always interested in learning more about how AI native startups operate and how they embed AI into every part of their business. A recent study by H Spot surveyed more than 1,000 top startups that are heavily adopting AI to scale their go-to-market process. The study found that AI is being used in a variety of ways, from customer targeting and segmentation to developing intelligent pricing models and improving logistics and supply chain processes.
In conclusion, AI has the potential to provide significant value in the area of knowledge management. Large language models, in particular, are well-suited to this task, as they can read and understand a wide range of data types and retrieve answers to user questions on the fly. To build a reliable and accurate AI application, it is important to carefully consider how you will provide the language model with the knowledge it needs to answer user questions. Whether you choose to fine-tune your own model or use retrieval augmented generation, there are a number of advanced tactics that you can use to improve the accuracy and relevancy of your AI application.