There is so much information about Generative AI (Gen AI) right now it can be hard as a Knowledge Manager to know what is important and what can be left to the technical experts. As we navigate this complex landscape, it’s essential for us to understand the key concepts that will enable us to successfully guide IT departments and vendors in building solutions that support knowledge work.
We’ve been hearing a lot about how AI can help capture and share expertise, but there’s often confusion, especially when it comes to tacit knowledge. Tacit knowledge isn’t just a collection of facts or how-to guides—it’s the wisdom experts use to solve complex problems where there are no clear rules. Today, I want to explain how we can use Generative AI (GenAI) to support this process in a way that helps us capture and share deep expertise while also addressing some practical concerns, like the legal risks of giving direct advice.
The Information vs. Knowledge Approach
One of the challenges is that many people approach this issue from an information perspective. They think that if you summarize all the documents, standards, and procedures, you’ll have the same value as an expert. But that’s not how experts work. Experts solve problems that don’t have clear answers—they go beyond the rulebook.
When it comes to using AI, we have to go beyond treating it like a static information system. Summarizing all the data in the world won’t give you expert-level judgment for unique, unfamiliar situations. This is where concepts like RAG (retrieval-augmented generation) and DAG (data-augmented generation) come into play. RAG helps AI pull in unstructured information from documents, policies, and customer data in real time, while DAG connects AI to structured data from relational databases, like product catalogs or financial records, to offer precise, data-driven insights. Together, these approaches allow AI to access both structured and unstructured knowledge, making it far more powerful in supporting decision-making.
The Inference Power of GenAI
The real strength of GenAI is its inference power. It doesn’t just pull facts—it can make connections between ideas in a way that mimics how experts think. This is where the concept of a query bot becomes powerful. Unlike a chatbot that simply tries to answer your question, a query bot can be made to use the information from your business’s RAG and DAG systems and apply inferences to lead users through a series of questions.
These questions are designed at first to better understand context, and later to guide you, much like an expert mentor, helping you think more critically about the problem. Rather than giving a straightforward answer, the AI uses these prompts to shift your perspective, so you arrive at the solution that fits your local context. This reflective process is key to handling complex or novel problems where predefined answers don’t exist, and it has a wonderful side-effect of transferring tacit knowledge right at the problems solvers point of need when they are most open to learning.
Pre-Prompting and Workflow Design
This guiding process can be enhanced with pre-prompting, where the AI is set up with initial prompts that help it steer the conversation toward reflective dialogue. For example, when asked, “How do I solve X?”, the AI might be encouraged to first respond, “What constraints are you working with? What resources are available?” By asking these kinds of questions, it ensures that the user reflects on the local context before moving to a solution. Next it might respond with more questions, but this time focused on expanding the user's viewpoint so potential solutions or relevant resources become obvious next steps.
Legal Advantages of Expert AI Systems approach
A side benefit of this expert AI system approach is the legal advantage it offers. In regulated industries, directly providing advice can lead to serious legal risks. If an AI system offers a specific solution, and it’s applied incorrectly, the organization might face legal consequences.
An expert AI system, however, minimises this risk by guiding users to think through the problem themselves, based on the context. It doesn’t provide direct instructions; instead, it leads the user to validated documents, standards, or references. Ultimately, the end user is responsible for applying that information, reducing the legal burden on the organization.
Building your GenAI Skillset as a Knowledge Manager
As I watch more and more knowledge managers beginning their AI journey, they need to navigate a jungle of information about AI and how it works. To be honest, much of it has a large dose of hype or fear. Of the good information out there, much of it is focused on information management and governance (See below for a few of my favourite GenAI YouTube channels). As we introduce Gen AI into our knowledge management ecosystems, data quality and currency is important, but unlike decades past, we can now use GenAI itself to cleanse, markup, summarise and label sensitive data. These are no longer massive manual projects for the users and/or the KM team.
Data quality and validity is undoubtably important, but for knowledge managers, there are several other concepts important to understand:
- LLMs (Large Language Models) are just engines that power these systems. The real value comes from designing workflows that combine AI’s inference power with tools like RAG, DAG, and pre-prompting to build meaningful expertise systems. Take Microsoft Copilot, for example—it uses the LLM not just to answer questions but to access relevant data, and then another time through the LLM to validate and refine answers. Roche Pharmaceutical is doing something similar, using AI workflows that process information through several layers of validation before submitting drug study documentation to regulatory bodies like the FDA.
- Next, it is important to know the difference between a Chat-bot and a Query-bot. Unlike a Chat-bot which will gladly talk with confidence about any topic, a Query-bot strictly answers from the source documents. Faced with no relevant data in it's sources, a Query-bot will simply say it lacks sufficient data to answer that question. In technical environments this is a much preferred behaviour.
- With all this focus of technology, the real key to capturing tacit knowledge is the expertise interviews themselves. The amazing Vanessa Liu from Sugarwork in New York is doing a wonderful job of this because she is applying long known KM principles:
- She lets peers interview the experts, people who have background knowledge and a relationship with them.
- She captures the entire dialogue, not just notes, and she retains the full content through processing and ingestion. GenAI works far better from this long narrative form data.
- She uses interview templates that provides a guide, but doesn't restrict the interviewer straying down rabbit holes, stories and old scenarios in order to capture nuggets of information, revealing key relationships, approaches, mindsets and problem solving techniques as well as information about that specific problem.
- Right now her clients conduct interviews periodically - every 6 or 12 months - but as tools and agents improve, we could scrape data from meeting logs, workshop transcriptions, peer reviews and debriefs to constantly augment the experts' model.
- Finally, a key part of capturing tacit knowledge in a GenAI system is building the semantic layer. This includes concepts like taxonomies, glossaries and ontologies that help the LLM understand local language, concepts and relationships between words. For example if you were working with animal keepers in a zoo, the terms "dog" and "Sheep dog" are obviously related, but the terms "Goanna" and "Lizard" aren't syntactically alike at all. A taxonomy links the species, helping with retrieval from your documents and guiding the LLM to respect related constraints and insights. Check out the Enterprise Knowledge blog posts below for some excellent information on the Semantic Layer and how they are used for AI. Many knowledge managers will be extremely intimate with these concepts already, but be aware there are subtle differences when building these tools for LLM consumption.
Conclusion
For knowledge managers, one real promise of GenAI is in capturing more tacit knowledge than ever possible. As you navigate your own AI journey, consider the tacit approach: capturing expert narrative and then guiding users through reflection and inquiry, not just giving ready-made answers. By combining expertise interviews, RAG and DAG for local data sources, and a well-designed workflow, we can begin to replicate the kind of mentorship that experts offer in the real world, giving users context-aware guidance. This approach not only mitigates legal risk but also helps capture some of the deep, tacit expertise that can be applied to solve real-world problems, making it an advanced form of succession planning; something usually preserved only for high level executives and particularly useful for high impact/low frequency situations. Well, ok, maybe more like having an apprentice who spent his time under a master, but it still sits beyond what KM has offered so far and the technologies keep improving.
By viewing AI as a tool for inquiry, not just for answers, we can build systems that help people solve complex problems while maintaining compliance, security, and accuracy. At the heart of that is the knowledge managers long-spoken importance of tacit knowledge.
Extra Resources
Some of my favourite Gen AI YouTube channels
Related Videos
AnythingLLM:
Tim Carambat. Creator of an AnythingLLM Desktop AI system for personal or business use.
Blazing Zebra:
A technical site allowing advanced knowledge managers to delve deeper into how these systems work, so they can better guide Executives and IT professionals alike.
NetworkChuck:
A great way to get into the actual implementation of your own LLM systems in a very easy-to-understand way. Chuck’s quirky voice and sense of humor make the content engaging.
Stanford Human AI Site:
Touches on a full range of GenAI related topics including human-centered AI concepts, ethics, and governance considerations.
Understanding the Semantic Layer
Check out these three fantastic blog-posts from Enterprise Knowledge to understand Semantic Layers:
0 Comments