To Be or Not To Be - Vectorless?

It looks like your code snippet for the reasoning-based retrieval with tree search got cut off. Let's complete it and ensure everything is clear.

Step 5: Reasoning-Based Retrieval with Tree Search

Here’s how you can use an LLM to identify relevant nodes in the document structure based on a user query:

python
 
1async def call_llm(prompt, model="meta-llama/Llama-3.1-8B-Instruct", temperature=0):
2    client = AsyncInferenceClient(token=HF_TOKEN)
3    response = await client.chat.completions.create(
4        model=model,
5        messages=[{"role": "user", "content": prompt}],
6        temperature=temperature
7    )
8    return response.choices[0].message.content.strip()
9
10# Define the search prompt template
11search_prompt_template = """
12Given the document structure below, which sections are most relevant to answer the question: "{query}"?
13
14Document Structure:
15{tree_without_text}
16
17Please provide a ranked list of node IDs that contain information related to the query.
18"""
19
20async def retrieve_relevant_nodes(tree_without_text, query):
21    prompt = search_prompt_template.format(query=query, tree_without_text=json.dumps(tree_without_text))
22    response = await call_ll
23
24[Read the full article at Towards AI - Medium](https://pub.towardsai.net/to-be-or-not-to-be-vectorless-7b2f7cd8a961?source=rss----98111c9905da---4)
25
26---
27
28**Want to create content about this topic?** [Use Nemati AI tools](https://nemati.ai) to generate articles, social posts, and more.