AI & Machine Learning

To Be or Not To Be - Vectorless?

47 sec read184 views0 listens

It looks like your code snippet for the reasoning-based retrieval with tree search got cut off. Let's complete it and ensure everything is clear.

Here’s how you can use an LLM to identify relevant nodes in the document structure based on a user query:

python
1async def call_llm(prompt, model="meta-llama/Llama-3.1-8B-Instruct", temperature=0): 2 client = AsyncInferenceClient(token=HF_TOKEN) 3 response = await client.chat.completions.create( 4 model=model, 5 messages=[{"role": "user", "content": prompt}], 6 temperature=temperature 7 ) 8 return response.choices[0].message.content.strip() 9 10# Define the search prompt template 11search_prompt_template = """ 12Given the document structure below, which sections are most relevant to answer the question: "{query}"? 13 14Document Structure: 15{tree_without_text} 16 17Please provide a ranked list of node IDs that contain information related to the query. 18""" 19 20async def retrieve_relevant_nodes(tree_without_text, query): 21 prompt = search_prompt_template.format(query=query, tree_without_text=json.dumps(tree_without_text)) 22 response = await call_ll 23 24[Read the full article at Towards AI - Medium](https://pub.towardsai.net/to-be-or-not-to-be-vectorless-7b2f7cd8a961?source=rss----98111c9905da---4) 25 26--- 27 28**Want to create content about this topic?** [Use Nemati AI tools](https://nemati.ai) to generate articles, social posts, and more.
Views184
Tags
On this page
Ali Nemati
Ali NematiBlogUI.author_articles
BlogUI.view_profile