阈值优化

设置 SemanticRouter 或 SemanticCache 后，最好调整 distance_threshold 以从系统中获得最佳性能。RedisVL 提供了辅助类，使这种轻量级优化变得容易。

注意：阈值优化依赖于 python > 3.9。

CacheThresholdOptimizer

假设您设置了具有 X 距离阈值的以下语义缓存并存储了条目

  • prompt: what is the capital of france? response: paris
  • prompt: what is the capital of morocco? response: rabat
from redisvl.extensions.llmcache import SemanticCache

sem_cache = SemanticCache(
    name="sem_cache",                    # underlying search index name
    redis_url="redis://localhost:6379",  # redis connection url string
    distance_threshold=0.5               # semantic cache distance threshold
)

paris_key = sem_cache.store(prompt="what is the capital of france?", response="paris")
rabat_key = sem_cache.store(prompt="what is the capital of morocco?", response="rabat")

这工作得很好，但我们想确保缓存仅适用于适当的问题。如果我们使用一个我们不希望得到响应的问题来测试缓存，我们会发现当前的 distance_threshold 太高了。

sem_cache.check("what's the capital of britain?")

[{'entry_id': 'c990cc06e5e77570e5f03360426d2b7f947cbb5a67daa8af8164bfe0b3e24fe3',
  'prompt': 'what is the capital of france?',
  'response': 'paris',
  'vector_distance': 0.421104669571,
  'inserted_at': 1741039231.99,
  'updated_at': 1741039231.99,
  'key': 'sem_cache:c990cc06e5e77570e5f03360426d2b7f947cbb5a67daa8af8164bfe0b3e24fe3'}]

定义 test_data 并优化

使用 CacheThresholdOptimizer，您可以通过提供以下形式的测试数据来快速调整距离阈值

[
    {
        "query": "What's the capital of Britain?",
        "query_match": ""
    },
    {
        "query": "What's the capital of France??",
        "query_match": paris_key
    },
    {
        "query": "What's the capital city of Morocco?",
        "query_match": rabat_key
    },
]

然后，阈值优化器将有效地执行并根据当前缓存中的内容对不同的阈值进行评分，并自动将缓存的阈值更新为最佳设置

from redisvl.utils.optimize import CacheThresholdOptimizer

test_data = [
    {
        "query": "What's the capital of Britain?",
        "query_match": ""
    },
    {
        "query": "What's the capital of France??",
        "query_match": paris_key
    },
    {
        "query": "What's the capital city of Morocco?",
        "query_match": rabat_key
    },
]

print(f"Distance threshold before: {sem_cache.distance_threshold} \n")
optimizer = CacheThresholdOptimizer(sem_cache, test_data)
optimizer.optimize()
print(f"Distance threshold after: {sem_cache.distance_threshold} \n")

Distance threshold before: 0.5 

Distance threshold after: 0.13050847457627118

我们还可以看到，我们不再匹配不正确的示例

sem_cache.check("what's the capital of britain?")

[]

但仍然匹配高度相关的提示

sem_cache.check("what's the capital city of france?")

[{'entry_id': 'c990cc06e5e77570e5f03360426d2b7f947cbb5a67daa8af8164bfe0b3e24fe3',
  'prompt': 'what is the capital of france?',
  'response': 'paris',
  'vector_distance': 0.0835866332054,
  'inserted_at': 1741039231.99,
  'updated_at': 1741039231.99,
  'key': 'sem_cache:c990cc06e5e77570e5f03360426d2b7f947cbb5a67daa8af8164bfe0b3e24fe3'}]

RouterThresholdOptimizer

与缓存情况非常相似，您可以优化您的路由器。

定义路由

from redisvl.extensions.router import Route

routes = [
        Route(
            name="greeting",
            references=["hello", "hi"],
            metadata={"type": "greeting"},
            distance_threshold=0.5,
        ),
        Route(
            name="farewell",
            references=["bye", "goodbye"],
            metadata={"type": "farewell"},
            distance_threshold=0.5,
        ),
    ]

初始化 SemanticRouter

import os
from redisvl.extensions.router import SemanticRouter
from redisvl.utils.vectorize import HFTextVectorizer

os.environ["TOKENIZERS_PARALLELISM"] = "false"

# Initialize the SemanticRouter
router = SemanticRouter(
    name="greeting-router",
    vectorizer=HFTextVectorizer(),
    routes=routes,
    redis_url="redis://localhost:6379",
    overwrite=True # Blow away any other routing index with this name
)

提供 test_data

test_data = [
    # Greetings
    {"query": "hello", "query_match": "greeting"},
    {"query": "hi", "query_match": "greeting"},
    {"query": "hey", "query_match": "greeting"},
    {"query": "greetings", "query_match": "greeting"},
    {"query": "good morning", "query_match": "greeting"},
    {"query": "good afternoon", "query_match": "greeting"},
    {"query": "good evening", "query_match": "greeting"},
    {"query": "howdy", "query_match": "greeting"},
    {"query": "what's up", "query_match": "greeting"},
    {"query": "yo", "query_match": "greeting"},
    {"query": "hiya", "query_match": "greeting"},
    {"query": "salutations", "query_match": "greeting"},
    {"query": "how's it going", "query_match": "greeting"},
    {"query": "how are you", "query_match": "greeting"},
    {"query": "nice to meet you", "query_match": "greeting"},
    # Farewells
    {"query": "goodbye", "query_match": "farewell"},
    {"query": "bye", "query_match": "farewell"},
    {"query": "see you later", "query_match": "farewell"},
    {"query": "take care", "query_match": "farewell"},
    {"query": "farewell", "query_match": "farewell"},
    {"query": "have a good day", "query_match": "farewell"},
    {"query": "see you soon", "query_match": "farewell"},
    {"query": "catch you later", "query_match": "farewell"},
    {"query": "so long", "query_match": "farewell"},
    {"query": "peace out", "query_match": "farewell"},
    {"query": "later", "query_match": "farewell"},
    {"query": "all the best", "query_match": "farewell"},
    {"query": "take it easy", "query_match": "farewell"},
    {"query": "have a good one", "query_match": "farewell"},
    {"query": "cheerio", "query_match": "farewell"},
    # Null matches
    {"query": "what's the capital of britain?", "query_match": ""},
    {"query": "what does laffy taffy taste like?", "query_match": ""},
]

优化

注意：默认情况下，路由距离阈值优化将使用随机搜索来找到最佳阈值，因为与缓存不同，有许多阈值需要同时优化。

from redisvl.utils.optimize import RouterThresholdOptimizer

print(f"Route thresholds before: {router.route_thresholds} \n")
optimizer = RouterThresholdOptimizer(router, test_data)
optimizer.optimize()

Route thresholds before: {'greeting': 0.5, 'farewell': 0.5} 

Eval metric F1: start 0.438, end 0.719 
Ending thresholds: {'greeting': 1.0858585858585856, 'farewell': 0.5545454545454545}

测试一下

# Query the router with a statement
route_match = router("hi there")
route_match

RouteMatch(name='greeting', distance=0.295984119177)

清理

router.delete()
sem_cache.delete()
评价本页
返回顶部 ↑