Vector database consulting support

Step 1: Create an index with dense_vector mapping Each document in our simple dataset will have:

Step 2: Index documents with embeddings

First, index a single document to understand the document structure.

Step 3: Search documents with embeddings

Now you can query these document vectors using a knn retriever. knn is a type of vector search, which finds the k most similar documents to a query vector. Here we’re simply using a raw vector for the query text, for demonstration purposes.

The dense_vector field type stores dense vectors of numeric values. Dense vector fields are primarily used for k-nearest neighbor (kNN) search.

A k-nearest neighbor (kNN) search finds the k nearest vectors to a query vector, as measured by a similarity metric.

Automatically quantize vectors for kNN search 

The dense_vector type supports quantization to reduce the memory footprint required when searching float vectors. The three following quantization strategies are supported:

Vector queries are specialized queries that work on vector fields to efficiently perform semantic search

Elasticsearch offers the usage of a wide range of NLP models, including  both dense and sparse vector models. Your choice of the language model is critical for implementing semantic search successfully.

After you decide which model you want to use for implementing semantic search, you need to deploy the model in Elasticsearch.

• Generate text embeddings
• Search the data
• Beyond semantic search with hybrid search

Reduce vector memory foot-print

Reduce vector dimensionality

Exclude vector fields from _source

Ensure data nodes have enough memory

Avoid page cache thrashing by using modest readahead values on Linux

Avoid heavy indexing during searches