VectorStore/Q&A, quickstart¶

NOTE: this uses Cassandra's experimental "Vector Similarity Search" capability. At the moment, this is obtained by building and running an early alpha from a specific branch of the codebase.

In [1]:

from langchain.indexes import VectorstoreIndexCreator
from langchain.text_splitter import (
    CharacterTextSplitter,
    RecursiveCharacterTextSplitter,
)
from langchain.docstore.document import Document
from langchain.document_loaders import TextLoader

from langchain.indexes import VectorstoreIndexCreator from langchain.text_splitter import ( CharacterTextSplitter, RecursiveCharacterTextSplitter, ) from langchain.docstore.document import Document from langchain.document_loaders import TextLoader

The following line imports the Cassandra flavor of a LangChain vector store:

In [2]:

from langchain.vectorstores.cassandra import Cassandra

from langchain.vectorstores.cassandra import Cassandra

As usual, a database connection is needed to access Cassandra. The following assumes that a vector-search-capable Cassandra cluster is running locally. Adjust as needed.

In [3]:

from cqlsession import getLocalSession, getLocalKeyspace
localSession = getLocalSession()
localKeyspace = getLocalKeyspace()

from cqlsession import getLocalSession, getLocalKeyspace localSession = getLocalSession() localKeyspace = getLocalKeyspace()

Both an LLM and an embedding function are required.

Below is the logic to instantiate the LLM and embeddings of choice. We choose to leave it in the notebooks for clarity.

In [4]:

from llm_choice import suggestLLMProvider

llmProvider = suggestLLMProvider()
# (Alternatively set llmProvider to 'VertexAI', 'OpenAI' ... manually if you have credentials)

if llmProvider == 'VertexAI':
    from langchain.llms import VertexAI
    from langchain.embeddings import VertexAIEmbeddings
    llm = VertexAI()
    myEmbedding = VertexAIEmbeddings()
    print('LLM+embeddings from VertexAI')
elif llmProvider == 'OpenAI':
    from langchain.llms import OpenAI
    from langchain.embeddings import OpenAIEmbeddings
    llm = OpenAI(temperature=0)
    myEmbedding = OpenAIEmbeddings()
    print('LLM+embeddings from OpenAI')
else:
    raise ValueError('Unknown LLM provider.')

from llm_choice import suggestLLMProvider llmProvider = suggestLLMProvider() # (Alternatively set llmProvider to 'VertexAI', 'OpenAI' ... manually if you have credentials) if llmProvider == 'VertexAI': from langchain.llms import VertexAI from langchain.embeddings import VertexAIEmbeddings llm = VertexAI() myEmbedding = VertexAIEmbeddings() print('LLM+embeddings from VertexAI') elif llmProvider == 'OpenAI': from langchain.llms import OpenAI from langchain.embeddings import OpenAIEmbeddings llm = OpenAI(temperature=0) myEmbedding = OpenAIEmbeddings() print('LLM+embeddings from OpenAI') else: raise ValueError('Unknown LLM provider.')

LLM+embeddings from OpenAI

A minimal example¶

The following is a minimal usage of the Cassandra vector store. The store is created and filled at once, and is then queried to retrieve relevant parts of the indexed text, which are then stuffed into a prompt finally used to answer a question.

Note: for the time being you have to explicitly turn on this experimental flag on the cassio side:

In [5]:

import cassio
cassio.globals.enableExperimentalVectorSearch()

import cassio cassio.globals.enableExperimentalVectorSearch()

The following creates an "index creator", which knows about the type of vector store, the embedding to use and how to preprocess the input text:

(Note: stores built with different embedding functions will need different tables. This is why we append the llmProvider name to the table name in the next cell.)

In [6]:

index_creator = VectorstoreIndexCreator(
    vectorstore_cls=Cassandra,
    embedding=myEmbedding,
    text_splitter=CharacterTextSplitter(
        chunk_size=400,
        chunk_overlap=0,
    ),
    vectorstore_kwargs={
        'session': localSession,
        'keyspace': localKeyspace,
        'table_name': 'vs_test1_' + llmProvider,
    },
)

index_creator = VectorstoreIndexCreator( vectorstore_cls=Cassandra, embedding=myEmbedding, text_splitter=CharacterTextSplitter( chunk_size=400, chunk_overlap=0, ), vectorstore_kwargs={ 'session': localSession, 'keyspace': localKeyspace, 'table_name': 'vs_test1_' + llmProvider, }, )

Loading a local text (a short story by E. A. Poe will do)

In [7]:

loader = TextLoader('texts/amontillado.txt', encoding='utf8')

loader = TextLoader('texts/amontillado.txt', encoding='utf8')

This takes a few seconds to run, as it must calculate embedding vectors for a number of chunks of the input text:

In [8]:

# Note: this is a temporary replacement for next cell, pending resolution of this error from VertexAI:
#    "InvalidArgument: 400 5 instance(s) is allowed per prediction. Actual: <N>"
# (i.e. one cannot request embedding of more than 5 strings in a single call)
if llmProvider == 'VertexAI':
    from langchain.indexes.vectorstore import VectorStoreIndexWrapper
    docs = loader.load()
    subdocs = index_creator.text_splitter.split_documents(docs)
    #
    print(f'sd {0} ...', end=' ')
    vs = index_creator.vectorstore_cls.from_documents(
        subdocs[:1],
        index_creator.embedding,
        **index_creator.vectorstore_kwargs,
    )
    print('done.')
    for sdi, sd in enumerate(subdocs[1:]):
        print(f'sd {sdi+1} ...', end=' ')
        vs.add_texts(texts=[sd.page_content], metadata=[sd.metadata])
        print('done.')
    #
    index = VectorStoreIndexWrapper(vectorstore=vs)

# Note: this is a temporary replacement for next cell, pending resolution of this error from VertexAI: # "InvalidArgument: 400 5 instance(s) is allowed per prediction. Actual: " # (i.e. one cannot request embedding of more than 5 strings in a single call) if llmProvider == 'VertexAI': from langchain.indexes.vectorstore import VectorStoreIndexWrapper docs = loader.load() subdocs = index_creator.text_splitter.split_documents(docs) # print(f'sd {0} ...', end=' ') vs = index_creator.vectorstore_cls.from_documents( subdocs[:1], index_creator.embedding, **index_creator.vectorstore_kwargs, ) print('done.') for sdi, sd in enumerate(subdocs[1:]): print(f'sd {sdi+1} ...', end=' ') vs.add_texts(texts=[sd.page_content], metadata=[sd.metadata]) print('done.') # index = VectorStoreIndexWrapper(vectorstore=vs)

In [9]:

if llmProvider != 'VertexAI':
    index = index_creator.from_loaders([loader])

if llmProvider != 'VertexAI': index = index_creator.from_loaders([loader])

In [10]:

query = "Who is Luchesi?"
index.query(query, llm=llm)

query = "Who is Luchesi?" index.query(query, llm=llm)

Out[10]:

" Luchesi is a friend of Fortunato's who has a critical turn and is known for his taste in wine."

Spawning a "retriever" from the index¶

In [11]:

retriever = index.vectorstore.as_retriever(search_kwargs={
    'k': 2,
})

retriever = index.vectorstore.as_retriever(search_kwargs={ 'k': 2, })

In [12]:

retriever.get_relevant_documents(
    "Check the motto of the Montresors"
)

retriever.get_relevant_documents( "Check the motto of the Montresors" )

Out[12]:

[Document(page_content='"A huge human foot d\'or, in a field azure; the foot crushes a serpent\nrampant whose fangs are imbedded in the heel."\n\n"And the motto?"\n\n"_Nemo me impune lacessit_."\n\n"Good!" he said.', metadata={'source': 'texts/amontillado.txt'}),
 Document(page_content='He raised it to his lips with a leer.  He paused and nodded to me\nfamiliarly, while his bells jingled.\n\n"I drink," he said, "to the buried that repose around us."\n\n"And I to your long life."\n\nHe again took my arm, and we proceeded.\n\n"These vaults," he said, "are extensive."\n\n"The Montresors," I replied, "were a great and numerous family."\n\n"I forget your arms."', metadata={'source': 'texts/amontillado.txt'})]