Haystack

🌾 Overview of the Haystack integration

Haystack is an open-source Python framework by deepset that’s used for building production-ready natural language processing (NLP) applications. Developers and enterprise businesses can easily try out the latest models in NLP, while maintaining flexibility and ease-of-use.

The Gradient and Haystack integration enables users to use their Gradient models for desired tasks in a Haystack workflow. Within the Haystack framework, users can call the Gradient API to run inference on their own LLMs that are built on the Gradient platform or call the Gradient Embeddings API to generate vector embeddings from text.

Installation

To install the Haystack and the Gradient integration, run pip install gradient-haystack

Indexing Documents into a Vector Database with Gradient Embeddings

You can create an indexing pipeline with GradientDocumentEmbedder to index documents and their embeddings into any one of their document stores, using Gradient embedding models.

To create a GradientDocumentEmbedder:

import os
from gradient_haystack.embedders.gradient_document_embedder import GradientDocumentEmbedder

document_embedder = GradientDocumentEmbedder(
    access_token=os.environ["GRADIENT_ACCESS_TOKEN"],
    workspace_id=os.environ["YOUR_WORKSPACE_ID"],
)

For example, below is a pipeline that indexes documents into an InMemoryDocumentStore:

from haystack import Pipeline
from haystack.components.preprocessors import DocumentSplitter
from haystack.components.writers import DocumentWriter
from haystack.document_stores import InMemoryDocumentStore

document_store = InMemoryDocumentStore()
writer = DocumentWriter(document_store=document_store)

indexing_pipeline = Pipeline()
indexing_pipeline.add_component(instance=document_embedder, name="document_embedder")
indexing_pipeline.add_component(instance=writer, name="writer")

indexing_pipeline.connect("document_embedder", "writer")

indexing_pipeline.run({"document_embedder": {"documents": docs}})

Retrieval Augmented Generation with Gradient Embeddings and Generators

You can create a RAG pipeline with GradientTextEmbedder for the retrieval and GradientGenerator for the generation steps.

To create these two components:

import os
from gradient_haystack.embedders.gradient_text_embedder import GradientTextEmbedder
from gradient_haystack.generator.base import GradientGenerator

text_embedder = GradientTextEmbedder(
    access_token=os.environ["GRADIENT_ACCESS_TOKEN"],
    workspace_id=os.environ["YOUR_WORKSPACE_ID"],
)
generator = GradientGenerator(
    access_token=os.environ["GRADIENT_ACCESS_TOKEN"],
    workspace_id=os.environ["YOUR_WORKSPACE_ID"],
    base_model_slug="llama2-7b-chat",
    max_generated_token_count=350,
)

For example, here is a full RAG pipeline that does question-answering on documents in an InMemoryDocumentStore:

from haystack.components.retrievers import InMemoryEmbeddingRetriever
from haystack.components.builders import PromptBuilder

prompt = """ Answer the query, based on the
content in the documents.
{{documents}}
Query: {{query}}
"""

retriever = InMemoryEmbeddingRetriever(document_store=document_store)
prompt_builder = PromptBuilder(template=prompt)

rag_pipeline = Pipeline()

rag_pipeline.add_component(instance=text_embedder, name="text_embedder")
rag_pipeline.add_component(instance=retriever, name="retriever")
rag_pipeline.add_component(instance=prompt_builder, name="prompt_builder")
rag_pipeline.add_component(instance=generator, name="generator")

rag_pipeline.connect("text_embedder", "retriever")
rag_pipeline.connect("retriever", "prompt_builder")
rag_pipeline.connect("prompt_builder", "generator")

question = "What are the steps for creating a custom component?"
result = rag_pipeline.run(
    data={"text_embedder":{"text": question}, "prompt_builder":{"query": question}},
)

Haystack documentation

You can read documentation about the integration here, or try it our for yourself in this example Colab.