Build RAG-based generative AI applications in AWS using Amazon FSx for NetApp ONTAP and Amazon Bedrock

This text was co-authored with NetApp’s Michael Shaul and Sasha Korman.

Generative synthetic intelligence (AI) purposes are sometimes constructed utilizing a method known as Retrieval-Augmented Technology (RAG), which permits the bottom mannequin (FM) to entry further information not out there throughout coaching. This information is used to complement generative AI cues to offer extra context-specific and correct responses with out the necessity to always retrain the FM, whereas additionally rising transparency and minimizing hallucinations.

On this article, we exhibit an answer utilizing Amazon FSx for NetApp ONTAP and Amazon Bedrock to offer a easy, quick, and safe solution to deliver company-specific unstructured person profile information to Amazon Bedrock on AWS. of generative AI purposes delivering RAG experiences.

Our resolution makes use of the FSx for ONTAP file system because the supply of unstructured information and constantly populates the Amazon OpenSearch Serverless vector database with the person’s present recordsdata and folders and related metadata. RAG situations might be applied utilizing Amazon Bedrock by enriching generative AI hints utilizing the Amazon Bedrock API and company-specific information retrieved from the OpenSearch Serverless vector database.

When utilizing RAG to develop generative AI purposes, equivalent to Q&A chatbots, clients are additionally involved about guaranteeing information safety and stopping finish customers from querying info from unauthorized sources. Our resolution additionally makes use of FSx for ONTAP to permit customers to increase their present information safety and entry mechanisms to boost mannequin responses from Amazon Bedrock. We use FSx for ONTAP as a supply of related metadata, particularly the person safety entry management record (ACL) configuration connected to its recordsdata and folders, and populate that metadata into OpenSearch Serverless. By combining entry management operations with file occasions to inform RAG purposes of latest and altered information on the file system, our resolution demonstrates how FSx for ONTAP permits Amazon Bedrock to detect solely particular information linked to our generated AI. The person makes use of the embedded software within the authorization file.

AWS serverless providers make it simple so that you can deal with constructing generative AI purposes by offering autoscaling capabilities, built-in excessive availability, and a pay-per-use billing mannequin. Occasion-driven computing with AWS Lambda is good for compute-intensive, on-demand duties equivalent to file embedding and versatile giant language mannequin (LLM) orchestration, and Amazon API Gateway supplies an API interface that permits pluggable front-ends and occasion jurisprudence Grasp’s Drive Name. Our resolution additionally demonstrates the right way to use API Gateway and Lambda to construct a scalable, automated, API-driven serverless software layer on prime of Amazon Bedrock and FSx for ONTAP.

Resolution overview

This resolution supplies FSx for ONTAP multi-AZ file system with storage digital machines (SVMs) joined to an AWS Managed Microsoft AD area. OpenSearch Serverless vector search assortment supplies scalable and high-performance similarity search capabilities. We use an Amazon Elastic Compute Cloud (Amazon EC2) Home windows server because the SMB/CIFS consumer for the FSx for ONTAP quantity and configure information sharing and ACLs for the SMB shares within the quantity. We used these profiles and ACLs to check permission-based embedded entry in Amazon Bedrock’s RAG state of affairs.

The embedded container element of our resolution is deployed on an EC2 Linux server and put in as an NFS consumer on an FSx for ONTAP quantity. It recurrently migrates present recordsdata and folders and their safety ACL configurations to OpenSearch Serverless. It populates the index within the OpenSearch Serverless vector search assortment utilizing company-specific information (and related metadata and ACLs) from the NFS share on the FSx for ONTAP file system.

The answer implements a RAG Retrieval Lambda perform that permits RAG utilizing Amazon Bedrock by utilizing the Amazon Bedrock API with company-specific profiles and related metadata (together with ACLs) retrieved from an OpenSearch Serverless index populated by the embedded container. Enrich generative AI immediate elements. The RAG Retrieval Lambda perform shops the dialog historical past of person interactions in an Amazon DynamoDB desk.

Finish customers work together with the answer by submitting pure language prompts via a chatbot software or instantly via the API gateway interface. The chatbot software container is constructed utilizing Streamlit and fronted by AWS Software Load Balancer (ALB). When a person submits a pure language immediate to the chatbot UI utilizing ALB, the chatbot container interacts with the API gateway interface after which calls the RAG Retrieval Lambda perform to acquire the person’s response. Customers may also submit immediate requests on to the API gateway and get responses. We exhibit permission-based entry to a RAG file by explicitly retrieving the person’s SID after which utilizing that SID in a chatbot or API gateway request. The RAG retrieval Lambda perform then matches the SID to the Home windows ACL configured for the file. As an extra authentication step in a manufacturing setting, you might also wish to authenticate the person towards the id supplier after which pair the person with the permissions configured for the file.

The diagram beneath illustrates the end-to-end circulate of our resolution. We first arrange information shares and ACLs utilizing FSx for ONTAP, that are then scanned periodically by the embedded container. The embedding container splits the file into chunks and builds vector embeddings from these chunks utilizing the Amazon Titan Embeddings mannequin. It then shops these vector embeddings and related metadata in a vector database by populating the indexes within the vector assortment in OpenSearch Serverless. The diagram beneath illustrates the end-to-end course of.

The next structure diagram illustrates the varied components of our resolution.

Stipulations

Full the next prerequisite steps:

1. Be sure you have mannequin entry permissions in Amazon Bedrock. On this resolution, we use Anthropic Claude v3 Sonnet on Amazon Bedrock.
2. Set up the AWS Command Line Interface (AWS CLI).
3. Set up Docker.
4. Set up Terraform.

Deploy resolution

The answer is out there for obtain on this GitHub repository. Cloning the repository and utilizing the Terraform template will present all elements with the required configuration.

1. Clone this resolution’s repository:

   sudo yum set up -y unzip
   git clone https://github.com/aws-samples/genai-bedrock-fsxontap.git
   cd genai-bedrock-fsxontap/terraform

2. From the terraform folder, deploy your complete resolution utilizing Terraform:

   terraform init
   terraform apply -auto-approve

This course of could take 15-20 minutes to finish. When full, the output of the terraform command ought to appear like this:

api-invoke-url = "https://9ng1jjn8qi.execute-api.<area>.amazonaws.com/prod"
fsx-management-ip = toset([
"198.19.255.230",])
fsx-secret-id = "arn:aws:secretsmanager:<area>:<account-id>:secret:AmazonBedrock-FSx-NetAPP-ONTAP-a2fZEdIt-0fBcS9"
fsx-svm-smb-dns-name = "BRSVM.BEDROCK-01.COM"
lb-dns-name = "chat-load-balancer-2040177936.<area>.elb.amazonaws.com"

Load information and set permissions

To check the answer we are going to use an EC2 Home windows server (ad_host) as an SMB/CIFS consumer to an FSx for ONTAP quantity to share pattern information and set person permissions, that are then utilized by the answer’s embedded container element to populate the OpenSearch Serverless index. Carry out the next steps to mount the FSx for ONTAP SVM quantity as a community drive, add information to the shared community drive, and set permissions based mostly on Home windows ACLs:

1. get ad_host Occasion DNS from Terraform template output.
2. Navigate to AWS Techniques Supervisor Fleet Supervisor on the AWS console and discover ad_host occasion and comply with the directions right here to log in utilizing the distant desktop. Use area admin person bedrock-01Admin And get the password from AWS Secrets and techniques Supervisor. Yow will discover the password utilizing Secrets and techniques Supervisor fsx-secret-id The key ID within the Terraform template output.
3. To mount an FSx for ONTAP quantity as a community drive, do as follows this pcchoose (right-click) community then choose Map community drive.
4. Choose the drive quantity and mount it utilizing the FSx for ONTAP share path
   (<svm>.<area >c$<volume-name>):
   
5. Add the Amazon Bedrock Consumer Information to a shared community drive and set permissions to Admin Consumer Solely (be sure you disable inheritance within the following instances) superior):
6. Add the Amazon FSx for ONTAP Consumer Information to the shared drive and ensure permissions are set to everybody:
7. superior ad_host server, open a command immediate and enter the next command to get the SID of the admin person:

   wmic useraccount the place title="Admin" get sid

Check permissions utilizing a chatbot

To check permissions with the chatbot, get lb-dns-name Output a URL from a Terraform template and entry it via an online browser:

For immediate inquiries, please ask any basic questions concerning the FSx for ONTAP Consumer Information, which is out there to everybody. In our state of affairs, we requested “The right way to create a FSx for ONTAP file system” and the mannequin responded within the chat window with detailed steps and supply properties to create an FSx for ONTAP file system utilizing the AWS Administration Console, AWS CLI, or the FSx API :

Now, let’s ask concerning the Amazon Bedrock Consumer Information for administrator entry solely. In our state of affairs, we requested “How do I take advantage of the bottom mannequin with Amazon Bedrock” and the mannequin responded that it didn’t have sufficient info to offer an in depth reply to the query:

Use the administrator SID within the Consumer (SID) filter search within the chat UI and ask the identical query within the immediate. This time, the mannequin ought to reply with steps detailing the right way to use FM with Amazon Bedrock, and supply the supply attribute that the mannequin makes use of in its response:

Check permissions utilizing API Gateway

You too can use API Gateway to question the mannequin instantly. get api-invoke-url Parameters from the Terraform template output.

curl -v '<api-invoke-url>/bedrock_rag_retreival' -X POST -H 'content-type: software/json' -d '{"session_id": "1","immediate": "What's an FSxN ONTAP filesystem?", "bedrock_model_id": "anthropic.claude-3-sonnet-20240229-v1:0", "model_kwargs": {"temperature": 1.0, "top_p": 1.0, "top_k": 500}, "metadata": "NA", "memory_window": 10}'

Then name the API gateway everybody Point out entry permissions for queries associated to the FSx for ONTAP Consumer Information by setting the worth of the metadata parameter to NA everybody Rights of use:

curl -v '<api-invoke-url>/bedrock_rag_retreival' -X POST -H 'content-type: software/json' -d '{"session_id": "1","immediate": "what's bedrock?", "bedrock_model_id": "anthropic.claude-3-sonnet-20240229-v1:0", "model_kwargs": {"temperature": 1.0, "top_p": 1.0, "top_k": 500}, "metadata": "S-1-5-21-4037439088-1296877785-2872080499-1112", "memory_window": 10}'

clear up

To keep away from recurring prices, please clear up your account after attempting the answer. From the terraform folder, delete the answer’s Terraform template:

terraform apply --destroy

in conclusion

On this article, we exhibit an answer that makes use of FSx for ONTAP with Amazon Bedrock and makes use of FSx for ONTAP’s help for file possession and ACLs to offer generative AI purposes in a RAG state of affairs. Entry permissions. Our resolution lets you construct generative AI purposes utilizing Amazon Bedrock, the place you possibly can enrich the generative AI prompts in Amazon Bedrock with company-specific, unstructured person profile information from the FSx for ONTAP file system. This resolution permits you to present extra related, particular, and correct responses whereas guaranteeing that solely licensed customers have entry to the information. Lastly, the answer demonstrates the right way to use AWS serverless providers with FSx for ONTAP and Amazon Bedrock to implement autoscaling, event-driven computing, and API interfaces for generative AI purposes on AWS.

For extra details about the right way to get began with Amazon Bedrock and FSx for ONTAP, see the next assets:

Concerning the writer

Kanishk Mahajan Is the AWS Resolution Structure Lead. He leads cloud transformation and resolution structure for AWS’s ISV clients and companions. Kanishk makes a speciality of containers, cloud operations, migration and modernization, AI/ML, resiliency, and safety and compliance. He’s a member of the Technical Area Group (TFC) in each space of ​​AWS.

Michael Sauer is the Chief Architect within the Workplace of the Chief Know-how Officer at NetApp. He has greater than 20 years of expertise constructing information administration methods, purposes and infrastructure options. He has distinctive and in-depth insights into cloud applied sciences, builders, and synthetic intelligence options.

Sasha Coleman Is the technical visionary chief of dynamic improvement and high quality assurance groups in Israel and India. He has labored at NetApp for 14 years, beginning as a programmer, and his hands-on expertise and management have been important in guiding complicated initiatives to success, with a deal with innovation, scalability and reliability.