Search 800 + Posts

Feb 5, 2026

The Proof is in the Prompt: Stress-Testing Select AI with RAG (in Oracle26ai)

In my previous posts, we laid out the Technical Blueprint and explored the Architectural Reasoning for bringing Select AI with RAG into the enterprise. But as any architect knows, a design is just a theory until the first query hits the engine.

Today, we move from "Building" to "Proving."

Testing a RAG system isn't just about checking for "Success" or "Failure." It’s about Grounding Truth. We need to ensure that when a warehouse manager asks about a "Wave," the AI isn't hallucinating a seaside vacation—it’s pulling directly from page 42 of your Oracle Warehouse Management Cloud User Guide.

For our stress-testing use case, we chose the Oracle Warehouse Management Cloud (WMS) User Guide. This isn’t just a simple document; it’s a 500+ page technical manual filled with complex workflows, status codes, and operational logic.

To make this "knowledge" accessible to Select AI, we first established a Secure Data Landing Zone in an Oracle Cloud Infrastructure (OCI) Object Storage Bucket. Loading the file into the bucket is only the first step. The real magic happens in how Oracle Database 26ai consumes this data. Unlike a standard file upload, Select AI performs a sophisticated 4-stage transformation to turn that PDF into a conversational asset.
1. The Automated Ingestion Pipeline
2. Intelligent Chunking
3. Vectorization (The "Meaning" Map)
4. The Vector Index Store

The Final Loop: The RAG Consumption
When a user asks a question like, "How do I close a manifest for an LTL shipment?", the consumption loop completes:
  1. Search: Select AI converts the user's question into a vector.
  2. Retrieve: It searches the "Meaning Map" for the top 3-5 chunks from the WMS manual that are most similar to the question.
  3. Augment: It sends the question PLUS those specific chunks to the LLM.
  4. Narrate: The LLM uses that "grounded" context to give you an answer that is 100% based on the WMS Guide, not its own imagination.

👉  Check our youtube video 

The Pre-Flight Check: Is the Engine Ready?

Before you issue your first natural language prompt, you must verify that the "Ingestion Pipeline" finished its heavy lifting. PDF processing is asynchronous; the database needs time to chunk the text and generate vectors.

1. The Health Dashboard

Check#1
Make you you have executed DBMS_CLOUD_AI.CREATE_VECTOR_INDEX,Below is my Example

BEGIN
DBMS_CLOUD_AI.CREATE_VECTOR_INDEX(
    index_name  => 'WMS_KNOWLEDGE_BASE',
    attributes  => '{"vector_db_provider": "oracle",
                    "location": "https://objectstorage.us-phoenix-1.oraclecloud.com/n/xxxxxx/b/WMS_PDF_BUCKET/o/",
                    "object_storage_credential_name":WMS_OCI_GENAI_CRED",
                    "profile_name": "WMS_GENAI_VECTOR_",
                    "vector_table_name":"ERP_VECTOR",
                    "vector_distance_metric": "cosine"
                }'
    );     
END; 

Note - PARAMETER : VECTOR_TABLE_NAME  - Its primary purpose is to manually define the name of the physical table that Oracle creates to store your chunked document text and their corresponding vector embeddings.

By default, if you don't specify this parameter, Oracle 26ai automatically generates a system-named table (typically following a pattern like VECTOR_$<INDEX_ID>). While the system manages this for you, manually providing a vector_table_name offers several architectural advantages:

Check#2  - Run Queries against your "Knowledge Base" to verify the quality of the chunks or the accuracy of the embeddings.
select * from ERP_VECTOR;

Check#3 - Check Status of INDEX in table USER_CLOUD_PIPELINES.This view tracks the progress and health of the automated background jobs that handle the document processing and embedding.

The STATUS_TABLE column in the USER_CLOUD_PIPELINES view identifies an internal tracking table created by the database to manage a specific pipeline's operations.



When you create a Vector Index using DBMS_CLOUD_AI.CREATE_VECTOR_INDEX, Oracle automatically launches a background pipeline to handle the heavy lifting (downloading files, chunking text, and generating embeddings). The STATUS_TABLE is where the database records the granular progress of every single file in your Object Storage location.

Query internal tracking table and if STATUS = COMPLETED then you are all Set.

The "WMS Masterclass": Five Levels of Testing

To truly stress-test our RAG implementation, we don't just ask simple questions. We test for Context, Process, and Synthesis. Here is the testing suite I use for our WMS pilot:

Level 1: Semantic Recall (The Definition)

  • Prompt: SELECT AI narrate what is an 'LPN' and why is it central to WMS?

  • What we’re testing: Can the AI find a specific definition buried in a 500-page doc?

  • Success Criteria: The response should mention "License Plate Number" and its role in tracking containers/pallets.

  • Result : 

"An 'LPN' in the context of Warehouse Management System (WMS) stands for License Plate Number. It is a unique barcode assigned to trailers or containers to trace them within the warehouse. The LPN is central to WMS because it helps in managing inventory within a Distribution Center by tracking the movement of goods from receipt through shipment. By using LPNs, the system can easily identify and manage the contents of containers or pallets without the need to scan detailed information during processes like picking or receiving.

 Sources:

  - oracle-warehouse-management-cloud-user-guide.pdf (https://objectstorage.us-phoenix-1.oraclecloud.com/nxxxxxx/b/WMS_PDF_BUCKET/o/oracle-warehouse-management-cloud-user-guide.pdf)"

Level 2: The Process Chain (Step-by-Step)

  • Prompt: SELECT AI narrate describe the step-by-step process for performing a 'Blind Cycle Count'.

  • What we’re testing: Can the AI retrieve and sequence multiple "chunks" of text correctly?

  • Success Criteria: A numbered list that matches the manual's operational flow.

  • Result

"To perform a Blind Cycle Count in Oracle WMS Cloud, follow these step-by-step instructions:

 1. Enter the RF transaction "Cycle Count LPN" to initiate the Blind Cycle Count process.

2. Scan the location that needs to be cycle counted.

3. Scan each LPN present in the location.

4. If you accidentally scan an LPN that does not exist in the location, the system will prompt a message to "Rescan the same LPN." Rescanning the same LPN will allow the system to make the necessary inventory adjustments to record the new LPN in the location.

 Sources:

  - oracle-warehouse-management-cloud-user-guide.pdf (https://objectstorage.us-phoenix-1.oraclecloud.com/n/xxxxxx/b/WMS_PDF_BUCKET/o/oracle-warehouse-management-cloud-user-guide.pdf)"

Level 3: Troubleshooting (Applied Logic)

  • Prompt: SELECT AI narrate a shipment is stuck in 'In-Transit' status even though it has arrived. What should I check?

  • What we’re testing: Does the RAG retrieve troubleshooting tips rather than just definitions?

  • Success Criteria: The AI provides 2-3 specific "checkpoints" mentioned in the manual.

  • Result

"To address the issue of a shipment being stuck in 'In-Transit' status even though it has arrived, you should check the following:

 

1. Ensure that the ASN (Advanced Shipment Notification) has been fully received and verified. The ASN status should be updated to 'Receiving Complete' or 'Verified' after the receipt confirmation is generated and sent.

 2. Verify that the ASN is still in the 'In-Transit' status and has not transitioned to 'Receiving Started' or 'Receiving Complete'. ASNs can only be modified while in 'In-Transit' status, so if it has moved to the next status, modifications cannot be made.

 3. Check for any discrepancies in the receiving process by reviewing the Receiving Discrepancies. You can filter discrepancies by 'OK', 'Missing', and 'Extra' to identify any issues with received quantities compared to shipped quantities.

 By ensuring the ASN is fully received, verifying its status, and checking for any discrepancies in the receiving process, you can troubleshoot why the shipment is stuck in 'In-Transit' status.

 Sources:

  - oracle-warehouse-management-cloud-user-guide.pdf (https://objectstorage.us-phoenix-1.oraclecloud.com/n/xxxxxx/b/WMS_PDF_BUCKET/o/oracle-warehouse-management-cloud-user-guide.pdf)"


Level 4: The Audit Trail (Citations)

  • Prompt: SELECT AI narrate what are the security requirements for RF device users? Provide citations.

  • What we’re testing: Is the AI properly referencing its source?

  • Success Criteria: The response includes "Sources: [File Name]" or ideally specific page/section references.

  • Result

"The security requirements for RF device users include creating new users through the "Users" screen where you can define the username, password, facilities and companies access, user's role, default group, language, default printer, and fixed equipment type. Additionally, users must ensure that the RF gun is in the correct facility and can change facilities by pressing Ctrl-F and typing in the Facility code.

Sources:

  - oracle-warehouse-management-cloud-user-guide.pdf (https://objectstorage.us-phoenix-1.oraclecloud.com/nxxxxxx/b/WMS_PDF_BUCKET/o/oracle-warehouse-management-cloud-user-guide.pdf)"


We have tested Same Prompt from on APEX UI ( tis is deigned by Oracle and available in Oracle Cloud)  and outcome was wonderful as expected 


Lessons from the Lab: What We Discovered

During our testing phase, we learned three critical lessons that every enterprise architect should keep in their back pocket:

1. The "Narrate" Action is Key

Standard SELECT AI might try to generate SQL. When testing RAG on documents, always use the narrate keyword. This tells the database: "Don't write code; tell me a story based on the retrieved facts."

2. Guarding against "Out-of-Bound" Answers

If the AI answers a question that isn't in your PDF, it’s using its "base knowledge" (hallucinating). To fix this, we refined our AI Profile attributes to increase the "grounding" requirement.

Conclusion: From Test to Production

Testing Select AI with RAG isn't just a technical exercise—it's about Building Trust. When Sarah the Lead Support Engineer (from our first blog) sees a response that perfectly mirrors the technical manual, she stops being a "searcher" and starts being an "expert."

By using the DBMS_CLOUD_AI package and monitoring our vector indexes, we’ve proven that the database isn't just a place to store data anymore. It’s a place to understand it.