Create a Rovo agent that matches ticket keywords to Confluence knowledge base articles, then run it from an automation rule triggered on ticket creation. What worked was instructing the agent in its prompt to answer the customer or escalate to a human based on its confidence, since parsing its output in the rule often failed.
To more efficiently manage requests and tickets, we must first evaluate our current resources and then optimize our processes. To begin.
To train Rovo for customer interactions:
Identify the top 5 to 10 most frequent keywords in incoming tickets.
Link relevant Confluence knowledge articles to these common requests.
Create a gap analysis of missing articles.
Categorize these missing articles with appropriate labels.
Analyzing the data as requested, Rovo provided a list of the top 10 most frequently used words.
On the Top 5 we have:
Data
Privacy
AI
Training
Model
Using this keyword list, we can now pair them with relevant articles, enabling Rovo to address common requests effectively. In Confluence, we've created five distinct articles, each with a header precisely matching one of the identified keywords.
We've created a new agent, 'Ticket Assistant,' designed to fulfill our requirements.
Instructions:
You are an automated assistant designed to help employees respond to tickets by utilizing pre-made knowledge base articles from Confluence.
You will analyze the content of incoming tickets and identify keywords to match with relevant articles.
If a ticket contains specific keywords, such as 'Data', you will provide the corresponding knowledge base article to assist in the response.
You should be efficient and accurate in identifying the right articles to ensure quick resolutions for the tickets.
Your primary job is to filter requests and provide the correct knowledge base article based on the ticket content.
Always try to be polite in your responses.
It's important to clarify that these requests are going to be raised inside the portal
Knowledge:
Confluence Space: Knowledge Base Articles
Actions:
Add a comment to a work item.
Assign a work item.
First part of the rule
An automation rule will be created, activating whenever a new ticket is generated and set to 'Waiting for support' status.
Then, enable the new Rovo Agent “Ticket Assistant”
Prompt: Analyze this ticket, and based on the content, look for a knowledge base article that is most likely going to help solve the question.
Evaluate your confidence on a scale from 1-10. If after that evaluation you got more than an 8, return 'STATUS:ready_to_go KB_URL: [YOUR_KNOWLEDGE_BASE_ARTICLE_URL_HERE]'. Otherwise, return 'STATUS:human_action_needed'.
The Rovo Agent prompt asks it to self-evaluate its confidence (1-10) in solving a ticket using its knowledge base. If highly confident (score > 8), it returns "STATUS:ready_to_go" with a KB URL, allowing automation to provide a direct solution. Otherwise, it returns "STATUS:human_action_needed," signaling that a human agent is required for escalation. This structured, confidence-based output enables smart, branched automation.
Then, create a Branch in which we are going to include the different IF conditionals
Conditionals and actions
The rule will include two separate "If" blocks, each evaluating the Rovo Agent's response with the expected values: "ready_to_go" or "human_action_needed."
If the state is ready_to_go, Rovo will add a comment that includes a link to the article deemed most likely to address the request effectively.
If the state is human_action_needed, Rovo will comment that, due to the absence of suitable articles, it is advisable for a human to take charge of the issue.
Assigns the issue to an agent, in this case, Nicolas.
is used to extract the first word from the Agent's response.
Here's how it works:
: This represents the entire text output generated by the "Use Rovo Agent" action.
.split(' '): This function takes the entire agentResponse string and breaks it into a list (or array) of smaller strings, using a space (' ') as the separator. For example, if agentResponse is "STATUS:ready_to_go KB_URL: AI", it would be split into ["STATUS:ready_to_go", "KB_URL:", "AI"].
[0]: After splitting, this [0] accesses the first item in the resulting list. In the example above, it would return "STATUS:ready_to_go".
This smart value is particularly useful for extracting a status or a key identifier that is expected to be the very first word of the Rovo Agent's output.
Rule Detailed View
Audit Log
Upon initial enablement, the rule successfully executed and provided a confidence-based answer; however, this was a one-time occurrence, as subsequent attempts consistently resulted in an error, preventing further execution.
The error Failed to get value for agentResponse.split(' ')[0] that the Smart Value expression is trying to operate like split()on a variable that is empty or null.
Specifically:
: This Smart Value is supposed to contain the text output from the Use Rovo Agent action.
.split(' ')[0]: This is a string manipulation function that attempts to break into parts using a space as a delimiter and then retrieve the first part.
The Error's Meaning: When you see "Failed to get value for...", it indicates that did not contain a valid text string for the .split() function to operate on. It's either empty, null, or undefined at that specific point in the rule's execution.
Error does not make sense, since it is proven there is a valid string inside ; this was tested with another rule.
Even after verifying the accurate setup of conditionals, actions, and smart value expressions, identifying and addressing the issue remained difficult. This challenge prompted us to explore an alternative rule.
For this attempt, we prompted the Rovo Agent to self-manage the conditional logic and perform actions based on its return value.
Rule Building
Similar to our initial attempt, we will establish an automation rule that triggers upon ticket creation when its status is "Waiting for support". Subsequently, the new "Ticket Assistant" Rovo Agent will be enabled.
Prompt: Analyze this ticket, and based on the content, look for a knowledge base article that is most likely going to help solving the question.
Evaluate your confidence on a scale from 1-10. If after that evaluation you got more than an 8, return 'STATUS:ready_to_go KB_URL: [YOUR_KNOWLEDGE_BASE_ARTICLE_URL_HERE]'. Otherwise, return 'STATUS:human_action_needed'.
If you return: 'STATUS:ready_to_go KB_URL: [YOUR_KNOWLEDGE_BASE_ARTICLE_URL_HERE]' attach the document and answer to the customer.
If you return: 'STATUS:human_action_needed'. Please respond to the customer saying that a human needs to be in charge of this issue
When responding to a customer, it is essential to provide an answer that is kind, professional, and knowledgeable, without revealing the value of your return or your internal thought process.
This prompt guides the Rovo Agent to analyze tickets, self-evaluate its confidence (1-10), and provide a structured "STATUS" response (ready_to_go or human_action_needed) along with a KB URL if applicable. Crucially, it then explicitly instructs the agent to perform the follow-up actions directly (attach document, answer customer, or indicate human needed) and to craft professional, hidden responses. This direct instruction for action, rather than just status, makes the automation more robust.
Upon enabling and triggering the rule, this is the result we observed.
Request: i would like to know more about Privacy inside my company
Outcome:
Rovo provided a relevant answer to the customer's question, along with the requested article.
Request: Can i go for holidays on Christmas?
Outcome:
Given we intentionally asked a question outside Rovo's knowledge base, it correctly escalated the request to a human agent.
We can confidently say that the Rovo Agent and the Automation Rule work seamlessly.
Inside of Atlassian Studio there’s an option to create an agent specialized on portal requests and customers
By clicking on Create then you will be asked to choose in which project the agent should work.
When Clicking on Continue we will be directed to the home page inside of Atlassian Studio, in order to enable our new agent we will go to the Help Center of our JSM project.
It’s important to activate the agent.
Enabling this agent allows customers to receive personalized, one-on-one assistance when submitting requests through the portal.
This document outlines our journey of integrating Rovo into Jira Service Management to automate repetitive tasks. Initially, our approach relied on external parsing of Rovo's output, which often failed due to empty responses. The breakthrough occurred when we directly instructed the Rovo Agent within its prompt to perform actions, such as attaching documents and responding to customers, based on its confidence levels. This method effectively bypassed parsing issues, resulting in a reliable solution. Furthermore, this initiative, along with the new Rovo Agent features in Atlassian Studio, marks a significant first step toward utilizing Rovo for more complex tasks and expanding knowledge bases in the future.