Unraveling Rovo Search Relevance

Atlassian's Rovo search is designed to make that possible by offering a unified search experience across all Atlassian products and more than 50 popular SaaS applications used by modern teams. Whether your documents live in Confluence, issues in Jira, files in Google Drive, or messages in Slack, Rovo can search across them all. It serves as a one stop knowledge discovery tool that spares you from having to search each platform individually. Rovo is also the foundation for Atlassian's AI features, powering everything from quick search results to AI driven assistants, so its ability to deliver fast, relevant answers is critical.

This detailed look at Rovo will explore how it provides different search experiences for users and the technology behind its search relevance. We will break down the key components that make Rovo's search powerful, including how queries are processed, how results are ranked and personalized, and how Rovo uses artificial intelligence to give smart answers with citations. By understanding the inner workings of Rovo, you can appreciate how advanced ranking algorithms, user activity signals, and large language models come together to help your team find what they need in seconds.

Key Terms in Rovo Search

• Connector: An integration that links an external application or data source with Rovo. Connectors allow Rovo to include content from other platforms in its search index. For example, a Google Drive connector or a SharePoint connector brings documents from those services into Rovo's search.

• OpenSearch: The search indexing platform Atlassian uses to store and query data for Rovo. It runs on AWS OpenSearch and stores all content as searchable "documents" including text and metadata.

• BM25: Stands for Best Match 25, a classic algorithm that scores how relevant a document is to a search query based on keyword matches. Rovo uses BM25 in its initial retrieval of results.

• KNN: Stands for K Nearest Neighbors, an algorithm that finds the closest matching items by semantic similarity. Rovo employs KNN based techniques to support semantic search, retrieving results that might be relevant even if they do not contain the exact query terms.

• LLM: Stands for Large Language Model, which is an AI model that can understand and generate humanlike text. Rovo uses multiple LLMs for tasks like query rewriting and generating natural language answers in Smart Answers.

Multiple Search Experiences for Every Need

• Quick Find: This is the search box at the top of Atlassian products. Quick Find is optimized to instantly surface content you have recently accessed or worked on. It is great for quickly finding content you engaged with recently, with minimal typing.

• Full Search: When you hit Enter from the Quick Find box, you go to the full search page. Here you can apply filters such as product, content type, author, date, and more to narrow results. The full search runs a deeper query to fetch the most relevant and authoritative results across all content you have access to, not just the items you touched recently. Your recent engagement will still boost certain items higher in the list, but full search will also uncover other important pages or tickets you might not have seen yet.

• Smart Answers: In the search interface, typing a natural language question can trigger a Smart Answer. This is an AI generated answer box that directly addresses your question, drawing on information from across your connected knowledge bases. For example, you might ask "How do I archive a project?" and get a concise answer with a step by step explanation drawn from relevant Confluence and Jira documentation. Each Smart Answer includes citation links to the source content so you can verify the details or read more.

• Rovo Chat: Rovo also offers an AI chat assistant that can answer questions or help with tasks in a conversational manner. Under the hood, this chat uses the same search technology as Smart Answers to find relevant information. When you ask the chatbot a question, it performs a search behind the scenes, then the AI formulates a helpful reply. This means Rovo Chat benefits from the same unified search index and relevance algorithms to provide accurate responses.

How Rovo Search Works Under the Hood

When you submit a query, Rovo goes through several steps to fetch and rank the best results, all in a fraction of a second. The process is sophisticated, but it can be understood in a few main layers:

Query processing: First, Rovo's query intelligence component interprets what you are looking for. It checks the query for things like typos, natural language phrasing, or known issue keys. The query might be rewritten or refined by an AI model to improve the results. For example, if you typed "next holiday", the system might expand it to "next holiday in your country" using your profile context, so that it knows to look up the correct regional information.

Initial retrieval: Once the query is understood, Rovo searches its index for content that matches. It does this across multiple indexes optimized for different content types. The search uses OpenSearch with both keyword based matching using BM25 and semantic matching using vector similarity techniques like KNN to cast a wide net for relevant items. At this stage, Rovo also filters results based on permissions ensuring you only get results from content you are allowed to access and it applies an initial ranking. This initial ranker may consider simple signals like how many times your query terms appear in a document and basic popularity metrics.

Advanced reranking: The top results from the initial search are then sent through more advanced machine learning models for a second round of ranking. Rovo utilizes a fine tuned cross encoder model to judge semantic relevance more deeply between the query and each result. It also uses a specialized Deep and Cross Network model that looks at behavioral signals such as which documents tend to get clicked for similar queries or how users have engaged with that content in the past. These models output scores indicating how likely each result is to be useful to you. The system combines these signals to adjust the ordering of results, pushing the most relevant and helpful items toward the top.

Blending results: If your search spans multiple product sources, for example results from Confluence, Jira, and Slack, Rovo then interleaves the top results from each source into one final ranked list. This blending considers your personal usage patterns and the context of the query. For instance, if you search for a specific Jira issue key, the system will prioritize the corresponding Jira ticket at the top. If you search for a query like "project plan Q4", it might show a Confluence page higher up. By mixing results intelligently, Rovo ensures that the final output isn’t dominated by one source and that you see the most relevant content across all your tools.

Throughout this process, Rovo emphasizes security and speed. The search is distributed across microservices so it can query many indexes in parallel, and multiple permission checks occur to make sure no result slips through that you shouldn’t see. The outcome is that within milliseconds, you get a list of results that feels almost intuitive: often the very thing you were hoping to find is right at the top.

Smart Answers: How AI Delivers Instant Answers

1. Classify the query: Not every search will trigger an AI generated answer. Rovo first classifies your query to decide how to handle it. If you typed the name of a person, a team, or a known internal bookmark, Rovo will recognize that and instantly show a relevant result such as a person’s profile card or a direct link. Those are straightforward navigational queries. If none of those special cases apply and your query looks like a natural language question, then the Smart Answers workflow kicks in.

2. Rewrite the question with context: Once Rovo determines you are asking a question in plain English, it uses a large language model to restate or enhance your query for better search results. It might add contextual details that it knows about you or your organization. For example, "when is the next holiday" could be automatically rephrased to "when is the next holiday in your country" based on your location, because the answer may depend on regional calendars. This rewritten query captures the intent more clearly.

3. Retrieve relevant documents: The enriched query is then sent through the same foundational search system described earlier to pull up relevant documents from all connected sources. At this stage, Rovo gathers a wide array of potential sources Confluence pages, Jira tickets, Google Drive files, Slack threads, and so on that might contain the answer.

4. Transform and chunk content: The documents found are converted into a standard format and then broken down into smaller passages or “chunks” of text. Splitting content into chunks, such as breaking a long page into individual paragraphs or sections, allows the system to isolate the specific pieces of information that could answer the question.

5. Rank the passages: Next, Rovo uses an AI model, specifically a cross encoder trained for passage ranking, to evaluate which of those chunks are most relevant to the query. It scores all the candidate passages and sorts them so that the most relevant information is identified.

6. Generate an answer with citations: Finally, a large language model takes the top ranked passages and formulates a concise answer to your question. Importantly, as it constructs the answer, it keeps track of which source each piece of information came from. Rovo then presents you with a written answer (often just a few sentences or a brief list) along with citation footnotes. Those citations link back to the original documents and even the exact sections that provided the information. This way, you not only get an instant answer but also the ability to click through and trust where the information is coming from.

Relevance Signals and Personalization

What counts as a "relevant" search result can vary from person to person, so Rovo incorporates many signals to tailor results to the individual user and the context. The aim is that the most useful content for you appears at the top.

Content and context in the index: All searchable content is stored with rich context in Rovo’s index. For chat platforms like Slack or Microsoft Teams, messages are indexed in groups by thread or channel so that a single message’s surrounding context is preserved. Additionally, each document or message in the index carries metadata like its author, how often it has been viewed or edited, and whether it lives in a personal space or a public/project space. Rovo takes these attributes into account when deciding relevance for instance, a document edited by many people or viewed frequently might be considered more important or authoritative than one that has sat untouched.

Personalization boosts: Rovo personalizes results by boosting content that is more likely to be relevant specifically to you. If you search for a topic and you yourself wrote a page about it, that page will rank very highly in your results. Similarly, if someone on your immediate team created the content, it gets an extra lift for you because there's a good chance that item is especially pertinent to your work. For communication platforms, Rovo will prioritize results from the channels or group chats you have been active in recently, since those conversations are the ones you’re closest to.

Popularity and freshness: The search algorithm also considers how popular and how recent content is. Popularity can be measured by things like the number of views, likes, or comments a document has received. Highly popular pages or frequently referenced files are more likely to be useful, so popularity can push them higher in the results. Freshness is also important something updated last week might be more relevant than a file from five years ago. Rovo boosts newer content and pushes very old content lower in the results. The overall effect is that new and actively useful information tends to rise to the top, while stale content appears further down.

All these factors work together to paint a picture of relevance. Imagine you search for "Search Relevance Deep Dive". If there is a recently updated Google Drive presentation on that exact subject, and you happen to be one of the contributors or viewers of it, Rovo will likely put that presentation at the very top of your results. Meanwhile, another user who searches the same keywords but has no connection to that file might see a different top result that better matches their context. By blending textual relevance with user-specific signals, Rovo aims to give each person the results that make the most sense for them.

Measuring Search Success

Rovo's job doesn’t end at delivering search results Atlassian also continually evaluates how well the search is performing and adjusts the system accordingly. To ensure that Rovo's results are truly helping users, a mix of user feedback, usage data, and even AI evaluations are used to measure success.

User engagement signals: One primary metric is Query Success Rate, or QSR for short. This “north star” metric looks at what happens after you perform a search. If you click a result and spend time on that page and do not immediately return to try a different result, that suggests you found what you needed. If a Smart Answer is shown and you get your answer directly without clicking anything else, that also counts as a success. QSR combines behaviors like clicks, dwell time on results, and whether users ultimately got their answer, to estimate the overall success rate of searches. In addition, Rovo collects explicit feedback where available: for example, if users give a thumbs up or thumbs down on a Smart Answer or on a particular result, that data is recorded and fed back into the system.

Offline relevance testing: Apart from live user interactions, Atlassian’s engineers conduct offline evaluations to fine tune Rovo’s ranking models. They compile sets of queries with known “right answers,” including some drawn from real usage patterns and others that are curated or synthesized for testing. Several metrics are used to evaluate relevance. For instance, Recall@k measures the percentage of test queries for which the expected document appears in the top k results. The team examines multiple values of k, such as 3, 5, or 10, to reflect typical result page sizes. NDCG, short for Normalized Discounted Cumulative Gain, assesses the quality of the ranking by comparing it to an ideal ordering, giving more credit when relevant items appear higher up. MRR, or Mean Reciprocal Rank, focuses on the rank of the first relevant result for each query. For example, if the best answer is the very first result, the MRR is 1. If the best answer is fourth on the list, the MRR is 0.25. A higher MRR means users are finding what they need faster. Beyond these metrics, the team also uses an "LLM as a judge" approach, where they feed sample queries and Rovo's results to a large language model and ask it to judge the relevance and ordering of the results, simulating a human reviewer. This provides additional insights and even training data to further refine Rovo's search algorithms.

By combining these evaluation methods, Atlassian makes sure that Rovo’s search relevance keeps improving. Before new connectors or features are rolled out, offline tests on these metrics give an early signal of search quality which is especially useful when there isn’t yet a lot of user traffic on a new data source. And once features are live, the ongoing monitoring of real world usage through metrics like QSR and direct user feedback guides iterative improvements. This commitment to measuring and refining the system means Rovo’s search should get smarter and more accurate over time.

In summary, Rovo search has evolved into a sophisticated platform for enterprise knowledge discovery. It leverages advanced technology under the hood from smart query understanding and multi layer ranking to AI driven answer generation all while integrating signals about what each user cares about. The result is a search experience that feels tailored and intuitive in a complex information landscape. Atlassian continues to enhance Rovo with new integrations and refinements, so its capabilities will only grow.