One major challenge that comes with building an AI agent is that its response quality is dependent on the quality of the data that it’s given. As a result, ensuring high quality responses from Aptible AI has been a journey, but we’ve made some big steps forward over the past few weeks.

Here’s what’s new:

Improved bot response quality

When your on-call engineers rely on accurate information to investigate and resolve incidents as quickly as possible. To ensure that Aptible AI provides the most relevant and helpful information that it can from your data sources. Here’s how we’ve improved Aptible AI’s responses:

• Self-rating: we’re now using AI to quantify how useful the most recent tool call results are to resolving the question that was asked. This self-rating system helps Aptible AI to choose tools and identify stopping criteria when searing for the correct answer.

  

• Links to sources: previously, when the bot would return a Grafana or Datadog dashboard, it would only embed the dashboard in its response. Now the response also includes the link to the source for easy access to further investigation if needed.

  

• Quicker responses: we’ve made adjustments to the bots data fetching that allow it to return a response faster by only grabbing relevant data.

Other improvements:

• UI update: We’ve improved the readability of long, complex conversations in the Threads tab of the Web UI so that it’s easier to understand the human messages vs. the bot’s messages
  
  

• New logging assistant: Improved usefulness of logging interactions by creating a standalone logging assistant that’s better at finding the specific section of logs that you need based on the incident

Aptible AI has come a long way in just a few short weeks.

Everything we’ve built has been inspired by the SRE teams we’ve spoken with, and this particular feature was developed in response to a few common problems. Specific and in-depth knowledge of certain systems typically resides with a few key engineers, making on-call difficult for anyone who may be less familiar with those systems (and leading to those tenured engineers getting pulled in to every incident). This generally leads to high stress and longer time to resolution.

To address that problem, Aptible AI now offers suggestions for next steps.

Suggested Next Steps

On-call engineers aren’t always confident about what to do next, either because of the high stress situation or because of a lack of knowledge about the affected services. Ask Aptible AI, “what should I do next?” and it will give you smart next step suggestions during an incident. Here’s how it works:

• Aptible AI has access to your runbooks and, with time, learns how you typically handle incidents by observing your team’s interactions and troubleshooting steps. Based on that information, it can provide suggestions on what you could try out next.

• From Slack, you can choose (and execute) your next course of action using Aptible AI.

  

Other improvements

• Single tenancy option for high security/compliance companies

• Improved bot response times and increased tool choice accuracy

• Improved support for:

  • Datadog

  • InfluxDB

  • Grafana

  • Sentry

  • PagerDuty

  • Papertrail

Every organization's infrastructure and needs are unique and complex. Incident response tools rarely account for your particular use cases and tools, often providing more guidance around process than actual assistance. If you need to investigate an incident by searching logs or metrics from your org's specific databases, you still have to do that manually, tracking down the right scripts to run or combing through thousands of lines of logs.

To speed things up, we developed a way for Aptible AI to integrate with your custom tools so that you can automate repetitive tasks, even the ones that are highly specific to your business.

Custom integrations

Engineers often waste time trying to track down the correct scripts to run to retrieve the information they need from log and metric providers or other internal tooling. We built support for custom tooling so that you can tell Aptible AI when to run your scripts for you. Here’s how it works:

• When you set up your integrations in chat, you can implement your custom integration in one of two ways:

  • A lightweight PubSub interface leveraging Redis as the intermediary

  • A websocket-based “safe” proxy interface modeled after Google’s implementation

• You ask questions in natural language and Aptible AI will make the appropriate queries to your log and metric providers

• Results will be returned directly in Slack, improving shared context and speeding up root cause analysis

  

Incident investigation and resolution occurs all over the place; sometimes there’s multiple chats and threads going on at the same time as a Slack huddle or a Zoom call, and it becomes overwhelming trying to keep up with what’s been done so far. It’s even harder to summarize all the findings, gather them into a single source, update runbooks, and learn from every incident…

That's why we built Incident Tracking and a new Web UI so that you can drill down into each incident, both as it's occurring and after it's resolved.

Incident tracking

Incident Tracking means Aptible AI can provide ongoing, updated summaries for all stakeholders directly within chat. Then it takes those learnings and records them in a Web UI so that your team can drill down into common time sinks, tools that you use most, most frequent root causes, and more. Here’s how it works:

• Aptible AI proactively participates in incident investigation, monitoring everything that’s happened within your Slack thread

• It provides up-to-date summaries for anyone who’s new to the incident so they can read what’s been done so far

• Once the incident is resolved, Aptible AI records all the information in an incident dashboard so that your team can drill down into what incidents are most common, how they’re being resolved, and where the most time is being spent

  

Thanks to the help of our design partners, we’ve been able to create an early iteration of the Aptible AI agent, focused on tackling complex, multi-faceted problems in incident response and system management. Aptible.ai isn’t just a chatbot; it’s a sophisticated system of agents, LLMs, and integrated training and research tools that all work together to provide proactive, expert assistance in real-time. 

Let's dive into the exciting developments that are shaping Aptible.ai into an indispensable tool for engineering teams.

Fast and comprehensive information retrieval

Use Aptible.ai to improve your document search efficiency and improve overall team productivity. Our focus on creating excellent search capabilities includes several key features so far:

• Background and bulk indexing

• Generic document indexing API that supports various document types, including PDFs

• Google Search fallback

• Notion indexing

Fit Aptible.ai into any of your existing workflows

We’ve designed Aptible.ai to be more than just a Slack integration (although you can certainly do that too), allowing developers to use the Agent wherever they choose:

• Multiple Slack channels with configurable assistants

• Dynamic SlackBot ID management and permalink storage

• Direct responses to Sentry and PagerDuty alerts in Slack

Easily troubleshoot and improve Aptible.ai performance 

To enhance transparency and facilitate troubleshooting, we've implemented the following logging and monitoring capabilities:

• Comprehensive request and access logging

• Logging of all exceptions from the message stream

This progress represents a significant step forward in our mission to provide support to SRE teams during an incident and to help them improve their incident response processes in the future. As always, we welcome your feedback and look forward to continuing our journey of innovation together.

Stay tuned for more updates as we continue to refine and expand Aptible.ai's capabilities!