Issue AI Analytics for Jira

Documentation and Knowledge Base

Functional Overview

Issue AI Analytics for Jira introduces new analytics functionality for Jira issues. It exposes an Enable AI Analytics button in the Issue View screen. Once button is pressed, the issue summary and description are sent for analysis to an external AI engine. The analysis results are displayed as tabs, through a new panel in the Issue View screen.

Example:

The tabs that are displayed and the analysis that is done by the AI engine is configurable, depending on the needs of your organization (reference Configuration section).

The following analytics tabs are available by default:

  • Instructions – Instructions with proposals and/or concrete solutions on how to solve tasks. It might contain steps to follow, general advise on how to approach the problem or in some cases, a direct solution. This section will become more accurate, as more detailed issue description is provided.
  • Description Improvements – Suggestions on how to improve clarity of an issue description. In some cases, the issue author might not provide a description that is detailed or clear enough – this section will help improve the quality and make it more understandable.
  • Definition of Done (DoD) – Most teams don’t spend enough time to write Definition of Done (DoD) for issues. This section will automatically generate the DoD. It will help the author to properly define, when an issue can be considered done.
  • Test Strategy – Complete strategy on how to test your issues are successfully completed. In most cases, a comprehensive suite of tests will be provided. This might include tests for non-functional requirements or corner cases, which are easily forgotten.

The following AI engines are supported:

  • OpenAI GPT-4o
  • OpenAI GPT-3.5-Turbo
  • Anthropic Claude 3 Opus
  • Anthropic Claude 3 Sonnet
  • Anthropic Claude 3 Haiku
  • Azure OpenAI Service

Architectural Overview

Issue AI Analytics integrates Jira with an external AI provider for analytics through a Heroku server. This can be visualized as follows:

The Deview Studios app server is hosted on the Heroku platform and is the entry point for the integration.

Integration process triggers whenever the Enable AI Analytics button is pressed in the Issue View screen. Only the issue summary and description are sent for analysis.

First time analytics generation for an issue can take between 20-60 seconds. This is primarily due to the processing time needed by the AI provider. Once generated, analytics are cached in an analytics database and every subsequent display of analytics results in the Issue View screen is immediate. Analytics results will be re-generated, if issue summary or description is changed.

The integration process can be visualized as follows

The analytics database only caches analytics results and never stores issue summary or description. Analytics results are stored fully encrypted in the database. Caching is required to provide better user experience, by not mandating the user to wait 20-60 seconds every time.

Requirements

The following conditions must be met in order to use Issue AI Analytics for Jira:

  • N/A

Installation

Issue AI Analytics for Jira is installed as a standard application through the Universal Plugin Manager (UPM).

Follow these steps:

  1. Login as a system administrator
  2. Navigate to Apps -> Explore More Apps
  3. In the Search for apps field type Issue AI Analytics for Jira
  4. The app will be filtered In the results panel. Click on the app icon
  5. Click Try it for free and wait for the app to install
Note that Issue AI Analytics for Jira uses standard Atlassian licensing model. Trial period is available.
 

Configuration

There is a separate configuration page available, with all configuration options. To navigate it, go to to the Manage Apps screen, open Issue AI Analytics for Jira and select the Configure button.

The following configuration settings are available:

  • Project analytics customization – Enable project administrators to customize analytics on a per-project basis. If enabled, an additional “Issue AI Analytics” page will become available under “Project Settings”, that allows creating unique analytics tabs, specific for the particular project.
  • Analytics language – language in which analytics should be displayed. This setting will attempt to instruct the AI model to respond in a particular language, however, it is not always guaranteed that this will be considered.
  • Analytics customization – allows customization of what analytics are produced, by modifying what queries are sent for analysis (additional details available below)
  • Enable advanced features – enable a set of advanced app features. Advanced features are only available if you provide your AI API token. This means that you will be responsible for AI charges when issue analytics are generated. Advanced features also enables you to use Azure OpenAI Service.
  • [Advanced feature] OpenAI API Token – Obtain an API token from your OpenAI account, to use advanced features with OpenAI models. Not needed if you haven’t selected an OpenAI model from the Analytics Customization section.
  • [Advanced feature] Anthropic API Token – Obtain an API token from your Anthropic account, to use advanced features with Anthropic models. Not needed if you haven’t selected an Anthropic model from the Analytics Customization section.
  • [Advanced feature] Automatically enable analytics on issue creation – All issues will have AI Analytics enabled by default when they’re created.
  • [Advanced feature] Enable Azure OpenAI Service – Enables Azure OpenAI Service integration. When enabled, by default all analytics tabs utilize Azure OpenAI Service.
  • [Azure OpenAI Service] API Key – Your Azure API token
  • [Azure OpenAI Service] Resource Name – The name of your Azure OpenAI Resource
  • [Azure OpenAI Service] Deployment ID – The deployment name you chose when you deployed the model
Analytics Customization

A system administrator may change what tabs are displayed and how analytics are generated (at the analytics tab level). This allows support for various different use cases, specific to your organization. To change the default behaviour, you may override the settings of the existing tabs by modifying the following parameters:

  • Enabled – if the tab should be displayed
  • AI Provider – external AI provider and model to be used. Note that this field will not be visible, if you select to use Azure OpenAI Service.
  • Tab Name – name of the tab to be shown in the issue view screen
  • AI Query – you may change the default AI query with your own, using this field
  • Copy analytics to Jira field – Use a Jira field to store analytics results, in addition to the analytics panel
  • Field Name – Specify the field name. Don’t use summary or description fields. If this is a custom field, use the ID, such as: customfield_10042
  • Field Type – Type of the field. Available options are Paragraph or Short text. Short text is limited to 255 characters – if the analytics block is bigger than 255 characters, the field will not be updated. If an incorrect field type is selected, assignment will fail.
Analysis is always done based on issue summary and issue description, through an AI query. The AI query is a question that is sent for analysis and represents a single tab in the AI Analytics issue view.
For example, if you want to get analytics for time estimations you may input the following:
 
   Tab name: Time estimation
   AI Query: Provide a time estimation needed to complete the task
 
The above will result in the following example query being sent for analysis:
 
   Password reset <– This is the Issue Summary
   Please reset my Active Directory password as it has expired <– This is the Issue Description
   —
   Provide a time estimation needed to complete the task <– This is the AI Query configured for one analytics tab
 
The results will be displayed in a tab called Time estimation
 
Project Analytics Customization

A project administrator may change what tabs are displayed and how analytics are generated (at the analytics tab level), for a particular project. The configuration functionality is accessible through:

  • Project Settings menu, under Issue AI Analytics

The following settings are available and are applicable only for the current project:

  • Analytics language – language in which analytics should be displayed. This setting will attempt to instruct the AI model to respond in a particular language, however, it is not always guaranteed that this will be considered.
Additionally, each analytics tab can be customized through the following settings:
  • Enabled – if the tab should be displayed
  • Tab Name – name of the tab to be shown in the issue view screen
  • AI Query – you may change the default AI query with your own, using this field
The AI Provider can’t be selected at the project level – it can only be changed by a system administrator globally. 
Analysis is always done based on issue summary and issue description, through an AI query. The AI query is a question that is sent for analysis and represents a single tab in the AI Analytics issue view.
For example, if you want to get analytics for time estimations you may input the following:
 
   Tab name: Time estimation
   AI Query: Provide a time estimation needed to complete the task
 
The above will result in the following example query being sent for analysis:
 
   Password reset <– This is the Issue Summary
   Please reset my Active Directory password as it has expired <– This is the Issue Description
   —
   Provide a time estimation needed to complete the task <– This is the AI Query configured for one analytics tab
 
The results will be displayed in a tab called Time estimation

Bulk Enable Analytics Wizard

A system administrator may bulk enable analytics for multiple issues at once. This functionality is accessible through a separate menu item. To navigate it, go to to the Manage Apps screen, under Issue AI Analytics section and select the Bulk Enable Analytics Wizard menu. You may use this menu if Advanced Features is enabled.

Select a project and optionally add additional JQL filters. Once ready, press the Bulk Enable button.  It’s recommended to narrow down the number of issues as much as possible for faster processing.

You must not close your browser, or navigate away from this page while the wizard is running.

Note that enabling analytics for a big amount of issues, can take a significant amount of time. Always make sure to test the functionality on a smaller project.

Usage

To use Issue AI Analytics for Jira, navigate to an already created issue and click the Enable AI Analytics button. Wait for the analytics to be displayed (can take 20-60 seconds the first time). The analysis results are displayed as tabs, through a new panel in the Issue View screen.

Once analytics for an issue are generated, they will be available immediately whenever the Issue View page is re-opened. Analytics will be re-generated automatically, when issue summary or description changes.

Limitations

Issue AI Analytics for Jira has the following limitations:

  • Maximum size of the issue summary + issue description that will be analyzed is 4000 symbols. Content bigger than 4000 symbols will be truncated.
  • AI analytics will be generated with a maximum of 2048 tokens per tab
  • Maximum of 4 analytics tabs can be configured

Known Issues

Issue AI Analytics for Jira has the following known issues:

  • Generated analytics are not relevant or insufficient. Try adding more information in the issue description field with sufficient context. Remember, the AI provider doesn’t have any context except what is provided in the summary and description fields.
  • The following error message is displayed when Enable AI Analytics is clicked – “License for this feature is not set or has expired. Please contact your system administrator”. Please ensure your app is licensed.
  • The following error message is displayed when Enable AI Analytics is clicked – “Unable to generate AI Analytics – this could be caused by slower analysis or busy AI servers. Please try again in a bit”. Sometimes the AI provider can be overloaded and analytics generation takes more time. The request will eventually be processed in the background, so try refreshing the page in some time.

Functional Overview

Issue AI Analytics for Jira introduces new analytics functionality for Jira issues. It exposes an Enable AI Analytics button in the Issue View screen. Once button is pressed, the issue summary and description are sent for analysis to an external AI engine. The analysis results are displayed as tabs, through a new panel in the Issue View screen.

Example:

The tabs that are displayed and the analysis that is done by the AI engine is configurable, depending on the needs of your organization (reference Configuration section).

The following analytics tabs are available by default:

  • Instructions – Instructions with proposals and/or concrete solutions on how to solve tasks. It might contain steps to follow, general advise on how to approach the problem or in some cases, a direct solution. This section will become more accurate, as more detailed issue description is provided.
  • Description Improvements – Suggestions on how to improve clarity of an issue description. In some cases, the issue author might not provide a description that is detailed or clear enough – this section will help improve the quality and make it more understandable.
  • Definition of Done (DoD) – Most teams don’t spend enough time to write Definition of Done (DoD) for issues. This section will automatically generate the DoD. It will help the author to properly define, when an issue can be considered done.
  • Test Strategy – Complete strategy on how to test your issues are successfully completed. In most cases, a comprehensive suite of tests will be provided. This might include tests for non-functional requirements or corner cases, which are easily forgotten.

The following AI engines are supported:

  • OpenAI GPT-4o
  • OpenAI GPT-3.5-Turbo
  • Anthropic Claude 2.1
  • Anthropic Claude 2.0
  • Private AI

Architectural Overview

Issue AI Analytics integrates Jira with an external AI provider for analytics through a Heroku server. This can be visualized as follows:

The Deview Studios app server is hosted on the Heroku platform and is the entry point for the integration.

Issue AI Analytics for Jira Data Center also supports Private AI Mode. If enabled, the Heroku cloud server is not used and no data is sent to the Internet, with the expectation that there is a Private AI model hosted within the same datacenter:

Integration process triggers whenever the Enable AI Analytics button is pressed in the Issue View screen. Only the issue summary and description are sent for analysis.

First time analytics generation for an issue can take between 20-60 seconds. This is primarily due to the processing time needed by the AI provider. Once generated, analytics are cached in an analytics database and every subsequent display of analytics results in the Issue View screen is immediate. Analytics results will be re-generated, if issue summary or description is changed.

The integration process for Jira Data Center can be visualized as follows. There’s a difference in how analytics are generated, depending if Cloud AI mode or Private AI mode is enabled.

In this case the analytics database is the local Jira database. Caching is required to provide better user experience, by not mandating the user to wait 20-60 seconds every time.

If Private AI mode is enabled, Heroku cloud server is not used and no data is sent to the Internet.

Requirements

The following conditions must be met in order to use Issue AI Analytics for Jira:

  • For Cloud AI mode – connectivity to deview-dc-prod-30130e64aa60.herokuapp.com via HTTP/TCP on port 443 (HTTP proxy is supported)
  • For Private AI mode – an internally hosted Private AI server must be available and reachable from Jira

Installation

Issue AI Analytics for Jira is installed as a standard application through the Universal Plugin Manager (UPM).

Follow these steps:

  1. Login as a system administrator
  2. Navigate to Jira Administration -> Manage Apps
  3. In the Search the Marketplace field type Issue AI Analytics for Jira
  4. The app will be filtered In the results panel. Click the Free Trial or Buy Now buttons
  5. Accept the terms and conditions
  6. Once app is installed, click on Get license and follow the instructions
  7. You will need to choose the application integration mode as Cloud AI or Private AI. To do that, navigate to the Manage Apps screen, open Issue AI Analytics for Jira and select the Configure button. Reference the Configuration section on this page for more information on which mode to choose. 
Note that Issue AI Analytics for Jira uses standard Atlassian licensing model. Trial period is available.
 

Configuration

Issue AI Analytics for Jira Data Center requires that you select the app Integration mode, before you can use it.

There is a separate configuration page available, with all configuration options. To navigate it, go to to the Manage Apps screen, open Issue AI Analytics for Jira and select the Configure button.

The following configuration settings are available:

  • Integration mode – specifies how the application integrates with the selected AI provider. This setting is not selected by default and must be configured by a system administrator for the app to function. There’s two options to choose from: Cloud AI mode allows you to utilize cloud-hosted models, such as ChatGPT. Internet connectivity is required. Private AI mode allows you to use an internally-hosted AI model, for maximum privacy. No Internet connectivity is required.
  • Integration script (only available in Private AI mode) – defines the connection protocol with the Private AI mode (additional details available below)
  • Proxy settings (only available in Cloud AI mode) – HTTP proxy configuration settings required for Jira Data Center instances that don’t have direct Internet connectivity
  • Project analytics customization – Enable project administrators to customize analytics on a per-project basis. If enabled, an additional “AI Analytics” page will become available under “Project Administration”, that allows creating unique analytics tabs, specific for the particular project.
  • Issue analytics policy – Enable or disable issue analytics button and panel for issues. Exceptions can be applied through the Policy exceptions field.
  • Policy exceptions – apply project exceptions to the “Issue analytics policy”. This value must be a list of project keys, separated by comma (no whitespaces). If “Issue analytics policy” is set to Enable, all projects specified in this field will have issue analytics disabled. If “Issue analytics policy” is set to Disable, all projects specified in this field will have issue analytics enabled.
  • Analytics language – language in which analytics should be displayed. This setting will attempt to instruct the AI model to respond in a particular language, however, it is not always guaranteed that this will be considered.
  • Analytics customization – allows customization of what analytics are produced, by modifying what queries are sent for analysis (additional details available below)
Analytics Customization

A system administrator may change what tabs are displayed and how analytics are generated (at the analytics tab level). This allows support for various different use cases, specific to your organization. To change the default behaviour, you may override the settings of the existing tabs by modifying the following parameters:

  • Enabled – if the tab should be displayed
  • AI Provider – external AI provider and model to be used (this setting is not available when Private AI mode is selected)
  • Tab Name – name of the tab to be shown in the issue view screen
  • AI Query – you may change the default AI query with your own, using this field
Analysis is always done based on issue summary and issue description, through an AI query. The AI query is a question that is sent for analysis and represents a single tab in the AI Analytics issue view.
For example, if you want to get analytics for time estimations you may input the following:
 
   Tab name: Time estimation
   AI Query: Provide a time estimation needed to complete the task
 
The above will result in the following example query being sent for analysis:
 
   Password reset <– This is the Issue Summary
   Please reset my Active Directory password as it has expired <– This is the Issue Description
   —
   Provide a time estimation needed to complete the task <– This is the AI Query configured for one analytics tab
 
The results will be displayed in a tab called Time estimation
 
Project Analytics Customization

A project administrator may change what tabs are displayed and how analytics are generated (at the analytics tab level), for a particular project. The configuration functionality is accessible through:

  • Project Administration menu, under AI Analytics

The following settings are available and are applicable only for the current project:

  • Analytics language – language in which analytics should be displayed. This setting will attempt to instruct the AI model to respond in a particular language, however, it is not always guaranteed that this will be considered.
Additionally, each analytics tab can be customized through the following settings:
  • Enabled – if the tab should be displayed
  • Tab Name – name of the tab to be shown in the issue view screen
  • AI Query – you may change the default AI query with your own, using this field
The AI Provider can’t be selected at the project level – it can only be changed by a system administrator globally. 
Analysis is always done based on issue summary and issue description, through an AI query. The AI query is a question that is sent for analysis and represents a single tab in the AI Analytics issue view.
For example, if you want to get analytics for time estimations you may input the following:
 
   Tab name: Time estimation
   AI Query: Provide a time estimation needed to complete the task
 
The above will result in the following example query being sent for analysis:
 
   Password reset <– This is the Issue Summary
   Please reset my Active Directory password as it has expired <– This is the Issue Description
   —
   Provide a time estimation needed to complete the task <– This is the AI Query configured for one analytics tab
 
The results will be displayed in a tab called Time estimation
 
Private AI Mode Integration Script

With Private AI mode enabled, you can bring your own AI model as an analytics engine, used by the application. You will need to configure a simple Groovy integration script, that defines the communication protocol with the AI engine. Without configuring the script, the app will not function properly and will not be able to generate analytics. By default, a sample script is available that serves as example – it can be modified or adapted. 

The script will be invoked every time when new analytics have to be generated (separately per analytics tab). For example, if you have 4 analytics tabs enabled, the script will be invoked sequentially 4 times.

Script invocation lifecycle is as follows:

  1. Analytics for 1 tab for an issue need to be generated
  2. Groovy script execution is triggered
  3. Groovy script integrates with a Private AI model over HTTP or other means
  4. Groovy script returns a single String result, which contains the generated analytics
  5. Resulting analytics are persisted in the Jira database as cache
  6. Groovy script is invoked for remaining 3 tabs

For proper execution the following best practises must be followed:

  • The script must always return the analytics as a String result
  • In case an error occur, an Exception should be thrown. Example: throw new Exception(“An error has occured…”)
  • Always ensure you put a timeout to all blocking operations, especially HTTP calls.

The following variables are available for use within the script:

  • tabId – contains ID of the tab that will store the generated analytics of this run
  • tabName – contains name of the tab that will store the generated analytics of this run.
  • query – query configured for the current tab. It should be sent for AI analysis. 
  • analyticsLanguage – selected analytics language 
  • issue – complete Issue object. See https://docs.atlassian.com/software/jira/docs/api/9.14.1/com/atlassian/jira/issue/Issue.html 
  • issueSummary – summary field for issue 
  • issueDescription – description field for issue 
  • logger – may be used for logging. Use as: logger.info(“message”) | logger.debug(“message”) | logger.error(“message”). The resulting logs will be available in the standard atlassian-jira.log files – please ensure you enable the required level of logging for the com.deview_studios.atlassian.jira.dc.issue_ai_analytics package from the Logging and profiling menu.
It’s recommended to enable DEBUG logging while developing the script to see if you’re receiving errors. Please see section Logging. 
If you need help developing the script, don’t hesitate to reach to a Deview Studios representative through our standard channels. No additional charges apply.


The following is an example integration with OpenAI GPT-3.5 through the Groovy script:
import com.atlassian.jira.util.json.*
def aiQuery = issueSummary + "\r\n" + issueDescription + "\r\n---\r\n" + query // Build the query that will be sent for analysis
def payload = new JSONObject() // Prepare body as per API specification
    .put("model", "gpt-3.5-turbo")
    .put("messages", new JSONArray()
        .put(new JSONObject()
            .put("role", "user")
            .put("content", aiQuery)))
    .put("temperature", 1.0)
    .toString();
def post = new URL("https://api.openai.com/v1/chat/completions").openConnection() // Create a new request to specified URL
logger.debug("Preparing POST body: " + payload)
post.setConnectTimeout(60000) // Set a timeout to 60 seconds - make sure to always set timeouts post.setReadTimeout(60000) // Set a timeout to 60 seconds - make sure to always set timeouts
post.setRequestMethod("POST") // Specify HTTP method
post.setDoOutput(true) // Specify our HTTP request has a body
post.setRequestProperty("Content-Type", "application/json") // Set HTTP header
post.setRequestProperty("Authorization", "Bearer YOUR_TOKEN") // Set HTTP header
post.getOutputStream().write(payload.getBytes("UTF-8")) // Set HTTP body
def postRC = post.getResponseCode() // Send POST request
def response = post.getInputStream().getText() // Read response body
logger.debug("Response code is: " + postRC + "; Response body is: " + response)
if (postRC.equals(200)) { // Verify request is successful
    def json = new JSONObject(response) // Read as JSON
    def result = json.getJSONArray("choices").getJSONObject(0).getJSONObject("message").get("content") // Traverse JSON tree
    return result // This script must always return the produced analytics
}
throw new Exception("Unsuccessful request! HTTP response code is " + postRC + "; Response body is: " + response) // Throw error due to unsuccessful request
 

Usage

To use Issue AI Analytics for Jira, navigate to an already created issue and click the Enable AI Analytics button. Wait for the analytics to be displayed (can take 20-60 seconds the first time). The analysis results are displayed as tabs, through a new panel in the Issue View screen.

Once analytics for an issue are generated, they will be available immediately whenever the Issue View page is re-opened. Analytics will be re-generated automatically, when issue summary or description changes.

Limitations

Issue AI Analytics for Jira has the following limitations:

  • Maximum size of the issue summary + issue description that will be analyzed is 4000 symbols. Content bigger than 4000 symbols will be truncated. Not applicable in Private AI mode.
  • AI analytics will be generated with a maximum of 2048 tokens per tabNot applicable in Private AI mode.
  • Maximum of 4 analytics tabs can be configured

Logging

To enable logging for Issue AI Analytics, go to Jira Adminsitration -> System -> Logging and Profiling and click on Configure under Default loggers. Add the following entry: com.deview_studios.atlassian.jira.dc.issue_ai_analytics and set the logging level to DEBUG.

Known Issues

Issue AI Analytics for Jira has the following known issues:

  • Generated analytics are not relevant or insufficient. Try adding more information in the issue description field with sufficient context. Remember, the AI provider doesn’t have any context except what is provided in the summary and description fields.
  • The following error message is displayed when Enable AI Analytics is clicked – “License for this feature is not set or has expired. Please contact your system administrator”. Please ensure your app is licensed.
  • The following error message is displayed when Enable AI Analytics is clicked – “Unable to generate AI Analytics – this could be caused by slower analysis or busy AI servers. Please try again in a bit”. Sometimes the AI provider can be overloaded and analytics generation takes more time. The request will eventually be processed in the background, so try refreshing the page in some time.
  • The following error message is displayed immediately when Enable AI Analytics is clicked – “Unable to generate AI Analytics – this could be caused by slower analysis or busy AI servers. Please try again in a bit.”. If Cloud AI mode is enabled, the app might not be able to connect to the Heroku server. Internet connectivity is required for analytics generation – please ensure you have connectivity. If not, an HTTP proxy can be configured from the app configuration page. 
Deview Studios Logo

 

Contact Us

Copyright © 2022 Deview Studios