Compare commits

...

3 Commits

Author SHA1 Message Date
Cat Wu
5484a86d28 Increase oncall triage engagement threshold to 50
Updates the oncall triage automation to require 50+ engagements
(comments + reactions) before applying the oncall label, making the
criteria more conservative to focus on the most critical issues.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-10-28 20:04:53 -07:00
Cat Wu
5d0e5cf15f Add oncall triage slash command for issue management
Creates a new /oncall-triage command that automates the process of triaging GitHub issues and labeling critical ones for oncall attention.

The command:
- Fetches open bugs updated in last 3 days with 5+ engagements
- Systematically evaluates each issue for blocking severity
- Adds "oncall" label to truly blocking issues
- Provides summary of all issues that received the label

Includes guidance to use individual gh commands instead of bash loops to avoid approval prompts.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-10-21 13:44:18 -07:00
Cat Wu
113ea425ac Add automated oncall triage workflow
Implements a GitHub Actions workflow that automatically identifies and labels critical blocking issues requiring oncall attention.

Features:
- Runs every 6 hours via cron schedule
- Fetches open issues updated in the last 3 days (5 per page to avoid context overflow)
- Orders by UPDATED_AT DESC and stops when hitting issues older than 3 days
- Evaluates each issue for bug status, engagement level (5+), and blocking severity
- Uses LLM comprehension to determine true blocking impact, not keyword matching
- Applies "oncall" label to qualifying issues via GitHub MCP tools
- Provides detailed summary including processed count, labeled issues, and close calls

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-10-20 23:13:36 -07:00
2 changed files with 160 additions and 0 deletions

View File

@@ -0,0 +1,40 @@
---
allowed-tools: Bash(gh issue list:*), Bash(gh issue view:*), Bash(gh issue edit:*), TodoWrite
description: Triage GitHub issues and label critical ones for oncall
---
You're an oncall triage assistant for GitHub issues. Your task is to identify critical issues that require immediate oncall attention and apply the "oncall" label.
Repository: anthropics/claude-code
Task overview:
1. First, get all open bugs updated in the last 3 days with at least 50 engagements:
```bash
gh issue list --repo anthropics/claude-code --state open --label bug --limit 1000 --json number,title,updatedAt,comments,reactions | jq -r '.[] | select((.updatedAt >= (now - 259200 | strftime("%Y-%m-%dT%H:%M:%SZ"))) and ((.comments | length) + ([.reactions[].content] | length) >= 50)) | "\(.number)"'
```
2. Save the list of issue numbers and create a TODO list with ALL of them. This ensures you process every single one.
3. For each issue in your TODO list:
- Use `gh issue view <number> --repo anthropics/claude-code --json title,body,labels,comments` to get full details
- Read and understand the full issue content and comments to determine actual user impact
- Evaluate: Is this truly blocking users from using Claude Code?
- Consider: "crash", "stuck", "frozen", "hang", "unresponsive", "cannot use", "blocked", "broken"
- Does it prevent core functionality? Can users work around it?
- Be conservative - only flag issues that truly prevent users from getting work done
4. For issues that are truly blocking and don't already have the "oncall" label:
- Use `gh issue edit <number> --repo anthropics/claude-code --add-label "oncall"`
- Mark the issue as complete in your TODO list
5. After processing all issues, provide a summary:
- List each issue number that received the "oncall" label
- Include the issue title and brief reason why it qualified
- If no issues qualified, state that clearly
Important:
- Process ALL issues in your TODO list systematically
- Don't post any comments to issues
- Only add the "oncall" label, never remove it
- Use individual `gh issue view` commands instead of bash for loops to avoid approval prompts

120
.github/workflows/oncall-triage.yml vendored Normal file
View File

@@ -0,0 +1,120 @@
name: Oncall Issue Triage
description: Automatically identify and label critical blocking issues requiring oncall attention
on:
push:
branches:
- add-oncall-triage-workflow # Temporary: for testing only
schedule:
# Run every 6 hours
- cron: '0 */6 * * *'
workflow_dispatch: # Allow manual trigger
jobs:
oncall-triage:
runs-on: ubuntu-latest
timeout-minutes: 15
permissions:
contents: read
issues: write
steps:
- name: Checkout repository
uses: actions/checkout@v4
- name: Create oncall triage prompt
run: |
mkdir -p /tmp/claude-prompts
cat > /tmp/claude-prompts/oncall-triage-prompt.txt << 'EOF'
You're an oncall triage assistant for GitHub issues. Your task is to identify critical issues that require immediate oncall attention.
Important: Don't post any comments or messages to the issues. Your only action should be to apply the "oncall" label to qualifying issues.
Repository: ${{ github.repository }}
Task overview:
1. Fetch all open issues updated in the last 3 days:
- Use mcp__github__list_issues with:
- state="open"
- first=5 (fetch only 5 issues per page)
- orderBy="UPDATED_AT"
- direction="DESC"
- This will give you the most recently updated issues first
- For each page of results, check the updatedAt timestamp of each issue
- Add issues updated within the last 3 days (72 hours) to your TODO list as you go
- Keep paginating using the 'after' parameter until you encounter issues older than 3 days
- Once you hit issues older than 3 days, you can stop fetching (no need to fetch all open issues)
2. Build your TODO list incrementally as you fetch:
- As you fetch each page, immediately add qualifying issues to your TODO list
- One TODO item per issue number (e.g., "Evaluate issue #123")
- This allows you to start processing while still fetching more pages
3. For each issue in your TODO list:
- Use mcp__github__get_issue to read the issue details (title, body, labels)
- Use mcp__github__get_issue_comments to read all comments
- Evaluate whether this issue needs the oncall label:
a) Is it a bug? (has "bug" label or describes bug behavior)
b) Does it have at least 50 engagements? (count comments + reactions)
c) Is it truly blocking? Read and understand the full content to determine:
- Does this prevent core functionality from working?
- Can users work around it?
- Consider severity indicators: "crash", "stuck", "frozen", "hang", "unresponsive", "cannot use", "blocked", "broken"
- Be conservative - only flag issues that truly prevent users from getting work done
4. For issues that meet all criteria and do not already have the "oncall" label:
- Use mcp__github__update_issue to add the "oncall" label
- Do not post any comments
- Do not remove any existing labels
- Do not remove the "oncall" label from issues that already have it
Important guidelines:
- Use the TODO list to track your progress through ALL candidate issues
- Process issues efficiently - don't read every single issue upfront, work through your TODO list systematically
- Be conservative in your assessment - only flag truly critical blocking issues
- Do not post any comments to issues
- Your only action should be to add the "oncall" label using mcp__github__update_issue
- Mark each issue as complete in your TODO list as you process it
7. After processing all issues in your TODO list, provide a summary of your actions:
- Total number of issues processed (candidate issues evaluated)
- Number of issues that received the "oncall" label
- For each issue that got the label: list issue number, title, and brief reason why it qualified
- Close calls: List any issues that almost qualified but didn't quite meet the criteria (e.g., borderline blocking, had workarounds)
- If no issues qualified, state that clearly
- Format the summary clearly for easy reading
EOF
- name: Setup GitHub MCP Server
run: |
mkdir -p /tmp/mcp-config
cat > /tmp/mcp-config/mcp-servers.json << 'EOF'
{
"mcpServers": {
"github": {
"command": "docker",
"args": [
"run",
"-i",
"--rm",
"-e",
"GITHUB_PERSONAL_ACCESS_TOKEN",
"ghcr.io/github/github-mcp-server:sha-7aced2b"
],
"env": {
"GITHUB_PERSONAL_ACCESS_TOKEN": "${{ secrets.GITHUB_TOKEN }}"
}
}
}
}
EOF
- name: Run Claude Code for Oncall Triage
uses: anthropics/claude-code-base-action@beta
with:
prompt_file: /tmp/claude-prompts/oncall-triage-prompt.txt
allowed_tools: "mcp__github__list_issues,mcp__github__get_issue,mcp__github__get_issue_comments,mcp__github__update_issue"
timeout_minutes: "10"
anthropic_api_key: ${{ secrets.ANTHROPIC_API_KEY }}
mcp_config: /tmp/mcp-config/mcp-servers.json
claude_env: |
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}