GuidesUpdated July 3, 2026

TFE Workspace Rename Guide - AIDE Project Standardization

how-toterraformtfeworkspacevaultjwtaide-project

TFE Workspace Rename Guide - AIDE Project Standardization

Overview

This guide provides step-by-step instructions for renaming TFE workspaces to include the aide-0085665- prefix for AIDE project standardization. This process preserves all Azure infrastructure resources while updating workspace names and configurations.

⚠️ CRITICAL COORDINATION REQUIREMENT: Workspace renames require three coordinated steps across different teams:

TFE Team: Rename workspace via API/GUI
Infrastructure Team: PR to update workspace names in code
Vault Team (Matt Iverson's team): Update Vault JWT authentication role

Failure to coordinate all three steps will result in authentication failures (context deadline exceeded errors).

Critical Prerequisites

Required Access

TFE token with workspace admin permissions (TFE_TOKEN environment variable)
Git repository access with branch creation permissions
Vault admin access to update JWT authentication roles (or contact Matt Iverson's team)
Terraform 1.5.0 or higher installed
jq command-line JSON processor installed
Access to TFE organization: uhg

Required Team Coordination

TFE Admin Team: Workspace rename permissions
Vault Team (Matt Iverson): JWT role update permissions
Infrastructure Team: PR review and approval

Important Safety Notes

⚠️ CRITICAL: This process does NOT delete or modify any Azure resources. It only renames TFE workspaces and updates configuration.

✅ Safe Operations:

Renaming TFE workspace via API
Updating Terraform configuration files
Using moved blocks for state migration
Adding lifecycle ignore_changes for workspace names
Updating Vault JWT authentication roles

❌ What Does NOT Happen:

Azure resources are NOT destroyed
Infrastructure is NOT modified
Service interruptions do NOT occur

Understanding the Three-Step Coordination

Why Three Steps Are Required

When a TFE workspace is renamed, three systems need to be updated:

TFE: The workspace name itself in Terraform Enterprise
Infrastructure Code: Backend configuration and workspace references in Git
Vault: JWT authentication role bound_claims that validate workspace identity

What Happens If Steps Are Not Coordinated

❌ If Vault JWT role is not updated:

Error: failed authenticating to Vault
error validating claims: claim "terraform_workspace_name" does not match
any associated bound claim values

This causes "context deadline exceeded" errors as TFE waits indefinitely for Vault credentials.

❌ If infrastructure code is not updated:

Terraform cannot find the backend workspace
State becomes inaccessible
Terraform operations fail

Coordination Workflow

sequenceDiagram
    participant TFE as TFE Admin
    participant Infra as Infrastructure Team
    participant Vault as Vault Team (Matt Iverson)

    TFE->>TFE: 1. Rename workspace via API
    TFE->>Infra: Notify: Workspace renamed
    Infra->>Infra: 2. Create PR with config updates
    Infra->>Vault: Notify: Update JWT
    Vault->>Vault: 3. Update JWT bound_claims
    Vault->>Infra: Confirm: JWT updated
    Infra->>Infra: Verify: terraform plan succeeds

Process Overview

Phase 1: Preparation

Identify workspaces to rename
Create feature branch
Set TFE token

Phase 2: TFE Workspace Module Updates (One-Time)

Add lifecycle block to ignore name changes
Add force_delete parameter

Phase 3: Individual Workspace Rename

Step 1: Rename workspace in TFE via API (TFE Admin)
Step 2: Update infrastructure code via PR (Infrastructure Team)
Step 3: Update Vault JWT authentication role (Vault Team)

Phase 4: Testing and Verification

Test TFE workspace changes
Test component workspace
Verify no infrastructure changes

Phase 5: Apply Changes

Apply TFE workspace changes
Apply component workspace changes
Verify Azure resources intact

Phase 6: Commit and Push

Commit all changes
Create pull request

Phase 1: Preparation

Step 1.1: Create Feature Branch

cd /path/to/your/repo
git checkout main
git pull origin main
git checkout -b feature/rename-workspaces-aide-prefix

Step 1.2: Identify Workspaces to Rename

List all workspaces that need the aide-0085665- prefix:

# List workspaces in terraform.tfvars
grep 'name.*=' tfe-workspace/terraform.tfvars | grep -v 'aide-0085665-' > workspaces_to_rename.txt

# Review the list
cat workspaces_to_rename.txt

Step 1.3: Set TFE Token

# Set your TFE token
export TFE_TOKEN="your-tfe-token-here"

# Verify token works
curl -s -X GET \
  --header "Authorization: Bearer $TFE_TOKEN" \
  --header "Content-Type: application/vnd.api+json" \
  "https://terraform.uhg.com/api/v2/organizations/uhg" | jq '.data.attributes.name'
# Should return: "uhg"

Phase 2: TFE Workspace Module Updates (One-Time Setup)

These changes apply to the tfe-workspace/workspace/main.tf module and need to be done once per repository.

Step 2.1: Add Lifecycle Block to Workspace Resource

File: tfe-workspace/workspace/main.tf

Add lifecycle { ignore_changes = [name] } to the tfe_workspace resource:

resource "tfe_workspace" "tfeworkspace" {
  name                   = var.name
  organization           = var.organization
  working_directory      = var.working_dir
  project_id             = var.project_id
  auto_apply_run_trigger = var.auto_apply_run_trigger
  auto_apply             = var.auto_apply
  trigger_prefixes       = var.trigger_prefixes
  file_triggers_enabled  = var.file_triggers_enabled
  force_delete           = var.force_delete
  vcs_repo {
    identifier         = var.vcs_repo
    branch             = "main"
    ingress_submodules = true
    oauth_token_id     = var.oauth_token_id
  }

  lifecycle {
    ignore_changes = [name]
  }
}

Purpose: Allows manual TFE workspace renames without Terraform trying to revert them.

Phase 3: Individual Workspace Rename Process

Repeat the following steps for each workspace you need to rename.

STEP 1: Rename Workspace in TFE (TFE Admin)

Get Workspace ID from TFE

Before renaming, retrieve the workspace ID:

# Set variables
OLD_WORKSPACE_NAME="tfews-epic-cogito-eastepic-pro-cus-001"
NEW_WORKSPACE_NAME="aide-0085665-tfews-epic-cogito-eastepic-pro-cus-001"

# Get workspace ID
WORKSPACE_ID=$(curl -s -X GET \
  --header "Authorization: Bearer $TFE_TOKEN" \
  --header "Content-Type: application/vnd.api+json" \
  "https://terraform.uhg.com/api/v2/organizations/uhg/workspaces/${OLD_WORKSPACE_NAME}" \
  | jq -r '.data.id')

echo "Workspace ID: $WORKSPACE_ID"
# Example output: ws-ABC123def456GHI

Verify Workspace Details (Safety Check)

# Verify you have the correct workspace
curl -s -X GET \
  --header "Authorization: Bearer $TFE_TOKEN" \
  --header "Content-Type: application/vnd.api+json" \
  "https://terraform.uhg.com/api/v2/workspaces/${WORKSPACE_ID}" \
  | jq '{
      id: .data.id,
      name: .data.attributes.name,
      workingDirectory: .data.attributes."working-directory",
      resourceCount: .data.attributes."resource-count",
      locked: .data.attributes.locked
    }'

Verify:

✅ Workspace name matches expected old name
✅ Working directory is correct
✅ Workspace is not locked (locked: false)

Rename Workspace via TFE API

# Rename the workspace in TFE
curl -X PATCH \
  --header "Authorization: Bearer $TFE_TOKEN" \
  --header "Content-Type: application/vnd.api+json" \
  --data "{
    \"data\": {
      \"type\": \"workspaces\",
      \"attributes\": {
        \"name\": \"${NEW_WORKSPACE_NAME}\"
      }
    }
  }" \
  "https://terraform.uhg.com/api/v2/workspaces/${WORKSPACE_ID}"

# Verify rename succeeded
curl -s -X GET \
  --header "Authorization: Bearer $TFE_TOKEN" \
  --header "Content-Type: application/vnd.api+json" \
  "https://terraform.uhg.com/api/v2/workspaces/${WORKSPACE_ID}" \
  | jq -r '.data.attributes.name'
# Should return: aide-0085665-tfews-epic-cogito-eastepic-pro-cus-001

✅ Checkpoint: Workspace renamed in TFE. Notify Infrastructure Team to proceed with Step 2.

STEP 2: Update Infrastructure Code (Infrastructure Team)

2.1: Update tfe-workspace/terraform.tfvars

File: tfe-workspace/terraform.tfvars

Find the workspace definition and update the name field:

Before:

  {
    name                = "tfews-epic-cogito-eastepic-pro-cus-001"
    directory           = "eastepicpro/cogitocentralus"
    trigger_prefixes    = [
      "eastepicpro/cogitocentralus/*"
    ]
  },

After:

  {
    name                = "aide-0085665-tfews-epic-cogito-eastepic-pro-cus-001"
    directory           = "eastepicpro/cogitocentralus"
    trigger_prefixes    = [
      "eastepicpro/cogitocentralus/*"
    ]
  },

2.2: Add Moved Block to tfe-workspace/main.tf

File: tfe-workspace/main.tf

Add a moved block at the end of the file:

moved {
  from = module.tfe_workspaces["tfews-epic-cogito-eastepic-pro-cus-001"]
  to   = module.tfe_workspaces["aide-0085665-tfews-epic-cogito-eastepic-pro-cus-001"]
}

Purpose: Handles the for_each key change in Terraform state without destroying/recreating the workspace.

2.3: Update Component Backend Configuration

File: <component-directory>/providers.tf

Example: eastepicpro/cogitocentralus/providers.tf

Update the backend workspace name:

Before:

terraform {
  backend "remote" {
    hostname     = "terraform.uhg.com"
    organization = "uhg"

    workspaces {
      name = "tfews-epic-cogito-eastepic-pro-cus-001"
    }
  }
  # ...
}

After:

terraform {
  backend "remote" {
    hostname     = "terraform.uhg.com"
    organization = "uhg"

    workspaces {
      name = "aide-0085665-tfews-epic-cogito-eastepic-pro-cus-001"
    }
  }
  # ...
}

2.4: Update workspace-id Tag

File: <component-directory>/terraform.tfvars.json

Example: eastepicpro/cogitocentralus/terraform.tfvars.json

Update the workspace-id tag in the tags section:

Before:

{
  "inputs": {
    "tags": {
      "aide-id": "AIDE_0085665",
      "workspace-id": "tfews-epic-cogito-eastepic-pro-cus-001",
      "source-code-repo": "https://github.com/optum-tech-compute/ohemr-epic-pro-001"
    }
  }
}

After:

{
  "inputs": {
    "tags": {
      "aide-id": "AIDE_0085665",
      "workspace-id": "aide-0085665-tfews-epic-cogito-eastepic-pro-cus-001",
      "source-code-repo": "https://github.com/optum-tech-compute/ohemr-epic-pro-001"
    }
  }
}

2.5: Update Vault Secret Lifecycle (If Applicable)

File: <component-directory>/vault.tf (if it exists) If the component has a vault_generic_secret resource that uses ${terraform.workspace} in the path, the secret will be recreated at the new path after the workspace rename.

Important: The secret at the old path will remain (Terraform won't delete it automatically). The password value from random_password in state will be preserved and written to the new path.

Example:

resource "vault_generic_secret" "admin_user_password" {
  provider = vault.production
  path = "kv/data/${terraform.workspace}/admin_user/admin_user_password"
  data_json = jsonencode({
    password = random_password.admin_user_password.result
  })
}

After workspace rename:

Old path: kv/data/tfews-epic-cogito-eastepic-pro-cus-001/admin_user/admin_user_password (remains untouched)
New path: kv/data/aide-0085665-tfews-epic-cogito-eastepic-pro-cus-001/admin_user/admin_user_password (created with same password)

Action required: Update any applications or scripts that reference the old Vault path. Note: The random_password resource maintains its value in state, so the same password will be written to the new Vault path automatically.

2.6: Commit and Create PR

cd /path/to/your/repo

# Add all modified files
git add tfe-workspace/workspace/main.tf
git add tfe-workspace/main.tf
git add tfe-workspace/terraform.tfvars
git add eastepicpro/cogitocentralus/providers.tf
git add eastepicpro/cogitocentralus/terraform.tfvars.json

# Commit with descriptive message
git commit -m "feat: rename workspace tfews-epic-cogito-eastepic-pro-cus-001 to aide-0085665- prefix

- Add lifecycle ignore_changes to tfe_workspace resource
- Rename workspace via TFE API to aide-0085665-tfews-epic-cogito-eastepic-pro-cus-001
- Add moved block for state migration
- Update backend configuration in component providers.tf
- Update workspace-id tag in terraform.tfvars.json
- Preserve all Azure infrastructure resources

NOTE: Vault JWT authentication role update required (Step 3) before apply succeeds."

# Push feature branch
git push origin feature/rename-workspaces-aide-prefix

# Create pull request
gh pr create \
  --title "feat: standardize workspace names with aide-0085665- prefix" \
  --body "Renames TFE workspaces to include AIDE project prefix.

⚠️ **VAULT TEAM ACTION REQUIRED BEFORE MERGE**:
Matt Iverson's team must update Vault JWT authentication role (Step 3).

## Changes
- TFE workspace renamed: tfews-epic-cogito-eastepic-pro-cus-001 → aide-0085665-tfews-epic-cogito-eastepic-pro-cus-001
- Backend configuration updated
- workspace-id tags updated
- No Azure infrastructure changes

## Testing
- terraform plan shows 0 infrastructure changes
- Only metadata (tags) updated

## Coordination Required
- [x] Step 1: TFE workspace renamed (completed)
- [x] Step 2: Infrastructure code updated (this PR)
- [ ] Step 3: Vault JWT role updated (Matt Iverson's team)

Refs: #<issue-number>"

✅ Checkpoint: PR created. Notify Vault Team to proceed with Step 3.

STEP 3: Update Vault JWT Authentication Role (Vault Team - Matt Iverson)

⚠️ CRITICAL: This step MUST be completed before the infrastructure PR is merged and applied, otherwise Terraform will fail to authenticate to Vault.

Problem

After renaming the TFE workspace, Vault's JWT authentication role still has the old workspace name in its bound_claims configuration. When TFE tries to authenticate with the new workspace name, Vault rejects the JWT token:

Error: failed authenticating to Vault
error validating claims: claim "terraform_workspace_name" does not match
any associated bound claim values

This causes "context deadline exceeded" errors as TFE waits indefinitely for Vault to provide Azure dynamic credentials.

Root Cause

The Vault JWT authentication role for TFE workspaces validates the workspace name claim in the JWT token. The bound claims must match the current workspace name in TFE.

Old configuration:

{
  "bound_claims": {
    "terraform_workspace_name": "tfews-epic-cogito-eastepic-pro-cus-001"
  }
}

Required new configuration:

{
  "bound_claims": {
    "terraform_workspace_name": "aide-0085665-tfews-epic-cogito-eastepic-pro-cus-001"
  }
}

Solution: Update Vault JWT Auth Role

Option 1: Via Vault CLI (Recommended)

# 1. Read current role configuration to get policies and JWT settings
vault read auth/tfe/role/aide-0085665-tfews-epic-cogito-eastepic-pro-cus-001

# 2. Update the role with new workspace name in bound_claims
# Note: Preserve existing bound_audiences and bound_issuer values from step 1
vault write auth/tfe/role/aide-0085665-tfews-epic-cogito-eastepic-pro-cus-001 \
  bound_claims='{"terraform_workspace_name":"aide-0085665-tfews-epic-cogito-eastepic-pro-cus-001"}' \
  bound_audiences="<value-from-step-1>" \
  bound_issuer="<value-from-step-1>" \
  policies="<policy-name-from-step-1>" \
  user_claim="terraform_workspace_id" \
  role_type="jwt" \
  token_policies="<policy-name-from-step-1>"

# 3. Verify update succeeded
vault read auth/tfe/role/aide-0085665-tfews-epic-cogito-eastepic-pro-cus-001

Option 2: Via Vault UI

Navigate to: https://vault.uhg.com/ui/vault/access/tfe/item/role/aide-0085665-tfews-epic-cogito-eastepic-pro-cus-001
Click Edit role

Update Bound Claims field:

{
  "terraform_workspace_name": "aide-0085665-tfews-epic-cogito-eastepic-pro-cus-001"
}

Click Save

Option 3: Via Terraform (If Vault roles are managed as IaC)

If there's a Terraform codebase that manages Vault auth roles:

resource "vault_jwt_auth_backend_role" "tfe_workspace" {
  backend         = "tfe"
  role_name       = "aide-0085665-tfews-epic-cogito-eastepic-pro-cus-001"
  role_type       = "jwt"
  bound_audiences = ["vault.workload.identity"]  # TFE default audience
  bound_issuer    = "https://terraform.uhg.com"  # Your TFE instance URL
  bound_claims = {
    terraform_workspace_name = "aide-0085665-tfews-epic-cogito-eastepic-pro-cus-001"
  }
  user_claim      = "terraform_workspace_id"
  token_policies  = ["<policy-name>"]
}

Verification

Transition Strategy: Support Both Names Temporarily

After updating the Vault JWT role:

cd /path/to/component/workspace
cd eastepicpro/cogitocentralus

# Reinitialize Terraform backend
terraform init -reconfigure

# Test authentication (should NOT see "context deadline exceeded")
terraform plan

Expected result: Terraform successfully authenticates to Vault and retrieves Azure credentials.

✅ Checkpoint: Vault JWT role updated. Notify Infrastructure Team that PR can be merged.

Phase 4: Testing and Verification

Step 4.1: Test TFE Workspace Changes

cd tfe-workspace

# Initialize (if needed)
terraform init

# Plan - should show moved resources, 0 to add, 0 to change
terraform plan

# Expected output:
# module.tfe_workspaces["tfews-epic-cogito-eastepic-pro-cus-001"] has moved to
# module.tfe_workspaces["aide-0085665-tfews-epic-cogito-eastepic-pro-cus-001"]
#
# Plan: 0 to add, 0 to change, 0 to destroy

Step 4.2: Test Component Workspace

cd ../eastepicpro/cogitocentralus

# Initialize with backend migration
terraform init -reconfigure

# Plan - should show minimal changes (workspace-id tag update only)
terraform plan

# Expected changes:
# - workspace-id tag updated
# - Vault secret may show replace (same password, new path)
# - All other resources should show no changes

Step 4.3: Verify No Infrastructure Changes

The plan should show:

✅ workspace-id tag changing on resources (safe metadata update)
✅ Vault secret recreated at new path (same password value)
❌ NO VM, storage, network, or infrastructure resource changes

If you see infrastructure resources being destroyed or modified, STOP and review the configuration.

Phase 5: Apply Changes

Step 5.1: Apply TFE Workspace Changes

cd tfe-workspace
terraform apply

# Review and confirm
# Type: yes

Step 5.2: Apply Component Workspace Changes

cd ../eastepicpro/cogitocentralus
terraform apply

# Review and confirm
# Type: yes

Step 5.3: Verify Azure Resources Intact

# Check VMs are running
az vm list --resource-group ohemr-rg-east_epic_cogito-pro-cus-001 --output table

# Verify workspace-id tag updated
az vm show --name <vm-name> --resource-group <rg-name> --query tags

Batch Processing Multiple Workspaces

For repositories with many workspaces (e.g., 48 workspaces renamed in commit bb0939d), you can script the process.

Batch Rename Script

#!/bin/bash
# batch_rename_workspaces.sh

set -e

# Workspace list (old name, component directory)
declare -A WORKSPACES=(
  ["tfews-epic-cogito-eastepic-pro-cus-001"]="eastepicpro/cogitocentralus"
  ["tfews-epic-odb-eastepic-pro-cus-001"]="eastepicpro/odb"
  ["tfews-epic-hyperspace-eastepic-pro-cus-001"]="eastepicpro/hyperspace"
)

# TFE configuration
TFE_ORG="uhg"
TFE_URL="https://terraform.uhg.com"
AIDE_PREFIX="aide-0085665-"

# Function to rename workspace in TFE
rename_workspace() {
  local old_name=$1
  local new_name=$2

  echo "==> Renaming: $old_name -> $new_name"

  # Get workspace ID
  WORKSPACE_ID=$(curl -s -X GET \
    --header "Authorization: Bearer $TFE_TOKEN" \
    --header "Content-Type: application/vnd.api+json" \
    "${TFE_URL}/api/v2/organizations/${TFE_ORG}/workspaces/${old_name}" \
    | jq -r '.data.id')

  if [ "$WORKSPACE_ID" == "null" ] || [ -z "$WORKSPACE_ID" ]; then
    echo "ERROR: Workspace $old_name not found"
    return 1
  fi

  echo "    Workspace ID: $WORKSPACE_ID"

  # Rename via API
  curl -s -X PATCH \
    --header "Authorization: Bearer $TFE_TOKEN" \
    --header "Content-Type: application/vnd.api+json" \
    --data "{\"data\":{\"type\":\"workspaces\",\"attributes\":{\"name\":\"${new_name}\"}}}" \
    "${TFE_URL}/api/v2/workspaces/${WORKSPACE_ID}" > /dev/null

  # Verify
  VERIFY_NAME=$(curl -s -X GET \
    --header "Authorization: Bearer $TFE_TOKEN" \
    --header "Content-Type: application/vnd.api+json" \
    "${TFE_URL}/api/v2/workspaces/${WORKSPACE_ID}" \
    | jq -r '.data.attributes.name')

  if [ "$VERIFY_NAME" == "$new_name" ]; then
    echo "    ✅ Rename successful"
    return 0
  else
    echo "    ❌ Rename failed"
    return 1
  fi
}

# Main execution
echo "Starting batch workspace rename..."
echo "=================================="

for old_name in "${!WORKSPACES[@]}"; do
  new_name="${AIDE_PREFIX}${old_name}"
  component_dir="${WORKSPACES[$old_name]}"

  rename_workspace "$old_name" "$new_name"

  echo ""
done

echo "=================================="
echo "Batch rename complete!"
echo ""
echo "⚠️ NEXT STEPS REQUIRED:"
echo "1. Update tfe-workspace/terraform.tfvars with new names"
echo "2. Add moved blocks to tfe-workspace/main.tf"
echo "3. Update backend in each component providers.tf"
echo "4. Update workspace-id tags in terraform.tfvars.json"
echo "5. **COORDINATE WITH VAULT TEAM** to update JWT roles"
echo "6. Run terraform plan in tfe-workspace"
echo "7. Run terraform plan in each component workspace"

Batch Vault JWT Role Update Script

#!/bin/bash
# batch_update_vault_jwt_roles.sh

set -e

# Workspace list (new names only)
WORKSPACES=(
  "aide-0085665-tfews-epic-cogito-eastepic-pro-cus-001"
  "aide-0085665-tfews-epic-odb-eastepic-pro-cus-001"
  "aide-0085665-tfews-epic-hyperspace-eastepic-pro-cus-001"
)

# Vault configuration
VAULT_AUTH_BACKEND="tfe"
POLICY_NAME="<your-policy-name>"  # Update with actual policy name

echo "Starting batch Vault JWT role updates..."
echo "========================================"

for workspace_name in "${WORKSPACES[@]}"; do
  echo "==> Updating JWT role for: $workspace_name"

  # Update Vault JWT role
  vault write "auth/${VAULT_AUTH_BACKEND}/role/${workspace_name}" \
    bound_claims="{\"terraform_workspace_name\":\"${workspace_name}\"}" \
    policies="${POLICY_NAME}" \
    user_claim="terraform_workspace_id" \
    role_type="jwt" \
    token_policies="${POLICY_NAME}"

  if [ $? -eq 0 ]; then
    echo "    ✅ JWT role updated"
  else
    echo "    ❌ JWT role update failed"
  fi

  echo ""
done

echo "========================================"
echo "Batch Vault JWT role update complete!"

Troubleshooting

Issue: "context deadline exceeded" Error

Symptom: Terraform hangs waiting for Vault credentials:

Operation failed: failed authenticating to Vault
error validating claims: claim "terraform_workspace_name" does not match
any associated bound claim values

Root Cause: Vault JWT authentication role not updated with new workspace name.

Solution: Complete Step 3 (Update Vault JWT Authentication Role) before applying changes.

# Verify current bound_claims in Vault
vault read auth/tfe/role/aide-0085665-tfews-epic-cogito-eastepic-pro-cus-001

# Update to new workspace name (preserve all existing JWT parameters)
vault write auth/tfe/role/aide-0085665-tfews-epic-cogito-eastepic-pro-cus-001 \
  bound_claims='{"terraform_workspace_name":"aide-0085665-tfews-epic-cogito-eastepic-pro-cus-001"}' \
  bound_audiences="<value-from-current-role>" \
  bound_issuer="<value-from-current-role>" \
  policies="<policy-name>" \
  user_claim="terraform_workspace_id" \
  role_type="jwt" \
  token_policies="<policy-name>"

Issue: Workspace Locked

Symptom: API returns "workspace is locked"

Solution:

# Unlock workspace via API
curl -X POST \
  --header "Authorization: Bearer $TFE_TOKEN" \
  --header "Content-Type: application/vnd.api+json" \
  "https://terraform.uhg.com/api/v2/workspaces/${WORKSPACE_ID}/actions/unlock"

Issue: Permission Denied

Symptom: API returns 403 Forbidden

Solution: Verify your TFE token has workspace admin permissions. Contact TFE admin or EPIC NATIONAL INSTANCE - SPT assignment group if you need access.

Issue: Workspace Not Found

Symptom: API returns 404 Not Found

Solution:

# List all workspaces to verify name
curl -s -X GET \
  --header "Authorization: Bearer $TFE_TOKEN" \
  --header "Content-Type: application/vnd.api+json" \
  "https://terraform.uhg.com/api/v2/organizations/uhg/workspaces" \
  | jq -r '.data[].attributes.name' | grep "epic"

Issue: Terraform Plan Shows Resource Destruction

Symptom: terraform plan shows Azure resources being destroyed

Solution:

STOP - Do not apply
Verify backend configuration points to correct workspace
Check terraform.workspace shows expected name:
```
terraform workspace show
```
Verify state file contains resources:
```
terraform state list
```
Review all configuration changes carefully

Issue: Moved Block Not Working

Symptom: Terraform still tries to create/destroy workspace

Solution:

Verify moved block syntax is correct
Ensure workspace was renamed in TFE UI/API before running plan

Check for_each keys match exactly:

# Old key
module.tfe_workspaces["tfews-epic-cogito-eastepic-pro-cus-001"]

# New key
module.tfe_workspaces["aide-0085665-tfews-epic-cogito-eastepic-pro-cus-001"]

Summary Checklist

For Each Workspace to Rename

Pre-Rename Coordination

Confirm all three teams available (TFE, Infrastructure, Vault)
Schedule coordination window
Document old and new workspace names

Step 1: TFE Admin Tasks

Get workspace ID from TFE API
Verify workspace details (name, directory, resource count)
Ensure workspace is not locked
Rename workspace in TFE via API
Verify rename succeeded
Notify Infrastructure Team: Workspace renamed

Step 2: Infrastructure Team Tasks

One-Time: Add lifecycle { ignore_changes = [name] } to tfe-workspace/workspace/main.tf
Update name in tfe-workspace/terraform.tfvars
Add moved block in tfe-workspace/main.tf
Update backend workspace name in <component>/providers.tf
Update workspace-id tag in <component>/terraform.tfvars.json
Commit all changes to feature branch
Push and create pull request
Notify Vault Team: PR ready, JWT update needed

Step 3: Vault Team Tasks (Matt Iverson)

Read current JWT role configuration
Update bound_claims with new workspace name
Verify JWT role update succeeded
Notify Infrastructure Team: JWT updated, PR can be merged

Testing (Infrastructure Team)

Run terraform plan in tfe-workspace (expect 0 changes, moved resources)
Run terraform init -reconfigure in component workspace
Run terraform plan in component workspace (expect tag updates only)
Verify no Azure infrastructure changes planned
NO "context deadline exceeded" errors

Apply (Infrastructure Team)

Get PR reviewed and merged
Apply changes in tfe-workspace
Apply changes in component workspace
Verify Azure resources intact

Post-Rename Verification

Check VMs running: az vm list
Verify workspace-id tags updated
Confirm Vault authentication working
Document completion in tracking issue

Affected Workspaces (AIDE-0085665 Project)

According to commit bb0939d, 48 workspaces were renamed with the aide-0085665- prefix. Each one requires coordination across TFE, Infrastructure, and Vault teams:

Cogito Workspaces (4)

aide-0085665-tfews-epic-cogito-eastepic-pro-cus-001
aide-0085665-tfews-epic-cogito-eastepic-pro-eus-001
aide-0085665-tfews-epic-cogito-westepic-pro-wus3-001
aide-0085665-tfews-epic-cogito-westepic-pro-cus-001

ODB Workspaces (8)

aide-0085665-tfews-epic-odb-eastepic-pro-cus-001
aide-0085665-tfews-epic-odb-eastepic-pro-eus-001
... (6 more)

Hyperspace Workspaces (6)

aide-0085665-tfews-epic-hyperspace-eastepic-pro-cus-001
... (5 more)

WSS, NetScaler, NetApp, Citrix (30)

aide-0085665-tfews-epic-wss-eastepic-pro-cus-001
... (29 more)

⚠️ CRITICAL: All 48 workspaces require Vault JWT role updates to prevent authentication failures.

Prevention Guide: Future Workspace Renames

To prevent authentication failures when renaming TFE workspaces in the future, follow this checklist:

Before Renaming

Document current state:
- List all workspaces to rename
- Document Vault JWT role configurations
- Identify component directories affected
Coordinate teams:
- Schedule coordination window with TFE, Infrastructure, and Vault teams
- Ensure all teams available during rename window
- Establish communication channel (Slack, Teams, etc.)
Prepare Vault updates:
- Pre-draft Vault CLI commands
- Verify Vault admin access
- Test Vault role update on non-production workspace first

During Rename

Execute in order:
- Step 1: TFE rename
- Step 2: Infrastructure PR created (DO NOT MERGE YET)
- Step 3: Vault JWT role updated
- Step 4: Infrastructure PR merged
- Step 5: Apply changes
Verify each step:
- TFE: Confirm workspace name in UI
- Vault: vault read auth/tfe/role/<workspace-name>
- Infrastructure: terraform plan shows no errors
Communication:
- Notify next team when step completes
- Document any issues immediately
- Do not proceed if any step fails

After Rename

Verify authentication:
- Run terraform plan in each renamed workspace
- Confirm no "context deadline exceeded" errors
- Check Vault audit logs for successful authentication
Update documentation:
- Document any issues encountered
- Update runbooks with lessons learned
- Share knowledge with team
Monitor:
- Watch for authentication errors in next 24 hours
- Check Azure resource health
- Verify no drift introduced

Support and Escalation

Contact Information

TFE Admin Team:

Assignment Group: EPIC NATIONAL INSTANCE - SPT
For: Workspace rename permissions, TFE API access

Vault Team (Matt Iverson):

Assignment Group: [Vault Admin Team]
For: JWT authentication role updates, Vault permissions

Infrastructure Team:

Assignment Group: EPIC NATIONAL INSTANCE - SPT
For: PR review, Terraform configuration questions

Escalation Path:

Team lead for respective area (TFE/Vault/Infrastructure)
AIDE-0085665 Project Lead
Platform Engineering Manager

Appendix: Quick Reference

TFE API Endpoints

# Get workspace by name
GET https://terraform.uhg.com/api/v2/organizations/{org}/workspaces/{workspace-name}

# Get workspace by ID
GET https://terraform.uhg.com/api/v2/workspaces/{workspace-id}

# Rename workspace
PATCH https://terraform.uhg.com/api/v2/workspaces/{workspace-id}

# Delete workspace (force)
DELETE https://terraform.uhg.com/api/v2/workspaces/{workspace-id}?force=true

# Unlock workspace
POST https://terraform.uhg.com/api/v2/workspaces/{workspace-id}/actions/unlock

Vault Commands

# List all JWT roles for TFE
vault list auth/tfe/role

# Read a specific role
vault read auth/tfe/role/<workspace-name>

# Update JWT role bound_claims (preserve existing JWT configuration)
vault write auth/tfe/role/<workspace-name> \
  bound_claims='{"terraform_workspace_name":"<new-workspace-name>"}' \
  bound_audiences="<value-from-existing-role>" \
  bound_issuer="<value-from-existing-role>" \
  policies="<policy-name>" \
  user_claim="terraform_workspace_id" \
  role_type="jwt" \
  token_policies="<policy-name>"

### Files Modified Per Workspace

1. `tfe-workspace/terraform.tfvars` - Update workspace name
2. `tfe-workspace/main.tf` - Add moved block
3. `<component>/providers.tf` - Update backend workspace name
4. `<component>/terraform.tfvars.json` - Update workspace-id tag

### Naming Convention

**Pattern**: `aide-0085665-{original-workspace-name}`

**Examples**:

- `tfews-epic-cogito-eastepic-pro-cus-001` → `aide-0085665-tfews-epic-cogito-eastepic-pro-cus-001`
- `tfews-epic-odb-westepic-test-wus3-001` → `aide-0085665-tfews-epic-odb-westepic-test-wus3-001`

---

**Document Version**: 2.0
**Last Updated**: January 21, 2026
**Author**: AIDE Project Team / Platform Engineering
**Contributors**: Amir Shahsavand (original TFE guide), Tom Hudak (Vault JWT integration)
**Repository**: ohemr-epic-megadoc

**Change Log**:

- v1.0 (2025-11-19): Original TFE workspace rename guide by Amir Shahsavand
- v2.0 (2026-01-21): Added Vault JWT authentication coordination (Step 3), updated with lessons learned from production issues