ODB SSH Access Method for Test and Administrative OS Users

Overview

1. SSH task code is in the tasks/ssh subdirectory of the utilities repo.
2. SSH playbook is located down the path playbooks/epic-on-azure/pb_odb_ssh.yml of the playbooks repo.
3. The SSH playbook is enabled within AWX as the ODB SSH v3 Job Template, which is further enabled by a set of 3 schedules - one each for CloudTest (non-functional due to environmental differences), non-prod, and prod. These run in the middle of the evening to hopefully reduce potential operational issues.

1. SSH Operational Documentation

The SSH automation pipeline:

1. Discovers the authoritative user population from AD groups.
   1. The ODB inventory in AWX is where the values for odb_grp_ssh_users are currently sourced from, as of 9/24/2025.
2. Mounts a secured network key share.
   1. The code and config does not mention this, but a user must have epic_nas_np for non-prod access or epic_nas access for prod access, as of 9/24/2025. The lack of these will typically manifest as an "unable to open file or directory" error visible in the user's mPuTTY window.
3. Ensures each discovered user has a key directory (and if absent, generates an ed25519 SSH keypair plus a PuTTY .ppk file).
4. Ensures each user’s public key is present in their ~/.ssh/authorized_keys.
5. Cleans up by unmounting the share.
6. Removes authorized_keys files for users who are no longer valid (deprovisioning enforcement).
   1. NOTE - users in the var odb_ssh_test_users are explicitly ignored when removing users, which is good for maintaining admin/OS CLI users outside of the other typical AD group pattern (i.e. when using the issue-tracker method to add an SSH key for a user to login), however this will present an issue when deprovisioning, as this list will need to be regularly maintained to reflect the current state of users allowed exempt from the typical process. If wanting to edit that var, it is also present in the ODB inventory in AWX, and the code to perform the delete is here.

The workflow expresses a closed-loop key lifecycle governed purely by AD group membership and a centralized storage share.

---

2. Deep Dive

2.1 Orchestration

Responsibilities:

• Drives the overall workflow; includes all other SSH task files.
• Mounts and unmounts the network share used as the canonical key repository.
• Prunes obsolete user authorized keys.

Notable Logic:

• Key generation is gated: a user directory’s absence triggers generation; presence implies reuse (idempotency).
• Removal of authorized_keys is conservative: it deletes the entire file if the username is not in valid_users + odb_ssh_test_users.
• Uses find /home -maxdepth 3 -path '/home/*/.ssh/authorized_keys' rather than the Ansible find module for precise pattern matching.

2.2 User Discovery

Purpose: Resolve AD group membership into a canonical, unique, sorted list of OS usernames (valid_users).

Operational Flow:

1. Ensure python-ldap library is installed (dependency for LDAP queries).
2. Include external DC discovery tasks (linux/ad_get_dcs.yml).
3. Pick a random domain controller (ad_site_dcs | random) to distribute load.
4. For each configured group (odb_ssh_user_groups), search AD for group object and extract the member attribute (DN list).
5. Flatten + dedupe collected DNs into all_member_dns.
6. For each DN, perform a base-scope lookup to retrieve sAMAccountName.
7. Aggregate, dedupe, sort, and register valid_users.

2.3 Key Material Creation

Purpose: For users missing a directory on the network share, generate and place new SSH credentials:

Artifacts generated (per user):

• id_ed25519 (private key)
• id_ed25519.pub (public key)
• privatekey.ppk (PuTTY format)

Key Steps:

1. Create a secure temporary directory (survives check mode).
2. Generate ed25519 keypair (community.crypto.openssh_keypair).
3. Convert private key to PPK via puttygen (guarded by creates: for idempotency).
4. Ensure per-user directory exists on share (0700).
5. Copy key files with restrictive permissions (0600).
6. Remove the temporary directory (sanitization).

Notes:

• Generation only occurs if a directory for the user is missing on the share.
• ansible_check_mode guarded copy prevents deploying fake/dry-run artifacts.

2.4 Authorized Key Enforcement

Purpose: Ensure each valid user’s ~/.ssh/authorized_keys contains the centrally managed public key.

Key Steps:

1. Ensure /home/<user> exists with correct owner/mode (0700).
2. Retrieve the user’s public key from the mounted share with slurp.
3. Install (append if necessary) the key using ansible.posix.authorized_key (non-exclusive mode).
4. Skip key installation when the share lacks the file (e.g., user’s key has not yet been created—shouldn’t normally happen if orchestration ordering is intact).

Considerations:

• Does not create parent directories above /home/<user>; expects standard home provisioning.
• exclusive: false allows coexisting manually managed keys (may be a policy decision—could be changed to enforce stricter control).

---

3. End-to-End Flow (Sequence)

flowchart TD
  A[Start Play] --> B[Include get_deduped_users]
  B --> C[ldap_search groups]
  C --> D[Resolve DNs -> sAMAccountNames]
  D --> E[valid_users fact]
  E --> F[Mount CIFS share]
  F --> G[Find existing share user dirs]
  G --> H{User dir exists?}
  H -- No --> I[Include create_missing_keypairs for user]
  H -- Yes --> J[Skip generation for user]
  I --> K[Repeat for remaining users]
  J --> K[All users evaluated]
  K --> L[Include ensure_pubkey_exists per user]
  L --> M[Unmount share]
  M --> N[Find authorized_keys under /home]
  N --> O{User still valid?}
  O -- No --> P[Remove authorized_keys]
  O -- Yes --> Q[Keep file]
  P --> R[End]
  Q --> R[End]

---

4. Variables & Facts Reference

4.1 Core Input Variables

4.2 Derived Facts

---

5. Operational Runbook Guidance

---

Escalations: If encountering an issue with the execution of this playbook or one of its effects, the best bet is to attend the EoA Office Hours call to seek help.

6. Example Ansible Role Invocation (Conceptual)

- name: Apply SSH key lifecycle enforcement
  hosts: linux_ssh_managed
  become: true
  vars:
    odb_ssh_user_groups:
      - UNIX_SSH_ENG
      - UNIX_SSH_PROD
    ssh_ad_base_dn: "DC=corp,DC=example,DC=com"
    ad_bind_dn_suffix: ",OU=Service Accounts,DC=corp,DC=example,DC=com"
    odb_ssh_mounted_key_share:
      unc: "//fileshare.example.com/sshkeys"
      mount_point: "/mnt/sshkeys"
      username: "svc_ssh_bind"
      password: "{{ vault_ssh_bind_password }}"
    ssh_user_group: "users"
    odb_ssh_test_users: ["labuser1"]
  roles:
    - ohemr-ansible-role-misc-utilities

---

7. Quick Reference: Task-to-Outcome Mapping

---