ODBA East Training/Dev/Test - Knowledge Transfer
ODBA Epic on Azure Knowledge Transfer
- East Training – MST, ACE 1-5, PLY, EXAM, REF, PREP
- East Dev– SBOX, POC
- East Test – OLDTST, TST, TSTRPT
The following Optum and Accenture team members were identified as key stakeholders for ODBA related knowledge transfer:
| Name | Organization |
|---|---|
| Jordan Lambert | Optum |
| Mason Chanthavatdy | Optum |
| John Keane | Optum |
| Laura Vaughn | Accenture |
| Christopher Land | Accenture |
The following items were identified as requiring Knowledge Transfer to transition the in-scope environments to the Optum ODBA team for ongoing support:
Server names and corresponding Epic groups
1. East Non-Prod VMs: Current Machines in Kuiper.csv
2. Kuiper URL: <https://kuiper.uhc.com>
3. System Pulse URL: <https://systempulse.uhc.com>
Server Naming Conventions
3.4 Virtual Machine Naming Convention
To ensure consistency across all resources it is important to follow a standard naming convention. A standardized format will allow for resources, such as virtual machines, to be easily identifiable.
VM Naming Convention
This standard naming convention will be used for naming hosts in Azure VMs:
-
Position 1: Hosting Platform (Z=Azure, A=AWS, G=Google, Oracle Cloud, etc.)
-
Position 2: Region (E, C, W)
-
Position 3: Environment (P=Prod, N=Non-Prod, D=Dev, R=Disaster Recovery, S=Shared)
-
Position 4: OS Platform Type (W=Windows, L=Linux)
-
Position 5: Purpose (AD=Active Directory, EPS=Epic Print Server, KUI= Kuiper, BCA, etc.)
-
Position 6: Instance Identifier (EE = Epic East, EW = Epic West, CL = Community Lead)
-
Position 7: Series Number starting at 001, incrementing as required.
-
Server hostnames will vary from 13-15 characters depending on role and/or purpose.
Z W P L EPS CL 001 Platform Region Environment OS Type Purpose Instance Identifier Series
VM Configuration Checks
- OS Changes
- ESMP
- Unit Infrastructure Test Cases (Unit Testing tab (filter on ODBA cases)): Optum_Epic to Azure Migration_Test Cases.xlsx
Confirm Server Access & corresponding tools required for access
- Ensure access to Hashicorp Vault (https://vault.uhg.com)
- Namespaces: Aide-0085665 (West), Aide-0085666 (East)
- Used for Static secrets
- Local admin passwords
- Follow-up with Infrastructure team to get naming decoder ring for local admin passwords Followed up with Indhu and Jeff 4/22 -
- Msnonprod service accounts
- Cloudtest infrastructure – ONLY Epic infrastructure in msnonprod domain
- Msnonprod.dsnonprod.uhc.com
- EMPs or ESMP passwords
- Etc.
- Local admin passwords
- Ensure access to Cyberark (same as on-prem) https://cyberark.optum.com/PasswordVault/v10/logon)
- View and Copy service account passwords
- Domain based secrets
- Epic service accounts: Epic on Azure Service Accounts.xlsx
- It is now accessible from Cloud SAW
- Ensure access to Cloud SAW
- VMWare Horizon
- Cloud SAW is the preferred way to RDP into Azure VMs
- Request Cloud SAW access via Secure
- Application: Secure Workbench
- Choose Create New ID to populate with Secondary ID
- If one does not exist, it will create a secondary ID for use.
- Role: Cloud SAW
- Request Cloud SAW access via Secure
- Ensure your elevated credentials are in the AD group below for your secondary account:
- ocnational_epic_azure_odba GPO is applied to Epic on Azure Windows VMs to allow admin access to this AD group
- Check adlookup.optum.com to ensure access has been granted
- Linux Server access
- Follow process outlined in this link: https://github.com/optum-tech-compute/ohemr-issue-tracker/issues
- Contact Manuel Palacios or Patrick O’Shea for any Linux access concerns
- Public/Private Key Access
-
Process for identifying Public Key (Putty)
- Search within this file path: \epicfilesnp.uhc.com\technical\Epic_Azure\Private_Keys
-
Search for your MS ID folder
-
the file with the “PUB” extension
-
technical > Epic_Azure > Private_Keys > jrobe224Name Date Modified Type id_ed25519 3/26/2025 12:45 PM File id_ed25519.pub 3/26/2025 12:45 PM PUB File privatekey.ppk 3/26/2025 12:45 PM PPK File
-
- Search within this file path: \epicfilesnp.uhc.com\technical\Epic_Azure\Private_Keys
-
Process for creating/identifying private key access
- \epicfilesnp.uhc.com\technical\Epic_Azure\Private_Keys
- Search for your MS ID folder
- Select the file with the “PUB” extension
- \epicfilesnp.uhc.com\technical\Epic_Azure\Private_Keys
-
technical > Epic_Azure > Private_Keys > jrobe224Name Date Modified Type id_ed25519 3/26/2025 12:45 PM File id_ed25519.pub 3/26/2025 12:45 PM PUB File privatekey.ppk 3/26/2025 12:45 PM PPK File
-
Backup and Restore
- Need to confirm the request process? ServiceNow Ticket? Working Session?
- Kunal performs ad hoc backup of the server using AZ Backup tool
- Kunal specifies the backup file to perform restore
- ODBA performs restore
- ODBA performs data integrity check
- ODBA confirms freeze/thaw script is working
- Kunal deletes restore file from server to prevent capacity issues~
Automation Handoff to ODBA
- Need to confirm Optum Stakeholders owning the automation configuration/maintenance
- Automation Team is using ansible/terraform tools to configure the Linux/storage components prior to handoff to ODBAs to initiate environment build
- OBDA Team will need to perform the following prior to handoff
- Confirm access to server following Github access process
- Validate ability to run instaserver to build the environment
- Here’s link to the Automation vs. ODBA responsibilities for the Linux servers: Automation Requirements to Solution Summary.xlsx
LDAP/User or Group Creation/SSH
- Need to collaborate with Larry Anderson to review /var logs for user access
- In future, this step will be owned by Optum’s operations
Azure Access
- Ensure log in and access to Virtual Machine details located in the portal (https://portal.azure.com)
- Currently not aware of the process to get “Contributor” access in Azure – Placeholder follow-up – Optum Cloud Operations – Followed up with Indhu and Jeff – 4/22 jm
- Use for Azure Bastion – Console level access to VMs if they are unreachable via RDP
List of deliverables
- Artifactory: repo1.uhc.com
- GitHub Scripts:
- Quick Reference Guide: Optum_Epic on Azure Infrastructure - Quick Reference Guide.xlsx
- Low-level Design Document: Low-Level_Design_v1.0.docx
- Deployment Plans: Deployment Plan
- Epic IP Address Allocation: EPIC IP Address Allocation-100%CDO.xlsx
- Network Architecture Diagram: Optum - Network Diagrams Draft v2.6-updated2.vsdx
Architecture & Business Continuity (DR considerations/config for specific environments)
This will be applicable for Production. It is not applicable for non-prod.
Server configuration details
- Please see the Bill of Materials that were used to request the infrastructure that has been deployed here: Deployed
Application Config details
This will be applicable for Production. It is not applicable for non-prod.
Monitoring
- System Pulse has been configured to match on-prem Alert Defs and users have been added to appropriate groups. Please ensure your account has Administrator access, the ECSA alerting group members are up-to-date, and that all the appropriate alerts are configured. (https://systempulse.uhc.com/SystemPulse)
- SMTP server: mailo2.uhc.com
- Netscaler VIP Status
- Follow up with Benny on read account to look at VIP status and configuration ( “followed up with Benny 4/22 -jm)
- Domain account? Local account?
SOP for admin tasks (e.g. add new disk, expand disk, upgrade SKU, add new machine, start/stop server, etc.)
- Disk Expansion/New Disk
- Work with Cloud Operations to update managed disk prior to the ODBA making the configuration
Linux Block Devices and Filesystem Information
Below is the output of lsblk, df -h, and xfs_growfs commands from the server.
Block Devices (lsblk output)
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINTS
sda 8:0 0 256G 0 disk
├─sda1 8:1 0 200M 0 part /boot/efi
├─sda2 8:2 0 2G 0 part /boot
└─sda3 8:3 0 253.5G 0 part
├─rootvg-tmplv 253:10 0 2G 0 lvm /tmp
├─rootvg-usrlv 253:11 0 10G 0 lvm /usr
├─rootvg-homelv 253:12 0 2G 0 lvm /home
├─rootvg-varlv 253:13 0 20G 0 lvm /var
└─rootvg-rootlv 253:14 0 20G 0 lvm /
sdb 8:16 0 490G 0 disk
└─epicvg-epiclvlv 253:8 0 490G 0 lvm /epic
sdc 8:32 0 500G 0 disk
└─epicvg-poc01lv 253:7 0 500G 0 lvm /epic/poc01
sdd 8:48 0 700G 0 disk
└─oldst01vg-oldst01lv 253:6 0 700G 0 lvm /epic/oldst01
sde 8:64 0 500G 0 disk
└─sbox01vg-sbox01lv 253:5 0 500G 0 lvm /epic/sbox01
sdf 8:80 0 700G 0 disk
└─tsrtpt01vg-tsrtpt01lv 253:4 0 700G 0 lvm /epic/tsrtpt01
sdg 8:96 0 700G 0 disk
└─tst01vg-tst01lv 253:3 0 700G 0 lvm /epic/tst01
sdh 8:112 0 600G 0 disk
└─epicjrnlvg-epicjrnlvl 253:2 0 600G 0 lvm /epic/jrn
sdi 8:128 0 590G 0 disk
└─epicaltjrnvg-epicaltjrnvl 253:1 0 590G 0 lvm /epic/altjrn
sdj 8:144 0 240G 0 disk
└─epicfilesvg-epicfileslv 253:0 0 235G 0 lvm /epicfiles/nonprdfiles
sr0 8:160 1 1.0G 0 rom
sdl 8:160 0 200G 0 disk
```text
---
## Filesystem Usage (`df -h /epic/oldst01`)
```text
Filesystem Size Used Avail Use% Mounted on
/dev/mapper/oldst01vg-oldst01lv 500G 3.6G 497G 1% /epic/oldst01
```text
---
## XFS Filesystem Info (`xfs_growfs /epic/oldst01`)
```text
meta-data=/dev/mapper/oldst01vg-oldst01lv isize=512 agcount=5, agsize=32112640 blks
= sectsz=4096 attr=2, projid32bit=1
= crc=1 finobt=1, sparse=1, rmapbt=0
= reflink=1 bigtime=1 inobtcount=1, nrext64=0
data = bsize=4096 blocks=131070976, imapct=25
= sunit=0 swidth=0 blks
naming =version 2 bsize=4096 ascii-ci=0, ftype=1
log =internal log bsize=4096 blocks=62720, version=2
= sectsz=4096 sunit=8 blks, lazy-count=1
realtime =none extsz=4096 blocks=0, rtextents=0
```text
- Data blocks changed from 131070976 to 183499776.
- Upgrade SKU
- .
# Patching Schedule/Process (Linux)
This is out of scope for the Epic on Azure team and details should be shared by Cloud Operations.
# Environment Servers
| **Environment Servers** | **West Hub** | **Central Hub** | **East Hub** |
|---------------------------|-----------------------------|-------------------------------|--------------|
| Training | | | |
| Development | | | |
| Full Sized Copies | | | |
| Disaster Recovery (DR) | | | |
| Prod Support | | | |
| Prod | 3rd Sunday<br>2am – 4am CST | 3rd Sunday<br>3am – 5am CST | |
---
Accenture Team will provide Hypercare through Friday, May 9, 2025; Optum’s ODBA team will take over ongoing support for this environment starting Monday, May 12, 2025.
Acknowledgement section:
| **Name** | **Organization** |
|-----------------------|------------------|
| Jordan Lambert | Optum |
| Mason Chanthavatdy | Optum |
| John Keane | Optum |
| Laura Vaughn | Accenture |
| Christopher Land | Accenture |