vCloud Director Troubleshooting.

By Karthic K 12 Dec, 2013 3 Comments
I started working with vCloud director from its initial release (vCloud 1.0) and I was thinking to write an article about the vCloud director troubleshooting for very long time but finally vCloud director(vCD) is going to die oops..I.e it is getting better and better , have a look at below articles
 
 
still it’s not late to write, as per the VMware KB2057589 support for vCD 5.5 is available till Q3 2017 and vCD 5.1 till Sep 2014.
 

What is in this post ?

Troubleshooting , resolving vCloud director issues are complex and tricky, there are no straight forward method to resolve those issues (For Ex:Go ahead just “Restart the service”, “Reboot the server” won’t work with vCloud director) . It needs more  patience and calm approach. To understand the behavior, it is important to understand all the components / services in the vCloud. I tried explaining my way of  approach, below are the topics I covered which I feel it is essential to read.

 
 

 

vCD & its components

You must understand what is vCloud director, what it can do, how it can be used to implement IaaS (Infrastructure as a Service) , there are numerous blog-post and references explaining what is vCloud director (vCD), some of them which I think it is important to read  are listed below.

 

vCloud Training :

 
I hope the above links will make clear about vCD, I would like to cover troubleshooting part of vCD, simply I listed the software components that makes vCD which is required for troubleshooting.
 

Components:

These are the collective software components which makes vCloud 
vCloud director
vCloud Networking and Security (Formerly vShield Manager)
vCenter Server
ESX(i)
vCenter Chargeback manager (optional)

 

vCD Cell Architecture:

 
To begin with vCD troubleshooting, one must understand its Architecture and components. I have summarized the component architecture and how it is connected.
Readers should keep this architecture diagram in mind. To approach an issue in vCD, for example if you are not able to view the vCloud director UI, it should strike that JAVA OSGI should be completely UP and running to access the UI, likewise the below diagram explains the internal connectivity and its relationship.
CellArchitechture
 
 
I have listed only the required components for troubleshooting, to know more about the vCD architechture review the below VMware KB.
 
Internal Components of vCD:
 

vCloud Cell is a Supported Linux Server with vCloud director software installed.

 

Transfer Service is involved when a user in vCloud director uploads a OVF / Media to their allocated resources

 

Console Proxy is proxy service for VMRC- Virtual Machine Remote Console, this component re-directs the VMRC from the ESXi host to vCloud director UI.

 

VC Proxy / Cell Listener service  is agent which sends / receives updates from the vCenter server, like power on/off VM, Create VM…Etc…

 

Active MQ – Message Bus is communication bus between multiple vCloud cells, it is for synchronization.

—

vCD log components:

 
If you encounter any issues in vCD you must review the logs  to find the cause and resolve the issue.
 
Login to your vCD cell via SSH (Often Putty) and navigate to vCD installation directory i.e /opt/vmware/vcloud-director/
 
Note: $VCLOUD_HOME is the environment variable for the vCloud director installation folder, use this to go to the location.
 
Example cd $VCLOUD_HOME/logs
 
These are the logs that will be useful in reviewing the vCD issues.

cell.log Start up log.
vcloud-container-debug.log  Debugging log messages from the cell
vcloud-container-info.log Warnings or errors in the cell
vmware-vcd-watchdog.log Log of the Watchdog service – which restarts the vCD service if it hangs or stops
 
Required services for vCloud director
  • vCenter services
  • VMware SSO services (Including all dependent services) (Applies on vCenter 5.1 and above)
  • VMware vSphere Profile-Driven storage Service  (Applies on vCenter 5.1 and above)
  • vShield Manager
  • vCloud services (vmware-vcd)

vCD Troubleshooting Checklist:

These are the common items must take into consideration before we begin troubleshooting

vCD Components:
  • vCenter and its DB
  • vCloud and its DB
  • vShield Manager
Check List:
  • Time Sync – Check if the time is synchronized of all vCD components
  • Name Resolution – Check if the vCD components are resolved by its FQDN
  • Valid SSL certificate on all vCD components
  • Connectivity between all vCD components
    • Perform Ping, telnet, netcat test to validate
  • Synchronization – Check if the synchronization of all vCD components
    • In vCloud director –> System tab –> vSphere resources –> vCenter, see if there is tick mark in the Status bar
    • In vCloud director –> System tab –> Cloud Resources –> Cells, check if the vCenter proxy is running
    • In vShield manager –> Configuration — >vCenter, Check the last inventory sync, it should contain the latest time stamp

 

vCD Issues:

 
Below listed issues are most common issues I have seen in vCloud Director.
  • Service Start-up Issue
  • VMRC Issues
  • Ovf Upload issue
  • Networking issues ..ahem
  • Resource allocation issues
  • Multi Cell and Load balancer
  • Guest Customization
  • Connectivity issues
    • vCenter
    • vShield
PS: I have published only the vCD startup issue, I am working on others and will publish shortly. 
 
This post first appeared on the Virtualization Express blog at virtualizationexpress.com, by Karthic Kumar +.
Copyright © 2013 – VirtualizationExpress +. All rights reserved. Not to be reproduced for commercial purposes without written permission.

3 thoughts on “vCloud Director Troubleshooting.”

  1. Pingback: vCloud Startup Issue | VirtualizationExpress

  2. Great post! I’m happy to find this your page. The internal connectivity and its relationship among vShield manager & vCD&vCenter servers also should be referred on this page isn’t it ? I really would like to know that..especially the internal provisioning procedure of Edge Gateway. Without those information, we cannot correspond to a trouble around vCloud network.

    Cheers,
    Kaz

  3. Hi Kaz,

    You are right, you need to understand the flow / concept to troubelshoot vCD networking issues, I am working on it and I will publish it at earliest.

    Thanks,
    Karthic

Comments are closed.