ICMP ping issue by Windows base End Point Operations Agent

As you know vRealize Operation Manager can monitor remote devices by ICMP check. It is a simple checking method, we use it to monitor at basic level, our remote routers or switches.

Implementation process:

  1. Go Environment / All Objects / EP OPS Adapter / Remote Cheks Word
  2. From Actions menu select “Monitor this object remotely”
    • Add Display nama E.g.: WAN-1
    • Monitored from select your Agent
    • Under Check Method select ICMP check
    • Hostname: IP address of device
  3. Under Advanced settings you can change “sotimeout” and “Collection Interval” (we use one minute)
  4. Click OK

Under Environment / All Objects / EP OPS Adapter / ICMP Check tree you will find all your ICMP checked objects, and you can select one of them for details or All Metrics data.

Now, you are able to create an Alert, base on resource availability state. Yes, but we received lots of alert from vROps, becouse the resources wasn’t available many times. We tried rise up sotimeout. Not worked.

We changed Agent log level to Debug and we saw many of this messages.

26-09-2017 20:23:36,043 CEST DEBUG [pool-1-thread-11] [Collector] name=ICMP Check, thread=pool-1-thread-11, result=Tue Sep 26 20:23:36 CEST 2017 (/192.168.xxx.xxx) null values={Availability=0.0, ResponseTime=19969.0}

(In my next Article I will show you how you can change EP OPS Agent’s log level and file size.)

ICMP

We opened a new case at GSS (VMware Global Service Support) about this problem, but fortunetly in the maintime (under weeks – waited for GSS) we find a Release Note for vRealize Operation Manager 6.2. The subscription in this note looks like very similar as our problem 🙂

No data returned when running ICMP check
When attempting to run an ICMP check for remote monitoring from an Agent running on a Windows platform, no data is returned.
Workaround: Do not use ICMP checking from an agent that is deployed on a Windows platform.

Link: http://pubs.vmware.com/Release_Notes/en/vrops/62/vrops-62-release-notes.html

We changed the Agent from Windows base OS to Linux OS, and in the last weeks we didn’t revecive fals alerts. I’m really sad becouse we use vRealize Operation 6.6.1 and the problem, wich was discovered in 6.2,  is still exists in the latest release.

SWAP drive Alerts in vROps

In our MS SQL environment we use dedicated swap drive wich almost is full becouse of we set it up Custom size. Despite of it is a normal behavior, vROps generate “Guest file system space usage” Alert. In this article I would like to show you how to manage this situation in vRealize Operation Manager and avoid unnecessary alerts.

In this example we have two similar VM with three drive: C: for OS, H: for Kernel and S: for SWAP. From Kernel drive you could figure out, it is a SAP environemt 🙂

SWAP_Alert_vRealize_Operation_001

We want to keep alert to C: and H: drive, but we don’t get alert from S: (swap) drive, unless it is full.

First of all we can create new symptoms to this VMs. You can see the table below. Create Warning (85%) , Immediate (90%) and Critical (95%) symptom to C: and H: drive too. As you can see in table.

For S: drive enough Info level and condition will be “is greather than or equal to” with value 100 (precent). So the swap drive is full, vROps will create an Alert. 100% is just an advice, you can change the value as you wish, customize your environment.

SWAP_Alert_vRealize_Operation_003

We can use “Guest File System stats / Guest File System Usage (%)” metric under effected drive. As you can see, in this list we don’t see S:\ drive.

In the interest of we see our VM’s S:\ drive click the little grey icon next to Metrics select list form.

In the new window you can find a list of all VMs, select that VM wich want to configure, and click OK, than you can see the all drive under Guest File System stats.

Choose drive eg. C:\ and under the drive letter you will find the “Guest File System Usage (%)” metric. Drag and drop to right side of window.

Create all symptoms from list of below.

 

 

 

 

SWAP_Alert_vRealize_Operation_004

Create this Symptomes

Base Object Type Metrics Threshold Name Level Condition Value
vCenter Adapter / Virtual Machine Guest File System stats / C:\ Guest File System Usage (%) Static C Drive space usage at Warning level Warning is greather than or equal to 85
vCenter Adapter / Virtual Machine Guest File System stats / C:\ Guest File System Usage (%) Static C Drive space usage at Immediate level Immediate is greather than or equal to 90
vCenter Adapter / Virtual Machine Guest File System stats / C:\ Guest File System Usage (%) Static C Drive space usage at Critical level Critical is greather than or equal to 95
vCenter Adapter / Virtual Machine Guest File System stats / H:\ Guest File System Usage (%) Static H Drive space usage at Warning level Warning is greather than or equal to 85
vCenter Adapter / Virtual Machine Guest File System stats / H:\ Guest File System Usage (%) Static H Drive space usage at Immediate level Immediate is greather than or equal to 90
vCenter Adapter / Virtual Machine Guest File System stats / H:\ Guest File System Usage (%) Static H Drive space usage at Critical level Critical is greather than or equal to 95
vCenter Adapter / Virtual Machine Guest File System stats / S:\ Guest File System Usage (%) Static S Drive space usage at Info level (SWAP) Info is greather than or equal to 100

From this symptoms we can create new Alert.

Create new Alert

Name Base Object Type Impact Criticality Alert Type and Subtype Wait Cycle Cancel Cycle
SAP LFP Apps virtual machine guest file systems are running out of disk space vCenter Adapter/Virtual Machine Health Symptom Based Virtualization/Hypervisor: Capacity 1 1

If you did everything right, you will see similar things as picture below.

SWAP_Alert_vRealize_Operation_002

Create new Custom Group

Ok, now you can create a new Custom group under Environement / Environment overview menu. Hit the green cross on top of the menu, and configure membership criteria. Part of Policy leave empty for now. In this eximpe I add exact VMs to this group. The name of the new group: “Disable SWAP drive alert”

SWAP_Alert_vRealize_Operation_005

Create new Policy

Go Administration / Policies / Policy Library and add new one. In the “6. Alert / Symptom Definitions” menu disable “One or more virtual machine guest file systems are running out of disk space” default alert, and Enable the newly created alert, in this case “SAP LFP Apps virtual machine guest file systems are running out of disk space”

SWAP_Alert_vRealize_Operation_006

In “8. Apply Policy to Groups” part select the custom group wich was created before, and click OK.

SWAP_Alert_vRealize_Operation_007

If you have any question pls. leave comment.

vRealize Operation Manager upgrade from 6.5 to 6.6 – License issue

I think you heard about new vRealize Operation Manager release. If not check this Release Notes

It was subject of many blogs:

I admit, the new UI is brilliant, my colleagues are sharing this opinion. So U can read about new features and integration with VMware Log Insigt and Business Management.

The upgrade process same as from 6.4 to 6.5. I found a good blog which is present it step by step: https://blogs.vmware.com/management/2017/05/upgrading-to-the-new-vrealize-operations-6-5-step-by-step-david-davis.html

After upgrade process was done we realize in vRealize 🙂 something strange. When we selected a VM we seen a watermarked message in the background, as you can see in the picture below. “License is invalid”

vRealize_operation_manager_6.6_license_001
License is invalid watermark

The solution is simple, you can go Administration / Management / Licensing page and select License Groups tab in vROPS.

Check the Licensable Usage column and if you see 0 value in Product Licensing row it will be the problem.

vRealize_operation_manager_6.6_license_002
Check Licensable Usage

In this case select Product Licensing row and click pencil icon on top. Under vRealize Operation Manager select your license key or keys depends on your environment and click next.

vRealize_operation_manager_6.6_license_003
Select your license key or keys

Define a membership criteria as you wish or use legacy settings. They didn’t change in our environment. If you could advice about filering I would give you, pls. contact me. Click next.

Check Preview then click Finish button.

!! Don’t forget push the refresh button on License Keys page however you can’t see the changes on License Groups page 🙂

vRealize_operation_manager_6.6_license_004
Push the Refresh button!!!

When you did everything right on License Groups page you will see a bigger value then 0 in Licensable usege column.

I hope this short article will help for you, when you experience similar problem after your upgrade.

Enable SSH VMware vRealize Operation Manager (vROps 6.x)

We have investigated our problem with VMware Support and they want run a query in Postgres database. Usually I use Putty for SSH connection, but before I could do this I should on the ssh on vROps:

  1. Open vROps Console and press “Alt+F1” to switch console
  2. Type “root” user
  3. In default vRealize root user hasan’t password
  4. Press enter for Old password and the promt will change New password
  5. Type new password and type again
  6. When we are able to login by root type: “chkconfig sshd on”
  7. Start service: “service sshd start”

Go back to Putty and connect to vRealize Operation Manager by ssh.

Cisco UCS memory inventory by PowerShell

We upgraded our Cisco UCS environment, every host got extra memory from 128 to 256GB. Before we did it we had mesured CPU usage by VMware vRealize Operation Manager. It was a great help to this process. We ware able to proof the CPU performance will be ok with this memory size. We would have liked to incrase our VMware environment density.

We had to coolect every memory module serial numbers, and information about that witch chassis consist of witch memory modul.

We used to Cisco PowerShell for this task. Yes, exist PowerShell tool for Cisco UCS 🙂

You can download Cisco UCS Power Too pack (click link) and install your computer.
(You must restart your computer after install process)

Collect memory serial numbers, and UCS chassis information:

Get-UcsMemoryUnit | Export-Csv c:\tmp\UCS_Memory.csv -Encoding utf8

Get UCS chassis serials:

Get-UcsBlade | Export-Csv c:\tmp\UCS_Serials.csv -Encoding utf8

Whit this two Excel you are able to determine witch memory modul are witch chassis.