What is Meghafind and how to use the Scouts?

 

  1. Introduction

Meghafind is a Silicon Valley based startup, with a unique technology to benchmark a server’s performance, whether it is local or remotely located in the cloud. Such a benchmarking measurement is performed in a fraction of a second, with almost no impact on the run-time functionality of the server, performed by a user level application.

It has several potential applications, such as to benchmark a server or hardware configuration, compare performance across servers, or setup a performance baseline for a server and then monitor the performance over time. Because the measurements are almost feather touch, with no perceptible impact on the other applications running on the same servers, these measurements can be done at periodic levels, and notify when the server’s performance goes above or below an established baseline.

The measurements can be a singleton, or decomposed into individual elements of a server such as its CPU, Memory and Storage sub-systems. This enables detailed comparisons across different server configurations and make a prudent selection for running an application, which may be CPU or I/O bound.

Another usage is in the case of a sudden performance slowdown, to diagnose the elements of a server that contributed to the slowdown, such as too many memory or file I/O transactions. All this is done in the background without any changes required in your application code or server configurations. Data is stored remotely so multiple servers can be compared over time.

Meghafind Scouts have a small run-time foot print on the server and thus offer some interesting use cases for the users of private or public clouds, such as follows:

  • Which Machine is best for my application?

Given a local server, on which your application runs well, how do you right size a new virtual machine in the cloud, so that you are not under provisioning, or over paying for it?

Solution is easy with Meghafind scouts. Meghafind scouts in a single binary available on multiple operating systems under the name megaApp. Simply run megaApp first on your local server, and then on a new cloud machine, to see how the two compare.

  • Installing Scouts

Visit http://www.meghafind.com/install to download the correct version of megaApp for your OS.

You can test if things are working correctly by running the following command in your local directory where installation was done.


% ./megaApp


SCOUT NAME  TIME   OPS/SEC
CPU         0.2446 408.80
Memory      0.5102 196.00
Storage     0.2431 411.31

  • Setting up repository

Meghafind has a front end, which gives you instant performance data on the machine where scouts are installed. However, instantaneous data changes over time, and you may also like to compare one machine with another. This requires a Meghafind account, for which you need to send a simple request to info@meghafind.com, or complete the form located at the bottom of http://www.meghafind.com. Soon you will get an account code and secret code, both need to be entered into scouts_config.json file, which you would have downloaded in the previous step when installing scouts.

  • How to compare one machine with another?

Now you are all ready to run and submit the results of scouts to our backend server, by simply adding submit=true as follows:


% ./megaApp -submit=true


SCOUT NAME TIME   OPS/SEC
CPU        0.2877 347.54
Memory     0.5916 169.02
Storage    0.2935 340.74
--------------------------
Measurement Completed!
-------------------------
Measurement Id        5625c2b1e50f074b0a71507d
Machine Id            55d3fbebb1a1b363f2dcfe32

You will get a measurement ID and Machine ID. Latter is important, and needs to be entered into the scouts_config.json file.

Repeat this process for another machine, or on the same machine after its hardware or software configuration changes, to submit additional scouts data in the database.

Visit https://apps.meghafind.com/#compare

to see how your various machines or their configurations relatively compare with each other. An example result is shown below:

S.No Category MacPro2 Score Windows7VM64Bit Score
1 Summary 1 0.3621
2 CPU 1 0.2171
3 Memory 1 0.6641
4 Storage 1 0.2049

 

  • Is my Cloud Machine slowing down?

Once the right cloud machine is picked, establish its baseline, and then run scouts in a monitoring loop. Whenever the performance falls below a certain threshold, you will get an alert. Furthermore, you can check what caused the alert, e.g., cpu is doing fine but storage performance fell, so you can take the necessary measures including a halt of other applications on that server or migrating your critical application to another server. That another server can be picked using the compare use-case described previously.

  • How to setup a Baseline?

This options establishes a new reference point for subsequent monitoring runs, e.g., the server can be in different loading conditions at different times, so a baseline is made relative to which future scout runs shall be placed to determine if that machine has become slower or faster. The baseline itself is not a single measure, but a set of them (say 20, it is configurable) in terms of 3 scouts, stored as their average and standard deviation values.


% ./megaApp -mode=monitor_baseline -submit=true
CPU       0.2478 403.60
Memory   0.4907 203.79
Storage   0.2393 417.86
Baseline created. Baseline id=55e54760b1a1b363f2dcfe36

Multiple baselines can be established and stored, e.g., at no-load, normal load or peak load conditions to compare against in future. This enables an IT manager to run megaApp and determine how many standard deviations away the current performance is from the previously established baseline.

  • How to monitor a Server’s Performance?

A server is monitored by iteratively running megaApp scouts in foreground or background. Latter mode is preferred with a limit set on how much of CPU can be used by megaApp. This limit is set by cpu_utilization value in json file, default is 10%, to minimize loading. A sample run is shown below:


% ./megaApp -mode=monitor_loop -submit=true
CPU       Memory   Storage   Status
102.43    168.21    101.62    GREEN
30.41      53.34      53.28   GREEN
97.10      93.05      52.19   YELLOW
38.84      65.02      49.36   ORANGE
36.23      98.14      102.09  RED

You can also visually see it online for your machine at https://apps.meghafind.com/#machines

Advanced: How much time my application will take on a new server?

There is an application performance comparison mode, given the scout measurements on two servers, and application’s run-time on a target server, its run-time can uniquely and accurately predicted using Meghafind’s proprietary machine learning and prediction algorithms. Stay tuned on how to do this.