Showing posts with label nmonchart. Show all posts
Showing posts with label nmonchart. Show all posts

Thursday, December 29, 2022

Using nmonchart and python to visualize the nmon output from linux server

 Objective: In the previous blog, we saw how to collect nmon data and convert the same to html files. Now we will see how we can host it in a web.

To host the nmon charts using a simple solution follow the below steps...

1) Let us recall the directory structure first...

Directory structure:
Nmon data loc: /pgBACKUP/nmondata
nmonchart bin: /pgBACKUP/nmonchartbin/nmonchart
nmonchart dest: /pgBACKUP/nmondata/nmonchartop

Let us one more refinement...
We will create one directory per server in nmonchart dest, this means we can use one url from one machine (using a remote nfs, we can collect data from all the servers) to access multiple machines chart.

Directory structure:
Nmon data loc: /pgBACKUP/nmondata
nmonchart bin: /pgBACKUP/nmonchartbin/nmonchart
nmonchart dest for 10.0.2.7 machine: /pgBACKUP/nmondata/nmonchartop/10.0.2.7/
nmonchart dest for 10.0.2.8 machine: /pgBACKUP/nmondata/nmonchartop/10.0.2.8/
nmonchart dest for 10.0.2.9 machine: /pgBACKUP/nmondata/nmonchartop/10.0.2.9/

So how it looks:

[root@localhost 10.0.2.8]# ls -altr
total 448
-rw-r--r--. 1 root root 457432 Dec 29 18:26 localhost_221229_1659.html
drwxr-xr-x. 2 root root     40 Dec 29 18:33 .
drwxr-xr-x. 3 root root     22 Dec 29 18:39 ..
[root@localhost 10.0.2.8]# pwd
/pgBACKUP/nmondata/nmonchartop/10.0.2.8
[root@localhost 10.0.2.8]


2) All we need now is python, if we already have python installed, you can just start a simple webpage from the location you have the host wise directory structure.


[root@localhost vagrant]# cd /pgBACKUP/nmondata/nmonchartop
[root@localhost nmonchartop]# ls -altr
total 0
drwxr-xr-x. 3 root root 193 Dec 29 18:13 ..
drwxr-xr-x. 2 root root  40 Dec 29 18:33 10.0.2.8
drwxr-xr-x. 3 root root  22 Dec 29 18:39 .
[root@localhost nmonchartop]# python -m SimpleHTTPServer
Serving HTTP on 0.0.0.0 port 8000 ...
>>>> now our website is up.


3) Before trying to access the web, please be sure to exempt the 8000 port from firewall or for time being stop it completely.


4) In case your testing this in a virtual box using NatNetwork, please do add port forwarding like below...

127.0.0.1:8000 -> 10.0.2.8:80000

5) Then try access the url...

http://127.0.0.1:8000/



6) Navigate through the url.






7) Observe the python command line output for incoming accesses....

[root@localhost nmonchartop]# python -m SimpleHTTPServer
Serving HTTP on 0.0.0.0 port 8000 ...
10.0.2.2 - - [29/Dec/2022 18:46:01] "GET / HTTP/1.1" 200 -
10.0.2.2 - - [29/Dec/2022 18:46:46] "GET /10.0.2.8/ HTTP/1.1" 200 -
10.0.2.2 - - [29/Dec/2022 18:47:22] "GET /10.0.2.8/localhost_221229_1659.html HTTP/1.1" 200 -


So now we have the data accessible from web url.

Thank you!

---this closes the blog.


Using nmon in linux to collect performance data

Objective:  Let us nmon to collect performance data from linux using the frequency as we need.

Step 1 - nmon installation in linux:

Search for nmon in repo:
[root@localhost ~]# yum search nmon
Loaded plugins: auto-update-debuginfo, fastestmirror
Loading mirror speeds from cached hostfile
 * base: anorien.csc.warwick.ac.uk
 * epel: mirrors.xtom.de
 * epel-debuginfo: mirrors.xtom.de
 * extras: uk.mirrors.clouvider.net
 * updates: mirrors.coreix.net
============================================== N/S matched: nmon ==============================================
nmon-debuginfo.x86_64 : Debug information for package nmon
xfce4-genmon-plugin-debuginfo.x86_64 : Debug information for package xfce4-genmon-plugin
conmon.x86_64 : OCI container runtime monitor
nmon.x86_64 : Nigel's performance Monitor for Linux                         <<<<<<<<< this is what we need
xfce4-genmon-plugin.x86_64 : Generic monitor plugin for the Xfce panel
  Name and summary matches only, use "search all" for everything.
[root@localhost ~]#
Installing nmon:
[root@localhost ~]# yum install nmon.x86_64
Loaded plugins: auto-update-debuginfo, fastestmirror
Loading mirror speeds from cached hostfile
epel/x86_64/metalink                                                                    |  26 kB  00:00:00
epel-debuginfo/x86_64/metalink                                                          |  26 kB  00:00:00
 * base: anorien.csc.warwick.ac.uk
 * epel: mirrors.xtom.de
 * epel-debuginfo: mirrors.xtom.de
 * extras: mirror.freethought-internet.co.uk
 * updates: mirror.bytemark.co.uk
base                                                                                    | 3.6 kB  00:00:00
base-debuginfo                                                                          | 2.5 kB  00:00:00
epel                                                                                    | 4.7 kB  00:00:00
epel-debuginfo                                                                          | 3.0 kB  00:00:00
extras                                                                                  | 2.9 kB  00:00:00
pgdg-common/7/x86_64/signature                                                          |  198 B  00:00:00
...
updates                                                                                 | 2.9 kB  00:00:00
(1/4): epel/x86_64/group_gz                                                             |  99 kB  00:00:00
(2/4): epel/x86_64/updateinfo                                                           | 1.0 MB  00:00:00
(3/4): epel/x86_64/primary_db                                                           | 7.0 MB  00:00:00
(4/4): epel-debuginfo/x86_64/primary_db                                                 | 901 kB  00:00:05
Resolving Dependencies
--> Running transaction check
---> Package nmon.x86_64 0:16g-3.el7 will be installed
--> Finished Dependency Resolution
Dependencies Resolved
===============================================================================================================
 Package                 Arch                      Version                       Repository               Size
===============================================================================================================
Installing:
 nmon                    x86_64                    16g-3.el7                     epel                     70 k
Transaction Summary
===============================================================================================================
Install  1 Package
Total download size: 70 k
Installed size: 156 k
Is this ok [y/d/N]: y
Downloading packages:
nmon-16g-3.el7.x86_64.rpm                                                               |  70 kB  00:00:05
Running transaction check
Running transaction test
Transaction test succeeded
Running transaction
  Installing : nmon-16g-3.el7.x86_64                                                                       1/1
  Verifying  : nmon-16g-3.el7.x86_64                                                                       1/1
Installed:
  nmon.x86_64 0:16g-3.el7
Complete!
[root@localhost ~]#


Launch nmon:

Test launch of nmon


lnmonq16gqqqqqqqqqqqqqqqqqqqqqHostname=localhostqqqqRefresh= 2secs qqq06:46.18qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqk
x                                                                                                             x
x   ------------------------------                                                                            x
x    _ __  _ __ ___   ___  _ __          For help type H or ...                                               x
x   | '_ \| '_ ` _ \ / _ \| '_ \          nmon -?  - hint                                                     x
x   | | | | | | | | | (_) | | | |         nmon -h  - full details                                             x
x   |_| |_|_| |_| |_|\___/|_| |_|                                                                             x
x                                        To stop nmon type q to Quit                                          x
x   ------------------------------                                                                            x
x                                                                                                             x
x   CentOS Linux release 7.9.2009 (Core) VERSION="7 (Core)"                                                   x
x   Vendor=GenuineIntel Model=11th Gen Intel(R) Core(TM) i5-1135G7 @ 2.40GHz                                  x
x   MHz=1382.406 bogomips=2764.81          lscpu:CPU=1 Little Endian                                          x
x   ProcessorChips=1 PhyscalCores=1              Sockets=1 Cores=1 Thrds=1                                    x
x                    VirtualCPUs =1              MHz=1382 max=0 min=0                                         x
x                                                                                                             x
x   Use these keys to toggle statistics on/off:                                                               x
x     c = CPU         l = CPU Long-term     - = Faster screen updates                                         x
x     C = " WideView  U = Utilisation       + = Slower screen updates                                         x
x     m = Memory      V = Virtual memory    j = File Systems                                                  x
x     d = Disks       n = Network           . = only busy disks/procs                                         x
x     r = Resource    N = NFS               h = more options                                                  x
x     k = Kernel      t = Top-processes     q = Quit                                                          x
xqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqx
x                                                                                                             x
x                                                                                                             x
x                                                                                                             x
x                                                                                                             x
x                                                                                                             x
x                                                                                                             x
mqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqj
Check Cpu usage:
lnmonq16gqqqqqqqqqqqqqqqqqqqqqHostname=localhostqqqqRefresh= 2secs qqq06:47.01qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqk
x CPU Utilisation qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqx
x---------------------------+-------------------------------------------------+                               x
xCPU User%  Sys% Wait%  Idle|0          |25         |50          |75       100|                               x
x  1   0.0   0.0   0.0 100.0|>                                                |                               x
x---------------------------+-------------------------------------------------+                               x
xqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqx
x
Cpu and network togather:
lnmonq16gqqqqqq[H for help]qqqHostname=localhostqqqqRefresh= 2secs qqq06:47.10qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqk
x CPU Utilisation qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqx
x---------------------------+-------------------------------------------------+                               x
xCPU User%  Sys% Wait%  Idle|0          |25         |50          |75       100|                               x
x  1   0.0   1.0   0.0  99.0|>                                                |                               x
x---------------------------+-------------------------------------------------+                               x
x Network I/O qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqx
xI/F Name Recv=KB/s Trans=KB/s packin packout insize outsize Peak->Recv Trans                                 x
x  enp0s3      0.2       4.4       2.0    3.5    98.5 1284.6        1.0     13.9                              x
x      lo      0.0       0.0       0.0    0.0     0.0    0.0        0.0      0.0                              x
x Network Error Counters qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqx
xI/F Name iErrors iDrop iOverrun iFrame oErrors   oDrop oOverrun oCarrier oColls                              x
x  enp0s3       0       0       0       0       0       0       0       0       0                             x
x      lo       0       0       0       0       0       0       0       0       0                             x
xqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqx
x


Now let us schedule nmon in cron to collect data every 10 secs for 360 times, which means 1hr:


59 * * * * cd /pgBACKUP/nmondata && nmon -f -tT -s10 -c360 1>/dev/null 2>&1


Resultant files are:
-rw-r--r--. 1 root     root     112123 Dec 29 18:13 localhost_221229_1759.nmon
-rw-r--r--. 1 root     root     335301 Dec 29 18:13 localhost_221229_1659.nmon
[root@localhost nmondata]# head localhost_221229_1759.nmon
AAA,progname,nmon
AAA,command,nmon -f -tT -s10 -c360
AAA,version,16g
AAA,disks_per_line,150
AAA,max_disks,256,set by -d option
AAA,disks,14,
AAA,host,localhost
AAA,user,root
AAA,OS,Linux,3.10.0-1160.6.1.el7.x86_64,#1 SMP Tue Nov 17 13:59:11 UTC 2020,x86_64
AAA,runname,localhost
[root@localhost nmondata]#


Also notice the next nmon sampler is running from cron... there is one nmon at any time; as per our setup. After 1hr the existing nmon will die and the next nmon will kick off.


[root@localhost nmondata]# ps -ef|grep -i nmon
root      6276     1  0 17:59 ?        00:00:01 nmon -f -tT -s10 -c360
root      7710  1770  0 18:22 pts/0    00:00:00 grep --color=auto -i nmon
[root@localhost nmondata]#


Now let us look how we can host this as a webpage...


Directory structure:
Nmon data loc: /pgBACKUP/nmondata
nmonchart bin: /pgBACKUP/nmonchartbin/nmonchart
nmonchart dest: /pgBACKUP/nmondata/nmonchartop

So we need to use nmonchart to push the output charts over to the dest location...
Prereqs: You need ksh


[root@localhost nmondata]# ksh
# pwd
/pgBACKUP/nmondata
#

/pgBACKUP/nmonchartbin/nmonchart localhost_221229_1659.nmon ./nmonchartop/localhost_221229_1659.html
Actual:
# /pgBACKUP/nmonchartbin/nmonchart localhost_221229_1659.nmon ./nmonchartop/localhost_221229_1659.html
#
[root@localhost nmondata]# cd nmonchartop/
[root@localhost nmonchartop]# ls -altr
total 448
drwxr-xr-x. 3 root root    193 Dec 29 18:13 ..
drwxr-xr-x. 2 root root     40 Dec 29 18:26 .
-rw-r--r--. 1 root root 457432 Dec 29 18:26 localhost_221229_1659.html
[root@localhost nmonchartop]# pwd
/pgBACKUP/nmondata/nmonchartop
[root@localhost nmonchartop]#

So the html output is available here. We just need to host it.
Let us try to use simple solution like python.
>>>next blog>>>

OKV platform certificate rotation - pitfall , awareness!!!!

OKV version: 21.9 Setup: Multimaster R/W cluster Plan: https://docs.google.com/spreadsheets/d/e/2PACX-1vSaXXTjj9cE1fvYpNmsDNBOkTIw78yTwQ6a9o...