A grid statistics page

classic Classic list List threaded Threaded
9 messages Options
Reply | Threaded
Open this post in threaded view
|

A grid statistics page

Jeff Kelley
Let's build a nice grid overview like this one :

http://www.pescadoo.net/tmp/gridstats.png

using the monitoring module :

http://opensimulator.org/wiki/Monitoring_Module

You should:

- Have a web server with PHP and mysqli
- Have SELECT access to the ROBUST database
- No fear uncommented code
- Supply a short file db_access.php with :

    function sqlUser() { return 'your_sql_user'; }
    function sqlPass() { return 'your_sql_pass'; }
    function sqlHost() { return 'your_sql_host'; }  // Probably 'localhost'
    function sqlBase() { return 'your_robust_database'; }

Works only in grid mode. Needs filtering local regions for stand-alone.
For large grids, you will have to paginate.

Let me know if it works for you. Improvements welcome.

PHP code follows. TAB is four spaces.


<!DOCTYPE html>
<html>
<head>
        <title>Grid Statistics</title>
        <meta http-equiv="refresh" content="5">
        <style type="text/css">
                body {  }
                table { margin: auto; xborder-collapse: collapse;
                                          border: 2px solid #000; }
                tr.divide { border: 1px solid #000; }
                table td { border: 1px solid #ddd;
                                          padding: 2px 4px 2px 4px; }
                td.name { text-align: left; }
                td { text-align: right;
                                          background: #eeeeee; }
                tr.divide td { height: 1px; padding: 0px;
                                           border: 1px solid #000; }
                td.RED { background: Salmon; }
                td.GREEN { background: LightGreen; }
                td.MAGENTA { background: Thistle; }

        </style>
</head>


<body>





<?php
        error_reporting(E_ALL ^ E_NOTICE);
        include "db_access.php";


        function GetRegionsFromDatabase() {

                // ATTENTION! Ne marche pas pour une stand-alone
(regions hypergrid dans la table)
                $query = "SELECT
regionName,serverIP,serverHttpPort,serverPort,uuid
                FROM regions ORDER BY serverHttpPort,serverPort,regionName;";

                $link = new mysqli(sqlHost(),sqlUser(),sqlPass(),sqlBase());
                if ($link->connect_errno)
                        die('Connect Error: ' . $link->connect_errno
. ' (' . $link->connect_error . ')');

                $answ = $link->query($query);
                if ($link->errno)
                        die('Select Error: ' . $link->errno . '(' .
$link->error . ')');

                $result = $answ->fetch_all(MYSQLI_ASSOC);

                $link->close();
                return $result;
        }


        function GetStatisticsFromSimulator($regions) {
                $result = array();
                foreach ($regions as $reg) {
                        $reg_name = $reg['regionName'];
                        $srv_host = $reg['serverIP'];
                        $srv_port = $reg['serverHttpPort'];
                        $reg_port = $reg['serverPort'];
                        $reg_uuid = $reg['uuid'];

                        $url =
"<a href="http://$srv_host:$srv_port/monitorstats/$reg_uuid/">http://$srv_host:$srv_port/monitorstats/$reg_uuid/";
                        $xml = file_get_contents ($url);
                        $dat = (array)simplexml_load_string ($xml);

                        $region_data = array();
                        $region_data['Name'] = $reg_name;
                        $region_data['ServerPort'] = $srv_port;
                        $region_data['RegionPort'] = $reg_port;

                        foreach ($dat as $key => $val) {
                                $key = shortenKey($key);
                                $region_data[$key] = $val;
                        }
                        array_push ($result, $region_data);
                }
                return $result;
        }


        function shortenKey($key) {
                if (preg_match ("/(.*)Monitor/", $key, $matches))
                        $key =  $matches[1];
                return $key;
        }

        function formatNum($key,$val) {
                if (preg_match("/FPS/" ,$key)) $val =
sprintf ("%3.2f",$val);
                if (preg_match("/Second/" ,$key)) $val =
sprintf ("%3.2f",$val);
                if (preg_match("/FrameTime/",$key)) $val = sprintf
("%3.2f",$val);
                if (preg_match("/Dilation/" ,$key)) $val =
sprintf ("%3.2f",$val);
                if (preg_match("/Memory/" ,$key)) $val =
sprintf ("%d",$val/1e+6);
                return $val;
        }

        function addUnits($key,$val) {
                if (preg_match("/FrameTime/",$key)) $val = $val." ms";
                if (preg_match("/Memory/" ,$key)) $val = $val." Mb";
                return $val;
        }

        function colorize($key,$val) {
                if (preg_match("/FrameTime/" ,$key)) return ($val >
20)?"RED" :"GREEN";
                if (preg_match("/Dilation/" ,$key)) return ($val
<  1)?"RED" :"GREEN";
                if (preg_match("/FPS/" ,$key)) return ($val
< 50)?"RED" :"GREEN";
                if (preg_match("/Pending/" ,$key)) return
($val >  0)?"RED" :"NOCOLOR";
                if (preg_match("/AgentCount/",$key)) return ($val >
0)?"MAGENTA":"NOCOLOR";
        }

        function divider($stats) {
                $ncols = 1 + count ($stats);
                printf ("<tr class=divide><td colspan=$ncols></tr>\n");
        }

        function PrintParam($stats,$key) {
                printf ("<tr>");
                printf ("<td class=name>%s</td>", $key);
                foreach ($stats as $column) {
                        $val = $column[$key];
                        $val = formatNum ($key,$val);
                        $val = addUnits  ($key,$val);
                        $col = colorize  ($key,$val);
                        printf ("<td class=$col>%s</td>", $val);
                }
                printf ("</tr>\n");
        }


        function PrintStatisticsFull($stats) {
                print "<table>\n";
                PrintParam ($stats, 'Name');
                PrintParam ($stats, 'ServerPort');
                PrintParam ($stats, 'RegionPort');
                divider    ($stats);
                PrintParam ($stats, 'TotalFrameTime');
                PrintParam ($stats, 'SpareFrameTime');
                PrintParam ($stats, 'SimulationFrameTime');
                PrintParam ($stats, 'PhysicsFrameTime');
                PrintParam ($stats, 'NetFrameTime');
                PrintParam ($stats, 'ImagesFrameTime');
                PrintParam ($stats, 'AgentFrameTime');
                divider    ($stats);
                PrintParam ($stats, 'SimFPS');
                PrintParam ($stats, 'PhysicsFPS');
                PrintParam ($stats, 'TimeDilation');
                divider    ($stats);
                PrintParam ($stats, 'AgentCount');
                PrintParam ($stats, 'ChildAgentCount');
                PrintParam ($stats, 'ObjectCount');
                PrintParam ($stats, 'ActiveScripts');
                PrintParam ($stats, 'ActiveObjectCount');
                PrintParam ($stats, 'AgentUpdatesPerSecond');
                PrintParam ($stats, 'ScriptEventsPerSecond');
                divider    ($stats);
                PrintParam ($stats, 'InPacketsPerSecond');
                PrintParam ($stats, 'OutPacketsPerSecond');
                PrintParam ($stats, 'PendingDownloads');
                PrintParam ($stats, 'PendingUploads');
                PrintParam ($stats, 'UnackedBytes');
                divider    ($stats);
                PrintParam ($stats, 'EventFrame');
                PrintParam ($stats, 'LastReportedObjectUpdates');
                PrintParam ($stats, 'SlowFrames');
                PrintParam ($stats, 'PhysicsFrame');
        # PrintParam ($stats, 'LastFrameTime');
                PrintParam ($stats, 'LandFrame');
                PrintParam ($stats, 'PhysicsUpdateFrame');
                PrintParam ($stats, 'TotalFrame');
                divider    ($stats);
                PrintParam ($stats, 'ThreadCount');
                PrintParam ($stats, 'GCMemory');
                PrintParam ($stats, 'PWSMemory');
                print "</table>\n";
        }

        function PrintStatisticsAvatars($stats) {
                print "<br><table>\n";
                print "<caption>Avatars</caption>\n";
                foreach ($stats as $row) {
                        $rname = $row['Name'];
                        $navat = $row['AgentCount'];
                        if ($navat > 0) {
                                printf ("<tr>");
                                printf ("<td class='MAGENTA'>$navat</td>");
                                printf ("<td class='MAGENTA
name'>$rname</td>");
                                printf ("<td class='MAGENTA
name'>91.121.159.37:8002:$rname</td>");
                                printf ("</tr>\n");
                        }
                }
                print "</table></br>\n";
        }

        // Main program

        $time_start = microtime(true);
        $regns = GetRegionsFromDatabase();
        $time_end = microtime(true);
        $time_database = 1000 * ($time_end - $time_start);

        $time_start = microtime(true);
        $stats =  GetStatisticsFromSimulator($regns);
        $time_end = microtime(true);
        $time_network = 1000 * ($time_end - $time_start);

        $time_start = microtime(true);
        PrintStatisticsFull($stats);
        $time_end = microtime(true);
        $time_edition = 1000 * ($time_end - $time_start);

        PrintStatisticsAvatars($stats);



  printf ("<br>\n");
  printf ("Database Time: %5.2f ms<br>\n", $time_database);
  printf ("Network Time: %5.2f ms<br>\n", $time_network);
  printf ("Edition Time: %5.2f ms<br>\n", $time_edition);


?>




_______________________________________________
Opensim-users mailing list
[hidden email]
http://opensimulator.org/cgi-bin/mailman/listinfo/opensim-users
Reply | Threaded
Open this post in threaded view
|

Re: A grid statistics page

Shaun T. Erickson
That's very nice. I'll have to set that up for my grid. I will
definitely have to figure out how to paginate it though. Thanks!

-ste

On 10/31/14, 11:43 PM, Jeff Kelley wrote:
> Let's build a nice grid overview like this one :
>
> http://www.pescadoo.net/tmp/gridstats.png
_______________________________________________
Opensim-users mailing list
[hidden email]
http://opensimulator.org/cgi-bin/mailman/listinfo/opensim-users
Reply | Threaded
Open this post in threaded view
|

Re: A grid statistics page

Jeff Kelley
At 12:55 AM -0400 1/11/14, Shaun T. Erickson wrote:

>That's very nice. I'll have to set that up for my grid.
>I will definitely have to figure out how to paginate it though. Thanks!

Add arguments ?startcolum= &numcolumns=

Narrow the foreach loop in PrintParam to :
startcolum to (startcolum + numcolumns - 1).

Echo a link backward to (startcolum - numcolumns)
and a link forward to (startcolum + numcolumns), checking bounds.


I think it does it.


-- Jeff




_______________________________________________
Opensim-users mailing list
[hidden email]
http://opensimulator.org/cgi-bin/mailman/listinfo/opensim-users
Reply | Threaded
Open this post in threaded view
|

Re: A grid statistics page

Shaun T. Erickson
In reply to this post by Jeff Kelley
Jeff,

I'm getting an error when I run it:

PHP Fatal error:  Call to undefined method mysqli_result::fetch_all() in
/var/www/html/stats/stats.php on line 43

That would be this line:

$result = $answ->fetch_all(MYSQLI_ASSOC);

 From perusing the web, it would seem my MySQL installation lacks the
mysqlnd driver, that this function relies upon.

-ste


On 10/31/14, 11:43 PM, Jeff Kelley wrote:
> Let's build a nice grid overview like this one :
>
> http://www.pescadoo.net/tmp/gridstats.png
[SNIP]
_______________________________________________
Opensim-users mailing list
[hidden email]
http://opensimulator.org/cgi-bin/mailman/listinfo/opensim-users
Reply | Threaded
Open this post in threaded view
|

Re: A grid statistics page

Shaun T. Erickson
It turns out that there was a php-mysqlnd rpm I could install in place
of the php-mysql rpm I had installed. Once I did that, the code worked.

Now, on to pagination. :)

-ste

On 11/1/14, 11:38 PM, Shaun T. Erickson wrote:

> Jeff,
>
> I'm getting an error when I run it:
>
> PHP Fatal error:  Call to undefined method mysqli_result::fetch_all()
> in /var/www/html/stats/stats.php on line 43
>
> That would be this line:
>
> $result = $answ->fetch_all(MYSQLI_ASSOC);
>
> From perusing the web, it would seem my MySQL installation lacks the
> mysqlnd driver, that this function relies upon.
>
> -ste
>
>
> On 10/31/14, 11:43 PM, Jeff Kelley wrote:
>> Let's build a nice grid overview like this one :
>>
>> http://www.pescadoo.net/tmp/gridstats.png
> [SNIP]
> _______________________________________________
> Opensim-users mailing list
> [hidden email]
> http://opensimulator.org/cgi-bin/mailman/listinfo/opensim-users

_______________________________________________
Opensim-users mailing list
[hidden email]
http://opensimulator.org/cgi-bin/mailman/listinfo/opensim-users
Reply | Threaded
Open this post in threaded view
|

Re: A grid statistics page

Shaun T. Erickson
Not so fast ...

Before I get to pagination, I see there's something else that needs
fixing. About 50% of the values are missing and the other 50% are all
zero values.

It seems I'm getting a 'permission denied' when this is run:

$url = "<a href="http://$srv_host:$srv_port/monitorstats/$reg_uuid/">http://$srv_host:$srv_port/monitorstats/$reg_uuid/";
$xml = file_get_contents ($url);

Now to figure out why ...

-ste

On 11/1/14, 11:48 PM, Shaun T. Erickson wrote:

> It turns out that there was a php-mysqlnd rpm I could install in place
> of the php-mysql rpm I had installed. Once I did that, the code worked.
>
> Now, on to pagination. :)
>
> -ste
>
> On 11/1/14, 11:38 PM, Shaun T. Erickson wrote:
>> Jeff,
>>
>> I'm getting an error when I run it:
>>
>> PHP Fatal error:  Call to undefined method mysqli_result::fetch_all()
>> in /var/www/html/stats/stats.php on line 43
>>
>> That would be this line:
>>
>> $result = $answ->fetch_all(MYSQLI_ASSOC);
>>
>> From perusing the web, it would seem my MySQL installation lacks the
>> mysqlnd driver, that this function relies upon.
>>
>> -ste
>>
>>
>> On 10/31/14, 11:43 PM, Jeff Kelley wrote:
>>> Let's build a nice grid overview like this one :
>>>
>>> http://www.pescadoo.net/tmp/gridstats.png
>> [SNIP]
>> _______________________________________________
>> Opensim-users mailing list
>> [hidden email]
>> http://opensimulator.org/cgi-bin/mailman/listinfo/opensim-users
>
> _______________________________________________
> Opensim-users mailing list
> [hidden email]
> http://opensimulator.org/cgi-bin/mailman/listinfo/opensim-users

_______________________________________________
Opensim-users mailing list
[hidden email]
http://opensimulator.org/cgi-bin/mailman/listinfo/opensim-users
Reply | Threaded
Open this post in threaded view
|

Re: A grid statistics page

Shaun T. Erickson
I'm not sure why I'm getting a "permission denied".

I put in a print statement to show me the value of $url, then tried to
go to some of them in my browser. I get there fine and see an XML block
with the various statistics in it. Why it doesn't like file_get_contents
doing that, I don't know.

-ste

On 11/1/14, 11:56 PM, Shaun T. Erickson wrote:

> Not so fast ...
>
> Before I get to pagination, I see there's something else that needs
> fixing. About 50% of the values are missing and the other 50% are all
> zero values.
>
> It seems I'm getting a 'permission denied' when this is run:
>
> $url = "<a href="http://$srv_host:$srv_port/monitorstats/$reg_uuid/">http://$srv_host:$srv_port/monitorstats/$reg_uuid/";
> $xml = file_get_contents ($url);
>
> Now to figure out why ...
>
> -ste
>
> On 11/1/14, 11:48 PM, Shaun T. Erickson wrote:
>> It turns out that there was a php-mysqlnd rpm I could install in
>> place of the php-mysql rpm I had installed. Once I did that, the code
>> worked.
>>
>> Now, on to pagination. :)
>>
>> -ste
>>
>> On 11/1/14, 11:38 PM, Shaun T. Erickson wrote:
>>> Jeff,
>>>
>>> I'm getting an error when I run it:
>>>
>>> PHP Fatal error:  Call to undefined method
>>> mysqli_result::fetch_all() in /var/www/html/stats/stats.php on line 43
>>>
>>> That would be this line:
>>>
>>> $result = $answ->fetch_all(MYSQLI_ASSOC);
>>>
>>> From perusing the web, it would seem my MySQL installation lacks the
>>> mysqlnd driver, that this function relies upon.
>>>
>>> -ste
>>>
>>>
>>> On 10/31/14, 11:43 PM, Jeff Kelley wrote:
>>>> Let's build a nice grid overview like this one :
>>>>
>>>> http://www.pescadoo.net/tmp/gridstats.png
>>> [SNIP]
>>> _______________________________________________
>>> Opensim-users mailing list
>>> [hidden email]
>>> http://opensimulator.org/cgi-bin/mailman/listinfo/opensim-users
>>
>> _______________________________________________
>> Opensim-users mailing list
>> [hidden email]
>> http://opensimulator.org/cgi-bin/mailman/listinfo/opensim-users
>
> _______________________________________________
> Opensim-users mailing list
> [hidden email]
> http://opensimulator.org/cgi-bin/mailman/listinfo/opensim-users

_______________________________________________
Opensim-users mailing list
[hidden email]
http://opensimulator.org/cgi-bin/mailman/listinfo/opensim-users
Reply | Threaded
Open this post in threaded view
|

Re: A grid statistics page

Shaun T. Erickson
Okay, now I'm getting values and all table coordinates filled in. I had
to tell selinux to get out of the way, with this command:

setsebool -P httpd_can_network_connect on

-ste

On 11/2/14, 12:11 AM, Shaun T. Erickson wrote:

> I'm not sure why I'm getting a "permission denied".
>
> I put in a print statement to show me the value of $url, then tried to
> go to some of them in my browser. I get there fine and see an XML
> block with the various statistics in it. Why it doesn't like
> file_get_contents doing that, I don't know.
>
> -ste
>
> On 11/1/14, 11:56 PM, Shaun T. Erickson wrote:
>> Not so fast ...
>>
>> Before I get to pagination, I see there's something else that needs
>> fixing. About 50% of the values are missing and the other 50% are all
>> zero values.
>>
>> It seems I'm getting a 'permission denied' when this is run:
>>
>> $url = "<a href="http://$srv_host:$srv_port/monitorstats/$reg_uuid/">http://$srv_host:$srv_port/monitorstats/$reg_uuid/";
>> $xml = file_get_contents ($url);
>>
>> Now to figure out why ...
>>
>> -ste
>>
>> On 11/1/14, 11:48 PM, Shaun T. Erickson wrote:
>>> It turns out that there was a php-mysqlnd rpm I could install in
>>> place of the php-mysql rpm I had installed. Once I did that, the
>>> code worked.
>>>
>>> Now, on to pagination. :)
>>>
>>> -ste
>>>
>>> On 11/1/14, 11:38 PM, Shaun T. Erickson wrote:
>>>> Jeff,
>>>>
>>>> I'm getting an error when I run it:
>>>>
>>>> PHP Fatal error:  Call to undefined method
>>>> mysqli_result::fetch_all() in /var/www/html/stats/stats.php on line 43
>>>>
>>>> That would be this line:
>>>>
>>>> $result = $answ->fetch_all(MYSQLI_ASSOC);
>>>>
>>>> From perusing the web, it would seem my MySQL installation lacks
>>>> the mysqlnd driver, that this function relies upon.
>>>>
>>>> -ste
>>>>
>>>>
>>>> On 10/31/14, 11:43 PM, Jeff Kelley wrote:
>>>>> Let's build a nice grid overview like this one :
>>>>>
>>>>> http://www.pescadoo.net/tmp/gridstats.png
>>>> [SNIP]

_______________________________________________
Opensim-users mailing list
[hidden email]
http://opensimulator.org/cgi-bin/mailman/listinfo/opensim-users
Reply | Threaded
Open this post in threaded view
|

Re: A grid statistics page

Jeff Kelley
At 12:37 AM -0400 2/11/14, Shaun T. Erickson wrote:

>Okay, now I'm getting values and all table coordinates filled in. I
>had to tell selinux to get out of the way, with this command:


For opensim servers, I usually disable the whole thing.

I have been lazy on http programming. file_get_contents may not be
the best choice.

And yes, you need the "native" mySQL driver. yum or apt-get install
php5-mysqlnd.

Anyway, since justincc pointed me to the GridService API, I found a
better way to get the regions : GetRegionRange ($serviceUri, 0, 0,
2147483647, 2147483647); You just need access to port 8003.

To be smart, we should cache the region data for a time. They
certainly don't change every 5 seconds, and GetRegionRange may tax
the grid service for a large grid.



-- Jeff
_______________________________________________
Opensim-users mailing list
[hidden email]
http://opensimulator.org/cgi-bin/mailman/listinfo/opensim-users