Help in CPU usage diagnosis, please

classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|

Help in CPU usage diagnosis, please

John Hopkin
I've noticed the past couple of builds I've used have sometimes make
heavy use of the CPU even when idle.  For example, today, after
leaving the system idle overnight, the two virtual CPUs (a single-core
Pentium 4) were running constantly at 96-99% each, whereas normally
I'd expect 35-40% and 12-15% during idle running, most of that
non-Opensim overhead.

When this happens, sometimes - but not always - a reload of the server
fixes it.  I'm only running a single OpenSim.exe, and using OSGrid's
UGAIM.

This is noticeable in-world as lag, and I'd like to find out what's
causing it, but I don't know how to narrow it down, other than by
eliminating various regions and also disabling services such as
scripting and prim physics.  With five regions, and more on the way,
it's pretty difficult to eliminate anything that way, especially with
something that's intermittent like this.

It strikes me that I know very little of what I, as a user, can do to
try to diagnose something like this.  I'd like to have more details
before I file a Mantis, since a description alone can't really be much
use to the devs.  The steps I could take with what knowledge I have
are:

1.  Examine OpenSim.log and look for anything suspicious-looking,
especially if it's happening repeatedly.  Do the same with the
on-screen messages (difficult, since the virtual console only scrolls
back so far).

2.  Look at other vital signs, such as network use and free memory -
in this case, these are normal.  In fact, at 33-38% user memory used
of 2GB for four regions, that's better lately than it's been for a
long time.

3.  Selectively disable various regions and services, as mentioned
above, and monitor the situation over time.  This is difficult, as it
naturally affects our use of the sim in the meantime.

4.  Make sure that it is the OpenSim.exe process that's causing it by
using "ps" (in *nix) and similar tools to examine the process.  (It
is).

What else can I do, in general terms?  What debugging tools and
techniques can a user with little direct experience of OS's internal
machinations use to help pinpoint a problem?  Is there any way of
finding out the resource usage of, say, individual modules?  Of
getting more detailed logs?  Anything?

Please note that the actual problem I'm having at the moment is only
an example - I'm interested in general in being able to be more
proactive in diagnosing Opensim problems, knowing as a developer
myself (though not in this project) how valuable more detail from
users can be in tacking a problem.
--
John Hopkin

_______________________________________________
Opensim-users mailing list
[hidden email]
https://lists.berlios.de/mailman/listinfo/opensim-users
Reply | Threaded
Open this post in threaded view
|

Re: Help in CPU usage diagnosis, please

Stefan Andersson
There are some diagnostics commands like
 
show statistics
and
show queues
 
that could sometimes be instructional.

Also, you can monitor the regions with the Statistics module that resides on /SStats on the region http server (if my memory serve me right)

Best regards,
Stefan Andersson
Tribal Media AB



 

> To: [hidden email]
> From: [hidden email]
> Date: Thu, 12 Feb 2009 19:07:21 +0000
> Subject: [Opensim-users] Help in CPU usage diagnosis, please
>
> I've noticed the past couple of builds I've used have sometimes make
> heavy use of the CPU even when idle. For example, today, after
> leaving the system idle overnight, the two virtual CPUs (a single-core
> Pentium 4) were running constantly at 96-99% each, whereas normally
> I'd expect 35-40% and 12-15% during idle running, most of that
> non-Opensim overhead.
>
> When this happens, sometimes - but not always - a reload of the server
> fixes it. I'm only running a single OpenSim.exe, and using OSGrid's
> UGAIM.
>
> This is noticeable in-world as lag, and I'd like to find out what's
> causing it, but I don't know how to narrow it down, other than by
> eliminating various regions and also disabling services such as
> scripting and prim physics. With five regions, and more on the way,
> it's pretty difficult to eliminate anything that way, especially with
> something that's intermittent like this.
>
> It strikes me that I know very little of what I, as a user, can do to
> try to diagnose something like this. I'd like to have more details
> before I file a Mantis, since a description alone can't really be much
> use to the devs. The steps I could take with what knowledge I have
> are:
>
> 1. Examine OpenSim.log and look for anything suspicious-looking,
> especially if it's happening repeatedly. Do the same with the
> on-screen messages (difficult, since the virtual console only scrolls
> back so far).
>
> 2. Look at other vital signs, such as network use and free memory -
> in this case, these are normal. In fact, at 33-38% user memory used
> of 2GB for four regions, that's better lately than it's been for a
> long time.
>
> 3. Selectively disable various regions and services, as mentioned
> above, and monitor the situation over time. This is difficult, as it
> naturally affects our use of the sim in the meantime.
>
> 4. Make sure that it is the OpenSim.exe process that's causing it by
> using "ps" (in *nix) and similar tools to examine the process. (It
> is).
>
> What else can I do, in general terms? What debugging tools and
> techniques can a user with little direct experience of OS's internal
> machinations use to help pinpoint a problem? Is there any way of
> finding out the resource usage of, say, individual modules? Of
> getting more detailed logs? Anything?
>
> Please note that the actual problem I'm having at the moment is only
> an example - I'm interested in general in being able to be more
> proactive in diagnosing Opensim problems, knowing as a developer
> myself (though not in this project) how valuable more detail from
> users can be in tacking a problem.
> --
> John Hopkin
>
> _______________________________________________
> Opensim-users mailing list
> [hidden email]
> https://lists.berlios.de/mailman/listinfo/opensim-users

_______________________________________________
Opensim-users mailing list
[hidden email]
https://lists.berlios.de/mailman/listinfo/opensim-users
Reply | Threaded
Open this post in threaded view
|

Re: Help in CPU usage diagnosis, please

Jürgen Hamel
In reply to this post by John Hopkin
On Thu, 12 Feb 2009 19:07:21 +0000
John Hopkin <[hidden email]> wrote:

> the two virtual CPUs (a single-core
> Pentium 4) were running constantly at 96-99% each, whereas normally
> I'd expect 35-40% and 12-15% during idle running, most of that
> non-Opensim overhead.
Hi, also important is the OS, I have the same problem at LINUX with
mono 1.9, update to 2.x are better and reduce it. So, not all is to
examine, something are simple bad things.

Jürgen Hamel

--
Cyrus-Computer GmbH Linux Server Support Jürgen Hamel
Cuon - Warenwirtschaft mit Linux  http://www.cuon.org

_______________________________________________
Opensim-users mailing list
[hidden email]
https://lists.berlios.de/mailman/listinfo/opensim-users

signature.asc (196 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Help in CPU usage diagnosis, please

Robert Klein
Jurgen,

Do you have instructions on how you upgraded your Mono 1.9 to 2.x? I would like to compare them with some I have before I try to upgrade.

Thanks,

-Robert

Jürgen Hamel wrote
On Thu, 12 Feb 2009 19:07:21 +0000
John Hopkin <opensim@jfhopkin.karoo.co.uk> wrote:

> the two virtual CPUs (a single-core
> Pentium 4) were running constantly at 96-99% each, whereas normally
> I'd expect 35-40% and 12-15% during idle running, most of that
> non-Opensim overhead.
Hi, also important is the OS, I have the same problem at LINUX with
mono 1.9, update to 2.x are better and reduce it. So, not all is to
examine, something are simple bad things.

Jürgen Hamel

--
Cyrus-Computer GmbH Linux Server Support Jürgen Hamel
Cuon - Warenwirtschaft mit Linux  http://www.cuon.org

 
_______________________________________________
Opensim-users mailing list
Opensim-users@lists.berlios.de
https://lists.berlios.de/mailman/listinfo/opensim-users
Reply | Threaded
Open this post in threaded view
|

Re: Help in CPU usage diagnosis, please

Jürgen Hamel
On Fri, 13 Feb 2009 07:59:46 -0800 (PST)
Robert Klein <[hidden email]> wrote:

> Do you have instructions on how you upgraded your Mono 1.9 to 2.x? I would
> like to compare them with some I have before I try to upgrade.
Hi,
I use debian and ubuntu systems, so I look at this page:

http://www.mono-project.com/Other_Downloads

perhaps it should help you -  or tell me your linux-distri.

Bye, Jürgen


--
Cyrus-Computer GmbH Linux Server Support Jürgen Hamel
Cuon - Warenwirtschaft mit Linux  http://www.cuon.org

_______________________________________________
Opensim-users mailing list
[hidden email]
https://lists.berlios.de/mailman/listinfo/opensim-users

signature.asc (196 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Help in CPU usage diagnosis, please

Robert Klein
Hey Jurgen, yes I am using Ubuntu 8.10 desktop. I see there is a section on Ubuntu in the link.

Thanks,

-Robert

Jürgen Hamel wrote
On Fri, 13 Feb 2009 07:59:46 -0800 (PST)
Robert Klein <rtkwebman@gmail.com> wrote:

> Do you have instructions on how you upgraded your Mono 1.9 to 2.x? I would
> like to compare them with some I have before I try to upgrade.
Hi,
I use debian and ubuntu systems, so I look at this page:

http://www.mono-project.com/Other_Downloads

perhaps it should help you -  or tell me your linux-distri.

Bye, Jürgen


--
Cyrus-Computer GmbH Linux Server Support Jürgen Hamel
Cuon - Warenwirtschaft mit Linux  http://www.cuon.org

 
_______________________________________________
Opensim-users mailing list
Opensim-users@lists.berlios.de
https://lists.berlios.de/mailman/listinfo/opensim-users
Reply | Threaded
Open this post in threaded view
|

Re: Help in CPU usage diagnosis, please

John Hopkin
In reply to this post by Jürgen Hamel
Jürgen Hamel wrote:

>On Thu, 12 Feb 2009 19:07:21 +0000
>John Hopkin <[hidden email]> wrote:
>
>> the two virtual CPUs (a single-core
>> Pentium 4) were running constantly at 96-99% each, whereas normally
>> I'd expect 35-40% and 12-15% during idle running, most of that
>> non-Opensim overhead.
>Hi, also important is the OS, I have the same problem at LINUX with
>mono 1.9, update to 2.x are better and reduce it. So, not all is to
>examine, something are simple bad things.

Thanks - I upgraded Mono a while ago, after a suggestion in a previous
thread here, and that cured some pretty nasty things.  Before I did
that, we were getting regular crashes, pretty much making OS unusable.
--
John Hopkin

_______________________________________________
Opensim-users mailing list
[hidden email]
https://lists.berlios.de/mailman/listinfo/opensim-users