bin/OpenSim.db-journal file begin written and deleted many times

classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|

bin/OpenSim.db-journal file begin written and deleted many times

aiaustin
As I shutdown a region simulator, I see a lot of flashing of an open
window showing the directory listing for the bin directory.

This seems to indicate that a 5K file called "OpenSim.db-journal" is
being created and immediately deleted a LOT of times as the
OpenSim.db file (for SQLIte data base) is changed. At least that's
what I assume is happening.  A translate scene command on a region
with only 600 objects seemed to take nearly half an hour to complete
on an otherwise unloaded region yesterday for example, and this
flashing wad happening throughout until the system finished its "shutdown".

This seems to be enabled  by the default level of debugging and I
wonder if this could this be a source of a slow down in using OpenSim?

Can anyone shed light on what is happening here and if the default
logging might itself be a cause of why such operations take such a log time?

_______________________________________________
Opensim-users mailing list
[hidden email]
http://opensimulator.org/cgi-bin/mailman/listinfo/opensim-users
Reply | Threaded
Open this post in threaded view
|

Re: bin/OpenSim.db-journal file begin written and deleted many times

Gwyneth Llewelyn
What version are you using, Ai? I have seen a fluctuation of viewer-side FPS on an apparently 'empty' region, usually a signal that there is some 'invisible' back-end processing at the simulator level, but haven't been able to track it down to anything yet. Selecting a lot of prims and linking them together now takes quite a lot of time (sometimes even a minute or so!). This is on 0.8.0 Dev 722f030. I suspected some delays while writing to the database, and having some sort of database journal being written to a lot of times would certainly account for that!

Also, I'm seeing some more unusual Flotsam cache errors (failing to open/delete files and directories). I deleted the disk-based asset cache, pulled the latest git (a083a48), found a completely unrelated error, reported it, but my point here is that it seems to work fine in terms of asset cache now :) I guess I had a waaaay old asset cache with lots of missing/invalid references after so many upgrades...


On 28 April 2014 12:43, Ai Austin <[hidden email]> wrote:
As I shutdown a region simulator, I see a lot of flashing of an open window showing the directory listing for the bin directory.

This seems to indicate that a 5K file called "OpenSim.db-journal" is being created and immediately deleted a LOT of times as the OpenSim.db file (for SQLIte data base) is changed. At least that's what I assume is happening.  A translate scene command on a region with only 600 objects seemed to take nearly half an hour to complete on an otherwise unloaded region yesterday for example, and this flashing wad happening throughout until the system finished its "shutdown".

This seems to be enabled  by the default level of debugging and I wonder if this could this be a source of a slow down in using OpenSim?

Can anyone shed light on what is happening here and if the default logging might itself be a cause of why such operations take such a log time?

_______________________________________________
Opensim-users mailing list
[hidden email]
http://opensimulator.org/cgi-bin/mailman/listinfo/opensim-users



--
"I'm not building a game. I'm building a new country."
  -- Philip "Linden" Rosedale, interview to Wired, 2004-05-08

_______________________________________________
Opensim-users mailing list
[hidden email]
http://opensimulator.org/cgi-bin/mailman/listinfo/opensim-users
--
"I'm not building a game. I'm building a new country."
  -- Philip "Linden" Rosedale, interview to Wired, 2004-05-08
Reply | Threaded
Open this post in threaded view
|

Re: bin/OpenSim.db-journal file begin written and deleted many times

Dahlia Trimble
In reply to this post by aiaustin
Translate scene should complete quite quickly, all it does is move the objects in the scene. What you are probably seeing is the changes being saved to the database, an entirely different process. If you are using Sqlite for your region database than storing changes to the scene will always be quite slow. It would likely take just as long for loading an oar with the same number of objects and saving them to the database. If you need faster region storage, try using one of the other database options such as mysql.


On Mon, Apr 28, 2014 at 4:43 AM, Ai Austin <[hidden email]> wrote:
As I shutdown a region simulator, I see a lot of flashing of an open window showing the directory listing for the bin directory.

This seems to indicate that a 5K file called "OpenSim.db-journal" is being created and immediately deleted a LOT of times as the OpenSim.db file (for SQLIte data base) is changed. At least that's what I assume is happening.  A translate scene command on a region with only 600 objects seemed to take nearly half an hour to complete on an otherwise unloaded region yesterday for example, and this flashing wad happening throughout until the system finished its "shutdown".

This seems to be enabled  by the default level of debugging and I wonder if this could this be a source of a slow down in using OpenSim?

Can anyone shed light on what is happening here and if the default logging might itself be a cause of why such operations take such a log time?

_______________________________________________
Opensim-users mailing list
[hidden email]
http://opensimulator.org/cgi-bin/mailman/listinfo/opensim-users


_______________________________________________
Opensim-users mailing list
[hidden email]
http://opensimulator.org/cgi-bin/mailman/listinfo/opensim-users
Reply | Threaded
Open this post in threaded view
|

Re: bin/OpenSim.db-journal file begin written and deleted many times

aiaustin
In reply to this post by aiaustin

>Douglas Maxwell....
>Hi Austin,  the reason I recommend you configure multiple processors
>for MIAB is because of all the different processes running in the
>VM.  In addition to OpenSim, there is Apache, MySQL, etc...  The
>extra processors help balance load.
>
>The only time I've seen multiprocessor systems benefit OpenSim was
>with our DGS experiments.  Currently your best bet for performance
>enhancement with regular OpenSimulator is with vertical scaling
>(faster computer).  Hope this helps.


It does indeed Doug, and thanks for clarifying that.  I did wonder if
it was because of te other non-OpenSim elements, so its good to know
that's the reason for your recommendation. Ai (Austin)


_______________________________________________
Opensim-users mailing list
[hidden email]
http://opensimulator.org/cgi-bin/mailman/listinfo/opensim-users
Reply | Threaded
Open this post in threaded view
|

Re: bin/OpenSim.db-journal file begin written and deleted many times

aiaustin
In reply to this post by aiaustin
Gwyneth wrote:
>What version are you using, Ai? I have seen a fluctuation of viewer-side
>FPS on an apparently 'empty' region, usually a signal that there is some
>'invisible' back-end processing at the simulator level, but haven't been
>able to track it down to anything yet. Selecting a lot of prims and linking
>them together now takes quite a lot of time (sometimes even a minute or
>so!). This is on 0.8.0 Dev 722f030. I suspected some delays while writing
>to the database, and having some sort of database journal being written to
>a lot of times would certainly account for that!

I was using the OSGrid latest add-on region release from a few days
ago (r/24670) Gwyneth.

Dahlia wrote:
>Translate scene should complete quite quickly, all it does is move the
>objects in the scene. What you are probably seeing is the changes being
>saved to the database, an entirely different process. If you are using
>Sqlite for your region database than storing changes to the scene will
>always be quite slow. It would likely take just as long for loading an oar
>with the same number of objects and saving them to the database. If you
>need faster region storage, try using one of the other database options
>such as mysql.#


That's right Dahlia.  The translate scene on 600 or so objects was
essentially instant and I was able to move the avatar about (using
ODE temporarily until Robert's r/24672 commit is in the OSGrid
release) in the new negative Z underwater area fine.  It was indeed
as I did the shutdown... and I did assume it was the persisting to
data base part.. that took nearly 30 minutes.  It was SQLite for the
add on regions we have for experiments.. not our MySQL usual data
base.  But still.. constantly creating and deleting the db-journal
file (perhaps once per DB write?) MUST make this persisting process
slower than needed?  I also thought I better make sure the persisting
process completed even though it as taking ages or else there may
have been only a partial translation.  If I had not had the
opensim\bin window open I would have thought the OpenSim.exe console
was jammed and stopped wit with the X on the console window... I
assume that would NOT have been good, so it could be a source fo
problems if persisting objects takes so long.  A better indication of
progress might be to put a dot out every 50 or 100 objects persisted
on the console to remind the user the region OpenSim.exe still need
time to complete.




_______________________________________________
Opensim-users mailing list
[hidden email]
http://opensimulator.org/cgi-bin/mailman/listinfo/opensim-users
Reply | Threaded
Open this post in threaded view
|

Re: bin/OpenSim.db-journal file begin written and deleted many times

Dahlia Trimble
That long shut down has really nothing to do with the translate scene command and everything to do with using Sqlite on a high object-density region where a lot of objects had changed. Sqlite has been extremely slow as long as I've used OpenSim, so I don't believe it's a new issue. I'm not sure of the meaning of the journal file but it's name suggests it may be doing some transaction journaling which can be deleted after a transaction is complete. I'm also not that familiar with the database layers in OpenSim but I'd think that if it was using transactions it would be a good thing as they can protect database integrity in the case of a crash or other failure during an update.

I use Sqlite for some high-prim regions and it's horribly slow. I just use the "backup" command while the region is still running so it can persist while the region is in use. Either that, or I tell it to shutdown and then go out for a walk or something.


On Tue, Apr 29, 2014 at 3:33 AM, Ai Austin <[hidden email]> wrote:
Gwyneth wrote:
What version are you using, Ai? I have seen a fluctuation of viewer-side
FPS on an apparently 'empty' region, usually a signal that there is some
'invisible' back-end processing at the simulator level, but haven't been
able to track it down to anything yet. Selecting a lot of prims and linking
them together now takes quite a lot of time (sometimes even a minute or
so!). This is on 0.8.0 Dev 722f030. I suspected some delays while writing
to the database, and having some sort of database journal being written to
a lot of times would certainly account for that!

I was using the OSGrid latest add-on region release from a few days ago (r/24670) Gwyneth.

Dahlia wrote:
Translate scene should complete quite quickly, all it does is move the
objects in the scene. What you are probably seeing is the changes being
saved to the database, an entirely different process. If you are using
Sqlite for your region database than storing changes to the scene will
always be quite slow. It would likely take just as long for loading an oar
with the same number of objects and saving them to the database. If you
need faster region storage, try using one of the other database options
such as mysql.#


That's right Dahlia.  The translate scene on 600 or so objects was essentially instant and I was able to move the avatar about (using ODE temporarily until Robert's r/24672 commit is in the OSGrid release) in the new negative Z underwater area fine.  It was indeed as I did the shutdown... and I did assume it was the persisting to data base part.. that took nearly 30 minutes.  It was SQLite for the add on regions we have for experiments.. not our MySQL usual data base.  But still.. constantly creating and deleting the db-journal file (perhaps once per DB write?) MUST make this persisting process slower than needed?  I also thought I better make sure the persisting process completed even though it as taking ages or else there may have been only a partial translation.  If I had not had the opensim\bin window open I would have thought the OpenSim.exe console was jammed and stopped wit with the X on the console window... I assume that would NOT have been good, so it could be a source fo problems if persisting objects takes so long.  A better indication of progress might be to put a dot out every 50 or 100 objects persisted on the console to remind the user the region OpenSim.exe still need time to complete.





_______________________________________________
Opensim-users mailing list
[hidden email]
http://opensimulator.org/cgi-bin/mailman/listinfo/opensim-users


_______________________________________________
Opensim-users mailing list
[hidden email]
http://opensimulator.org/cgi-bin/mailman/listinfo/opensim-users