What is the best way to manage asset cache on an OSgrid set of regions

classic Classic list List threaded Threaded
10 messages Options
Reply | Threaded
Open this post in threaded view
|

What is the best way to manage asset cache on an OSgrid set of regions

tringate
I run three servers each have a few instances of opensim running on them and all of the regions are on OSgrid.

I have a common asset cache  for each servers, that all the instances use.  Maybe that is not a good idea, but I thought it might help keep it’s size
smaller than every instance having it’s own asset cache.

Up to now I have not expired any items in the cache.  Yes, they are massive in size.

I have been thinking about having one instance expire the cache and using a retention time of 48 hours.

Is that a good or bad idea?

I do have a fair amount of activity on each server as I run free home sites for many OSgrid users.

I am wondering a few things:

- Is running a single cache for several instances of opensim a good or bad idea?
- Is not expiring the cache a good or bad idea?
- Can one instance do the expiring of the cache for all instances since they all share the same cache?

Tom



_______________________________________________
Opensim-users mailing list
[hidden email]
http://opensimulator.org/cgi-bin/mailman/listinfo/opensim-users
Reply | Threaded
Open this post in threaded view
|

Re: What is the best way to manage asset cache on an OSgrid set of regions

AJLDuarte
Hi,
        A) yes it is good idea to share assets cache
        B and C)
        Cache will grow for ever taking 2 much disk space possible with useless entries.
        Using one instance to do expires is a option, it will even try to not delete assets in use, but just on that instance, so not that useful in this case, just waste of cpu.
        Using a external tool (both windows cmd and bash have usable commands) to delete files modified long ago is another option.
        Wrong deletes should not be a major issue, assuming all assets are on grid databases, if not you are in trouble anyways :)
Regards,
Ubit

-----Original Message-----
From: [hidden email] [mailto:[hidden email]] On Behalf Of [hidden email]
Sent: Saturday, January 06, 2018 04:08
To: OS-Opensim Users
Subject: [Opensim-users] What is the best way to manage asset cache on an OSgrid set of regions

I run three servers each have a few instances of opensim running on them and all of the regions are on OSgrid.

I have a common asset cache  for each servers, that all the instances use.  Maybe that is not a good idea, but I thought it might help keep it’s size smaller than every instance having it’s own asset cache.

Up to now I have not expired any items in the cache.  Yes, they are massive in size.

I have been thinking about having one instance expire the cache and using a retention time of 48 hours.

Is that a good or bad idea?

I do have a fair amount of activity on each server as I run free home sites for many OSgrid users.

I am wondering a few things:

- Is running a single cache for several instances of opensim a good or bad idea?
- Is not expiring the cache a good or bad idea?
- Can one instance do the expiring of the cache for all instances since they all share the same cache?

Tom



_______________________________________________
Opensim-users mailing list
[hidden email]
http://opensimulator.org/cgi-bin/mailman/listinfo/opensim-users

_______________________________________________
Opensim-users mailing list
[hidden email]
http://opensimulator.org/cgi-bin/mailman/listinfo/opensim-users
Reply | Threaded
Open this post in threaded view
|

Re: What is the best way to manage asset cache onan OSgrid set of regions

tringate
Ubit,

Thanks for the information.  It gives me the needed information to set up a
"trim" function from cron nightly.

I have one additional question, is an asset's modified date updated by
opensim if it uses the asset or does it remain the date it was downloaded
from the asset server once it is placed in the cache?

Reason I ask is to better understand if cache is more for an immediate
experience for a user on their present login, or, if it is intended to speed
things up for days after an asset was loaded.  I'm trying to understand the
"hit" on the OSgrid asset server of reloading the same assets day after day.

I usually see messages when getting things from the grids asset server of
long load times in the 5 to 6 second range.

Tom

-----Original Message-----
From: AJLDuarte
Sent: Saturday, January 6, 2018 3:29 PM
To: [hidden email]
Subject: Re: [Opensim-users] What is the best way to manage asset cache onan
OSgrid set of regions

Hi,
A) yes it is good idea to share assets cache
B and C)
Cache will grow for ever taking 2 much disk space possible with useless
entries.
Using one instance to do expires is a option, it will even try to not delete
assets in use, but just on that instance, so not that useful in this case,
just waste of cpu.
Using a external tool (both windows cmd and bash have usable commands) to
delete files modified long ago is another option.
Wrong deletes should not be a major issue, assuming all assets are on grid
databases, if not you are in trouble anyways :)
Regards,
Ubit

-----Original Message-----
From: [hidden email]
[mailto:[hidden email]] On Behalf Of
[hidden email]
Sent: Saturday, January 06, 2018 04:08
To: OS-Opensim Users
Subject: [Opensim-users] What is the best way to manage asset cache on an
OSgrid set of regions

I run three servers each have a few instances of opensim running on them and
all of the regions are on OSgrid.

I have a common asset cache  for each servers, that all the instances use.
Maybe that is not a good idea, but I thought it might help keep it’s size
smaller than every instance having it’s own asset cache.

Up to now I have not expired any items in the cache.  Yes, they are massive
in size.

I have been thinking about having one instance expire the cache and using a
retention time of 48 hours.

Is that a good or bad idea?

I do have a fair amount of activity on each server as I run free home sites
for many OSgrid users.

I am wondering a few things:

- Is running a single cache for several instances of opensim a good or bad
idea?
- Is not expiring the cache a good or bad idea?
- Can one instance do the expiring of the cache for all instances since they
all share the same cache?

Tom



_______________________________________________
Opensim-users mailing list
[hidden email]
http://opensimulator.org/cgi-bin/mailman/listinfo/opensim-users

_______________________________________________
Opensim-users mailing list
[hidden email]
http://opensimulator.org/cgi-bin/mailman/listinfo/opensim-users 

_______________________________________________
Opensim-users mailing list
[hidden email]
http://opensimulator.org/cgi-bin/mailman/listinfo/opensim-users
Reply | Threaded
Open this post in threaded view
|

Re: What is the best way to manage asset cache onan OSgrid set of regions

AJLDuarte
Yes, floatsam cache does update the files using
        File.SetLastAccessTime(filename, DateTime.Now);
Ubit


-----Original Message-----
From: [hidden email] [mailto:[hidden email]] On Behalf Of [hidden email]
Sent: Sunday, January 07, 2018 19:26
To: [hidden email]
Subject: Re: [Opensim-users] What is the best way to manage asset cache onan OSgrid set of regions

Ubit,

Thanks for the information.  It gives me the needed information to set up a "trim" function from cron nightly.

I have one additional question, is an asset's modified date updated by opensim if it uses the asset or does it remain the date it was downloaded from the asset server once it is placed in the cache?

Reason I ask is to better understand if cache is more for an immediate experience for a user on their present login, or, if it is intended to speed things up for days after an asset was loaded.  I'm trying to understand the "hit" on the OSgrid asset server of reloading the same assets day after day.

I usually see messages when getting things from the grids asset server of long load times in the 5 to 6 second range.

Tom

-----Original Message-----
From: AJLDuarte
Sent: Saturday, January 6, 2018 3:29 PM
To: [hidden email]
Subject: Re: [Opensim-users] What is the best way to manage asset cache onan OSgrid set of regions

Hi,
A) yes it is good idea to share assets cache B and C) Cache will grow for ever taking 2 much disk space possible with useless entries.
Using one instance to do expires is a option, it will even try to not delete assets in use, but just on that instance, so not that useful in this case, just waste of cpu.
Using a external tool (both windows cmd and bash have usable commands) to delete files modified long ago is another option.
Wrong deletes should not be a major issue, assuming all assets are on grid databases, if not you are in trouble anyways :) Regards, Ubit

-----Original Message-----
From: [hidden email]
[mailto:[hidden email]] On Behalf Of [hidden email]
Sent: Saturday, January 06, 2018 04:08
To: OS-Opensim Users
Subject: [Opensim-users] What is the best way to manage asset cache on an OSgrid set of regions

I run three servers each have a few instances of opensim running on them and all of the regions are on OSgrid.

I have a common asset cache  for each servers, that all the instances use.
Maybe that is not a good idea, but I thought it might help keep it’s size smaller than every instance having it’s own asset cache.

Up to now I have not expired any items in the cache.  Yes, they are massive in size.

I have been thinking about having one instance expire the cache and using a retention time of 48 hours.

Is that a good or bad idea?

I do have a fair amount of activity on each server as I run free home sites for many OSgrid users.

I am wondering a few things:

- Is running a single cache for several instances of opensim a good or bad idea?
- Is not expiring the cache a good or bad idea?
- Can one instance do the expiring of the cache for all instances since they all share the same cache?

Tom



_______________________________________________
Opensim-users mailing list
[hidden email]
http://opensimulator.org/cgi-bin/mailman/listinfo/opensim-users

_______________________________________________
Opensim-users mailing list
[hidden email]
http://opensimulator.org/cgi-bin/mailman/listinfo/opensim-users 

_______________________________________________
Opensim-users mailing list
[hidden email]
http://opensimulator.org/cgi-bin/mailman/listinfo/opensim-users

_______________________________________________
Opensim-users mailing list
[hidden email]
http://opensimulator.org/cgi-bin/mailman/listinfo/opensim-users
Reply | Threaded
Open this post in threaded view
|

Re: What is the best way to manage asset cache onan OSgrid set of regions

Drang Po
ajlduarte where do I find the floatsam cache files?

"Yes, floatsam cache does update the files using
        File.SetLastAccessTime(filename, DateTime.Now); "

On Sun, Jan 7, 2018 at 11:34 AM, AJLDuarte <[hidden email]> wrote:

> Yes, floatsam cache does update the files using
>         File.SetLastAccessTime(filename, DateTime.Now);
> Ubit
>
>
> -----Original Message-----
> From: [hidden email] [mailto:
> [hidden email]] On Behalf Of [hidden email]
> Sent: Sunday, January 07, 2018 19:26
> To: [hidden email]
> Subject: Re: [Opensim-users] What is the best way to manage asset cache
> onan OSgrid set of regions
>
> Ubit,
>
> Thanks for the information.  It gives me the needed information to set up
> a "trim" function from cron nightly.
>
> I have one additional question, is an asset's modified date updated by
> opensim if it uses the asset or does it remain the date it was downloaded
> from the asset server once it is placed in the cache?
>
> Reason I ask is to better understand if cache is more for an immediate
> experience for a user on their present login, or, if it is intended to
> speed things up for days after an asset was loaded.  I'm trying to
> understand the "hit" on the OSgrid asset server of reloading the same
> assets day after day.
>
> I usually see messages when getting things from the grids asset server of
> long load times in the 5 to 6 second range.
>
> Tom
>
> -----Original Message-----
> From: AJLDuarte
> Sent: Saturday, January 6, 2018 3:29 PM
> To: [hidden email]
> Subject: Re: [Opensim-users] What is the best way to manage asset cache
> onan OSgrid set of regions
>
> Hi,
> A) yes it is good idea to share assets cache B and C) Cache will grow for
> ever taking 2 much disk space possible with useless entries.
> Using one instance to do expires is a option, it will even try to not
> delete assets in use, but just on that instance, so not that useful in this
> case, just waste of cpu.
> Using a external tool (both windows cmd and bash have usable commands) to
> delete files modified long ago is another option.
> Wrong deletes should not be a major issue, assuming all assets are on grid
> databases, if not you are in trouble anyways :) Regards, Ubit
>
> -----Original Message-----
> From: [hidden email]
> [mailto:[hidden email]] On Behalf Of
> [hidden email]
> Sent: Saturday, January 06, 2018 04:08
> To: OS-Opensim Users
> Subject: [Opensim-users] What is the best way to manage asset cache on an
> OSgrid set of regions
>
> I run three servers each have a few instances of opensim running on them
> and all of the regions are on OSgrid.
>
> I have a common asset cache  for each servers, that all the instances use.
> Maybe that is not a good idea, but I thought it might help keep it’s size
> smaller than every instance having it’s own asset cache.
>
> Up to now I have not expired any items in the cache.  Yes, they are
> massive in size.
>
> I have been thinking about having one instance expire the cache and using
> a retention time of 48 hours.
>
> Is that a good or bad idea?
>
> I do have a fair amount of activity on each server as I run free home
> sites for many OSgrid users.
>
> I am wondering a few things:
>
> - Is running a single cache for several instances of opensim a good or bad
> idea?
> - Is not expiring the cache a good or bad idea?
> - Can one instance do the expiring of the cache for all instances since
> they all share the same cache?
>
> Tom
>
>
>
> _______________________________________________
> Opensim-users mailing list
> [hidden email]
> http://opensimulator.org/cgi-bin/mailman/listinfo/opensim-users
>
> _______________________________________________
> Opensim-users mailing list
> [hidden email]
> http://opensimulator.org/cgi-bin/mailman/listinfo/opensim-users
>
> _______________________________________________
> Opensim-users mailing list
> [hidden email]
> http://opensimulator.org/cgi-bin/mailman/listinfo/opensim-users
>
> _______________________________________________
> Opensim-users mailing list
> [hidden email]
> http://opensimulator.org/cgi-bin/mailman/listinfo/opensim-users
>



--
Kalasiddhi Grid Co-Coordinator

HG: kalasiddhigrid.com:8002
Web: http://www.kalasiddhigrid.com
Buddhism in Virtual Reality

Drang Po
_______________________________________________
Opensim-users mailing list
[hidden email]
http://opensimulator.org/cgi-bin/mailman/listinfo/opensim-users
Reply | Threaded
Open this post in threaded view
|

Re: What is the best way to manage asset cache onan OSgrid set of regions

AJLDuarte
Hi,
        They are included and should be enabled using default configuration or the examples.
        Review files on bin/config-include
Ubit


-----Original Message-----
From: [hidden email] [mailto:[hidden email]] On Behalf Of Drang Po
Sent: Thursday, February 01, 2018 02:24
To: [hidden email]
Subject: Re: [Opensim-users] What is the best way to manage asset cache onan OSgrid set of regions

ajlduarte where do I find the floatsam cache files?

"Yes, floatsam cache does update the files using
        File.SetLastAccessTime(filename, DateTime.Now); "

On Sun, Jan 7, 2018 at 11:34 AM, AJLDuarte <[hidden email]> wrote:

> Yes, floatsam cache does update the files using
>         File.SetLastAccessTime(filename, DateTime.Now); Ubit
>
>
> -----Original Message-----
> From: [hidden email] [mailto:
> [hidden email]] On Behalf Of [hidden email]
> Sent: Sunday, January 07, 2018 19:26
> To: [hidden email]
> Subject: Re: [Opensim-users] What is the best way to manage asset cache
> onan OSgrid set of regions
>
> Ubit,
>
> Thanks for the information.  It gives me the needed information to set up
> a "trim" function from cron nightly.
>
> I have one additional question, is an asset's modified date updated by
> opensim if it uses the asset or does it remain the date it was downloaded
> from the asset server once it is placed in the cache?
>
> Reason I ask is to better understand if cache is more for an immediate
> experience for a user on their present login, or, if it is intended to
> speed things up for days after an asset was loaded.  I'm trying to
> understand the "hit" on the OSgrid asset server of reloading the same
> assets day after day.
>
> I usually see messages when getting things from the grids asset server of
> long load times in the 5 to 6 second range.
>
> Tom
>
> -----Original Message-----
> From: AJLDuarte
> Sent: Saturday, January 6, 2018 3:29 PM
> To: [hidden email]
> Subject: Re: [Opensim-users] What is the best way to manage asset cache
> onan OSgrid set of regions
>
> Hi,
> A) yes it is good idea to share assets cache B and C) Cache will grow for
> ever taking 2 much disk space possible with useless entries.
> Using one instance to do expires is a option, it will even try to not
> delete assets in use, but just on that instance, so not that useful in this
> case, just waste of cpu.
> Using a external tool (both windows cmd and bash have usable commands) to
> delete files modified long ago is another option.
> Wrong deletes should not be a major issue, assuming all assets are on grid
> databases, if not you are in trouble anyways :) Regards, Ubit
>
> -----Original Message-----
> From: [hidden email]
> [mailto:[hidden email]] On Behalf Of
> [hidden email]
> Sent: Saturday, January 06, 2018 04:08
> To: OS-Opensim Users
> Subject: [Opensim-users] What is the best way to manage asset cache on an
> OSgrid set of regions
>
> I run three servers each have a few instances of opensim running on them
> and all of the regions are on OSgrid.
>
> I have a common asset cache  for each servers, that all the instances use.
> Maybe that is not a good idea, but I thought it might help keep it’s size
> smaller than every instance having it’s own asset cache.
>
> Up to now I have not expired any items in the cache.  Yes, they are
> massive in size.
>
> I have been thinking about having one instance expire the cache and using
> a retention time of 48 hours.
>
> Is that a good or bad idea?
>
> I do have a fair amount of activity on each server as I run free home
> sites for many OSgrid users.
>
> I am wondering a few things:
>
> - Is running a single cache for several instances of opensim a good or bad
> idea?
> - Is not expiring the cache a good or bad idea?
> - Can one instance do the expiring of the cache for all instances since
> they all share the same cache?
>
> Tom
>
>
>
> _______________________________________________
> Opensim-users mailing list
> [hidden email]
> http://opensimulator.org/cgi-bin/mailman/listinfo/opensim-users
>
> _______________________________________________
> Opensim-users mailing list
> [hidden email]
> http://opensimulator.org/cgi-bin/mailman/listinfo/opensim-users
>
> _______________________________________________
> Opensim-users mailing list
> [hidden email]
> http://opensimulator.org/cgi-bin/mailman/listinfo/opensim-users
>
> _______________________________________________
> Opensim-users mailing list
> [hidden email]
> http://opensimulator.org/cgi-bin/mailman/listinfo/opensim-users
>



--
Kalasiddhi Grid Co-Coordinator

HG: kalasiddhigrid.com:8002
Web: http://www.kalasiddhigrid.com
Buddhism in Virtual Reality

Drang Po
_______________________________________________
Opensim-users mailing list
[hidden email]
http://opensimulator.org/cgi-bin/mailman/listinfo/opensim-users

_______________________________________________
Opensim-users mailing list
[hidden email]
http://opensimulator.org/cgi-bin/mailman/listinfo/opensim-users
Reply | Threaded
Open this post in threaded view
|

Re: What is the best way to manage asset cache on an OSgrid set of regions

David Saunders-2
In reply to this post by AJLDuarte
Hey,

 This what I have found out.

 The cache will grow.
 You can delete it at any time, it will be rebuilt. So each of our servers
deletes the cache files from time to time.
cache file reduced the load on the asset server, this is very important
When the asset server is in a different location on the internet.
Cache is good when you allot of shared assets between simulators., it
allows local files to be used when you get cache hits.

We run 20 of single region simulators on several servers It does help with
the load on the asset server and the database chop.
We do use the floatsome cache, thought about a RAM cache a few times but I
studied it some and thing the size of some or regions would make it
unpractical to implement.

David


On Sat, Jan 6, 2018 at 3:29 PM, AJLDuarte <[hidden email]> wrote:

> Hi,
>         A) yes it is good idea to share assets cache
>         B and C)
>         Cache will grow for ever taking 2 much disk space possible with
> useless entries.
>         Using one instance to do expires is a option, it will even try to
> not delete assets in use, but just on that instance, so not that useful in
> this case, just waste of cpu.
>         Using a external tool (both windows cmd and bash have usable
> commands) to delete files modified long ago is another option.
>         Wrong deletes should not be a major issue, assuming all assets are
> on grid databases, if not you are in trouble anyways :)
> Regards,
> Ubit
>
> -----Original Message-----
> From: [hidden email] [mailto:
> [hidden email]] On Behalf Of [hidden email]
> Sent: Saturday, January 06, 2018 04:08
> To: OS-Opensim Users
> Subject: [Opensim-users] What is the best way to manage asset cache on an
> OSgrid set of regions
>
> I run three servers each have a few instances of opensim running on them
> and all of the regions are on OSgrid.
>
> I have a common asset cache  for each servers, that all the instances
> use.  Maybe that is not a good idea, but I thought it might help keep it’s
> size smaller than every instance having it’s own asset cache.
>
> Up to now I have not expired any items in the cache.  Yes, they are
> massive in size.
>
> I have been thinking about having one instance expire the cache and using
> a retention time of 48 hours.
>
> Is that a good or bad idea?
>
> I do have a fair amount of activity on each server as I run free home
> sites for many OSgrid users.
>
> I am wondering a few things:
>
> - Is running a single cache for several instances of opensim a good or bad
> idea?
> - Is not expiring the cache a good or bad idea?
> - Can one instance do the expiring of the cache for all instances since
> they all share the same cache?
>
> Tom
>
>
>
> _______________________________________________
> Opensim-users mailing list
> [hidden email]
> http://opensimulator.org/cgi-bin/mailman/listinfo/opensim-users
>
> _______________________________________________
> Opensim-users mailing list
> [hidden email]
> http://opensimulator.org/cgi-bin/mailman/listinfo/opensim-users
>
_______________________________________________
Opensim-users mailing list
[hidden email]
http://opensimulator.org/cgi-bin/mailman/listinfo/opensim-users
Reply | Threaded
Open this post in threaded view
|

Re: What is the best way to manage asset cache on an OSgrid set of regions

tringate
I run my regions on three servers.  After ubit's explanation of how the
cache works I have set up a common cache on each server and run a cron
process nightly to remove any file that has not been accessed in the past 26
hours.

Remember the file "date modified" is updated whenever that file is accessed
in the cache.

This keeps the number of files to a reasonable number in the cache, and
allows for frequently accessed files to remain in cache indefinitely.

Deleting the entire cache on a periodic bases does not sound like a good
idea to me in terms of load on the asset server.  If you have a file that is
constantly being accessed within a 24 hour period it will be server from
cache but if it is less frequently accessed then the asset server would be
the source.  I have been thinking about changing the trim time to be 48
hours to see if the number of files changes appreciably.

I restart all of my regions every night so items on the region should always
remain in cache.  That is why I set the trim function to remove on a basis
of 26 hours.

Tom

-----Original Message-----
From: David Saunders
Sent: Thursday, February 1, 2018 12:57 PM
To: [hidden email]
Subject: Re: [Opensim-users] What is the best way to manage asset cache on
an OSgrid set of regions

Hey,

This what I have found out.

The cache will grow.
You can delete it at any time, it will be rebuilt. So each of our servers
deletes the cache files from time to time.
cache file reduced the load on the asset server, this is very important
When the asset server is in a different location on the internet.
Cache is good when you allot of shared assets between simulators., it
allows local files to be used when you get cache hits.

We run 20 of single region simulators on several servers It does help with
the load on the asset server and the database chop.
We do use the floatsome cache, thought about a RAM cache a few times but I
studied it some and thing the size of some or regions would make it
unpractical to implement.

David


On Sat, Jan 6, 2018 at 3:29 PM, AJLDuarte <[hidden email]> wrote:

> Hi,
>         A) yes it is good idea to share assets cache
>         B and C)
>         Cache will grow for ever taking 2 much disk space possible with
> useless entries.
>         Using one instance to do expires is a option, it will even try to
> not delete assets in use, but just on that instance, so not that useful in
> this case, just waste of cpu.
>         Using a external tool (both windows cmd and bash have usable
> commands) to delete files modified long ago is another option.
>         Wrong deletes should not be a major issue, assuming all assets are
> on grid databases, if not you are in trouble anyways :)
> Regards,
> Ubit
>
> -----Original Message-----
> From: [hidden email] [mailto:
> [hidden email]] On Behalf Of [hidden email]
> Sent: Saturday, January 06, 2018 04:08
> To: OS-Opensim Users
> Subject: [Opensim-users] What is the best way to manage asset cache on an
> OSgrid set of regions
>
> I run three servers each have a few instances of opensim running on them
> and all of the regions are on OSgrid.
>
> I have a common asset cache  for each servers, that all the instances
> use.  Maybe that is not a good idea, but I thought it might help keep it’s
> size smaller than every instance having it’s own asset cache.
>
> Up to now I have not expired any items in the cache.  Yes, they are
> massive in size.
>
> I have been thinking about having one instance expire the cache and using
> a retention time of 48 hours.
>
> Is that a good or bad idea?
>
> I do have a fair amount of activity on each server as I run free home
> sites for many OSgrid users.
>
> I am wondering a few things:
>
> - Is running a single cache for several instances of opensim a good or bad
> idea?
> - Is not expiring the cache a good or bad idea?
> - Can one instance do the expiring of the cache for all instances since
> they all share the same cache?
>
> Tom
>
>
>
> _______________________________________________
> Opensim-users mailing list
> [hidden email]
> http://opensimulator.org/cgi-bin/mailman/listinfo/opensim-users
>
> _______________________________________________
> Opensim-users mailing list
> [hidden email]
> http://opensimulator.org/cgi-bin/mailman/listinfo/opensim-users
>
_______________________________________________
Opensim-users mailing list
[hidden email]
http://opensimulator.org/cgi-bin/mailman/listinfo/opensim-users 

_______________________________________________
Opensim-users mailing list
[hidden email]
http://opensimulator.org/cgi-bin/mailman/listinfo/opensim-users
Reply | Threaded
Open this post in threaded view
|

Re: What is the best way to manage asset cache on an OSgrid set of regions

AJLDuarte
Hi
        Better have a larger expire time if disk is not critical.
        Assets will only be touched if anything like viewer asks for them, and viewers also have cache.
Ubit

-----Original Message-----
From: [hidden email] [mailto:[hidden email]] On Behalf Of [hidden email]
Sent: Thursday, February 01, 2018 18:38
To: [hidden email]
Subject: Re: [Opensim-users] What is the best way to manage asset cache on an OSgrid set of regions

I run my regions on three servers.  After ubit's explanation of how the cache works I have set up a common cache on each server and run a cron process nightly to remove any file that has not been accessed in the past 26 hours.

Remember the file "date modified" is updated whenever that file is accessed in the cache.

This keeps the number of files to a reasonable number in the cache, and allows for frequently accessed files to remain in cache indefinitely.

Deleting the entire cache on a periodic bases does not sound like a good idea to me in terms of load on the asset server.  If you have a file that is constantly being accessed within a 24 hour period it will be server from cache but if it is less frequently accessed then the asset server would be the source.  I have been thinking about changing the trim time to be 48 hours to see if the number of files changes appreciably.

I restart all of my regions every night so items on the region should always remain in cache.  That is why I set the trim function to remove on a basis of 26 hours.

Tom

-----Original Message-----
From: David Saunders
Sent: Thursday, February 1, 2018 12:57 PM
To: [hidden email]
Subject: Re: [Opensim-users] What is the best way to manage asset cache on an OSgrid set of regions

Hey,

This what I have found out.

The cache will grow.
You can delete it at any time, it will be rebuilt. So each of our servers deletes the cache files from time to time.
cache file reduced the load on the asset server, this is very important When the asset server is in a different location on the internet.
Cache is good when you allot of shared assets between simulators., it allows local files to be used when you get cache hits.

We run 20 of single region simulators on several servers It does help with the load on the asset server and the database chop.
We do use the floatsome cache, thought about a RAM cache a few times but I studied it some and thing the size of some or regions would make it unpractical to implement.

David


On Sat, Jan 6, 2018 at 3:29 PM, AJLDuarte <[hidden email]> wrote:

> Hi,
>         A) yes it is good idea to share assets cache
>         B and C)
>         Cache will grow for ever taking 2 much disk space possible
> with useless entries.
>         Using one instance to do expires is a option, it will even try
> to not delete assets in use, but just on that instance, so not that
> useful in this case, just waste of cpu.
>         Using a external tool (both windows cmd and bash have usable
> commands) to delete files modified long ago is another option.
>         Wrong deletes should not be a major issue, assuming all assets
> are on grid databases, if not you are in trouble anyways :) Regards,
> Ubit
>
> -----Original Message-----
> From: [hidden email] [mailto:
> [hidden email]] On Behalf Of
> [hidden email]
> Sent: Saturday, January 06, 2018 04:08
> To: OS-Opensim Users
> Subject: [Opensim-users] What is the best way to manage asset cache on
> an OSgrid set of regions
>
> I run three servers each have a few instances of opensim running on
> them and all of the regions are on OSgrid.
>
> I have a common asset cache  for each servers, that all the instances
> use.  Maybe that is not a good idea, but I thought it might help keep
> it’s size smaller than every instance having it’s own asset cache.
>
> Up to now I have not expired any items in the cache.  Yes, they are
> massive in size.
>
> I have been thinking about having one instance expire the cache and
> using a retention time of 48 hours.
>
> Is that a good or bad idea?
>
> I do have a fair amount of activity on each server as I run free home
> sites for many OSgrid users.
>
> I am wondering a few things:
>
> - Is running a single cache for several instances of opensim a good or
> bad idea?
> - Is not expiring the cache a good or bad idea?
> - Can one instance do the expiring of the cache for all instances
> since they all share the same cache?
>
> Tom
>
>
>
> _______________________________________________
> Opensim-users mailing list
> [hidden email]
> http://opensimulator.org/cgi-bin/mailman/listinfo/opensim-users
>
> _______________________________________________
> Opensim-users mailing list
> [hidden email]
> http://opensimulator.org/cgi-bin/mailman/listinfo/opensim-users
>
_______________________________________________
Opensim-users mailing list
[hidden email]
http://opensimulator.org/cgi-bin/mailman/listinfo/opensim-users 

_______________________________________________
Opensim-users mailing list
[hidden email]
http://opensimulator.org/cgi-bin/mailman/listinfo/opensim-users

_______________________________________________
Opensim-users mailing list
[hidden email]
http://opensimulator.org/cgi-bin/mailman/listinfo/opensim-users
Reply | Threaded
Open this post in threaded view
|

Re: What is the best way to manage asset cache on an OSgrid set of regions

AJLDuarte
In reply to this post by David Saunders-2
Hi
        The use of ram cache has problems
        Framework memory garbage collector (gc) will consider memory allocated for certain amount of time as allocated for the rest of application life. Either mono or windows, both their own way and different criteria.
        So if when our memory cache expires a asset, its memory may never be released staying there dead, impossible to use until the simulator (or robust) shuts down.
        We already have this issue with objects with shorter use time than such assets cache would have.
Ubit

-----Original Message-----
From: [hidden email] [mailto:[hidden email]] On Behalf Of David Saunders
Sent: Thursday, February 01, 2018 17:57
To: [hidden email]
Subject: Re: [Opensim-users] What is the best way to manage asset cache on an OSgrid set of regions

Hey,

 This what I have found out.

 The cache will grow.
 You can delete it at any time, it will be rebuilt. So each of our servers deletes the cache files from time to time.
cache file reduced the load on the asset server, this is very important When the asset server is in a different location on the internet.
Cache is good when you allot of shared assets between simulators., it allows local files to be used when you get cache hits.

We run 20 of single region simulators on several servers It does help with the load on the asset server and the database chop.
We do use the floatsome cache, thought about a RAM cache a few times but I studied it some and thing the size of some or regions would make it unpractical to implement.

David


On Sat, Jan 6, 2018 at 3:29 PM, AJLDuarte <[hidden email]> wrote:

> Hi,
>         A) yes it is good idea to share assets cache
>         B and C)
>         Cache will grow for ever taking 2 much disk space possible
> with useless entries.
>         Using one instance to do expires is a option, it will even try
> to not delete assets in use, but just on that instance, so not that
> useful in this case, just waste of cpu.
>         Using a external tool (both windows cmd and bash have usable
> commands) to delete files modified long ago is another option.
>         Wrong deletes should not be a major issue, assuming all assets
> are on grid databases, if not you are in trouble anyways :) Regards,
> Ubit
>
> -----Original Message-----
> From: [hidden email] [mailto:
> [hidden email]] On Behalf Of
> [hidden email]
> Sent: Saturday, January 06, 2018 04:08
> To: OS-Opensim Users
> Subject: [Opensim-users] What is the best way to manage asset cache on
> an OSgrid set of regions
>
> I run three servers each have a few instances of opensim running on
> them and all of the regions are on OSgrid.
>
> I have a common asset cache  for each servers, that all the instances
> use.  Maybe that is not a good idea, but I thought it might help keep
> it’s size smaller than every instance having it’s own asset cache.
>
> Up to now I have not expired any items in the cache.  Yes, they are
> massive in size.
>
> I have been thinking about having one instance expire the cache and
> using a retention time of 48 hours.
>
> Is that a good or bad idea?
>
> I do have a fair amount of activity on each server as I run free home
> sites for many OSgrid users.
>
> I am wondering a few things:
>
> - Is running a single cache for several instances of opensim a good or
> bad idea?
> - Is not expiring the cache a good or bad idea?
> - Can one instance do the expiring of the cache for all instances
> since they all share the same cache?
>
> Tom
>
>
>
> _______________________________________________
> Opensim-users mailing list
> [hidden email]
> http://opensimulator.org/cgi-bin/mailman/listinfo/opensim-users
>
> _______________________________________________
> Opensim-users mailing list
> [hidden email]
> http://opensimulator.org/cgi-bin/mailman/listinfo/opensim-users
>
_______________________________________________
Opensim-users mailing list
[hidden email]
http://opensimulator.org/cgi-bin/mailman/listinfo/opensim-users

_______________________________________________
Opensim-users mailing list
[hidden email]
http://opensimulator.org/cgi-bin/mailman/listinfo/opensim-users