Working With Rails

Hello Balaji

300 MB is not a big deal I think. And ruby itself has memory-leak such as symbol which will not be gc-ed. memcache is not related to memory-leak. It’s just a cache and use memory in its memory space. I think you can profile it. You may want to google the rails profiling.

Hope this helps

Regards

Thanks for the reply Naveen,

But the figure 300 MB i mentioned was for a single instance and i use 20 clusters, so i feel the cumulative figure to be bit on the higher side.

what is the recommended memory to be allocated for memcache ( for an application which provides search feature for more than 2 million records )?

Thanks to inform that symbols are not considered in normal GC, is there any alternative fort his case?

I run monit on my servers following Ezra’s recommendations which restarts a mongrel instance at 110mb plus several other issues. Ever since I put monit into the stack everything runs great. Much more reliable than before.

Check out the monit config sample here. You can adjust the config to meet your setup. Just a note, you need an entry for each mongrel instance you are running, so 20 mongrels equals 20 ‘check process’ blocks in your config file. At least that’s how I did it. There may be a shorter config solution but I’m not a Linux expert.

Balaji

Whenever I had such issues I use to profile the app. however I am unsure what alternative you are referring ?

You may want to send me an email and explain, I would try to help as much as I can

Thanks

300 IS A BIG DEAL.

Please use something like Evan Weaver’s bleak_house to find out the problem.

You can run ruby-prof ad measure for memory, rather than processor-cycles. Google for it to get the details of how.
Then you can discover which parts of your code are causing the memory to leap upwards and improve the nasty areas.

Can Monit be configured in Windows ?

oh sorry. Monit isn’t avaliable for Windows. However, there are lots of commercial products on the market for Windows server monitoring that range from the simple to the complex with full email and paging support, auto-recovery, full blown reporting, real-time GUI status and remote administration.

I’m sure there must be something open source as well but I haven’t looked around for anything like it yet.

Thanks for your Reply Raul

Can someone name tools similar to Monit that can be used in Windows

Windows itself can recover from stopped or stuck services. If you go into the Computer Management > Services admin you can set what you want Windows to do when a service hangs or stops. It will also log the event and/or run a program when directed to so you can use that to launch scripts that can email you or do more complex tasks.

An example of a commercial offering is something like Nimsoft. This is only one example of the hundreds that are on the market. Are you in an enterprise environment? Do you have a network admin on site?

Actually i was looking for some open source tool which i can use to monitor my mongrel clusters, restart them when they are overloaded or down.

You can definitely install Mongrel as a Windows service and gain the recovery and monitoring benefits but I don’t know if you can still run a cluster of Mongrels as a service. You’d have to research that a bit.

There is a good post on google groups where Tom Werner walks through his process of finding a memory leak that was in an earlier version of God

http://groups.google.com/group/god-rb/browse_thread/thread/1cca2b7c4a581c2

A good read from start to finish to see some leak debugging in action..

As per the article in http://mongrel.rubyforge.org/wiki/FAQ, we can track memory usage by starting mongrel using

mongrel_rails start -B

But this command just dumps a lot of data in the log file , I am not able to interpret it

the dump is something like this

run,classname,last,count,delta,lenmean,lensd,lenmax
1,Float,106655,117957,11302,,,
1,String,59251,64611,5360,24.858182,136.757360,15791.000000
1,Array,7851,7092,-759,5.576283,25.001869,934.000000
1,Proc,548,1722,1174,,,
1,Bignum,5054,1623,-3431,,,
1,Hash,947,1470,523,10.513605,51.913194,692.000000
1,Regexp,1341,1311,-30,,,
1,Class,1161,1192,31,,,
1,Rational,2761,1026,-1735,,,

can anyone explain what conclusion can be drawn from this dump?

you should get a number of files generated by logging in the mongrel_debug directory that let you look at different aspects. the one you’re looking at looks like objects.log

now, objects log shows you the top object types (in terms of number) and shows you the number of each object in existence before and after the request (run)

so, in your case, the first run meant an increase of 11302 in the number of floats, etc.

i think the idea here is if you have a particular object type that is always increasing in number, then it’s going to increase memory usage. if it always increases by a significant amount on a particular action then, again, it looks like it’s not being cleaned up.

Thanks a lot for the reply Chris

Is there any threshold level for this counts, i mean beyond which we can declare that they are not being cleaned up properly.

Is it advisable to call GC explicitly using GC start ?