More load problems

[Snake]

Ok so i have done loads of testing.

Over a period of 24hrs my load rises steadily into the 30.00's

A server restart fixes it for another 24hrs.

If i disable the mod i do not get this problem.

Do not say that with mod disabled there is less traffic as that is not true, google still sends the same traffic with mod disabled according to my stats.

Please explain, load is driving me mad.

[tavenger5]

Sounds like bots are hitting the translated pages when the mod is on. You should look into optimizing apache or getting a server with more cpu power. Are you running vboptimise or any type of caching mechanism, like memcached?

[Snake]

The fact is my forum gets 30,000 uniques a day, if i disable the mod load drops instantly, and bots and users are still show pages so are still demanding server power, it is simply that showing translated pages use 10x the resources than standard vb pages from the normal database. It's poorly written code, and needs to be fixed! The other mods never did this, just vbet, wish i never changed over, but going back is too late now.

[moman]

It really sounds like you're running on an underpowered server. I'm on a 8-core nehalem machine (so we're looking at 8 more virtual cores due to the HT, for a total of 16). I've also extensively optimized this machine using my own techniques as well as pointers from the folks at vbulletin.com.

vBET raises my load from about 2.5 to 3.0-3.5, depending on number of users, and this obviously grows based on the cache size. However, I don't think that's too bad at all, as my simultaneous users range from a fairly high 800 to an even higher 1,200.

[vBET]

Hi Of course translation MUST take some resources - there is no magic. Translate page means take output result and change it. Because of that translation will always take longer than normal page.

I understand that you are not too happy having more server load, but please note that vBET is taking resources ONLY FOR TRANSLATION. For normal pages it adds only flags. So all this additional load comes from additional traffic to translated pages. As you wrote your traffic didn't go down instantly after disabling vBET (if you have it a while then it will decrease after disabling - trust me) and server load is lower - it is obvious - robots are still crawling URLs to translated pages, users are still finding in Google links to your translated pages. So you still have same traffic, but now under translated links is simply duplicated content - normal page which is not translated. If you want to stay with disabled vBET we strongly recommend to add rule in your .htaccess file which will redirect all translated pages to normal one, otherwise you can loose your SEO because of duplicated content.

Please note that we have already planned supporting of other cache systems and our translation algorithms are instantly optimized. I.e. we just discovered how drastically decreases PHP performance when working on large strings and we modified our algorithm. It is already released in vBET 4.2.0 with additional configuration options. And we will move all improvements also to vBET 3.x which is still supported

I understand that in your opinion our code is weak. I do not know on what you are basing your expectations. We have fastest translation mod for vB - there is nothing what works better. Translation will take some resources and our mod takes it less that any other. You can see how fast vBET can work on many forums. If you have issues on your server, then please consider configuration changes or adding server resources. You will not put 20 liters of water into 10 liters bucket.

Our devise is: "We have lot to change". And that is why we are experimenting, changing algorithms, profiling and spend lot of time looking for solutions which will require less resources. Still we know no mod which could be any competition to vBET and there are some other translation mods. We made lot of algorithm changes which we had to trow away because they didn't help, during this process we also discovered many improvements. You can have your impression basing on your server issues, but please consider do you have any better solution? What could you give a hint that maybe vBET is not wrong solution since is working on thousands of forums, maybe you are just trying to put 20 liters of water into 10 liters bucket. Still - we have lot to change and great TODO list in optimization section (about 70% to experiment will it help or not) And you are 100% right - we can do it better, we will and we ARE doing it all the time Just wait till we move all improvements which we made during vBET4.x implementation

If I can give you some hints - please check how you can optimize vBET: http://www.vbenterprisetranslator.co...rformance.html
Especially consider disabling some languages and blocking irrelevant pages by robots.txt

What is time of response for translated pages? What is your CPU usage? What is your memory usage? If it is acceptable then you have nothing to worry about. People often are scared by increasing server load and even don't know what does it means. 10 times more server load does NOT means 10 time more resources used. It just means that more threads are waiting in queue, what is perfectly normal, since right now your threads have to wait for Google response if some translation is not cached yet. So thread is waiting for Google response and it takes NO CPU AT ALL during this time. Because of that your server load WILL be bigger even if vBET could take no resources at all (what of course is not possible).

About crashing your server - it is obviously your server issue. It happens periodically. I had similar issue on my server. It was caused by some Apache bug so one Apache thread was growing and growing with memory usage until whole memory was consumed ans server cached. Only one thread behaves like that - other Apache threads was normal. I played with Apache configuration and issue is solved. I think that Apache just had some memory leakage - as I remember I put lower value of requests which can be hold by one thread. There were also other changes. I suggest to check your memory usage and monitor it for some time. Also it can be helpful to check average amount of memory used by one Apache thread, made some calculations and set appropriate value of max threads for Apache.

If you have any more questions please just ask

[vBET]

Hey - I just was on your forum it translates ULTRA FAST... So what you are claiming about and why so angry attitude about vBET, when you have super fast translation service?...

Please consider what server load means. Understanding its meaning can be very helpful to understand what happens on server and how it can be related to threads which are waiting in queue not because you have no resources, but are waiting for response from other server (Google in this case).

In my opinion you have super fast translations and you have nothing to worry about

[tavenger5]

How many languages do you have enabled? How many posts do you have? Do you have vbseo and sitemap generator installed? How many bots are hitting the site a day?

[Snake]

Hi!

32 languages enabled.

100,000 posts.

vbseo and sitemap installed.

sitemap plugin says 1000,000 pages crawled a day.


The fact is, if i disable the mod and restart the server, bot and users are still hitting my server the same as before, the same pages are in still in googles index and thus i get the same traffic enabled or not.

The only difference is that with the mod disabled the bots and google searchers get the english page, so, traffic is identical, the only difference is they are not seeing a translated page, thus avoiding the mods database issues.

It's clear as day to me this mod has been released without proper testing, and clearly the owner is not interested in addressing the issues.

Fail.

[vBET]

It's clear as day to me this mod has been released without proper testing, and clearly the owner is not interested in addressing the issues.

Fail.

If you have doubts about proper testing I propose to check history of vBET - it was tested by hundreds of real forums before it advanced to paid version

About addressing the issue. I'm sorry. I wrongly assumed that giving you in first response this informations, I was clear that we are addressing the issue:

Please note that we have already planned supporting of other cache systems and our translation algorithms are instantly optimized. I.e. we just discovered how drastically decreases PHP performance when working on large strings and we modified our algorithm. It is already released in vBET 4.2.0 with additional configuration options. And we will move all improvements also to vBET 3.x which is still supported

...

Our devise is: "We have lot to change". And that is why we are experimenting, changing algorithms, profiling and spend lot of time looking for solutions which will require less resources. Still we know no mod which could be any competition to vBET and there are some other translation mods. We made lot of algorithm changes which we had to trow away because they didn't help, during this process we also discovered many improvements. You can have your impression basing on your server issues, but please consider do you have any better solution? What could you give a hint that maybe vBET is not wrong solution since is working on thousands of forums, maybe you are just trying to put 20 liters of water into 10 liters bucket. Still - we have lot to change and great TODO list in optimization section (about 70% to experiment will it help or not) And you are 100% right - we can do it better, we will and we ARE doing it all the time Just wait till we move all improvements which we made during vBET4.x implementation

If I can give you some hints - please check how you can optimize vBET: http://www.vbenterprisetranslator.co...rformance.html
Especially consider disabling some languages and blocking irrelevant pages by robots.txt

I feel fully responsible for this misunderstanding. Once again very sorry. Please tell us, which way we have to tell you that we are improving vBET all the time, and how we can assure you again that vBET3.x will have another performance improvements, to make you clear that we are addressing the issue? We will always gladly improve our way of communication with client

[Snake]

Michael, you are wrong, this load is not caused by an increase in traffic, it is caused by reading and writing translations to mysql.

If vbulletin team can read and write to mysql with low load why cant you?

Poor code my friend.