Best unofficial Apache Server developers community
Username
Forgot password?
Sign in with Twitter account
Sign in with Facebook account
List archives

Redis high CPU and fork rate on OpenVZ

Redis running very slow
(19 lines)
Feb 6, 2012
Damon
Damon
Hi,

I'm currently having some issues with Redis spiking in CPU and have been 
unable to determine what is causing it.

Host OS: Centos 5.5 on an OpenVZ VM
Memory: 12GB soft limit, 16GB hard limit
Redis version: 2.2.4
Memory: 2.45GB, 10.71GB RSS
Keys in db0: ~4M with ~150k expired
Total "expired_keys" from redis-cli info: 49631528
Persistence is disabled

Any time the fork rate on the system spikes above about 100, redis hits 
100% CPU and starts to become inaccessible. Sometimes I can't even connect 
to the redis port while this is happening. I've done a lot of debugging, 
but have yet to be able to figure out what is going on. I'm not sure what's

causing the forking on the system as any redis writes to disk are disabled.

This box is a backend for Resque and I've tried disabling all workers with 
only a limited amount (50 ops/sec) of rpush'es happening, and CPU still 
thrashes.

There are no keys commands, or large unions or anything like that. Any help

in trying to figure out quite what is going on is appreciated.

Thanks,

Damon





Reply
Tags: host osmemorysoft limit
Messages in this thread
Redis high CPU and fork rate on OpenVZ
reply Re: Redis high CPU and fork rate on OpenVZ
(31 lines) Feb 7, 2012 04:05
reply Re: Redis high CPU and fork rate on OpenVZ
(51 lines) Feb 7, 2012 09:55
Memcached 1.2.2:High hit rate but %mem very low
August 19, 2011 01:17:13 PM
Dear Memcached expert I'm using a memcached 1.2.2 which installed in a RHEL5.But I have a problem with the memcahced about a performance.The memcached is high a hit rate (up to 90%) but the %MEM value is low (estimate 3%). I think the…
High ingest rate and FIN_WAIT1 problems
July 16, 2010 09:33:29 AM
Hi, I am a complete HBase and HDFS newbie, so I apologize in advance for the inevitable bloopers. We are doing feasibility testing on NoSql data store options, with rather high ingest rate requirements. So far, HBase is looking good, with only…
Activemq CMS - high data rate lockup
July 15, 2011 02:09:12 PM
Perhaps my search foo is weak, but I have been unable to find any recent examples of people sending high data data rates over activemq CMS. http://www.mostly-useless.com/blog/2007/12/27/playing-with-activemq/ -> has some good data (although…
High latency (40ms) and low request rate (10 rps) under windows
October 25, 2011 02:22:51 PM
Hi! I installed CouchDB 1.1.0 on Windows (Winows 7, 32 bit) and tried to test performance using ab.exe (Apache Benchmark http://httpd.apache.org/docs/2.2/programs/ab.html) ab.exe -k -c 1 -n 1000 -p ab.json -T "application/json"…
high 500 error rate on file metadata operations
January 22, 2011 10:46:40 AM
Howdy, Lately our puppetmasters have been returning 500 errors at a reasonably high rate. On any given day, 200-300 (out of many thousands) of our clients are reporting that they've received a 500 during a run. So far, the only firm reports I've…
High ham rate in darxus corpora for URIBL_WS_SURBL Re: ham scores
September 19, 2011 12:02:05 PM
On 09/17, Mark wrote: > I noticed that the ham rate of WS in the darxus corpus is rather high. > > ruleqa.spamassassin.org/20110916-r1171450-n/URIBL_WS_SURBL/detail > > Could you check what is causing this please? That shows 39 of…
Re: High ham rate in darxus corpora for URIBL_WS_SURBL Re: ham scores
September 19, 2011 04:38:11 PM
On 09/19, Mark wrote: > darx### @chaosreigns.com wrote: > > The other 38 were notifications from livejournal.com, nothing spam > > related, from 2011-08-02 to 2011-08-11. It looks like you just had > > livejournal.com listed as…
Re: redis without fork
March 27, 2011 11:10:37 PM
On Mar 27, 2011, at 20:53 , redis-db+nor### @googlegroups.com wrote: [...] > Javier Guerra Giraldez <javi### @guerrag.com> Mar 27 09:52PM -0500 ^ > > > I'm interested in opinions on whether this is feasible or not. > >…
redis without fork
March 27, 2011 02:36:30 PM
Hello All, I am looking into modifying redis to use threads rather than fork, so that it can run on architectures that don't support fork (iOS, windows). My idea is to implement my own copy on write for all data structures. For example, if copy…
REDIS server fork
December 16, 2010 12:24:11 PM
This issue was discussed multiple times, however no working solution was proposed. When doing background child process for saving data, REDIS server calls fork(). This becomes slower with bigger amount of data that REDIS stores. For example on our…
redis dump fork question
August 11, 2011 02:04:54 PM
Hi, I like to know if Redis when forks for BGSAVE, does it really takes up double the memory? I looked at pmap -x <PID> for redis process and bgsave forked process and both seem to share the same address space (000000000063b000 ) so I am…
timeseries data and lsbarel fork of redis?
July 21, 2011 01:07:24 PM
Hi, I was looking at Redis for timeseries data, and found this for that addresses the lack of a timeseries specific data structure: https://github.com/lsbardel/redis/blob/quantredis/timeseries.rst#tslen I've been using it for several weeks, and…
Looking for a Redis Expert for quick optimization help of very high transaction redis setup
December 30, 2011 12:32:05 PM
Hi All, I am not sure if its a right forum for such queries thus please accept my apologies in advance. I am looking for a redis expert for quick optimization help of our high transaction redis setup.. The setup have 10000+ get/set per sec and…
Some high-level overview of Redis?
April 17, 2011 03:07:47 AM
Interested in knowing the following while starting to get hands dirty with Redis. Could someone on the list give an overview of them or point some ref. links? Thanks! - Scalability ( is it a concern if too many traffic - r/w request will overload…
Redis failover and high availability
November 18, 2010 02:34:09 AM
Hi all While the replication guide is fairly clear on how one can achieve master-slave replication, what is not clear is how one achieves high availability. I wish to confirm my understanding below - * One can setup redis replication to replicate…
Related – A Redis-backed high performance distributed graph database
January 30, 2012 01:53:30 PM
Hi, I've been working on a project using Redis for a while that I think can be useful for other people than me. So I would like to share it and perhaps get some feedback on it. It's a distributed graph database called Related implemented as a Ruby…
Puppet + OpenVZ
August 6, 2010 12:00:10 AM
Hi Everyone, I just wanted to shoot a quick post out there,hope everyone is doing well. I've been searching for people sharing this type of code, and it seemed to be lacking so I decided to jump in. I'm working on developing what would be an…
mongo & openvz
August 31, 2011 07:37:04 AM
Is there currently any fix/patch for running mongodb on openvz? ISSUE - https://jira.mongodb.org/browse/SERVER-1121 - has no new updated (last change Apr 11 2011). Thanks for any info ... - Mitja
OpenVZ support in Virt module
July 21, 2011 11:38:15 AM
Reliant Security funded some new work by Carla Souza that has extended what's available within puppet to manage virtual machines. The Puppet virt module provides the configuration and correct activation of virtual machines with multiple…
DO NOT REPLY Tomcat breaks in serving large files (>=1.8MB) under high load and high threading.
June 2, 2010 02:48:43 PM
https://issues.apache.org/bugzilla/show_bug.cgi?id=48760 --- Comment #21 from Matthew Mullins <mattmull82@gmail.com> 2010-06-02 15:48:37 EDT --- (In reply to comment #20) > This has been fixed in 6.0.x and will be included in 6.0.27…
Diagnosing a Memcache server with a high cache miss rate
March 16, 2011
Hi Guys, What tools are available to debug a memcache server with a high miss rate (62%)? Ideally I would like to know the keys for the top cache…
High IO Rate when writing on disk and blocked SQL INSERT statements
March 23, 2011
Hi there, I've a problem with high IO wait. I've tested some things with the following hardware and software -physical machine (HP ProLiant), Debian…
Running mongodb with openvz
June 18, 2011
Has anyone had any luck getting mongodb working on openvz? Was looking for some tips, tricks, hacks to get it from running out of memory I am…
Is it possible to fork a mysqldump of data?
February 6, 2011
I am restoring a mysql database with perl on a remote server with about 30 million records. It's taking > 2 days & looking at my network…
Fork shell script (not &)
January 27, 2011
I'm accessing a webserver via PHP. I want to update some info in the Apache configs, so I start a shell script that makes the changes. Then I want…
Block OUTGOING port 25 traffic from specific IPs on openvz host
June 8, 2011
I have a server running CentOS 5.5 and acting as an OpenVZ host. One of the containers has been sending out spam and I need to block his ability to…
SQL parser for Wordpress NoSQL Fork
October 10, 2010
Call me crazy but I'm planning to Fork wordpress. I'm planning to swap out MySQL for Apache Cassandra. Call it ambitious but I'm planning to devote…
I'm considering Redis backed web application, but I'm not sure about the hardware requirements for Redis, or really precisely how Redis "works"?
March 4, 2011
I'm new to Redis, as in... I'm really not sure how it works. But I'm considering it for a web app with a relatively uncomplicated data structure…
Why does adding redis to Gemfile fix error with: require 'redis'
March 3, 2011
I have had a problem trying use the redis (2.1.1) gem with rails. I've fixed the problem but I don't know why the fix works and would like to…
Is there any way to get resque-web to work with Redis To Go hosted redis instance?
January 22, 2011
Is there any way to get resque-web to work with a Redis To Go hosted redis instance?
Does anyone know what X-Accel-Limit-Rate really does?
June 13, 2011
These are the docs for X-Accel-Limit-Rate : Sets the rate limit for this single request. Off means unlimited. Not much there. Most of the examples…
Rate limiting in php and memcached
April 25, 2011
I was reading this article: http://simonwillison.net/2009/Jan/7/ratelimitcache/ (which I think is great BTW). I understand his methods and can…
Postfix rate limiting
March 2, 2011
Hi there, I did add a new slow transport to my Postfix configuration but this doesn't looks to work Messages passes correctly in the slow transport…
Very high cost for low limit LIMIT / high OFFSET
January 18, 2011
I'm having very large table with products. I need to select several products at very high offset (example below). Postgresql manual on indexes and…
Acceptable rate of http requests
June 10, 2011
I want to add a feature to my applications which parses an HTML page and jumps out to each <a href="http:\\foo" > in turn. Im gathering…
SQL Query to check rate limit
June 6, 2011
Lets say I have a table of messages that users have sent, each with a timestamp. I want to make a query that will tell me (historically) the most…
Limit apache request rate by IP
January 30, 2011
Is it possible to limit each IP requesting a page from my Apache server to, say, 1 request per second?
A simple rate limiter in php and memcached
May 5, 2011
I'm trying to make a simple rate limiter, based on the comments from here : function set_session_rate_limit($memcache, $name, $user_session, $time)…
After comping redis 2.2.5 redis-stat is not created
April 30, 2011
Hi guy, I wget the latest version of redis 2.2.5, then make, and cd redis-2.2.5/src, but there is no redis-stat found. Why? Could someone tell me…
Postfix delivery rate based on origin
March 2, 2011
How can I rate the outgoing email based on who is trying to send it. For example, if one email address is trying to send many emails or big ones…