Best unofficial Apache Server developers community
Username
Forgot password?
Sign in with Twitter account
Sign in with Facebook account
List archives

Issue 312 in redis: redis-server hanging when being populated with keys-values w/expire with VM enabled

Re: Issue 179 in redis: redis-cli fails after error in multi/exec
(17 lines)
Re: Is there a Redis GUI
(84 lines)
Aug 24, 2010
Redis
Redis
Status: New
Owner: ----
Labels: Type-Defect Priority-Medium

New issue 312 by themig.adx: redis-server hanging when being populated
with  
keys-values w/expire with VM enabled
http://code.google.com/p/redis/issues/detail?id=312

What steps will reproduce the problem?
1. Start redis-server with
    vm-enabled yes
    vm-max-memory 2147483648
    vm-page-size 32
    vm-pages 134217728
    etc
2. From a standalone executable issue commands in a loop that will do the 

following:
    getset random_key random_value
    expire random_key 2592000
3. When the server gets across boundary of its virtual machine (2 G)it  
stops responding to cli and to "redis-stat vmstat" with about 48% CPU busy
 
on our LINUX system, probably falling into a loopwhole of VM swapping.

What is the expected output? What do you see instead?


What version of the product are you using? On what operating system?


Please provide any additional information below.






Reply
Tags: stepsenabled
Similar Threads
Re: Issue 54 in redis: EXPIRE accepts seamingly too large values without complaining
Updates:
	Status: Verified

Comment #8 on issue 54 by antirez: EXPIRE accepts seamingly too large  
values without complaining
http://code.google.com/p/redis/issues/detail?id=54

Fixed in a general way (overflow detection was not good in other places of
 
the code as well, btw the code base changed a lot since this bug and the  
patches was submitted in the first instance).

Thanks for reporting,
Salvatore





Re: Issue 97 in redis: Ubuntu 9.10 /etc/init.d/redis-server [stop|restart] does not force a sync to
Updates:
	Status: Verified

Comment #4 on issue 97 by antirez: Ubuntu 9.10 /etc/init.d/redis-server  
[stop|restart] does not force a sync to disk
http://code.google.com/p/redis/issues/detail?id=97

this is now fixed (as Chris suggested)





Re: Issue 157 in redis: EXPIRE and EXPIREAT resets INCR sequence
Updates:
	Status: Verified

Comment #10 on issue 157 by antirez: EXPIRE and EXPIREAT resets INCR  
sequence
http://code.google.com/p/redis/issues/detail?id=157

Fixed in Redis master, now it's possible to write against volatile keys
it will not get backported to 2.0 so there is to wait the (not so far) 2.2
 
at this point.

Cheers,
Salvatore





Re: Issue 140 in redis: calling setnx on existing key with expire returns 1
Updates:
	Status: Verified

Comment #12 on issue 140 by antirez: calling setnx on existing key with  
expire returns 1
http://code.google.com/p/redis/issues/detail?id=140

The new behavior allows writes against volatile keys, starting from Redis 

2.2 (Redis master) this issue is definitely solved.

Cheers,
Salvatore





Re: Issue 132 in redis: Comment at the top of utils/redis-copy.rb applies to redis-sha1.rb
Updates:
	Status: Done

Comment #1 on issue 132 by antirez: Comment at the top of  
utils/redis-copy.rb applies to redis-sha1.rb
http://code.google.com/p/redis/issues/detail?id=132

done, thanks





Re: Issue 151 in redis: The order of keys
Updates:
	Status: WontFix

Comment #1 on issue 151 by antirez: The order of keys
http://code.google.com/p/redis/issues/detail?id=151

keys will return keys in random order for design. To take ordered elements
 
please take a look at sorted sets.

Cheers,
Salvatore





Issue 311 in redis: speed enhancement to keys *
Status: New
Owner: ----
Labels: Type-Defect Priority-Medium

New issue 311 by dsrthorne: speed enhancement to keys *
http://code.google.com/p/redis/issues/detail?id=311

This is a diff against 1.2.1.  The problem I encountered is that keys *  
isn't optimized and is a very common thing in our environment.  This
change  
sped up the unit tests by 1.5-2.5s, and gave us much more notable benefits
 
in production.  The change involved moving the if (pattern[0] == '*'
&&  
pattern[1] == '\0') outside of the while loop, so it is not evaluated for 

each key or before the other kind of glob pattern matching.

diff --git a/redis.c b/redis.c
index d2491a1..abd165a 100644
--- a/redis.c
+++ b/redis.c
@@ -3243,12 +3243,17 @@ static void keysCommand(redisClient *c) {
      di = dictGetIterator(c->db->dict);
      addReply(c,lenobj);
      decrRefCount(lenobj);
-    while((de = dictNext(di)) != NULL) {
-        robj *keyobj = dictGetEntryKey(de);

-        sds key = keyobj->ptr;
-        if ((pattern[0] == '*' && pattern[1] == '\0') ||
-            stringmatchlen(pattern,plen,key,sdslen(key),0)) {
+    //
+    // match on the general common case before while loop to prevent
+    // this conditional from being run for each key
+    // and/or before any other glob pattern match attempt
+    //
+    if (pattern[0] == '*' && pattern[1] == '\0') {
+        while((de = dictNext(di)) != NULL) {
+            robj *keyobj = dictGetEntryKey(de);
+
+            sds key = keyobj->ptr;
              if (expireIfNeeded(c->db,keyobj) == 0) {
                  if (numkeys != 0)
                      addReply(c,shared.space);
@@ -3258,6 +3263,22 @@ static void keysCommand(redisClient *c) {
              }
          }
      }
+    else {
+        while((de = dictNext(di)) != NULL) {
+            robj *keyobj = dictGetEntryKey(de);
+
+            sds key = keyobj->ptr;
+            if (stringmatchlen(pattern,plen,key,sdslen(key),0)) {
+                if (expireIfNeeded(c->db,keyobj) == 0) {
+                    if (numkeys != 0)
+                        addReply(c,shared.space);
+                    addReply(c,keyobj);
+                    numkeys++;
+                    keyslen += sdslen(key);
+                }
+            }
+        }
+    }
      dictReleaseIterator(di);
      lenobj->ptr = sdscatprintf(sdsempty(),"$%lu\r\n",keyslen+(numkeys
?  
(numkeys-1) : 0));
      addReply(c,shared.crlf);







Issue 310 in redis: Add multiple values to set
Status: New
Owner: ----
Labels: Type-Defect Priority-Medium

New issue 310 by VnVi### @gmail.com: [Feature Request] Add multiple values 

to set
http://code.google.com/p/redis/issues/detail?id=310

I often need to get intersected ids, and then sort those ids. In my app,  
because the ids are stored in different sets and zsets, I can't use SINTER
 
or ZINTERSTORE on them. Instead I get all ids from those sets and zsets
and  
perform the intersection at the application level (using function  
array_intersect() in PHP). But I need to sort the ids in redis (need to  
sort BY a hash field). So I have to add the intersected ids to a temporary
 
set to do a SORT command against it. I use SADD in a loop to accomplish  
this. However from my test using Predis lib, adding 100,000 integer values
 
to a set by using a loop takes 7 seconds, which is surely slow. I think  
there should be a way to add multiple values to a set in one go.

I test setting 100,000 keys/values with MSET (also using Predis) and it  
only takes about 1 second. So I think a MSADD or something like that would
 
be just about as fast. And while we can set multiple string values I think
 
it's logical to have a command to add multiple values to a set. Beside my 

case, I can think of a number of other cases where this command is useful.

Thanks





Re: Issue 167 in redis: Dissapointing responsiveness of Redis when using append only file mode
Comment #2 on issue 167 by antirez: Dissapointing responsiveness of Redis 

when using append only file mode
http://code.google.com/p/redis/issues/detail?id=167

updates about this:

1) With the current Linux kernel it is not possible to flush on a
different  
thread, as write(2) will block anyway in the main thread. This sucks but I
 
don't think this is going to get fixed in little time.
2) We have now in Redis master an option so that fsync(2) is not called  
when there is a background saving/log-rewrite operation in progress. It's
a  
trick... but works.
3) All this is highly dependent on the file system used and the mount  
options.
4) "fsync none" is a trivial way to completely fix this problem but the  
drawback is that up to 30 seconds of logs can be lost.
5) fsync always is now optimized, it is still very very slow but much  
faster than before.

I'm leaving this open as it's an open problem but I don't think there are 

very good way to fix this at the moment, still with the latest changes we 

mitigated the problem enough. Currently when very low latency is required
a  
two box setup with a saving slave may be the best option.





Re: Issue 106 in redis: Incorrect parsing of redis.conf save parameters
Updates:
	Status: Accepted

Comment #1 on issue 106 by antirez: Incorrect parsing of redis.conf save  
parameters
http://code.google.com/p/redis/issues/detail?id=106

yep not cool at all, accepted. I've some plan to rewrite most of the  
configuration file parsers and for sure sanity checks including parameters
 
types and limits will be supported as well.





Issue 300 in redis: redis-check-dump shows an overflowed value with huge .rdb.
Status: New
Owner: ----
Labels: Type-Defect Priority-Medium

New issue 300 by hatemogi: redis-check-dump shows an overflowed value with
 
huge .rdb.
http://code.google.com/p/redis/issues/detail?id=300

What steps will reproduce the problem?
1. make a huge dump.rdb file.
2. run redis-check-dump with that file.
3. I see an overflowed value in the result.

What is the expected output? What do you see instead?

I see below

$ ls -la data/dump.rdb
-rw-rw-r--  1 dante dante 11853271020 Aug  5 15:24 data/dump.rdb
$ ./redis-check-dump data/dump.rdb
==== Processed 9971202 valid opcodes (in -1031630877 bytes)  

Re: Issue 133 in redis: SUNIONSTORE and SINTERSTORE produce different key values for non-existant ke
Updates:
	Status: Verified

Comment #3 on issue 133 by antirez: SUNIONSTORE and SINTERSTORE produce  
different key values for non-existant keys
http://code.google.com/p/redis/issues/detail?id=133

This is now fixed, thanks.

Cheers,
Salvatore





Re: Issue 137 in redis: redis.php: rpop and lpop method should not have $value parameter
Updates:
	Status: WontFix

Comment #1 on issue 137 by antirez: redis.php: rpop and lpop method should
 
not have $value parameter
http://code.google.com/p/redis/issues/detail?id=137

please use Predis or Rediska PHP Redis libs! The old PHP lib is
deprecated.

Cheers,
Salvatore





Issue 296 in redis: redis-cli does not handle EOF in monitor mode correctly
Status: New
Owner: ----
Labels: Type-Defect Priority-Medium

New issue 296 by bjarni.runar: redis-cli does not handle EOF in monitor  
mode correctly
http://code.google.com/p/redis/issues/detail?id=296

What steps will reproduce the problem?
1. run redis-cli in monitor mode
2. killall redis-server

What is the expected output? What do you see instead?

redis-cli should exit, as the server is no longer running. It does not,  
instead it spews infinite newlines.


What version of the product are you using? On what operating system?

redis 2.0.0rc3


Please provide any additional information below.

Patch attached, fixes the issue.

Attachments:
	redis-cli-newlines.patch  798 bytes





Re: Issue 179 in redis: redis-cli fails after error in multi/exec
Updates:
	Status: Verified

Comment #1 on issue 179 by antirez: redis-cli fails after error in  
multi/exec
http://code.google.com/p/redis/issues/detail?id=179

Fixed in redis master,

Cheers,
Salvatore





Re: Issue 168 in redis: Feature Request] redis-cli needs a auth-option
Updates:
	Status: Verified

Comment #3 on issue 168 by antirez: Feature Request] redis-cli needs a  
auth-option
http://code.google.com/p/redis/issues/detail?id=168

This was fixed indeed (see the -a option of redis-cli)





Re: Issue 111 in redis: read-only redis.conf option
Comment #1 on issue 111 by antirez: [FEATURE REQUEST] read-only redis.conf
 
option
http://code.google.com/p/redis/issues/detail?id=111

not a bad idea indeed... I'll think a bit more about it but sounds cool





Re: Issue 162 in redis: need a redis-cli command to close all connection
Comment #1 on issue 162 by antirez: need a redis-cli command to close all 

connection
http://code.google.com/p/redis/issues/detail?id=162

can't find a valid use case for this ;) Care to explain?

Cheers,
Salvatore





Re: Issue 108 in redis: PHPRedis fails for values longer than 1024 characters
Updates:
	Status: Invalid

Comment #2 on issue 108 by antirez: PHPRedis fails for values longer than 

1024 characters
http://code.google.com/p/redis/issues/detail?id=108

sorry here we handle only problems related to the Redis server itself. Btw
 
the way to go is using a modern PHP client lib, that is Predis or Rediska.

Cheers,
Salvatore





Re: Issue 145 in redis: redis-1.2.0 Makefile fixes
Updates:
	Status: WontFix

Comment #2 on issue 145 by antirez: [patch] redis-1.2.0 Makefile fixes
http://code.google.com/p/redis/issues/detail?id=145

(No comment was entered for this change.)