Best unofficial Apache Server developers community
Username
Forgot password?
Sign in with Twitter account
Sign in with Facebook account
List archives

New to redis , using it to store feed

read operations have lots of small mallocs - compounds memory fragmentation
(28 lines)
redis.org
(27 lines)
Sep 2, 2010
Drkiran
Drkiran
Hi ,

 I am learning redis , Facing a problem when ever i restart redis
server the incr("global:uid") or incr("global:feed")
 started from 1 . I lost my previous keys .

 please help ,

 Thanks





Reply
Tags: redishelp
Messages in this thread
New to redis , using it to store feed
reply Re: New to redis , using it to store feed
(24 lines) Sep 2, 2010 04:44
Similar Threads
Re: Issue 280 in redis: Add a `STORE dstkey` option to all commands returning a result
Comment #7 on issue 280 by antirez: [Feature Request] Add a `STORE dstkey`
 
option to all commands returning a result
http://code.google.com/p/redis/issues/detail?id=280

I need to think at this better. I don't like the general idea of ... meta 

commands, because it is somewhat a step towards an ad-hoc hill conceived  
scripting language. But anyway let's talk theoretically :) In order to  
implement such a thing without breaking the command semantics or without  
changing a lot of code something like this is needed instead:

STORE foo
LRANGE mylist 0 10

so it should be a command that sets the state to "let's store the next  
command output there".

Also in order to make this possible, internally Redis should be able to  
target the format of the object and not the protocol. This means that  
addreply() family functions should be check some client flag to understand
 
if what to output is the protocol or a Redis object.

Also... I'm not sure that it is possible to simply output the same type as
 
the input type... think at SORT for instance, and in general to all the  
things that will go from unordered to ordered outputs and vice versa. Or
to  
sorted sets operations where the score is lost.

Btw such a feature will hardly be incorporated in such a way :(

This should be IMHO part of the scripting engine that is getting more and 

more clear in my mind (but is still far... clustering is the first
priority  
after 2.2), and will be the subject of the next weekly update.

Taking this issue open for a few weeks, it's an interesting issue and I  
would like to get some feedback.

Cheers,
Salvatore





CouchDB with RSS or Atom feed
Can CouchDB be configured to return RSS or Atom XML for HTML GETs?

Nathan


Re: expected behaviour of changes feed
Mikeal & Sebastian

tks for explanation ......... all clear in my head now.

J Chris

That's just perfect for validation. Appreciate the link.

Thank you all for your help

very best regards

Cliff

On 07/08/10 18:41, J Chris Anderson wrote:
 For your use case you might look at these versioning techniques:


http://blog.couch.io/post/632718824/s...ning-with-couchdb


 On Aug 7, 2010, at 9:55 AM, Mikeal Rogers wrote:

    
> the changes feed only shows the update sequences for the current
documents
> in the database. it doesn't show sequences for old revisions of
documents
> because they will go away after compaction anyway.
>
> every time a document is updated it removes it's previous
sequence from the
> by-sequence index and the revision information is only available
by querying
> for it on the document itself (until you compact).
>
> -Mikeal
>
> On Sat, Aug 7, 2010 at 7:56 AM, Cliff
Williams<cliff### @aol.com>  wrote:
>
>      
>> All,
>>
>> Good afternoon.
>>
>> using this url
>>
http://10.0.0.10:5984/test/_changes?s...amp;all_docs=true
>>
>> I get the following results
>>
>> {"results":[
>>
>>
{"seq":14971,"id":"ab53d85b92247bcbf251ef9abed4d998","changes":[{"rev":"2-b9cfa00cc907767d6bfdeca7dee5008f"}]},
>>
>>
{"seq":14973,"id":"ab53d85b92247bcbf251ef9abed4d512","changes":[{"rev":"3-ec350b40276b7c16c09ef485dd8f14aa"}]},
>>
>>
{"seq":14981,"id":"ab53d85b92247bcbf251ef9abed4a63a","changes":[{"rev":"2-fe0cb06e7046e9d41d326b3e8c30ed0c"}]},
>>
>>
{"seq":14982,"id":"_design/test","changes":[{"rev":"8-5a565b32bca9b42479543441f23a2e61"}]},
>>
>>
{"seq":14983,"id":"ab53d85b92247bcbf251ef9abed4b4d8","changes":[{"rev":"2-ed899403c21b6cca22f7e68f05d92b0c"}]},
>>
>>
{"seq":14984,"id":"ab53d85b92247bcbf251ef9abed47f2f","changes":[{"rev":"2-30d5cf6764f2e6c1795d65f272817238"}]},
>>
>>
{"seq":15006,"id":"ab53d85b92247bcbf251ef9abed4e544","changes":[{"rev":"10-e6856aac928ff1e2e019b556de8f1760"}]},
>>
>>
{"seq":15007,"id":"ab53d85b92247bcbf251ef9abed49e50","changes":[{"rev":"19-d7629eac5a5bd2133a19178fdc9277e6"}]}
>> ],
>> "last_seq":15007}
>>
>> now if I make an amendment to the document at sequence number
15006 and
>> using the same URL I get the following results.
>>
>> {"results":[
>>
>>
{"seq":14971,"id":"ab53d85b92247bcbf251ef9abed4d998","changes":[{"rev":"2-b9cfa00cc907767d6bfdeca7dee5008f"}]},
>>
>>
{"seq":14973,"id":"ab53d85b92247bcbf251ef9abed4d512","changes":[{"rev":"3-ec350b40276b7c16c09ef485dd8f14aa"}]},
>>
>>
{"seq":14981,"id":"ab53d85b92247bcbf251ef9abed4a63a","changes":[{"rev":"2-fe0cb06e7046e9d41d326b3e8c30ed0c"}]},
>>
>>
{"seq":14982,"id":"_design/test","changes":[{"rev":"8-5a565b32bca9b42479543441f23a2e61"}]},
>>
>>
{"seq":14983,"id":"ab53d85b92247bcbf251ef9abed4b4d8","changes":[{"rev":"2-ed899403c21b6cca22f7e68f05d92b0c"}]},
>>
>>
{"seq":14984,"id":"ab53d85b92247bcbf251ef9abed47f2f","changes":[{"rev":"2-30d5cf6764f2e6c1795d65f272817238"}]},
>>
>>
{"seq":15007,"id":"ab53d85b92247bcbf251ef9abed49e50","changes":[{"rev":"19-d7629eac5a5bd2133a19178fdc9277e6"}]},
>>
>>
{"seq":15008,"id":"ab53d85b92247bcbf251ef9abed4e544","changes":[{"rev":"11-e01ef09eef56397e345f598fd2069f22"}]}
>> ],
>> "last_seq":15008}
>>
>>
>> I think that it is fairly straight forward to ascertain that
we are looking
>> at latest revision sequences.
>>
>> Is there a way to see all sequence numbers?? I am of course
aware of the
>> continuous feed but do not wish to use it.
>> My use case is for a pharma regulated clinical trials
database where i need
>> a total audit of every single change made to the database.
>>
>> best regards
>>
>> Cliff
>>
>>        
    


georss atom feed
Hi there - I'm looking to find ways to get CouchDB to serve georss (in this
case, a point with latitude and longitude).

I'm taking sofa as my starting point, because it has atom feed capability
built in.

So, sofa can generate this:
http://mick.couchone.com/blog/_design...0&format=atom

In the couchapp files on my local machine I'm looking to amend
sofa/lists/index.js so that the feed entry includes georss:point as
specified here:
http://www.georss.org/simple#Point

Currently the relevant part of sofa/lists/index.js looks like this:

        // generate the entry for this row
        var feedEntry = Atom.entry({
          entry_id :
path.absolute('/'+encodeURIComponent(req.info.db_name)+'/'+encodeURIComponent(
row.id)),
          title : row.value.title,
          content : html,
          updated : new Date(row.value.created_at),
          author : row.value.author,
          alternate : path.absolute(path.show('post', row.id))
        });

Has anyone out there done this already? Specifically I wonder how to
handle
the fact of georss:point containing a colon and getting that to work in
the
js file.

Also - apologies if this is a dumb question - is it simply a matter of
including latitude and longitude in my couchdb database, then
concatenating
these at the correct position in this js file -

georss:point : row.value.latitude + ' ' + row.value.longitude

- or are there other things I need to consider before referencing these db
elements?

Thanks!

Michael


Re: Issue 132 in redis: Comment at the top of utils/redis-copy.rb applies to redis-sha1.rb
Updates:
	Status: Done

Comment #1 on issue 132 by antirez: Comment at the top of  
utils/redis-copy.rb applies to redis-sha1.rb
http://code.google.com/p/redis/issues/detail?id=132

done, thanks





Re: Issue 260 in redis: r.keys() from redis-py v1.34.1 failing against redis 2.0.0-rc1
Updates:
	Status: Done

Comment #6 on issue 260 by antirez: r.keys() from redis-py v1.34.1 failing
 
against redis 2.0.0-rc1
http://code.google.com/p/redis/issues/detail?id=260

(No comment was entered for this change.)





Re: Issue 212 in redis: Interactive redis-cli crashes when issuing a command after period of inactiv
Updates:
	Status: Verified

Comment #3 on issue 212 by pcnoordhuis: Interactive redis-cli crashes when
 
issuing a command after period of inactivity
http://code.google.com/p/redis/issues/detail?id=212

Reconnecting when the connection has timed out in interactive mode is
fixed  
in both 2.0 and master.





Re: Issue 303 in redis: Disk space full crashes redis-server (upon restart)
Comment #1 on issue 303 by antirez: Disk space full crashes redis-server  
(upon restart)
http://code.google.com/p/redis/issues/detail?id=303

Accepted, thanks for reporting, It is unlikely this will be fixed in 1.2
as  
2.0 is going to be released.

I'll fix the issue and report here asap.

Cheers,
Salvatore





Issue 300 in redis: redis-check-dump shows an overflowed value with huge .rdb.
Status: New
Owner: ----
Labels: Type-Defect Priority-Medium

New issue 300 by hatemogi: redis-check-dump shows an overflowed value with
 
huge .rdb.
http://code.google.com/p/redis/issues/detail?id=300

What steps will reproduce the problem?
1. make a huge dump.rdb file.
2. run redis-check-dump with that file.
3. I see an overflowed value in the result.

What is the expected output? What do you see instead?

I see below

$ ls -la data/dump.rdb
-rw-rw-r--  1 dante dante 11853271020 Aug  5 15:24 data/dump.rdb
$ ./redis-check-dump data/dump.rdb
==== Processed 9971202 valid opcodes (in -1031630877 bytes)  

Re: Issue 202 in redis: Redis does not include man pages in the upstream distribution (Github)
Updates:
	Status: Accepted

Comment #1 on issue 202 by antirez: Redis does not include man pages in
the  
upstream distribution (Github)
http://code.google.com/p/redis/issues/detail?id=202

Ok we should actually ping the author of this man pages and ask if we can 

include this in the Redis distribution... but then there is the problem of
 
a complex "make install" target that we are currently avoiding at all.

Anyway at least including the man pages makes sense. Accepted.

Cheers,
Salvatore





Re: Issue 167 in redis: Dissapointing responsiveness of Redis when using append only file mode
Comment #2 on issue 167 by antirez: Dissapointing responsiveness of Redis 

when using append only file mode
http://code.google.com/p/redis/issues/detail?id=167

updates about this:

1) With the current Linux kernel it is not possible to flush on a
different  
thread, as write(2) will block anyway in the main thread. This sucks but I
 
don't think this is going to get fixed in little time.
2) We have now in Redis master an option so that fsync(2) is not called  
when there is a background saving/log-rewrite operation in progress. It's
a  
trick... but works.
3) All this is highly dependent on the file system used and the mount  
options.
4) "fsync none" is a trivial way to completely fix this problem but the  
drawback is that up to 30 seconds of logs can be lost.
5) fsync always is now optimized, it is still very very slow but much  
faster than before.

I'm leaving this open as it's an open problem but I don't think there are 

very good way to fix this at the moment, still with the latest changes we 

mitigated the problem enough. Currently when very low latency is required
a  
two box setup with a saving slave may be the best option.





Re: Issue 226 in redis: strsep not found on solaris - needed for redis-cli.c to compile.
Updates:
	Status: Verified

Comment #8 on issue 226 by antirez: strsep not found on solaris - needed  
for redis-cli.c to compile.
http://code.google.com/p/redis/issues/detail?id=226

(No comment was entered for this change.)





Re: Issue 294 in redis: [redis-cli] faulty commands mess subsequent reponse
Updates:
	Status: Verified

Comment #2 on issue 294 by antirez: [BUG][redis-cli] faulty commands mess 

subsequent reponse
http://code.google.com/p/redis/issues/detail?id=294

(No comment was entered for this change.)





Re: Issue 225 in redis: redis-cli REPL loop 'wrong number of arguments'
Updates:
	Status: Verified

Comment #2 on issue 225 by antirez: redis-cli REPL loop 'wrong number of  
arguments'
http://code.google.com/p/redis/issues/detail?id=225

This is now fixed.





Re: Issue 235 in redis: redis server not starting on linux redhat RHEL 3
Updates:
	Status: Started

Comment #1 on issue 235 by antirez: redis server not starting on linux  
redhat RHEL 3
http://code.google.com/p/redis/issues/detail?id=235

Hello, this bug sounds very strange! Can you please provide the redis.conf
 
used?

Cheers,
Salvatore





Re: Issue 106 in redis: Incorrect parsing of redis.conf save parameters
Updates:
	Status: Accepted

Comment #1 on issue 106 by antirez: Incorrect parsing of redis.conf save  
parameters
http://code.google.com/p/redis/issues/detail?id=106

yep not cool at all, accepted. I've some plan to rewrite most of the  
configuration file parsers and for sure sanity checks including parameters
 
types and limits will be supported as well.





Re: Issue 97 in redis: Ubuntu 9.10 /etc/init.d/redis-server [stop|restart] does not force a sync to
Updates:
	Status: Verified

Comment #4 on issue 97 by antirez: Ubuntu 9.10 /etc/init.d/redis-server  
[stop|restart] does not force a sync to disk
http://code.google.com/p/redis/issues/detail?id=97

this is now fixed (as Chris suggested)





Issue 312 in redis: redis-server hanging when being populated with keys-values w/expire with VM enab
Status: New
Owner: ----
Labels: Type-Defect Priority-Medium

New issue 312 by themig.adx: redis-server hanging when being populated
with  
keys-values w/expire with VM enabled
http://code.google.com/p/redis/issues/detail?id=312

What steps will reproduce the problem?
1. Start redis-server with
    vm-enabled yes
    vm-max-memory 2147483648
    vm-page-size 32
    vm-pages 134217728
    etc
2. From a standalone executable issue commands in a loop that will do the 

following:
    getset random_key random_value
    expire random_key 2592000
3. When the server gets across boundary of its virtual machine (2 G)it  
stops responding to cli and to "redis-stat vmstat" with about 48% CPU busy
 
on our LINUX system, probably falling into a loopwhole of VM swapping.

What is the expected output? What do you see instead?


What version of the product are you using? On what operating system?


Please provide any additional information below.