Capped collection Vs Normal collection

If you had one database per month you could just move the datafiles
for that month to archival storage at the end of the month.

On Wed, Aug 18, 2010 at 6:54 AM, ram <rams.aw### @gmail.com> wrote:
 Hi,

 I am using mongo-db for realtime analytics, I am getting 2 million
 data points per day which will increase in future to 5 million at
 least. we require to have at least 1 month data online then we will
 move it to archive. Presently I am running on single machine No
 sharding and replication.

 Now my question is that:
 Capped collections are circular so my data will be overwritten after
 it reaches to the cap and
 Normal collections do not release disk space even if we rename and
 drop the collection.

 So what approach I should follow?  I am thinking to have capped
 collection and take dump of data in CSV format before it reaches to
 the cap.

 Or having normal collection and rename it on 1st day of month  but
 this will not put data offline.


 Regards,
 Ram


 --
 

CollectionUtils.removeAll(Collection collection, Collection remove) calls
ListUtils.retainAll(collection, remove)

db.createCollection("mycoll", {capped:true, size:100000})
db['mycoll'].insert({user:1})
so, I want to know after insert, the mycoll collection used size, how
to? or, the insert size.

I have an application that I'm developing that uses some capped
collections (that are created dynamically) to store some temporary
data from a larger data source so a client can poll the capped
collection for new records.

My question is this: As a client polls for new records from the
collection how should it determine which record to start from on the
next polling request? Obviously, polling on a timestamp type of column
is normally used. Currently I index on the _id field and figured I'd
just request the next batch that is { $gt => $prev_id }. Is this the
best way to do it? I ask because capped collections don't normally
have any indexes (not even on _id) so I wanted to see if there was a
more efficient manner to do this. I want the capped collection to be
as fast as possible.

Thanks.

There was a discussion last year about using a collection for each
user's data

http://groups.google.com/group/mongod.../e0254fa52339b4a4

After a year of doing this, I was wondering if Nicolas can share some
insights on experiences using this approach.

Thanks.

Hi,

I just upgraded to 1.6 and noticed that db.stats() is now showing one
extra collection in a specific database than in 1.4.4.  Not a big
deal, but I was wondering if you could tell me why.

In the following simple example, I insert one collection into a new
database.  'show collections' only shows two collections, but db.stats
is showing 3.

MongoDB shell version: 1.6.0
connecting to: localhost:27017/admin
 use mydb
switched to db mydb
 show collections
 db.stats()
{
	"collections" : 0,
	"objects" : 0,
	"avgObjSize" : NaN,
	"dataSize" : 0,
	"storageSize" : 0,
	"numExtents" : 0,
	"indexes" : 0,
	"indexSize" : 0,
	"fileSize" : 0,
	"ok" : 1
}
 j = { name : "mongo" };
{ "name" : "mongo" }
 db.things.save(j);
 show collections
system.indexes
things
 db.stats()
{
	"collections" : 3,
	"objects" : 5,
	"avgObjSize" : 40,
	"dataSize" : 200,
	"storageSize" : 7424,
	"numExtents" : 3,
	"indexes" : 1,
	"indexSize" : 8192,
	"fileSize" : 16777216,
	"ok" : 1
}

Thanks!

db.capped_table.exists() function can show the options of a capped
collection

 db.mycollection.exists();
{
        "name" : "my.capped_table",
        "options" : {
                "create" : "capped_table",
                "capped" : true,
                "size" : 1024000,
                "max" : 10
        }
}

but when convertToCapped function or command is used, the 'max' and
'create' options are lost.

So, how can i change the collection options, such as max?

Hi All,

I been trying mapReduce and I'm currently stuck because the resulting
collection looks like:

{ _id: { MY_KEY_STUFF }, value: { MY_VALUE_STUFF } }

But it happens that I would like to have only the contents of
MY_VALUE_STUFF
in my permanent collection, because I want it easy to index by a couple of
fields in there and it makes it diffcult by having them embedded.

I tried the finalize function but it doesn't seem to have tha purpose as
my
resulting collection looks the same.

Thanks in advance
Kevin

Every so often I see intances of ModSecurity set a session variable and
then
not be able to retrieve the variable later on. In other words, I have a
rule
that creates a SESSION.username variable, which gets set as follows:

[08/Jul/2010:09:30:24 +1000]
[xxx.xxx.xxx/sid#8fb6de7][rid#cfa8c10][/index.asp][9] Set variable
"SESSION.username" to "123456".

but sometimes (and only sometimes) when I try to use the SESSION.username
variable the variable does not resolve to a value. When this happens I
don't
see "Resolved macro %SESSION.username" in the logs, but I do see the "Set
variable "SESSION.username to ..." which tells me that the variable is
set,
but sometimes seems to disappear. Has anyone experienced this before?

Thanks,
Chris

Hi,

I'm using Tomcat 6.0.26, Java 1.6 and wondering what tools/strategies you
use to tune your garbage collection parameters?

Further, does anyone know how to read entries in the garbage collection
log? 
Entries in my log look like

Desired survivor size 10944512 bytes, new threshold 1 (max 15)
 [PSYoungGen: 129311K->3232K(136512K)] 558882K->434085K(585920K),
0.0090900
secs]

Thanks, - Dave

Hi -

First-time poster, new to Mongo, so apologies if this is a n00b
question.

Is there an efficient way to import a whole collection into another,
e.g. to aggregate several existing collections into one? Just now I
have a find() on the source collection, followed by iterating the
cursor and inserting each document into the destination collection.
This works but feels inefficient and brute force. Maybe I'm splitting
hairs.

I happen to be using the PHP drivers, but happy to hear general
approaches.

Best regards,
Simon

On 24/06/2010 17:41, Finn Bock wrote:
 2010/6/24 Adrian A:
> What if we make OptionGroup extend Option?
> Wouldn't that simplify the problem?
 
 Indeed it would, but with a tiny risk of breaking existing user code
like:

I don't think we need to enforce inheritance onto Option and OptionGroup.
The issue at hand is minor
at best, no need to complicate the framework.

Bob

Is there a way to set a collection variable from a Lua script? Perhaps
something like m.setvar. Alternatively, can I return the value I want to
set
in a collection value to modsec and use the rules language to set a
collection variable?

Thanks

I am missing something but I am not sure what.
I want to fill a list of strings.
Currently, the SimpleDomain does return a name with a null list.

Java:
public class SimpleDomain
{
    String name;
    List<String> someStrings;
    //getters and setters here
}

Mapper:
<select id="MySuperQuery" resultMap="MyMap">
     select tbl1.lname as NAME, tlb2.lstring as MYSTRING
        from tbl1
        left outer join tbl2 on tbl1.id = tbl2.id
</select>

<resultMap id="MyMap" type="SimpleDomain">
  <result property="name" column="NAME"/>
  <collection property="someStrings" javaType="list" column="MYSTRING"
ofType="string"/>
</resultMap>

Thank you.

Use typesafe collection

Hi,

I have this class generated by wsdl2java (comments remove):
@XmlAccessorType(XmlAccessType.FIELD)
@XmlType(name = "RunExtensionArgumentsType", propOrder = {
     "extensionArgument"
})
public class RunExtensionArgumentsType
     implements Serializable
{

     @XmlElement(name = "ExtensionArgument")
     protected List<RunExtensionType> extensionArgument;

     public List<RunExtensionType> getExtensionArgument() {
         if (extensionArgument == null) {
             extensionArgument = new ArrayList<RunExtensionType>();
         }
         return this.extensionArgument;
     }

}


I have been serialising it for ages using the following code:
     static {
         aegisContext.initialize();

         TypeMapping tm = aegisContext.getTypeMapping();
         Type aegisType = tm.getType( RunExtensionArgumentsType.class );
         if( aegisType == null )
         {
             // Make a new type and register it for the future.
             log.debug( "Registering type for RunExtensionArgumentsType"
);
             TypeCreator tc = tm.getTypeCreator();
             aegisType = tc.createType( RunExtensionArgumentsType.class );

             QName qName = new QName( "http://groupgti.com/s-service", 
"RunExtensionType");
             tm.register( RunExtensionArgumentsType.class, qName, 
aegisType );
         }
     }

     public static String extensionArgsToString( 
RunExtensionArgumentsType extensionArgs )
     {
         try
         {
             StringWriter stringWriter = new StringWriter();
             XMLStreamWriter xmlWriter = 
xmlOutputFactory.createXMLStreamWriter( stringWriter );

             AegisWriter<XMLStreamWriter> writer = 
aegisContext.createXMLStreamWriter();

             TypeMapping tm = aegisContext.getTypeMapping();
             Type aegisType = tm.getType( RunExtensionArgumentsType.class
);
             writer.write( extensionArgs, aegisType.getSchemaType(), 
true, xmlWriter, aegisType );

             xmlWriter.close();
             return stringWriter.toString();
         }
         catch( Exception ex )
         {
             log.error( ExceptionStringConverter.ExceptionWithCauses( ex 
) );
             return null;
         }
     }

Now I come to deserialize it using this:
     public static RunExtensionArgumentsType extensionArgsFromString( 
String extensionArgsString )
     {
         try
         {
             StringReader stringReader = new StringReader( 
extensionArgsString );
             XMLStreamReader xmlReader = 
xmlInputFactory.createXMLStreamReader( stringReader );

             AegisReader<XMLStreamReader> reader = 
aegisContext.createXMLStreamReader();

             TypeMapping tm = aegisContext.getTypeMapping();
             Type aegisType = tm.getType( RunExtensionArgumentsType.class
);
             Object result = reader.read( xmlReader, aegisType );

             xmlReader.close();
             return (RunExtensionArgumentsType)result;
         }
         catch( Exception ex )
         {
             StringBuilder msg = new StringBuilder();
             msg.append( "Exception whilst trying to convert \" 
").append( extensionArgsString ).append( "\" to 
RunExtensionArgumentsType:\n\t" );
             log.error( ExceptionStringConverter.ExceptionWithCauses( 
msg, ex ) );
             return null;
         }
     }

but it hits this exception:
         org.apache.cxf.aegis.DatabindingException:
  No write method for property 
{http://groupgti.com/s-service}extensionArgument in class 
com.groupgti.s_service.RunExtensionArgumentsType:
  No write method for property 
{http://groupgti.com/s-service}extensionArgument in class 
com.groupgti.s_service.RunExtensionArgumentsType:
  org.apache.cxf.aegis.type.basic.BeanType:writeProperty @
BeanType.java:272
  org.apache.cxf.aegis.type.basic.BeanType:readObject @ BeanType.java:163
  org.apache.cxf.aegis.AegisXMLStreamDataReader:read @ 
AegisXMLStreamDataReader.java:82
  org.apache.cxf.aegis.AegisXMLStreamDataReader:read @ 
AegisXMLStreamDataReader.java:31
 
com.groupgti.services.service.XmlConversionSupport:extensionArgsFromString
@ XmlConversionSupport.java:162
  com.groupgti.services.XmlConversionsTest:testStringToExtensionArgs @ 
XmlConversionsTest.java:65


Now clearly this class can be deserialized 'cos it's used in by the web 
service, so what do I have to do to make it use a get().add() instead of 
a set() for collections?

Thanks

Jim

Hi. I'm a beginner with MyBatis.

I just want to know how to insert a collection of objects from an
instance of a class. Say I have a class User related to a Note in one-
to-many relationship. I just like to mention that I built my schema
using JPA 2 annotations via Hibernate's hbm2ddl. I'll add the key JPA
annotations I used in the sample code below.

Here's a sample:

@Entity
public class User {
    ...
    @OneToMany
    @JoinColumn(name="user")
    public List<Note> getNotes() {...}
    ...
}

Now, everytime I insert something into User table I have to insert
instances into the Note table if the list is not empty. Take note of
the @JoinColumn in Note table which should have the id of the inserted
User, which I have set to be autogenerated.

Has anyone got something like this working? Thanks.

Hi all!

I'm trying to understand the correct use of collections mapping of the
jackrabbit OCM.

	@Collection(proxy=true, 
			  elementClassName=SomeClass.class,
collectionConverter=NTCollectionConverterImpl.class)

I found something like this but I really don't understand what's going
on with this.

Is there any good explanation about how to use? The main page is almost
empty...

Thank you.

Hi,

Is there a way to integrate the JAXB 2.1 Collection setter injection
plugin[1] to cxf-codegen-plugin ?

Regards.

[1] https://jaxb2-commons.dev.java.net/co...-setter-injector/

This is a multi-part message in MIME format.
Hi, all:

   what's the Succinctest way of using Complex Collection Properties ?  

   As we know ,when we do a 1:1 query , a bean has a complex bean property
for instance a Product has a Category property, we can write the select
statement query a product and its category like below:

    <select id="getProductWithCategory"  resultClass="product" 
parameterClass="int">
        select
           p.prd_id pid,
           p.prd_name name,
           p.prd_description description,
           c.cat_id as "category.cid",
           c.cat_name as "category.name"
        from product p,category c
         where p.prd_id=#pid# and p.cat_id=c.cat_id
    </select>
 
    that is , we don't need do as guide doc says, define two resultMap for
that, just use dot referring to the relationship. How succinct , graceful
it is!  But how about Complex Collection Properties  , how about a Category
has a list property referring to all the products related?
    To query a category and its product list , I have to do fully
according to what guide says,define two resultMap as below, it does work,
but a little boring. Is there better , succint way to it?

 <typeAlias alias="product" type="xdg.model.Product"/>
    <typeAlias alias="category" type="xdg.model.Category"/>
   <resultMap id="productList" class="product">
       <result property="pid" column="prd_id"/>
       <result property="name" column="prd_name"/>
   </resultMap>
   <resultMap id="categoryResult" class="category" groupBy="cid">
       <result property="cid" column="cat_id"/>
       <result property="name"  column="cat_name"/>
       <result property="products"
resultMap="category.productList"/>
   </resultMap>
    <select  id="getCategoryList" resultClass="category">
        select *  
        from category c,product p
        where c.cat_id=p.cat_id
        order by c.cat_id;
    </select>


  thank so much.

Kurt
2010-08-14 



fxbird