Re: performance tuning - where does the slowness come from?

Hi Gurus,

Any advise on initial tuning values for apache MinSpareServers,
MaxSpareServers, and StartServers, tcp tunings, ulimits, sysctl....

I'm running svn 1.6 over apache2 pre-fork. System load goes high as
much as 10 during heavy usage.

What would be your recommended MaxRequestsPerChild for heavy svn usage?

Thanks,


West

Hi,

I'm using Tomcat 6.0.26, Java 1.6 and wondering what tools/strategies you
use to tune your garbage collection parameters?

Further, does anyone know how to read entries in the garbage collection
log? 
Entries in my log look like

Desired survivor size 10944512 bytes, new threshold 1 (max 15)
 [PSYoungGen: 129311K->3232K(136512K)] 558882K->434085K(585920K),
0.0090900
secs]

Thanks, - Dave

Hi All,

I noticed using Firefox with a plugin yslow that it recommends using only
4 virtual hosts instead of the 16 I have for serving images. I am using a 6
core system and want to make sure I take advantage of it using Apache APR
so I setup more than 4 virtual hosts.

I am using multiple virtual hosts for serving images since I heard that
can improve user response time where they have browsers with 4 or more
concurrent connections supported. I have a screenshots page where I have 24
thumbnails being loaded into the browser and I am trying to tune APR for
the best user response time.

So any best practice for this you all would recommend?

Thanks,
-Tony


      

Oleg..

*       Here is the wire error and context debug.
o       From my end I made 3 service calls, (which translates into 4 total
calls because the very first call is to create a security token to be
reused on subsequent requests).
o       Before I start asking questions let me know what you are seeing in
here and your thoughts.
*       Also, in a separate exercise I changed over from httpclient to
httpcore.
o       That made almost no difference in response times.
o       Prior to the changeover I had upgraded to httpclient from trunk
and had disabled a bunch of interceptors (cookies, proxy, authcache) as
well as turned off the stale checking so I think httpclient was running
about as thin as I could make it.
o       Definitely did not see a 40% increase in performance tho..
o       It is reusing the connection.. (before you ask)  :)


2010-07-20 01:20:40,235 DEBUG tsccm.ThreadSafeClientConnManager
(ThreadSafeClientConnManager.java:199)    - Get connection:
HttpRoute[{}->http://localhost:7001], timeout = 0
2010-07-20 01:20:40,235 DEBUG tsccm.ConnPoolByRoute
(ConnPoolByRoute.java:332)    - [HttpRoute[{}->http://localhost:7001]]
total kept alive: 0, total issued: 0, total allocated: 0 out of 20
2010-07-20 01:20:40,235 DEBUG tsccm.ConnPoolByRoute
(ConnPoolByRoute.java:505)    - No free connections
[HttpRoute[{}->http://localhost:7001]][null]
2010-07-20 01:20:40,235 DEBUG tsccm.ConnPoolByRoute
(ConnPoolByRoute.java:351)    - Available capacity: 2 out of 2
[HttpRoute[{}->http://localhost:7001]][null]
2010-07-20 01:20:40,235 DEBUG tsccm.ConnPoolByRoute
(ConnPoolByRoute.java:531)    - Creating new connection
[HttpRoute[{}->http://localhost:7001]]
2010-07-20 01:20:40,266 DEBUG conn.DefaultClientConnectionOperator
(DefaultClientConnectionOperator.java:145)    - Connecting to
localhost/127.0.0.1:7001
2010-07-20 01:20:40,282 DEBUG client.DefaultHttpClient
(DefaultRequestDirector.java:625)    - Attempt 2 to execute request
2010-07-20 01:20:40,298 DEBUG conn.DefaultClientConnection
(DefaultClientConnection.java:258)    - Sending request: POST
/RemotingLoopback-current/essd-remote/ HTTP/1.1
2010-07-20 01:20:40,298 DEBUG http.headers     
(DefaultClientConnection.java:262)    - >> POST
/RemotingLoopback-current/essd-remote/ HTTP/1.1
2010-07-20 01:20:40,298 DEBUG http.headers     
(DefaultClientConnection.java:265)    - >> Content-Type:
application/x-java-serialized-object
2010-07-20 01:20:40,298 DEBUG http.headers     
(DefaultClientConnection.java:265)    - >> Content-Encoding: none
2010-07-20 01:20:40,298 DEBUG http.headers     
(DefaultClientConnection.java:265)    - >> Serialization-Type: java
2010-07-20 01:20:40,298 DEBUG http.headers     
(DefaultClientConnection.java:265)    - >> Content-Length: 628
2010-07-20 01:20:40,298 DEBUG http.headers     
(DefaultClientConnection.java:265)    - >> Host: localhost:7001
2010-07-20 01:20:40,313 DEBUG http.headers     
(DefaultClientConnection.java:265)    - >> Connection: Keep-Alive
2010-07-20 01:20:40,313 DEBUG conn.DefaultClientConnection
(DefaultClientConnection.java:243)    - Receiving response: HTTP/1.1 200
OK
2010-07-20 01:20:40,313 DEBUG http.headers     
(DefaultClientConnection.java:246)    - << HTTP/1.1 200 OK
2010-07-20 01:20:40,313 DEBUG http.headers     
(DefaultClientConnection.java:249)    - << Date: Tue, 20 Jul 2010
05:20:40 GMT
2010-07-20 01:20:40,313 DEBUG http.headers     
(DefaultClientConnection.java:249)    - << Transfer-Encoding:
chunked
2010-07-20 01:20:40,313 DEBUG http.headers     
(DefaultClientConnection.java:249)    - << Content-Type:
application/x-java-serialized-object
2010-07-20 01:20:40,313 DEBUG http.headers     
(DefaultClientConnection.java:249)    - << Content-Encoding: none
2010-07-20 01:20:40,329 DEBUG http.headers     
(DefaultClientConnection.java:249)    - << Serialization-Type: java
2010-07-20 01:20:40,329 DEBUG http.headers     
(DefaultClientConnection.java:249)    - << X-Powered-By: Servlet/2.5
JSP/2.1
2010-07-20 01:20:40,329 DEBUG client.DefaultHttpClient
(DefaultRequestDirector.java:478)    - Connection can be kept alive for
ever
2010-07-20 01:20:40,345 DEBUG tsccm.ThreadSafeClientConnManager
(ThreadSafeClientConnManager.java:248)    - Released connection is
reusable.
2010-07-20 01:20:40,345 DEBUG tsccm.ConnPoolByRoute
(ConnPoolByRoute.java:416)    - Releasing connection
[HttpRoute[{}->http://localhost:7001]][null]
2010-07-20 01:20:40,345 DEBUG tsccm.ConnPoolByRoute
(ConnPoolByRoute.java:442)    - Pooling connection
[HttpRoute[{}->http://localhost:7001]][null]; keep alive for ever
2010-07-20 01:20:40,345 DEBUG tsccm.ConnPoolByRoute
(ConnPoolByRoute.java:661)    - Notifying no-one, there are no waiting
threads
2010-07-20 01:20:40,485 DEBUG tsccm.ThreadSafeClientConnManager
(ThreadSafeClientConnManager.java:199)    - Get connection:
HttpRoute[{}->http://localhost:7001], timeout = 0
2010-07-20 01:20:40,501 DEBUG tsccm.ConnPoolByRoute
(ConnPoolByRoute.java:332)    - [HttpRoute[{}->http://localhost:7001]]
total kept alive: 1, total issued: 0, total allocated: 1 out of 20
2010-07-20 01:20:40,501 DEBUG tsccm.ConnPoolByRoute
(ConnPoolByRoute.java:480)    - Getting free connection
[HttpRoute[{}->http://localhost:7001]][null]
2010-07-20 01:20:40,501 DEBUG client.DefaultHttpClient
(DefaultRequestDirector.java:625)    - Attempt 2 to execute request
2010-07-20 01:20:40,501 DEBUG conn.DefaultClientConnection
(DefaultClientConnection.java:258)    - Sending request: POST
/RemotingLoopback-current/essd-remote/ HTTP/1.1
2010-07-20 01:20:40,501 DEBUG http.headers     
(DefaultClientConnection.java:262)    - >> POST
/RemotingLoopback-current/essd-remote/ HTTP/1.1
2010-07-20 01:20:40,501 DEBUG http.headers     
(DefaultClientConnection.java:265)    - >> Content-Type:
application/x-java-serialized-object
2010-07-20 01:20:40,501 DEBUG http.headers     
(DefaultClientConnection.java:265)    - >> Content-Encoding: none
2010-07-20 01:20:40,501 DEBUG http.headers     
(DefaultClientConnection.java:265)    - >> Serialization-Type: java
2010-07-20 01:20:40,501 DEBUG http.headers     
(DefaultClientConnection.java:265)    - >> Content-Length: 1768
2010-07-20 01:20:40,501 DEBUG http.headers     
(DefaultClientConnection.java:265)    - >> Host: localhost:7001
2010-07-20 01:20:40,501 DEBUG http.headers     
(DefaultClientConnection.java:265)    - >> Connection: Keep-Alive
2010-07-20 01:20:40,501 DEBUG conn.DefaultClientConnection
(DefaultClientConnection.java:243)    - Receiving response: HTTP/1.1 200
OK
2010-07-20 01:20:40,501 DEBUG http.headers     
(DefaultClientConnection.java:246)    - << HTTP/1.1 200 OK
2010-07-20 01:20:40,501 DEBUG http.headers     
(DefaultClientConnection.java:249)    - << Date: Tue, 20 Jul 2010
05:20:40 GMT
2010-07-20 01:20:40,501 DEBUG http.headers     
(DefaultClientConnection.java:249)    - << Transfer-Encoding:
chunked
2010-07-20 01:20:40,501 DEBUG http.headers     
(DefaultClientConnection.java:249)    - << Content-Type:
application/x-java-serialized-object
2010-07-20 01:20:40,501 DEBUG http.headers     
(DefaultClientConnection.java:249)    - << Content-Encoding: none
2010-07-20 01:20:40,501 DEBUG http.headers     
(DefaultClientConnection.java:249)    - << Serialization-Type: java
2010-07-20 01:20:40,501 DEBUG http.headers     
(DefaultClientConnection.java:249)    - << X-Powered-By: Servlet/2.5
JSP/2.1
2010-07-20 01:20:40,501 DEBUG client.DefaultHttpClient
(DefaultRequestDirector.java:478)    - Connection can be kept alive for
ever
2010-07-20 01:20:40,501 DEBUG tsccm.ThreadSafeClientConnManager
(ThreadSafeClientConnManager.java:248)    - Released connection is
reusable.
2010-07-20 01:20:40,516 DEBUG tsccm.ConnPoolByRoute
(ConnPoolByRoute.java:416)    - Releasing connection
[HttpRoute[{}->http://localhost:7001]][null]
2010-07-20 01:20:40,516 DEBUG tsccm.ConnPoolByRoute
(ConnPoolByRoute.java:442)    - Pooling connection
[HttpRoute[{}->http://localhost:7001]][null]; keep alive for ever
2010-07-20 01:20:40,516 DEBUG tsccm.ConnPoolByRoute
(ConnPoolByRoute.java:661)    - Notifying no-one, there are no waiting
threads
2010-07-20 01:20:40,516 DEBUG tsccm.ThreadSafeClientConnManager
(ThreadSafeClientConnManager.java:199)    - Get connection:
HttpRoute[{}->http://localhost:7001], timeout = 0
2010-07-20 01:20:40,516 DEBUG tsccm.ConnPoolByRoute
(ConnPoolByRoute.java:332)    - [HttpRoute[{}->http://localhost:7001]]
total kept alive: 1, total issued: 0, total allocated: 1 out of 20
2010-07-20 01:20:40,516 DEBUG tsccm.ConnPoolByRoute
(ConnPoolByRoute.java:480)    - Getting free connection
[HttpRoute[{}->http://localhost:7001]][null]
2010-07-20 01:20:40,516 DEBUG client.DefaultHttpClient
(DefaultRequestDirector.java:625)    - Attempt 2 to execute request
2010-07-20 01:20:40,516 DEBUG conn.DefaultClientConnection
(DefaultClientConnection.java:258)    - Sending request: POST
/RemotingLoopback-current/essd-remote/ HTTP/1.1
2010-07-20 01:20:40,516 DEBUG http.headers     
(DefaultClientConnection.java:262)    - >> POST
/RemotingLoopback-current/essd-remote/ HTTP/1.1
2010-07-20 01:20:40,516 DEBUG http.headers     
(DefaultClientConnection.java:265)    - >> Content-Type:
application/x-java-serialized-object
2010-07-20 01:20:40,516 DEBUG http.headers     
(DefaultClientConnection.java:265)    - >> Content-Encoding: none
2010-07-20 01:20:40,516 DEBUG http.headers     
(DefaultClientConnection.java:265)    - >> Serialization-Type: java
2010-07-20 01:20:40,516 DEBUG http.headers     
(DefaultClientConnection.java:265)    - >> Content-Length: 1768
2010-07-20 01:20:40,516 DEBUG http.headers     
(DefaultClientConnection.java:265)    - >> Host: localhost:7001
2010-07-20 01:20:40,516 DEBUG http.headers     
(DefaultClientConnection.java:265)    - >> Connection: Keep-Alive
2010-07-20 01:20:40,516 DEBUG conn.DefaultClientConnection
(DefaultClientConnection.java:243)    - Receiving response: HTTP/1.1 200
OK
2010-07-20 01:20:40,516 DEBUG http.headers     
(DefaultClientConnection.java:246)    - << HTTP/1.1 200 OK
2010-07-20 01:20:40,516 DEBUG http.headers     
(DefaultClientConnection.java:249)    - << Date: Tue, 20 Jul 2010
05:20:40 GMT
2010-07-20 01:20:40,516 DEBUG http.headers     
(DefaultClientConnection.java:249)    - << Transfer-Encoding:
chunked
2010-07-20 01:20:40,516 DEBUG http.headers     
(DefaultClientConnection.java:249)    - << Content-Type:
application/x-java-serialized-object
2010-07-20 01:20:40,516 DEBUG http.headers     
(DefaultClientConnection.java:249)    - << Content-Encoding: none
2010-07-20 01:20:40,516 DEBUG http.headers     
(DefaultClientConnection.java:249)    - << Serialization-Type: java
2010-07-20 01:20:40,516 DEBUG http.headers     
(DefaultClientConnection.java:249)    - << X-Powered-By: Servlet/2.5
JSP/2.1
2010-07-20 01:20:40,516 DEBUG client.DefaultHttpClient
(DefaultRequestDirector.java:478)    - Connection can be kept alive for
ever
2010-07-20 01:20:40,516 DEBUG tsccm.ThreadSafeClientConnManager
(ThreadSafeClientConnManager.java:248)    - Released connection is
reusable.
2010-07-20 01:20:40,516 DEBUG tsccm.ConnPoolByRoute
(ConnPoolByRoute.java:416)    - Releasing connection
[HttpRoute[{}->http://localhost:7001]][null]
2010-07-20 01:20:40,516 DEBUG tsccm.ConnPoolByRoute
(ConnPoolByRoute.java:442)    - Pooling connection
[HttpRoute[{}->http://localhost:7001]][null]; keep alive for ever
2010-07-20 01:20:40,516 DEBUG tsccm.ConnPoolByRoute
(ConnPoolByRoute.java:661)    - Notifying no-one, there are no waiting
threads
2010-07-20 01:20:40,532 DEBUG tsccm.ThreadSafeClientConnManager
(ThreadSafeClientConnManager.java:199)    - Get connection:
HttpRoute[{}->http://localhost:7001], timeout = 0
2010-07-20 01:20:40,532 DEBUG tsccm.ConnPoolByRoute
(ConnPoolByRoute.java:332)    - [HttpRoute[{}->http://localhost:7001]]
total kept alive: 1, total issued: 0, total allocated: 1 out of 20
2010-07-20 01:20:40,532 DEBUG tsccm.ConnPoolByRoute
(ConnPoolByRoute.java:480)    - Getting free connection
[HttpRoute[{}->http://localhost:7001]][null]
2010-07-20 01:20:40,532 DEBUG client.DefaultHttpClient
(DefaultRequestDirector.java:625)    - Attempt 2 to execute request
2010-07-20 01:20:40,532 DEBUG conn.DefaultClientConnection
(DefaultClientConnection.java:258)    - Sending request: POST
/RemotingLoopback-current/essd-remote/ HTTP/1.1
2010-07-20 01:20:40,532 DEBUG http.headers     
(DefaultClientConnection.java:262)    - >> POST
/RemotingLoopback-current/essd-remote/ HTTP/1.1
2010-07-20 01:20:40,532 DEBUG http.headers     
(DefaultClientConnection.java:265)    - >> Content-Type:
application/x-java-serialized-object
2010-07-20 01:20:40,532 DEBUG http.headers     
(DefaultClientConnection.java:265)    - >> Content-Encoding: none
2010-07-20 01:20:40,532 DEBUG http.headers     
(DefaultClientConnection.java:265)    - >> Serialization-Type: java
2010-07-20 01:20:40,532 DEBUG http.headers     
(DefaultClientConnection.java:265)    - >> Content-Length: 1768
2010-07-20 01:20:40,532 DEBUG http.headers     
(DefaultClientConnection.java:265)    - >> Host: localhost:7001
2010-07-20 01:20:40,532 DEBUG http.headers     
(DefaultClientConnection.java:265)    - >> Connection: Keep-Alive
2010-07-20 01:20:40,532 DEBUG conn.DefaultClientConnection
(DefaultClientConnection.java:243)    - Receiving response: HTTP/1.1 200
OK
2010-07-20 01:20:40,532 DEBUG http.headers     
(DefaultClientConnection.java:246)    - << HTTP/1.1 200 OK
2010-07-20 01:20:40,532 DEBUG http.headers     
(DefaultClientConnection.java:249)    - << Date: Tue, 20 Jul 2010
05:20:40 GMT
2010-07-20 01:20:40,532 DEBUG http.headers     
(DefaultClientConnection.java:249)    - << Transfer-Encoding:
chunked
2010-07-20 01:20:40,532 DEBUG http.headers     
(DefaultClientConnection.java:249)    - << Content-Type:
application/x-java-serialized-object
2010-07-20 01:20:40,532 DEBUG http.headers     
(DefaultClientConnection.java:249)    - << Content-Encoding: none
2010-07-20 01:20:40,532 DEBUG http.headers     
(DefaultClientConnection.java:249)    - << Serialization-Type: java
2010-07-20 01:20:40,532 DEBUG http.headers     
(DefaultClientConnection.java:249)    - << X-Powered-By: Servlet/2.5
JSP/2.1
2010-07-20 01:20:40,532 DEBUG client.DefaultHttpClient
(DefaultRequestDirector.java:478)    - Connection can be kept alive for
ever
2010-07-20 01:20:40,532 DEBUG tsccm.ThreadSafeClientConnManager
(ThreadSafeClientConnManager.java:248)    - Released connection is
reusable.
2010-07-20 01:20:40,532 DEBUG tsccm.ConnPoolByRoute
(ConnPoolByRoute.java:416)    - Releasing connection
[HttpRoute[{}->http://localhost:7001]][null]
2010-07-20 01:20:40,532 DEBUG tsccm.ConnPoolByRoute
(ConnPoolByRoute.java:442)    - Pooling connection
[HttpRoute[{}->http://localhost:7001]][null]; keep alive for ever
2010-07-20 01:20:40,532 DEBUG tsccm.ConnPoolByRoute
(ConnPoolByRoute.java:661)    - Notifying no-one, there are no waiting
threads



This transmission may contain information that is privileged,
confidential, legally privileged, and/or exempt from disclosure
under applicable law.  If you are not the intended recipient, you
are hereby notified that any disclosure, copying, distribution, or
use of the information contained herein (including any reliance
thereon) is STRICTLY PROHIBITED.  Although this transmission and
any attachments are believed to be free of any virus or other
defect that might affect any computer system into which it is
received and opened, it is the responsibility of the recipient to
ensure that it is virus free and no responsibility is accepted by
JPMorgan Chase & Co., its subsidiaries and affiliates, as
applicable, for any loss or damage arising in any way from its use.
 If you received this transmission in error, please immediately
contact the sender and destroy the material in its entirety,
whether in electronic or hard copy format. Thank you.

Hi,

I have set up a ring with a couple of servers and wanted to run some
stress tests.

Unfortunately, there is some kind of bottleneck at the client side.
I'm using Hector and Cassandra 0.6.1.

The subsequent profile results are based on a small Java program that
inserts sequentially records, with a couple of columns, into Cassandra
(no-multithreading or something that increases the stress). The nodes
are not too busy while inserting the records (approx. 20%-25% CPU
utilization).

Log-Level is on Info and I don't see any exception flying around. The
client has also registered all available node IPs.

According to my Profiler
operateWithFailover(me.prettyprint.cassandra.service.Operation)
consumes ~86% of the execution time and further down the hierarchy the
method executeAndSetResult(org.apache.cassandra.thrift.Cassandra$Client)
ist responsible for ~73%.

I'm inserting the columns one-by-one is such way:

ColumnPath cp = new ColumnPath(colFamilyName);
cp.setColumn(bytes(colName));
cp.setSuper_column(bytes(superColName));
keySpace.insert(key, cp, value.getBytes());

Can anyone point me out in what I could look into to resolve this issue?

/SJ

Oleg,

We're replacing a t3 solution with one that is similar to SpringRemoting..
Serializing POJOs over http.

I've been able to optimize the CPU utilization and memory usage very
well.. We're running in a smaller memory footprint and less CPU than on
t3.

The one thing I'm struggling with is the actual response times.
We're about 2-6x slower over http.

Keep in mind, the service itself can serve both t3 and http so its not an
issue of the serverside.. it is responding consistently.

One thing that I'm focusing in on is what Jprofiler is telling me about
httpclient.

Please see the attached html file.. it's a Call tree output showing the
CPU time for the call tree.
It shows that 60% of my call time is buried in httpclient.execute.
I would expect that, but what I don't get is that when I drill down to
doSendRequest and doSendResponse I find out that the majority of the time
is eaten up by sendRequestHeader and receiveResponseHeader.. I would have
thought that it would have been the retrieving the entity that would
consume the majority of the time..

I understand that some of the overhead might be related to t3 being
persistent and http not really (having to do the stale check, etc.. ).

But the difference is pretty significant... 2000 invocations on t3 take
5300ms, 2000 invocations on http takes 30000ms..
14 of those 30 seconds are spent inside of httpclient (and that's just CPU
time)..

What in the world am I missing?
Is that just to be expected.. or am I doing something wrong?

What further information do you need from me?
-k




This transmission may contain information that is privileged,
confidential, legally privileged, and/or exempt from disclosure
under applicable law.  If you are not the intended recipient, you
are hereby notified that any disclosure, copying, distribution, or
use of the information contained herein (including any reliance
thereon) is STRICTLY PROHIBITED.  Although this transmission and
any attachments are believed to be free of any virus or other
defect that might affect any computer system into which it is
received and opened, it is the responsibility of the recipient to
ensure that it is virus free and no responsibility is accepted by
JPMorgan Chase & Co., its subsidiaries and affiliates, as
applicable, for any loss or damage arising in any way from its use.
 If you received this transmission in error, please immediately
contact the sender and destroy the material in its entirety,
whether in electronic or hard copy format. Thank you.

Hi, 

I have a multiproject build that takes about 35 seconds using buildr
1.3.5/jruby1.4.0(or jruby 1.4.1) on cygwin.

On rubyforge i found a buildr-1.4.0 bundled distribution. When I ran the
build with that it now took about 50 seconds.

Any thoughts on why buildr-1.4.0 appears to be so much slower than 1.3.5 ?

kind regards
Magnus


 		 	   		  

Hi

I want to use the Jackrabbit JCR for a CR in the company I'm working for.
But my boss has some concerns about performance, reliability, scaling and
security of the Jackrabbit implementation. Can anybody please tell me about
his or her experiences? Or do you know some known sw products, where
Jackrabbit is used?

Regards,

Robert

AEB-Fachforum am 3. November 2010
Logistik zum Anfassen. Au?enwirtschaft zum Erleben. AEB l?dt Sie ein zu
Fachvortr?gen,
Workshops, F?hrungen und zum Networken am Flughafen Stuttgart.
www.aeb-event.de

At first I started using ibatis almost exclusively with auto mapping
(after I solved the camel case<->underscore problem).

Unfortunately later I had to convert to explicit result maps to
achieve the
performance that was required.
Now I have hundreds of boilerplate lines like these:

    <resultMap id="CompanyMap"
type="com.acme.myapp.dtos.CompanyDto">
        <id column="id" property="id"/>
        <result column="city" property="city"/>
        <result column="contact_person" property="contactPerson"/>
        <result column="country" property="country"/>
        <result column="department" property="department"/>
        <result column="is_customer" property="isCustomer"/>
        <result column="name" property="name"/>
        <result column="sap_debitor_no" property="sapDebitorNo"/>
        <result column="short_name" property="shortName"/>
        <result column="state" property="state"/>
        <result column="street" property="street"/>
        <result column="tlc" property="tlc"/>
        ...

I would very much like to get rid of all these superfluous xml!

IMHO there is no fundamental reason why auto mapping should be
noticeably slower
than explicit result maps. Or am I missing something?

I had a look at what's going on and it seems to me that reflection is
done
much too often. Shouldn't it be sufficient to reflect only once on a
class and
from there on treat it exactly like explicitly mapped?

Hi,

Our production environment consists of >= 500 websites and are
currently 
all written in PHP. We're considering moving from PHP to Java but are 
unsure if its the right choice and if it is even possible with that many 
sites. (memory?)

At the moment we're considering to use Felix as the container for all 
applications. So Felix will have > 500 bundles. Each bundle (at least 
the webapps) will be around 100kb each. I'd like to know whether it is 
possible to have that many bundles deployed in Felix and would like to 
know more about the performance aspect.

So in short: is it possible to deploy ~ 500 bundles on one container and 
have the same or even better performance than the same server using PHP? 
Or do you have any other (better)  suggestions?

Regards,

Sander de Groot

Hello,

For comparing performance of Apache Tomcat 7 with APR and Jboss Web Server

with APR version 2.1.4 with Tomcat 7   we have done a load test of an app.
The Open SSL and APR libtcnative files were in the path using the
LD_LIBRARY_PATH env variable. The protocol used was HTTPS.

While doing load testing with HP Load Runner we observed the following
response times(seconds) for 100 concurrent users:
	                       Tomcat7	JbossWebServer-2.1.4	Jboss4.2.3

Hello,

While developing an application using Cayenne I noticed that the
performance of  DataContext.commitChanges()  degrades with the number of
commits done.

To be more specific, I have a model with two entities: Result and
ResultStream, with a many to one relation from Result to ResultStream.

The test I do is simple: I generate a lot of Result objects, all linked to
the same ResultStream and I call commit after each Result added to the
DataContext and measure the time it takes to perform the commit. I notice
that this time grows fairly linear. Of course, the growth is noticeably
after generating many results.

Attached is the test source.

If I don't make the link between result and resultStream, the performance
is not degrading

Could you please help me with some advice?
Is this a known issue, can it be fixed by tweaking cayenne?

Many thanks,
Cosmin

Sorry,

The attached file was not uploaded.
Here it is:

package insert;

import org.apache.cayenne.access.DataContext;
import org.apache.cayenne.query.SelectQuery;
import org.apache.commons.lang.time.DurationFormatUtils;
import org.apache.commons.lang.time.StopWatch;

public class Main {

    public static void main(String[] args) {
        DataContext dc = DataContext.createDataContext();

        final int stepSize = 500;
        final int nbSteps = 50;

        SelectQuery sq = new SelectQuery(ResultStream.class);

        ResultStream rs = (ResultStream) dc.performQuery(sq).get(0);

        StopWatch sw = new StopWatch();
        sw.start();

        for (int i = 1; i <= nbSteps * stepSize; ++i) {
            Result t = new Result();
            t.setValue(i);
            t.setResultStream(rs);

            dc.commitChanges();

            if (i % stepSize == 0) {
                sw.stop();
                System.out.printf("%6d, %s, %5d\n", i,
DurationFormatUtils.formatDurationHMS(sw.getTime()),
sw.getTime());
                sw.reset();
                sw.start();
            }
        }
    }
}


 Hello,

 While developing an application using Cayenne I noticed that the
 performance of  DataContext.commitChanges()  degrades with the number
of
 commits done.

 To be more specific, I have a model with two entities: Result and
 ResultStream, with a many to one relation from Result to
ResultStream.

 The test I do is simple: I generate a lot of Result objects, all
linked to
 the same ResultStream and I call commit after each Result added to
the
 DataContext and measure the time it takes to perform the commit. I
notice
 that this time grows fairly linear. Of course, the growth is
noticeably
 after generating many results.

 Attached is the test source.

 If I don't make the link between result and resultStream, the
performance
 is not degrading

 Could you please help me with some advice?
 Is this a known issue, can it be fixed by tweaking cayenne?

 Many thanks,
 Cosmin

Hello everyone,

I have a "conf" db with the following documents:

(env)(db) [17:38] sauce $curl localhost:5984/conf/_all_docs
{"total_rows":8,"offset":0,"rows":[
{"id":"_design/amis","key":"_design/amis","value":{"rev":"1-d210474cd70114aa12a9a0a45e5d0f20"}},
{"id":"_design/takos","key":"_design/takos","value":{"rev":"1-e35bbf33d4b29a86eb60fafbab7aa42c"}},
{"id":"ami-a5b559cc","key":"ami-a5b559cc","value":{"rev":"1-5f08c036a73967cb6b72fd0e580a4b84"}},
{"id":"ami-a9e90fc0","key":"ami-a9e90fc0","value":{"rev":"1-3f3a5d33f3db78c971ceade5c6a2b960"}},
{"id":"ami-cf608da6","key":"ami-cf608da6","value":{"rev":"1-9c8c728402e151e0ad39e67909fd6af9"}},
{"id":"ami-db608db2","key":"ami-db608db2","value":{"rev":"1-5d5683582b4c513536c0a05fe4d494a0"}},
{"id":"ami-dd608db4","key":"ami-dd608db4","value":{"rev":"1-7187ff6b10fad61c79ed2c0cbf64b374"}},
{"id":"expediter","key":"expediter","value":{"rev":"1-9b56984574a38d0c69e33abb055ea3e6"}}
]}

There's a view called "amis/active" which says:
curl localhost:5984/conf/_design/amis | jsonpretty
{
   "_rev": "1-d210474cd70114aa12a9a0a45e5d0f20",
   "_id": "_design/amis",
   "language": "javascript",
   "filters": {},
   "views": {
       "active": {
           "map": "function(doc)\n{\n  if ((doc.doc_type ==
\"ChefAMI\" || doc.doc_type == \"AMI\") && doc.active) {\n
emit(doc._id, doc);\n  }\n}"
       }
   }
}


However when I run the view:
   curl localhost:5984/conf/_design/amis/_view/active

It just hangs and "beam.smp" takes 10-14% CPU.

CouchDB 0.11, Ubuntu 10.4

Any ideas would be greatly appreciated!

Adam Christian

Hello,

Yes, I am on Cayenne 3.0 release.

I profiled the test but got a bit lost. Eclipse profiler is not very easy
to use and cayenne is quite complex.

From what I have seen, ObjectStore.postprocessAfterCommit() takes a lot of
time.

Actually, from my two tests (I set the relation -> slow, I don't set it
->
constant time) I noticed that postprocessAfterCommit() takes much more
time in the first case.

If you have an idea on what to look for in a profile or for a better test,
let me know, I will do the tests.

Thanks,
Cosmin

Andrus Adamchik wrote:
 This is odd indeed. Are you on Cayenne 3.0?

 I am not aware of this issue. All operations involved are constant
time
ops regardless of the number of objects previously cached (such as entry
lookup in a HashMap or a list.add(..) operation).

 So I may try to run this code in profiler to see why it is slowing
down.
If you have access to a profiler and willing to profile it yourself,
this will help a lot of course :-)

 Andrus

Hi All,
I found that our security implementation is much slower than RI's when
get an algorithm instance.
I have done some investigation and found that most time are consumed
on loading and initialising providers.
Following is the test cases and results to show this problem.

                      RI5         |      RI6      |     Harmony5    |
  Harmony6
Testcase1[1]    78ms      | 90~100ms  |     450ms        |   475~490ms
Testcase2[2]    75ms      |  85~91ms  |                       |
Testcase3[3]                  |                 |   447~455ms    |
470~490ms
Testcase4[4]    9~10ms  |   6ms         |                       |
Testcase5[5]                 |                  |    448~462ms   | 
479~501ms
Testcase6[6]    9ms       |    8~10ms   |   448~459ms   |   481~490ms

From[1], we can see our implementation is much slower than RI's when
loading all the providers.
Compare [2] & [3], both loading JSSE provider, our implementation is
much slower.
Compare [2] & [4], the time differs, which means RI can load providers
one by one.
Compare [1],[3] & [5], we can see our implementation consumes almost
same time whether it loads one or more providers.
Form[6], I guess RI knows what services a provider provides so that it
can just pick up the providers the application need.

If you look into the Services.java, you will find that our
implementation will load all the providers when this class is loaded
even this application does NOT need some of providers.
Then it put all the providers and services into a cache which can be
called "pre-loading".

Base on the above investigation, I suggest to change the "pre-loading"
to "lazy-loading" if possible which means only loading necessary
providers, no more, no less.

But I found that it is very hard to achieve this target, because we
don't know what services a provider provides before we load the
provider.
I don't know what RI does to achieve this.
Maybe we can store the map relationships between these default
providers and services to a file or internal cache table?

Any comments or suggestions?

Jimmy, Kevin,
I talked to you about this issue before, feel free to fix me if
anything is wrong.
Or you can give more information about this issue.

[1]
public static void main(String[] args) {
        long start = System.currentTimeMillis();
        for (Provider p : Security.getProviders()) {
            p.getServices();
        }
        System.out.println(System.currentTimeMillis() - start);
}

[2]
public static void main(String[] args) throws NoSuchAlgorithmException {
    long start = System.currentTimeMillis();
    Security.getProvider("SUNJSSE");
    System.out.println(System.currentTimeMillis() - start);
}

[3]
public static void main(String[] args) throws NoSuchAlgorithmException {
     long start = System.currentTimeMillis();
     Security.getProvider("HarmonyJSSE");
     System.out.println(System.currentTimeMillis() - start);
}

[4]
public static void main(String[] args) throws NoSuchAlgorithmException {
        long start = System.currentTimeMillis();
        Security.getProvider("SUN");
        System.out.println(System.currentTimeMillis() - start);
}

[5]
public static void main(String[] args) throws NoSuchAlgorithmException {
    long start = System.currentTimeMillis();
    Security.getProvider("BC");
    System.out.println(System.currentTimeMillis() - start);
}

[6]
public static void main(String[] args) throws NoSuchAlgorithmException {
    long start = System.currentTimeMillis();
    MessageDigest.getInstance("SHA-1");
    System.out.println(System.currentTimeMillis() - start);
}


Ray Chen

For every file should be served by apache httpd, apache httpd tries to
lstat
all directory in path more times:

This is normally seen when you have .htaccess usage turned on (all
subdirectories have to be checked for the presence of .htaccess). Do you
have AllowOverride set to off?

Joost

Since your data comes in ever second, couldn't you constrain the query with
WHERE CURRENT_TIMESTAMP - Tstamp < 10000

This would give you all data less than ten seconds old

Sent from my iPhone

On Jul 9, 2010, at 7:47 PM, Robert Brewer <rbre### @lava.net> wrote:

Hello. I'm using Derby in embedded mode in a Java web application, and
have run into a query that runs slowly, though I would expect it to be
fast. I'm somewhat of an SQL newbie, so please forgive me if this is
obvious.

The table in question stores power data from sensors. New rows are being
inserted every few seconds, and I want to retrieve the most recent data
from a particular sensor (a "Source" in our terminology).

The table schema is:

create table SensorData (
  Tstamp TIMESTAMP NOT NULL,
  Tool VARCHAR(128) NOT NULL,
  Source VARCHAR(256) NOT NULL,
  Properties VARCHAR(32000),
  LastMod TIMESTAMP NOT NULL,
  PRIMARY KEY (Source, Tstamp) )

I added the following indices, in the hope that they might speed things up
(they did not):

CREATE INDEX TstampIndex ON SensorData(Tstamp asc)
CREATE INDEX TstampIndexDesc ON SensorData(Tstamp desc)
CREATE INDEX SourceIndex ON SensorData(Source asc)

To retrieve the most recent data entry, my query is:

SELECT * FROM SensorData WHERE Source = ? ORDER BY Tstamp DESC FETCH FIRST
ROW ONLY

The table has over 3 million rows, and the select statement above is
taking 10-12 seconds to execute. Naively, I would expect to be able to find
the row with most recent Tstamp fairly quickly since it is indexed.

Any suggestions on what I can do to improve the performance of this query
(hopefully to 1 second or less) would be most appreciated. Mahalo!

Hi,

I want to optimize the performance of tomcat.
Can anyone please point to some docs/wiki/direct answer.


Regards
Harpreet