Re: Memory fragmentation

I put in a counter for each time zmalloc() (in zmalloc.c) is called
which gets reset each time call() (in redis.c) is called

Here are the number of malloc's per read op
GET                   -> 39 mallocs
LRANGE of 1     -> 57 mallocs
SISMEMBER      -> 39 mallocs
ZSCORE            -> 42 mallocs
HGET                 -> 46 mallocs

many of the malloc's have to do with command line arguments and the
response and most are small sizes (mean size is maybe 30bytes) and
they seem to always be for the same sizes independent of the command.

These are read operations and should not require much malloc'ing.

This is a great place to introduce some static buffers and cut done on
the number of malloc's which would help greatly w/ memory
fragmentation issues.

thoughts?

Updates:
	Status: WontFix

Comment #6 on issue 248 by antirez: Real memory exceeded maxmemory with  
Virtual Memory mechanism in redis-1.3.12
http://code.google.com/p/redis/issues/detail?id=248

(No comment was entered for this change.)

I know this subject has been discussed in the past on the list and I've
read
through all discussions but I haven't been able to find a solution to the
memory problems listed below... so here again...

It seems that the cassandra cluster I'm using is either leaking memory or
just using more mem than I expected it to use.
Each host in the ring uses about 12G of ram while in some cases its entire
dataset is only 1.5G (take for example .252.124 below with 1.54G)
I use extensive row caching so I expect memory consumption to be >=
1.5G but
I don't understand why it gets up to 12G. Most of the times I don't care
so
much since I have plenty of memory however at times this gets me into GC
storms and very slow responses. Also, I'd like to be able to load more
data
to the cluster and I'm hitting the memory wall, which I didn't expect.

In the cassandra.in.sh you'd notice that I do provide Xmx=12G but given
that
there's so little data I wouldn't expect the process to be using all of
that. As a matter of fact I wanted to insert more data to the cluster but
I
stopped since it wasn't handling the load very well.

I suppose that at the end of the day I only need to know which knobs
configure but after having played with the configuration for a long time
I'm
a little lost.

I'm running a 0.6.2 cluster consisting of 6 physical hosts (some with 16G
and some 32G ram) distributed b/w two DCs.
RF is 2 (one replica in each DC).
HH is turned off.
File access is standard (no m-mapped files, I tried that and the system
just
kept swapping itself to death so I switched back to normal).

I've pasted below the output of nodetool ring and cfstats as well as some
vmstat and iostat (not that I think it matters...)
Also jmap -heap and attached is the jmap -histo so I hope this output can
help shed some light on memory usage.
Currently the logs don't say anything out of the ordinary so I didn't
include them.

Thanks :)

$ nodetool -h cass99 -p 9004 ring
Address       Status     Load          Range
     Ring

170141183460469231731687303715884105727
192.168.252.99Up         6.16 GB
28356863910078205288614550619314017621     |<--|
192.168.252.124Up         1.54 GB
56713727820156410577229101238628035242     |   ^
192.168.252.125Up         1.54 GB
85070591730234615865843651857942052863     v   |
192.168.254.57Up         6.15 GB
113427455640312821154458202477256070485    |   ^
192.168.254.58Up         1.54 GB
141784319550391026443072753096570088106    v   |
192.168.254.59Up         1.54 GB
170141183460469231731687303715884105727    |-->|

 <Keyspace Name="outbrain_kvdb">
      <ColumnFamily CompareWith="BytesType" Name="KvImpressions"
                    KeysCached="0"
                    RowsCached="10000000"/>
      <ColumnFamily CompareWith="BytesType" Name="KvAds"
                    KeysCached="0"
                    RowsCached="10000000"/>
      <ColumnFamily CompareWith="BytesType" Name="KvRatings"
                    KeysCached="0"
                    RowsCached="10000000"/>


<ReplicaPlacementStrategy>org.apache.cassandra.locator.RackAwareStrategy</ReplicaPlacementStrategy>
      <ReplicationFactor>2</ReplicationFactor>


<EndPointSnitch>org.apache.cassandra.locator.EndPointSnitch</EndPointSnitch>
    </Keyspace>


$ cat bin/cassandra.in.sh
# Licensed to the Apache Software Foundation (ASF) under one
...
# Arguments to pass to the JVM
JVM_OPTS=" \
        -ea \
        -Xms4G \
        -Xmx*12*G \
        -XX:+UseParNewGC \
        -XX:+UseConcMarkSweepGC \
        -XX:+CMSParallelRemarkEnabled \
        -XX:SurvivorRatio=8 \
        -XX:MaxTenuringThreshold=1 \
        -XX:+HeapDumpOnOutOfMemoryError \
        -Dcom.sun.management.jmxremote.port=9004 \
        -Dcom.sun.management.jmxremote.ssl=false \
        -Dcom.sun.management.jmxremote.authenticate=false"




Keyspace: outbrain_kvdb
        Read Count: 5608010
        Read Latency: 8.52211627029909 ms.
        Write Count: 42794
        Write Latency: 0.10353956162078796 ms.
        Pending Tasks: 0

                Column Family: KvAds
                SSTable count: 11
                Space used (live): 9331647391
                Space used (total): 9331647391
                Memtable Columns Count: 84928
                Memtable Data Size: 21400502
                Memtable Switch Count: 1
                Read Count: 5602705
                Read Latency: 2.023 ms.
                Write Count: 42794
                Write Latency: 0.060 ms.
                Pending Tasks: 0
                Key cache: disabled
                Row cache capacity: 10000000
                Row cache size: 698671
                Row cache hit rate: 0.5535463700149053
                Compacted row minimum size: 391
                Compacted row maximum size: 76890
                Compacted row mean size: 635



top - 10:23:26 up 96 days, 23:04,  1 user,  load average: 5.03, 6.21, 6.08
Tasks:  93 total,   1 running,  92 sleeping,   0 stopped,   0 zombie
Cpu(s): 92.1%us,  4.1%sy,  0.0%ni,  1.8%id,  0.0%wa,  0.5%hi,  1.5%si,
 0.0%st
Mem:  16443880k total, 16357676k used,    86204k free,    43448k buffers
Swap:  4194296k total,    13912k used,  4180384k free,  2625024k cached

  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND



 5757 cassandr  25   0 13.6g  12g 9860 S 197.2 82.3   9445:17 java



$ jmap -heap 5757
Attaching to process ID 5757, please wait...
Debugger attached successfully.
Server compiler detected.
JVM version is 16.3-b01

using parallel threads in the new generation.
using thread-local object allocation.
Concurrent Mark-Sweep GC

Heap Configuration:
   MinHeapFreeRatio = 40
   MaxHeapFreeRatio = 70
   MaxHeapSize      = 12884901888 (12288.0MB)
   NewSize          = 21757952 (20.75MB)
   MaxNewSize       = 43581440 (41.5625MB)
   OldSize          = 65404928 (62.375MB)
   NewRatio         = 7
   SurvivorRatio    = 8
   PermSize         = 21757952 (20.75MB)
   MaxPermSize      = 88080384 (84.0MB)

Heap Usage:
New Generation (Eden + 1 Survivor Space):
   capacity = 39256064 (37.4375MB)
   used     = 6779480 (6.465415954589844MB)
   free     = 32476584 (30.972084045410156MB)
   17.26989236618322% used
Eden Space:
   capacity = 34930688 (33.3125MB)
   used     = 2490360 (2.3749923706054688MB)
   free     = 32440328 (30.93750762939453MB)
   7.12943300744606% used
From Space:
   capacity = 4325376 (4.125MB)
   used     = 4289120 (4.090423583984375MB)
   free     = 36256 (0.034576416015625MB)
   99.16178385416667% used
To Space:
   capacity = 4325376 (4.125MB)
   used     = 0 (0.0MB)
   free     = 4325376 (4.125MB)
   0.0% used
concurrent mark-sweep generation:
   capacity = 12841320448 (12246.4375MB)
   used     = 10867324872 (10363.888618469238MB)
   free     = 1973995576 (1882.5488815307617MB)
   84.62778353679785% used
Perm Generation:
   capacity = 30380032 (28.97265625MB)
   used     = 18100520 (17.262001037597656MB)
   free     = 12279512 (11.710655212402344MB)
   59.580319072738305% used

Hi,

After storing a while data (about 80.000 nodes) into an CR, an out of
memory error occurred:

2010-08-25 00:22:58,439 ERROR (IndexMerger.java:568) - Error while merging
indexes:
java.lang.OutOfMemoryError: Java heap space
            at java.util.HashMap.resize(HashMap.java:508)
            at java.util.HashMap.addEntry(HashMap.java:799)
            at java.util.HashMap.put(HashMap.java:431)
            at
org.apache.jackrabbit.core.query.lucene.CachingIndexReader$CacheInitializer$1.collect(CachingIndexReader.java:433)
            at
org.apache.jackrabbit.core.query.lucene.CachingIndexReader$CacheInitializer.collectTermDocs(CachingIndexReader.java:515)
            at
org.apache.jackrabbit.core.query.lucene.CachingIndexReader$CacheInitializer.initializeParents(CachingIndexReader.java:425)
            at
org.apache.jackrabbit.core.query.lucene.CachingIndexReader$CacheInitializer.run(CachingIndexReader.java:386)
            at
org.apache.jackrabbit.core.query.lucene.CachingIndexReader.<init>(CachingIndexReader.java:135)
            at
org.apache.jackrabbit.core.query.lucene.AbstractIndex.getReadOnlyIndexReader(AbstractIndex.java:315)
            at
org.apache.jackrabbit.core.query.lucene.MultiIndex.replaceIndexes(MultiIndex.java:665)
            at
org.apache.jackrabbit.core.query.lucene.IndexMerger$Worker.run(IndexMerger.java:551)
            at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:417)
            at
java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:269)
            at java.util.concurrent.FutureTask.run(FutureTask.java:123)
            at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:65)
            at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:168)
            at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:650)
            at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:675)
            at java.lang.Thread.run(Thread.java:595)

If I add more properties or change the binary data mime type for full text
indexing, then I'll get the error sooner.

If I quit the application and restart it, it works again for a while. It
seems to me, that lucene is stacking up memory.

How can I handle that situation?

Kindly regards,

Robert Seidel

When starting my web app under Tomcat 6.0.26, I get the following logged..

(The application otherwise appears to be working fine so I'm just 
wondering if I should be concerned or how to prevent this error from 
happening upon startup)

20-Jul-2010 9:16:40 AM org.apache.catalina.loader.WebappClassLoader 
clearReferencesJdbc
SEVERE: A web application registered the JBDC driver 
[com.ibm.db2.jcc.DB2Driver] but failed to unregister it when the web 
application was stopped. To prevent a memory leak, the JDBC Driver has 
been forcibly unregistered.
20-Jul-2010 9:16:40 AM org.apache.catalina.loader.WebappClassLoader 
clearReferencesThreads
SEVERE: A web application appears to have started a thread named 
[cayenne-edt-597457953-0] but has failed to stop it. This is very likely 
to create a memory leak.
20-Jul-2010 9:16:40 AM org.apache.catalina.loader.WebappClassLoader 
clearReferencesThreads
SEVERE: A web application appears to have started a thread named 
[cayenne-edt-597457953-1] but has failed to stop it. This is very likely 
to create a memory leak.
20-Jul-2010 9:16:40 AM org.apache.catalina.loader.WebappClassLoader 
clearReferencesThreads
SEVERE: A web application appears to have started a thread named 
[cayenne-edt-597457953-2] but has failed to stop it. This is very likely 
to create a memory leak.
20-Jul-2010 9:16:40 AM org.apache.catalina.loader.WebappClassLoader 
clearReferencesThreads
SEVERE: A web application appears to have started a thread named 
[cayenne-edt-597457953-3] but has failed to stop it. This is very likely 
to create a memory leak.
20-Jul-2010 9:16:40 AM org.apache.catalina.loader.WebappClassLoader 
clearReferencesThreads
SEVERE: A web application appears to have started a thread named 
[cayenne-edt-597457953-4] but has failed to stop it. This is very likely 
to create a memory leak.
20-Jul-2010 9:16:40 AM org.apache.catalina.loader.WebappClassLoader 
clearReferencesThreads
SEVERE: A web application appears to have started a thread named 
[PoolManagerCleanup-2013151593] but has failed to stop it. This is very 
likely to create a memory leak.
log4j:WARN No appenders could be found for logger 
(org.apache.cayenne.conf.DefaultConfiguration).
log4j:WARN Please initialize the log4j system properly.
log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for 
more info.

Thanks,

Andrew

Hi,

I have a Cassandra cluster and I set java Xmx to 2G in cassandra.in.sh in
every node, but the memory usage of cassandra daemon is 3 or 4 times of
that
(so 6-8 Gb).

Have you got any idea to decrease the memory usage of cassandra?

There are 4 nodes in the cluster and the system has approx 1000 reads and
writes per sec.
Every node has debian.

Hi,

Can anyone help me with this problem? I've been running tests running a
small number of threads starting over a 1 minute period with no problem
with
JMeter. I increased the size of the number of threads to 500 to start up
over an 8 hour period. JMeter threw an 'Out of Memory' error.
I had already increased available RAM to Jmeter and the only thing left to
me is to remove one of the two listeners from the script.
The available RAM was set at "set HEAP=-Xms1024m -Xmx1536m". Is this OK???
Should the Xms value be lower? My PC has 2Gb of RAM available.

Any help with this much appreciated...

 Sorry for not responding yet.  Busy at what I actually 
 get paid for, and trying to generate some real numbers 
 for the question at hand.  More later. 

No worries, I just want to keep the thread alive until something
definitive comes from it if possible. :-)




Disclaimer - June 21, 2010 
This email and any files transmitted with it are confidential and intended
solely for Tomcat Users List. If you are not the named addressee you should
not disseminate, distribute, copy or alter this email. Any views or
opinions presented in this email are solely those of the author and might
not represent those of . Warning: Although  has taken reasonable
precautions to ensure no viruses are present in this email, the company
cannot accept responsibility for any loss or damage arising from the use of
this email or attachments. 
This disclaimer was added by Policy Patrol: http://www.policypatrol.com/

 So what is the impact say (picking some random 
 numbers of the air) if you have an app that is quite 
 happy with 256MB of heap space but has 1024MB allocated?  

My question exactly!







Disclaimer - June 21, 2010 
This email and any files transmitted with it are confidential and intended
solely for Tomcat Users List. If you are not the named addressee you should
not disseminate, distribute, copy or alter this email. Any views or
opinions presented in this email are solely those of the author and might
not represent those of . Warning: Although  has taken reasonable
precautions to ensure no viruses are present in this email, the company
cannot accept responsibility for any loss or damage arising from the use of
this email or attachments. 
This disclaimer was added by Policy Patrol: http://www.policypatrol.com/

> For what it's worth, either this is not the case in our 
> real-world situation or the effect is negligible.

 Not surprising - you'd have to be very unlucky to be right 
 at the edge and see a lot of GC activity and be able to 
 continue running.  Usually you'll be over the edge a bit, 
 encounter OOMEs, increase the heap significantly, and be 
 well away from the frequent tight heap scenario. 

What qualifies as a tight heap and what qualifies as a significant
increase? Usually when we see OOMEs we increase the allocation by 32MB
and they go away. 

To return to the original question, is it generally better to custom-fit
the RAM allocation (as we currently do) or to unilaterally set all
instances to a higher amount that we know will not generate OOMEs, such
as 512MB?





Disclaimer - June 20, 2010 
This email and any files transmitted with it are confidential and intended
solely for Tomcat Users List. If you are not the named addressee you should
not disseminate, distribute, copy or alter this email. Any views or
opinions presented in this email are solely those of the author and might
not represent those of . Warning: Although  has taken reasonable
precautions to ensure no viruses are present in this email, the company
cannot accept responsibility for any loss or damage arising from the use of
this email or attachments. 
This disclaimer was added by Policy Patrol: http://www.policypatrol.com/

This email contained a .zip file attachment. Raytheon does not allow email
attachments that are considered likely to contain malicious code. For your
protection this attachment has been removed.

If this email is from an unknown source, please simply delete this email.

If this email was expected, and it is from a known sender, you may follow
the below suggested instructions to obtain these types of attachments.

+ Instruct the sender to enclose the file(s) in a ".zip" compressed file,
and rename the ".zip" compressed file with a different extension, such as,
".rtnzip".  Password protecting the renamed ".zip" compressed file adds an
additional layer of protection. When you receive the file, please rename it
with the extension ".zip".

Additional instructions and options on how to receive these attachments
can be found at:

http://security.it.ray.com/antivirus/extensions.html
http://security.it.ray.com/news/2007/zipfiles.html

Should you have any questions or difficulty with these instructions,
please contact the Help Desk at 877.844.4712

---

 From: Caldarale, Charles R [mailto:Chuck.Ca### @unisys.com]
 Subject: RE: Setting the Right Amount of Memory
 
 Let's see if a .java attachment makes it through...

So let's try it as a zip file...

 - Chuck


THIS COMMUNICATION MAY CONTAIN CONFIDENTIAL AND/OR OTHERWISE PROPRIETARY
MATERIAL and is thus for use only by the intended recipient. If you
received this in error, please contact the sender and delete the e-mail and
its attachments from all computers.

 Having a borderline heap size can, in the worst case, result
 in almost continual GC activity, if there is only room to 
 allocate a minimal number of objects between GC occurrences.

For what it's worth, either this is not the case in our real-world
situation or the effect is negligible. Even though we have 160 instances
of tomcat on the same server, and all of them are trimmed to use the
minimum RAM (usually 64-96MB) and all instances are quite actively being
used, the server is still 90% idle during peak load. The server is an
8-core 2.6GHz Xeon with 32GB RAM (10GB free and no swapping).


Disclaimer - June 20, 2010 
This email and any files transmitted with it are confidential and intended
solely for Tomcat Users List. If you are not the named addressee you should
not disseminate, distribute, copy or alter this email. Any views or
opinions presented in this email are solely those of the author and might
not represent those of . Warning: Although  has taken reasonable
precautions to ensure no viruses are present in this email, the company
cannot accept responsibility for any loss or damage arising from the use of
this email or attachments. 
This disclaimer was added by Policy Patrol: http://www.policypatrol.com/

As I alluded to in another post, we just moved from 2-4 nodes. Since then,
the cluster has been incredibly

The memory problems I've posted about before have gotten much worse and
our
nodes are becoming incredibly slow/unusable every 24 hours or so.
Basically,
the JVM reports that only 14GB is committed, but the RSS of the process is
22GB, and cassandra is completely unresponsive, but still having requests
routed to it internally, so it completely destroys performance.

I'm at a loss for how to diagnose this issue.

In addition to that, read performance has gone way downhill, and query
latency is much slower than it was with a 2 node cluster. Perhaps this was
to be expected, though.

We really like cassandra for the most part, but these stability issues are
going to force us to abandon it. Our application is like a yoyo right now,
and we can't live with that.

Help resolving these issues would be greatly appreciated.

 Just a note here : 160 X 512 MB = 81 GB. If each Tomcat's 
 JVM Is allowed to use up to 512 MB of Heap, there might be 
 moments where a lot of JVM's will be using close to that 
 amount.  Unless your system can really support that amount 
 of real RAM, you may be in for massive swapping.

Thanks, but no worries there. I would never allow swapping. If I have to
allocate 512MB RAM to each instance, it will be on a new server with
adequate physical RAM.

 During a full GC, the JVM does nothing else (applications are
 frozen). So there may be an impact on the user experience during 
 that time.

That's the effect I'm trying to anticipate.

 Note also that the 81 GB above count only the Tomcat Heaps, and 
 disregard any other memory needed by your 160 instances.

Very good point.







Disclaimer - June 19, 2010 
This email and any files transmitted with it are confidential and intended
solely for Tomcat Users List. If you are not the named addressee you should
not disseminate, distribute, copy or alter this email. Any views or
opinions presented in this email are solely those of the author and might
not represent those of . Warning: Although  has taken reasonable
precautions to ensure no viruses are present in this email, the company
cannot accept responsibility for any loss or damage arising from the use of
this email or attachments. 
This disclaimer was added by Policy Patrol: http://www.policypatrol.com/

On 17/06/2010 08:59, Robinson, Eric wrote:
> If your heap size is right on the edge of your minimum for a
Tomcat 
> instance, you may be doing more GC work than is really needed.
> However, if you're satisfied with the response time and CPU 
> utilization, you should be ok.

 Time to hit the vendor around the head with the cluebat. If 
 the app is happy with less heap space then increasing it 
 is only going to cause problems - mainly that GC when it 
 happens will take longer and trigger longer pauses. You 
 can mitigate this with GC config (later VMs may make the 
 right choices for you anyway) but all this is just adding 
 unecessary complexity.

 Mark

With 160 instances of tomcat on the server, and most of them happy with
64-96MB of RAM, could you take an educated guess at the negative impact
on the server of raising the RAM to 512MB for each instance? How much
extra CPU utilization do you think I could possibly see from all the
extra GC?

--Eric


Disclaimer - June 19, 2010 
This email and any files transmitted with it are confidential and intended
solely for Tomcat Users List. If you are not the named addressee you should
not disseminate, distribute, copy or alter this email. Any views or
opinions presented in this email are solely those of the author and might
not represent those of . Warning: Although  has taken reasonable
precautions to ensure no viruses are present in this email, the company
cannot accept responsibility for any loss or damage arising from the use of
this email or attachments. 
This disclaimer was added by Policy Patrol: http://www.policypatrol.com/

 Time to hit the vendor around the head with the cluebat. 
 If the app is happy with less heap space then increasing 
 it is only going to cause problems - mainly that GC when it 
 happens will take longer and trigger longer pauses. You can 
 mitigate this with GC config (later VMs may make the right 
 choices for you anyway) but all this is just adding unnecessary 
 complexity.

I agree completely. I wish the vendor accepted feedback like this.
Unfortunately if you try to show them a better way, they tend to get
defensive and think of you as a troublemaker. :-(





Disclaimer - June 17, 2010 
This email and any files transmitted with it are confidential and intended
solely for Tomcat Users List. If you are not the named addressee you should
not disseminate, distribute, copy or alter this email. Any views or
opinions presented in this email are solely those of the author and might
not represent those of . Warning: Although  has taken reasonable
precautions to ensure no viruses are present in this email, the company
cannot accept responsibility for any loss or damage arising from the use of
this email or attachments. 
This disclaimer was added by Policy Patrol: http://www.policypatrol.com/

 Just wondering, what tools do you use to manage all the 
 instances? Also, what do you use to look for the OutOfMemory 
 in logs?  I am looking at Splunk too.

Just shell scripts. We have scripts that...

	-- set up a new instance and customize it for a new customer
	-- stops, starts, restarts, optionally clearing the work folders
	-- analyzes jasper logs for system responsiveness (both realtime
and historically)
	-- synchronizes certain files between instances on different
tomcat servers
	-- watches for OutOfMemory in various logs
 	-- grooms logs periodically
	-- monitors various logs in realtime

..and so on. Never found anything we couldn't do with bash and cron.




Disclaimer - June 17, 2010 
This email and any files transmitted with it are confidential and intended
solely for Tomcat Users List. If you are not the named addressee you should
not disseminate, distribute, copy or alter this email. Any views or
opinions presented in this email are solely those of the author and might
not represent those of . Warning: Although  has taken reasonable
precautions to ensure no viruses are present in this email, the company
cannot accept responsibility for any loss or damage arising from the use of
this email or attachments. 
This disclaimer was added by Policy Patrol: http://www.policypatrol.com/

Hi

When running activemq in memory (vm://localhost) it seems that the queue
messages are still persisted ssomewhere. Is there a way to persist the
message only in memory so the queue messages will dead with the broker.

This is for unit test.
Regards

 If your heap size is right on the edge of your minimum for a Tomcat
 instance, you may be doing more GC work than is really needed.  
 However, if you're satisfied with the response time and 
 CPU utilization, you should be ok. 

My thoughts exactly. Just wanted to check it with the community. FYI,
with 160 instances of tomcat running at the same time, CPU is still 90%
idle even during peak production hours. Now the software vendor is
coming along and forcing us to set the heap for each instance to 512MB
when 64MB or 96MB works fine. It's unnecessarily expensive and super
frustrating.





Disclaimer - June 17, 2010 
This email and any files transmitted with it are confidential and intended
solely for Tomcat Users List. If you are not the named addressee you should
not disseminate, distribute, copy or alter this email. Any views or
opinions presented in this email are solely those of the author and might
not represent those of . Warning: Although  has taken reasonable
precautions to ensure no viruses are present in this email, the company
cannot accept responsibility for any loss or damage arising from the use of
this email or attachments. 
This disclaimer was added by Policy Patrol: http://www.policypatrol.com/

Hi,

there are some places where httpd uses more memory than necessary, 
which can increase cache misses and reduce performance on current 
hardware. Things I would like to change:

1) reorder structs to have fewer holes on 64bit arches

httpd.h has this nice comment since the days of 1.3:

/* Things placed at the end of the record to avoid breaking binary
 * compatibility.  It would be nice to remember to reorder the entire
 * record to improve 64bit alignment the next time we need to break
 * binary compatibility for some other reason.
 */

Pro: Examples for space savings on 64bit:

request_rec:   704 -> 672 bytes
server_rec:    208 -> 192 bytes
proxy_worker:  264 -> 208 bytes

Con: In some cases reordering the struct members makes the code harder 
to read because related members may no longer be grouped together.

Is it worth changing? Would somebody be -1?

(BTW, pahole is useful for finding such structs).


2) Use char instead of int for the module_levels vector in struct 
ap_logconfig. The length of the vector is ~100.

Pro: This may save ~300 bytes per server conf which are read often and 
therefore occupy CPU-cache memory

Con: On some architectures, accessing chars is slower than accessing 
ints.

Does anyone have an idea what is better, here? Int or char?

The same argument could be made for boolean flags in various other 
structs. But I don't think those are worth the effort.


3) In server/config.c, many config vectors are created (allocated and 
zeroed) with length (total_modules + DYNAMIC_MODULE_LIMIT). But this 
is only necessary before dynamic modules are loaded. Afterwards, 
total_modules is set to the total number of modules. Therefore 
allocating a vec of length total_modules should be enough. This would 
save zeroing 128 pointers per request and connection.

It seems the attached patch works and I could not find any problems 
when adding/removing modules during graceful restart. Objections?

Cheers,
Stefan