Fsync and Global Lock Percentages

if i set a key cache size of 100% the way i understand how that works is:

- the cache is not write through, but read through
- a key gets added to the cache on the first read if not already available
- the size of the cache will always increase for ever item read.  so if
you
have 100mil items your key cache will grow to 100mil

Here are my questions:

if that is the case then what happens if you only have enough mem to store
10mil items in your key cache?
do you lose the other 90% how is it determined what is removed?
will the server keep adding til it gets OOM?
if you add a row cache as well how does that affect your percentage?
if there a priority between the cache? or are they independant so both
will
try to be satisfied which would result in an OOM?

thanx,
artie

Hi
I'm experiencing contention in my Felix bundle due to its use of
Xerces. This is a well known issue that
can be resolved by pooling the SAX parsers obtained from the factory,
but I'd like to understand why
I get into this situation. Basically, the contention happens because
Xerces is calling loadclass on the
current classloader, and my threads get stuck at:
"Thread-34746" daemon prio=10 tid=0x0000000051b42400 nid=0x47db
waiting for monitor entry [0x000000004b6d0000..0x000000004b6d0c90]
   java.lang.Thread.State: BLOCKED (on object monitor)
        at
org.apache.felix.framework.cache.DirectoryContent.getEntryAsBytes(DirectoryContent.java:72)
        - waiting to lock <0x00002aaab55fb4a0> (a
org.apache.felix.framework.cache.DirectoryContent)
        at
org.apache.felix.framework.searchpolicy.ModuleImpl$ModuleClassLoader.findClass(ModuleImpl.java:1492)
        at
org.apache.felix.framework.searchpolicy.ModuleImpl.findClassOrResourceByDelegation(ModuleImpl.java:562)
        at
org.apache.felix.framework.searchpolicy.ModuleImpl.access$100(ModuleImpl.java:59)
        at
org.apache.felix.framework.searchpolicy.ModuleImpl$ModuleClassLoader.loadClass(ModuleImpl.java:1446)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:252)
        at
org.apache.felix.framework.searchpolicy.ModuleImpl.getClassByDelegation(ModuleImpl.java:481)
        at
org.apache.felix.framework.Felix.loadBundleClass(Felix.java:1354)
        at
org.apache.felix.framework.BundleImpl.loadClass(BundleImpl.java:755)
The pb is that I get this contention not only once, but all the time
in the code - and Xerces is always doing a loadClass for the same
class name, so I was under the impression the ModuleClassLoader would
have cached the bytes and class and speedily
return - in contrast it seems to always get to the 'DirectoryContent'
(hence re-reading the bytes each time ?? not sure of this)
Any insight on this ?
On a side note, the BundleImpl.loadClass() call is actually triggered
by a delegating classloader above, that very straightforwardly
delegates the call (no call to findLoadedClass or other magic).
Thanks
       /jog

is there a way to lock data while updating
so that anybody else can't edit the data and can't start any raise
condition

otherwise i would implement locking with a memcache
but that wouldn't be a nice way .....


reguards
Dawit

I suggest you to eliminate taking of the lock on cqi_freelist.
In order to it we should done all work on cqi_freelist by one thread
-- dispatcher thread in memcached.
I made some changes in queue code, including improvements for cache
line.
It will work on any platform.
There are patch file (diff -C3 memcached-1.4.5/thread.c <new
thread.c>)

*** thread.c.orig       Sat Apr  3 11:07:16 2010
--- thread.c    Tue Aug 17 14:09:28 2010
***************
*** 12,16 ****
  #include <pthread.h>

! #define ITEMS_PER_ALLOC 64

  /* An item in the connection queue. */
--- 12,17 ----
  #include <pthread.h>

! #define CACHE_LINE_SIZE 64
! #define ITEMS_PER_ALLOC 256

  /* An item in the connection queue. */
***************
*** 29,35 ****
  struct conn_queue {
      CQ_ITEM *head;
      CQ_ITEM *tail;
      pthread_mutex_t lock;
!     pthread_cond_t  cond;
  };

--- 30,40 ----
  struct conn_queue {
      CQ_ITEM *head;
+     char pad0[CACHE_LINE_SIZE - sizeof(CQ_ITEM *)];
+     CQ_ITEM *divider;
+     char pad1[CACHE_LINE_SIZE - sizeof(CQ_ITEM *)];
      CQ_ITEM *tail;
+     char pad2[CACHE_LINE_SIZE - sizeof(CQ_ITEM *)];
      pthread_mutex_t lock;
!     char pad3[CACHE_LINE_SIZE - sizeof(pthread_mutex_t)];
  };

***************
*** 45,49 ****
  /* Free list of CQ_ITEM structs */
  static CQ_ITEM *cqi_freelist;
- static pthread_mutex_t cqi_freelist_lock;

  static LIBEVENT_DISPATCHER_THREAD dispatcher_thread;
--- 50,53 ----
***************
*** 65,68 ****
--- 69,75 ----
  static void thread_libevent_process(int fd, short which, void *arg);

+ static CQ_ITEM *cqi_new(void);
+ static void cqi_free(CQ_ITEM *item);
+
  /*
   * Initializes a connection queue.
***************
*** 70,76 ****
  static void cq_init(CQ *cq) {
      pthread_mutex_init(&cq->lock, NULL);
!     pthread_cond_init(&cq->cond, NULL);
!     cq->head = NULL;
!     cq->tail = NULL;
  }

--- 77,81 ----
  static void cq_init(CQ *cq) {
      pthread_mutex_init(&cq->lock, NULL);
!     cq->head = cq->divider = cq->tail = cqi_new();
  }

***************
*** 78,96 ****
   * Looks for an item on a connection queue, but doesn't block if
there isn't
   * one.
!  * Returns the item, or NULL if no item is available
   */
! static CQ_ITEM *cq_pop(CQ *cq) {
!     CQ_ITEM *item;

      pthread_mutex_lock(&cq->lock);
!     item = cq->head;
!     if (NULL != item) {
!         cq->head = item->next;
!         if (NULL == cq->head)
!             cq->tail = NULL;
      }
      pthread_mutex_unlock(&cq->lock);

!     return item;
  }

--- 83,105 ----
   * Looks for an item on a connection queue, but doesn't block if
there isn't
   * one.
!  * Returns 1 if there are new item, or 0 if no item is available
   */
! static int cq_pop(CQ *cq, CQ_ITEM *item) {
!       int res = 0;
!
!     if (NULL == cq->divider->next)
!         return 0;

      pthread_mutex_lock(&cq->lock);
!     if (NULL != cq->divider->next) {
!         *item = *cq->divider->next;
!         res = 1;
!         cq->divider = cq->divider->next;
      }
      pthread_mutex_unlock(&cq->lock);

!     item->next = NULL;
!
!     return res;
  }

***************
*** 102,112 ****

      pthread_mutex_lock(&cq->lock);
!     if (NULL == cq->tail)
!         cq->head = item;
!     else
!         cq->tail->next = item;
!     cq->tail = item;
!     pthread_cond_signal(&cq->cond);
      pthread_mutex_unlock(&cq->lock);
  }

--- 111,124 ----

      pthread_mutex_lock(&cq->lock);
!     cq->tail->next = item;
!     cq->tail = cq->tail->next;
      pthread_mutex_unlock(&cq->lock);
+
+     while(cq->head != cq->divider)
+     {
+         CQ_ITEM *tmp = cq->head;
+         cq->head = cq->head->next;
+         cqi_free(tmp);
+     }
  }

***************
*** 116,125 ****
  static CQ_ITEM *cqi_new(void) {
      CQ_ITEM *item = NULL;
-     pthread_mutex_lock(&cqi_freelist_lock);
      if (cqi_freelist) {
          item = cqi_freelist;
          cqi_freelist = item->next;
      }
-     pthread_mutex_unlock(&cqi_freelist_lock);

      if (NULL == item) {
--- 128,135 ----
***************
*** 139,148 ****
              item[i - 1].next = &item[i];

-         pthread_mutex_lock(&cqi_freelist_lock);
          item[ITEMS_PER_ALLOC - 1].next = cqi_freelist;
          cqi_freelist = &item[1];
-         pthread_mutex_unlock(&cqi_freelist_lock);
      }

      return item;
  }
--- 149,158 ----
              item[i - 1].next = &item[i];

          item[ITEMS_PER_ALLOC - 1].next = cqi_freelist;
          cqi_freelist = &item[1];
      }

+     item->next = NULL;
+
      return item;
  }
***************
*** 153,160 ****
   */
  static void cqi_free(CQ_ITEM *item) {
-     pthread_mutex_lock(&cqi_freelist_lock);
      item->next = cqi_freelist;
      cqi_freelist = item;
-     pthread_mutex_unlock(&cqi_freelist_lock);
  }

--- 163,168 ----
***************
*** 254,258 ****
  static void thread_libevent_process(int fd, short which, void *arg)
{
      LIBEVENT_THREAD *me = arg;
!     CQ_ITEM *item;
      char buf[1];

--- 262,267 ----
  static void thread_libevent_process(int fd, short which, void *arg)
{
      LIBEVENT_THREAD *me = arg;
!     CQ_ITEM _item;
!     CQ_ITEM *item = &_item;
      char buf[1];

***************
*** 261,267 ****
              fprintf(stderr, "Can't read from libevent pipe\n");

!     item = cq_pop(me->new_conn_queue);
!
!     if (NULL != item) {
          conn *c = conn_new(item->sfd, item->init_state, item-
event_flags,
                             item->read_buffer_size,
item->transport,
me->base);
--- 270,274 ----
              fprintf(stderr, "Can't read from libevent pipe\n");

!     if (cq_pop(me->new_conn_queue, item)) {
          conn *c = conn_new(item->sfd, item->init_state, item-
event_flags,
                             item->read_buffer_size,
item->transport,
me->base);
***************
*** 280,284 ****
              c->thread = me;
          }
-         cqi_free(item);
      }
  }
--- 287,290 ----
***************
*** 595,599 ****
      pthread_cond_init(&init_cond, NULL);

-     pthread_mutex_init(&cqi_freelist_lock, NULL);
      cqi_freelist = NULL;

--- 601,604 ----



Also I tested lock- & wait- free single-producer single-consumer
conn_queue using volatile variables. It works on my SMP server (2*Xeon
E55**), centos 5.5, gcc 4.1. But I'm not sure that it will work on
other platforms without memory fence instructions/calls, for example,
on Itanium. I can send patch if anybody interested in it.

Lock Problem when accessing JCR

I keep getting lots of warnings like these:

 Sun Aug  8 08:11:08 [conn10] warning: ClientCursor::yield can't
unlock b/c of recursive lock
 ...
 Sun Aug  8 08:11:33 [conn10] warning: ClientCursor::yield can't
unlock b/c of recursive lock

All writes on existing collections fail. Writes to new collections
work. I have a replica set with 2 nodes and an arbiter.

Are the warnings related with my problem? Any ideas on how to approach
the issue? Thanks!

     [
https://issues.apache.org/activemq/br...nels:all-tabpanel
]

Rob Davies reassigned AMQ-2738:

     [
https://issues.apache.org/activemq/br...nels:all-tabpanel
]

Rob Davies resolved AMQ-2738.

     [
https://issues.apache.org/activemq/br...nels:all-tabpanel
]

Gary Tully updated AMQ-2738:

ability to lock the database

Is it possible to create / use Result Map that can be used by any of
mapper?

Hello puppeteers,

We are in the process of upgrading to 0.25.4 and I've discovered an
issue. I have a global class called global that looks like this:

class globals {

  $java          = "/usr/bin/java"
  $javac         = "/usr/bin/javac"
  $java_home     = "/usr/java/jdk1.5.0_15"
  $dynamo_home   = "/usr/local/ATG/ATG2007.1/home"
  $jboss_home    = "/usr/local/jboss"
  $jboss_release = "4.0.5.GA"
  $atg_release   = "2007.1"
  $ant_home      = "/opt/apache-ant-1.7.1"
}


If I include it in a node and try to access $java_home, I get no
value:

node 'homer.simpson.com' {
  include globals
  notify { "globalchk" : message => "Here's my java_home:
($java_home)" }
}

Here's the output:
notice: Here's my java_home: ()
notice: //Node[homer.simpson.com]/Notify[globalchk]/message: defined
'message' as 'Here's my java_home: ()'

Did something change since 0.24.4?

    [
https://issues.apache.org/jira/browse...4#action_12888764
] 

Shashank commented on JCR-2438:

     [
https://issues.apache.org/jira/browse...nels:all-tabpanel
]

Niall Pemberton updated IO-216:

     [
https://issues.apache.org/jira/browse...nels:all-tabpanel
]

Knut Anders Hatlen updated DERBY-360:

Database lock driver override not found for : [oracle_jdbc_driver].  Will
use default implementation.

     [
https://issues.apache.org/jira/browse...nels:all-tabpanel
]

Shashank updated JCR-2438:

     [
https://issues.apache.org/jira/browse...nels:all-tabpanel
]

Olga Natkovich resolved PIG-602.

     [
https://issues.apache.org/jira/browse...nels:all-tabpanel
]

Olga Natkovich reassigned PIG-602:

    [
https://issues.apache.org/jira/browse...0#action_12891800
] 

Olga Natkovich commented on PIG-602: